Special Features¶
Automator supports a few special options and features.
Packet Captures (tcpdump)¶
Download tcpdump through your package manager. On Ubuntu, run:
Now we need to modify the permissions so that Automator can run it without root privileges. Run the following commands:
sudo groupadd pcap
sudo usermod -a -G pcap $USER
sudo chgrp pcap /usr/sbin/tcpdump
sudo setcap cap_net_raw,cap_net_admin=eip /usr/sbin/tcpdump
sudo ln -s /usr/sbin/tcpdump /usr/bin/tcpdump
Note that this will not persist across upgrades of the tcpdump package. If you upgrade tcpdump, you will need to run the above commands again.
FFmpeg is already part of our Docker image! 
Now, with the following flag, a tcpdump recording can be made during a study:
This will record data on any interface and write the output to PCAP file in your system's temporary directory. The path to the file will be printed in the log, and it is in the format:
Use --tcpDumpDir to set a different directory.
Additional Browser Extensions¶
If you want to load additional browser extensions with the browser start, you first have to download the extension from the Chrome Web Store.
Use the download_crx.sh script, which is bundled with Automator, to download additional extensions. For instance, to download the uBlock Origin ad blocker:
You can then load this extension along with the Automator:
./surfmeter-automator-headless startStudy \
--studyId STUDY_YOUTUBE \
--additionalExtensions $HOME/surfmeter/extensions/ublockOrigin
Specify multiple extensions by repeating the --additionalExtensions flag.
Note that we cannot guarantee that all extensions will work properly with Automator. Please report any issues you encounter and we will try to fix them.
Environment Variables¶
When running the Surfmeter Automator executables, you can pass the following additional environment variables:
| Variable | Description |
|---|---|
TMPDIR |
Override the path for temporary lockfiles |
SURFMETER_AUTOMATOR_EXEC |
Set the path to the surfmeter-automator executable when using a different directory |
HEADLESS_RESOLUTION |
Specify the X window resolution in the format <width>x<height>x<bitDepth>, defaults to 1920x1080x24. Note that this will not work if the X window daemon is running or if you are using the GUI version. |
CONFIG_DIR |
Set the path to an alternative configuration directory |
LOG_LEVEL |
Set the log level, e.g. LOG_LEVEL=debug to print additional messages. Available log levels are: error, warning, info (default), debug, trace, extreme |
SURFMETER_DISPLAY |
Set the X display number that will be used for Simulated Headless mode, defaults to :123. This also applies to the Docker container and will change the internal Xvfb server display number. |
SURFMETER_DISABLE_PULSEAUDIO |
Set to 1 to disable PulseAudio. This only works for the Docker version. |
SURFMETER_AUTO_LOAD_SCHEDULE |
Set to 1 to automatically load the schedule when starting the Automator inside Docker. Has no effect for native deployments. |
SURFMETER_AUTO_REGISTER |
Set to 1 to automatically register the Docker container with the server. Has no effect for native deployments. |
SURFMETER_DEFAULT_REGISTRATION_KEY |
Set the default registration key to use when automatically registering with the server. |
Network Throttling Chrome¶
For studies that use the browser (i.e., non-Lighthouse web measurements using the website subject, video measurements, and non-native speed tests), Chrome's capabilities to emulate a slow network can be used.
--latency: Sets the latency in milliseconds--uploadThroughput: Sets the upload throughput in kBit/s. Set to -1 to disable throttling (default).--downloadThroughput: Sets the download throughput in kBit/s. Set to -1 to disable throttling (default)
Note that with extreme settings of the above values, functionality of the Automator software may not be guaranteed, and videos may run into timeouts. The sending of the measurement data will not be affected by the throttling.
Network Throttling on the System Level¶
You can use a tool like comcast to throttle the network on the system level. This is useful for studies that do not use the browser, such as the DNS measurement. Note, however, that this will also affect the sending of the measurement data.
Under the hood, it uses tc, which requires root privileges. To install it on Ubuntu, run:
As root, run:
Then add this to your /root/.bashrc:
Now you can run it, as root, like this:
Check out the GitHub repository for more information.
Docker usage
For Docker, so far, it appears that only latency throttling works with Docker.
To run the above commands you need to make sure you run the container with the --cap-add=NET_ADMIN flag, and drop into a root shell to perform the above commands:
Cache Clearing¶
Automator attempts to create a clean environment for running measurements, by performing the following steps every time the browser is started:
- Clearing the system DNS cache
- Clearing browser cookies
- Clearing the browser cache
These steps can be disabled via the respective command-line flags:
--noClearSystemDns--noClearCookies--noClearCache
Note
To clear the system DNS cache, the user must be able to run a specific binary with sudo without having to enter the password interactively. The command depends on the operating system. For Linux, it is resolvectl; for macOS it is dscacheutil. Please refer to the man sudoers manual on how to allow the user to execute these commands noninteractively.
For instance, to allow a user to run Linux resolvectl without a password, add the following line to the /etc/sudoers file:
Please replace <username> with your user's name, and edit this file with visudo to avoid syntax errors.
On older systems, the binary might be located at /usr/bin/systemd-resolve.