Logging and Sending Network Requests

Surfmeter Automator supports logging and sending network requests for additional debugging information. This captures the timing, size, remote IP, and other details of the network requests made by the browser for any study you run. This can be tremendously helpful for diagnosing issues with the browser or the network.

Logging Network Requests

Via the --logNetworkRequests option, all network requests made by the browser will be logged to a file in the system's temporary directory and used for further analysis.

Using this option enables additional statistics like the video response time and content server hostname/IP address. It is not enabled by default due to the possible performance penalty on low-end devices.

Tip

If you want to globally set this option for all your studies, you can add the following to your automatorConfig.json file:

automatorConfig.json

{
  "version": 1,
  // other properties like "updates"
  "globalScheduleSettings": {
    "options": {
        "logNetworkRequests": true
    }
  },
  "studySchedules": [
    // your individual study schedules
  ]
}

For more information on the globalScheduleSettings option, see the reference.

The log file will be deleted unless the keepNetworkRequests option is specified. If you want to keep the log file for later analysis, you can do so by adding this flag to your Automator command (--keepNetworkRequests) or specifying the "keepNetworkRequests": true option in the automatorConfig.json file.

The path to the file will be printed in the log, and it is in a gzipped line-delimited JSON format. So you can use a command like gunzip -c <path> to decompress and view the file.

Sending Network Requests

You can also send all network requests to the server with the --sendNetworkRequests option.

The data will be available in the Surfmeter Export API under the network_requests resource. Note that the data is not indexed to our Analytics Dashboard.

Warning

This causes a lot of network traffic and should be used with care. Sending requests may multiple send processes to run. We do not yet support retry mechanisms, so if the connection is lost, the request will fail.

Data Format

Internally, the Chrome DevTools Protocol is used to intercept and log the network requests, so the fields are in the same format as when using the DevTools Protocol. For more information, see the Chrome DevTools Protocol documentation on the following events:

• requestWillBeSent
• responseReceived