Public Config Reference
The publicConfig.json file is a JSON file that contains the configuration for the studies that you can run.
The file is usually located in the config directory of the Automator installation, and it is also implicitly stored inside the Surfmeter Lab extension.
Example File
Here is an example file without content, which you can use as a template for filling out:
{
"$schema": "https://aveq.info/jsonschema/PublicConfigSectionDataV1.json",
"version": 1,
"credentials": {},
"studies": [
// (1)
],
}
- This is where you should specify each study. See Study Descriptions for more information.
The $schema property is optional, but it is recommended to include it for better validation in some editors.
Now let's go through the different sections of the config file.
Properties
Root Properties
These are the root items of the config file.
| Property | Value | Description |
|---|---|---|
version | 1 | Internal version. Do not change this! |
studies | array of Study Descriptions | A list of studies |
credentials | Credentials (optional) | Credentials for individual services |
revision | string (optional) | The revision of the config file. Can be a Git hash, a date, or anything else that makes sense for version control. |
Study Descriptions
A study description is an object that describes a study. It has the following properties:
| Property | Value | Description |
|---|---|---|
id | string | A unique identifier for the study |
type | string | The general type of study, must be one of WEB, VIDEO, SPEEDTEST, or NETWORK |
subject | Subject | The subject of the study (i.e., the specific measurement technology) |
params | Study Params | The parameters for the study |
names | Study Names (optional) | Study names (textual descriptions) |
native | Native Study Params (optional) | Custom options for native studies |
Examples
- Website Study
- Netflix Study
- Native Ping Study
This runs a web study using the browser.
{
"id": "STUDY_TOP5_WEBSITES",
"subject": "website",
"type": "WEB",
"params": {
"timeout": 30,
"urls": [ "google.com", "youtube.com", "facebook.com", "amazon.com", "yahoo.com" ]
}
}
This runs a Netflix study. Note that this will only work if you have specified the credentials for Netflix.
{
"id": "STUDY_NETFLIX",
"subject": "netflix",
"type": "VIDEO",
"params": {
"duration": 60,
"timeout": 30,
"video": {
"src": "https://www.netflix.com/watch/81035325?t=0",
"contentType": "feature"
},
"playMode": "normal"
}
}
This runs a ping test against a list of hosts.
{
"id": "STUDY_PING",
"subject": "icmpping",
"type": "NETWORK",
"params": {
"host": ["google.com", "youtube.com", "facebook.com", "amazon.com", "yahoo.com"]
}
}
Subjects
The subject keyword defines the specific measurement technology that is used for the study, or, for video, the service that will be called.
The following subjects are supported per type of measurement. Click on the name of the subject to see the parameters that can be configured.
Most video subjects use the shared parameters. The Lightweight Video Player has its own native parameter set. Links take you to the service-specific documentation, which is also available here.
VIDEO:3q3satamazon_trailerardmediathekdashjsdazndisneyplusfacebookjoynlightweightvideoplayermagentadenetflixnetflix_trailernflorfrakutenrtlplusservustvsrftiktokinstagramwaiputvyoutubezattoozdfmediathek
WEB:
NETWORK:
CONFERENCING:
Study Params
For each type of study and subject, there are different parameters that can be specified in the params object of the study description.
Forcing the video resolution
Set params.player.forceResolution to request a video height instead of leaving the choice entirely to the player's native adaptive bitrate logic:
{
"params": {
"player": {
"forceResolution": 1080
}
}
}
The supported values are 144, 240, 360, 480, 720, 1080, 1440, and 2160, expressed as the video height in pixels. A single value requests one resolution. A [min, max] tuple requests a permitted range:
{
"params": {
"player": {
"forceResolution": [720, 1080]
}
}
}
Omit forceResolution, or set it to "auto", to leave native automatic resolution switching enabled. Resolution forcing is currently implemented for these subjects:
-
dashjsdisables dash.js video ABR and selects an offered rendition within the requested range. If the manifest has no matching rendition, it selects the highest rendition below the requested maximum, or the lowest available rendition when none is below it. -
facebookselects one fixed quality from Facebook's player menu. A tuple uses its maximum as the target because the player menu cannot express an ABR range. Videos that Facebook does not stream adaptively have no heights in that menu, and for those the setting has no effect. -
netflix_trailerconstrains the trailer rendition ladder. A tuple allows Netflix to adapt among the renditions in the requested range. -
waiputvrestricts the player's adaptive logic to the renditions in the requested range and switches to one of them immediately. If no rendition matches, the setting has no effect. -
youtubepins a single value or constrains YouTube's adaptive logic to a requested range.
Other video subjects do not currently implement this setting. Available rendition heights also vary by service and video. When an exact height is unavailable, a service may select a nearby lower rendition; the measurement reports the resolution that was actually played.
Standard Video Study Params
| Property | Type | Description |
|---|---|---|
credentials | Credentials (optional) | The credentials to use when requesting the video |
duration | number | The length of the video to play in seconds |
interactive | Interactive (optional) | Interactivity parameters for video browsing. These currently apply to reels-based social media platforms, but could be extended to other services in the future. In order to enable interactive mode, you must at least specify the swipeDelay parameter, and point the study "url" attribute to a playlist of videos (e.g., a YouTube shorts channel). |
playMode | string (one of: embedded, lab_embedded, normal, unknown) | The type of video player mode that is used. |
player | PlayerParams (optional) | The parameters to use when playing the video. Most fields apply to the browser-based players (dash.js/hls.js), but several (autoplay, skipAds, width, height, forceResolution) also apply to the native service players. |
statisticsCalculation | StatisticsCalculationParams (optional) | The parameters to use when calculating statistics |
timeout | number (optional) | The timeout to use when requesting the video |
video | ChannelTitleVideoDescription, or SrcVideoDescription | The description of the video to load |
ChannelTitleVideoDescription
The parameters to use when playing a video from a channel title.
| Property | Type | Description |
|---|---|---|
channelTitle | string | The title of the channel to play. The list of possible channel titles depends on the service itself. Please ask our support for that list. |
contentType | string (one of: advertisement, feature, lab, live, ...) | The type of video content that is played. |
Credentials
The credentials to use when requesting the video
| Property | Type | Description |
|---|---|---|
password | string | |
username | string |
Interactive
Interactivity parameters for video browsing. These currently apply to reels-based social media platforms, but could be extended to other services in the future.
In order to enable interactive mode, you must at least specify the swipeDelay parameter, and point the study "url" attribute to a playlist of videos (e.g., a YouTube shorts channel).
| Property | Type | Description |
|---|---|---|
maxDuration | number (optional) | Maximum duration of the test. Whereas "duration" in the study parameters refers to the maximum duration of a single video, this one is the maximum duration of the entire study before it is completed. Note that this may make the last video very short. Alternatively, specify "maxNumVideos" which will limit the overall number of tested videos. |
maxNumVideos | number (optional) | The total number of videos that will be tested. The study will finish after playing the last video for the amount of the "duration" attribute. |
swipeAfterPlaybackOnly | boolean (optional) | If true, swipe only after the video playback has started. Otherwise, swiping will be based on a timer that starts when the last swipe has completed. This means that the delay value will be applied after the video starts playing, rather than immediately after the previous swipe. This can be useful to ensure that the user has enough time to watch the video before swiping, especially if there are any loading times before the video starts. |
swipeDelay | array of number, or number (optional) | Delay after which to swipe to the next video. Can be a single number or a set of delays, where if the number of videos exceeds the number of swipe delays, the last one will be repeated until the study ends. Note that if a video has a shorter duration than the swipe delay for that given video, the study will automatically swipe to the next video after the current one finishes, even if the swipe delay has not yet been reached. E.g.: [5, 2, 2, ...] |
PlayerParams
The parameters to use when playing the video. Most fields apply to the browser-based players (dash.js/hls.js), but several (autoplay, skipAds, width, height, forceResolution) also apply to the native service players.
| Property | Type | Description |
|---|---|---|
autoplay | boolean (optional) | Whether the video should start playing automatically |
customParams | object (optional) | Custom params to be passed to the dash.js or hls.js player. Please refer to https://cdn.dashjs.org/latest/jsdoc/module-Settings.html for dash.js, and https://github.com/video-dev/hls.js/blob/master/docs/API.md for hls.js. |
drmParams | DrmParams (optional) | The parameters to use when playing DRM protected video. |
forceResolution | array (tuple: [number, number]), or one of: 1080, 144, 1440, 2160, ... (optional) | Force the streamed video resolution instead of leaving it to the player's native adaptive bitrate logic. A single value pins the resolution exactly; a [min, max] tuple caps the ceiling while still allowing adaptation down to min; 'auto' (or omitting the field) keeps native ABR. |
height | number (optional) | The height in pixels of the video element |
keepState | boolean (optional) | Keep the state that the player remembers from one run to the next, per service, as for a returning viewer. Off by default. Only valid for YouTube at the moment, where this keeps the player's bandwidth estimate (yt-player-bandwidth). Other services ignore this option. |
playerVersion | string (optional) | Custom version of the player, useful for dash.js to select a specific hosted version. |
skipAds | boolean (optional) | Whether any ads should be skipped, if possible |
width | number (optional) | The width in pixels of the video element |
SrcVideoDescription
The parameters to use when playing a video from a URL source.
| Property | Type | Description |
|---|---|---|
contentType | string (one of: advertisement, feature, lab, live, ...) | The type of video content that is played. |
src | string | The URL of the video to play. Note that this depends on the particular service; some services have no permanent video URLs, or the URLs may be dynamically generated, so you should use a channelTitle instead. In case of questions you can ask us for a sample configuration. |
StatisticsCalculationParams
The parameters to use when calculating statistics
| Property | Type | Description |
|---|---|---|
forceNominalBitrate | boolean (optional) | Whether to calculate the bitrate of the video using manifest bitrates |
DrmParams
The parameters to use when playing DRM protected video.
| Property | Type | Description |
|---|---|---|
httpRequestHeaders | object (optional) | DRM HTTP Headers |
serverUrl | string | DRM Server Url |
type | string (one of: widevine, playready, clearkey) | DRM Type |
Lightweight Video Player Study Params
| Property | Type | Description |
|---|---|---|
connectTimeout | number (optional) | Connection timeout in seconds. Default 10. |
dashSchedulingThreshold | number (optional) | Buffer level (seconds) below which the DASH profile fetches the next segment. Default 6. |
dashTargetBuffer | number (optional) | Forward buffer target (seconds) the DASH profile fills up to. Default 18. |
device | string (one of: mobile, pc) (optional) | Device identifier reported in the P.1203 metadata. Default "pc". |
displaySize | string (optional) | Simulated display resolution as "widthxheight" in pixels, used for P.1203 scoring. Default "1920x1080". |
duration | number | The length of the measurement in seconds. |
headers | object (optional) | Extra HTTP request headers to send with every request. |
hlsSchedulingThreshold | number (optional) | Buffer level (seconds) below which the HLS profile fetches the next segment. Default 10. |
hlsTargetBuffer | number (optional) | Forward buffer target (seconds) the HLS profile fills up to. Default 30. |
manifestAttempts | number (optional) | Number of HTTP attempts for the manifest before giving up. Default 2. |
maxBitrate | number (optional) | Highest declared video bitrate in bit/s that ABR may select. Audio is not affected. |
maxEventLogBytes | number (optional) | Maximum retained request event-log size in bytes. Default 4194304 (4 MiB). |
maxExpandedSegments | number (optional) | Maximum number of segments expanded from a single timeline. Default 20000. |
maxInFlightBodyBytes | number (optional) | Maximum retained response-body memory in bytes. Default 8388608 (8 MiB). |
maxManifestBytes | number (optional) | Maximum manifest body size in bytes. Default 2097152 (2 MiB). |
maxPlaylistLineBytes | number (optional) | Maximum HLS playlist line length in bytes. Default 16384. |
maxResolution | number (optional) | Highest video height in pixels that ABR may select, for example 1080. Audio is not affected. |
maxSegmentBytes | number (optional) | Maximum segment body size in bytes. Default 4194304 (4 MiB). |
maxUrlBytes | number (optional) | Maximum URL length in bytes. Default 8192. |
maxXmlDepth | number (optional) | Maximum DASH XML nesting depth. Default 64. |
minBitrate | number (optional) | Lowest declared video bitrate in bit/s that ABR may select. Audio is not affected. |
minResolution | number (optional) | Lowest video height in pixels that ABR may select, for example 480. Audio is not affected. |
profile | unknown, or string (one of: dash, hls) (optional) | ABR compatibility profile to emulate. 'hls' and 'dash' select the latest supported profile for that protocol; a specific versioned profile name may also be passed for reproducibility. Auto-detected from the manifest extension when omitted. |
proxy | string (optional) | HTTP/HTTPS proxy URL to route all requests through. |
referer | string (optional) | Value for the HTTP Referer header. |
rendition | string (optional) | Fixed video rendition, which turns off ABR for video: a height such as "720p" (the highest bitrate at that height), "lowest", "highest", or "id:<representation id>". Cannot be combined with the min/max options. The measurement fails at startup if a rendition constraint matches no video rendition. |
requestTimeout | number (optional) | Per-request transfer timeout for manifests and segments, in seconds. Default 30. |
resolve | array of string (optional) | Fixed addresses for host names, each as "HOST:PORT:ADDR" (as with curl's --resolve), for example "cdn.example.com:443:192.0.2.1". Write an IPv6 address in brackets. TLS and the Host header keep the original host name. Cannot be combined with proxy. |
segmentAttempts | number (optional) | Number of HTTP attempts per segment before giving up. Default 3. |
sensitiveHeaders | array of string (optional) | Header names whose values must be redacted from the report and diagnostics. Common auth headers (e.g. Authorization, Cookie) are always redacted; list any additional names here. |
timeout | number (optional) | Timeout for the measurement in seconds. This includes the load/playback duration. If a measurement is not finished by this timeout, it will be aborted and marked as failed. Default: Twice the duration. |
url | string | The HLS or DASH manifest, or the progressive MP4 file, to measure. Choose a canonical URL. |
viewingDistance | string (optional) | Viewing distance reported in the P.1203 metadata. Default "150cm". |
Web Study Params
website
A web study for one or more websites loaded through the browser.
| Property | Type | Description |
|---|---|---|
randomize | boolean (optional) | randomize the URLs |
timeout | number (optional) | How long in seconds before load is cancelled |
urls | array of string | A list of URLs to load in the study |
lighthouse
A web study for one or more websites loaded with Google Lighthouse.
Requires the lighthouse CLI tool.
For details, see the Lighthouse specification
| Property | Type | Description |
|---|---|---|
emulatedFormFactor | string (one of: desktop, mobile, none) (optional) | How emulation (useragent, device screen metrics, touch) should be applied. none indicates Lighthouse should leave the host browser as-is |
emulatedUserAgent | string (optional) | The user agent string that the emulation should use |
maxWaitForLoad | number (optional) | The maximum amount of time to wait for a page to load, in ms |
randomize | boolean (optional) | randomize the URLs |
throttling | object with properties: cpuSlowdownMultiplier, downloadThroughputKbps, requestLatencyMs, rttMs, throughputKbps, uploadThroughputKbps (optional) | The network throttling to apply to the run |
throttlingEnabled | boolean (optional) | Whether to enable network throttling |
throttlingMethod | string (one of: devtools, provided, simulate) (optional) | The method used to throttle the network |
timeout | number (optional) | How long in seconds before load is cancelled |
urls | array of string | A list of URLs to load in the study |
Speedtest Study Params
fastcom
A speedtest study on fast.com.
| Property | Type | Description |
|---|---|---|
maxConnections | number (optional) | The maximum number of connections to use |
maxDuration | number (optional) | Maximum test duration in seconds |
measureUploadLatency | boolean (optional) | Whether to measure loaded latency |
minConnections | number (optional) | The minimum number of connections to use |
minDuration | number (optional) | Minimum test duration in seconds |
nperf
This subject has no parameters to configure.
Network Study Params
dns
A network study implementing the DNS protocol, making one or more DNS queries against a specified host, optionally using a specific nameserver.
If DNS caching is used, this means that the first request will take longer, as the cache is warmed up, and subsequent requests will be faster.
Running a DNS study requires the dig CLI tool, unless the useGetAddrInfo flag is set to true,
in which case, the getaddrinfo system call is used via Node.js.
When dig is used, it will make a simple A query by default, and we measure the time for the single request. When using getaddrinfo, an A and AAAA query are made at the same time, and the time to complete both queries is measured.
| Property | Type | Description |
|---|---|---|
host | array of string, or string | The host(s) to query |
nameserver | string (optional) | The DNS nameserver to use (if unset, will use system default). Not applicable when useGetAddrInfo is true. |
parallel | boolean (optional) | Parallelize DNS lookups across hosts |
queryCount | number (optional) | The number of queries to send, default: 1 |
randomize | boolean (optional) | Randomize the order of the hosts |
recordType | string (one of: A, AAAA) (optional) | The DNS record type to query with dig. Default: A. Not applicable when useGetAddrInfo is true. |
sample | number (optional) | Sample only up to N hosts from the given list |
useGetAddrInfo | boolean (optional) | Use getaddrinfo instead of dig, and make an A and AAAA query at the same time |
icmpping
A network study implementing the ICMP ping protocol.
Requires the ping CLI tool.
| Property | Type | Description |
|---|---|---|
discardFirstPacket | boolean (optional) | Discard the first packet (e.g. to avoid DNS lookup interfering with) |
host | array of string, or string | The host(s) to ping |
interval | number (optional) | Interval between pings, in seconds |
ipVersion | string (one of: v4, v6) (optional) | IP version to use. Default: v4 |
packetCount | number (optional) | How many packets to send. Default: 10, unless totalTime is specified, which causes the packet count to be adjusted accordingly. |
packetSize | number (optional) | Size of each packet, in bytes |
parallel | boolean (optional) | Parallelize the pings across hosts |
randomize | boolean (optional) | Randomize the order of the hosts |
totalTime | number (optional) | Total time to ping for, in seconds. In parallel mode, this is the total time for all parallel pings. In sequential mode, the total time will be split among the hosts (i.e., time per host = totalTime / host count). If packetCount is specified, the test will end after packetCount is reached, or totalTime, whichever comes first. |
ooklaspeedtest
A network study implementing the Ookla Speedtest CLI.
Requires the speedtest CLI tool.
| Property | Type | Description |
|---|---|---|
host | string (optional) | The server via its host's fully qualified domain name |
server | string (optional) | The server via its ID |
traceroute
A network study implementing the traceroute CLI. This is based on the Linux (and, where possible, the macOS) traceroute CLI.
| Property | Type | Description |
|---|---|---|
asLookup | boolean (optional) | Perform AS path lookups in routing registries and print results directly after the corresponding addresses. Only with GNU traceroute or macOS traceroute. |
gateway | string (optional) | Add an IP source routing option to the outgoing packet that tells the network to route the packet through the specified gateway(s). Comma-separate multiple gateways. |
host | array of string, or string | The host(s) to traceroute to |
interface | string (optional) | Specifies the interface through which traceroute should send packets. By default, the interface is selected according to the routing table. |
ipVersion | string (one of: v4, v6) (optional) | Explicitly force IPv4 or IPv6 traceouting. By default, the program will try to resolve the name given, and choose the appropriate protocol automatically. If resolving a host name returns both IPv4 and IPv6 addresses, traceroute will use IPv4. |
maxHops | number (optional) | Specifies the maximum number of hops (max time-to-live value) traceroute will probe. The default is 30. |
parallel | boolean (optional) | Parallelize the traceroutes across hosts |
probeCount | number (optional) | Sets the number of probe packets per hop. The default is 3. |
probeInterval | number (optional) | Minimal time interval between probes (default 0). If the value is more than 10, then it specifies a number in milliseconds, else it is a number of seconds (float point values allowed too). Useful when some routers use rate-limit for icmp messages. |
useIcmp | boolean (optional) | Set to true to explicitly use ICMP (if unset, will default to what traceroute does by default) |
useTcp | boolean (optional) | Set to true to explicitly use TCP (if unset, will default to what traceroute does by default) |
waitTime | number (optional) | Set the time (in seconds) to wait for a response to a probe (default 5.0 sec). |
httpprober
A network study implementing the HTTP Prober tool for testing CDN delivery of large static files.
| Property | Type | Description |
|---|---|---|
abortedReason | string (optional) | Aborted reason if the main CLI call was aborted |
disableHeaders | boolean (optional) | Disable headers in output (default: false) |
disableTimings | boolean (optional) | Disable timing information collection (default: false) |
disabledTests | array of string (optional) | Test types to disable |
enabledTests | array of string (optional) | Specific test types to enable (default: all test types) |
freshConnections | boolean (optional) | Force fresh TCP/TLS connections for each request to simulate multiple clients (default: false) |
hostHeader | string (optional) | Custom Host header value to send with all requests |
http2 | boolean (optional) | Whether to use HTTP/2 for protocol testing (default: false) |
noRedirect | boolean (optional) | Do not follow HTTP redirects (default: false) |
parallel | number (optional) | Number of parallel requests for error rate and byte range testing (default: 1) |
range | string (optional) | Byte range to request for applicable tests (default: "0-1023") |
requests | number (optional) | Number of requests for error rate and byte range testing (default: 10) |
timeout | number (optional) | Timeout for requests in seconds (default: 30) |
traceRedirects | boolean (optional) | Collect redirect chain details when following redirects (default: false) |
urls | array of string, or string | Single URL or array of URLs to test |
userAgents | array of string (optional) | Custom user agents to test (default: predefined set) |
Conferencing Study Params
googlemeet
A conferencing study that joins a conference and measures the performance and quality.
| Property | Type | Description |
|---|---|---|
duration | number | Duration of the study in seconds |
mode | string (one of: host, join) (optional) | The mode of the conference, if host, will attempt to let others join |
timeout | object with properties: joinTimeout, partnerWaitTimeout, webuiLoadTimeout (optional), or number (optional) (optional) | Timeout for joining in seconds |
url | string | The URL of the conference to join |
microsoftteams
A conferencing study that joins a conference and measures the performance and quality.
| Property | Type | Description |
|---|---|---|
duration | number | Duration of the study in seconds |
mode | string (one of: host, join) (optional) | The mode of the conference, if host, will attempt to let others join |
timeout | object with properties: joinTimeout, partnerWaitTimeout, webuiLoadTimeout (optional), or number (optional) (optional) | Timeout for joining in seconds |
url | string | The URL of the conference to join |
Native Study Params
These are custom options for native studies. Native studies run on the host operating system — they are not run in the browser.
| Property | Type | Description |
|---|---|---|
enabled | boolean | Must be set to true by user to actively enable this study |
cmdPrefix | array of string (optional) | Prepend to the native command |
cmdSuffix | array of string (optional) | Append to the native command |
cwd | string (optional) | Override the current working directory |
env | object (optional) | Pass custom environment variables |
executable | string (optional) | Override the executable for the native measurement |
You must set enable to true for a native study to work.
Automator Parmas
These are custom options for the Automator.
| Property | Type | Description |
|---|---|---|
enableFullscreen | boolean (optional) | Enable fullscreen playback for video if possible. This is true by default. Set to false to disable. |
Examples
- Enabling a Native Study
- Adding a command suffix
Always pass enable: true to enable a native study.
{
"id": "STUDY_LIGHTHOUSE",
"type": "WEB",
"subject": "lighthouse",ed": true,
}
}
This may be a bit contrived, but you can pass any command line arguments to the native study.
{
"id": "STUDY_DNS",
"type": "NETWORK",
"subject": "dns",ed": true,
"cmdSuffix": ["-p", "53"] // overriding the port
}
}
Study Names
The study names are used to display the study in the UI. The names are specified in a dictionary, with the language code as the key. The following languages are supported:
| Language | Code |
|---|---|
| English | en_US |
| German | de_DE |
| French | fr_FR |
Each object has a short and long key.
To specify the names, you must at least use the en_US key.
Example
The following example shows the names for a study in English and German:
{
// ...
"studies": [
{
"id": "STUDY_WEB",
// ... other params": {
"short": "Web Top 5",
"long": "Test-Queue für 5 der meistbesuchten Websites in den USA"
}
}
}
// ... other studies
]
}
Credentials
A credentials dictionary is required when measuring a closed-access service such as Netflix. It contains one key for each service, and the value is a dictionary with the username and password for that service.
For a list of possible service keys, see the list of subject values.
These credentials are likely to be different per device. Use the local config feature to specify different credentials for different devices.
The credentials are stored in plain text in the publicConfig.json file. Do not commit this file to a public repository. Ideally, change the permissions of the file so that only the user running the Automator can read it, i.e. chmod 600 publicConfig.json.
Example
{
"credentials": {
"netflix": {
"username": "foo@example.com",
"password": "bar"
}
},
"studies": [
// ...
]
}