Logging¶

Surfmeter Automator uses pino for structured JSON logging. Each measurement run produces log output that includes startup information, browser events, measurement progress, and results. Understanding how this output is captured depends on whether you run Automator natively or inside Docker.

Log Levels¶

You can control the verbosity of log output with the LOG_LEVEL environment variable. The available levels, from least to most verbose, are:

• error – only errors
• warning – errors and warnings
• info – the default; normal operational messages
• debug – additional diagnostic output
• trace – very detailed tracing
• extreme – maximum verbosity

Set the variable when running Automator directly:

LOG_LEVEL=debug ./surfmeter-automator-headless startStudy --studyId STUDY_YOUTUBE

Or add it to a schedule entry's env block in your automatorConfig.json:

{
    "id": "STUDY_YOUTUBE",
    "studyId": "STUDY_YOUTUBE",
    "cronSchedule": "*/15 * * * *",
    "env": {
        "LOG_LEVEL": "debug"
    }
}

You can also set it globally for all schedules via globalScheduleSettings.env:

"globalScheduleSettings": {
    "env": {
        "LOG_LEVEL": "debug"
    }
}

Logging in Native Deployments¶

When Automator runs natively (outside Docker), scheduled measurements are executed by cron. By default, cron delivers command output via its mail transfer agent (MTA). On most systems this MTA is not configured, which means log output is silently discarded.

To persist the logs, set cronLogFile in your automatorConfig.json. You can set it per schedule entry or globally:

"globalScheduleSettings": {
    "cronLogFile": "$HOME/logs/surfmeter-automator.log"
}

The $HOME and $TMP variables are expanded automatically. Make sure the target directory exists before loading the schedule:

mkdir -p ~/logs

When you run a study manually (not via cron), output goes to your terminal's stdout as usual.

Logging in Docker Deployments¶

Docker containers write their stdout and stderr to the Docker logging driver, which you can inspect with:

docker compose logs -f surfmeter

The Surfmeter Docker entrypoint produces its own structured JSON log lines for container lifecycle events (startup, registration, schedule loading, etc.).

Cron output forwarding¶

Cron jobs inside a container cannot write to container stdout directly, because cron runs as a separate process without access to PID 1's file descriptors. To solve this, the Docker entrypoint sets up a named pipe (FIFO) at the path defined by the SURFMETER_CRON_LOG environment variable (default: /tmp/surfmeter-cron.log). When a schedule entry does not specify a cronLogFile, Automator automatically redirects the cron job's output to this FIFO. The entrypoint reads from the pipe in the background and forwards lines to container stdout:

• JSON lines (pino output from Automator) are forwarded as-is
• Non-JSON lines are wrapped in the entrypoint's own JSON log format

This means that with the default Docker setup, all measurement logs appear in docker compose logs without any extra configuration.

Overriding with cronLogFile¶

If you set a cronLogFile in your automatorConfig.json, cron output is written to that file instead of the FIFO. This bypasses Docker's logging entirely for those schedule entries.

Disabling the FIFO¶

To disable the FIFO forwarding entirely, set the environment variable to an empty string in your docker-compose.yml:

environment:
  SURFMETER_CRON_LOG: ""

Be aware that without the FIFO and without a cronLogFile, cron output inside Docker is lost.

Built-in Cron Logfile Safeguard¶

Starting with Automator 1.63.1, every invocation checks all cronLogFile paths in the merged automatorConfig.json and automatorConfig.local.json. The check runs in the background, so it does not delay the requested action. When a regular file has reached 1024 MiB, Automator truncates it in place. Missing paths and non-regular files, including the Docker FIFO, are ignored.

Set SURFMETER_CRON_LOG_MAX_SIZE_MB to change the limit in MiB:

export SURFMETER_CRON_LOG_MAX_SIZE_MB=512

Set it to 0 to disable the safeguard. Invalid values produce a warning and retain the 1024 MiB default. The variable must be present in the environment of the scheduled Automator process; see Using Overrides with Scheduled Jobs for the available configuration methods.

Truncation clears the current file and retains no history. It is a last-resort disk-usage safeguard for systems without logrotate, not a replacement for rotation when you need historical logs.

Summary¶

Log Rotation¶

Use log rotation when you want to retain historical file logs. Automator's built-in safeguard only clears an oversized current file.

On Linux, the standard tool for this is logrotate. Create a configuration file, for example /etc/logrotate.d/surfmeter:

/home/surfmeter/logs/surfmeter-automator.log {
    daily
    rotate 14
    compress
    missingok
    notifempty
    copytruncate
}

This rotates the log daily, keeps 14 compressed copies, and uses copytruncate so that the running process does not need to be signaled. Adjust the path and retention to your needs.

{
    "log-driver": "json-file",
    "log-opts": {
        "max-size": "50m",
        "max-file": "5"
    }
}

services:
  surfmeter:
    logging:
      driver: json-file
      options:
        max-size: "50m"
        max-file: "5"