Skip to main content
Version: 2.0 (Preview)

Scheduling Daemon

The scheduling daemon runs studies at their scheduled times and keeps the probe up to date. It replaces the cron jobs of Automator 1.x. To define what it runs, see Scheduling Studies.

The setup script installs and starts the daemon. In Docker, the container starts it if you set SURFMETER_AUTO_START_DAEMON=1. In both cases, you do not have to start it yourself.

Files and Services​

The daemon runs as surfmeter daemon start under the measurement user. There is one daemon per user, and it needs a registered client.

On a Linux installation, these are its files and services:

  • Program: the installation in ~/surfmeter
  • Service: surfmeter-daemon.service, a systemd user service in ~/.config/systemd/user/
  • Watchdog: surfmeter-daemon-watchdog.timer and surfmeter-daemon-watchdog.service, in the same directory
  • Schedule: ~/.surfmeter/config/automatorConfig.json, with the last valid version in ~/.surfmeter/lastGoodConfig/
  • Log: the systemd journal, see Viewing the Log
  • Control socket: /run/user/<uid>/surfmeter/daemon.sock, which the surfmeter daemon commands use to talk to the daemon
Important: Use the measurement user's service

Run the commands on this page as the measurement user. The daemon is a user service, so systemctl needs --user. If you switch users with sudo -u, also set the runtime directory:

sudo -u <username> XDG_RUNTIME_DIR=/run/user/$(id -u <username>) systemctl --user status surfmeter-daemon.service

In Docker, the daemon runs inside the container, under the surfmeter user. There is no systemd in the container. See Using the Daemon in Docker.

How the Daemon Works​

Every second, the daemon compares the current time with the next run of each entry in the schedule. When an entry is due, the daemon starts a separate surfmeter study start process, with the options of the entry. It does not run studies itself. A study that hangs or crashes does not affect the schedule. If a run takes too long, the daemon ends it. See Timeouts.

The daemon can run any number of different entries at the same time, but runs of the same entry cannot overlap. It skips runs missed during downtime.

System Jobs​

The daemon also runs these jobs on its own:

  • system.update-config: downloads the configuration after each start, and then every 15 minutes, unless updates.config.schedule sets different times
  • system.flush-db: sends results that could not be sent before, every 15 minutes
  • system.update-self: updates Automator, only if updates.automator.schedule is set

The prefix system. is reserved and cannot be used for your own entries.

Configuration Changes​

When a new schedule config arrives from the server, or when you change it with surfmeter config set or surfmeter config edit, the daemon applies it automatically. It does not interrupt runs that are active. If the new schedule config is invalid, the daemon keeps running the previous schedule, and reports the problem in its status.

If the schedule config cannot be read when the daemon starts, it uses the last valid version.

Restarts​

The daemon keeps running when you log out and starts again after a reboot. The setup script enables systemd lingering for the measurement user to allow this. See The Daemon Stops When I Log Out.

If the daemon exits, systemd restarts it after 10 seconds. The watchdog also checks every minute whether the daemon responds. If it fails to respond twice in a row, 15 seconds apart, the watchdog restarts it. The watchdog does not restart a daemon that you stopped manually.

When the daemon stops, it asks all active runs to end, and waits up to 15 seconds for them. It then kills the runs that are still active.

Checking the Daemon​

surfmeter daemon status shows what the daemon is doing:

$ surfmeter daemon status
Surfmeter daemon v2.0.0 (pid 2211) is running
  started:   2026-11-02T08:00:12.000Z (3h 14m ago)
  last tick: now, every 1000 ms
  timezone:  Europe/Vienna (UTC+01:00), set in the config
  config:    /home/surfmeter/.surfmeter/config/automatorConfig.json revision 2026-11-02 hash 3f9a61c07d2e
  client id: 0f8c7a52-6a3e-4b7e-9d11-2c4f6e8a1b90
  reload:    1h 2m ago, accepted

Jobs (3):
  youtube-hourly               study        0 * * * *          next in 45m 12s      last success   runs 3, skipped 0, missed 0, active 0
    jitter 192s | studies STUDY_YOUTUBE
  system.update-config         updateConfig every 15m          next in 8m 41s       last success   runs 13, skipped 0, missed 0, active 0
  system.flush-db              flushDb      */15 * * * *       next in 2m 5s        last success   runs 12, skipped 0, missed 0, active 0

Recent runs (2, newest first):
  system.update-config         success    exit 0        schedule ended 6m 19s ago
  youtube-hourly               success    exit 0        schedule ended 12m 30s ago

Run counters and recent run history reset when the daemon restarts. The output contains these sections:

  • Jobs: all entries and system jobs, with their timing, their next run, and the result of their last run. skipped counts runs that were due while the previous run of the entry was still active. missed counts runs that the daemon could not start in time, for example because the machine was suspended.
  • Active runs: what is running at the moment. This part only appears if something is running.
  • Recent runs: the result of each run, which is one of success, failed, timeout, killed, or spawnError.
  • Issues: problems in the schedule config. This part only appears if there are problems.

Use --json to get all details for a script, or --tsv to get the main fields as tab-separated values.

To only check that the daemon responds, use surfmeter daemon ping.

Exit Codes​

The commands return these exit codes, which you can use in monitoring scripts:

  • 0: OK
  • 1: an error occurred, for example because a reload was rejected
  • 2: no daemon answered
  • 3: a run that you waited for was not successful

Running an Entry Now​

To test an entry of the schedule, run it with its own options:

surfmeter daemon run <entryId>           # Start the run, and return immediately
surfmeter daemon run <entryId> --wait    # Wait for the run, and show its result

The daemon refuses to start the entry if it is already running. A manual run counts as a regular run. For an interval entry, the interval before the next run begins when the manual run ends.

This is different from surfmeter study start, which you can run at any time with your own options, without the daemon.

Reloading the Schedule​

The daemon automatically detects changes from the server and from surfmeter config commands. If you edit the files by hand, reload them:

surfmeter daemon reload

The command reports whether the daemon accepted the schedule and lists any errors. Changes to the measurement config need no reload, since each run reads this file when it starts.

Viewing the Log​

The daemon and its runs write to the systemd journal:

journalctl --user -u surfmeter-daemon -f                  # Follow the log
journalctl --user -u surfmeter-daemon --since "1 hour ago"
journalctl --user -u surfmeter-daemon-watchdog            # Log of the watchdog

To filter the log for one run, and to write the output to files, see Logging.

Starting, Stopping, and Restarting​

The daemon is controlled by systemd:

systemctl --user status surfmeter-daemon.service
systemctl --user restart surfmeter-daemon.service
systemctl --user stop surfmeter-daemon.service

A daemon that you stopped starts again at the next reboot. To keep it off, use disable:

systemctl --user disable --now surfmeter-daemon.service

surfmeter daemon stop prints instructions for stopping the daemon through its supervisor. Use the systemd commands above to stop the service.

Setting Environment Variables​

The daemon passes its environment variables to all runs. To add some, for example for a proxy, edit the service and restart it:

systemctl --user edit surfmeter-daemon.service
systemctl --user restart surfmeter-daemon.service

To set variables for certain entries only, use env in the schedule config.

Installing and Removing the Service​

The setup script does this for you, but you can also do it by hand:

surfmeter daemon install      # Install, enable, and start the service and the watchdog
surfmeter daemon uninstall    # Stop and remove them

surfmeter daemon install also adds the SURFMETER_* variables of your current shell to the service.

Running in the Foreground​

To watch the daemon in a terminal, for example when testing a local schedule, stop the service first, and run:

surfmeter daemon start

Press Ctrl+C to stop it. If another daemon is already running, the command refuses to start.

Using the Daemon in Docker​

If you set SURFMETER_AUTO_START_DAEMON=1, the container runs the daemon, and restarts it if it exits. Without this variable, nothing is scheduled.

The surfmeter daemon commands work the same way, but you run them in the container:

docker exec --user surfmeter surfmeter surfmeter daemon status
docker exec --user surfmeter surfmeter surfmeter daemon run <entryId> --wait

Docker deployments differ in these ways:

  • The log is part of the container output. Run docker compose logs -f.
  • To stop the daemon, stop the container with docker compose stop.
  • To restart only the daemon, run docker exec surfmeter /command/s6-svc -r /run/service/surfmeter-daemon.
  • The commands surfmeter daemon install and surfmeter daemon uninstall, and all systemctl commands, do not work.
  • The control socket is /run/surfmeter/daemon.sock.
An unhealthy container is not restarted automatically

There is no watchdog in Docker. If the daemon stops responding, the health check marks the container as unhealthy. Docker does not restart it based on that status.