Skip to content

Webhooks

Webhooks send Surfmeter events to your own systems, or to a Slack or Microsoft Teams channel, as soon as they happen. For example, you can post a message to a channel when a client goes offline or an anomaly is detected.

You can find webhooks in the sidebar under APIs and Keys > Webhooks. Only users with the Admin or Organization Admin role can see and manage them.

Creating a webhook

Click New Webhook and fill in these fields:

  • Name – A name for the webhook, for example "Ops alerting".
  • Type – Choose Generic JSON for your own receiver, Slack for a Slack incoming webhook, or Microsoft Teams for a Teams Workflows webhook.
  • URL – The address that receives the events. For Slack and Teams, paste the webhook URL that you created in Slack or Teams.
  • Description – An optional description.
  • Events – The events this webhook receives. Select at least one. See Event types.
  • Filters – Optional filters to receive fewer events. See Filters.
  • Enabled – Whether the webhook sends events. It is on by default.

For Slack webhooks, you can also mention users, user groups, @here, or @channel in each message. For Generic JSON webhooks, you can add up to 10 custom request headers, for example an Authorization header.

An organization can have up to 20 webhooks.

Note

The URL must use HTTPS and must not point to a private network. Operators of on-premise installations can lift these restrictions in the server settings.

Warning

Treat webhook URLs like passwords. Slack and Teams accept any message sent to the URL. If a URL leaks, replace it in Slack or Teams.

Event types

  • Clients – client.created (a client registered), client.updated (for example, its label or tags changed), client.disabled, and client.reenabled.
  • Client health – client.offline, client.online, and client.health_changed (see Fleet Health).
  • Operational issues – study.execution_failed and study.execution_recovered (see Study failure notifications).
  • Anomalies – anomaly.detected and anomaly.resolved (see Anomalies).
  • Measurements – measurement.created, sent once new measurements are available through the Export API.
  • Users – client_admin_user.created, client_admin_user.updated, and client_admin_user.deleted.

The dialog shows a description and a sample payload for each event type.

Client health and study failure events are only sent if the matching detection is turned on in Organization Settings, and if the client has not muted the notification type (see Notification preferences).

Tip

measurement.created events contain only the ID, type, time, and client UUID of each measurement, in batches of up to 500 per minute. Use the IDs to fetch the full measurements from the Export API.

Filters

Filters are only shown if they apply to the selected events:

  • Client tags – Only send events for clients with at least one of these tags.
  • Measurement types – Only send measurement and anomaly events for these measurement types.
  • Minimum severity – Only send anomaly and health events of at least this severity (warning or critical). Recoveries are always sent.
  • Health metrics – Only send health events that involve these metrics (CPU, memory, or disk).

Testing and delivery log

Click Send test event to send a sample event to the webhook.

Click a webhook to open its detail page. The Deliveries table lists every event sent in the past seven days, with its status, number of attempts, the response status, and the time it took. Click a delivery to see the request body and the start of the response body. Click Redeliver to send it again.

Retries and automatic disabling

A delivery counts as successful if the receiver answers with any 2xx status code within 10 seconds. Redirects are not followed.

If a delivery fails, Surfmeter tries again after 1 minute, 5 minutes, 30 minutes, 2 hours, and 8 hours. After six failed attempts, it stops trying. You can still redeliver it by hand.

If a webhook fails 50 times in a row, or only fails for 72 hours, Surfmeter disables it and sends an email to all admins. Once the receiver works again, open the webhook and click Enable.

Note

Events may not arrive in the order in which they happened. Use the timestamp field in the body if the order matters.

Receiving Generic JSON events

Each event is sent as an HTTP POST with a JSON body:

{
  "id": "6f1c0e2e-3d7e-4a6f-9d0b-2c8d1e5f7a90",
  "type": "client.updated",
  "timestamp": "2026-09-16T10:41:07.221Z",
  "data": {
    "client": { "id": 42, "label": "probe-vienna-01", "tags": ["vienna"] },
    "changes": { "label": { "previous": "probe-vienna", "current": "probe-vienna-01" } }
  }
}

The type field holds the event type, timestamp is when the event happened, and data holds the event content. Update events include a changes object with the previous and current value of each changed field.

A retry uses the same id as the first attempt. Store the IDs of processed events and ignore repeats.

Verifying the signature

Generic JSON requests are signed according to the Standard Webhooks specification. Slack and Teams requests are not signed.

The signing secret is shown only once, after you create the webhook. To get a new secret, click Rotate secret on the detail page. The old secret keeps working for 24 hours, so you have time to update your receiver.

Each request has these headers:

  • webhook-id – The event ID.
  • webhook-timestamp – The time the request was sent, in Unix seconds.
  • webhook-signature – One or more signatures in the form v1,<signature>, separated by spaces. During the 24 hours after a secret rotation, there are two.

To check a request, compute an HMAC-SHA256 over <webhook-id>.<webhook-timestamp>.<raw body>. Use the base64-decoded part of the secret after whsec_ as the key. Compare the base64-encoded result with each signature. You can also use one of the Standard Webhooks libraries instead.

For example, in Python:

import base64, hashlib, hmac, time

def verify(secret: str, headers: dict, body: bytes) -> bool:
    key = base64.b64decode(secret.removeprefix("whsec_"))
    signed = f"{headers['webhook-id']}.{headers['webhook-timestamp']}.".encode() + body
    expected = base64.b64encode(hmac.new(key, signed, hashlib.sha256).digest()).decode()
    fresh = abs(time.time() - int(headers["webhook-timestamp"])) <= 300
    return fresh and any(
        hmac.compare_digest(s, f"v1,{expected}") for s in headers["webhook-signature"].split()
    )

Warning

Always verify the signature. Anyone who knows your URL can send JSON to it. Also reject requests whose timestamp is more than five minutes off, to prevent old requests from being replayed.

To manage webhooks through the API, see the Webhooks API reference.