Skip to content

Web Quality Test (iOS)

The WebQualityTest measures how long a web page takes to load and render. It loads the page in a WKWebView, collects Web Vitals and navigation timing from it, and computes a web QoE score on the device. For a complete example, see WebTestView.swift in the demo app.

This test is available on iOS only, since it relies on WKWebView.

Starting the Test

You create the builder with the URL to test, give it a view for the web view to live in, and attach a result listener:

import SurfmeterQualitySDK

let test = try WebQualityTestBuilder(url: URL(string: "https://www.example.com")!)
    .setTimeout(timeoutMs: 30_000)
    .setContainerView(view: myContainerView)
    .setResultListener(listener: self)
    .build()

try test.start()

To stop a running test early, call stop(). The test then finishes with whatever it has collected so far and delivers a result.

try test.stop()

The web view has to be on screen

An off-screen WKWebView never renders, and paint metrics such as First Contentful Paint and Largest Contentful Paint are never reported for it. Pass a view that is part of a visible view hierarchy through setContainerView, otherwise the test only returns the navigation timing metrics and no web QoE score can be computed.

Builder Options

WebQualityTestBuilder is created with the URL and supports the following methods:

  • setTimeout(timeoutMs:) – the timeout for the test in milliseconds, 30000 by default
  • setContainerView(view:) – the UIView the web view is added to
  • setResultListener(listener:) – the QualityTestResultListener that receives results, errors and state changes
  • setMetadata(_:) – arbitrary key-value pairs that are stored alongside the measurement

How the Test Ends

The test ends in one of three ways:

  • The page finishes loading. The SDK then waits two more seconds for Web Vitals to settle before computing the result, because Largest Contentful Paint can still change after the load event.
  • The timeout elapses. Whatever has been collected so far is used.
  • You call stop().

If navigation fails outright, for example because the host cannot be resolved, onTestError is called instead.

Receiving Test Results

To receive the outcomes of the test, your class must conform to the QualityTestResultListener protocol. This protocol includes methods for handling successful results, errors, and state changes. The onPlaybackProgress method is not used by this test.

import SurfmeterQualitySDK

class TestRunner: QualityTestResultListener {
    func onTestResult(_ result: [String: AnyHashable]) {
        // the finished measurement report
    }

    func onTestError(_ error: String) {
        // the test could not be completed
    }

    func onTestStateChanged(_ state: QualityTestState) {
        // the test moved to a new state
    }

    func onPlaybackProgress(_ progress: Int) {
        // not called by the web test
    }
}

Exactly one of onTestResult or onTestError is called per test run. The SDK holds a strong reference to the listener until that final callback has been delivered, then releases it, so a listener that also owns the test does not leak.

Test Success

When the test completes, onTestResult is called with the full measurement report. The metrics are under statistic_values:

func onTestResult(_ result: [String: AnyHashable]) {
    guard let stats = result["statistic_values"] as? [String: AnyHashable] else { return }
    print(stats)
}

{
  "serverResponseTime": 0.152,
  "documentReadyLoadTime": 0.814,
  "pageLoadTime": 1.523,
  "dnsLoadTime": 0.021,
  "firstContentfulPaint": 0.454,
  "largestContentfulPaint": 0.682,
  "cumulativeLayoutShift": 0.03,
  "webQoeScore": 84
}

All times are in seconds. cumulativeLayoutShift is unitless and webQoeScore runs from 0 to 100. For a full description of each metric, refer to the measurement data page.

The report has the same shape as the one the Android SDK produces, so both platforms can be read the same way. The individual Web Vitals appear only under statistic_values, and the raw navigation timing is not part of the report, since it is an input to the calculation rather than a result.

Test Errors

If the page cannot be loaded, onTestError is called with a message:

func onTestError(_ error: String) {
    print("Test error: \(error)")
}

Test State Changes

You can follow the lifecycle of the test through onTestStateChanged:

func onTestStateChanged(_ state: QualityTestState) {
    print("Test state: \(state)")
}

The web test passes through the following states, in this order:

  • STATE_READY — the test has been built but not started
  • STATE_STARTING — start() was called, setup is in progress
  • STATE_STARTED — the web view has been created and the page is loading
  • STATE_ANALYZING — the page load is over, statistics are being computed
  • STATE_END — the test is finished, the result callback follows

If the test is aborted, it moves to STATE_ABORTED instead.

Collected Metrics

The SDK injects the web-vitals library into the page and registers callbacks for First Contentful Paint, Largest Contentful Paint and Cumulative Layout Shift. It also reads the browser's navigation timing entry, from which the server response time, DNS lookup time, document ready time and page load time are derived.

Interaction to Next Paint is deliberately not collected. The metric needs a real user interaction with the page, which never happens during an automated test. The Android SDK does not collect it either.

When Web Vitals reports no layout shift at all, cumulativeLayoutShift is set to zero rather than left out. The web QoE score is only defined for a few fixed combinations of metrics, so a missing value would mean no score is computed.

Differences from Android

The iOS web test does not support overriding the user agent. The setUserAgent option that the Android SDK offers has no counterpart here yet.

Background Execution

Background execution is not supported. The test needs the app to be visible on screen, both because WKWebView stops rendering otherwise and because the paint metrics would never fire.