Skip to content

AVPlayer Quality Test

The AVPlayerQualityTest plays a video stream with AVFoundation's AVPlayer and scores the playback with the ITU-T Rec. P.1203 model. It is the iOS counterpart of the Android ExoPlayer Quality Test, and reports its results under the subject avplayer.

The test plays for a configurable duration, then stops playback, builds a P.1203 input report from what the player did, and computes the quality scores on the device. Have a look at AVPlayerTestView.swift in the demo app for a complete example.

Starting the Test

In the example below the test plays an HLS stream for 30 seconds. The maximum test duration is a timeout for the whole test in case of network problems, and the analysis duration is a timeout for computing the statistics once playback is over.

import SurfmeterQualitySDK

let test = try AVPlayerQualityTestBuilder(manifest: URL(string: "https://example.com/stream.m3u8")!)
    .setMaxPlaybackDuration(durationMs: 30_000)
    .setMaxTestDuration(durationMs: 45_000)
    .setMaxAnalysisDuration(durationMs: 10_000)
    .setProgressInterval(intervalMs: 1000)
    .setResultListener(listener: self)
    .build()

try test.start()

Both build() and start() throw, so wrap them in a do/catch and handle the error. build() rejects invalid combinations, for example a maximum test duration shorter than the maximum playback duration. start() throws when the test has already been started.

To stop a running test early, call stop(). If playback was already running, the test finishes normally and you get a result for what was played so far. If it had not started yet, you get an error instead.

try test.stop()

Builder Options

AVPlayerQualityTestBuilder is created with the manifest URL and supports the following methods, all of which return the builder so they can be chained:

  • setMaxPlaybackDuration(durationMs:) – maximum playback duration in milliseconds, 30000 by default
  • setMaxTestDuration(durationMs:) – maximum test duration in milliseconds, 45000 by default. Must be at least the playback duration
  • setMaxAnalysisDuration(durationMs:) – timeout for computing the statistics, in milliseconds, 10000 by default
  • setProgressInterval(intervalMs:) – how often onPlaybackProgress is called, in milliseconds, 1000 by default
  • setResultListener(listener:) – the QualityTestResultListener that receives results, errors and state changes
  • setCalculationSettings(calculationSettings:) – P.1203 calculation settings, see below
  • attachToView(view:) – attach the player to a UIView so the video is visible during the test
  • setAllowsBackgroundPlayback(_:) – allow playback and measurement to continue after the host app enters the background, false by default
  • setReportedDisplaySize(width:height:) – the display size reported to P.1203, in device pixels
  • setMetadata(_:) – arbitrary key-value pairs that are stored alongside the measurement

There is also a setExternalPlayer(player:) method for handing the test an AVPlayer you created yourself. See Using an External Player for its lifecycle rules. It cannot be combined with attachToView; build() throws if both are set.

Tip

You can get test URLs from the hls.js demo. Note that AVPlayer supports HLS and progressive MP4, but not DASH, so the DASH streams that work with the Android SDK will not play here.

Reported Display Size

P.1203 scores partly on how far the video has to be scaled to fit the display, so the model needs to know which display the measurement stands for.

This is deliberately independent of attachToView. A test running in a small preview, or with no view at all, is usually meant to represent what a full-screen viewing would look like, so the SDK does not measure the player's actual size. Left unset, the report assumes a 1920x1080 display.

try builder.setReportedDisplaySize(width: 1920, height: 1080)

Calculation Settings

Calculation settings select modules and adjustments within the combined video QoE calculation. In the Video QoE model, pv is the video-quality module. It produces a video MOS value for each second (O.22). The pq module then combines those video values with the audio quality and stalling events to produce the final session MOS (O.46). See the P.1203 module diagram for this data flow.

You normally do not need to call setCalculationSettings. The iOS SDK uses these defaults:

  • pv: P12043BitstreamMode0. Despite this legacy internal name, it selects the ITU-T P.1204.1 Mode 0 video-quality model. This metadata-based model uses the codec, bitrate, resolution, frame rate, and reported display size; it does not inspect the encoded bitstream.

  • pq: P1203PqExtended. This is the P.1203.3 integration model with additional diagnostic outputs.

  • amendment1Audiovisual: true. This increases the effect of very low audiovisual quality on the final MOS.

  • amendment1Stalling: true. This increases the effect of stalling on the final MOS.

You can override individual defaults as follows:

try builder.setCalculationSettings(calculationSettings: [
    "pv": "P1203PvRetrained",
    "pq": "P1203PqExtended",
    "amendment1Audiovisual": true,
    "amendment1Stalling": true,
    "amendment1App2": false
])

The current on-device calculator supports P12043BitstreamMode0 and P1203PvRetrained for pv. P1203PvRetrained uses P.1203.1 Mode 0 with retrained coefficients for additional codecs. The P.1203 extensions and variants page compares both models and their supported inputs.

P1203PqExtended is the supported pq value. amendment1App2 applies an Appendix 2 adjustment when externally calculated bitstream- or pixel-based video scores are used; leave it false for the metadata-based models above.

The builder's validation also accepts the server-side model identifiers P1203PvExtended, P1203PvHveiExtended, and P1203PqM, as well as the longMode key. The current on-device statistics calculator does not implement these model choices, and longMode does not change its calculation. Do not use them for an AVPlayerQualityTest.

Codec and Bitrate Fallbacks

AVFoundation can expose a stream codec using a FourCC value that the quality model does not recognize. In that case, the SDK logs a warning and uses H.264 for an unknown video codec or AAC for an unknown audio codec. If AVFoundation reports no video bitrate, the SDK uses the same resolution-based nominal bitrate estimate as the Android SDK and logs another warning.

These values keep the quality calculation from returning an empty result, but they are estimates. Filter the app's Console logs on the AVPlayerObserver category when you need to check whether a result used a codec or bitrate fallback.

Using an External Player

Use setExternalPlayer(player:) when your app already owns and presents the player. Load the stream passed to the builder into that player, then start the test before starting playback so the complete interval is measured:

import AVFoundation
import SurfmeterQualitySDK

let manifestURL = URL(string: "https://example.com/stream.m3u8")!
let player = AVPlayer(url: manifestURL)

let test = try AVPlayerQualityTestBuilder(manifest: manifestURL)
    .setExternalPlayer(player: player)
    .setResultListener(listener: self)
    .build()

try test.start()
player.play()

The test observes the external player but does not start it, pause it, replace its current item, or release it. Your app remains responsible for the player throughout the test. For background playback, follow the additional external-player setup below.

Receiving Test Results

To receive the outcomes of the test, your class must conform to the QualityTestResultListener protocol. This protocol includes methods for handling successful results, errors, state changes, and playback progress.

import SurfmeterQualitySDK

class TestRunner: QualityTestResultListener {
    func onTestResult(_ result: [String: AnyHashable]) {
        // the finished measurement report
    }

    func onTestError(_ error: String) {
        // the test could not be completed
    }

    func onTestStateChanged(_ state: QualityTestState) {
        // the test moved to a new state
    }

    func onPlaybackProgress(_ progress: Int) {
        // playback progress in percent
    }
}

Exactly one of onTestResult or onTestError is called per test run. The SDK holds a strong reference to the listener until that final callback has been delivered, then releases it, so a listener that also owns the test does not leak.

Test Success

When the test completes, onTestResult is called with the full measurement report:

func onTestResult(_ result: [String: AnyHashable]) {
    guard let stats = result["statistic_values"] as? [String: AnyHashable] else { return }
    print("Overall MOS: \(stats["p1203OverallMos"] ?? "n/a")")
}

The report has the same shape as the one the Android SDK produces, so both platforms can be read the same way. The quality scores are under statistic_values, and include p1203OverallMos, p1203AverageVideoQuality, p1203AverageAudioQuality and p1203StallingQuality, alongside playback statistics such as initialLoadingDelay, averageVideoBitrate and averageStallingTime. The full P.1203 computation that produced them is under statistics_input, and the raw input report and per-segment performance data are under client_reports.

The values correspond to the measurement data, so please refer to that page for a description of each one.

To serialize the report, for example to store it or to display it:

if let data = try? JSONSerialization.data(withJSONObject: result, options: .prettyPrinted),
   let json = String(data: data, encoding: .utf8) {
    print(json)
}

Test Errors

If the test cannot be completed, onTestError is called with a message instead:

func onTestError(_ error: String) {
    print("Test error: \(error)")
}

This happens when the test is stopped before playback started, or when computing the statistics times out.

Playback Progress

While the video is playing, onPlaybackProgress reports the position as a percentage of the expected playback duration, at the interval you set through setProgressInterval. It is called once more with 100 when playback finishes.

func onPlaybackProgress(_ progress: Int) {
    print("Progress: \(progress)%")
}

Test State Changes

You can follow the lifecycle of the test through onTestStateChanged:

func onTestStateChanged(_ state: QualityTestState) {
    print("Test state: \(state)")
}

The video test passes through the following states, in this order:

  • STATE_READY — the test has been built but not started
  • STATE_STARTING — start() was called, setup is in progress
  • STATE_PLAYING — playback is running
  • STATE_ANALYZING — playback is over, statistics are being computed
  • STATE_END — the test is finished, the result callback follows

If the test is aborted, it moves to STATE_ABORTED instead.

Showing the Video

By default the test plays without any visible output, which is enough for a measurement. To show the video, hand the test a view to attach the player layer to:

builder.attachToView(view: myPlayerView)

The player layer is sized to the view's bounds when the test is built, so make sure the view has its final size by then.

Background Execution

AVPlayerQualityTest can continue when the user leaves the app after a test has started. This is the iOS counterpart to Android background measurements, but iOS does not provide a service or wake-lock equivalent. The host app must qualify for background media playback and keep a real media playback session active.

Background playback is off by default. Enable it only for an app whose purpose includes media playback. It does not add periodic scheduling, launch a test while the app is suspended, or enable background execution for VideoQualityTest and WebQualityTest.

Host App Setup

In Xcode, open the app target's Signing & Capabilities settings, add the Background Modes capability, and select Audio, AirPlay, and Picture in Picture. This adds the following entry to the app's Info.plist:

<key>UIBackgroundModes</key>
<array>
    <string>audio</string>
</array>

Before starting the test, configure and activate the app's shared audio session for playback:

import AVFoundation

let audioSession = AVAudioSession.sharedInstance()
try audioSession.setCategory(.playback, mode: .moviePlayback)
try audioSession.setActive(true)

The SDK does not change the shared AVAudioSession, because doing so could conflict with the host app's audio and interruption policy. The app is also responsible for handling interruptions, route changes, and deactivating the session after playback when appropriate.

Do not use silent audio solely to keep the app running. Background modes must match the app's real function and the playback presented to the user.

SDK-Owned Player

For the player created by the SDK, enable background playback on the builder:

let test = try AVPlayerQualityTestBuilder(manifest: manifestURL)
    .setAllowsBackgroundPlayback(true)
    .setMaxPlaybackDuration(durationMs: 60_000)
    .setMaxTestDuration(durationMs: 75_000)
    .setResultListener(listener: self)
    .build()

try test.start()

The SDK sets the player's audiovisualBackgroundPlaybackPolicy to .continuesIfPossible. This works with headless playback and with a player attached through attachToView(view:).

External Player Background Playback

The SDK does not change a caller-owned player. Set its background policy before building the test:

let player = AVPlayer(url: manifestURL)
player.audiovisualBackgroundPlaybackPolicy = .continuesIfPossible

let test = try AVPlayerQualityTestBuilder(manifest: manifestURL)
    .setExternalPlayer(player: player)
    .setAllowsBackgroundPlayback(true)
    .setResultListener(listener: self)
    .build()

try test.start()
player.play()

Validation and Completion

When background playback is enabled, build() checks that the host app declares the audio background mode. For an external player it also checks that audiovisualBackgroundPlaybackPolicy is .continuesIfPossible. Either missing requirement throws QualityTestError.backgroundPlaybackUnavailable, with error code 1015.

The SDK cannot verify whether the process-wide audio session is active and correctly configured. If it is not, iOS can suspend playback after the app enters the background.

When playback finishes, the SDK requests a short UIKit background task while it computes the statistics, stores the report in the measurement queue, and delivers the final callback. If iOS does not grant that time, or the background task expires, the test reports an error and records the run as aborted instead of returning incomplete or empty statistics.