Validating an Encode¶

Use analyze to check the quality of an encode when you do not have the source. For example, you can check a file delivered by a partner, a recording of a broadcast channel, or the output of a new encoder configuration.

analyze runs three layers on every input:

bitstream: QP, frame types, frame sizes and motion statistics per frame
pixels: no-reference pixel metrics per frame (blockiness, blur, noise, freezes, black frames, and more)
p1204: the ITU-T P.1204.3 quality score per second, per GOP and for the whole file

All three layers use the same decoding pass. See Metrics for a description of each value.

Running the analysis¶

surfmeter-media analyze -o results/channel1 channel1.ts

The progress is shown on standard error. When the run is done, results/channel1/ contains records.ndjson with all per-frame records and summary.json with the overall results.

To explore the results interactively, add --tui:

surfmeter-media analyze --tui -o results/channel1 channel1.ts

The timeline shows QP, frame sizes, the pixel metrics and the P.1204.3 score over time. Scene changes, freezes and black frames appear as events. Use n to jump from event to event, and m to choose the metrics shown.

Reading the results¶

The overall P.1204.3 score is in summary.json, under model_scores of each input:

jq '.inputs[] | {label, p1204: [.model_scores[] | {score, device, display}]}' \
  results/channel1/summary.json

The statistics of every metric (count, mean, standard deviation, minimum, maximum, and the 1^st, 5^th, 50^th and 95^th percentile) are in summaries. Statistics of bitstream metrics are also given per frame type. For example, to show the average QP per frame type:

jq -r '.inputs[].summaries[] |
  select(.record_type == "frame_bitstream" and .metric == "qp_avg") |
  "\(.frame_type // "all")\tmean=\(.mean)\tp95=\(.p95)\tmax=\(.max)"' \
  results/channel1/summary.json

To find the seconds with the lowest quality, read the per_second scores from the records:

jq -s -c '[.[] | select(.type == "model_score" and .scope == "per_second" and .score)]
  | sort_by(.score) | .[:10][] | {start, end, score}' \
  results/channel1/records.ndjson

Or to list frames with a decoder error (damaged or missing data):

jq -c 'select(.type == "frame_bitstream" and .decode_error == true) | {n, pts, frame_type}' \
  results/channel1/records.ndjson

Configuring the P.1204.3 model¶

The P.1204.3 score depends on the viewing situation. By default, the toolkit uses the PC/TV model with a 3840×2160 display. Set the device and display to match your audience:

# Full HD TV
surfmeter-media analyze --display 1920x1080 -o results input.mp4

# Mobile viewing
surfmeter-media analyze --device mobile --display 1920x1080 -o results input.mp4

The device and display are stored in each model_score record.

Analyzing part of a file¶

To analyze only the beginning of each file, use --duration with the number of seconds:

surfmeter-media analyze --duration 60 -o results channel1.ts

Choosing layers and metrics¶

To speed up the analysis, select only the layers you need with --layers:

# Only QP, frame types and frame sizes
surfmeter-media analyze --layers bitstream -o results input.ts

# Bitstream statistics and P.1204.3, no pixel metrics
surfmeter-media analyze --layers bitstream,p1204 -o results input.ts

The pixel layer is the most expensive. By default, it computes blockiness, blurriness, noise, brightness, black frames, SI/TI, freezes, scene changes and jerkiness. --pixel-features changes the set:

# All 13 features of the video analyzer (about twice the CPU time)
surfmeter-media analyze --pixel-features all -o results input.ts

# Only the listed features
surfmeter-media analyze --pixel-features blockiness,noise,temporalchange -o results input.ts

The available features are blurriness, blockiness, brightness, blackscreen, spatialcomplexity, chroma, contrast, radialprofile, noise, autoenhancement, finedetail, temporalchange and jerkiness. See Metrics for the values each feature produces.

For long files, --pixel-step <n> analyzes only every n-th frame for pixel metrics. Temporal metrics then compare frames that are n frames apart.

surfmeter-media analyze --pixel-step 5 -o results long-recording.ts

Analyzing several files¶

analyze accepts several inputs and processes them in parallel. By default, the number of parallel inputs depends on the number of CPU cores. Set it with -j:

surfmeter-media analyze -j 4 -o results/batch recordings/*.ts

All inputs end up in the same output directory, and each record carries the ID of its input. In the terminal view, select an input with 1–9 or [ and ].

Next steps¶

If you have the source of the encode, compare the encode against it to get VMAF, PSNR and SSIM in addition to the no-reference analysis.