Skip to main content

Output Format

All results of the toolkit are records in one format, shared by all commands. This page describes the records and the files they are written to. For the individual metrics, see Metrics.

Files​

With -o <directory>, the toolkit writes these files:

  • records.ndjson: all records of the run, one JSON object per line (format ndjson)
  • summary.json: one JSON document with the most important results per input (format json)
  • <record_type>.csv: one CSV file per record type, with nested fields flattened into columns (format csv)

--format selects the formats. The default is ndjson,json. Without -o, the NDJSON records are written to standard output.

Records​

Every record has a type field and a schema_version field. The record types are:

  • run_info: information about the run: tool version, command line, inputs with their IDs, start and end time, versions of the components, host, options, and the final status and errors
  • stream_info: stream information of one input (container, programs, streams)
  • scte35, scte35_window: cue messages and their time ranges in inputs with SCTE-35
  • frame_bitstream: bitstream statistics of one video frame
  • frame_pixels: pixel metrics of one video frame
  • model_score: a quality model score for a second, a GOP, or a whole stream
  • alignment: the alignment of one encode with the reference (compare only)
  • frame_fullref: VMAF, PSNR and SSIM of one frame against the reference (compare only)
  • summary: statistics of one metric of one stream
  • comparison: the difference of one metric's statistics between an encode and the reference (compare only)
  • loudness: the loudness of one audio stream, every 100 ms
  • audio_block: the levels of each channel of one audio stream, every second
  • audio_event: a problem found in the audio, such as silence or clipping
  • audio_metadata: the AC-3 or E-AC-3 metadata of one audio stream, such as dialnorm
  • loudness_summary: the totals of one audio stream and the result of each loudness target

The run_info record is written twice: at the start of the run with status set to running, and at the end with the final status. When you read records.ndjson, use the last run_info record.

Common fields​

Records that refer to an input have these fields:

  • input: the ID of the input, as listed in run_info.inputs. Inputs are numbered from "0" in the order of the command line. In compare, "0" is the reference.
  • stream: the index of the stream within the input, as numbered by FFmpeg
  • program, pid: the program number and PID, for transport streams

Frame records also have:

  • n: the frame index in presentation order, starting at 0
  • pts: the presentation timestamp in seconds
  • pts_raw and time_base: the original timestamp and its time base, without loss of precision

All timestamps are the original timestamps of the file, including the start offset of transport streams. This lets you locate a frame with other tools, such as ffprobe. The terminal view shows times from the start of each input instead.

Optional fields are left out of a record when they have no value, rather than written as null.

Example​

A frame_bitstream record of an H.264 frame (shortened):

{
"schema_version": "0.1.9",
"type": "frame_bitstream",
"input": "0",
"stream": 0,
"program": 1,
"pid": 256,
"n": 42,
"pts": 3.16,
"pts_raw": 284400,
"time_base": {"num": 1, "den": 90000},
"frame_type": "P",
"is_idr": false,
"size": 18734,
"qp_avg": 28.4,
"qp_stdev": 2.1,
"qp_min": 24,
"qp_max": 35,
"motion_avg": 3.2
}

A model_score record with the overall P.1204.3 score:

{
"schema_version": "0.1.9",
"type": "model_score",
"input": "0",
"stream": 0,
"model": "p1204.3",
"scope": "overall",
"start": 1.48,
"end": 7.48,
"device": "pc",
"display": {"width": 3840, "height": 2160},
"score": 3.87
}

Audio records​

The audio layer writes these records for each audio stream. stream is the index of the audio stream. The times are the presentation times of the file, as for video. Loudness is in LUFS, loudness differences in LU, true peaks in dBTP, and sample peaks and RMS levels in dBFS. See Metrics for what the values mean.

A loudness record is written every 100 ms. pts is the end of the 100 ms interval. Its fields:

  • momentary: loudness of the last 400 ms
  • short_term: loudness of the last 3 seconds
  • integrated and lra: integrated loudness and loudness range from the start of the stream up to this point
  • true_peak and sample_peak: the highest peak of all channels in the last 100 ms

A loudness value is left out when it cannot be measured yet, for example before the first 400 ms are complete, or when the audio is digital silence.

An audio_block record is written every second, from start to end. channels has one entry per channel, in the order of the stream. Each entry has the channel name (FL, FR, FC, LFE, SL, SR and so on), sample_peak, true_peak, rms, clipped_samples, and silent if the channel was silent.

An audio_event record describes a problem found in the audio. kind is one of silence, clipping, true_peak_over and dialnorm_change. start marks its time; end is omitted for a point event such as a dialnorm change. channel is set for events of one channel. value and limit depend on the kind (for example the new and the previous dialnorm for dialnorm_change), and message describes the event in words. More kinds may be added later.

An audio_metadata record is written for AC-3 and E-AC-3 streams, at the first frame and every time a value changes. changed lists the fields that changed. Its fields are codec (ac3 or eac3), dialnorm (in dB, -1 to -31), bsid, bsmod (bitstream mode), acmod (channel mode), lfe, compr, sample_rate, bitrate, frame_size and channel_layout, and for E-AC-3 also stream_type, substream_id and blocks_per_frame.

A loudness_summary record is written at the end of each audio stream. Its fields:

  • codec, language, sample_rate, channels, channel_layout
  • start, end and duration of the measured audio
  • integrated, lra with its lower and upper bounds lra_low and lra_high
  • momentary_max and short_term_max: the highest momentary and short-term loudness
  • true_peak and sample_peak: the highest peaks of the whole stream
  • silence_duration: the time in seconds that the stream was silent
  • clipped_samples: the number of clipped samples
  • decode_errors: the number of audio packets that could not be decoded
  • for AC-3 and E-AC-3: dialnorm (the value used for the longest time), dialnorm_min, dialnorm_max, dialnorm_changes, and dialnorm_difference (the integrated loudness minus the dialnorm)
  • targets: one entry per loudness target, with its name (for example ebu_r128 or atsc_a85), the target level, tolerance and max_true_peak, the deviation from the target level, and the results loudness_ok, true_peak_ok, dialnorm_ok (only for targets that check dialnorm) and ok

A loudness_summary record of an AC-3 stream (shortened):

{
"schema_version": "0.1.9",
"type": "loudness_summary",
"input": "0",
"stream": 1,
"program": 1,
"pid": 257,
"codec": "ac3",
"duration": 20.0,
"channels": 6,
"channel_layout": "5.1(side)",
"integrated": -30.988,
"lra": 0.0,
"true_peak": -9.99,
"sample_peak": -9.998,
"silence_duration": 0.0,
"clipped_samples": 0,
"decode_errors": 0,
"dialnorm": -31,
"dialnorm_changes": 0,
"dialnorm_difference": 0.012,
"targets": [
{"name": "ebu_r128", "target": -23.0, "tolerance": 0.2, "max_true_peak": -1.0,
"deviation": -8.0, "loudness_ok": false, "true_peak_ok": true, "ok": false},
{"name": "atsc_a85", "target": -24.0, "tolerance": 2.0, "max_true_peak": -2.0,
"deviation": -7.0, "loudness_ok": false, "true_peak_ok": true, "dialnorm_ok": true, "ok": false}
]
}

The targets of a run are listed in run_info.options.loudness_targets.

summary.json​

summary.json groups the results by input. It holds the run_info of the run and one entry per input:

{
"schema_version": "0.1.9",
"run_info": { "...": "..." },
"inputs": [
{
"id": "0",
"uri": "channel1.ts",
"label": "channel1.ts",
"stream_info": [ { "...": "..." } ],
"alignments": [ { "...": "..." } ],
"model_scores": [ { "model": "p1204.3", "scope": "overall", "...": "..." } ],
"summaries": [ { "record_type": "frame_bitstream", "metric": "qp_avg", "...": "..." } ],
"comparisons": [ { "...": "..." } ],
"loudness": [ { "stream": 1, "integrated": -23.994, "targets": [ { "...": "..." } ], "events": {} } ]
}
]
}

The entries are the records of the NDJSON output, without their type and schema_version fields. model_scores holds only the overall scores; the scores per second and per GOP are only in records.ndjson and model_score.csv. comparisons are listed under the encode they belong to. loudness holds the loudness_summary of each audio stream, with events, the number of audio events of each kind. Event kinds with no occurrences are omitted. Empty lists are left out.

summary.json is written at the end of the run. When you use the terminal view and compute additional metrics on demand, it is written when you close the view.

CSV files​

The CSV files contain the same records as records.ndjson, one file per record type, for example frame_bitstream.csv or summary.csv. Nested fields are flattened into columns with dotted names (for example vmaf.score in frame_fullref.csv). run_info is not written to CSV. stream_info.csv has one row per stream.

For audio:

  • loudness.csv: the loudness over time, one row every 100 ms
  • audio_block.csv: one row per channel and second, with a channel column
  • audio_event.csv: one row per audio event
  • audio_metadata.csv: one row per AC-3 or E-AC-3 metadata record
  • loudness_summary.csv: one row per audio stream, with a group of columns per target, for example targets.ebu_r128.deviation, targets.ebu_r128.ok or targets.atsc_a85.dialnorm_ok

JSON Schema​

surfmeter-media schema prints the JSON Schema of all record types:

surfmeter-media schema > surfmeter-media-records.schema.json

Use it to validate the output, or to generate types for your own code, for example with quicktype or datamodel-code-generator for Python.

The schema_version field of each record tells you which version of the format it follows. New fields may be added in later versions; your code should ignore fields it does not know.

Records of on-demand jobs​

When you compute additional metrics in the terminal view (see Basic Usage), their records are added to the output files with a job field (a job column in CSV). The final run_info lists these jobs under jobs.