Output Format
All results of the toolkit are records in one format, shared by all commands. This page describes the records and the files they are written to. For the individual metrics, see Metrics.
Files
With -o <directory>, the toolkit writes these files:
records.ndjson: all records of the run, one JSON object per line (formatndjson)summary.json: one JSON document with the most important results per input (formatjson)<record_type>.csv: one CSV file per record type, with nested fields flattened into columns (formatcsv)
--format selects the formats. The default is ndjson,json. Without -o, the NDJSON records are written to standard output.
Records
Every record has a type field and a schema_version field. The record types are:
run_info: information about the run: tool version, command line, inputs with their IDs, start and end time, versions of the components, host, options, and the finalstatusanderrorsstream_info: stream information of one input (container, programs, streams)scte35,scte35_window: cue messages and their time ranges in inputs with SCTE-35frame_bitstream: bitstream statistics of one video frameframe_pixels: pixel metrics of one video framemodel_score: a quality model score for a second, a GOP, or a whole streamalignment: the alignment of one encode with the reference (compareonly)frame_fullref: VMAF, PSNR and SSIM of one frame against the reference (compareonly)summary: statistics of one metric of one streamcomparison: the difference of one metric's statistics between an encode and the reference (compareonly)loudness: the loudness of one audio stream, every 100 msaudio_block: the levels of each channel of one audio stream, every secondaudio_event: a problem found in the audio, such as silence or clippingaudio_metadata: the AC-3 or E-AC-3 metadata of one audio stream, such as dialnormloudness_summary: the totals of one audio stream and the result of each loudness target
The run_info record is written twice: at the start of the run with status set to running, and at the end with the final status. When you read records.ndjson, use the last run_info record.
Common fields
Records that refer to an input have these fields:
input: the ID of the input, as listed inrun_info.inputs. Inputs are numbered from"0"in the order of the command line. Incompare,"0"is the reference.stream: the index of the stream within the input, as numbered by FFmpegprogram,pid: the program number and PID, for transport streams
Frame records also have:
n: the frame index in presentation order, starting at 0pts: the presentation timestamp in secondspts_rawandtime_base: the original timestamp and its time base, without loss of precision
All timestamps are the original timestamps of the file, including the start offset of transport streams. This lets you locate a frame with other tools, such as ffprobe. The terminal view shows times from the start of each input instead.
Optional fields are left out of a record when they have no value, rather than written as null.
Example
A frame_bitstream record of an H.264 frame (shortened):
{
"schema_version": "0.1.9",
"type": "frame_bitstream",
"input": "0",
"stream": 0,
"program": 1,
"pid": 256,
"n": 42,
"pts": 3.16,
"pts_raw": 284400,
"time_base": {"num": 1, "den": 90000},
"frame_type": "P",
"is_idr": false,
"size": 18734,
"qp_avg": 28.4,
"qp_stdev": 2.1,
"qp_min": 24,
"qp_max": 35,
"motion_avg": 3.2
}
A model_score record with the overall P.1204.3 score:
{
"schema_version": "0.1.9",
"type": "model_score",
"input": "0",
"stream": 0,
"model": "p1204.3",
"scope": "overall",
"start": 1.48,
"end": 7.48,
"device": "pc",
"display": {"width": 3840, "height": 2160},
"score": 3.87
}
Audio records
The audio layer writes these records for each audio stream. stream is the index of the audio stream. The times are the presentation times of the file, as for video. Loudness is in LUFS, loudness differences in LU, true peaks in dBTP, and sample peaks and RMS levels in dBFS. See Metrics for what the values mean.
A loudness record is written every 100 ms. pts is the end of the 100 ms interval. Its fields:
momentary: loudness of the last 400 msshort_term: loudness of the last 3 secondsintegratedandlra: integrated loudness and loudness range from the start of the stream up to this pointtrue_peakandsample_peak: the highest peak of all channels in the last 100 ms
A loudness value is left out when it cannot be measured yet, for example before the first 400 ms are complete, or when the audio is digital silence.
An audio_block record is written every second, from start to end. channels has one entry per channel, in the order of the stream. Each entry has the channel name (FL, FR, FC, LFE, SL, SR and so on), sample_peak, true_peak, rms, clipped_samples, and silent if the channel was silent.
An audio_event record describes a problem found in the audio. kind is one of silence, clipping, true_peak_over and dialnorm_change. start marks its time; end is omitted for a point event such as a dialnorm change. channel is set for events of one channel. value and limit depend on the kind (for example the new and the previous dialnorm for dialnorm_change), and message describes the event in words. More kinds may be added later.
An audio_metadata record is written for AC-3 and E-AC-3 streams, at the first frame and every time a value changes. changed lists the fields that changed. Its fields are codec (ac3 or eac3), dialnorm (in dB, -1 to -31), bsid, bsmod (bitstream mode), acmod (channel mode), lfe, compr, sample_rate, bitrate, frame_size and channel_layout, and for E-AC-3 also stream_type, substream_id and blocks_per_frame.
A loudness_summary record is written at the end of each audio stream. Its fields:
codec,language,sample_rate,channels,channel_layoutstart,endanddurationof the measured audiointegrated,lrawith its lower and upper boundslra_lowandlra_highmomentary_maxandshort_term_max: the highest momentary and short-term loudnesstrue_peakandsample_peak: the highest peaks of the whole streamsilence_duration: the time in seconds that the stream was silentclipped_samples: the number of clipped samplesdecode_errors: the number of audio packets that could not be decoded- for AC-3 and E-AC-3:
dialnorm(the value used for the longest time),dialnorm_min,dialnorm_max,dialnorm_changes, anddialnorm_difference(the integrated loudness minus the dialnorm) targets: one entry per loudness target, with itsname(for exampleebu_r128oratsc_a85), thetargetlevel,toleranceandmax_true_peak, thedeviationfrom the target level, and the resultsloudness_ok,true_peak_ok,dialnorm_ok(only for targets that check dialnorm) andok
A loudness_summary record of an AC-3 stream (shortened):
{
"schema_version": "0.1.9",
"type": "loudness_summary",
"input": "0",
"stream": 1,
"program": 1,
"pid": 257,
"codec": "ac3",
"duration": 20.0,
"channels": 6,
"channel_layout": "5.1(side)",
"integrated": -30.988,
"lra": 0.0,
"true_peak": -9.99,
"sample_peak": -9.998,
"silence_duration": 0.0,
"clipped_samples": 0,
"decode_errors": 0,
"dialnorm": -31,
"dialnorm_changes": 0,
"dialnorm_difference": 0.012,
"targets": [
{"name": "ebu_r128", "target": -23.0, "tolerance": 0.2, "max_true_peak": -1.0,
"deviation": -8.0, "loudness_ok": false, "true_peak_ok": true, "ok": false},
{"name": "atsc_a85", "target": -24.0, "tolerance": 2.0, "max_true_peak": -2.0,
"deviation": -7.0, "loudness_ok": false, "true_peak_ok": true, "dialnorm_ok": true, "ok": false}
]
}
The targets of a run are listed in run_info.options.loudness_targets.
summary.json
summary.json groups the results by input. It holds the run_info of the run and one entry per input:
{
"schema_version": "0.1.9",
"run_info": { "...": "..." },
"inputs": [
{
"id": "0",
"uri": "channel1.ts",
"label": "channel1.ts",
"stream_info": [ { "...": "..." } ],
"alignments": [ { "...": "..." } ],
"model_scores": [ { "model": "p1204.3", "scope": "overall", "...": "..." } ],
"summaries": [ { "record_type": "frame_bitstream", "metric": "qp_avg", "...": "..." } ],
"comparisons": [ { "...": "..." } ],
"loudness": [ { "stream": 1, "integrated": -23.994, "targets": [ { "...": "..." } ], "events": {} } ]
}
]
}
The entries are the records of the NDJSON output, without their type and schema_version fields. model_scores holds only the overall scores; the scores per second and per GOP are only in records.ndjson and model_score.csv. comparisons are listed under the encode they belong to. loudness holds the loudness_summary of each audio stream, with events, the number of audio events of each kind. Event kinds with no occurrences are omitted. Empty lists are left out.
summary.json is written at the end of the run. When you use the terminal view and compute additional metrics on demand, it is written when you close the view.
CSV files
The CSV files contain the same records as records.ndjson, one file per record type, for example frame_bitstream.csv or summary.csv. Nested fields are flattened into columns with dotted names (for example vmaf.score in frame_fullref.csv). run_info is not written to CSV. stream_info.csv has one row per stream.
For audio:
loudness.csv: the loudness over time, one row every 100 msaudio_block.csv: one row per channel and second, with achannelcolumnaudio_event.csv: one row per audio eventaudio_metadata.csv: one row per AC-3 or E-AC-3 metadata recordloudness_summary.csv: one row per audio stream, with a group of columns per target, for exampletargets.ebu_r128.deviation,targets.ebu_r128.okortargets.atsc_a85.dialnorm_ok
JSON Schema
surfmeter-media schema prints the JSON Schema of all record types:
surfmeter-media schema > surfmeter-media-records.schema.json
Use it to validate the output, or to generate types for your own code, for example with quicktype or datamodel-code-generator for Python.
The schema_version field of each record tells you which version of the format it follows. New fields may be added in later versions; your code should ignore fields it does not know.
Records of on-demand jobs
When you compute additional metrics in the terminal view (see Basic Usage), their records are added to the output files with a job field (a job column in CSV). The final run_info lists these jobs under jobs.