ExoPlayer Quality Test¶

Once registered, you can create a Builder for the respective QualityTest which can also be launched in the onCreate method of your activity. In our example we will be using the ExoPlayerQualityTest builder. Have a look at the ExoplayerTestActivity in the demo app for more info.

For our ExoPlayerQualityTest, we will play a DASH stream with a known manifest URL. The playback duration is 30 seconds in the below example, and the maximum test duration is a timeout for the test in case of network problems. The analysis duration is a timeout for the actual analysis of the results on the remote server.

Starting the Test¶

Here is how you can start the test:

ExoPlayerQualityTest mQTest;

ExoPlayerQualityTest.Builder builder = new ExoPlayerQualityTest.Builder(
    this,
    Uri.parse("https://example.com/manifest.mpd")
)
.setMaxPlaybackDuration(30 * 1000)
.setMaxTestDuration(60 * 1000)
.setMaxAnalysisDuration(10 * 1000)
.setResultListener(this)
.setProgressInterval(1000);

try {
    mQTest = builder.build();
    mQTest.start();
} catch (Exception e) {
    Log.e(TAG, "Test start failed", e);
    // Handle exception
}

Builder Options¶

The ExoPlayerQualityTest.Builder supports the following configuration methods:

• setMaxPlaybackDuration(long ms) – maximum playback duration in milliseconds
• setMaxTestDuration(long ms) – maximum test duration (timeout) in milliseconds
• setMaxAnalysisDuration(long ms) – maximum analysis duration in milliseconds
• setResultListener(ExoPlayerQualityTestResultListener listener) – listener for test results, errors, and state changes
• setProgressInterval(int ms) – interval for progress callbacks in milliseconds
• setCalculationSettings(HashMap<String, Object> settings) – custom calculation settings (e.g., pv and pq model names)
• setExternalPlayer(ExoPlayer player) – use an externally created ExoPlayer instance instead of the internal one
• attachToView(FrameLayout frame) – attach the player to a view for visual playback during the test
• setInitialBitrateEstimate(long bps) – initial bitrate estimate for the bandwidth meter in bits per second. Higher values cause the player to start with higher quality renditions (e.g., 50_000_000 for 50 Mbps).
• setForceHighestSupportedBitrate(boolean force) – force the player to always select the highest supported bitrate, bypassing adaptive bitrate selection
• setIgnoreViewportConstraints(boolean ignore) – ignore viewport size constraints imposed by the PlayerView. When a PlayerView is attached, ExoPlayer limits track selection to the view's layout dimensions (e.g., a small view may cap selection at 480p). Setting this to true clears those constraints so the ABR algorithm can select higher quality tracks regardless of display size.
• setLoopPlayback(boolean loop) – loop the media within a single test until the max playback duration is reached. This is different from the demo app's "Loop test" feature described below, which repeats whole test runs rather than looping the media of one test.

Calculation Settings¶

Calculation settings select modules and adjustments within the combined video QoE calculation. In the Video QoE model, pv is the video-quality module. It produces a video MOS value for each second (O.22). The pq module then combines those video values with the audio quality and stalling events to produce the final session MOS (O.46). See the P.1203 module diagram for this data flow.

You normally do not need to call setCalculationSettings. The Android SDK uses these defaults:

• pv: P12043BitstreamMode0. Despite this legacy internal name, it selects the ITU-T P.1204.1 Mode 0 video-quality model. This metadata-based model uses the codec, bitrate, resolution, frame rate, and reported display size; it does not inspect the encoded bitstream.

• pq: P1203PqExtended. This is the P.1203.3 integration model with additional diagnostic outputs.

• amendment1Audiovisual: true. This increases the effect of very low audiovisual quality on the final MOS.

• amendment1Stalling: true. This increases the effect of stalling on the final MOS.

You can override individual defaults as follows:

HashMap<String, Object> calculationSettings = new HashMap<>();
calculationSettings.put("pv", "P1203PvRetrained");
calculationSettings.put("pq", "P1203PqExtended");
calculationSettings.put("amendment1Audiovisual", true);
calculationSettings.put("amendment1Stalling", true);
calculationSettings.put("amendment1App2", false);

builder.setCalculationSettings(calculationSettings);

The current on-device calculator supports P12043BitstreamMode0 and P1203PvRetrained for pv. P1203PvRetrained uses P.1203.1 Mode 0 with retrained coefficients for additional codecs. The P.1203 extensions and variants page compares both models and their supported inputs.

P1203PqExtended is the supported pq value. amendment1App2 applies an Appendix 2 adjustment when externally calculated bitstream- or pixel-based video scores are used; leave it false for the metadata-based models above.

The builder's validation also accepts the server-side model identifiers P1203PvExtended, P1203PvHveiExtended, and P1203PqM, as well as the longMode key. The current on-device statistics calculator does not implement these model choices, and longMode does not change its calculation. Do not use them for an ExoPlayerQualityTest.

Receiving Test Results¶

To receive the outcomes of the test, your activity or class must implement the ExoPlayerQualityTestResultListener interface. This interface includes methods for handling successful results, errors, and state changes.

Test Success¶

When the test completes successfully, the onTestResult method is called:

@Override
public void onTestResult(@NonNull Map<String, Object> result) {
    Log.i(TAG, "Test result received: " + result);
    // Convert the result to a JSON string for display or storage
    Gson gson = new GsonBuilder().setPrettyPrinting().create();
    String resultJson = gson.toJson(result);
    // Process or display the resultJson
}

The results correspond to the measurement data, so please refer to that page for more information on the values.

Test Errors¶

If an error occurs during the test, the onTestError method is invoked:

@Override
public void onTestError(@NonNull QualityTestException error) {
    // Handle the error, e.g., display a message to the user
    Log.e(TAG, "Test error: " + error.getMessage(), error);
}

Test State Changes¶

You can monitor the lifecycle of the test by implementing the onTestStateChanged method from the ExoPlayerQualityTestResultListener interface:

@Override
public void onTestStateChanged(QualityTestState state) {
    Log.d(TAG, "Test state: " + state);
}

Performance Monitoring Datasource¶

We provide a special loader for ExoPlayer that monitors the performance of each segment request. This is useful for performance monitoring, connection troubleshooting, etc.

To use it, you must instantiate the ExoPlayer yourself (i.e., not use the ExoPlayerQualityTest's internal ExoPlayer instance). Before you instantiate the ExoPlayer itself, call:

import com.aveq.qoereporting.PerformanceMonitoringHttpDataSourceFactory;
import com.aveq.qoereporting.PerformanceMonitoringHttpDataSource.PerformanceListener;

// ...

DefaultHttpDataSource.Factory defaultHttpDataSourceFactory = new DefaultHttpDataSource.Factory();
HttpDataSource.Factory performanceMonitoringFactory =
        new PerformanceMonitoringHttpDataSourceFactory(defaultHttpDataSourceFactory, this);
MediaSource.Factory mediaSourceFactory = new DefaultMediaSourceFactory(performanceMonitoringFactory);

ExoPlayer mPlayer = new ExoPlayer.Builder(mContext)
    .setMediaSourceFactory(mediaSourceFactory)
    .build();

Then pass that ExoPlayer instance to the ExoPlayerQualityTest builder:

ExoPlayerQualityTest.Builder builder = new ExoPlayerQualityTest.Builder(
    this,
    Uri.parse("https://example.com/manifest.mpd")
)
.setExternalPlayer(mPlayer);

Now make sure your activity implements the PerformanceMonitoringHttpDataSource.PerformanceListener interface:

/**
 * Called when performance metrics are available for a segment request.
 *
 * @param url              The URL of the segment.
 * @param bytesTransferred The number of bytes transferred.
 * @param totalTime        The total time taken to transfer the segment in seconds.
 * @param ttfb             The time to first byte in milliseconds.
 * @param throughput       The throughput in kilobits per second.
 */
@Override
public void onRequestPerformance(String url, long bytesTransferred, double totalTime, long ttfb, double throughput) {
    // Handle the performance data
}

You will receive onRequestPerformance calls for each segment request.

Instrumented Usage¶

The ExoPlayerQualityTest can be initiated via adb by passing intent extras to the demo ExoplayerTestActivity. This is useful for automated testing scenarios.

adb shell am start -n "com.aveq.qualitytestlibdemo/.ExoplayerTestActivity" \
  --ez autoRunTest true \
  --el maxPlaybackDuration 30000 \
  --el maxTestDuration 45000 \
  --ez displayPlayer true

The following intent extras are supported:

• autoRunTest (boolean) – start the test automatically on activity launch
• maxPlaybackDuration (long) – max playback duration in ms
• maxTestDuration (long) – max test duration in ms
• manifestUri (string) – manifest URI to test
• displayPlayer (boolean) – show the player view during the test
• pvModel (string) – Pv model name
• pqModel (string) – Pq model name
• initialBitrateEstimate (long) – initial bitrate estimate in bps
• forceHighestBitrate (boolean) – force highest supported bitrate
• ignoreViewportConstraints (boolean) – ignore viewport size constraints for track selection
• loopTest (boolean) – repeat the test automatically until stopped, see Looping Tests below

We provide a shell script to run the quality test using the above call, which extracts the test results directly from the Android log. See ./run_exoplayer_quality_test.sh for more info.

Looping Tests¶

The demo ExoplayerTestActivity can repeat a test automatically until you press Stop, using the "Loop test" checkbox in the settings screen (or the loopTest intent extra above). This repeats the whole test with the same settings, waiting a couple of seconds between two runs — it is distinct from setLoopPlayback, which loops the media within a single test.

While a loop is running, the result screen is not shown after each individual test, so the app can keep going unattended. Each test's result is still delivered as usual through ExoPlayerQualityTestResultListener#onTestResult. Press Stop at any time to end the loop after the current test finishes.