Video Quality Test¶
For testing video playback quality on web-based streaming platforms like YouTube or Netflix, you can use the VideoQualityTest. This test allows you to analyze video quality metrics for various streaming services. Have a look at the VideoTestActivity in the demo app for more info.
Warning
Do not instantiate the builder before the registration is complete, as you will receive an exception. Make sure your application is checking the registration before starting the test.
Note that this is slightly different from the ExoPlayerQualityTest, and uses a different builder class/result listener class.
Here is a breakdown of the essential parts of the test. Note that in contrast to the ExoPlayer quality test, the VideoQualityTest requires a FrameLayout to be attached to the test, to display the video.
Starting the Test¶
This is how you would start the test:
// attach the FrameLayout to the test --> see the demo app, `activity_video.xml` layout for more info
FrameLayout videoFrame = findViewById(R.id.videoFrame);
VideoQualityTest.Builder builder = new VideoQualityTest.Builder(
this,
"netflix_trailer", // the subject identifier, can be "youtube", "netflix_trailer", or "facebook"
Uri.parse("https://www.netflix.com/tudum/videos/your-trailer-url"),
30 * 1000 // 30 seconds playback duration
)
.setMaxTestDuration(60 * 1000) // 60 seconds timeout
.setVideoQualityTestResultListener(this)
.attachToView(videoFrame); // attach to a FrameLayout in your layout
try {
mQTest = builder.build();
mQTest.start();
} catch (Exception e) {
Log.e(TAG, "Test start failed", e);
// Handle exception
}
Which URLs/subjects are supported? Please check the below:
For Netflix content, currently only trailer URLs from Netflix's Tudum site are supported:
For YouTube content, you can use any YouTube video URL from the main YouTube site:
new VideoQualityTest.Builder(
this,
"youtube",
Uri.parse("https://www.youtube.com/watch?v=your-video-id"),
30 * 1000
);
However, this may play back the video in a smaller resolution, hence we recommend using embed URLs instead:
Receiving Test Results¶
To receive the outcomes of the test, your activity or class must implement the VideoQualityTestResultListener interface. This interface includes methods for handling successful results, errors, and state changes.
Test Success¶
When the test completes successfully, the onTestResult method is called:
@Override
public void onTestResult(@NonNull Map<String, Object> result) {
Log.i(TAG, "Test result received: " + result);
// Convert the result to a JSON string for display or storage
Gson gson = new GsonBuilder().setPrettyPrinting().create();
String resultJson = gson.toJson(result);
// Process or display the resultJson
}
The results correspond to the measurement data, so please refer to that page for more information on the values.
Test Errors¶
If an error occurs during the test, the onTestError method is invoked:
@Override
public void onTestError(@NonNull QualityTestException error) {
// Handle the error, e.g., display a message to the user
Log.e(TAG, "Test error: " + error.getMessage(), error);
}
Test State Changes¶
You can monitor the lifecycle of the test by implementing the onTestStateChanged method from the VideoQualityTestResultListener interface:
@Override
public void onTestStateChanged(QualityTestState state) {
Log.d(TAG, "Test state: " + state);
}
Getting Periodic Statistics¶
You can also receive periodic statistics updates by implementing the VideoStatisticsListener interface in your class, then setting it in the builder:
This will trigger the onStatisticsUpdate method in your class, which you can use to update your UI or store the statistics in a database:
The values might look like this:
{
"averageAudioBitrate": 126.92089792420876,
"averageStallingTime": 0.0,
"averageTotalBitrate": 2068.632738783725,
"averageVideoBitrate": 1941.711840859516,
"averageVideoFramerate": 60.05286424104944,
"contentServerHostname": "",
"contentServerIpAddress": "",
"initialLoadingDelay": 1.549,
"initialResolution": 1080.0,
"largestPlayedVideoSize": 1080.0,
"longestPlayedChunk": 1080.0,
"mostUsedAudioCodec": "aaclc",
"mostUsedVideoCodec": "h264",
"numberOfQualitySwitches": 0.0,
"numberOfStallingEvents": 0.0,
"p1203AverageAudioQuality": 4.553492832693684,
"p1203AverageVideoQuality": 4.2279949249820765,
"p1203MaxMosRatio": 0.8531772087203596,
"p1203MaxTheoreticalMos": 4.77197412302386,
"p1203OverallAudiovisualQuality": 5.0,
"p1203OverallMos": 4.071339562367283,
"p1203StallingQuality": 4.171710787150921,
"qualitySwitchDownCount": 0.0,
"qualitySwitchUpCount": 0.0,
"stallingRatio": 0.0,
"totalPlayTime": 10.316056,
"totalStallingTime": 0.0,
"videoCompletionRatio": 0.016255971862634947
}
Please refer to our statistic values reference for more information on the values.
Video State and Progress Callbacks¶
The VideoQualityTestResultListener interface also provides methods to track video state changes, duration, and playback progress:
@Override
public void onVideoStateChanged(String videoState, Double clientTime, Double mediaTime) {
// videoState can be: 'unstarted', 'stalling', 'playing', 'seeking', 'ended', 'paused'
Log.i(TAG, "Video state changed: " + videoState +
", clientTime=" + clientTime +
", mediaTime=" + mediaTime);
}
@Override
public void onVideoDuration(Double duration) {
// Called when the video duration is known (in seconds)
Log.i(TAG, "Video duration: " + duration);
}
@Override
public void onPlaybackProgress(int progress) {
// progress is an integer between 0-100 representing playback percentage
Log.i(TAG, "Playback progress: " + progress + "%");
// update a progress bar or other UI element
progressBar.setProgress(progress);
}
These callbacks make it easier to track the video playback state and update your UI accordingly.
Instrumented Usage¶
The VideoQualityTest can be initiated via adb by passing intent extras to the demo VideoTestActivity. This is useful for automated testing scenarios.
adb shell am start -n "com.aveq.qualitytestlibdemo/.VideoTestActivity" \
--ez autoRunTest true \
--es url "https://www.netflix.com/tudum/videos/your-trailer-url"
--es subject "netflix_trailer"
--el maxPlaybackDuration 30000
Note that these intent parameters are directly passed to the video quality test; check the demo app source code for more info on how they are handled.
We provide a shell script to run the quality test using the above call, which extracts the test results directly from the Android log. See ./run_video_quality_test.sh for more info.
Background Execution¶
The SDK supports two modes of running tests in the background. This is useful for automated testing scenarios or when you want to run tests periodically without user interaction. Here are the modes:
- A background service that runs in the background while your app is visible on screen
- A foreground service that runs in the background while your app is not visible on screen. This is useful for running tests periodically without user interaction.
And yes, it's a head-scratcher why it's called a foreground service when it is actually running in the background 🤨 … to clear up the confusion, the demo app calls the actual foreground tests "in-activity tests."
We recommend you use the background service for most use cases, as it is less intrusive and does not require an extra permission. A foreground service displays a visible notification and causes an extra notification prompt, which is not desirable for some users. However, if you cannot guarantee your app to stay on-screen, then you must use the foreground service.
Background Service¶
To instantiate a background service based test, you need to use the QualityTestService class.
Bundle parameters = new Bundle();
parameters.putParcelable(QualityTestService.ParameterKeys.URL, Uri.parse("https://www.youtube.com/watch?v=dQw4w9WgXcQ"));
parameters.putString(QualityTestService.ParameterKeys.SUBJECT, "youtube");
parameters.putLong(QualityTestService.ParameterKeys.MAX_PLAYBACK_DURATION_MS, 30 * 1000);
parameters.putLong(QualityTestService.ParameterKeys.TIMEOUT_MS, 60 * 1000);
Intent intent = new Intent(this, QualityTestService.class);
intent.setAction(QualityTestService.ACTION_START_TEST);
intent.putExtra(QualityTestService.EXTRA_TEST_TYPE, QualityTestService.TEST_TYPE_VIDEO);
intent.putExtra(QualityTestService.EXTRA_TEST_PARAMETERS, parameters);
startService(intent);
This will start a background test that will run for 30 seconds and timeout after 60 seconds. The test will be delivered via broadcast intents to the TestResultReceiver class. Have a look at the setupTestResultReceiver method in the BaseTestActivity class in the demo app for more info on how to set it up and register it.
Regularly Scheduling Tests
Our demo app further includes a scheduling feature using the WorkManager library. This is a good example of how to schedule periodic tests, but note that the minimum interval for periodic tests is 15 minutes because of a WorkManager limitation. See the schedulePeriodicBackgroundWork method in the VideoTestActivity class for more info.
Foreground Service¶
Before you can use the foreground service, your app needs to have the following permissions in the AndroidManifest.xml file:
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MEDIA_PLAYBACK"/>
<uses-permission android:name="android.permission.POST_NOTIFICATIONS"/>
You also need to request the permissions at runtime. Check out the getRequiredPermissionsList() function in the MainActivity.java file in the demo app.
To then instantiate a foreground service based test, you can also use the QualityTestService class, but you need to set the EXTRA_RUN_AS_FOREGROUND_SERVICE flag to true in the intent:
Bundle parameters = new Bundle();
parameters.putParcelable(QualityTestService.ParameterKeys.URL, Uri.parse("https://www.youtube.com/watch?v=dQw4w9WgXcQ"));
parameters.putString(QualityTestService.ParameterKeys.SUBJECT, "youtube");
parameters.putLong(QualityTestService.ParameterKeys.MAX_PLAYBACK_DURATION_MS, 30 * 1000);
parameters.putLong(QualityTestService.ParameterKeys.TIMEOUT_MS, 60 * 1000);
Intent intent = new Intent(this, QualityTestService.class);
intent.setAction(QualityTestService.ACTION_START_TEST);
intent.putExtra(QualityTestService.EXTRA_TEST_TYPE, QualityTestService.TEST_TYPE_VIDEO);
intent.putExtra(QualityTestService.EXTRA_TEST_PARAMETERS, parameters);
startService(intent);
intent.putExtra(QualityTestService.EXTRA_RUN_AS_FOREGROUND_SERVICE, true);
This will start a foreground service that will run for 30 seconds and timeout after 60 seconds. The test will be delivered via broadcast intents to the TestResultReceiver class. Have a look at the setupTestResultReceiver method in the BaseTestActivity class in the demo app for more info on how to set it up and register it.
Receiving Test Results via Broadcast¶
Of course, when a test is done, you want to receive the results.
When you start a test using QualityTestService (for either background or foreground service mode), the results are delivered via broadcast intents. To receive these, you need to implement and register a BroadcastReceiver.
The service will send intents with the following actions:
com.aveq.qualitytestlib.action.TEST_RESULT: Indicates a successful test completion. The results are in aBundleপাওয়া যাবেcom.aveq.qualitytestlib.extra.EXTRA_RESULT_DATAextra key. This bundle contains a serializableMap<String, Object>with the key"map".com.aveq.qualitytestlib.action.TEST_ERROR: Indicates a test failure. Error details are in aBundlein thecom.aveq.qualitytestlib.extra.EXTRA_ERROR_DATAextra. This bundle contains a"message"(String) and"code"(int).
You can also get the test type from the com.aveq.qualitytestlib.extra.EXTRA_TEST_TYPE extra in the intent.
To see an example of how to implement the TestResultReceiver class, check out the BaseTestActivity class in the demo app.
Note that you need to dynamically register your BroadcastReceiver in your Activity or another component that should listen for the results. Make sure to unregister it when it's no longer needed to prevent memory leaks.