Improving video playback with ExoPlayer
Video has become an important part of our lives and is now commonly integrated into various mobile applications.
Reddit is no exception. We have over 10 different video surfaces:
In this article, I will share practical tips, supported by production data, on how to improve playback from different perspectives and effectively use ExoPlayer in your Android app.
This article will be beneficial if you are an Android Engineer and familiar with the basics of the ExoPlayer and Media 3.
Delivery
There are several popular ways to deliver Video on Demand (VOD) from the server to the client.
Binary
The simplest way is to get a binary file, like an mp4, and play it on the client. It works great and works on all devices.
However, there are drawbacks: For instance, the binary approach doesn’t automatically adapt to changes in the network and only provides one bitrate and resolution. This may be not ideal for longer videos, as there may not be enough bandwidth to download the video quickly.
Adaptive
To tackle the bandwidth drawback with binary delivery, there’s another way — adaptive protocols, like HLS developed by Apple and DASH by MPEG.
Instead of directly getting the video and audio segments, these protocols work by getting a manifest file first. This manifest file has various segments for each bitrate, along with separate tracks for audio and video.
After the manifest file is downloaded, the protocol’s implementation will choose the best video quality based on your device’s available bandwidth. It’s smart enough to adapt the video quality on the fly, depending on your network’s condition. This is especially useful for longer videos.
It’s not perfect, however. For example, to start playing the video in DASH, it may take at least 3 round trips, which involve fetching the manifest, audio segments, and video segments.
This may increase the chance of a network error.
On the other hand, in HLS, it may take 4 round trips, including fetching the master manifest, manifest, audio segments, and video segments.
Reddit experience
Historically, we have used DASH for all video content for Android and HLS for all video content for Web and iOS. However, about 75% of our video content is less than 45 seconds long.
For short videos, we hypothesize that it is not necessary to be switching bitrate during the playbacks.
To verify our theory, we conducted an experiment where we served certain videos in MP4 format instead of DASH, with different duration limitations.
We observed that a 45-second limitation showed the most pragmatic result:
- Playback errors
decreased by 5.5% - Cases where users left a video before playback started (Exit Before Video Start in the future)
decreased by 2.5% - Overall video view
increased by 1.7%
Based on these findings, we’ve made the decision to serve all videos that are under 45 seconds in pure MP4 format. For longer videos, we'll continue to serve them in adaptive streamable format.
Caching & Prefetching
The concept of prefetching involves fetching content before it is displayed and showing it from the cache when the user reaches it. However, first we need to implement caching, which may not be straightforward.
Let’s review the potential problems we may encounter with this.
ExternalDir isn’t available everywhere
In Android, we have two options for caching: internal cache or external cache. For most apps, using internalDir is a practical choice, unless you need to cache very large video files. In that case, externalDir may be a better option.
It's important to note that the system may clean up the internalDir if your application reaches a certain quota, while the external cache is only cleaned up if your application is deleted (if it's stored under the app folder).
At Reddit, we initially attempted to cache video in the externalDir, but later switched to the internalDir to avoid compatibility issues on devices that do not have it, such as OPPO.
SimpleCache may clean other files
If you take a look at the implementation of SimpleCache, you’ll notice that it’s not as simple as its name suggests.
So, SimpleCache could potentially remove other cache files unless there is a specific dedicated folder that may affect other app logic, be careful with this.
By the way, I spent a lot of time studying the implementation, but I missed those lines. Thanks to Maxim Kachinkin for bringing them to my attention.
SimpleCache hits disk on the constructor
We encountered a lot of ANRs (Application Not Responding) while SimpleCache was being created. Diving into the implementation, I realized it was hitting disk in constructor:
So make sure to create this instance on a background thread to avoid this.
URL uses as a cache-key
This is by default. However, if your URL is different due to signing signature or additional parameters, make sure to provide a custom cache key factory for the data source. This will help increase cache-hit and optimize performance.
Eviction should be explicitly enabled
Eviction is a pretty nifty strategy to prevent cached data from piling up and causing trouble. Lots of libraries, like Glide, actually use it under the hood. If video content is not the main focus of your app, SimpleCache also allows for easy implementation in just one line:
Prefetching options
Well. You have 5 prefetching options to choose from: DownloadManager, DownloadHelper, DashUtil, DashDownloader, and HlsDownloader.
In my opinion, the easiest way to accomplish this is by using DownloadManager. You can integrate it with ExoPlayer, and it uses the same SimpleCache instance to work:
It’s also really customizable: for instance, it lets you pause, resume, and remove downloads, which can be really handy when users scroll too quickly and ongoing download processes are no longer necessary. It also provides a bunch of options for threading and parallelization.
For prefetching adaptive streams, you can also use DownloadManager in combination with DownloadHelper that simplifies that job.
Unfortunately, one disadvantage is that there is currently no option to preload a specific amount of video content (e.g., around 500kb), as mentioned in this discussion.
Reddit experience
We tried out different options, including prefetching only the next video, prefetching 2 next videos in parallel or one after the other, and only for short video content (mp4).
After evaluating these prefetching approaches, we discovered that implementing a prefetching feature for only the next video yielded the most practical outcome.
- Video load time < 250 ms:
didn’t change - Video load time < 500 ms:
increased by 1.9% - Video load time > 1000 ms:
decreased by 0.872% - Exit before video starts:
didn’t change
To further improve our experiment, we want to consider the users’ internet connection strength as a factor for prefetching. We conducted a multi-variant experiment with various bandwidth options, starting from 2 mbps up to 20 mbps.
Unfortunately, this experiment wasn’t successful. For example, with a speed of 2 mbps:
- Video load time < 250 ms:
decreased by 0.9% - Video load time < 500 ms:
decreased by 1.1% - Video load time > 1000 ms:
increased by 3%
In the future, we also plan to experiment with this further and determine if it would be more beneficial to partially prefetch N videos in parallel.
LoadControl
Load control is a mechanism that allows for managing downloads. In simple terms, it addresses the following questions:
- Do we have enough data to start playback?
- Should we continue loading more data?
And a cool thing is that we can customize this behavior!
bufferForPlaybackMs, default: 2500
Refers to the amount of video content that should be loaded before the first frame is rendered or playback is interrupted by the user (e.g., pause/seek).
bufferForPlaybackAfterRebufferMs, default: 5000
Refers to the amount of data that should be loaded after playback is interrupted due to network changes or bitrate switch
minBuffer & maxBuffer, default: 50000
During playback, ExoPlayer buffers media data until it reaches maxBufferMs. It then pauses loading until the buffer decreases to the minBufferMs, after which it resumes loading.
You may notice that by default, these values are set to the same value. However, in earlier versions of ExoPlayer, these values were different. Different buffer configuration value could lead to increased rebuffering when the network is unstable.
By setting these values to the same value, the buffer is consistently filled up. (This technique is called Drip-Feeding).
If you want to dig deeper, there are very good articles about buffers:
- How to get 4 times better re-buffering with drip-feeding technique in ExoPlayer on Android
- Enhancing Video Streaming Quality for Exoplayer
Reddit Experience
Since most of our videos are short, we noticed that the default buffer values were a bit too lengthy. So, we thought it would be a good idea to try out some different values and see how they work for us.
We found that setting bufferForPlaybackMs and bufferForPlaybackAfterRebufferMs = 1 000, and minBuffer and maxBuffer = 20,000, gave us the most pragmatic results:
- Video load time < 250 ms:
increased by 2.7% - Video load time < 500 ms:
increased by 4.4% - Video load time > 1000 ms:
decreased by 11.9% - Video load time > 2000 ms:
decreased by 17.7% - Rebuffering
decreased by 4.8% - Overall video views
increased by 1.5%
So far this experiment has been one of the most impactful that we ever conducted from all video experiments.
Improving adaptive bitrate with BandwidthMeter
mproving video quality can be challenging because higher quality often leads to slower download speeds, so it’s important to find a proper balance in order to optimize the viewing experience.
To select the appropriate video bitrate and ensure optimal video quality based on the network, ExoPlayer uses BandwidthMeter.
It calculates the network bandwidth required for downloading segments and selects appropriate audio and video tracks based on that for subsequent videos.
Reddit experience
At some point, we noticed that although users have good network bandwidth, we don’t always serve the best video quality.
The first issue we identified was that prefetching doesn’t contribute to overall network bandwidth in BandwidthMeter, as DataSource in DownloadManager doesn’t know anything about it. The fix is to include prefetching when considering the overall bandwidth.
And conducted experiment to confirm on production, which yielded the following result:
- Better video resolution:
increased by 1.4% - Overall chained video viewing:
increased by 0.5% - Bitrate changing during playback:
decreased by 0.5% - Video load time > 1000 ms:
increased by 0.3%(Which is a trade-off)
It is worth mentioning that the current BandwidthMeter is still not perfect in calculating the proper video bitrate. In media 1.0.1, an ExperimentalBandwidthMeter has been added, which will eventually replace the old one that should improve the state of things.
Additionally, by default, BandwidthMeter uses hardcoded values which are different depending on network type and country. It may be not relevant for the current network and in general could be not accurate. For instance, it considers Great Britain 3G faster than 4G.
We haven’t experimented with this yet, but one way to address this would be to remember the latest network bandwidth and setting it up when application starts:
There are also a few customizations available in AdaptiveTrackSelection.Factory to manage when to switch between better and worse quality: minDurationForQualityIncreaseMs (default value: 15 000) and minDurationForQualityDecreaseMs (default value: 25000) that may help with this.
Choosing a bitrate for MP4 Content
If videos are not the primary focus of your application and you only use them, for instance, to showcase updates or year reviews, sticking with an average bitrate may be pragmatic.
At Reddit, when we first transitioned short videos to mp4, we began sending the current bandwidth to receive the next video batch.
However, this solution is not very precise as bandwidth may fluctuate more frequently. We decided to improve it this way:
The main difference between this implementation (second diagram) and adaptive bitrate (DASH/HLS) is that we do not need to prefetch the manifest first (as we obtain it when fetching the video batch), reducing the chances of network errors. Also, the bitrate will remain constant during playback.
When we were experimenting with this approach, we initially relied on approximate bitrates for each video and audio, which was not precise. As a result, the metrics did not move in the right direction:
- Better video quality:
increased by 9.70% - Video load time > 1000 ms:
increased by 12.9% - Overall video view
decreased by 2%
In the future, we will experiment with exact video and audio bitrates, as well as with thresholds, to achieve a good balance between download time and quality.
Decoders & Player instances
At some point, we noticed a spike of 4001 playback error, which indicates that the decoder is not available. This problem appeared on almost every android vendor.
Each device has limitations in terms of available decoders and this issue may occur, for instance, when another app has not released the decoder properly.
While we may not be able to mitigate the decoder issue 100%, ExoPlayer provides an opportunity to switch to a software decoder if a primary one isn't available:
Although this solution is not ideal, as falling back to software decoder can perform slower than hardware decoder, it is better than not being able to play the video. Enabling the fallback option during experimentation resulted in a 0.9% decrease in playback errors.
To reduce such cases, ExoPlayer uses the audio manager and can request focus on your behalf. However, you need to explicitly do so:
Another thing that could help is to use only one instance of ExoPlayer per app. Initially, this may seem like a simple solution. However, if you have videos in feeds, manually managing thumbnails and last frames can be challenging. Additionally, if you want to reuse already initialized decoders, you need to avoid calling stop() and call prepare() with new video on top of current playback.
On the other hand, synchronizing multiple instances of ExoPlayer is also a complex task and may result in audio bleeding issues as well.
At Reddit, we reuse video players when navigating between surfaces. However, when scrolling, we currently create a new instance for each video playback, which adds unnecessary overhead.
We are currently considering two options: a fixed player pool based on the availability of decoders, or using a single instance. Once we conduct the experiment, we will write a new blog post to share our findings.
Rendering
We have two choices: TextureView or SurfaceView. While TextureView is a regular view that is integrated into the view hierarchy, SurfaceView has a different rendering mechanism. It draws in a separate window directly to the GPU, while TextureView renders to the application window and needs to be synchronized with the GPU, which may create overhead in terms of performance and battery consumption.
However, if you have a lot of animations with video, keep in mind that prior to Android N, SurfaceView had issues in synchronizing animations.
ExoPlayer also provides default controls (play/pause/seekbar) and allows you to choose where to render video.
Reddit experience
Historically, we’ve been using TextureView to render videos. However, we are planning to switch to SurfaceView for better efficiency.
Currently, we are migrating our features to Jetpack Compose and have created composable wrappers for videos. One issue we face is that, since most of our main feeds are already in Compose, we need to constantly reinflate videos, which can take up to 30ms according to traces, causing frame drops.
To address this, Jetpack Compose 1.4 introduced a ViewPool where you need to override callbacks:
However, we decided to implement our own ViewPool to potentially reuse inflated views across different screens and have more control in the future, like pre-initializing them before displaying the first video:
This implementation resulting in the following benefits:
- Video load time < 250 ms:
increased by 1.7% - Video load time < 500 ms:
increased by 0.3% - Video minutes watched
increased by 1.4% - Creation P50:
1ms, improvedx30 - Creation P90:
24ms, improvedx1.5
Additionally, since default ExoPlayer controls are implemented by using old-fashioned views, I’d recommend always implementing your own controls to avoid unnecessary inflation.
There are wrappers for SurfaceView is already available in Jetpack Compose 1.6: AndroidExternalSurface and AndroidEmbeddedExternalSurface.
In Summary
One of the key things to keep in mind when working with videos is the importance of analytics and regularly conducting A/B testing with various improvements.
This not only helps us identify positive changes, but also enables us to catch any regression issues.
If you just started to working with videos, consider to have at least next events:
- First frame rendered (time)
- Rebuffering
- Playback started/stopped
- Playback error
ExoPlayer also provides an AnalyticsListener which can help with that.
Additionally, I must say that working with videos has been quite a challenging experience for me. But hey, don’t worry if things don’t go exactly as planned for you too — it’s completely normal.
In fact, it’s meant to be like this.
If working with videos were a song, it would be “Trouble” by Cage the Elephant.
Thanks for reading. If you want to connect and discuss this further, please feel free to reach out. Also props to my past colleague Jameson Williams, who had direct contributions to some of the improvements mentioned here.
Thanks to the following folks for helping me review this — Irene Yeh, Merve Karaman, Farkhad Khatamov, Matt Ewing, and Tony Lenzi.
