TLDR: Mobile app users are highly impatient, in particular when it comes to video streaming: users start abandoning a video after only 2 seconds of buffering, and buffering events can reduce video watch time by ~40%. Optimizations such as using a CDN and adjusting video size are crucial for ensuring fast video delivery, but they are not enough as the wireless last mile makes video startup time highly volatile. At Codavel, we ran our own testing to evaluate HTTP performance and, like many others in the industry, what we see is a highly unstable video delivery, with a very long tail of bad user experiences. I also provide an overview of what Codavel has to offer to solve these challenges.

High-quality video, without buffering: is it possible in a mobile app?

In a previous post, I went through the challenges that mobile apps face today with respect to video startup time. The data we’ve collected showed more than 68% of user sessions with a video startup time above 2 seconds, which is a strong sign of how hard it is to ensure a good user experience when streaming video to a mobile app. 

At Codavel, we felt the urge to further investigate the challenges of mobile video streaming. A good user experience requires way more than a fast startup: users demand to watch their favorite shows in high-quality video and without interruptions. Buffering events alone can reduce video watch time by ~40% and, as you have probably already felt that yourself, buffering happens, a lot. According to a report by Conviva on their customer base, a staggering 5.1B viewing hours were lost in Q2 2018 due to buffering. During this period, the rebuffering ratio was at 0.78% and engagement was at 27 minutes. If rebuffering was under 0.4%, up to 18 minutes of engagement could have been gained. The problem can be seen not only on the average behavior, with instability being a major source of user frustration. For example, according to data from Snapchat, 44% of users experience a 90th-percentile buffering duration.

A very long tail of bad experiences

At Codavel, we decided to take a deeper look at this phenomenon, by analyzing the video streaming experience in 213 user sessions in India. We focused our analysis on the three major KPIs for video streaming performance: video quality, buffering, and startup time. In a nutshell, what we observe is a long tail of bad experiences. 

The video stream under analysis allowed for playback in 8 different video qualities, via Dynamic Adaptive Streaming over HTTP (DASH). The available qualities range from 150Kbps up to the maximum quality of 3.5Mbps, where the quality to be requested was decided by the video player, according to the perceived available bandwidth (as well as other factors). Looking at the average video quality per user session, we observed an average quality of 2,06Mbps, considerably far from the maximum quality of 3.5Mbps.

When we look at the playback time on each different quality version of the video, we can see that more than 40% of the playback time is in qualities of 1.1Mbps or lower. Even worse, when we look at 10% worst user sessions, the average quality is below 580Kbps. The 1% worst sessions never leave the lower quality available, 150Kbps.

These video quality results are a clear sign of a highly unstable streaming experience. Let us look into what startup time and rebuffering data have to tell us.

While the median video startup time lies at 1.5s, the 95th-percentile is 4.3s and the 99th-percentile is 5.1s. In other words, from the data we collected, video startup time is not only high for some users but also very volatile. 

Rebuffering time presents a similar behavior, but with an even more pronounced discrepancy on the long-tail sessions. While the median rebuffering time was 6.1s, the 95th-percentile was 18.8s, and the 99th-percentile was 34.2s. This clearly indicates that highly frustrating buffering events are being experienced in some sessions. Recall the previous reference from Snapchat: 44% of users experience a 90th-percentile buffering duration.

The one to blame? HTTP

Why is this happening? In a nutshell, as I’ve already discussed in the previous post, the one to blame is HTTP and its incapacity to handle the natural instability of WiFi or 4G links. Leading-edge mobile apps like Uber are experiencing highly volatile network response times across the globe, with the scenario being even more challenging in regions like India, Southeast Asia, or South America. Quoting a post from Twitter Engineering Blog, “in many locations, latency is too high for users to consistently access their Twitter content”, whereas at the LinkedIn Engineering Blog we can find that optimizing the existing HTTP/TCP paradigm implies “forcing a “selective penalty” on some members, while improving the content delivery for others”. 

The mobile app industry has already identified the need to replace existing HTTP with new protocols prepared to handle the volatility of wireless links. This is why industry leaders like Google, Facebook, or Uber have been investing heavily in a new protocol, QUIC, which evolved to HTTP3. However, QUIC brings two major drawbacks: it requires an engineering and operational nightmare to use, and QUIC performance improvements are somewhat limited. 

Improving video delivery with Codavel

At Codavel, we devote ourselves to this fundamental challenge: building a mobile-first protocol, robust to wireless link instability, that can be used effortlessly by mobile apps. That is what you get with Codavel Performance Service and our groundbreaking protocol, Bolina, that clearly outperforms both HTTP and QUIC / HTTP3.

Using the same methodology as in the aforementioned performance results, we analyzed the behavior of Codavel Performance Service for the same DASH stream with 8 different video qualities. Same video stream, same mobile app, same user behavior, but now with Codavel Performance Service optimizing HTTP network calls, for a set of users in India. In a nutshell, with Codavel Performance Service you get:

Want to know more about our testing and results? Download the full performance report here and talk with us, we can help you achieve the same results!