UMass Amherst researcher quantifies the effectiveness of video ads

Online video is a “killer application” of the Internet, predicted to soon make up 86 percent of consumer traffic on the web, says computer science researcher Ramesh Sitaraman at the University of Massachusetts Amherst. But advertisers who want to capitalize on it face a huge question: How effective are video ads?

“Video usage is growing so rapidly,” says Sitaraman, “but making online videos economically sustainable and profitable is a burning question of key importance to the future economics and evolution of the Internet.” He and research partner S. Shunmuga Krishnan of Akamai recently completed perhaps the largest-ever scientific study of the effectiveness of video ads, a fundamental question for business. Sitaraman will present their findings at the ACM Internet Measurement Conference in Barcelona on October 24.

Sitaraman, who led the study, says a simple, well-accepted measure of ad effectiveness is whether viewers watch the video ads to completion or not. “Our goal was to scientifically understand what factors influence people to complete watching video ads and what contributes to them abandoning the ad before it completes.”

“Viewers by and large watch video ads to completion, though we found that the position in the video where the ad was inserted had the largest impact,” he adds. Ads inserted in the middle of a video where viewers are presumably most engaged with the content had the highest ad completion rate of 97 percent, while ads inserted at the beginning or end were completed at lesser rates, 74 and 45 percent, respectively.

The research analyzed anonymized viewer data linked to more than 257 million ads inserted into 367 million videos from over 3,000 publishers and viewed by 65 million unique viewers around the world, so statistical power was high. Anonymized viewer data, representing real people but without personal information attached, were collected over a 15-day period in April 2013 using Akamai's media analytics platform.

All viewers did not behave similarly. Viewers who repeatedly came to a particular site had noticeably higher completion rates in all ad categories than casual one-time visitors. “Repeat visitors are likely more interested in the videos offered by that site than one-time visitors, and hence more willing to watch the ads,” says Sitaraman.

The video into which an ad is inserted matters, too. Viewers were much more likely to completely watch an ad inserted into long-form content such as movie or a TV episode than short-form content such as a news clip. Sitaraman notes, “A plausible reason is that viewers assign more uniqueness and value to movies and TV episodes that makes it seem more worthwhile for them to sit through an ad.”

The study also compared two scenarios that could test the patience of a video viewer: One set of viewers experienced a slow-to-load video, while another set watched a pre-roll ad that must complete before the video begins. While both sets of viewers were waiting to watch their chosen video, the viewers watching the slow-to-load ad abandoned more than three times faster as the other group.

Further, at the 10-second mark, more than 45 percent of viewers waiting for the slow-loading video had abandoned, compared to only 13 percent of the viewers abandoning the pre-roll ad. A reason for this may be that viewers opt to watch an ad as an implicit form of payment for the content. Waiting for this ad to complete is a predictable experience with a known duration. But a slow-loading video occurs more unexpectedly and the wait is often unknown, causing more frustration and viewer abandonment, says Sitaraman.

There were some surprises in the research outcomes, he points out. Internet folklore has been that people are much more likely to complete watching video ads in the more relaxed setting of evenings or weekends. However, the research did not find significant time-of-day or weekday-weekend variations in ad completion rates.

“The primary scientific challenge in understanding video ads is the large number of conflicting factors that could influence viewer behavior. For instance, if short 15-second ads inserted in the middle of videos complete more often, it is unclear if the higher completion rates are because of the ad's mid-roll position or its shorter length. To quantitatively assess the impact of the various factors, we devised novel data analysis techniques that extract key nuggets of information that are hidden within large amounts of video viewing data,” says Sitaraman.