Marc Sullivan

The Effect of Content Desirability on Subjective Video Quality Ratings

Objective: The purpose of this study was to examine the impact of the desirability of content on viewers’ ratings of subjective video quality. Background: Most subjective video quality studies use short-duration clips that are specially designed to exercise the encoding algorithms and do not consider the desirability of the content as a variable. Method: In four studies, we employed a total of 100 participants and 180 movie clips encoded at nine levels from 550 kbps up to DVD quality. Participants viewed clips that were 2 min in length and then were asked about video quality of the clips and desirability of the movie content. Results: The results of these studies show that there is a strong correlation between the desirability of movie content and subjective ratings of video quality. This strong relationship holds across a wide range of encoding levels and movie content when that content is viewed under longer, more naturalistic viewing conditions. Conclusion: The effects of content should be considered when evaluating the subjective quality of encoded video content, as these effects can be as large as those seen between low- and high-quality encodings. Application: Researchers and practitioners trying to determine acceptable levels of video quality for actual consumption by consumers may find that the results and methods described here allow for a more accurate assessment of levels of video quality that are acceptable in a fielded service.

Practical issues in subjective video quality evaluation: human factors vs. psychophysical image quality evaluation

In this paper, we describe subjective video quality testing using naturally occurring, realistic, video content. The use of realistic content poses a number of challenges that are not encountered when using standard video test clips. The methods that we have developed are targeted to define levels of video quality that will be acceptable to customers of a video service. The methods were developed in the context of home TV viewing, but are applicable to other emerging areas such as mobile video. Some of the key elements of testing with natural content are controlling for natural content variability, measuring content desirability, and using a testing method that encourages natural viewing behaviors.