Alexander Raake

Study of Rating Scales for Subjective Quality Assessment of High-Definition Video

With the constant evolution of video technology and the deployment of new video services, content providers and broadcasters always face the challenge of delivering an adequate video quality which meets end-users expectations. The development of reliable quality testing and quality monitoring tools that can be used by broadcasters ultimately requires reliable objective video quality metrics. In turn, the validation of these objective models requires reliable subjective assessment, the most accurate representation of the quality perceived by end-users. Many different subjective assessment methodologies exist, and each has its advantages and drawbacks. One important element in a subjective testing methodology is the choice of the rating scale. In this paper, we make a direct comparison between four scales, which are either included in existing international standards or proposed to be used in future standardization activities related to video quality. We examine the subjective data from the points of view of response behavior from participants, similarity and variability of subjective scores. We discuss these results within the context of the subjective quality assessment of high-definition video compressed and transmitted over error-prone networks. Our experimental data show no overall statistical differences between the different scales. Results also show that the single-stimulus presentation provides highly repeatable results even if different scales or groups of participants are used.

Speech Quality Estimation: Models and Trends

This article presents a tutorial overview of models for estimating the quality experienced by users of speech transmission and communication services. Such models can be classified as either parametric or signal based. Signal-based models use input speech signals measured at the electrical or acoustic interfaces of the transmission channel. Parametric models, on the other hand, depend on signal and system parameters estimated during network planning or at run time. This tutorial describes the underlying principles as well as advantages and limitations of existing models. It also presents new developments, thus serving as a guide to an appropriate usage of the multitude of current and emerging speech quality models.

T-V-model: Parameter-based prediction of IPTV quality

The paper presents a parameter-based model for predicting the perceived quality of transmitted video for IPTV applications. The core model we derived can be applied both to service monitoring and network or service planning. In its current form, the model covers H.264 and MPEG-2 coded video (standard and high definition) transmitted over IP-links. The model includes factors like the coding bit-rate, the packet loss percentage and the type of packet loss handling used by the codec. The paper provides an overview of the model, of its integration into a multimedia model predicting audio-visual quality, and of its application to service monitoring. A performance analysis is presented showing a high correlation with the results of different subjective video quality perception tests. An outlook highlights future model extensions.

Short- and Long-Term Packet Loss Behavior: Towards Speech Quality Prediction for Arbitrary Loss Distributions

A speech-quality-oriented classification of packet loss distributions is proposed according to both the short- and long-term loss behavior. While the short-term behavior (microscopic loss behavior) relates to the effect of packet loss on the coder and packet loss concealment performance, the long-term loss behavior (macroscopic loss behavior) is defined so that it reflects the loss behavior that ultimately leads to speech quality that perceptively changes over time. Based on this classification, different parametric (objective) modeling approaches for predicting speech quality are discussed. To this aim, a packet loss averaging approach is presented for modeling speech quality under short-term loss. Starting from this model, two different ways for predicting speech quality under long-term-dependent packet loss are analyzed and compared to auditory (subjective) test results: quality prediction based on the averaging at packet trace level as provided, for example, by the E-model (2005), and the prediction based on the time-averaging of estimated instantaneous quality profiles, as suggested, for example, by L. Gros and N. Chateau (2001) (1998). From this comparison, the suitability of the different approaches for network planning are discussed, and their limitations in case of particular loss distributions are pointed out

Spatial Sound With Loudspeakers and Its Perception: A Review of the Current State

This paper reviews the current state of loudspeaker-based spatial sound reproduction methods from technical perspective as well as perceptual perspective. A nomenclature is developed that allows for a strict separation between these two perspectives. The physical fundamentals, practical realization, and results from perceptual studies are discussed for a number of well-established and emerging reproduction techniques. Further, the paper outlines novel approaches to spatial sound evaluation in terms of perceived quality and provides a comparison of current approaches.

Evaluating Depth Perception of 3D Stereoscopic Videos

3D video quality of experience (QoE) is a multidimensional problem; many factors contribute to the global rating like image quality, depth perception and visual discomfort. Due to this multidimensionality, it is proposed in this paper, that as a complement to assessing the quality degradation due to coding or transmission, the appropriateness of the non-distorted signal should be addressed. One important factor here is the depth information provided by the source sequences. From an application-perspective, the depth-characteristics of source content are of relevance for pre-validating whether the content is suitable for 3D video services. In addition, assessing the interplay between binocular and monocular depth features and depth perception are relevant topics for 3D video perception research. To achieve the evaluation of the suitability of 3D content, this paper describes both a subjective experiment and a new objective indicator to evaluate depth as one of the added values of 3D video.

Impairment Factor Framework for Wide-Band Speech Codecs

A new method is described for quantifying the quality degradation introduced by wide-band speech codecs via a one-dimensional impairment factor. The method is based on auditory listening-only tests, but the resulting impairment factors may be used for predicting speech quality in an instrumental way, e.g., for network planning purposes. Following the method, auditory test results are first transformed to an overall quality rating scale, and then adjusted to rule out test-specific effects. The derived impairment factors fit into the common framework which is defined by the E-model for narrow-band telephone networks, and which is hereby extended towards wide-band speech transmission. This paper presents the necessary auditory test data, describes the derivation and adjustment methodology, and provides numerical values for a range of wide-band speech codecs. The values are tested for their robustness in case of codec tandems and adjusted to represent the effects of packet loss

Quality of Experience and HTTP adaptive streaming : A review of subjective studies

HTTP adaptive streaming technology has become widely spread in multimedia services because of its ability to provide adaptation to characteristics of various viewing devices and dynamic network conditions. There are various studies targeting the optimization of adaptation strategy. However, in order to provide an optimal viewing experience to the end-user, it is crucial to get knowledge about the Quality of Experience (QoE) of different adaptation schemes. This paper overviews the state of the art concerning subjective evaluation of adaptive streaming QoE and highlights the challenges and open research questions related to QoE assessment.

Quality and Quality of Experience

The chapter discusses the processes of human perception and experiencing, and of quality formation. In this context, definitions of relevant terms are re-visited and adapted to the presented, updated view, and different aspects of research into quality at large and into Quality of Experience are summarized. Using a conceptual model, the quality formation process is analyzed in view of different contexts and tasks, such as taking part in a quality test under controlled conditions, experiencing a video presentation or concert, or exploring a system or device when considering a purchase in a shop. We provide a short overview of different quality assessment methods, and outline related trends in QoE research.

Towards content-related features for parametric video quality prediction of IPTV services

This paper investigates video content-related features, such as measures of spatio-temporal complexity, for inclusion into parametric video quality models. Our goal is to find a parametric content description that correlates with perceived video quality. In the course of the development of a parametric IPTV video quality prediction model (T-V-model), a large number of subjective tests have been conducted for standard definition and high definition video with different types of content. As expected from previous studies, we observed content dependencies that were different for different types of degradations. As descriptors of the content, we employ spatio-temporal related information obtained either before encoding and from the decoder or obtained from the decoder only. We compare those two approaches and explore their application to a reduced- or no-reference parametric model. An outlook highlights future steps for integrating the spatio- temporal features into the parametric model.