Marie-Neige Garcia

Study of Rating Scales for Subjective Quality Assessment of High-Definition Video

With the constant evolution of video technology and the deployment of new video services, content providers and broadcasters always face the challenge of delivering an adequate video quality which meets end-users expectations. The development of reliable quality testing and quality monitoring tools that can be used by broadcasters ultimately requires reliable objective video quality metrics. In turn, the validation of these objective models requires reliable subjective assessment, the most accurate representation of the quality perceived by end-users. Many different subjective assessment methodologies exist, and each has its advantages and drawbacks. One important element in a subjective testing methodology is the choice of the rating scale. In this paper, we make a direct comparison between four scales, which are either included in existing international standards or proposed to be used in future standardization activities related to video quality. We examine the subjective data from the points of view of response behavior from participants, similarity and variability of subjective scores. We discuss these results within the context of the subjective quality assessment of high-definition video compressed and transmitted over error-prone networks. Our experimental data show no overall statistical differences between the different scales. Results also show that the single-stimulus presentation provides highly repeatable results even if different scales or groups of participants are used.

T-V-model: Parameter-based prediction of IPTV quality

The paper presents a parameter-based model for predicting the perceived quality of transmitted video for IPTV applications. The core model we derived can be applied both to service monitoring and network or service planning. In its current form, the model covers H.264 and MPEG-2 coded video (standard and high definition) transmitted over IP-links. The model includes factors like the coding bit-rate, the packet loss percentage and the type of packet loss handling used by the codec. The paper provides an overview of the model, of its integration into a multimedia model predicting audio-visual quality, and of its application to service monitoring. A performance analysis is presented showing a high correlation with the results of different subjective video quality perception tests. An outlook highlights future model extensions.

Towards content-related features for parametric video quality prediction of IPTV services

This paper investigates video content-related features, such as measures of spatio-temporal complexity, for inclusion into parametric video quality models. Our goal is to find a parametric content description that correlates with perceived video quality. In the course of the development of a parametric IPTV video quality prediction model (T-V-model), a large number of subjective tests have been conducted for standard definition and high definition video with different types of content. As expected from previous studies, we observed content dependencies that were different for different types of degradations. As descriptors of the content, we employ spatio-temporal related information obtained either before encoding and from the decoder or obtained from the decoder only. We compare those two approaches and explore their application to a reduced- or no-reference parametric model. An outlook highlights future steps for integrating the spatio- temporal features into the parametric model.

Parametric packet-layer video quality model for IPTV

This paper presents a parametric packet-layer model for predicting perceived quality of High Definition (HD, 1920×1080) and Standard Definition (SD, 720×576) videos for IPTV services. The model can be applied both for network planning and service monitoring. It takes as input the video resolution, packet-layer information such as bit-rate, packet-loss-rate and burstiness factor, and information on the encoder/decoder settings such as the number of slices per frame and the packet-loss-concealment. Addressed degradations are compression artifacts related to H.264 and MPEG2 encoding, and transmission errors leading to slicing errors or freezing, depending on the applied packet-loss-concealment. Extensive subjective video quality tests have been conducted for measuring the perceived quality of the original and degraded video sequences. The model is developed based on the obtained test results. The model prediction shows a correlation of 0.98 for HD and 0.96 for SD with the mean subjective ratings.

Impairment-Factor-Based Audiovisual Quality Model for IPTV: Influence of Video Resolution, Degradation Type, and Content Type

This paper presents an audiovisual quality model for IPTV services. The model estimates the audiovisual quality of standard and high definition video as perceived by the user. The model is developed for applications such as network planning and packet-layer quality monitoring. It mainly covers audio and video compression artifacts and impairments due to packet loss. The quality tests conducted for model development demonstrate a mutual influence of the perceived audio and video quality, and the predominance of the video quality for the overall audiovisual quality. The balance between audio quality and video quality, however, depends on the content, the video format, and the audio degradation type. The proposed model is based on impairment factors which quantify the quality-impact of the different degradations. The impairment factors are computed from parameters extracted from the bitstream or packet headers. For high definition video, the model predictions show a correlation with unknown subjective ratings of 95%. For comparison, we have developed a more classical audiovisual quality model which is based on the audio and video qualities and their interaction. Both quality- and impairment-factor-based models are further refined by taking the content-type into account. At last, the different model variants are compared with modeling approaches described in the literature.

IP-Based Mobile and Fixed Network Audiovisual Media Services

This article provides a tutorial overview of current approaches for monitoring the quality perceived by users of IP-based audiovisual media services. The article addresses both mobile and fixed network services such as mobile TV or Internet Protocol TV (IPTV). It reviews the different quality models that exploit packet- header-, bit stream-, or signal-information for providing audio, video, and audiovisual quality estimates, respectively. It describes how these models can be applied for real-life monitoring, and how they can be adapted to reflect the information available at the given measurement point. An outlook gives insight into emerging trends for near- and mid-term future requirements and solutions.

No-reference video quality assessment for SD and HD H.264/AVC sequences based on continuous estimates of packet loss visibility

In this paper, a novel method for predicting the visibility of packet losses in SD and HD H.264/AVC video sequences and modeling their impact on perceived quality is proposed. Based on the findings of a new subjective experiment it is initially shown that the classification of packet loss visibility in a binary fashion is not sufficient to model the perceptual degradations caused by the transmission errors. The proposed no-reference algorithm extracts a set of features from the video bit-stream to account for the spatial and temporal characteristics of the video content and the induced distortion due to the network impairments. Subsequently, the visibility of packet losses is predicted in a continuous fashion using Support Vector Regression. Finally, a no-reference bit-stream based video quality assessment model that explicitly employs the predicted packet loss visibility estimates is presented. The evaluation of the proposed model demonstrates that the use of continuous estimates for the visibility of packet losses improves the performance of the video quality assessment model.

No-reference bit stream model for video quality assessment of h.264/AVC video based on packet loss visibility

In this paper, a no reference bit stream model for quality assessment of SD and HD H.264/AVC video sequences based on packet loss visibility is proposed. The method considers the impact of network impairments on human perception and uses the visibility of packet losses to predict objective scores. Also, a new subjective experiment has been designed to provide insight into the perceptual effect of degradations caused by transmission errors. The proposed algorithm extracts a set of features from the received bit stream. Then, the visibility of each packet loss event is determined by classifying the extracted features using a Support Vector Machines classifier. Finally, analytical expressions are developed to account for visual degradation due to compression and channel induced distortion based on the outcome of the visibility classifier. The evaluation demonstrates the validity of the proposed method.

Impairment-factor-based audio-visual quality model for IPTV

This paper presents an audio-visual quality model for IPTV services. The model estimates the quality of High Definition (HD, 1920x1080 pixels) audio-video as perceived by the user. It is developed for being applied to network planning and packet layer quality monitoring. Addressed degradations are mainly audio and video compression artifacts and packet loss. Results demonstrate amutual influence of the perceived audio and video qualities, and the predominance of the video quality for the overall audio-visual quality. The model is based on impairment factors which express the quality impact of the different degradations. Those factors are computed from parameters extracted from the bit-stream or packet headers, such as audio and video bit-rates and packet-loss-rates, making the model computationally efficient. The model correlates at 97% with unknown subjective ratings. For comparison, we have developed a more classical audio-visual quality model based on audio and video qualities and their interaction, which shows lower performance than the impairment-based approach. Also provided is a comparison with models described in the the literature.

Quality impact due to initial loading, stalling, and video bitrate in progressive download video services

The paper targets the audiovisual quality assessment of progressive download type video services such as Video on Demand (VoD) and YouTube. To this aim, we investigate the quality impact due to initial loading, stalling, and video bitrate for High Definition (HD, 1920×1080 pixels) audiovisual sequences. The analysis is based on the results of three audiovisual subjective tests conducted in a laboratory environment. The paper extends the work described in previous studies by looking at high video resolution (HD) for short (30 s) and longer (45-60 s) test sequences, and by addressing the combined effect of initial loading, stalling and video bitrates on perceived quality. The distribution of stalling events within a video sequence and the recency effect are also analysed.