Bernhard Feiten

Audio adaptation according to usage environment and perceptual quality metrics

Future audio applications will need to handle a wide variety of end-devices and networks from low quality mobile services to very high quality home entertainment services. The MPEG-21 Digital Item Adaptation standard specifies a variety of tools that aim to enhance the user experience and maximize the perceived quality of service. This paper focuses on the audio-related tools, including audio usage environment descriptions and techniques for audio resource adaptation. Additionally, subjective and objective measures are discussed in the context of bit-rate adaptation. New quality measures, brightness, cleanness, and wideness are proposed for controlling the quality of the audio transmission.

IP-Based Mobile and Fixed Network Audiovisual Media Services

This article provides a tutorial overview of current approaches for monitoring the quality perceived by users of IP-based audiovisual media services. The article addresses both mobile and fixed network services such as mobile TV or Internet Protocol TV (IPTV). It reviews the different quality models that exploit packet- header-, bit stream-, or signal-information for providing audio, video, and audiovisual quality estimates, respectively. It describes how these models can be applied for real-life monitoring, and how they can be adapted to reflect the information available at the given measurement point. An outlook gives insight into emerging trends for near- and mid-term future requirements and solutions.

Parametric model for audiovisual quality assessment in IPTV: ITU-T Rec. P.1201.2

A parametric packet-based model has been created to estimate user perceived audiovisual quality of Internet Protocol Television (IPTV) services. It is divided into three modules, for audio, video and audiovisual quality. The model is applicable to the quality monitoring of encrypted and non-encrypted audiovisual streams. Typical audio and video degradations for IPTV are covered for Standard Definition (SD) and High Definition (HD) video formats. The model supports the H.264 video codec and the audio codecs MPEG-I Layer II, MPEG-2 AAC-LC, MPEG-4 HE-AACv2 and AC3. It handles various types of IP-network layer transmission errors. The model was developed and validated using a large database of subjective tests. The underlying concept is based on an impairment factor approach, which enables detection of how users build their individual judgment of quality of a given audiovisual signal. Each impairment factor captures the perceived quality impact of a possible degradation and therefore enables diagnostic analysis of quality problems. The model shows high performance results, both in terms of Pearsons Correlation coefficient (r) and Root-Mean-Square-Error (RMSE). The model is standardized as ITU-T Recommendation P.1201.2, the higher resolution (IPTV and Video on Demand (VoD)) algorithm of Recommendation P.1201.

Parametric quality prediction for IP‐based audio

Different multimedia services are more and more transmitted over a common network infrastructure, e.g. using the Internet Protocol (IP). Examples are the widespread voice over Internet Protocol (VoIP), and Internet Protocol Television (IPTV). The streaming of pure audio over IP even has a longer tradition, with applications such as internet radio. For an efficient development, planning and monitoring of such services, models can be used that predict user‐perceived quality based on technical service characteristics. Speech quality models for telephony are among the most advanced ones in this context, with different model types like the signal‐based PESQ (ITU‐T Rec. P.862, 2001) or the parametric E‐model (ITU‐T Rec. G.107, 2005). In this paper, we describe a parametric approach for predicting the quality of IP‐based audio. The main parameters are the audio codec, codec bitrate, packet loss characteristics and the audio content. We base our considerations on own listening tests conducted in the framework of IPTV quality assessment, on approaches and test data described in the literature and on complementary knowledge from the fields of speech and video quality models. In this context, we identify similarities and discrepancies between different types of services in the light of a common model framework.

A framework for QoE analysis of encrypted video streams

Today most internet traffic is generated by video streaming. YouTube and other video streaming platforms are using encrypted streams (HTTPS) for transport of video content. Encryption will lead to more requirements on network and content providers, e.g. caching mechanisms will not work direct. Estimation of video quality for measuring users satisfaction is also harder because there is no direct access to the video bitstream. We are building up a framework for analyzing video quality that allows us to store client information, decrypted network traffic and encrypted messages. Our approach is based on a man-in-the-middle proxy for storing the decrypted video bitstream, active probing and traffic shaping. Using these data, we are able to calculate video QoE values for example using a model such as ITU-T Rec. P.1203. Our framework will be used for generating datasets for encrypted video stream analysis, analyzing internal behavior of video streaming platforms, and more. For experimental evaluation, in this paper we analyze the influence of our man-in-the-middle proxy on key-performance indicators (KPIs) for video streaming quality.

A bitstream-based, scalable video-quality model for HTTP adaptive streaming: ITU-T P.1203.1

The paper presents the scalable video quality model part of the P.1203 Recommendation series, developed in a competition within ITU-T Study Group 12 previously referred to as P.NATS. It provides integral quality predictions for 1 up to 5 min long media sessions for HTTP Adaptive Streaming (HAS) with up to HD video resolution. The model is available in four modes of operation for different levels of media-related bitstream information, reflecting different types of encryption of the media stream. The video quality model presented in this paper delivers short-term video quality estimates that serve as input to the integration component of the P.1203 model. The scalable approach consists in the usage of the same components for spatial and temporal scaling degradations across all modes. The third component of the model addresses video coding artifacts. To this aim, a single model parameter is introduced that can be derived from different types of bitstream input information. Depending on the complexity of the available input, one of four scaling-levels of the model is applied. The paper presents the different novelties of the model and scientific choices made during its development, the test design, and an analysis of the model performance across the different modes.

Parametric audio quality model for IPTV services - ITU-T P.1201.2 audio

This paper describes a parametric audio quality model for IPTV services. The model is standardized under the ITU-T P.1201.2 Recommendation. Subjective test databases used to develop and validate the model are also presented. The model is applicable to network planning and packet-layer quality monitoring of encrypted audio. Addressed degradations are audio compression (MPEG-1 Layer II & III, MPEG-2 AAC-LC, MPEG-4 HE-AACv2 and AC-3) and audio frame loss for various frame-loss-patterns. Two implementations (Dolby, Nero) have been tested for the AAC-LC and HE-AAC codecs. The proposed model takes as input parameters the audio codec, bitrate, frame loss percentage and a burst-length-related parameter. It shows high performance results on unknown data with a Pearsons correlation coefficient of 0.92 with the subjective quality scores and an RMSE of 0.39 on the 5-point scale.

Scene change detection in encrypted video bit streams

In this paper, a novel method to detect scene changes in encrypted video streams is presented. Typically, in IPTV systems, the media stream is transmitted in encrypted form, and therefore the only available information to determine the scene changes are the packet headers which transport the video signal. Thus, the proposed method estimates the size and the type of each picture of the video sequence by extracting information from the packet headers. Then, based on the GOP structure, a set of rules are determined to predict changes of frame sizes which are indicative of scene changes in the video sequences. Furthermore, the application of the proposed method in the recently standardized ITU-T Recommendation P.1201.2 for no-reference audio-visual quality assessment for IPTV-grade services is presented to highlight how such method could be deployed. Finally, the proposed method is evaluated on a large set of video databases to demonstrate the validity of the proposed method.

HTTP adaptive streaming QoE estimation with ITU-T rec. P. 1203: open databases and software

This paper describes an open dataset and software for ITU-T Ree. P.1203. As the first standardized Quality of Experience model for audiovisual HTTP Adaptive Streaming (HAS), it has been extensively trained and validated on over a thousand audiovisual sequences containing HAS-typical effects (such as stalling, coding artifacts, quality switches). Our dataset comprises four of the 30 official subjective databases at a bitstream feature level. The paper also includes subjective results and the model performance. Our software for the standard was made available to the public, too, and it is used for all the analyses presented. Among other previously unpublished details, we show the significant performance improvements of using bitstream-based models over metadata-based ones for video quality analysis, and the robustness of combining classical models with machine-learning-based approaches for estimating user QoE.