Kjell Brunnström

Techniques for measuring quality of experience

Quality of Experience (QoE) relates to how users perceive the quality of an application. To capture such a subjective measure, either by subjective tests or via objective tools, is an art on its own. Given the importance of measuring users’ satisfaction to service providers, research on QoE took flight in recent years. In this paper we present an overview of various techniques for measuring QoE, thereby mostly focusing on freely available tools and methodologies.

Factors Influencing Quality of Experience

In this chapter different factors that may influence Quality of Experience (QoE) in the context of media consumption, networked services, and other electronic communication services and applications, are discussed. QoE can be subject to a range of complex and strongly interrelated factors, falling into three categories: human, system and context influence factors (IFs). With respect to Human IFs, we discuss variant and stable factors that may potentially bear an influence on QoE, either for low-level (bottom-up) or higher-level (top-down) cognitive processing. System IFs are classified into four distinct categories, namely content-, media-, network- and device-related IFs. Finally, the broad category of possible Context IFs is decomposed into factors linked to the physical, temporal, social, economic, task and technical information context. The overview given here illustrates the complexity of QoE and the broad range of aspects that potentially have a major influence on it.

VQeg validation and ITU standardization of objective perceptual video quality metrics [Standards in a Nutshell]

For industry, the need to access accurate and reliable objective video metrics has become more pressing with the advent of new video applications and services such as mobile broadcasting, Internet video, and Internet Protocol television (IPTV). Industry-class objective quality- measurement models have a wide range of uses, including equipment testing (e.g., codec evaluation), transmission- planning and network-dimensioning tasks, head-end quality assurance, in- service network monitoring, and client-based quality measurement. The Video Quality Experts Group (VQEG) is the primary forum for validation testing of objective perceptual quality models. The work of VQEG has resulted in International Telecommunication Union (ITU) standardization of objective quality models designed for standard- definition television and for multimedia applications. This article reviews VQEGs work, paying particular attention to the groups approach to validation testing.

Active fixation for scene exploration

It is well-known that active selection of fixation points in humans is highly context and task dependent. It is therefore likely that successful computational processes for fixation in active vision should be so too. We are considering active fixation in the context of recognition of man-made objects characterized by their shapes. In this situation the qualitative shape and type of observed junctions play an important role. The fixations are driven by a grouping strategy, which forms sets of connected junctions separated from the surrounding at depth discontinuities. We have furthermore developed a methodology for rapid active detection and classification of junctions by selection of fixation points. The approach is based on direct computations from image data and allows integration of stereo and accommodation cues with luminance information. This work form a part of an effort to perform active recognition of generic objects, in the spirit of Malik and Biederman, but on real imagery rather than on line-drawings.

Quality of Experience and HTTP adaptive streaming : A review of subjective studies

HTTP adaptive streaming technology has become widely spread in multimedia services because of its ability to provide adaptation to characteristics of various viewing devices and dynamic network conditions. There are various studies targeting the optimization of adaptation strategy. However, in order to provide an optimal viewing experience to the end-user, it is crucial to get knowledge about the Quality of Experience (QoE) of different adaptation schemes. This paper overviews the state of the art concerning subjective evaluation of adaptive streaming QoE and highlights the challenges and open research questions related to QoE assessment.

Active Detection and Classsification of Junctions by Foveation with a Head-Eye System Guided by the Scale-Space Primal Sketch

We consider how junction detection and classification can be performed in an active visual system. This is to exemplify that feature de-tection and classification in general can be done by both simple and robust methods, if the vision system is allowed to look at the world rather than at prerecorded images. We address issues on how to attract the attention to salient local image structures, as well as on how to characterize those.

Subjective Multimedia Quality Assessment

The Video Quality Experts Group (VQEG) is preparing a programme of subjective multimedia quality tests. The results from these tests will be used to evaluate the performance of competing objective multimedia quality metrics. The reliability of the subjective test data is of great importance for VQEGs task. This paper provides an overview of VQEGs multimedia ad-hoc group. The work of this group will require subjective tests to be performed by laboratories located in Europe, Asia and North America. For VQEGs multimedia work to be successful, the subjective assessment methodology must be precisely defined and produce reliable and repeatable subjective quality data. Although international standards covering multimedia quality assessment methods are in force, there remains some uncertainty regarding the most effective approach to assessing the subjective quality of multimedia. A review of existing methods is provided. Two experiments are presented investigating the suitability of alternative subjective assessment methods (single-stimulus ACR and SAMVIQ). The results of these experiments are discussed within the context of the VQEG multimedia testing programme.

Subjective quality assessment of error concealment strategies for 3DTV in the presence of asymmetric transmission errors

The transmission of 3DTV sequences over packet based networks may result in degradations of the video quality due to packet loss. In the conventional 2D case, several different strategies are known for extrapolating the missing information and thus concealing the error. In 3D however, the residual error after concealment of one view might leads to binocular rivalry with the correctly received second view. In this paper, three simple alternatives are presented: frame freezing, a reduced playback speed, and displaying only a single view for both eyes, thus effectively switching to 2D presentation. In a subjective experiment the performance in terms of quality of experience of the three methods is evaluated for different packet loss scenarios. Error-free encoded videos at different bit rates have been included as anchor conditions. The subjective experiment method contains special precautions for measuring the Quality of Experience (QoE) for 3D content and also contains an indicator for visual discomfort. The results indicate that switching to 2D is currently the best choice but difficulties with visual discomfort should be expected even for this method.

Subjective quality assessment of an adaptive video streaming model

With the recent increased popularity and high usage of HTTP Adaptive Streaming (HAS) techniques, various studies have been carried out in this area which generally focused on the technical enhancement of HAS technology and applications. However, a lack of common HAS standard led to multiple proprietary approaches which have been developed by major Internet companies. In the emerging MPEG-DASH standard the packagings of the video content and HTTP syntax have been standardized; but all the details of the adaptation behavior are left to the client implementation. Nevertheless, to design an adaptation algorithm which optimizes the viewing experience of the enduser, the multimedia service providers need to know about the Quality of Experience (QoE) of different adaptation schemes. Taking this into account, the objective of this experiment was to study the QoE of a HAS-based video broadcast model. The experiment has been carried out through a subjective study of the end user response to various possible clients’ behavior for changing the video quality taking different QoE-influence factors into account. The experimental conclusions have made a good insight into the QoE of different adaptation schemes which can be exploited by HAS clients for designing the adaptation algorithms.

Standardized toolchain and model development for video quality assessment — The mission of the Joint Effort Group in VQEG

Since 1997, the Video Quality Experts Group (VQEG) has been active in the field of subjective and objective video quality assessment. The group has validated competitive quality metrics throughout several projects. Each of these projects requires mandatory actions such as creating a testplan and obtaining databases consisting of degraded video sequences with corresponding subjective quality ratings. Recently, VQEG started a new open initiative, the Joint Effort Group (JEG), for encouraging joint collaboration on all mandatory actions needed to validate video quality metrics. Within the JEG, effort is made to advance the field of both subjective and objective video quality measurement by providing proper software tools and subjective databases to the community. One of the subprojects of the JEG is the joint development of a hybrid H.264/AVC objective quality metric. In this paper, we introduce the JEG and provide an overview of the different ongoing activities within this newly started group.