Satu Jumisko-Pyykkö

Does context matter in quality evaluation of mobile television

Subjective quality evaluation is used to optimize the produced audiovisual quality from fundamental signal processing algorithms to consumer services. These studies typically follow the basic principles of controlled psychoperceptual experiments. However, when compromising compression and transmission parameters for consumer services, the ecological validity of conventional quality evaluation methods can be questioned. To tackle this, we firstly present a novel user-oriented quality evaluation method for mobile television in its usage contexts. Secondly, we present the results of an experiment conducted with 30 participants comparing acceptability and satisfaction of quality as well as goals of viewing in three mobile contexts and under four different residual transmission error rates, when the participants also performed simultaneous assessment tasks. Finally, we compare the results with a previous laboratory experiment. The studied error rates impacted negatively on all measured tasks with some contextual differences. Moreover, the evaluations were more favorable and less discriminate in the mobile contexts compared to the laboratory.

Evaluation of subjective video quality of mobile devices

Subjectively perceived video quality is a critical factor when adopting new mobile video applications. When video is used in mobile networks the most important requirements are related to low bitrates, framerates and the screen size of mobile device. In two tests we investigated the effects of codecs and combinations of audio and video streams with low bitrates and different contents on the perceived video quality of mobile devices. The first test showed that the codec H.264 produced the most satisfying video quality, but the quality was not high enough for the presentation of textual information. In the second test, the audio-video ratio 32/128kbps was found to be the most pleasant, but there were content dependent variations.

Acceptance Threshold: A Bidimensional Research Method for User-Oriented Quality Evaluation Studies

Subjective quality evaluation is widely used to optimize system performance as a part of end-products. It is often desirable to know whether a certain system performance is acceptable, that is, whether the system reaches the minimum level to satisfy user expectations and needs. The goal of this paper is to examine research methods for assessing overall acceptance of quality in subjective quality evaluation methods. We conducted three experiments to develop our methodology and test its validity under heterogeneous stimuli in the context of mobile television. The first experiment examined the possibilities of using a simplified continuous assessment method for assessing overall acceptability. The second experiment explored the boundary between acceptable and unacceptable quality when the stimuli had clearly detectable differences. The third experiment compared the perceived quality impacts of small differences between the stimuli close to the threshold of acceptability. On the basis of our results, we recommend using a bidimensional retrospective measure combining acceptance and satisfaction in consumer-/user-oriented quality evaluation experiments.

Experienced quality factors: qualitative evaluation approach to audiovisual quality

Subjective evaluation is used to identify impairment factors of multimedia quality. The final quality is often formulated via quantitative experiments, but this approach has its constraints, as subjects quality interpretations, experiences and quality evaluation criteria are disregarded. To identify these quality evaluation factors, this study examined qualitatively the criteria participants used to evaluate audiovisual video quality. A semi-structured interview was conducted with 60 participants after a subjective audiovisual quality evaluation experiment. The assessment compared several, relatively low audio-video bitrate ratios with five different television contents on mobile device. In the analysis, methodological triangulation (grounded theory, Bayesian networks and correspondence analysis) was applied to approach the qualitative quality. The results showed that the most important evaluation criteria were the factors of visual quality, contents, factors of audio quality, usefulness - followability and audiovisual interaction. Several relations between the quality factors and the similarities between the contents were identified. As a research methodological recommendation, the focus on content and usage related factors need to be further examined to improve the quality evaluation experiments.

Comparison of Error Protection Methods for Audio-Video Broadcast over DVB-H

The paper discusses methods for robust audio-video broadcast over the digital video broadcasting-handheld (DVB-H) system. DVB-H includes a link-layer forward error correction (FEC) scheme known as multiprotocol encapsulation (MPE) FEC, which provides equal error protection (EEP) to the transmitted media streams. Several approaches for unequal error protection (UEP) have been proposed in the literature, and the applicability of some of them to DVB-H is analyzed in the paper. A link-layer UEP method based on priority segmentation of the media streams is chosen for more detailed analysis. According to the method, audio and the most important coded video pictures are protected by MPE-FEC more robustly compared to the remaining coded pictures. In order to compare EEP and UEP in a DVB-H environment, an error-prone DVB-H channel was simulated, audio-visual clips were sent through it, and a comprehensive subjective quality evaluation was conducted in a controlled laboratory environment. The results of the subjective evaluation revealed that the use of UEP improves the subjective quality of some test clips noticeably when the channel conditions were severe, while in other tested channel conditions and clips, UEP and EEP performed equally well.

A Hybrid Method for Quality Evaluation in the Context of Use for Mobile (3D) Television

Controlled psychoperceptual quality evaluation experiments are used to assess the excellence of produced audiovisual quality from fundamental signal processing algorithms to consumer services. When compromising produced quality for consumer services, used in dynamic and heterogeneous mobile contexts, the ecological validity of conventional quality evaluation methods can be questioned. The goal of this paper is to develop a method for evaluating the experienced multimedia quality in the context of use. We conducted three studies where the quality of mobile 2D and 3D television was assessed in three different field contexts, one simulated context and one controlled laboratory situation when audio-video compression and transmission parameters were varied. We propose a hybrid method for the design, data-collection and analysis of the experiments in the contexts of use. Its novelty is to complement conventional quantitative quality evaluation with concrete tools to identify factors that surround the assessment in the context. The methodological framework is part of our long-term aim to measure and understand the user-centered quality of experience.

Descriptive quality of experience for mobile 3D video

Perceptual quality evaluation experiments are used to assess the excellence of multimedia quality. However, these studies disregard qualitative experiential descriptions, interpretations, and impressions of quality. The goal of this paper is to identify general descriptive characteristics of experienced quality of 3D video on mobile devices. We conducted five studies in which descriptive data was collected after the psychoperceptual quality evaluation experiment. Qualitative semi-structured interviews and written attribute description tasks were conducted with over 90 naïve participants. The experiments contained an extensive and heterogeneous set of produced quality by varying content, level of depth, compression and transmission parameters, and audio and display factors for 3D. The results showed that quality of experience is constructed from four main components, 1) visual quality, 2) viewing experience, 3) content, and 4) quality of other modalities and their interactions.

I would like to see the subtitles and the face or at least hear the voice: Effects of picture ratio and audio---video bitrate ratio on perception of quality in mobile television

In new mobile video applications, the subjectively perceived visual and audiovisual qualities are critical factors in the wide audience adoption of mass products. The critical factors in mobile video coding are related to low bitrates, framerates and screen size of the devices. The first part of this study examined the effects of codecs, bitrates and picture ratio on the perceived visual video quality. In the second study, effects of two audio–video bitrate ratios are presented as factors in the perceived audiovisual video quality and the results are compared to the previous study that showed the content dependency in the audio–video bitrate ratio comparisons. In the results of the visual quality study, H.264 was rated as the most satisfying, but the quality was still not enough for text legibility and perceiving important details of image due to the QCIF picture ratio. The codec XviD was in generally experienced as worse than the H.264, but with the screen size SIF-SP with XviD codec was rated better with several contents. In the audiovisual study, the results also showed that in the low total bitrates the significance of audio is increased. However, the higher and lower bitrates had similar content dependent effects.

Mobile 3D television: development of core technological elements and user-centered evaluation methods toward an optimized system

A European consortium of six partners has been developing core technological components of a mobile 3D television system over DVB-H channel. In this overview paper, we present our current results on developing optimal methods for stereo-video content creation, coding and transmission and emphasize their significance for the power-constrained mobile platform, equipped with auto-stereoscopic display. We address the user requirements by applying modern usercentered approaches taking into account different user groups and usage contexts in contrast to the laboratory assessment methods which, though standardized, offer limited applicability to real applications. To this end, we have been aiming at developing a methodological framework for the whole system development process. One of our goals has been to further develop the user-centered approach towards experienced quality of critical system components. In this paper, we classify different research methods and technological solutions analyzing their pros and constraints. Based on this analysis we present the user-centered methodological framework being used throughout the whole development process of the system and aimed at achieving the best performance and quality appealing to the end user.

Simulator sickness — Five experiments using autostereoscopic mid-sized or small mobile screens

Visual comfort is one of the factors of 3D experience. The goal of this paper is to evaluate simulator sickness as a part of five different video quality evaluation studies. The experiments are conducted using three different dual-view autostereoscopic midsized and small mobile screens with varying constructed video quality levels, under different lengths of viewing, and with a total of 200 participants. The simulator sickness questionnaire (SSQ) was used in the data-collection in the pre- and postimmersion. The results showed only a small short term increase in the symptoms after exposure in four studies indicating a good applicability of these displays for short term video viewing.