Ragnhild Eg

Randomised pair comparison: an economic and robust method for audiovisual quality assessment

Subjective quality perception studies with human observers are essential for multimedia system design. Such studies are known to be expensive and difficult to administer. They require time, a detailed knowledge of experimental designs and a level of control which can often only be achieved in a laboratory setting. Hence, only very few researchers consider running subjective studies at all. In this paper we present Randomised Pair Comparison (r/pc), an easy-to-use, flexible, economic and robust extension to conventional pair comparison methods. r/pc uses random sampling to select a unique and small subset of pairs for each assessor, thus separating session duration from the experimental design. With r/pc an experimenter can freely define the duration of sessions and balance between costs and accuracy of an experiment. On a realistic example study we show that r/pc is able to create stable results with an accuracy close to full factorial designs, yet much lower costs. We also provide initial evidence that r/pc can avoid unpleasant fatigue and learning effects which are common in long experiment sessions.

Spatial flicker effect in video scaling

Scalable video streaming may result in flicker effects — visual artifacts in video presentation due to adaptive layer switching. In our work, we have identified three types of flicker, noise, blur and motion flicker. Here, we investigate the blur and noise flicker, which are both related to the spatial domain. The perceptual impact of blur and noise flicker is explored through subjective assessments, focusing on acceptance of variations in the amplitudes and frequencies of the quality changes, across four content types. Our results indicate that the perception and acceptance of different layer variations are jointly influenced by multiple factors. When video scaling is required, frequency can be adjusted to relieve the annoyance of flicker artifacts, while amplitude still plays the dominant role in delivering satisfactory video quality. In addition, contents with complex details are more affected by blur flicker than other contents. This difference is not observed for noise flicker.

Can gamers detect cloud delay

In many games, a win or a loss is not only contingent on the speedy reaction of the players, but also on how fast the game can react to them. From our ongoing project, we aim to establish perceptual thresholds for visual delays that follow user actions. In this first user study, we eliminated the complexities of a real game and asked participants to adjust the delay between the push of a button and a simple visual presentation. At the most sensitive, our findings reveal that some perceive delays below 40 ms. However, the median threshold suggests that motor-visual delays are more likely than not to go undetected below 51-90 ms. These results will in future investigations be compared to thresholds for more complex visual stimuli, and to thresholds established from different experimental approaches.

Audio-visual identification of place of articulation and voicing in white and babble noisea)

Research shows that noise and phonetic attributes influence the degree to which auditory and visual modalities are used in audio-visual speech perception (AVSP). Research has, however, mainly focused on white noise and single phonetic attributes, thus neglecting the more common babble noise and possible interactions between phonetic attributes. This study explores whether white and babble noise differentially influence AVSP and whether these differences depend on phonetic attributes. White and babble noise of 0 and -12 dB signal-to-noise ratio were added to congruent and incongruent audio-visual stop consonant-vowel stimuli. The audio (A) and video (V) of incongruent stimuli differed either in place of articulation (POA) or voicing. Responses from 15 young adults show that, compared to white noise, babble resulted in more audio responses for POA stimuli, and fewer for voicing stimuli. Voiced syllables received more audio responses than voiceless syllables. Results can be attributed to discrepancies in the acoustic spectra of both the noise and speech target. Voiced consonants may be more auditorily salient than voiceless consonants which are more spectrally similar to white noise. Visual cues contribute to identification of voicing, but only if the POA is visually salient and auditorily susceptible to the noise type.

Crowdsourcing as self-fulfilling prophecy: Influence of discarding workers in subjective assessment tasks

Crowdsourcing has established itself as a powerful tool for multimedia researchers, and is commonly used to collect human input for various purposes. It is also a fairly widespread practice to control the contributions of users based on the quality of their input. This paper points to the fact that applying this practice in subjective assessment tasks may lead to an undesired, negative outcome. We present a crowdsourcing experiment and a discussion of the ways in which control in crowdsourcing studies can lead to a phenomenon akin to a self-fulfilling prophecy. This paper is intended to trigger discussion and lead to more deeply reflective crowdsourcing practices in the multimedia context.

Adaptive media streaming to mobile devices: challenges, enhancements, and recommendations

Video streaming is predicted to become the dominating traffic in mobile broadband networks. At the same time, adaptive HTTP streaming is developing into the preferred way of streaming media over the Internet. In this paper, we evaluate how different components of a streaming system can be optimized when serving content to mobile devices in particular. We first analyze the media traffic from a Norwegian network and media provider. Based on our findings, we outline benefits and challenges for HTTP streaming, on the sender and the receiver side, and we investigate how HTTP-based streaming affects server performance. Furthermore, we discuss various aspects of efficient coding of the video segments from both performance and user perception point of view. The final part of the paper studies efficient adaptation and delivery to mobile devices over wireless networks. We experimentally evaluate and improve adaptation strategies, multilink solutions, and bandwidth prediction techniques. Based on the results from our evaluations, we make recommendations for how an adaptive streaming system should handle mobile devices. Small changes, or simple awareness of how users perceive quality, can often have large effects.

Audiovisual robustness: exploring perceptual tolerance to asynchrony and quality distortion

Rules-of-thumb for noticeable and detrimental asynchrony between audio and video streams have long since been established from the contributions of several studies. Although these studies share similar findings, none have made any discernible assumptions regarding audio and video quality. Considering the use of active adaptation in present and upcoming streaming systems, audio and video will continue to be delivered in separate streams; consequently, the assumption that the rules-of-thumb hold independent of quality needs to be challenged. To put this assumption to the test, we focus on the detection, not the appraisal, of asynchrony at different levels of distortion. Cognitive psychologists use the term temporal integration to describe the failure to detect asynchrony. The term refers to a perceptual process with an inherent buffer for short asynchronies, where corresponding auditory and visual signals are merged into one experience. Accordingly, this paper discusses relevant causes and concerns with regards to asynchrony, it introduces research on audiovisual perception, and it moves on to explore the impact of audio and video quality on the temporal integration of different audiovisual events. Three content types are explored, speech from a news broadcast, music presented by a drummer, and physical action in the form of a chess game. Within these contexts, we found temporal integration to be very robust to quality discrepancies between the two modalities. In fact, asynchrony detection thresholds varied considerably more between the different content than they did between distortion levels. Nevertheless, our findings indicate that the assumption concerning the independence of asynchrony and audiovisual quality may have to be reconsidered.

The Effects of Delay on Game Actions: Moving Target Selection with a Mouse

In modern computer systems, user input, particularly for computer games, is affected by delay from local systems, networks and servers. While general awareness of the degradation effects of delay on player performance and quality of experience are well known, an understanding quantifying how specific player actions are impacted by delay is missing. This work presents a user study that gathers data on player actions for a range of delay and game conditions for the fundamental game action of selecting a moving target with a mouse. Analysis shows player sensitivity to delays in all conditions, with particular sensitivity when targets are fast. From the data, we derive a simple analytic model that is a promising step towards a broadly applicable tool to better understand and compensate for delay in games and interactive applications.

The Cameraman Operating My Virtual Camera is Artificial: Can the Machine Be as Good as a Humanq

In this article, we argue that the energy spent in designing autonomous camera control systems is not spent in vain. We present a real-time virtual camera system that can create smooth camera motion. Similar systems are frequently benchmarked with the human operator as the best possible reference; however, we avoid a priori assumptions in our evaluations. Our main question is simply whether we can design algorithms to steer a virtual camera that can compete with the user experience for recordings from an expert operator with several years of experience? In this respect, we present two low-complexity servoing methods that are explored in two user studies. The results from the user studies give a promising answer to the question pursued. Furthermore, all components of the system meet the real-time requirements on commodity hardware. The growing capabilities of both hardware and network in mobile devices give us hope that this system can be deployed to mobile users in the near future. Moreover, the design of the presented system takes into account that services to concurrent users must be supported.

Instantaneous Human-Computer Interactions: Button Causes and Screen Effects

Many human-computer interactions are highly time-dependent, which means that an effect should follow a cause without delay. In this work, we explore how much time can pass between a cause and its effect without jeopardising the subjective perception of instantaneity. We ran two experiments that involve the same simple interaction: A click of a button causes a spinning disc to change its direction of rotation, following a variable delay. In our adjustment experiment, we asked participants to adjust the delay directly, but without numerical references, using repeated attempts to achieve a value as close to zero as possible. In the discrimination task, participants made judgements on whether the single rotation change happened immediately following the button-click, or after a delay. The derived thresholds revealed a marked difference between the two experimental approaches, participants could adjust delays down to a median of 40 ms, whereas the discrimination mid-point corresponded to 148 ms. This difference could possibly be an artefact of separate strategies adapted by participants for the two tasks. Alternatively, repeated presentations may make people more sensitive to delays, or provide them with additional information to base their judgements on. In either case, we have found that humans are capable of perceiving very short temporal delays, and these empirical results provide useful guidelines for future designs of time-critical interactions.