Tobias Hoßfeld

Quantification of YouTube QoE via Crowdsourcing

This paper addresses the challenge of assessing and modeling Quality of Experience (QoE) for online video services that are based on TCP-streaming. We present a dedicated QoE model for You Tube that takes into account the key influence factors (such as stalling events caused by network bottlenecks) that shape quality perception of this service. As second contribution, we propose a generic subjective QoE assessment methodology for multimedia applications (like online video) that is based on crowd sourcing - a highly cost-efficient, fast and flexible way of conducting user experiments. We demonstrate how our approach successfully leverages the inherent strengths of crowd sourcing while addressing critical aspects such as the reliability of the experimental data obtained. Our results suggest that, crowd sourcing is a highly effective QoE assessment method not only for online video, but also for a wide range of other current and future Internet applications.

An Evaluation of QoE in Cloud Gaming Based on Subjective Tests

Cloud Gaming is a new kind of service, which combines the successful concepts of Cloud Computing and Online Gaming. It provides the entire game experience to the users remotely from a data center. The player is no longer dependent on a specific type or quality of gaming hardware, but is able to use common devices. The end device only needs a broadband internet connection and the ability to display High Definition (HD) video. While this may reduce hardware costs for users and increase the revenue for developers by leaving out the retail chain, it also raises new challenges for service quality in terms of bandwidth and latency for the underlying network. In this paper we present the results of a subjective user study we conducted into the user-perceived quality of experience (QoE)in Cloud Gaming. We design a measurement environment, that emulates this new type of service, define tests for users to assess the QoE, derive Key Influence Factors (KFI) and influences of content and perception from our results.

Efficient simulation of large-scale p2p networks: packet-level vs. flow-level simulations

Peer-to-peer (P2P) networks can reduce the distribution cost of large media files for the original provider of the data significantly. Thereby, the BitTorrent protocol is widely used in the Internet today. Most research work studies the protocol analytically, by simulations at the flow-level or real world experiments. Thereby, for flow-level simulations the influence of neglecting packet-level characteristics is not yet quantified. Therefore, this paper compares packet-level simulation results with flow-level values and analytically derived bounds. Our findings show that BitTorrent is near to optimal at flow-level for different scenarios. Naturally, packet-level results deviate more from the optimal values but differences are at most around 30% in our simulations. Furthermore, we show that the propagation delay can significantly influence the download performance of BitTorrent.

“Time is bandwidth”? Narrowing the gap between subjective time perception and Quality of Experience

Over the last couple of years, the scope of Quality of Experience (QoE) research has been constantly extended, most recently to the field of Web QoE in the context of HTTP-based applications. In this paper, we address the question whether it is sufficient to reduce typical Web QoE assessment scenarios to the temporal aspects of waiting for task completion, which would allow to attribute the resulting logarithmic laws to well-known psychological insights on human time perception. We demonstrate that while this attribution is valid for simple waiting tasks which are typical for simple data services like e.g. file downloads, the case of interactive web browsing is much more complex. We show that this is not only because technical issues prevent bandwidth and download time from being directly correlated with each other in a simple manner, but also because user perceived web page load times strongly deviate from technical page load times. Consequently, existing approaches towards assessment and modeling of web browsing QoE have to be critically reviewed and redesigned.

Internet video delivery in youtube: from traffic measurements to quality of experience

This chapter investigates HTTP video streaming over the Internet for the YouTube platform. YouTube is used as concrete example and case study for video delivery over the Internet, since it is not only the most popular online video platform, but also generates a large share of traffic on todays Internet. We will describe the YouTube infrastructure as well as the underlying mechanisms for optimizing content delivery. Such mechanisms include server selection via DNS as well as application-layer traffic management. Furthermore, the impact of delivery via the Internet on the user experienced quality (QoE) of YouTube video streaming is quantified. In this context, different QoE monitoring approaches are qualitatively compared and evaluated in terms of the accuracy of QoE estimation.

Analyzing costs and accuracy of validation mechanisms for crowdsourcing platforms

Abstract Crowdsourcing is becoming more and more important for commercial purposes. With the growth of crowdsourcing platforms like Amazon Mechanical Turk or Microworkers, a huge work force and a large knowledge base can be easily accessed and utilized. But due to the anonymity of the workers, they are encouraged to cheat the employers in order to maximize their income. In this paper, we analyze two widely used crowd-based approaches to validate the submitted work. 1 Both approaches are evaluated with regard to their detection quality, their costs and their applicability to different types of typical crowdsourcing tasks.

From packets to people: quality of experience as a new measurement challenge

Over the course of the last decade, the concept of Quality of Experience (QoE) has gained strong momentum, both from an academic research and an industry perspective. Being linked very closely to the subjective perception of the end user, QoE is supposed to enable a broader, more holistic understanding of the qualitative performance of networked communication systems and thus to complement the traditional, more technology-centric Quality of Service (QoS) perspective. The purpose of this chapter is twofold: firstly, it introduces the reader to QoE by discussing the origins and the evolution of the concept. Secondly, it provides an overview of the current state of the art of QoE research, with focus on work that particularly addresses QoE as a measurement challenge on the technology as well as on the end-user level. This is achieved by surveying the different streams of QoE research that have emerged in the context of Video, Voice and Web services with respect to the following aspects: fundamental relationships and perceptual principles, QoE assessment, modeling and monitoring.

Gaming in the clouds: QoE and the users’ perspective

Abstract Cloud Gaming is a new kind of service, which combines the successful concepts of Cloud Computing and Online Gaming. It provides the entire game experience to the users remotely from a data center. The player is no longer dependent on a specific type or quality of gaming hardware, but is able to use common devices. The end device only needs a broadband internet connection and the ability to display High Definition (HD) video. While this may reduce hardware costs for users and increase the revenue for developers by leaving out the retail chain, it also raises new challenges for service quality in terms of bandwidth and latency for the underlying network. In this paper we present the results of a subjective user study we conducted into the user-perceived quality of experience (QoE) in Cloud Gaming. We design a measurement environment, that emulates this new type of service, define tests for users to assess the QoE, derive Key Influence Factors (KIF) and influences of content and perception from our results.

Quantification of quality of experience for edge-based applications

In future Internet, multi-network services correspond to a new paradigm that intelligence in network control is gradually moved to the edge of the network. As a consequence, the application itself can influence or determine the amount of consumed bandwidth. Thus the user behaviour may change dramatically. This impacts the Quality of Service (QoS) and the Quality of Experience (QoE), a subjective measure from the user perspective of the overall value of the provided service or application. A selfish user or application tries to maximize its own QoE rather than to optimize the network QoS, in contrast to a legacy altruistic user. In this paper we present the IQX hypothesis which assumes an exponential functional relationship between QoE and QoS. This contribution is a first step towards the quantification of the QoE for edge-based applications, where an example of VoIP is taken into account. Starting from a measurement of the Skype application, we show the basic properties of selfish and altruistic user behaviour in accordance to edge-based intelligence. The QoE is quantified in terms of MOS in dependence of the packet loss of the end-to-end connection, whereby Skypes iLBC voice codec is used exemplarily. It is shown that the IQX hypothesis is verified in this application scenario. Furthermore, selfish user behaviour with replicated sending of voice datagrams is investigated with respect to the obtained QoE of a single user. In addition, the impact of this user behaviour on congestion in the network is outlined by means of simulations.

Assessing effect sizes of influence factors towards a QoE model for HTTP adaptive streaming

HTTP Adaptive Streaming (HAS) is employed by more and more video streaming services in the Internet. It allows to adapt the downloaded video quality to the current network conditions, and thus, avoids stalling (i.e., playback interruptions) to the greatest possible extend. The adaptation of video streams is done by switching between different quality representation levels, which influences the user perceived quality of the video stream. In this work, the influence of several adaptation parameters, namely, switch amplitude (i.e., quality level difference), switching frequency, and recency effects, on Quality of Experience (QoE) is investigated. Therefore, crowdsourcing experiments were conducted in order to collect subjective ratings for different adaptation-related test conditions. The results of these subjective studies indicate the influence of the adaptation parameters, and based on these findings a simplified QoE model for HAS is presented, which only relies on the switch amplitude and the playback time of each layer.