Andreas Sackl

YouTube & Facebook Quality of Experience in mobile broadband networks

YouTube and Facebook are two of the most consumed applications in todays Internet. Together they account for more than 30% of the overall Internets traffic volume and span more than half of the Internet users worldwide. Such popularity has attracted a growing interest from the research community, particularly regarding content characterization, network performance, and privacy issues in these applications. Their extensive and ever-growing usage in cellular networks has also captured the attention of mobile operators, who need to engineer their systems to handle the load while providing good quality levels. In this paper we take a user-centric approach for YouTube and Facebook performance evaluation, analyzing the Quality of Experience (QoE) as assessed by a group of 33 mobile broadband users in a field trial. Spanning an evaluation period of 31 days and using their own 3.5G-connected laptops, users regularly reported their perceived experience on surfing their preferred YouTube and Facebook contents under changing network conditions, artificially modified through traffic shaping. Our approach is holistic and considers a three-layered evaluation methodology, including control and monitoring of the network-layer QoS, content monitoring at the network and application layers, and QoE assessment at the user-layer. To the best of our knowledge, this paper presents the first results on the evaluation of different 3.5G network conditions on YouTube and Facebook from the enduser perspective, considering everyday life Web usage scenarios.

The QoE alchemy: Turning quality into money. Experiences with a refined methodology for the evaluation of willingness-to-pay for service quality

Studying the connection between QoE and QoS and its implications for network and content providers has yielded a couple of well-established experimental approaches. However, research in this field has mainly neglected economical aspects of user trials. This paper describes and evaluates an extended experimental setup to investigate willingness-to-pay for enhanced video quality inferred from network quality parameters. The results confirm that our realistic setup provides valid results indicating the suitability for upcoming studies bringing economical aspects and QoE together.

The role of cognitive dissonance for QoE evaluation of multimedia services

In contrast to traditional QoS-based charging, charges for QoE (Quality of Experience) play a double role both as part of the evaluation context and as counterpart of the users willingness-to-pay for service quality. To further investigate this link between pricing and QoE, we combine some unexpected results from empirical quality perception experiments with the socio-psychological theory of cognitive dissonance. With this approach, we are able to explain the influence of active user decisions on QoE. Moreover, economical implications, further research opportunities and practical advices for quality testing are presented.

The trouble with choice: An empirical study to investigate the influence of charging strategies and content selection on QoE

Despite the fact that novel QoE based charging mechanism are vitally needed, the complex interrelation of payment and quality perception has been examined only marginally so far. In this paper we want to describe a comprehensive experiment which investigates the intricate interplay of content selection, quality decisions & evaluation and payment strategies in the context of a video on demand scenario. Beside depicting methodological challenges and providing recommendations for further empirical work, we also compare current findings with our previous work to reveal new sights and research attempts.

A fixed-point model for QoE-based charging

Within the current paradigm change from Quality-of-Service (QoS) towards Quality-of-Experience (QoE), the question of how to charge for QoE is widely neglected in the research community despite of its obvious importance. In this paper, we present and analyze a fixed point model which specifically reflects the double role of prices, i.e. as regulating factor for demand size and at the same time as part of the QoE-based user context. The model is validated through comprehensive user trial results which allow interesting insights into the temporal behavior of end users who are confronted with a fine-grained scale of choices on video quality and corresponding tariffs.

Next to You: Monitoring Quality of Experience in Cellular Networks From the End-Devices

A quarter of the world population will be using smartphones to access the Internet in the near future. In this context, understanding the quality of experience (QoE) of popular apps in such devices becomes paramount to cellular network operators, who need to offer high-quality levels to reduce the risks of customers churning for quality dissatisfaction. In this paper, we address the problem of QoE provisioning in smartphones from a double perspective, combining the results obtained from subjective laboratory tests with end-device passive measurements and QoE crowd-sourced feedback obtained in operational cellular networks. The study addresses the impact of both access bandwidth and latency on the QoE of five different services and mobile apps: YouTube, Facebook, Web browsing through Chrome, Google Maps, and WhatsApp. We evaluate the influence of both constant and dynamically changing network access conditions, tackling in particular the case of fluctuating downlink bandwidth, which is typical in cellular networks. As a main contribution, we show that the results obtained in the laboratory are highly applicable in the live scenario, as mappings track the QoE provided by users in real networks. We additionally provide hints and bandwidth thresholds for good QoE levels on such apps, as well as discussion on end-device passive measurements and analysis. The results presented in this paper provide a sound basis to better understand the QoE requirements of popular mobile apps, as well as for monitoring the underlying provisioning network. To the best of our knowledge, this is the first paper providing such a comprehensive analysis of QoE in mobile devices, combining network measurements with users QoE feedback in laboratory tests, and operational networks.

YouTube in the move: Understanding the performance of YouTube in cellular networks

YouTube is the most popular and volume-dominant service in todays Internet, and is changing the way ISPs manage their networks. Understanding the performance of YouTube traffic is paramount for ISPs, specially for mobile operators, who must handle the huge surge of traffic with the constraints and challenges of cellular networks. In this paper we present an empirical analysis of the performance of YouTube flows accessed through a national-wide cellular network, considering download throughput as well as end-user Quality of Experience (QoE) metrics. The analysis considers the characteristics and impacts of the Content Delivery Network hosting YouTube, and compares its behavior with other popular HTTP video streaming services accessed through cellular networks. The QoE analysis is performed through end-user device measurements, which directly reflect the experience of the end-users. Our study additionally shows the potentiality of monitoring YouTube performance in cellular networks directly from the smart-phones of the users, bypassing the traffic visibility loss at the core of the network introduced by traffic encryption (e.g., HTTPS).

Where's the music? comparing the QoE impact of temporal impairments between music and video streaming

We are witnessing a dramatic rise of cloud-based media services nowadays, not only in terms of importance to end users but also in terms of traffic volume - a challenge for network and service providers alike. However, while the Quality of Experience of cloud-based video streaming services like YouTube has already become a well studied subject, this is less the case for music streaming services like Spotify and Deezer. In this paper we investigate the impact of temporal impairments on the QoE of music streaming. We present results from a subjective lab experiment which exposed participants to initial delay and interruptions (stallings) of playback when accessing streamed music via a Spotify service mock-up. Furthermore, we compare the results with other experiments conducted in the same lab that used equivalent study designs and setups but addressed video streaming instead. Our comparison reveals many similarities between music and video streaming (users preferring initial delay over stalling, generic relationships between impairments and QoE) but also substantial differences (e.g. user tolerance to stalling) between both service types.

On the quest for new KPIs in mobile networks: The impact of throughput fluctuations on QoE

Downlink throughput is the most widely used and accepted Quality of Service (QoS) related feature within the networking community, specially in the operational field. Current quality monitoring and reporting systems as well as quality benchmarking campaigns use the Average Downlink Throughput (ADT) as metric to assess the performance of the network. For example, flow-based monitoring systems normally report the per flow ADT as a throughput-based Key Performance Indicator (KPI), which is then aggregated at different temporal (e.g., 15 minutes), geographical (e.g., per radio cell) and/or logical (e.g., per service) scales to reflect the health of the network. A similar direction is currently followed within the Quality of Experience (QoE) research domain, where ADT is translated into a measure of user satisfaction for bandwidth-sensitive services (e.g., video streaming, file sharing, etc.). We claim that the ADT is not always an accurate KPI in terms of QoE for bandwidth-sensitive services, and present results showing that the variation of downlink throughput can actually have a major impact on the perceived quality of the end user. In this paper we present a complete study of the QoE undergone by 52 mobile users in controlled subjective lab tests, using different mobile applications such as YouTube, Facebook and Gmaps. By shaping the traffic of the users through multiple bandwidth fluctuation patterns, we conclude that novel downlink-throughput related KPIs must be defined for QoE-based traffic analysis in mobile networks. Based on this observation, we propose some very first simple throughput fluctuation models to define such KPIs.

Market entrance, user interaction and willingness-to-pay: Exploring fundamentals of QoE-based charging for VoD services

Strongly rising interest in Quality of Experience (QoE) research together with the current economical struggle of Network Service Providers (NSPs) are about to lead towards monetization of QoE overtaking outright quality optimization in importance. While there is still a significant lack of quantitative studies for backing QoE-based charging recommendations, the present work presents results and conclusions from a recent empirical user study explicitly measuring the maximum willingness-to-pay of end users for High Definition (HD) Video on Demand (VoD) services. Beyond the observation of unexpectedly high expenditures by test subjects, we focus on phenomena with respect to market entrance pricing strategies, which may be explained by anchoring effects well-known in psychology. Thus, the present work provides valuable clues on how to approach VoD markets in an optimal and economically efficient way.