Toni Mäki

A reduced-reference parametric model for audiovisual quality of IPTV services

In this paper we present a parametric model for audiovisual quality estimation in IPTV and similar services. The proposed model takes advantage of signal characteristics calculated at the sender (in particular related to levels of motion in the content), but is purely parametric on the estimation (i.e. it does not require peeking into the bitstream), which makes it suitable for large-scale real-time monitoring applications. In order to obtain the model, we followed the Pseudo-Subjective Quality Assessment (PSQA) methodology, and compared different kinds of statistical estimators, namely Multilayer Perceptrons (MLP) and Random Neural Networks (RNN).

Bridging the gap between eye tracking and crowdsourcing

Visual attention constitutes a very important feature of the human visual system (HVS). Every day when watching videos, images or browsing the Internet, people are confronted with more information than they are able to process, and analyze only part of the information in front of them. In parallel, crowdsourcing has become a particularly hot topic, enabling to scale subjective experiments to a large crowd with diversity in terms of nationalities, social background, age, etc. This paper describes a novel framework with the aim to bridge these two fields, by providing a new way of measurements of users experience in a subjective crowdsourcing experiment. This study goes beyond self-reported methods, and provide a new kind of information for the context of crowdsourcing: visual attention. The results show that it is possible to estimate visual attention, in a non-intrusive manner and without using self-reported methods or specialized equipment, with a precision as high as 14.1% in the horizontal axis and 17.9% in the vertical axis. This accuracy is sufficient for many kinds of measurements that can be efficiently executed only in non-controlled environments..

Towards an understanding of visual appeal in website design

Visual appeal (VA), as judged by users accessing different websites, has been shown to impact overall user preferences and satisfaction. In the context of Web Quality of Experience (QoE), we hypothesize that VA has a non-trivial impact on end user quality perception and hence needs to be accounted for when modeling overall Web QoE. In this paper, we aim to quantify the impact of VA factors on perceived visual design quality for different types of tested websites. We carried out two large-scale (>350 users each) crowd-sourced campaigns to test the influence of several factors often found in design best practices recommendations. While results have identified color goodness and font goodness as Key Influence Factors (KIF) for VA, an analysis of user demographics has further shown a strong impact of users origin on subjective ratings. By quantifying the impact of identified KIFs on VA, we provide guidelines for successful VA manipulations to be considered in future Web QoE subjective tests.

QoE in the Web: A dance of design and performance

Web services and applications are an increasingly pervasive fixture of modern life and business. This paper reports on a Web-based crowdsourced subjective assessment campaign in which we studied the effects of network performance and design aspects of manipulated Web sites on the perceived performance, perceived ease-of-use, visual appeal, as well as on the overall Quality of Experience of users browsing a given Web site. We consider the simultaneous impact of multiple factors on different dimensions of QoE, as well as their interplay, with a focus on the interactions between page/element loading times and Web site visual appeal and ease-of-use. Key results have shown the following: page loading times and visual appeal have a significant effect on overall user QoE; both higher perceived aesthetics and higher perceived ease-of-use result in an increased user tolerance to delay; and overall QoE is strongly correlated with perceived aesthetics, perceived ease-of-use, and perceived network performance.

A Layered Model for Quality Estimation of HTTP Video from QoS Measurements

HTTP video is quickly becoming a dominating type of traffic on the Internet, with popular services such as YouTube and Netflix being used by hundreds of millions of users daily, and showing ever-growing usage numbers. Understanding Quality of Experience (QoE) for these services is an important topic, and one that has been addressed in the literature. However, the available works focus on the impact of application-level events (e.g. stalls) on the perceived quality, but not on the underlying cause, i.e., network-level impairments, as the relation between Quality of Service (QoS) and QoE is significantly more complex than it was in the case of RTP/UDP based video, due to HTTP video being streamed over TCP. In this paper we present a first step in the direction of solving this QoS-to-QoE mapping for HTTP video, by providing a (parametric) layered model approach for network-side QoE monitoring.

Chamber QoE: a multi-instrumental approach to explore affective aspects in relation to quality of experience

Evaluating (audio)visual quality and Quality of Experience (QoE) from the user’s perspective, has become a key element in optimizing users’ experiences and their quality. Traditionally, the focus lies on how multi-level quality features are perceived by a human user. The interest has however gradually expanded towards human cognitive, affective and behavioral processes that may impact on, be an element of, or be influenced by QoE, and which have been underinvestigated so far. In addition, there is a major discrepancy between the new, broadly supported and more holistic conceptualization of QoE proposed by Le Callet et al. (2012) and traditional, standardized QoE assessment. This paper explores ways to tackle this discrepancy by means of a multi-instrumental approach. More concretely, it presents results from a lab study on video quality (N=27), aimed at going beyond the dominant QoE assessment paradigm and at exploring affective aspects in relation to QoE and in relation to perceived overall quality. Four types of data were collected: ‘traditional’ QoE self-report measures were complemented with ‘alternative’, emotional state- and user engagement-related self-report measures to evaluate QoE. In addition, we collected EEG (physiological) data, gazetracking data and facial expressions (behavioral) data. The video samples used in test were longer in duration than is common in standard tests allowing us to study e.g. more realistic experience and deeper user engagement. Our findings support the claim that the traditional QoE measures need to be reconsidered and extended with additional, affective staterelated measures.

Is affective crowdsourcing reliable

Affective content annotations are typically acquired from subjective manual assessments by experts in supervised laboratory tests. While well manageable, such campaigns are expensive, time-consuming and results may not be generalizable to larger audiences. Crowdsourcing constitutes a promising approach for quickly collecting data with wide demographic scope and reasonable costs. Undeniably, affective crowdsourcing is particularly challenging in the sense that it attempts to collect subjective perceptions from humans with different cultures, languages, knowledge background, etc. In this study we analyze the validity of well-known user affective scales in a crowdsourcing context by comparing results with the ones obtained in laboratory tests. Experimental results demonstrate that pictorial scales possess promising features for affective crowdsourcing.

Network quality differentiation: Regional effects, market entrance, and empirical testability

While for offline business models it does not seem necessary to reiterate the close relationship between quality and price, for Internet services the quality-based, i.e., Quality of Experience (QoE), and customer-centric pricing is non-trivial. As insufficient data exists today to successfully commercialise QoE, this paper collects the integral empirical Willingness-To-Pay (WTP) data for the case of online video services. This work reproduces and extends a previous study in two dedicated campaigns in Austria and Finland. The campaigns study QoE and WTP related to Dynamic Adaptive Streaming over HTTP (DASH). They also confirm or disprove previous studies, openly share the data, and provide empirical background information on the purchasing behavior of customers. Due to the testing at two locations, we can further first time study whether cultural or regional differences affect the purchasing behaviors of such services. Additionally this paper gives insights and updated methodological guidance on conducting future WTP studies.

QuoTe an extensible platform for QoE monitoring and benchmarking of telemedicine applications

Telemedicine applications provide many opportunities for improving health care in a variety of conditions, in particular for people living in remote or geographically isolated areas without fast access to doctors (“medical deserts”), such as some parts of Finland. In many applications, the technologies used are actually off-the shelf solutions for videoconferencing, in some cases even used in an Over The Top (OTT) fashion over best-effort networks. In these contexts, the quality (both Quality of Experience and Quality of Service) of the whole system can have a greater significance than in other contexts such as entertainment, yet there are no quality monitoring and assessment systems specifically conceived for this purpose. In this paper we present an on-going effort to develop an extensible quality monitoring and benchmarking platform designed with video-based telemedicine applications, and the particular issues associated with them, in mind.

Dynamic and intelligent SAND-enabled CDN management

In this demonstration, an advanced Content Delivery Network (CDN) solution for enhancing the delivery of adaptive HTTP-based video streaming is introduced. The demonstration showcases intelligent and scalable CDN testbed which is based on intelligent CDN management functionalities and scalable CDN architecture. The intelligent CDN management functionalities and the associated signaling are based on the upcoming MPEG standard for Server and Network assisted DASH (SAND). The scalable CDN architecture achieved via lightweight service virtualization allows dynamic scaling and balancing of available resources according to the current needs. The demonstration showcases a scenario where end-users are streaming MPEG-DASH video from the advanced CDN featuring intelligent CDN management and monitoring functionalities to dynamically add or remove virtualized edge servers and reroute end-users based on the resource needs and the location of end-users to achieve more balanced traffic load within the network and better Quality of Experience (QoE) for end-users.