Marwin Schmitt

A QoE testbed for socially-aware video-mediated group communication

Video-Mediated group communication is filtering into everyday use, as commercial products enable people to connect with friends and relatives. Current solutions provide basic support, so that communication can happen, but do they enable conversations? This paper argues that the purpose and the context of the conversation are influential factors that are rarely taken into consideration. The aim should be on the development of underlying mechanisms that can seamless palliate the effects of networking variances (e.g., delays) and optimize media and connection for every single participant. In particular, our interest is on how to improve remote multi-party gatherings by dynamically adjusting network and communication parameters, depending on the ongoing conversation. If we are to provide a software component that can, in real-time, monitor the Quality of Experience (QoE), we would have to carry out extensive experiments under different varying (but controllable) conditions. Unfortunately, there are no tools available that provide us the required fined-grained level of control. This paper reports on our efforts implementing such a testbed. It provides the experiment conductor with the possibility of modifying and monitoring network and media conditions in real-time.

Asymmetric delay in video-mediated group discussions

Delay has been found as one of the most crucial factors determining the Quality of Experience (QoE) in synchronous video-mediated communication. The effect has been extensively studied for dyadic conversations and recently the study of small group communications has become the focus of the research community. Contrary to dyads, in which the delay is symmetrically perceived, this is not the case for groups. Due to the heterogeneous structure of the internet, asymmetric delays between participants are likely to occur. We conducted a trial with video-mediated group discussions with five participants in symmetric and asymmetric delay conditions. In this paper, we focus on the asymmetric conditions, in which only one participant has an added delay. Previous research showed that interaction and context are essential factors in the QoE of participants. Therefore, we perform an analysis of our data towards these factors and compare our data between the asymmetric and the symmetric delay conditions. We discuss the differences in perception of delay reported from multi-party to dyadic video-conferencing. Our data shows that already one person with a high delay, affects the experience of the whole group as strong as the person with delay. This disruption is perceived more intensely by active participants rather than non-active ones- regardless of whether they have the delay or not. Compared to dyadic situations the group scenario shows a less intense perception of delay.

The Influence of Interactivity Patterns on the Quality of Experience in Multi-party Video-mediated Conversations under Symmetric Delay Conditions

As commercial, off-the-shelf, services enable people to easily connect with friends and relatives, video-mediated communication is filtering into our daily activities. With the proliferation of broadband and powerful devices, multi-party gatherings are becoming a reality in home environments. With the technical infrastructure in place and has been accepted by a large user base, researchers and system designers are concentrating on understanding and optimizing the Quality of Experience (QoE) for participants. Theoretical foundations for QoE have identified three crucial factors for understanding the impact on the individuals perception: system, context, and user. While most of the current research tends to focus on the system factors (delay, bandwidth, resolution), in this paper we offer a more complete analysis that takes into consideration context and user factors. In particular, we investigate the influence of delay (constant system factor) in the QoE of multi-party conversations. Regarding the context, we extend the typical one-to-one condition to explore conversations between small groups (up to five people). In terms of user factors, we take into account conversation analysis, turn-taking and role-theory, for better understanding the impact of different user profiles. Our investigation allows us to report a detailed analysis on how delay influences the QoE, concluding that the actual interactivity pattern of each participant in the conversation results on different noticeability thresholds of delays. Such results have a direct impact on how we should design and construct video-communication services for multi-party conversations, where user activity should be considered as a prime adaptation and optimization parameter.

A QoE study of different stream and layout configurations in video conferencing under limited network conditions

One particular problem of QoE research in video conferencing is, that most research in the past concentrated on one-to-one video conferencing or simply video consumption. However, video conferencing with two people (one-to-one) and within a group (multi-party) is different. Particularly, limitations of one participant might have an effect on the QoE of the whole group. This possible effect however is not well studied. Therefore, this paper aims to better understand the impact of individual limitations towards the groups QoE. To do so, we show a study about different video stream configurations and layouts for multi-party conferencing in respect to individual network limitations. For this, we conduct a user study with 20 participants in 5 groups, in a semi-controlled setup. Such a setup, combines supervising participants locally while still using our software infrastructure deployed in the internet. Furthermore, we use an asymmetric experiment design, by putting every participant under a different condition, as this proposes a more realistic scenario. Within our study, we look at three different factors: layout, video quality and network limitations. To foster conversation between participants, the group engaged in a discussion based team building exercise about survival in the wilderness. Our findings show that packet loss and the resulting distortions have a greater impact on the QoE as reducing the video quality by its resolution. Furthermore, our findings indicate that participants are more satisfied in a visually equal layout (showing participants in a similar size) and a more balanced stream configuration.

1Mbps is enough: Video quality and individual idiosyncrasies in multiparty HD video-conferencing

Most video platforms deliver HD video in high bitrate encoding. Modern video-conferencing systems are capable of handling HD streams, but using multiparty conferencing, average internet connections in the home are on their bandwidth limit. For properly managing the encoding bitrate in videoconferencing, we must know what is the minimum bitrate requirement to provide users an acceptable experience, and what is the bitrate level after which QoE saturates?. Most available subjective studies in this area used rather dated technologies. We report on a multiparty study on video quality with HD resolution. We tested different encoding bitrates (256kbs, 1024kbs and 4096kbs) and packet loss rates (0, 0.5%) in groups of 4 participants with a scenario based on the ITU building blocks task. We discuss the influence of group interaction and individual idiosyncrasies based on different mixed models, and look at covariates engagement and enjoyment as further explanatory factors. We found that 256kbs is still sufficient to provide a fair overall experience, but video quality is noticed to be poor. On the higher bitrate end, most people will not perceive the difference between 1024kbs and 4096kbs, considering in both cases the quality to be close to excellent. Independent on bitrate, packet loss has a small but significant impact, quantifiable in, on average, less than half a point difference on a 5-point ITU scale.

Towards Individual QoE for Multiparty Videoconferencing

Videoconferencing is becoming an essential part in everyday life. The visual channel allows us for interactions that were not possible over audio-only communication systems, such as the telephone. However, being a de-facto over-the-top service, the quality of the delivered videoconferencing experience is subject to variations, depending on network conditions. Videoconferencing systems adapt to network conditions by changing, for example, encoding bit rate of the video. For this adaptation not to hamper the benefits related to the presence of a video channel in the communication, it needs to be optimized according to a measure of the quality of experience (QoE) as perceived by the user. The latter is highly dependent on the ongoing interaction and individual preferences, which have hardly been investigated so far. In this paper, we focus on the impact that video quality has on conversations that revolve around objects that are presented over the video channel. To this end, we conducted an empirical study where groups of four people collaboratively build a Lego model over a videoconferencing system. We examine the requirements for such a task by showing when the interaction, measured by visual and auditory cues, changes depending on the encoding bit rate and loss. We then explore the impact that prior experience with the technology and affective state have on QoE of participants. We use these factors to construct predictive models that double the accuracy compared to a model based on the system factors alone. We conclude with a discussion of how these factors could be applied in real-world scenarios.

Towards context-aware interactive Quality of Experience evaluation for audiovisual multiparty conferencing

In modern video conferencing services, just as in common video delivery, most of the resource optimization is taken care of in the codec layer. Modern codecs like H.264 use detailed perceptual models to optimize the data reduction in way that it is least noticed by us. Already early evaluations of telecommunication systems could establish that there are different thresholds for a good quality depending on the situation. It is further known that subjective quality perceptions vary from user to user. But the space of user and context factors is still largely unexplored. To gain insight in which parameters are key in differentiating quality perception, we need to explore the interaction in different situations while keeping a tight control over the system parameters. In this paper we explore how clustering participants by their interaction or rating behavior can reveal subgroups that show significantly different perception of the QoE delivered by the same videoconferencing system. While for a cluster of users we find video quality to influence other QoE dimensions such as audio, for another cluster this is not the case. We explore whether this effect is due to conversational dynamics (contextual factor) or individual preferences (user factor) and discuss what this would mean for the design of future video-conferencing systems, that want to dynamically adapt to situation and participants.

A Quality of Experience Testbed for Video-Mediated Group Communication

Video-Mediated group communication is quickly moving from the office to the home, where network conditions might fluctuate. If we are to provide a software component that can, in real-time, monitor the Quality of Experience (QoE), we would have to carry out extensive experiments under different varying (but controllable) conditions. Unfortunately, there are no tools available that provide us the required fined-grained level of control. This paper reports on our efforts implementing such a test bed. The test bed provides the experiment conductor full control over the complete media pipeline, and the possibility of modifying in real-time network and media conditions. Additionally, it has facilities to easily develop an experiment with custom layouts, task integration, and assessment of subjective ratings through questionnaires. We have already used the test bed in a number of evaluations, reported in this paper for discussing the benefits and drawbacks of our solution. The test bed have been proven to be a flexible and effective canvas for better understanding QoE on video-mediated group communication.

Methods for Human-Centered Evaluation of MediaSync in Real-Time Communication

In an ideal world people interacting using real-time multimedia links experience perfectly synchronized media, and there is no latency of transmission: the interlocutors would hear and see each other with no delay. Methods to achieve the former are discussed in other chapters in this book, but for a variety of practical and physical reasons, delay-free communication will never be possible. In some cases, the delay will be very obvious since it will be possible to observe the reaction time of the listeners modified by the delay, or there may be some acoustic echo from the listeners’ audio equipment. However, in the absence of echo, the users themselves do not always explicitly notice the presence of delay, even for quite large values. Typically, they notice something is wrong (for example “we kept interrupting each other!”), but are unable to define what it is. Some useful insights into the impact of delay on a conversation can be obtained from the linguistic discipline of Conversation Analysis, and especially the analysis of “turn-taking” in a conversation. This chapter gives an overview of the challenges in evaluating media synchronicity in real-time communications, outlining appropriate tasks and methods for subjective testing and how in-depth analysis of such tests can be performed to gain a deep understanding of the effects of delay. The insights are based on recent studies of audio and audiovisual communication, but also show examples from other media synchronization applications like networked music interaction.

Mitigating problems in video-mediated group discussions: Towards conversation aware video-conferencing systems

In this paper we discuss communication problems in video-mediated small group discussions. We present results from a study in which ad-hoc groups of five people, with moderator, solved a quiz question-select answer style task over a video-conferencing system. The task was performed under different delay conditions, of up to 2000ms additional one-way delay. Even with a delay up to 2000ms, we could not observe any effect on the achieved quiz scores. In contrast, the subjective satisfaction was severely negatively affected. While we would have suspected a clear conversational breakdown with such a high delay, groups adapted their communication style and thus still managed to solve the task. This is, most groups decided to switch to a more explicit turn-taking scheme. We argue that future video-conferencing systems can provide a better experience if they are aware of the current conversational situation and can provide compensation mechanisms. Thus we provide an overview of what cues are relevant and how they are affected by the video-conferencing system and how recent advancements in computational social science can be leveraged. Further, we provide an analysis of the suitability of normal webcam data for such cue recognition. Based on our observations, we suggest strategies that can be implemented to alleviate the problems.