Simon Gunkel

From IPTV to Synchronous Shared Experiences: Challenges in Design: Distributed Media Synchronization

This paper discusses results from research related to the use of television as a device that supports social interaction between close-knit groups in settings that include more than two locations, each location being potentially equipped with more than one camera. The paper introduces the notion of a framing experience, as a specific scenario or situation within which social communication takes place. It reports on the evaluation of some of the key attributes of social communication through semi-structured interviews, with 16 families across four European countries. The inferences drawn from this study are reduced to four system capabilities including the ability to support: excitement, engagement and entertainment; high quality, reliable audiovisual communications; flexibility and adaptability sufficient to support the unpredictable and reactive nature of human interaction and discourse. These system requirements are, in turn, reduced to a number of technology challenges which if solved will help enable effective social communications between groups, mediated by the television. These technology challenges include: high quality reliable audio visual communication; interaction orchestration, multimedia interpretation and multimedia composition. Finally the paper reflects on the impact the use of framing experiences, such as those described here, could have on strategy and policy for service providers and regulators.

A QoE testbed for socially-aware video-mediated group communication

Video-Mediated group communication is filtering into everyday use, as commercial products enable people to connect with friends and relatives. Current solutions provide basic support, so that communication can happen, but do they enable conversations? This paper argues that the purpose and the context of the conversation are influential factors that are rarely taken into consideration. The aim should be on the development of underlying mechanisms that can seamless palliate the effects of networking variances (e.g., delays) and optimize media and connection for every single participant. In particular, our interest is on how to improve remote multi-party gatherings by dynamically adjusting network and communication parameters, depending on the ongoing conversation. If we are to provide a software component that can, in real-time, monitor the Quality of Experience (QoE), we would have to carry out extensive experiments under different varying (but controllable) conditions. Unfortunately, there are no tools available that provide us the required fined-grained level of control. This paper reports on our efforts implementing such a testbed. It provides the experiment conductor with the possibility of modifying and monitoring network and media conditions in real-time.

Asymmetric delay in video-mediated group discussions

Delay has been found as one of the most crucial factors determining the Quality of Experience (QoE) in synchronous video-mediated communication. The effect has been extensively studied for dyadic conversations and recently the study of small group communications has become the focus of the research community. Contrary to dyads, in which the delay is symmetrically perceived, this is not the case for groups. Due to the heterogeneous structure of the internet, asymmetric delays between participants are likely to occur. We conducted a trial with video-mediated group discussions with five participants in symmetric and asymmetric delay conditions. In this paper, we focus on the asymmetric conditions, in which only one participant has an added delay. Previous research showed that interaction and context are essential factors in the QoE of participants. Therefore, we perform an analysis of our data towards these factors and compare our data between the asymmetric and the symmetric delay conditions. We discuss the differences in perception of delay reported from multi-party to dyadic video-conferencing. Our data shows that already one person with a high delay, affects the experience of the whole group as strong as the person with delay. This disruption is perceived more intensely by active participants rather than non-active ones- regardless of whether they have the delay or not. Compared to dyadic situations the group scenario shows a less intense perception of delay.

The Influence of Interactivity Patterns on the Quality of Experience in Multi-party Video-mediated Conversations under Symmetric Delay Conditions

As commercial, off-the-shelf, services enable people to easily connect with friends and relatives, video-mediated communication is filtering into our daily activities. With the proliferation of broadband and powerful devices, multi-party gatherings are becoming a reality in home environments. With the technical infrastructure in place and has been accepted by a large user base, researchers and system designers are concentrating on understanding and optimizing the Quality of Experience (QoE) for participants. Theoretical foundations for QoE have identified three crucial factors for understanding the impact on the individuals perception: system, context, and user. While most of the current research tends to focus on the system factors (delay, bandwidth, resolution), in this paper we offer a more complete analysis that takes into consideration context and user factors. In particular, we investigate the influence of delay (constant system factor) in the QoE of multi-party conversations. Regarding the context, we extend the typical one-to-one condition to explore conversations between small groups (up to five people). In terms of user factors, we take into account conversation analysis, turn-taking and role-theory, for better understanding the impact of different user profiles. Our investigation allows us to report a detailed analysis on how delay influences the QoE, concluding that the actual interactivity pattern of each participant in the conversation results on different noticeability thresholds of delays. Such results have a direct impact on how we should design and construct video-communication services for multi-party conversations, where user activity should be considered as a prime adaptation and optimization parameter.

A QoE study of different stream and layout configurations in video conferencing under limited network conditions

One particular problem of QoE research in video conferencing is, that most research in the past concentrated on one-to-one video conferencing or simply video consumption. However, video conferencing with two people (one-to-one) and within a group (multi-party) is different. Particularly, limitations of one participant might have an effect on the QoE of the whole group. This possible effect however is not well studied. Therefore, this paper aims to better understand the impact of individual limitations towards the groups QoE. To do so, we show a study about different video stream configurations and layouts for multi-party conferencing in respect to individual network limitations. For this, we conduct a user study with 20 participants in 5 groups, in a semi-controlled setup. Such a setup, combines supervising participants locally while still using our software infrastructure deployed in the internet. Furthermore, we use an asymmetric experiment design, by putting every participant under a different condition, as this proposes a more realistic scenario. Within our study, we look at three different factors: layout, video quality and network limitations. To foster conversation between participants, the group engaged in a discussion based team building exercise about survival in the wilderness. Our findings show that packet loss and the resulting distortions have a greater impact on the QoE as reducing the video quality by its resolution. Furthermore, our findings indicate that participants are more satisfied in a visually equal layout (showing participants in a similar size) and a more balanced stream configuration.

WebVR meets WebRTC: Towards 360-degree social VR experiences

Virtual Reality (VR) and 360-degree video are reshaping the media landscape, creating a fertile business environment. During 2016 new 360-degree cameras and VR headsets entered the consumer market, distribution platforms are being established and new production studios are emerging. VR is evermore becoming a hot topic in research and industry and many new and exciting interactive VR content and experiences are emerging. The biggest gap we see in these experiences are social and shared aspects of VR. In this demo we present our ongoing efforts towards social and shared VR by developing a modular web based VR framework, that extends current video conferencing capabilities with new functionalities of Virtual and Mixed Reality. It allows us to connect two people together for mediated audio-visual interaction, while being able to engage in interactive content. Our framework allows to run extensive technological and user based trials in order to evaluate VR experiences and to build immersive multi-user interaction spaces. Our first results indicate that a high level of engagement and interaction between users is possible in our 360-degree VR set-up utilizing current web technologies.

Social VR Platform: Building 360-degree Shared VR Spaces

Virtual Reality (VR) and 360-degree video are set to become part of the future social environment, enriching and enhancing the way we share experiences and collaborate remotely. In this demo, we present our ongoing efforts towards social and shared VR; a modular web based VR framework that extends current video conferencing capabilities with new VR functionalities. The framework allows for two people to come together for mediated audio-visual interaction, while engaging in (interactive) content. First results show that a majority of users appreciate the quality and feel highly immersed and present. Thus, with our demo we show that current web technologies can enable a high level of engagement and interaction in VR.

The Optimiser: monitoring and improving switching delays in video conferencing

With the growing popularity of video communication systems, more people are using group video chat, rather than only one-to-one video calls. In such multi-party sessions, remote participants compete for the available screen space and bandwidth. A common solution is showing the current speaker prominently. Bandwidth limitations may not allow all streams to be sent at a high resolution at all times, especially with many participants in a call. This can be mitigated by only switching on higher resolutions when they are required. This switching encounters delays due to latency and the properties of encoded video streams. In this paper, we analyse and improve the switching delay of our video conferencing system. Our server-centric system offers a next-generation video chat solution, providing end-to-end video encoding. To evaluate our system we use a testbed that allows us to emulate different network conditions. We measure the video switching delay between three clients, each connected via different network profiles. Our results show that missing Intra-Frames in the transmission has a strong influence on the switching delay. Based on this, we provide an optimization mechanism that improves those delays by resending Intra-Frames.

A Quality of Experience Testbed for Video-Mediated Group Communication

Video-Mediated group communication is quickly moving from the office to the home, where network conditions might fluctuate. If we are to provide a software component that can, in real-time, monitor the Quality of Experience (QoE), we would have to carry out extensive experiments under different varying (but controllable) conditions. Unfortunately, there are no tools available that provide us the required fined-grained level of control. This paper reports on our efforts implementing such a test bed. The test bed provides the experiment conductor full control over the complete media pipeline, and the possibility of modifying in real-time network and media conditions. Additionally, it has facilities to easily develop an experiment with custom layouts, task integration, and assessment of subjective ratings through questionnaires. We have already used the test bed in a number of evaluations, reported in this paper for discussing the benefits and drawbacks of our solution. The test bed have been proven to be a flexible and effective canvas for better understanding QoE on video-mediated group communication.

Virtual reality conferencing: multi-user immersive VR experiences on the web

Virtual Reality (VR) and 360-degree video are set to become part of the future social environment, enriching and enhancing the way we share experiences and collaborate remotely. While Social VR applications are getting more momentum, most services regarding Social VR focus on animated avatars. In this demo, we present our efforts towards Social VR services based on photo-realistic video recordings. In this demo paper, we focus on two parts, the communication between multiple people (max 3) and the integration of new media formats to represent users as 3D point clouds. We enhance a green screen (chroma key) like cut-out of the person with depth data, allowing point cloud based rendering in the client. Further, the paper presents a user study with 54 people evaluating a three-people communication use case and a technical analysis to move towards 3D representations of users. This demo consists of two shared virtual environments to communicate and interact with others, i.e. i) a 360-degree virtual space with users being represented as 2D video streams (with the background removed) and ii) a 3D space with users being represented as point clouds (based on color and depth video data).