Mario Montagud

Inter-Destination Multimedia Synchronization: Schemes, Use Cases and Standardization

Traditionally, the media consumption model has been a passive and isolated activity. However, the advent of media streaming technologies, interactive social applications, and synchronous communications, as well as the convergence between these three developments, point to an evolution towards dynamic shared media experiences. In this new model, geographically distributed groups of consumers, independently of their location and the nature of their end-devices, can be immersed in a common virtual networked environment in which they can share multimedia services, interact and collaborate in real-time within the context of simultaneous media content consumption. In most of these multimedia services and applications, apart from the well-known intra and inter-stream synchronization techniques that are important inside the consumers’ playout devices, also the synchronization of the playout processes between several distributed receivers, known as multipoint, group or Inter-destination multimedia synchronization (IDMS), becomes essential. Due to the increasing popularity of social networking, this type of multimedia synchronization has gained in popularity in recent years. Although Social TV is perhaps the most prominent use case in which IDMS is useful, in this paper we present up to 19 use cases for IDMS, each one having its own synchronization requirements. Different approaches used in the (recent) past by researchers to achieve IDMS are described and compared. As further proof of the significance of IDMS nowadays, relevant organizations’ (such as ETSI TISPAN and IETF AVTCORE Group) efforts on IDMS standardization (in which authors have been and are participating actively), defining architectures and protocols, are summarized.

Enhanced adaptive RTCP-based Inter-Destination Multimedia Synchronization approach for distributed applications

Newer social multimedia applications, such as Social TV or networked multi-player games, enable independent groups (or clusters) of users to interact among themselves and share services within the context of simultaneous media content consumption. In such scenarios, concurrently synchronized playout points must be ensured so as not to degrade the user experience on such interaction. We refer to this process as Inter-Destination Multimedia Synchronization (IDMS). This paper presents the design, implementation and evaluation of an evolved version of an RTCP-based IDMS approach, including an Adaptive Media Playout (AMP) scheme that aims to dynamically and smoothly adjust the playout timing of each one of the geographically distributed consumers in a specific cluster if an allowable asynchrony threshold between their playout states is exceeded. For that purpose, we previously had also to develop a full implementation of RTP/RTCP protocols for NS-2, in which we included the IDMS approach as an optional functionality. Simulation results prove the feasibility of such IDMS and AMP proposals, by adopting several dynamic master reference selection policies, to maintain an overall synchronization status (within allowable limits) in each cluster of participants, while minimizing the occurrence of long-term playout discontinuities (such as skips/pauses) which are subjectively more annoying and less tolerable to users than small variations in the media playout rate.

On the Use of Adaptive Media Playout for Inter-Destination Synchronization

Inter-Destination Multimedia Synchronization (IDMS) is essential in most of the emerging social multimedia applications. In this paper we present a novel Adaptive Media Playout (AMP) scheme that aims to acquire an overall synchronization status between distributed receivers by means of smoothly adjusting their playout timing. Simulation results show that the proposed solution minimizes long-term playout discontinuities (skips and/or pauses) which are subjectively more annoying to users than small variations in the media playout rate.

The need for inter-destination synchronization for emerging social interactive multimedia applications

Currently, the media consumption paradigm is changing from a single end user to a group shared experience. Now, social communication opportunities may be exploited (e.g., conferencing while watching television), facing lots of technological (e.g., synchronization, universal session handling, scalability) and perceptual (e.g., presence awareness, QoE) challenges. This article focuses on one of these major challenges ahead in new emerging social interactive multimedia applications, which is the synchronization of different media streams across multiple locations, known as inter-destination multimedia synchronization (IDMS). We describe the three kinds of temporal multimedia synchronization, and summarize some related work and examples of applications in which IDMS is needed. The article includes a discussion about an RTP/RTCP-based IDMS solution the authors have been working on, as well as the standardization status regarding this kind of synchronization.

Understanding Timelines Within MPEG Standards

Nowadays, media content can be delivered via diverse broadband and broadcast technologies. Although these different technologies have somehow become rivals, their coordinated usage and convergence, by leveraging of their strengths and complementary characteristics, can bring many benefits to both operators and customers. For example, broadcast TV content can be augmented by on-demand broadband media content to provide enriched and personalized services, such as multi-view TV, audio language selection, and inclusion of real-time web feeds. A piece of evidence is the recent Hybrid Broadcast Broadband TV (HbbTV) standard, which aims at harmonizing the delivery and consumption of (hybrid) broadcast and broadband TV content. A key challenge in these emerging scenarios is the synchronization between the involved media streams, which can be originated by the same or different sources, and delivered via the same or different technologies. To enable synchronized (hybrid) media delivery services, some mechanisms providing timelines at the source side are necessary to accurately time align the involved media streams at the receiver-side. This paper provides a comprehensive review of how clock references (timing) and timestamps (time) are conveyed and interpreted when using the most widespread delivery technologies, such as DVB, RTP/RTCP and MPEG standards (e.g., MPEG-2, MPEG-4, MPEG-DASH, and MMT). It is particularly focused on the format, resolution, frequency, and the position within the bitstream of the fields conveying timing information, as well as on the involved components and packetization aspects. Finally, it provides a survey of proofs of concepts making use of these synchronization related mechanisms. This complete and thorough source of information can be very useful for scholars and practitioners interested in media services with synchronization demands.

Design and Simulation of a Distributed Control Scheme for Inter-destination Media Synchronization

Inter-Destination Multimedia Synchronization(IDMS) is a key requirement to enable satisfying group shared media experiences. This paper presents the design and simulation of a Distributed Control Scheme (DCS) for IDMS. In this scheme, geographically dispersed receivers exchange(multicast) RTCP reports on their arrival and play out timing for RTP packets, and each one of them uses the collected reports from the other receivers belonging to the same logical group (or cluster) to adaptively adjust its local play out timing such that an overall synchronization status (within allowable limits) is maintained. Also, the strengths (interactivity, flexibility, scalability, and robustness) and weaknesses (traffic overhead, security) of a DCS for IDMS are discussed, in comparison with centralized approaches. Moreover, a simple technique to enhance the performance in terms of coherence when using the DCS for IDMS is proposed. Simulation results prove the effectiveness of the proposed control scheme and techniques to guarantee concurrently synchronized play outpoints for independent clusters of receivers.

Master Selection Policies for Inter-destination Multimedia Synchronization in Distributed Applications

This paper presents an enhanced version of a proposed extension of the RTP/RTCP standard protocols to acquire Inter-Destination Multimedia Synchronization (IDMS) in one-way distributed applications. Several dynamic policies for selecting the master reference play out point to synchronize are introduced and their suitability is analyzed, for specific network conditions and application requirements, according to the impact of the play out adjustments (such as skips and/or pauses) on the overall quality and the buffer fullness variation. Simulation results prove the ability of our approach, for all the proposed master selection algorithms, to maintain the play out asynchrony between distributed receivers within acceptable limits, by dynamically adjusting their play out timing every time an allowed play out time discrepancy between their play out states is detected.

Multimedia group synchronization approach for one-way cluster-to-cluster applications

Nowadays, many multimedia cluster-to-cluster applications exist, such as 3D tele-immersion (3DTI), computer-supported collaborative workspaces (CSCWs), distributed multimedia presentations (DMP)…. All these applications have sophisticated data transport requirements due to the use of multiple, semantically related flows of information. A synchronization mechanism must be used to synchronize the playout of the streams, regardless of the number of receivers and the number of streams played on the receiver clusters. In this paper, we present a new solution for multimedia group synchronization in such applications. Rather than base the solution on the definition of a new synchronization protocol (as other authors do), we base it on small modifications or extensions to RTP/RTCP standard protocols already used in most multimedia applications. Due to this, the overload introduced by the approach is minimal. The suitability of the approach was evaluated in a real one-way cluster-to-cluster application, with satisfactory results.

Early event-driven (EED) RTCP feedback for rapid IDMS

Inter-Destination Media Synchronization (IDMS) is essential in the emerging media consumption paradigm, which is radically evolving from passive and isolated services towards dynamic and interactive group shared experiences. This paper concentrates on improving a standardized RTP/RTCP-based solution for IDMS. In particular, novel Early Event-Driven (EED) RTCP feedback reporting mechanisms are designed to overcome latency issues and to enable higher flexibility, dynamism and accuracy when using RTP/RTCP for IDMS. The faster reaction on dynamic situations (e.g., detection of asynchrony or channel change delays) and a finer granularity for synchronizing media-related events, while preserving the RTCP bandwidth bounds, are validated through simulation tests.

Design, development and assessment of control schemes for IDMS in a standardized RTCP-based solution

Currently, several media sharing applications that allow social interactions between distributed users are gaining momentum. In these networked scenarios, synchronized playout between the involved participants must be provided to enable truly interactive and coherent shared media experiences. This research topic is known as Inter-Destination Media Synchronization (IDMS). This paper presents the design and development of an advanced IDMS solution, which is based on extending the capabilities of RTP/RTCP standard protocols. Particularly, novel RTCP extensions, in combination with several control algorithms and adjustment techniques, have been specified to enable an adaptive, highly accurate and standard compliant IDMS solution. Moreover, as different control or architectural schemes for IDMS exist, and each one is best suited for specific use cases, the IDMS solution has been extended to be able to adopt each one of them. Simulation results prove the satisfactory responsiveness of our IDMS solution in a small scale scenario, as well as its consistent behavior, when using each one of the deployed architectural schemes.