Marcelo Ferreira Moreno

Ginga-NCL: Declarative middleware for multimedia IPTV services

This article presents the innovative features of Ginga-NCL, an open middleware specification for multimedia IPTV services. Ginga-NCL relies on the Nested Context Language, a domain-specific declarative language targeting multimedia application authoring. As a glue language, NCL relates media objects in time and space without restricting or imposing any media content type, including media objects with imperative and declarative code written using other languages. Other NCL features include support for multidevice presentations, content adaptations, presentation adaptations, and advanced code reuse. Ginga-NCL allows NCL applications to be modified on the fly by means of live editing commands. Initially defined as the standard middleware for the Brazilian terrestrial DTV system, Ginga-NCL has recently become part of ISDB standards and an ITU-T Recommendation for IPTV services.

Intermedia synchronization management in DTV systems

Intermedia synchronization is related with spatial and temporal relationships among media objects that compound a DTV application. From the server side (usually a broadcasters server or a Web Server) to receivers, end-to-end intermedia synchronization support must be provided. Based on application specifications, several abstract data structures should be created to guide all synchronization control processes. A special data structure, a labeled digraph called HTG (Hypermedia Temporal Graph) is proposed in this paper as the basis of all other data structures. From HTG, receivers derive a presentation plan to orchestrate media content presentations that make up a DTV application. From this plan other data structures are derived to estimate when media players should be instantiated and when data contents should be retrieved from a DSM-CC carousel or from a return channel. If the return channel provides QoS support, another data structure is derived from the presentation plan, in order to determine when resource reservation should take place. For content pushed by broadcasters, HTG is used in the server side as the basis for building the carousel plan, a data structure that guides the order and frequency that media objects should be broadcasted. The papers proposals were partially put into practice in the current open source reference implementation of the standard middleware of the Brazilian Terrestrial Digital TV System. However, this reference implementation is used just as a proof of concept. The ideas presented can be extended to any multimedia document presentation player (user agent) and content distribution server.

Multiple exhibition devices in DTV systems

Nested Context Language (NCL) is the declarative language of the Brazilian Terrestrial Digital TV System. NCL is part of ISDB (International Standard for Digital Broadcasting) standards and also the ITU-T Recommendation H.761 for IPTV services. This paper presents, discusses, and illustrates the NCL hierarchical control model for multiple exhibition device support. Based on this model, multiple devices are orchestrated to run a DTV application in cooperation. Two types of multiple device exhibitions are distinguished. Those where the same content is shown in a set of devices under a unique control, and those where content is under each individual device control, working completely independent. In this last case, depending on viewer interactions, the resulting presented content can differ from a device to another. Examples of NCL applications using both options are presented and discussed.

Composer: meeting non-functional aspects of hypermedia authoring environment

This paper discusses the importance of non-functional requirements in the design of hypermedia authoring tools, which typically provides multiple graphical abstractions (views). It focuses on creating products and services that operate robustly across a broad range of environments, and that take into account the changeable needs of their users over time, as they become more familiar with the tool. In order to meet these non-functional aspects, this paper proposes a microkernel-based architecture for authoring tools, where the microkernel is responsible for instantiating the requested extensions (plugins), maintaining the core data model that represents the hypermedia document under development, and notifying changes in this model to plugins interested in them. Based on the proposed architecture, a new version of Composer (an NCL authoring tool) is presented, rewritten from scratch. Results from experiments show that the discussed non-functional requirements are adequately met.

Relating declarative hypermedia objects and imperative objects through the NCL glue language

This paper focuses on the support provided by NCL (Nested Context Language) to relate objects with imperative code content and declarative hypermedia-objects (objects with declarative code content specifying hypermedia documents). NCL is the declarative language of the Brazilian Terrestrial Digital TV System (SBTVD) supported by its middleware called Ginga. NCL and Ginga are part of ISDB standards and also of ITU-T Recommendations for IPTV services. The main contribution of this paper is the seamless way NCL integrates imperative and declarative language paradigms with no intrusion, maintaining a clear limit between embedded objects, independent of their coding content, and defining a behavior model that avoids side effects from one paradigm use to another.

Deepening the separation of concerns in the implementation of multimedia systems

The presentation of multimedia content requires devices with proper computational resources to guarantee intermedia synchronization requirements. When a multimedia system is executed in resource-constrained devices, some synchronization mismatches may occur. This problem becomes harder to solve when considering support for features like user interaction, systemic resilience, and distributed multimedia processing. This paper discusses the separation of concerns between multimedia presentation orchestration and multimedia content decoding and rendering. A multimedia system architecture and its implementation are also presented in this paper, where the separation of concerns goes beyond modularization by being deeply realized as process isolation. This design decision not only assures improved system resilience to the proposed solution but also enables it to be applied in distributed multimedia processing/presentations. Experiments carried out have indicated that the presentation orchestration does not require much resource, and may be run on lightweight devices. On the other hand, multimedia decoding/rendering is suitable to run on devices that are more resourceful.

Towards the convergence of digital TV systems

To allow producing digital TV applications independently from receiver’s hardware and operating system, and also to provide better support to application designs, middleware layer is introduced in digital TV system architectures. At first, middleware systems were developed aiming at specific transport platforms (IPTV, terrestrial DTV, etc.), offering support to services specifically designed for those platforms. However, the next generation of digital TV pulls all TV services present in all current platforms together into a single core of distributed services, as a result of the transport platforms convergence. In this hybrid TV, transport systems shall be concealed by the middleware to applications, as other operating system and hardware resources are hidden.This paper emphasizes the middleware natural role as key technology for this upcoming convergent digital TV, raising some requirements to be committed. NCL and Ginga-NCL features—technologies recommended by ITU-T for IPTV services, and ISDB standards for terrestrial DTV—are used as examples of some proposed solutions, as well as to illustrate some issues which deserve future research attention and new better results.

STorM: A Hypermedia Authoring Model for Interactive Digital Out-of-Home Media

Among the several vehicles of social communication, digital signage displays have been playing a remarkable role in both public and private spaces. Such Digital Out-of-Home (DOOH) media allows for the rapid dissemination of collective information to a large number of people. It is observed, however, that there is a large distance between the graphical abstractions offered by DOOH authoring tools and the underlying language for the representation of hyper/multimedia content. Document representation becomes complex, sometimes makes use of scripting languages, and therefore is illegible by authors and even difficult for automated information extraction. In this context, this paper proposes STorM, a hypermedia model and its language STorML that defines higher-level entities related to the concepts found in the audiovisual industry, such as scenes, tracks and media.

Hypermedia Content Transmission Plan: Managing the Broadcast/Multicast Delivery

Nowadays hypermedia content may be delivered over different platforms, such as Terrestrial DTV, IPTV and the Web. Therefore hypermedia presentation engines must be designed taking into consideration the specificities of their supported networks to provide the expected QoE levels. Presentation engines employ some content analysis to help on the task of maintaining QoE. However, when the content includes pushed data, this analysis should be transferred to the server side for building the so-called Transmission Plan. The Transmission Plan is a data structure that predicts the time when media objects should be transmitted and for how long, in order to optimize end-to-end resource usage. This work proposes a framework for managing the push-mode delivery of hypermedia content. The framework is generic enough to be instantiated over different content delivery scenarios. Introductory instantiation scenarios and initial results are also discussed.

Interaction and audience analysis in interactive digital TV systems

This paper proposes and compares two approaches for data capture to support the analysis of user interaction and audience measurement in Interactive digital TV (IDTV) systems. The first approach relies on the development of extensions to existent standards and the second one depends only on resources commonly available in IDTV systems. Tools for audience and interaction analysis are also presented, which, independently from the approach chosen, allow for the handling of data originated from the interaction between users and their digital receivers.