Rodrigo Costa Mesquita Santos

Textual authoring of interactive digital TV applications

Authoring tools for hypermedia languages usually provide visual abstractions, which hide the source code from the author aiming to simplify and accelerate the development process. Among other drawbacks, these abstractions modify or even break the communication process between the author and the language designer, since these languages were designed to be readable and understandable by its target audience. This paper presents a textual approach to hypermedia authoring that does not have these inconveniences, but rather uses typographical accessories, such as program visualization, hypertextual navigation, and semi-automatic error correction. The proposed approach exploits concepts known to the author and does not imply in extra cognitive overload. A use case is presented, namely the NCL Eclipse authoring environment, for Nested Context Language, the Brazilian Digital TV and ITU-T standard.

CÉU-MEDIA: Local Inter-Media Synchronization using CÉU

The semantics of current multimedia languages is informal and may lead to the development of ambiguous applications. In this paper we investigate the use of the synchronous language Céu for programming local multimedia applications, in particular, those applications that can be described as a set of synchronized media objects. Céu has a deterministic, concise and accurate semantics which, along with high-level programming constructs, makes the language an attractive alternative for developing multimedia applications. We also present Céu-Media, a library for programming multimedia in Céu. Céu-Media implementation guarantees that the properties of the Céu synchronous semantics are reflected in the multimedia presentation, ensuring inter-media synchronization. We compare the synchronization paradigm of Céu with those of NCL and SMIL, and examine the implementation of representative use cases.

Deepening the separation of concerns in the implementation of multimedia systems

The presentation of multimedia content requires devices with proper computational resources to guarantee intermedia synchronization requirements. When a multimedia system is executed in resource-constrained devices, some synchronization mismatches may occur. This problem becomes harder to solve when considering support for features like user interaction, systemic resilience, and distributed multimedia processing. This paper discusses the separation of concerns between multimedia presentation orchestration and multimedia content decoding and rendering. A multimedia system architecture and its implementation are also presented in this paper, where the separation of concerns goes beyond modularization by being deeply realized as process isolation. This design decision not only assures improved system resilience to the proposed solution but also enables it to be applied in distributed multimedia processing/presentations. Experiments carried out have indicated that the presentation orchestration does not require much resource, and may be run on lightweight devices. On the other hand, multimedia decoding/rendering is suitable to run on devices that are more resourceful.

An architecture to assist multimedia application authors and presentation engine developers

This paper presents an architecture for monitoring the presentation of multimedia declarative applications, providing feedback about variables states, object properties, media presentation times, among others. Monitoring tools that follow the proposed architecture are able to detect if visual problems are being caused by programming errors or by player malfunctioning. The architecture presents a communication protocol designed to be independent of the declarative language used in the development of multimedia applications. The main goal is to provide an open and generic architecture that can assist multimedia application authors and presentation engine developers. As an example of the architecture use, the paper also presents a monitoring tool integrated into a graphical user interface developed for the ITU-T reference implementation of the Ginga-NCL middleware.

Eventline: representation of the temporal behavior of multimedia applications

This paper presents the concept of eventline as a visual representation proposal for the temporal behavior of multimedia applications. The main difference of this proposal when compared to the timeline paradigm is the replacement of the temporal continuous axis to a temporal discrete axis based on the occurrence of events. Media synchronization in each eventline is based on the relative time of a main media. User interaction is expressed as the navigation between multiple eventlines. The objective of this work is to provide a simplified alternative for authoring of interactive multimedia applications.

Multimedia authoring based on templates and semi-automatic generated wizards

Templates have been used to engage non-expert multimedia authors as content producers. In template-based authoring, templates with most of the relevant application logic and application constraints are developed by experts, who must also specify the template semantics, report which are the required gaps to be filled in, and how to do so. Filling templates gaps is the single task left to inexperienced users to produce the final applications. To do that, they usually must understand the padding instructions reported by template authors and learn some specific padding language. An alternative is using specific GUI components created specifically to each new developed template. This paper proposes a semi-automatic generation of GUI Wizards to guide end(-user) authors to create multimedia applications. The wizard can be tuned to improve the communication between the template author and the template end user, and also if the template specification is not complete. Many successful trial cases show that the generated wizards are usually simple enough to be used by non-experts. The contributions coming from this paper is not constrained to any specific template language or final-application format. Nevertheless, aiming at testing the proposal it was instantiated to work with TAL (Template Authoring Language) whose template processors can generate applications in different target languages.

Incremental validation of digital TV applications in nested context language

This paper presents an incremental validation process for multimedia documents describing interactive digital TV (iTV) applications. This work delves more specifically into documents defined in Nested Context Language (NCL), a declarative language standard adopted by ISDB-Tb and ITU-T for IPTV. The focus of incremental validation, the approach proposed in this work, relies on checking only the pieces of code that have been modified since the latest validation as well as those parts that have suffered any side effects, thus eliminating the need for a thorough checking of the document. The implications of performing this type of validation in a declarative language such as NCL are discussed and implementation details are given. Performance tests comparing the standard and incremental validation approaches have been carried out revealing a significant improvement concerning the latter.

A memory-bounded, deterministic and terminating semantics for the synchronous programming language Céu

Céu is a synchronous programming language for embedded soft real-time systems. It focuses on control-flow safety features, such as safe shared-memory concurrency and safe abortion of lines of execution, while enforcing memory bounded, deterministic, and terminating reactions to the environment. In this work, we present a small-step structural operational semantics for Céu and a proof that reactions have the properties enumerated above: that for a given arbitrary timeline of input events, multiple executions of the same program always react in bounded time and arrive at the same final finite memory state.

Cognitive Computing Systems in Reactive Multimedia Applications Developed in CÉU

The integration of multimedia frameworks and APIs for accessing cognitive computing systems into a unified programming model may not be straightforward. Current solutions use programming models that lead to problems like callback hell or synchronization of threads. In this paper, we explore the use of the reactive programming paradigm to create multimedia applications that explore cognitive computing systems features. By using CÉU as instance of reactive language, we discuss how this approach promotes a straightforward integration between these domains, while avoiding the aforementioned issues. We discuss the implementation of two case studies to illustrate the advantages of this approach.

Catalog of Templates: An Approach for Documenting Families of Hypermedia Documents

Two actors are identified in template-oriented authoring: the template author and the application author. Classifying hypermedia templates, aiming to support application authors in searching for suitable templates, in inferring on their semantic and in knowing how to fill them to generate applications is the focus of this paper. In doing so, it can be said that the proposed catalog somehow mediates the communication between template and application authors, favoring reuse and reducing the cognitive load; thereby facilitating application authoring. As an example of use, an indexing and storage apparatus, associated with a template authoring tool, and a search plug-in coupled with an application authoring tool are presented.