Roberto Gerson de Albuquerque Azevedo

Composer: meeting non-functional aspects of hypermedia authoring environment

This paper discusses the importance of non-functional requirements in the design of hypermedia authoring tools, which typically provides multiple graphical abstractions (views). It focuses on creating products and services that operate robustly across a broad range of environments, and that take into account the changeable needs of their users over time, as they become more familiar with the tool. In order to meet these non-functional aspects, this paper proposes a microkernel-based architecture for authoring tools, where the microkernel is responsible for instantiating the requested extensions (plugins), maintaining the core data model that represents the hypermedia document under development, and notifying changes in this model to plugins interested in them. Based on the proposed architecture, a new version of Composer (an NCL authoring tool) is presented, rewritten from scratch. Results from experiments show that the discussed non-functional requirements are adequately met.

Textual authoring of interactive digital TV applications

Authoring tools for hypermedia languages usually provide visual abstractions, which hide the source code from the author aiming to simplify and accelerate the development process. Among other drawbacks, these abstractions modify or even break the communication process between the author and the language designer, since these languages were designed to be readable and understandable by its target audience. This paper presents a textual approach to hypermedia authoring that does not have these inconveniences, but rather uses typographical accessories, such as program visualization, hypertextual navigation, and semi-automatic error correction. The proposed approach exploits concepts known to the author and does not imply in extra cognitive overload. A use case is presented, namely the NCL Eclipse authoring environment, for Nested Context Language, the Brazilian Digital TV and ITU-T standard.

Embedding 3D objects into NCL multimedia presentations

This paper discusses how 3D objects, in special X3D documents, can be embedded into Digital TV middleware, aiming at providing 3D interactive content for both IPTV and terrestrial DTV systems. Particularly, the paper focuses on embedding 3D objects into Ginga-NCL, the declarative environment of the Japanese-Brazilian ISDB-TB terrestrial DTV middleware, and ITU-T Recommendation for IPTV services. Thus, we propose a well-defined interface that allows 3D objects to be declaratively embedded into NCL -- the declarative language of Ginga-NCL -- multimedia applications. Moreover, the proposed solution will allow for 3D objects to take advantage of the high-level abstractions of NCL to specify temporal behavior of 3D scenes, and contributes to the convergence process between different multimedia (2D and 3D) technologies.

Extending multimedia languages to support multimodal user interactions

Historically, the Multimedia community research has focused on output modalities, through studies on timing and multimedia processing. The Multimodal Interaction community, on the other hand, has focused on user-generated modalities, through studies on Multimodal User Interfaces (MUI). In this paper, aiming to assist the development of multimedia applications with MUIs, we propose the integration of concepts from those two communities in a unique high-level programming framework. The framework integrates user modalities —both user-generated (e.g., speech, gestures) and user-consumed (e.g., audiovisual, haptic)— in declarative programming languages for the specification of interactive multimedia applications. To illustrate our approach, we instantiate the framework in the NCL (Nested Context Language) multimedia language. NCL is the declarative language for developing interactive applications for Brazilian Digital TV and an ITU-T Recommendation for IPTV services. To help evaluate our approach, we discuss a usage scenario and implement it as an NCL application extended with the proposed multimodal features. Also, we compare the expressiveness of the multimodal NCL against existing multimedia and multimodal languages, for both input and output modalities.

Reducing the complexity of NCL player implementations

In this paper, we present an approach for reducing the complexity of NCL player implementations. This approach consists, basically, in introducing in the players architecture an initial conversion step that removes all syntactic sugar and reuse features from the source language. The output of this step, a redundancy-free version of the original input, is then fed to the player that interprets it and creates a corresponding multimedia presentation. In particular, we propose the use of the NCL Raw profile as this intermediate language. The Raw profile is an (almost) redundancy-free profile that is compatible with the NCL 3.0 EDTV (Enhanced Digital TV) profile, a property that guarantees a seamless integration with current EDTV profile implementations. The main targets of the proposed approach are NCL players running on HTML browsers. We discuss how the solutions presented by NCL4Web, WebNCL, and Ginga Plug-in can be tuned to overcome some problems pointed their authors. The same problems arise in similar contexts for other declarative languages, e.g., SMIL, and the solutions presented here can also be extended to those systems.

An approach for textual authoring of hypermedia documents based on the use of programmatic visualization and hypertextual navigation

Authoring tools for hypermedia languages often provide visual abstractions that hide the source code from the author with the objective of simplifying and accelerating the development process. Among other drawbacks, these abstractions modify or even prevent the communication process between the author and the designer of the language, which has been designed to be streamlined and readable by its target audience. This paper presents a textual approach to hypermedia authoring that does not inherit these inconveniences, but also makes use of typographical accessories such as programmatic views and hypertextual navigation. By setting an environment for hypermedia authoring similar to the presentation engine of the source language, the proposed approach uses concepts known by the author and does not imply in cognitive overload. A case of use is exemplified by the NCL Eclipse authoring environment, toward for Nested Context Language, the standard for the Brazilian Digital TV System.

Converting NCL Documents to Smix and Fixing Their Semantics and Interpretation in the Process

In this paper, we present the conversion of NCL to Smix and discuss its main implications. NCL is a declarative language for the specification of interactive multimedia presentations which was adopted by the ITU-T H.761 recommendation for interoperable IPTV services. Smix is a synchronous domain-specific language with a similar purpose, but with a simpler and more precise semantics. By implementing NCL over Smix, we bring to the former the notions of reaction and execution instants, and with them some benefits. From a practical perspective, we fix the semantics of the converted documents, get a leaner NCL player (the Smix interpreter), and simplify further conversions. From a systems-design perspective, the structured conversion of NCL to Smix helps us tame the complexity of mapping the user-oriented constructs of NCL into the machine-oriented primitives that realize them as a multimedia presentation. In the paper, we present NCL and Smix, discuss related work on document conversion, and detail the conversion process and a prototype implementation.

Real-Time Depth-Image-Based Rendering for 3DTV Using OpenCL

This paper proposes a real-time (performance of at least 30 fps for full-HD video) Depth-Image-based Rendering (DIBR) approach for stereoscopic 3DTV using OpenCL. Many stereoscopic 3DTV, multi-view, and Free-view-point TV (FTV) technologies have been based on DIBR, aiming at saving bandwidth, and to allow for user adaptation in the client-side. Unlike related work, this paper uses OpenCL for all the DIBR steps, including the re-projection in the virtual views (which is commonly performed using OpenGL, even when implemented in GPGPUs). The use of OpenCL-only can, in some cases, outperform the OpenGL z-testing performance. Two execution models have been implemented (per-line parallel, and per-pixel parallel) and tested against standard video-plus-depth test sequences to show the approach performance.

Extending NCL to Support Multiuser and Multimodal Interactions

Recent advances in technologies for speech, touch and gesture recognition have given rise to a new class of user interfaces that does not only explore multiple modalities but also allows for multiple interacting users. Even so, current declarative multimedia languages e.g. HTML, SMIL, and NCL?support only limited forms of user input (mainly keyboard and mouse) for a single user. In this paper, we aim at studying how the NCL multimedia language could take advantage of those new recognition technologies. To do so, we revisit the model behind NCL, named NCM (Nested Context Model), and extend it with first-class concepts supporting multiuser and multimodal features. To evaluate our approach, we instantiate the proposal and discuss some usage scenarios, developed as NCL applications with our extended features.

Specification of Multimodal Interactions in NCL

This paper proposes an approach to integrate multimodal events--both user-generated, e.g., audio recognizer, motion sensors; and user-consumed, e.g., speech synthesizer, haptic synthesizer--into programming languages for the declarative specification of multimedia applications. More precisely, it presents extensions to the NCL (Nested Context Language) multimedia language. NCL is the standard declarative language for the development of interactive applications for Brazilian Digital TV and an ITU-T Recommendation for IPTV services. NCL applications extended with the multimodal features are presented as results. Historically, Human-Computer Interaction research community has been focusing on user-generated modalities, through studies on the user interaction. On the other hand, Multimedia community has been focusing on output modalities, through studies on timing and multimedia processing. The proposals in this paper is an attempt to integrate concepts of both research communities in a unique high-level programming framework, which aims to assist the authoring of multimedia/multimodal applications.