Erick Lazaro Melo

WebNCL: a web-based presentation machine for multimedia documents

Presentation machines for multimedia declarative languages especially the ones related with Interactive Digital TV (iDTV) and Internet Protocol TV (IPTV) are usually embedded in devices and strongly coupled with the platforms when native code and API for the devices platform are used. Since much of the complexity to implement presentation machines lies on presenting and controlling different types of media (video, audio, image, text), and given that most of the modern browsers natively support those requirements, it becomes interesting to implement presentation machines using Web technologies to reduce their coupling with platforms. In this paper we discuss the advantages of a presentation machine for declarative multimedia languages implemented on top of Web technologies. As a proof of concept we implemented the WebNCL, a lightweight NCL presentation machine based on the web technologies stack (HTML 5/ JavaScript/ CSS). By using WebNCL, NCL documents can be presented in any device that has a HTML5 compatible browser, such as tablets, smartphones, smart TVs and PCs.

User-media interaction with interactive TV

Watching TV is a practice many people enjoy and feel comfortable with. We propose the capture of the user interaction while interacting with a remote control to watch TV: such detailed information is most valuable to many applications and services. We discuss our proposed approach in the context of the Brazilian Interactive Digital TV platform.

A multimodal interaction component for digital television

In most current digital TV applications the user interaction takes place by pressing keys on a remote control. For simple applications this type of interaction is sufficient --- however, as interactive applications become more popular new input devices are demanded. After discussing motivating scenarios, this paper presents an architecture that offers to applications running on a set-top-box the possibility of receiving multimodal data (audio, video, image, ink, accelerometer, text, voice and customized data) from multiple devices (such as mobile phones, PDAs, tablet PCs, notebooks or even desktops). We validated the architecture by implementing a corresponding multimodal interaction component which extends the Brazilian Digital TV middleware, and by building applications which use the component.

Taking advantage of contextualized interactions while users watch TV

While watching TV, viewers use the remote control to turn the TV set on and off, change channel and volume, to adjust the image and audio settings, etc. Worldwide, research institutes collect information about audience measurement, which can also be used to provide personalization and recommendation services, among others. The interactive digital TV offers viewers the opportunity to interact with interactive applications associated with the broadcast program. Interactive TV infrastructure supports the capture of the user–TV interaction at fine-grained levels. In this paper we propose the capture of all the user interaction with a TV remote control—including short term and instant interactions: we argue that the corresponding captured information can be used to create content pervasively and automatically, and that this content can be used by a wide variety of services, such as audience measurement, personalization and recommendation services. The capture of fine grained data about instant and interval-based interactions also allows the underlying infrastructure to offer services at the same scale, such as annotation services and adaptative applications. We present the main modules of an infrastructure for TV-based services, along with a detailed example of a document used to record the user–remote control interaction. Our approach is evaluated by means of a proof-of-concept prototype which uses the Brazilian Digital TV System, the Ginga-NCL middleware.

End-user live editing of iTV programmes

Watching TV is a practice many people enjoy and feel comfortable with. While watching a TV programme, users can be offered the opportunity to, while making annotations, create their own edited versions of the programme. In this scenario, it is a challenge to allow the user to add comments in a ubiquitous, transparent way. In this paper, we exploit the concept of end-user live editing of interactive video programmes by detailing an environment where users are able to live edit a video using the iTV remote control. We contextualise our approach in the context of the Brazilian Interactive Digital TV platform.

Multimedia multi-device educational presentations preserved as interactive multi-video objects

The capture of lectures or similar presentations is of interest for several reasons. From the attendees perspective, students may use the recordings when working on homework assignments or preparing for exams, or to watch the contents of a missed class. From the instructors perspective, a captured lecture may be evaluated, recaptured for improvements, or reused as complementary learning material. Moreover, captured lectures may be a valuable resource for e-learning and distance education courses. In this paper we detail the design rationale associated with the development of a prototype platform for the ubiquitous capture of live presentations and their transformation into a corresponding interactive multi-video object. Our approach includes capturing important context information which, when incorporated into the multimedia object, enables one to interact with the recorded lecture in novel dimensions. We tested our prototype by using case studies involving instructors and students, which allowed us to identify important features and novel uses for the platform.

Multimedia presentation integrating interactive media produced in real time with high performance processing

Sophisticated interactive animation may be an interesting element to compose a multimedia document. However, potential demand for high performance computing can make this option impractical in digital interactive TV and mobile device environments, due to computing power restrictions of these platforms. In previous work we proposed a solution to overcome such restrictions based on video streaming. Moving forward the solution, it is described in this paper how to take advantage of media-agnostic characteristic, when present in the multimedia presentation machine, to manage this new type of media in multimedia documents. As a proof of concept we extended and tested an NCL presentation engine to add suport to this new type of media.

Discrimination of media moments and media intervals: sticker-based watch-and-comment annotation

In this paper we discuss the problem of how to discriminate moments of interest on videos or live broadcast shows. The primary contribution is a system which allows users to personalize their programs with previously created media stickers—pieces of content that may be temporarily attached to the original video. We present the system’s architecture and implementation, which offer users operators to transparently annotate videos while watching them. We offered a soccer fan the opportunity to add stickers to the video while watching a live match: the user reported both enjoying and being comfortable using the stickers during the match—relevant results even though the experience was not fully representative.

Go beyond boundaries of iTV applications

The development of multimedia applications that require the manipulation and the synchronization of multiple media and the handling of different types of user interactions usually requires specialized knowledge in imperative languages. Declarative languages have been proposed in order to make this task easier, especially when applications are restricted to certain classes, as it is the case of Interactive TV applications in which user interactions are restricted to a few simple models. However, those simple models may be too simple when documents are reused in other platforms: for instance, when watching a video most web users expect an interactive timeline to be available --- which is not the case in interactive TV videos. This paper presents a component-based approach to the enrichment of declarative languages for multimedia so that desirable user-media interactions are made possible at the same time that the original ease of authoring is maintained. We detail the components and present a corresponding proof-of-concept prototype. We also discuss design decisions associated with the development of the components, which should be useful in further extensions.

Ubiquitous end-user live editing of interactive multimedia programs

Watching TV is a practice many people enjoy and feel comfortable with. In the context of interactive TV, a program is defined by means of a structured multimedia document delivered to the viewers digital TV equipment. While watching TV, a user can be offered the opportunity to edit the program with explicit or implicit edit operations. In this paper we exploit the concept of end-user live editing of interactive video programs by proposing an environment where such a live editing can take place. We contextualize our proposed approach in the context of the Brazilian Interactive Digital TV platform.