Dick C. A. Bulterman

Structured multimedia authoring

Authoring context sensitive, interactive multimedia presentations is much more complex than authoring either purely audiovisual applications or text. Interactions among media objects need to be described as a set of spatio-temporal relationships that account for synchronous and asynchronous interactions, as well as on-demand linking behavior. This article considers the issues that need to be addressed by an authoring environment. We begin with a partitioning of concerns based on seven classes of authoring problems. We then describe a selection of multimedia authoring environments within four different authoring paradigms: structured, timeline, graph and scripting. We next provide observations and insights into the authoring process and argue that the structured paradigm provides the most useful framework for presentation authoring. We close with an example application of the structured multimedia authoring paradigm in the context of our own structure-based system GRiNS.

Strategic directions in human-computer interaction

Human-computer interaction (HCI) is the study of how people design, implement, and use interactive computer systems and how computers affect individuals, organizations, and society. This encompasses not only ease of use but also new interaction techniques for supporting user tasks, providing better access to information, and creating more powerful forms of communication. It involves input and output devices and the interaction techniques that use them; how information is presented and requested; how the computer’s actions are controlled and monitored; all forms of help, documentation, and training; the tools used to design, build, test, and evaluate user interfaces; and the processes that developers follow when creating interfaces. This report describes the historical and intellectual foundations of HCI and then summarizes selected strategic directions in human-computer interaction research. Previous important reports on HCI directions include the results of the 1991 [Sibert and Marchionini 1993] and 1994 [Strong 1994] NSF studies on HCI in general, and the 1994 NSF study on the World-Wide Web [Foley and Pitkow 1994].

Usages of the Secondary Screen in an Interactive Television Environment: Control, Enrich, Share, and Transfer Television Content

This paper investigates a number of techniques and services around a unifying concept: the secondary screen. Far too often television consumption is considered a passive activity. While there are specific genres and programs that immerse the viewer into the media experience, there are other times in which whilst watching television, people talk, scan the program guide, record another program or recommend a program by phone. This paper identifies four major usages of the secondary screen in an interactive digital television environment: control, enrich, share, and transfer television content. By control we refer to the decoupling of the television stream, optional enhanced content, and television controls. Moreover, the user can use the secondary screen to enrich or author media content by, for example, including personalized media overlays such as an audio commentary that can be shared with his peer group. Finally, the secondary screen can be used to bring along the television content. This paper reviews previous work on the secondary screen, identifies the key usages, and based on a working system provides the experiences of developing relevant scenarios as well as an initial evaluation of them.

Enhancing social sharing of videos: fragment, annotate, enrich, and share

Media consumption is an inherently social activity, serving to communicate ideas and emotions across both small- and large-scale communities. The migration of the media experience to personal computers retains social viewing, but typically only via a non-social, strictly personal interface. This paper presents an architecture and implementation for media content selection, content (re)organization, and content sharing within a user community that is heterogeneous in terms of both participants and devices. In addition, our application allows the user to enrich the content as a differentiated personalization activity targeted to his/her peer-group. We describe the goals, architecture and implementation of our system in this paper. In order to validate our results, we also present results from two user studies involving disjoint sets of test participants.

Links in hypermedia: the requirement for context

Taking the concept of a link from hypertext and adding to it the rich collection of information formats found in multimedia systems provides an extension to hypertext that is often called hypermedia. Unfortunately, the implicit assumptions under which hypertext links work do not extend well to time-based presentations that consist of a number of simultaneously active media items. It is not obvious where links should lead and there are no standard rules that indicate what should happen to other parts of the presentation that are active. This paper addresses the problems associated with links in hypermedia. In order to provide a solution, we introduce the notion of context for the source and the destination of a link. A context makes explicit which part of a presentation is affected when a link is followed from an anchor in the presentation. Given explicit source and destination contexts for a link, an author is able to state the desired presentation characteristics for following a link, including whether the presentation currently playing should continue playing or be replaced. We first give an intuitive description of contexts for links, then present a structure-based approach. We go on to describe the implementation of contexts in our hypermedia authoring system CMIFed.

The implications of program genres for the design of social television systems

In this paper, we look at how television genres can play a role in the use of social interactive television systems (social iTV). Based on a user study of a system for sending and receiving enriched video fragments to and from a range of devices, we discuss which genres are preferred for talking while watching, talking about after watching and for sending to users with different devices. The results show that news, soap, quiz and sport are genres during which our participants talk most while watching and are thus suitable for synchronous social iTV systems. For asynchronous social iTV systems film, news, documentaries and music programs are potentially popular genres. The plot structure of certain genres influences if people are inclined to talk while watching or not, and to which device they would send a video fragment. We also discuss how this impacts the design and evaluation of social iTV systems.

Is it time for a moratorium on metadata

This work discusses the author suggestion for locating content in mixed-media for context-sensitive queries. Also provides examples of nontextual approach, a system for organizing digital photographs in which all of the instances of a particular person can be found based on face recognition rather than keyword matching, and compared with a conventional metadata effort that uses predictive labelling of objects in the base data set.

Fragment, tag, enrich, and send: Enhancing social sharing of video

The migration of media consumption to personal computers retains distributed social viewing, but only via nonsocial, strictly personal interfaces. This article presents an architecture, and implementation for media sharing that allows for enhanced social interactions among users. Using a mixed-device model, our work allows targeted, personalized enrichment of content. All recipients see common content, while differentiated content is delivered to individuals via their personal secondary screens. We describe the goals, architecture, and implementation of our system in this article. In order to validate our results, we also present results from two user studies involving disjoint sets of test participants.

Automatic generation of video narratives from shared UGC

This paper introduces an evaluated approach to the automatic generation of video narratives from user generated content gathered in a shared repository. In the context of social events, end-users record video material with their personal cameras and upload the content to a common repository. Video narrative techniques, implemented using Narrative Structure Language (NSL) and ShapeShifting Media, are employed to automatically generate movies recounting the event. Such movies are personalized according to the preferences expressed by each individual end-user, for each individual viewing. This paper describes our prototype narrative system, MyVideos, deployed as a web application, and reports on its evaluation for one specific use case: assembling stories of a school concert by parents, relatives and friends. The evaluations carried out through focus groups, interviews and field trials, in the Netherlands and UK, provided validating results and further insights into this approach.

Using SMIL to encode interactive, peer-level multimedia annotations

This paper discusses applying facilities in SMIL 2.0 to the problem of annotating multimedia presentations. Rather than viewing annotations as collections of (abstract) meta-informa-tion for use in indexing, retrieval or semantic processing, we view annotations as a set of peer-level content with temporal and spatial relationships that are important in presenting a coherent story to a user. The composite nature of the collection of media is essential to the nature of peer-level annotations: you would typically annotate a single media item much differently than that same media item in the context of a total presentation.This paper focuses on the document engineering aspects of the annotation system. We do not consider any particular user interface for creating the annotations or any back-end storage architecture to save/search the annotations. Instead, we focus on how annotations can be represented within a common document architecture and we consider means of providing docu-ment facilities that meet the requirements of our user model. We present our work in the context of a medical patient dossier example.