Stefano Bocconi

Towards ontology-driven discourse: from semantic graphs to multimedia presentations

Traditionally, research in applying Semantic Web technology to multimedia information systems has focused on using annotations and ontologies to improve the retrieval process. This paper concentrates on improving the presentation of the retrieval results. First, our approach uses ontological domain knowledge to select and organize the content relevant to the topic the user is interested in. Domain ontologies are valuable in the presentation generation process, because effective presentations are those that succeed in conveying the relevant domain semantics to the user. Explicit discourse and narrative knowledge allows selection of appropriate presentation genres and creation of narrative structures, which are used for conveying these domain relations. In addition, knowledge of graphic design and media characteristics is essential to transform abstract presentation structures in real multimedia presentations. Design knowledge determines how the semantics and presentation structure are expressed in the multimedia presentation. In traditional Web environments, this type of design knowledge remains implicit, hidden in style sheets and other document transformation code. Our second use of Semantic Web technology is to model design knowledge explicitly, and to let it drive the transformations needed to turn annotated media items into structured presentations.

Automatic generation of matter-of-opinion video documentaries

In this paper we describe a model for automatically generating video documentaries. This allows viewers to specify the subject and the point of view of the documentary to be generated. The domain is matter-of-opinion documentaries based on interviews. The model combines rhetorical presentation patterns used by documentary makers with a data-driven approach. Rhetorical presentation patterns provide the viewer with an engaging viewing experience, while a data-driven approach can be applied to growing media repositories. To date, the modeling of rhetoric has been achieved in a top-down manner using closed repositories, while data-driven generation approaches were unable to implement non-trivial rhetorical presentation patterns. We describe an implementation of our model in a system, Vox Populi, and apply it to an online documentary shot by a group of independent amateur documentarists.

Automatic generation of biased video sequences

We describe our experimental rhetoric engine <i>Vox Populi</i> that generates biased video-sequences from a repository of video interviews and other related audio-visual web sources. Users are thus able to explore their own opinions on controversial topics covered by the repository. The repository contains interviews with United States residents stating their opinion on the events occurring after the terrorist attack on the United States on the 11th of September 2001. We present a model for biased documentary statements, such as interviews, and explain in detail how this model facilitates the automatic generation of rhetorical arguments on a micro-level. We outline the required representations of relevant rhetorical structures and the way they can be processed. The processes are described via examples generated by our experimental engine. The first example shows how to logically counter an opinion using semantics contained in the audio tracks from the database, while the second example describes the generation of an emotional counterargument using visual material.

Vox populi: a tool for automatically generating video documentaries

Vox Populi is a system that automatically generates video documentaries. Our application domain is video interviews about controversial topics. Via a Web interface the user selects one of the possible topics and a point of view she would like the generated sequence to present, and the engine selects and assembles video material from the repository to satisfy the user request.