Marian Florin Ursu

Interactive TV narratives: Opportunities, progress, and challenges

This article is motivated by the question whether television should do more than simply offer interactive services alongside (and separately from) traditional linear programs, in the context of its dominance being seriously challenged and threatened by interactive forms of screen media entertainment. It suggests: yes. Interactive narrativity, that is, the ability to interact with (and influence) stories whilst they are being told, represents one clear development path for interactive television. The capabilities of computing technology are ripe for exploring this new form of storytelling, from creation to commercial distribution. The article starts by looking at the relationship between narrativity and interactivity in the current context of screen media, and identifies clear signs of interest from certain European public broadcasters in interactive TV narratives. It then presents in detail four recent experimental interactive TV productions in the genres of drama, news, and documentary, developed in collaboration with public broadcasters, which illustrate the potential and richness of this new form of storytelling, but also highlight new technological capabilities necessary for such productions. A number of essential technological requirements are then discussed in more detail in the final part. The article suggests that the ShapeShifting Media Technology, employed in the implementation of the four productions, has made significant advances both at the technological and the creative ends in supporting the development of interactive TV narrativity, but, however, that further developments are required before being able to answer questions such as “Would end users want such a form of screen media entertainment?” and “Would it be effective for both end users and producers?”

Automatic generation of video narratives from shared UGC

This paper introduces an evaluated approach to the automatic generation of video narratives from user generated content gathered in a shared repository. In the context of social events, end-users record video material with their personal cameras and upload the content to a common repository. Video narrative techniques, implemented using Narrative Structure Language (NSL) and ShapeShifting Media, are employed to automatically generate movies recounting the event. Such movies are personalized according to the preferences expressed by each individual end-user, for each individual viewing. This paper describes our prototype narrative system, MyVideos, deployed as a web application, and reports on its evaluation for one specific use case: assembling stories of a school concert by parents, relatives and friends. The evaluations carried out through focus groups, interviews and field trials, in the Netherlands and UK, provided validating results and further insights into this approach.

ShapeShifting TV: interactive screen media narratives

This paper presents a paradigm, called ShapeShifting TV, for the realisation of interactive TV narratives or, more generally, of interactive screen-media narratives. These are productions whose narrations respond on the fly (i.e. in real time) to interaction from active viewers. ShapeShifting TV refers to productions made mainly with pre-recorded time-based material, in which variation is achieved by selecting and rearranging atomic elements of content (e.g. video clips) into individual narrations. The aimed quality of the productions (e.g. narrative continuity and aesthetics) is at least that of good traditional linear TV programmes. The artefact which determines the way individual stories unfold, called the narrative space, is authored and tested by experts before the delivery of the programme. However, the adaptation of narrations to input, at delivery time, is automatic. ShapeShifting TV is a generic paradigm; it is neither production nor genre specific. Furthermore, it is not confined to television; it is about screen media in general. ShapeShifting TV is founded on a computational language called Narrative Structure Language (NSL) and is accompanied by a comprehensive software system for authoring and delivery (which implements NSL). These were successfully employed to the creation of a number of ShapeShifting TV productions, which extended genres such as drama, documentary and news with interactivity. This paper defines the ShapeShifting TV paradigm, outlines NSL and the associated software, and presents two ShapeShifting TV productions.

Reasoning for video-mediated group communication

In this paper we present an approach to the reasoning required to support multi-location, multi-camera group-to-group video communication, which we call orchestration. Orchestration is akin to virtual directing: it has to ensure that each location displays the most adequate shots from all the other available sources. Its input is low-level cues extracted automatically from the AV streams. They are processed to detect higher-level events that determine the state of the communication. Directorial decisions are then inferred, reflecting social communication as well as stylistic criteria. Finally, they are transformed into camera and editing commands, directly executable by the AV infrastructure. Here, we present the architecture of the Orchestrator and sketch our rule-based approach to reasoning.

Conceiving ShapeShifting TV: a computational language for truly-interactive TV

iTV does not yet have truly interactive programmes, that is programmes whose content adapts to the preferences of their viewers. In commercially deployed iTV productions, the programmes themselves are essentially linear and therefore non-interactive. In the research arena, the main bulk of work in computational support for interactive narratives focuses on wrapping interactions up in meaningful and interesting narratives, rather than on expanding traditional linear narratives with interactivity. This paper presents a validated approach to the development of truly interactive programmes called ShapeShifting TV. In focus is a representation language for narrative structures.

Ontological Representations of Software Patterns

This paper is based on and advocates the trend in software engineering of extending the use of software patterns as means of structuring solutions to software development problems (be they motivated by best practice or by company interests and policies). The paper argues that, on the one hand, this development requires tools for automatic organisation, retrieval and explanation of software patterns. On the other hand, that the existence of such tools itself will facilitate the further development and employment of patterns in the software development process. The paper analyses existing pattern representations and concludes that they are inadequate for the kind of automation intended here. Adopting a standpoint similar to that taken in the semantic web, the paper proposes that feasible solutions can be built on the basis of ontological representations.

Interactive documentaries: A Golden Age

This article is motivated by the opportunity presented by recent advances in information and communication technology—particularly by faster broadband connections and faster digital media processing capabilities—for interactive television to extend and develop interactive storytelling or interactive narratives. This will give viewers the ability to shape and configure the programs they watch, while watching, according to their needs and desires. Rather than consuming a predefined linear narration—represented by the traditional dramatic or factual program—which has to address the potential audience as a whole, individuals or groups of viewers can receive tailored-made personal narratives. Each viewer can thus potentially become an active explorer of a narrative space rather than a receiver of a predefined narration. This article presents the production of A Golden Age, an interactive configurable documentary about the arts of the Renaissance in England, as a comprehensive illustration of the potential offered by interactive narration. At the same time, it is also a successful example of the employment of the recently developed, production- and genre-independent, ShapeShifting Media technology in the realization of a good quality interactive narrative. This article describes the concept of A Golden Age, the content production process, carried out from the outset with the aim of producing an interactive experience, and, finally, its authoring and delivery with the ShapeShifting Media toolkit. The focus of the presentation is on the design and implementation of the computational interactive narrative structures expressed in the Narrative Structure Language (NSL), the declarative representation language underlying ShapeShifting Media. A Golden Age places a distinct emphasis on the quality and style of each emerging individual narration, aiming at levels at least comparable to those of (good quality) linearly compiled documentaries. NSL and the ShapeShifting Media toolkit provided the means to achieve this. A Golden Age is a production realized by Illuminations Television Ltd, London, in collaboration with Goldsmiths, University of London and BT over a period of more or less two years. Approximately 50 hours of rushes were filmed for its production. A Golden Age has already inspired the production of another similar documentary, Films of Fact, soon to be released in the public domain as an installation at the Science Museum, London, and, it is hoped, will continue to serve as inspiration for other interactive documentaries.

A Rule-Based Virtual Director Enhancing Group Communication

Audiovisual group communication systems deal with a large number of video streams, and, unlike less advanced videoconferencing systems, require intelligence for selecting adequate views for each of the connected rooms, in order to convey best what is happening in the other locations. Such a decision making component, in our implementation called Orchestration Engine (OE), acts as a Virtual Director. It processes low level events, emitted by content analysis sensors, into editing commands. The OE has two main components: one that semantically lifts low-level events into communication events and one that associates editing decisions to communication contexts. The former has to deal with uncertain and delayed information. The latter subsumes knowledge that reflects both conversation and narrative principles. Both components include contradicting bodies of knowledge. We investigate a rule-based event processing approach and reflect the scalability of our solution regarding competing and contradicting rules.

Improving video-mediated communication with orchestration

Video-mediated communication (VMC) has become a popular communication medium. However, research to date suggests that the inherent constraints of VMC impair effective and efficient communication and task performance. We propose that these negative findings could be attributed to how the technology was used and propose the novel concept of communication orchestration aimed at mitigating some of the signaled limitations. Orchestration is a selection process for displaying information that is deemed relevant for accomplishing an effective and efficient task performance and communicative experience. We report an experiment that confirmed this suggestion. The results indicate that orchestration could be an important novel feature to aid humans when communicating via VMC, but also suggest that there is potential for further improvements in orchestration.

Interactive video stories from user generated content: a school concert use case

This paper describes a web-based narrative system able to generate video compilations, framed as event stories, from a shared repository of video recordings of the event itself and possibly of related events. For this, it employs narrative techniques informed by TV documentary. The generated stories are dynamically personalised, in that the system is able to adapt them to the choices and preferences expressed by the active viewers during narration. The system has been prototyped for the case of school concerts. User evaluations indicate that experiences founded on story navigation rather than sharing individual media assets is a rewarding one and point to further areas of development in interactive storytelling in the context of user generated content.