Gert Kienast

Towards a format-agnostic approach for production, delivery and rendering of immersive media

The media industry is currently being pulled in the often-opposing directions of increased realism (high resolution, stereoscopic, large screen) and personalization (selection and control of content, availability on many devices). We investigate the feasibility of an end-to-end format-agnostic approach to support both these trends. In this paper, different aspects of a format-agnostic capture, production, delivery and rendering system are discussed. At the capture stage, the concept of layered scene representation is introduced, including panoramic video and 3D audio capture. At the analysis stage, a virtual director component is discussed that allows for automatic execution of cinematographic principles, using feature tracking and saliency detection. At the delivery stage, resolution-independent audiovisual transport mechanisms for both managed and unmanaged networks are treated. In the rendering stage, a rendering process that includes the manipulation of audiovisual content to match the connected display and loudspeaker properties is introduced. Different parts of the complete system are revisited demonstrating the requirements and the potential of this advanced concept.

A Video Browsing Tool for Content Management in Postproduction

We propose an interactive video browsing tool for supporting content management and selection in postproduction. The approach is based on a process model for multimedia content abstraction. A software framework based on this process model and desktop and Web-based client applications are presented. For evaluation, we apply two TRECVID style fact finding approaches (retrieval and question answering tasks) and a user survey to the evaluation of the video browsing tool. We analyze the correlation between the results of the different methods, whether different aspects can be evaluated independently with the survey, and if a learning effect can be measured with the different methods, and we also compare the full-featured desktop and the limited Web-based user interface. The results show that the retrieval task correlates better with the user experience according to the survey. The survey rather measures the general user experience while different aspects of the usability cannot be analyzed independently.

SALERO: Semantic AudiovisuaL Entertainment Reusable Objects

The Integrated Project SALERO aims to advance the state of the art in digital media to the point where it becomes possible to create audiovisual content for cross-platform delivery using intelligent content tools, with greater quality at lower cost, to provide audiences with more engaging entertainment and information at home or on the move. SALERO will build on and extend research in media technologies, web semantics and context based image retrieval, to reverse the trend toward ever-increasing cost of creating media.

The fascinate production scripting engine

In the realm of a format agnostic live event broadcast system, the FascinatE Scripting Engines are software components that automate taking decisions on what is visible and audible at each playout device and prepare the audiovisual content streams for display. Essentially, they act together as a Virtual Director with the production team possibly steering it via a backend user interface. We present an architecture for this real-time system and describe interfaces to other production components. Details of subcomponents of the distributed engine, design decisions and technology choices are discussed.

Integration of Media Sources for Situation Analysis in the Different Phases of Disaster Management: The QuOIMA Project

In this paper we describe work in progress on a cross-media content analysis approach and framework, which is currently being developed within the QuOIMA project. We describe the role of media, and how possible links between social and traditional media and terminology and communication patterns are envisioned to be connected to the different phases of a disaster model. The paper continues with a discussion of potential benefits for decision makers and planners and concludes with an outlook on further planned activities and developments.

Video documentation of urban areas

Abstract The massive amount of information that is currently available is challenging the ability of urban communities to organize, process, and analyze all the information they require. This paper focuses on the concepts and principles for multimedia assets (especially videos) aiming to provide easier access to geographically relevant views on urban areas. Geo-referenced videos can be a great support in the management process of urban communities. This paper explains the process of acquisition and geo-referencing videos using low cost equipment. Furthermore, it describes the architecture of a Geo-Multimedia service infrastructure which enables users to store, retrieve, and share geo-referenced video content. Via a standard and state-of-the-art Web Service interface, geographic and multimedia objects can be retrieved from the Geo-Multimedia service by systems and applications already in use for administration (e.g. street management) or marketing purposes.

Cross-Media Analysis for Communication during Natural Disasters

In this paper we describe the role of media in the context of natural disasters. Traditional media have a long history in covering disasters and will continue to be a major provider of information in the future. In recent years, however there has been a significant change: information about natural disasters has increasingly been disseminated on a large scale on social media platforms. These media are typically faster but may be less reliable. They provide additional and complementary angles on events and, combined with traditional media, provide a wider spectrum of coverage. We argue that cross-media information combined with multi-lingual data provides huge opportunities for first-responders and decision makers to gain improved situational awareness allowing for improved disaster relief, support, mitigation and resilience measures.

Data Fusion Across Traditional and Social Media

Crises and disasters are covered continuously and without interruption by today’s media, especially social media. There is not a single significant occurrence within the flow of events which they do not document. Consequently, the information contained in media—especially social media like Facebook and Twitter—provides an often neglected potential which should not be overlooked. Through fusion of sources, diverse, mixed, and complementary types of information can be tapped into and combined. The difficulty of this process is to view, channel, prepare, and exploit this inhomogeneous and enormous amount of information. Automatic monitoring of traditional as well as social media sources allows to deriving risk factors and risk indicators for crises and disaster events quickly. Intelligence derived from this process allows for earlier and swifter reaction to potential situations of crisis and interrelationships. Current publicly described technical and electronic infrastructure for national and international crisis and disaster management is not able to perform comprehensive analyses of all media channels automatically. The continuous developments in the areas of multimedia and social media demand the creation of adequate methods of processing. Relevant manifestations of events are to be identified automatically from documents from traditional (TV, radio, web) as well as social media and document clusters of the examined multimedia documents are to be presented to situational awareness experts. The focus of the Quelloffene IntegrierteMultimedia Analyse (QuOIMA) project is on the research on and development of algorithms and methods to achieve this goal. Automatic analysis of content in the multimedia and social media domain forms a fundamental innovation. From a technical, social studies, and scientific point of view, the targeted insights and findings of this project, form a fundamental contribution to security research, reaching far beyond the quality of existing systems. The integration of findings regarding situational awareness will provide more realistic risk assessment increasing their possibilities to (re)act. End users extend their expertise and as a consequence the ability of the overall organizations to act.

Video Assisted Geographical Information Systems

In recent years the communities working on geographic information systems (GIS) and multimedia applications have developed highly advanced technologies which are specialized and well accepted in their respective domains. Until recently the functionalities offered by these applications have not been integrated to build a video enabled geographic information system. This paper investigates possible synergies from both of these research fields to enhance currently existing applications. A short overview of relevant standardization efforts for data and metadata is given and the concept of smart media objects is described. To demonstrate the concept, this paper describes a number of use cases and applications of such video assisted GIS in the area of multimedia archives and territorial networks.