Philippe Aigrain

Content-Based Representation and Retrieval of Visual Media: A State-of-the-Art Review

This paper reviews a number of recently available techniques in content analysis of visual media and their application to the indexing, retrieval, abstracting, relevance assessment, interactive perception, annotation and re-use of visual documents.

The automatic real-time analysis of film editing and transition effects and its applications

Abstract We describe an adaptive method for the automatic analysis of film editing. Unlike previously proposed methods, which were restricted to the detection of cut transitions between sequence shots, our method achieves detection of shot change even when done through smooth transition effects (dissolves, wipes), and identifies classes of transition special effects. This method can be applied in real-time on a flow of digital or analog motion picture data of any resolution. We describe experimental implementations of the method. Applications of such a tool are the temporal analysis of motion picture data to be stored in large databases and the automatic choice of entry points in compressed motion picture. The fact that the analysis can be done in real-time allows for its use at digitizing time, or on the output of a video production site. Temporal analysis and description of motion picture is presently a major time and money bottleneck in the establishment of motion picture databases, and we argue that our method can significantly improve the situation in this field.

Evaluation of navigational links between images

Abstract We describe a general framework for the evaluation of sets of links designed for user navigation between images in pictorial data banks. An example is given of construction of a connection graph for a large image bank. The method used for construction of the connection graph has been shown in previous work. We describe an experiment conducted for the observation of user navigation behaviour in this set of links. The corresponding evaluation results are discussed in the framework of our evaluation model.

Representation-based user interfaces for the audiovisual library of the year 2000

The audiovisual library of the future will be based on computerized access to digitized documents. In this communication, we address the user interface issues which will arise from this new situation. One cannot simply transfer a user interface designed for the piece by piece production of some audiovisual presentation and make it a tool for accessing full-length movies in an electronic library. One cannot take a digital sound editing tool and propose it as a means to listen to a musical recording. In our opinion, when computers are used as mediations to existing contents, document representation-based user interfaces are needed. With such user interfaces, a structured visual representation of the document contents is presented to the user, who can then manipulate it to control perception and analysis of these contents. In order to build such manipulable visual representations of audiovisual documents, one needs to automatically extract structural information from the documents contents. In this communication, we describe possible visual interfaces for various temporal media, and we propose methods for the economically feasible large scale processing of documents. The work presented is sponsored by the Bibliotheque Nationale de France: it is part of the program aiming at developing for image and sound documents an experimental counterpart to the digitized text reading workstation of this library.

Software research for video libraries and archives

Digital video is a field in which huge industry and government programmes are aiming at building a coherent framework for the coding, comprehension, storage, networking and deliverability of video programmes. Video libraries and archives follow this trend with great interest and try to build pilot projects of digital video libraries, but economical and technical constraints have yet prevented most organizations from building fully operational digital systems. This article emphasizes another face of digital video; the ability to use digital processing of video documents for the indexing, retrieval and access to video documents.

Representation and Retrieval of Video Data in Multimedia Systems

Representation and Retrieval of Video Data in Multimedia Systems brings together in one place important contributions and up-to-date research results in this important area. Representation and Retrieval of Video Data in Multimedia Systems serves as an excellent reference, providing insight into some of the most important research issues in the field.

A model for the evaluation of expansion techniques in information retrieval systems

We describe an evaluation model for expansion systems in information retrieval, that is, systems expanding a user selection of documents in order to provide the user with a larger set of documents sharing the same or related characteristics. Our model leads to a test protocol and practical estimates of the efficiency of an expansion system provided that it is possible for a sample of users to exhaustively scan the content of a subset of the database in order to decide which documents would have been selected by an «ideal» expansion system. This condition is met only by databases whose unit contents can be quickly apprehended, such as still image databases or synthetic bibliographical references. We compare our model with other types of possible indicators, and discuss the precision to which our measure can be estimated, using data from experimentation with an image database system developed by our research team

Introduction to Special Issue on Represenatation and Retrieval of Visual Media in MultiMedia Systems (2)

This is the second issue of the special issue of Multimedia Tools and Applications, devoted to Representation and Retrieval of Visual Media in Multimedia System, with the emphasis on implemented prototypes, tools and applications. This issue includes three papers focus-ing on video content analysis and representation. The first paper by Lee, Li, and Xiong presents an experimental video information management system. It describes briefly the video parsing subsystem for extracting low-level visual features of video data, and gives a detailed description on the conceptual clustering scheme implemented in the system for video content representation. The paper by Ardizzone and La Cascia reports a video in-dexing and retrieval system that supports automatic video structure parsing, shot feature extraction and content-based retrieval. Experimental results are discussed in detail. The last paper by Nack and Parkes presents a case study of the applications of a video semantics and theme representation scheme in automated editing and generation of humorous video from stock material. It addresses the problem of video editing modeling to realize a thematic specification by knowledge representation approaches, a very first step towards intelligent video editing.

Introduction to Audio Retrieval and Navigation Interfaces

This chapter pays justice to the preexisting specialized work that has made multimedia content processing technology possible. It also refers to several papers, in particular in emerging domains that would have been included in this collection if size limitations had not prevented it. The recent developments of audio retrieval and navigation interfaces are the history of the meeting between multimedia research and music and speech technology research. One of the reasons for multimedia researchs success in delivering new technology and finding overlooked potential of existing technology is that it has worked within precisely defined paradigms of usage. Despite recent developments, the content-based retrieval of audio still faces major challenges. Efficient indexing and retrieval for large databases of music and for speech databases with many different locutors are still to be demonstrated. An approach combining navigation techniques and query/retrieval techniques is still in its infancy. Nevertheless, a core set of techniques has been developed, from which more systematic work can proceed.