James M. Turner

Towards computer-vision software tools to increase production and accessibility of video description for people with vision loss

This paper presents the status of a R&D project targeting the development of computer-vision tools to assist humans in generating and rendering video description for people with vision loss. Three principal issues are discussed: (1) production practices, (2) needs of people with vision loss, and (3) current system design, core technologies and implementation. The paper provides the main conclusions of consultations with producers of video description regarding their practices and with end-users regarding their needs, as well as an analysis of described productions that lead to propose a video description typology. The current status of a prototype software is also presented (audio-vision manager) that uses many computer-vision technologies (shot transition detection, key-frame identification, key-face recognition, key-text spotting, visual motion, gait/gesture characterization, key-place identification, key-object spotting and image categorization) to automatically extract visual content, associate textual descriptions and add them to the audio track with a synthetic voice. A proof of concept is also briefly described for a first adaptive video description player which allows end users to select various levels of video description.

From ABC to http: The Effervescent Evolution of Indexing for Audiovisual Materials

Indexing methods for audiovisual materials had not yet settled when the arrival of the World Wide Web upset any stability that existed in this area. New possibilities have now opened up for indexing digital audiovisual materials in a networked environment. This article, traces some of the methods used for organizing collections of audiovisual materials, give a general portrait of how various types of them are organized today, and using indicators that have become manifest, speculate on some future developments in this area.

A Digital Archives Framework for the Preservation of Cultural Artifacts with Technological Components

The preservation of artistic works with technological components, such as musical works, is recognised as an issue by both the artistic community and the archival community. Preserving such works involves tackling the difficulties associated with digital information in general, but also raises its own specific problems, such as constantly evolving digital instruments embodied within software and idiosyncratic human-computer interactions. Because of these issues, standards in place for archiving digital information are not always suitable for the preservation of these works. The impact on the organisation and the descriptions of such archives need to be conceptualised in order to provide these technological components with readability, authenticity and intelligibility. While previous projects emphasized readability and authenticity, less effort has been dedicated to addressing intelligibility issues. The research into the specification of significant properties and its extension, namely significant knowledge, offers some grounds for reflecting on this question. Furthermore, the relevance of taking into account the creative process involved in the production of technological components offers an opportunity to redefine the status of technological agents in the performative aspect of digital records. Altogether, the research on significant knowledge and creative processes provide us with a conceptual framework that we propose to bring together with digital archives models to form a coherent framework.

Using Ancillary Text to Index Web-based Multimedia Objects

PeriCulture is the name of a research project at the Universite de Montreal which is part of a larger project based at the Universite de Sherbrooke. The parent project aimed to form a research network for managing Canadian digital cultural content. The general research objective of PeriCulture was to study indexing methods for web-based non-textual cultural content, specifically still images. The research results reported here build on work in image indexing and automatic (text) indexing by studying properties of text associated with images in a networked environment to try to gain some understanding of how the ancillary text associated with images on web pages can be exploited to index the corresponding images. We studied this question in the context of selected web sites, i.e. that contained multimedia objects, that had text associated with these objects (broader than file names and captions), that were bilingual (English and French), and that housed Canadian digital cultural content. We identified keywords that were useful in indexing and studied their proximity to the object described. Potential indexing terms were identified in various HTML tags and full text (each considered a different source of ancillary text). Our study found that a large number of useful indexing terms are available in the ancillary text of many web sites with cultural content, and that ancillary text of different sources have variable usefulness in retrieval. Our results suggest that these terms can be manipulated in a number of ways in automated retrieval systems to improve search results.

Modeling videos as works

SUMMARY Defining works is complex, but defining video works is extremely complex because of the large number of instantiations available and because of the intricate relationships among them. The terrorist attacks on the World Trade Center on September 11, 2001 and other important news events offer rich examples of the complexity of the problem and help us gain an understanding of how managing video news material can be modelled. The need for a taxonomy of some of the instantiations of material that need to be managed in the context of a news video library is identified.

Information et visualisation : enjeux, recherches et applications (review)

L’essor de l’information numérique, notamment sur le web, rend plus importante que jamais la nécessité de favoriser la compréhension de la part des usagers, en leur offrant des moyens avancés de présenter l’information de toutes sortes. La visualisation est une approche qui cherche à amplifier la perception et la compréhension en manipulant la forme visuelle que prend la présentation de l’information. Face aux quantités massives d’information, de simples listes de résultats d’une recherche, par exemple, ne suffisent plus aux utilisateurs pour décoder les contenus. Au contraire, il faut offrir diverses vues sur l’information ainsi que la possibilité de les paramétrer, afin de faire ressortir divers aspects. Cela se traduit par « des schémas, des diagrammes et des graphes interactifs, par des représentations cartographiques telles que les cartes heuristiques et conceptuelles, les nuages de mots clés. . . et de métaphores. . . en passant par l’interface graphique » (p. 11). Heureusement, la visualisation est un champ de recherche très actif en ce moment. Caractérisé par la pluridisciplinarité, c’est un domaine dont les recherches ont besoin d’aboutir à des produits utilisables et satisfaisants si on espère retenir longtemps l’attention des internautes.

Putting motion into the image retrieval interface

When video, still and 3D images meet, how do we design a universal interface? This panel looks at current research in the imaging field focused on bridging the gap between image retrieval system designers and users. Creating interfaces that move with the format in heterogeneous databases is only part of the challenge. This panel brings together a wide range of research from the imaging field. It is important to monitor the progress of the imaging field since user expectation, technology and information needs are constantly changing. Current research, such as the research offered in this panel, will help the practitioners and system designers meet the information needs of all the stakeholders involved.

Current research in digital image management. Sponsored by SIG VIS

This panel brings together four LIS researchers in the area of the management of digital images. It offers a look at some of their research activities with a view to providing an update on a number of aspects of research in this important area. Each panelist will provide a brief summary of current research activities with regard to still or moving images. This will be followed by a discussion of how these activities fit into the broader portrait of image research, identification of the main research issues in image management, and an indication of where development is needed.

Classification and automatic indexing in a persistent object environment

The goal of this project is to study classification and automatic indexing for multimedia data in the context of digital libraries. The general objectives are: (1) to propose methods for digital media classification and indexing and (2) to implement them effectively by using a persistent environment provided by object-oriented database systems.