Lynn Wilcox

Temporal event clustering for digital photo collections

Organizing digital photograph collections according to events such as holiday gatherings or vacations is a common practice among photographers. To support photographers in this task, we present similarity-based methods to cluster digital photos by time and image content. The approach is general and unsupervised, and makes minimal assumptions regarding the structure or statistics of the photo collection. We present several variants of an automatic unsupervised algorithm to partition a collection of digital photographs based either on temporal similarity alone, or on temporal and content-based similarity. First, interphoto similarity is quantified at multiple temporal scales to identify likely event clusters. Second, the final clusters are determined according to one of three clustering goodness criteria. The clustering criteria trade off computational complexity and performance. We also describe a supervised clustering method based on learning vector quantization. Finally, we review the results of an experimental evaluation of the proposed algorithms and existing approaches on two test collections.

A hidden Markov model framework for video segmentation using audio and image features

This paper describes a technique for segmenting video using hidden Markov models (HMM). Video is segmented into regions defined by shots, shot boundaries, and camera movement within shots. Features for segmentation include an image-based distance between adjacent video frames, an audio distance based on the acoustic difference in intervals just before and after the frames, and an estimate of motion between the two frames. Typical video segmentation algorithms classify shot boundaries by computing an image-based distance between adjacent frames and comparing this distance to fixed, manually determined thresholds. Motion and audio information is used separately. In contrast, our segmentation technique allows features to be combined within the HMM framework. Further, thresholds are not required since automatically trained HMMs take their place. This algorithm has been tested on a video data base, and has been shown to improve the accuracy of video segmentation over standard threshold-based systems.

NoteLook: taking notes in meetings with digital video and ink

NoteLook is a client-server system designed and built to support multimedia note taking in meetings with digital video and ink. It is integrated into a conference room equipped with computer controllable video cameras, video conference camera, and a large display rear video projector. The NoteLook client application runs on wireless pen-based notebook computers. Video channels containing images of the room activity and presentation material are transmitted by the NoteLook servers to the clients, and the images can be interactively and automatically incorporated into the note pages. Users can select channels, snap in large background images and sequences of thumbnails, and write freeform ink notes. A smart video source management component enables the capture of high quality images of the presentation material from a variety of sources. For accessing and browsing the notes and recorded video, NoteLook generates Web pages with links from the images and ink strokes correlated to the video.

Dynomite: a dynamically organized ink and audio notebook

Dynomite is a portable electronic notebook for the capture and retrieval of handwritten and audio notes. Dynomite provides a casual, paper like interface, allowing the user to write or draw on a screen resembling an ordinary paper notebook. Indexing is provided by allowing users to attach properties, or data types, to specific ink, and by allowing the user to associate text keywords with pages of notes. Dynomite notes can be organized into sub notebooks based on these indexes. In addition, Dynomite records audio synchronously with notes and allows users to mark or highlight significant sections, so that they may be easily located and replayed. In order to understand how well the features of Dynomite support natural note taking practices, we conducted usage studies, which showed that Dynomites indexing capabilities clearly improve the ability to find information in notes. (4 pages)

An intelligent media browser using automatic multimodal analysis

1. ABSTWCT Rlany techniques can e~act information from an multimedia stream, such as speaker identi~ or shot boundaries. We present a browser that usw t~ls information to navigate through stored media. Because automatica~y-derived information is not wholly retiable, it is transformed into a time-dependent “cotiIdence score~’ When presented graphicaUy,confidence scores enable users to make informed decisionsabout regions of inter~t in themedia, so thatnon-interesting areas maybe skipped. Additiondy, index points maybe determined automati~y for my navigation, selection, editing, and annotation and tin support analysistyp= other than the speaker identification and shot detection used here.

Leveraging face recognition technology to find and organize photos

With digital still cameras, users can easily collect thousands of photos. We have created a photo management application with the goal of making photo organization and browsing simple and quick, even for very large collections. A particular concern is the manage-ment of photos depicting people. We present a semi-automatic approach designed to facilitate the task of labeling photos with people that opportunistically takes advantage of the strengths of current state-of-the-art technology in face detection and recognition. In particular, an accurate face detector is used to automatically extract faces from photos while the less accurate face recog-nizer is used not to classify the detected faces, but to sort faces by their similarity to a chosen model. The sorted faces are presented as candidates within a user interface designed for quick and easy face labeling. We present results of a simulation of the usage model that demonstrate the improved ease that is achieved by our method

Room with a rear view. Meeting capture in a multimedia conference room

Using advances in audio and video technology, capturing meetings can be much more than recording slide presentations and videotaping speakers. The FX Palo Alto Laboratory combined notetaking software and behind-the-scenes network hardware to create an unobtrusive multimedia conference room that is capable of capturing a range of presentation styles.

Keyframe-based user interfaces for digital video

Three visual interfaces that use keyframes - still images automatically pulled from video - to provide access points for efficient navigation of recorded content can help identify potentially useful or relevant video segments.

Training and search algorithms for an interactive wordspotting system

Algorithms for a speaker-dependent wordspotting system based on hidden Markov models (HMMs) are described. The system allows a user to specify keywords dynamically and to train the associated HMMs via a single repetition of a keyword. Nonkeyword speech is modeled using an HMM trained from a prerecorded sample of continuous speech. The wordspotter is intended for interactive applications, such as the editing of voice mail or mixed-media documents, and for keyword indexing in audio or video recordings. The forward-backward search algorithm used in the wordspotter is compared with the Viterbi decoder on the basis of speed and accuracy. In addition, an algorithm for speaker adaptation is described which allows indexing by a user into another talkers speech.<<ETX>>

FlySPEC: a multi-user video camera system with hybrid human and automatic control

FlySPEC is a video camera system designed for real-time remote operation. A hybrid design combines the high resolution of an optomechanical video camera with the wide field of view always available from a panoramic camera. The control system integrates requests from multiple users so that each controls a virtual camera. The control system seamlessly integrates manual and fully automatic control. It supports a range of options from untended automatic to full manual control. The system can also learn control strategies from user requests. Additionally, the panoramic view is always available for an intuitive interface, and objects are never out of view regardless of the zoom factor. We present the system architecture, an information-theoretic approach to combining panoramic and zoomed images to optimally satisfy user requests, and experimental results that show the FlySPEC system significantly assists users in a remote inspection tasks.