Ross Cutler

View-based interpretation of real-time optical flow for gesture recognition

The authors have developed a real-time, view-based gesture recognition system. Optical flow is estimated and segmented into motion blobs. Gestures are recognized using a rule-based technique based on characteristics of the motion blobs such as relative motion and size. Parameters of the gesture (e.g., frequency) are then estimated using context specific techniques. The system has been applied to create an interactive environment for children.

Person identification using automatic height and stride estimation

We present a parametric method to automatically identify people in monocular low-resolution video by estimating the height and stride parameters of their gait. Stride parameters (stride length and cadence) are functions of body height, weight, and gender Previous work has demonstrated an effective use of these biometrics for identification and verification of people. In this paper, we show that performance is significantly improved by using height as an additional discriminant feature. Height is estimated by segmenting the person from the background and fitting their apparent height to a time-dependent model. With a database of 45 people and 4 samples of each, we show that a person is correctly identified with 49% probability when using both height and stride parameters, compared with 21% when using stride parameters only. Height estimates for this configuration are accurate to within /spl sigma/=3.5 cm. This method works with low-resolution images of people, and is robust to changes in lighting, clothing, and tracking errors.

Look who's talking: speaker detection using video and audio correlation

The visual motion of the mouth and the corresponding audio data generated when a person speaks are highly correlated. This fact has been exploited for lip/speech-reading and for improving speech recognition. We describe a method of automatically detecting a talking person (both spatially and temporally) using video and audio data from a single microphone. The audio-visual correlation is learned using a time-delayed neural network, which is then used to perform a spatio-temporal search for a speaking person. Applications include videoconferencing, video indexing and improving human-computer interaction (HCI). An example HCI application is provided.

View-based detection and analysis of periodic motion

We describe a technique that detects periodic motion. Assuming a static camera, we first segment moving objects from the background. By tracking objects of interest, we compute the objects self-similarity as it evolves in time. For periodic motion, the self-similarity metric is periodic, and is Fourier analyzed to detect and characterize periodicity. Examples on real image sequences are given.

Backpack: detection of people carrying objects using silhouettes

We described a video-rate surveillance algorithm to detect and track people from a stationary camera, and to determine if they are carrying objects or moving unencumbered. The contribution of the paper is the shape analysis algorithm that both determines if a person is carrying an object and segments the object from the person so that it can be tracked, e.g., during an exchange of objects between two people. As the object is segmented an appearance model of the object is constructed. The method combines periodic motion estimation with static symmetry analysis of the silhouettes of a person in each frame of the sequence. Experimental results demonstrate robustness and real-time performance of the proposed algorithm.

Real-time periodic motion detection, analysis, and applications

We describe a new technique to detect and analyze periodic motion as seen from both a static and moving camera. By tracking objects of interest, we compute an objects self-similarity as it evolves in time. For periodic motion, the self-similarity measure is also periodic, and we apply time-frequency analysis to detect and characterize the periodic motion. A real-time system has been implemented to track and classify objects using periodicity. Examples of object classification, person counting, and non-stationary periodicity are provided.

Robust periodic motion and motion symmetry detection

We describe a robust technique for detecting nonstationary periodic motion from a moving and static camera. We also describe a robust technique for discriminating motion symmetries (periodic motion classification), which we apply to classifying running humans (bipeds) and canines (quadrupeds). The system has been implemented to run in real-time (30 Hz) on standard PC workstations.

Monitoring human and vehicle activities using Airborne Video

Ongoing work in Activity Monitoring (AM) for the Airborne Video Surveillance (AVS) project is described. The goal for AM is to recognize activities of interest involving humans and vehicles using airborne video. AM consists of three major components: (1) moving object detection, tracking, and classification; (2) image to site-model registration; (3) activity recognition. Detecting and tracking humans and vehicles form airborne video is a challenging problem due to image noise, low GSD, poor contrast, motion parallax, motion blur, and camera blur, and camera jitter. We use frame-to- frame affine-warping stabilization and temporally integrated intensity differences to detect independent motion. Moving objects are initially tracked using nearest-neighbor correspondence, followed by a greedy method that favors long track lengths and assumes locally constant velocity. Object classification is based on object size, velocity, and periodicity of motion. Site-model registration uses GPS information and camera/airplane orientations to provide an initial geolocation with +/- 100m accuracy at an elevation of 1000m. A semi-automatic procedure is utilized to improve the accuracy to +/- 5m. The activity recognition component uses the geolocated tracked objects and the site-model to detect pre-specified activities, such as people entering a forbidden area and a group of vehicles leaving a staging area.

Multimedia authoring systems

In this paper we survey three multimedia authoring systems (Multimedia Toolbook 3.0, Director 4.0, and Authorware 3.0). Each system uses a different metaphor (book, movie, and icon-based flowchart) for creating multimedia applications. A sample application is developed in each MAS and the effects of the corresponding metaphors are compared. We also discuss current technologies like ODBC, OLE, DDE, DLL, and MCI, and describe how MAS’s are benefiting from them. In the last section of the paper, we look at the limitations of current systems, and discuss the future of the research in this area.