Steven C. Hsu

A machine-vision system for iris recognition

This paper describes a prototype system for personnel verification based on automated iris recognition. The motivation for this endevour stems from the observation that the human iris provides a particularly interesting structure on which to base a technology for noninvasive biometric measurement. In particular, it is known in the biomedical community that irises are as distinct as fingerprints or patterns of retinal blood vessels. Further, since the iris is an overt body, its appearance is amenable to remote examination with the aid of a machine-vision system. The body of this paper details the design and operation of such a system. Also presented are the results of an empirical study in which the system exhibits flawless performance in the evaluation of 520 iris images.

Mosaic based representations of video sequences and their applications

Recently, there has been a growing interest in the use of mosaic images to represent the information contained in video sequences. The paper systematically investigates how to go beyond thinking of the mosaic simply as a visualization device, but rather as a basis for efficient representation of video sequences. We describe two different types of mosaics called the static and the dynamic mosaic that are suitable for different needs and scenarios. We discuss a series of extensions to these basic mosaics to provide representations at multiple spatial and temporal resolutions and to handle 3D scene information. We describe techniques for the basic elements of the mosaic construction process, namely alignment, integration, and residual analysis. We describe several applications of mosaic representations including video compression, enhancement, enhanced visualization, and other applications in video indexing, search, and manipulation.<<ETX>>

Robust Video Mosaicing through Topology Inference and Local to Global Alignment

The problem of piecing together individual frames in a video sequence to create seamless panoramas (video mosaics) has attracted increasing attention in recent times. One challenge in this domain has been to rapidly and automatically create high quality seamless mosaics using inexpensive cameras and relatively free hand motions.

Efficient representations of video sequences and their applications

Abstract Recently, there has been a growing interest in the use of mosaic images to represent the information contained in video sequences. This paper systematically investigates how to go beyond thinking of the mosaic simply as a visualization device, but rather as a basis for an efficient and complete representation of video sequences. We describe two different types of mosaics called the static and the dynamic mosaics that are suitable for different needs and scenarios. These two types of mosaics are unified and generalized in a mosaic representation called the temporal pyramid. To handle sequences containing large variations in image resolution, we develop a multiresolution mosaic. We discuss a series of increasingly complex alignment transformations (ranging from 2D to 3D and layers) for making the mosaics. We describe techniques for the basic elements of the mosaic construction process, namely sequence alignment, sequence integration into a mosaic image, and residual analysis to represent information not captured by the mosaic image. We describe several powerful video applications of mosaic representations including video compression, video enhancement, enhanced visualization, and other applications in video indexing, search, and manipulation.

Alignment of continuous video onto 3D point clouds

We propose a general framework for aligning continuous (oblique) video onto 3D sensor data. We align a point cloud computed from the video onto the point cloud directly obtained from a 3D sensor. This is in contrast to existing techniques where the 2D images are aligned to a 3D model derived from the 3D sensor data. Using point clouds enables the alignment for scenes full of objects that are difficult to model; for example, trees. To compute 3D point clouds from video, motion stereo is used along with a state-of-the-art algorithm for camera pose estimation. Our experiments with real data demonstrate the advantages of the proposed registration algorithm for texturing models in large-scale semi-urban environments. The capability to align video before a 3D model is built from the 3D sensor data offers new practical opportunities for 3D modeling. We introduce a novel modeling-through-registration approach that fuses 3D information from both the 3D sensor and the video. Initial experiments with real data illustrate the potential of the proposed approach.

Registration of video to geo-referenced imagery

The ability to locate scenes and objects visible in aerial video imagery with their corresponding locations and coordinates in a reference coordinate system is important in visually-guided navigation, surveillance and monitoring systems. However, a key technical problem of locating objects and scenes in a video with their geo-coordinates needs to be solved in order to ascertain the geo-location of objects seen from the camera platforms current location. We present the key algorithms for the problem of accurate mapping between camera coordinates and geo-coordinates, called geo-spatial registration. Current systems for geo-location use the position and attitude information for the moving platform in some fixed world coordinates to locate the video frames in the reference database. However, the accuracy achieved is only of the order of 100s of pixels. Our approach utilizes the imagery and terrain information contained in the geo-spatial database to precisely align dynamic videos with the reference imagery and thus achieves a much higher accuracy. Applications of geo-spatial registration include aerial mapping, target location and tracking and enhanced visualization such as the overlay of textual/graphical annotations of objects of interest in the current video using the stored annotations in the reference database.

VIDEO COMPRESSION USING MOSAIC REPRESENTATIONS

Abstract We describe a technique for video compression based on a mosaic image representation obtained by aligning all frames of a video sequence, giving a panoramic view of the scene. We describe two types of mosaics, static and dynamic, which are suited for storage and transmission applications, respectively. In each case, the mosaic construction process aligns the images using a global parametric motion transformation, usually canceling the effect of camera motion on the dominant portion of the scene. The residual motions that are not compensated by the parametric motion are then analyzed for their significance and coded. The mosaic representation exploits large scale spatial and temporal correlations in image sequences. In many applications where there is significant camera motion (e.g., remote surveillance), it performs substantially better than traditional interframe compression methods and offers the potential for very low bit-rate transmission. In storage applications, such as digital libraries and video editing environments, it has the additional benefit of enabling direct access and retrieval of single frames at a time.

Accurate computation of optical flow by using layered motion representations

This paper presents a framework combining two prevailing approaches to motion analysis: optical flows which describes motion at each point, and methods that define global motions for larger regions. Image motion is represented by layers-image regions whose coherent motion can be approximated by some parametric motion model. The motion at every point is obtained by the parametric motion estimate of the entire layer, corrected by a residual flow which captures the difference between the real image motion and the layers motion model. The new approach is able to construct accurate flow fields in the presence of multiple motions, motion boundaries, and transparent motions.

Video georegistration: algorithm and quantitative evaluation

An algorithm is presented for video georegistration, with a particular concern for aerial video, i.e., video captured from an airborne platform. The algorithms input is a video stream with telemetry (camera model specification sufficient to define an initial estimate of the view) and geodetically calibrated reference imagery (coaligned digital orthoimage and elevation map). The output is a spatial registration of the video to the reference so that it inherits the available geodetic coordinates. The video is processed in a continuous fashion to yield a corresponding stream of georegistered results. Quantitative results of evaluating the developed approach with real world aerial video also are presented. The results suggest that the developed approach may provide valuable input to the analysis and interpretation of aerial video.

Vehicle fingerprinting for reacquisition & tracking in videos

Visual recognition of objects through multiple observations is an important component of object tracking. We address the problem of vehicle matching when multiple observations of a vehicle are separated in time such that frames of observations are not contiguous, thus prohibiting the use of standard frame-to-frame data association. We employ features extracted over a sequence during one time interval as a vehicle fingerprint that is used to compute the likelihood that two or more sequence observations are from the same or different vehicles. The challenges of change in pose, aspect and appearances across two disparate observations are handled by combining feature-based quasi-rigid alignment with flexible matching between two or more sequences. The current work uses the domain of vehicle tracking from aerial platforms where typically both the imaging platform and the vehicles are moving and the number of pixels on the object are limited to fairly low resolutions. Extensive evaluation with respect to ground truth is reported in the paper.