Arturo Donate

Human-aware robot motion planning with velocity constraints

This paper addresses the issue of how high-speed robots may move among humans such that the robots complete their tasks efficiently while the humans in the environment feel safe and comfortable. It describes the Segway robotic platform used for this research and then discusses the three primary research areas needed to develop the human-aware motion planner. First, it is necessary to conduct experiments with humans to develop human aware velocity constraints as a function of the distance of the robot from a human. Next, these velocity constraints must be used to plan the robot motion in real time. Finally, practical implementation of this motion planner requires the ability to robustly detect humans using the available vision sensors. The approach taken to each of these problems is described in this paper along with preliminary results.

Improved Reconstruction of Images Distorted by Water Waves

This paper describes a new method for removing geometric distortion in images of submerged objects observed from outside shallow water. We focus on the problem of analyzing video sequences when the water surface is disturbed by waves. The water waves will affect the appearance of the individual video frames such that no single frame is completely free of geometric distortion. This suggests that, in principle, it is possible to perform a selection of a set of low distortion sub-regions from each video frame and combine them to form a single undistorted image of the observed object. The novel contribution in this paper is to use a multi-stage clustering algorithm combined with frequency domain measurements that allow us to select the best set of undistorted sub-regions of each frame in the video sequence. We evaluate the new algorithm on video sequences created both in our laboratory, as well as in natural environments. Results show that our algorithm is effective in removing distortion caused by water motion.

Efficient Path-Based Stereo Matching With Subpixel Accuracy

This paper presents an efficient algorithm to achieve accurate subpixel matchings for calculating correspondences between stereo images based on a path-based matching algorithm. Compared with point-by-point stereo-matching algorithms, path-based algorithms resolve local ambiguities by maximizing the cross correlation (or other measurements) along a path, which can be implemented efficiently using dynamic programming. An effect of the global matching criterion is that cross correlations at all pixels contribute to the criterion; since cross correlation can change significantly even with subpixel changes, to achieve subpixel accuracy, it is no longer sufficient to first find the path that maximizes the criterion at integer pixel locations and then refine to subpixel accuracy. In this paper, by writing bilinear interpolation using integral images, we show that cross correlations at all subpixel locations can be computed efficiently and, thus, lead to a subpixel accuracy path-based matching algorithm. Our results show the feasibility of the method and illustrate significant improvement over existing path-based matching methods.

Shot boundary detection in videos using robust three-dimensional tracking

The use of three dimensional information from video is rare in the video analysis literature due to the inherent difficulties of extracting accurate 3D measurements from a single view of a scene. Several methods have been published in recent years, however, that attempt to solve such a problem. They all use the same underlying meaning of exploiting camera motion in order to measure the parallax of visible objects in the scene. In this paper, we employ the use of such algorithms towards solving the problem of automatic shot boundary detection. The idea is to extract salient features from a video sequence and track them over time in order to estimate shot boundaries within the video. We apply many ideas from previously published SLAM techniques in order to model the inherent three dimensional structure of a scene, and accurately track various salient features across frames. We detect shot boundaries in videos by observing the systems ability to successfully track features across frames.

Classification of Textures Distorted by WaterWaves

In this paper, we approach the novel problem of classifying images of underwater textures as observed from outside the water. Our main contribution is to combine a geometric distortion removal algorithm with a texture classification method to solve the problem of classifying images of submerged textures when the water is disturbed by waves. We show that by modeling the separate types of distortion, we can extract enough texture information to correctly classify textures using spatial statistical measurements on the texton representations. We evaluate our algorithm on both natural and artificial textures acquired in our laboratory. Results are promising and show the feasibility of our algorithm

Efficient and accurate subpixel path based stereo matching

This paper presents an efficient algorithm to achieve accurate subpixel matchings for calculating correspondences between stereo images based on a path-based matching algorithm. Compared to point-by-point stereo matching algorithms, path-based algorithms resolve local ambiguities by maximizing the cross correlation (or other measurements) along a path, which can be implemented efficiently using dynamic programming. An effect of the global matching criterion is that the cross correlation at all pixels can contribute to the criterion; since cross correlation can change significantly even with subpixel changes, to achieve subpixel accuracy, it is no longer sufficient to first find the path that maximizes the criterion and then refine to subpixel accuracy. In this paper, by writing bilinear interpolation using integral images, we show that cross correlations at all subpixel locations can be computed efficiently and thus lead to a subpixel accuracy path based matching algorithm. Our results show the feasibility of the method and illustrate the significant improvements over the original path-based matching method.

Kernel functions for robust 3D surface registration with spectral embeddings

Registration of 3D surfaces is a critical step for shape analysis. Recent studies show that spectral representations based on intrinsic pairwise geodesic distances between points on surfaces are effective for registration and alignment due to their invariance under rigid transformations and articulations. Kernel functions are often applied to the pairwise geodesic distances to make the registration process based on spectral embedding robust to elastic deformations. The Gaussian kernel is most commonly used, but the effect of the choice of the kernel function has not been studied in the previous works. In this paper, we compare the results obtained with several different choices and show empirically that significant improvements can be achieved in shape registration with appropriate choices.

Viewing scenes occluded by smoke

In this paper, we focus on the problem of reconstructing images of scenes occluded by thick smoke. We propose a simple and effective algorithm that creates a single clear image of the scene given only a video sequence as input. Our method is based on two key observations. First, an increase in smoke density induces a decrease in both image contrast and color saturation. Measuring the decay of the high-frequency content in each video frame provides an effective way of quantifying the amount of contrast reduction. Secondly, the dynamic nature of the smoke causes the scene to be partially visible at times. By dividing the video sequence into subregions, our method is able to select the subregion-frame containing the least amount of smoke occlusion over time. Current experiments on different data sets show very promising results.

3D structure estimation from monocular video clips

This paper explores the idea of extracting three dimensional features from a previously recorded video, in an attempt to provide three dimensional information about a video clip in order to improve the performance of various video analysis tasks. Although video analysis is a very prevalent area of research, the use of 3D features is scarce in the literature due to the inherent difficulties associated with extracting accurate 3D representations of videos in cases where no previous knowledge of the scene or camera is known. In this paper, we present a framework that attempts to compute a dense three dimensional representation of a scene using only the available video sequence. Our proposed system exploits the motion of the camera in order to estimate the relative 3D positions of salient features located in the video frames. Additionally, we incorporate the use of appearance-based models to estimate their relative poses and fit a 3D human model into the reconstructed scenes. We test our method using various video clips obtained from online databases in order to show the feasibility of this approach.

Three Dimensional Information Extraction and Applications to Video Analysis

This chapter explores the idea of extracting three dimensional features from a video, and using such features to aid various video analysis and mining tasks. The use of 3D information in video analysis is scarce in the literature due to the inherent difficulties of such a system. When the only input to the system is a video stream with no previous knowledge of the scene or camera (a typical scenario in video analysis), computing an accurate 3D representation becomes a difficult task; however, several recently proposed methods can be applied to solving the problem efficiently, including simultaneous localization and mapping, structure from motion, and 3D reconstruction. These methods are surveyed and presented in the context of video analysis and demonstrated using videos from TRECVID 2005; their limitations are also discussed. Once an accurate 3D representation of a video is obtained, it can be used to increase the performance and accuracy of existing systems for various video analysis and mining tasks. Advantages of utilizing 3D representation are illustrated using several of these tasks, including shot boundary detection, object recognition, content-based video retrieval, as well as human activity recognition. The chapter concludes with a discussion on limitations of existing 3D methods and future research directions.