Joshua Gluckman

Ego-motion and omnidirectional cameras

Recent research in image sensors has produced cameras with very large fields of view. An area of computer vision research which will benefit from this technology is the computation of camera motion (ego-motion) from a sequence of images. Traditional cameras stiffer from the problem that the direction of translation may lie outside of the field of view, making the computation of camera motion sensitive to noise. In this paper, we present a method for the recovery of ego-motion using omnidirectional cameras. Noting the relationship between spherical projection and wide-angle imaging devices, we propose mapping the image velocity vectors to a sphere, using the Jacobian of the transformation between the projection model of the camera and spherical projection. Once the velocity vectors are mapped to a sphere, we show how existing ego-motion algorithms can be applied and present some experimental results. These results demonstrate the ability to compute ego-motion with omnidirectional cameras.

Planar catadioptric stereo: geometry and calibration

By using mirror reflections of a scene, stereo images can be captured with a single camera (catadioptric stereo). Single camera stereo provides both geometric and radiometric advantages over traditional two camera stereo. In this paper we discuss the geometry and calibration of catadioptric stereo with two planar mirrors and show how the relative orientation, the epipolar geometry and the estimation of the focal length are constrained by planar motion. In addition, we have implemented a real-time system which demonstrates the viability of stereo with mirrors as an alternative to traditional two camera stereo.

Catadioptric stereo using planar mirrors

By using mirror reflections of a scene, stereo images can be captured with a single camera (catadioptric stereo). In addition to simplifying data acquisition single camera stereo provides both geometric and radiometric advantages over traditional two camera stereo. In this paper, we discuss the geometry and calibration of catadioptric stereo with two planar mirrors. In particular, we will show that the relative orientation of a catadioptric stereo rig is restricted to the class of planar motions thus reducing the number of external calibration parameters from 6 to 5. Next we derive the epipolar geometry for catadioptric stereo and show that it has 6 degrees of freedom rather than 7 for traditional stereo. Furthermore, we show how focal length can be recovered from a single catadioptric image solely from a set of stereo correspondences. To test the accuracy of the calibration we present a comparison to Tsai camera calibration and we measure the quality of Euclidean reconstruction. In addition, we will describe a real-time system which demonstrates the viability of stereo with mirrors as an alternative to traditional two camera stereo.

Rectified catadioptric stereo sensors

It has been previously shown how mirrors can be used to capture stereo images with a single camera, an approach termed catadioptric stereo. In this paper we present novel catadioptric sensors which use mirrors to produce rectified stereo images. The scan-line correspondence of these images benefits real-time stereo by avoiding the computational cost and image degradation due to resampling when rectification is performed after image capture. First, we develop a theory which determines the number of mirrors that must be used and the constraints on those mirrors that must be satisfied to obtain rectified stereo images with a single camera. Then we discuss in detail the use of both one and three mirrors. In addition, we show how the mirrors should be placed in order to minimize sensor size for a given baseline, an important design consideration.

Rectifying transformations that minimize resampling effects

Image rectification is the process of warping a pair of stereo images in order to align the epipolar lines with the scan-lines of the images. Once a pair of images is rectified, stereo matching can be implemented in an efficient manner. Given the epipolar geometry, it is straightforward to define a rectifying transformation, however, many transformations will lead to unwanted image distortions. In this paper, we present a novel method for stereo rectification that determines the transformation that minimizes the effects of resampling that can impede stereo matching. The effects we seek to minimize are the loss of pixels due to under-sampling and the creation of new pixels due to over-sampling. To minimize these effects we parameterize the family of rectification transformations and solve for the one that minimizes the change in local area integrated over the area of the images.

Higher order whitening of natural images

Natural images are approximately scale invariant resulting in long range statistical regularities that typically obey a power law. For example, images have considerable regularity in their second order spatial correlations as measured by the power spectrum. Processing images to remove these expected correlations is known as whitening an image. Because the expected value of the power spectrum has a regular form (a power law) linear processing such as convolution can be used to whiten an image. After whitening an image, higher order regularities that cannot be removed with linear processing still exist in the form of correlations in the magnitude. In this paper it is shown that these correlations also obey a power law and a non-linear method is used to remove them, a process referred to as higher order whitening. The method is invertible demonstrating that while redundancy is removed no information is lost. Experiments are given showing that after higher order whitening the coefficients can be severely quantized yet a good reconstruction is possible despite the nonlinearities.

Catadioptric video sensors

Conventional video cameras have limited fields of view which make them restrictive in a variety of applications. A catadioptric sensor uses a combination of lenses and mirrors placed in a carefully arranged configuration to capture a much wider field of view. At Columbia University, we have developed a wide range of catadioptric sensors. Some of these sensors have been designed to produce unusually large fields of view. Others have been constructed for the purpose of depth computation. All our sensors perform in real time using just a PC.