Padmanabhan Anandan

The Robust Estimation of Multiple Motions

Most approaches for estimating optical flow assume that, within a finite image region, only a single motion is present. Thissingle motion assumptionis violated in common situations involving transparency, depth discontinuities, independently moving objects, shadows, and specular reflections. To robustly estimate optical flow, the single motion assumption must be relaxed. This paper presents a framework based onrobust estimationthat addresses violations of the brightness constancy and spatial smoothness assumptions caused by multiple motions. We show how therobust estimation frameworkcan be applied to standard formulations of the optical flow problem thus reducing their sensitivity to violations of their underlying assumptions. The approach has been applied to three standard techniques for recovering optical flow: area-based regression, correlation, and regularization with motion discontinuities. This paper focuses on the recovery of multiple parametric motion models within a region, as well as the recovery of piecewise-smooth flow fields, and provides examples with natural and synthetic image sequences.

A Computational Framework and an Algorithm for the Measurement of Visual

The robust measurement of visual motion from digitized image sequences has been an important but difficult problem in computer vision. This paper describes a hierarchical computational framework for the determination of dense displacement fields from a pair of images, and an algorithm consistent with that framework. Our framework is based on a scale-based separation of the image intensity information and the process of measuring motion. The large-scale intensity information is first used to obtain rough estimates of image motion, which are then refined by using intensity information at smaller scales. The estimates are in the form of displacement (or velocity) vectors for pixels and are accompanied by a direction-dependent confidence measure. A smoothness constraint is employed to propagate measurements with high confidence to neighboring areas where the confidences are low. At all levels, the computations are pixel-parallel, uniform across the image, and based on information from a small neighborhood of a pixel. Results of applying our algorithm to pairs of real images are included. In addition to our own matching algorithm, we also show that two different hierarchical gradient-based algorithms are consistent with our framework.

A framework for the robust estimation of optical flow

The authors consider the problem of robustly estimating optical flow from a pair of images using a new framework based on robust estimation which addresses violations of the brightness constancy and spatial smoothness assumptions. They also show the relationship between the robust estimation framework and line-process approaches for coping with spatial discontinuities. In doing so, the notion of a line process is generalized to that of an outlier process that can account for violations in both the brightness and smoothness assumptions. A graduated non-convexity algorithm is presented for recovering optical flow and motion discontinuities. The performance of the robust formulation is demonstrated on both synthetic data and natural images.<<ETX>>

Mosaic based representations of video sequences and their applications

Recently, there has been a growing interest in the use of mosaic images to represent the information contained in video sequences. The paper systematically investigates how to go beyond thinking of the mosaic simply as a visualization device, but rather as a basis for efficient representation of video sequences. We describe two different types of mosaics called the static and the dynamic mosaic that are suitable for different needs and scenarios. We discuss a series of extensions to these basic mosaics to provide representations at multiple spatial and temporal resolutions and to handle 3D scene information. We describe techniques for the basic elements of the mosaic construction process, namely alignment, integration, and residual analysis. We describe several applications of mosaic representations including video compression, enhancement, enhanced visualization, and other applications in video indexing, search, and manipulation.<<ETX>>

A unified approach to moving object detection in 2D and 3D scenes

The detection of moving objects is important in many tasks. Previous approaches to this problem can be broadly divided into two classes: 2D algorithms which apply when the scene can be approximated by a flat surface and/or when the camera is only undergoing rotations and zooms, and 3D algorithms which work well only when significant depth variations are present in the scene and the camera is translating. We describe a unified approach to handling moving object detection in both 2D and 3D scenes, with a strategy to gracefully bridge the gap between those two extremes. Our approach is based on a stratification of the moving object detection problem into scenarios which gradually increase in their complexity. We present a set of techniques that match the above stratification. These techniques progressively increase in their complexity, ranging from 2D techniques to more complex 3D techniques. Moreover, the computations required for the solution to the problem at one complexity level become the initial processing step for the solution at the next complexity level. We illustrate these techniques using examples from real-image sequences.

Robust dynamic motion estimation over time

A novel approach to incrementally estimating visual motion over a sequence of images is presented. The authors start by formulating constraints on image motion to account for the possibility of multiple motions. This is achieved by exploiting the notions of weak continuity and robust statistics in the formulation of a minimization problem. The resulting objective function is non-convex. Traditional stochastic relaxation techniques for minimizing such functions prove inappropriate for the task. A highly parallel incremental stochastic minimization algorithm is presented which has a number of advantages over previous approaches. The incremental nature of the scheme makes it dynamic and permits the detection of occlusion and disocclusion boundaries.<<ETX>>

Efficient representations of video sequences and their applications

Abstract Recently, there has been a growing interest in the use of mosaic images to represent the information contained in video sequences. This paper systematically investigates how to go beyond thinking of the mosaic simply as a visualization device, but rather as a basis for an efficient and complete representation of video sequences. We describe two different types of mosaics called the static and the dynamic mosaics that are suitable for different needs and scenarios. These two types of mosaics are unified and generalized in a mosaic representation called the temporal pyramid. To handle sequences containing large variations in image resolution, we develop a multiresolution mosaic. We discuss a series of increasingly complex alignment transformations (ranging from 2D to 3D and layers) for making the mosaics. We describe techniques for the basic elements of the mosaic construction process, namely sequence alignment, sequence integration into a mosaic image, and residual analysis to represent information not captured by the mosaic image. We describe several powerful video applications of mosaic representations including video compression, video enhancement, enhanced visualization, and other applications in video indexing, search, and manipulation.

About Direct Methods

This report provides a brief summary of the review of “Direct Methods”, which was presented by Michal Irani and P. Anandan.

Representation of scenes from collections of images

The goal of computer vision is to extract information about the world from collections of images. This information might be used to recognize or manipulate objects, to control movement through the environment, to measure or determine the condition of objects, and for many other purposes. The goal of this paper is to consider the representation of information derived from a collection of images and how it may support some of these tasks. By collection of images we mean any set of images relevant to a given scene. This includes video sequences, multiple images from a single still camera, or multiple images from different cameras. The central thesis of this paper is that the traditional approach to representation of information about scenes by relating each image to an abstract three dimensional coordinate system may not always be appropriate. An approach that more directly represents the relationships among the collection of images has a number of advantages. These relationships can also be computed using practical and efficient algorithms. We present a hierarchical framework for scene representation. We develop the algorithms used to build these representations and demonstrate results on real image sequences. Finally, the application of these representations to real world problems is discussed.

VIDEO COMPRESSION USING MOSAIC REPRESENTATIONS

Abstract We describe a technique for video compression based on a mosaic image representation obtained by aligning all frames of a video sequence, giving a panoramic view of the scene. We describe two types of mosaics, static and dynamic, which are suited for storage and transmission applications, respectively. In each case, the mosaic construction process aligns the images using a global parametric motion transformation, usually canceling the effect of camera motion on the dominant portion of the scene. The residual motions that are not compensated by the parametric motion are then analyzed for their significance and coded. The mosaic representation exploits large scale spatial and temporal correlations in image sequences. In many applications where there is significant camera motion (e.g., remote surveillance), it performs substantially better than traditional interframe compression methods and offers the potential for very low bit-rate transmission. In storage applications, such as digital libraries and video editing environments, it has the additional benefit of enabling direct access and retrieval of single frames at a time.