Julian Magarey

Motion estimation using a complex-valued wavelet transform

This paper describes a new motion estimation algorithm that is potentially useful for both computer vision and video compression applications. It is hierarchical in structure, using a separable two-dimensional (2-D) discrete wavelet transform (DWT) on each frame to efficiently construct a multiresolution pyramid of subimages. The DWT is based on a complex-valued pair of four-tap FIR filters with Gabor-like characteristics. The resulting complex DWT (CDWT) effectively implements an analysis by an ensemble of Gabor-like filters with a variety of orientations and scales. The phase difference between the subband coefficients of each frame at a given subpel bears a predictable relation to a local translation in the region of the reference frame subtended by that subpel. That relation is used to estimate the displacement field at the coarsest scale of the multiresolution pyramid. Each estimate is accompanied by a directional confidence measure in the form of the parameters of a quadratic matching surface. The initial estimate field is progressively refined by a coarse-to fine strategy in which finer scale information is appropriately incorporated at each stage. The accuracy, efficiency, and robustness of the new algorithm are demonstrated in comparison testing against hierarchical implementations of intensity gradient-based and fractional-precision block matching motion estimators.

Wavelet Transforms in Image Processing

This chapter is designed to be partly tutorial in nature and partly a summary of recent work by the authors in applying wavelets to various image processing problems. The tutorial part describes the filter-bank implementation of the discrete wavelet transform (DWT) and shows that most wavelets which permit perfect reconstruction are similar in shape and scale. We then discuss an important drawback of these wavelet transforms, which is that the distribution of energy between coefficients at different scales is very sensitive to shifts in the input data. We propose the Complex Wavelet Transform (CWT) as a solution to this problem and show how it may be applied in two dimensions. Finally we give brief details of applications of the CWT to motion estimation and image de-noising.

Motion estimation using complex wavelets

This paper describes a new wavelet-based approach to the motion estimation problem for digital video. A complex- valued discrete wavelet transform is used to decompose each frame into a subsampled directionally bandpass filtered hierarchy. The transform is defined so that at each level there is an approximate correspondence between local translation and coefficient phase shift. This relationship is used to estimate motion with each orientation subband. The estimates are combined over all orientations and scales using a coarse-to-fine refinement strategy to produce a fractional-pel accurate motion field with a directional confidence measure. The technique is suitable for video compression schemes and can also be used for stereo vision and image registration.

Multiresolution stereo image matching using complex wavelets

This paper describes a multiresolution image-matching strategy, based on the complex discrete wavelet transform (CDWT), to derive a dense disparity field with hierarchical (coarse-to-fine) refinement. The CDWT feature space efficiently provides fractionally accurate matching results which are robust to typical image formation perturbations such as offsets, global scaling, and additive noise. At each level of the hierarchy, the disparity field is regularised to provide a global compromise between feature similarity and disparity field continuity, resulting in feature-sensitive smoothing. The algorithm is well suited to analysing facial images, for which we demonstrate striking reconstruction results.

An improved motion estimation algorithm using complex wavelets

This paper describes some improvements to our hierarchical wavelet-based algorithm for estimating motion in digital video (Magarey and Kingsbury, 1996). The algorithm uses a separable discrete wavelet transform (DWT) to analyse each frame into a multiresolution pyramid of subimages. Our analysis filters are Gabor-like (complex valued), so each subband at a given level of the pyramid contains information from a distinct spatial orientation. Starting at the coarsest (most sparsely sampled) level, our algorithm produces a single field of motion estimates from the oriented subimages at that level. Accompanying each motion estimate is a directionally dependent confidence measure. The estimate field is then progressively refined by incorporating the finer level subimages, using a standard coarse-to-fine control strategy. This paper describes an iterative strategy including a new interpolation scheme which increases accuracy at a given level, and a curvature correction method which improves the propagation of estimates from coarse to fine.

Robust Motion Estimation Using Chrominance Information in Color Image Sequences

This paper describes a method for incorporating the chrominance information when estimating motion in a colour image sequence. It is based on a Maximum-Likelihood (ML) formulation of the motion estimation problem which assumes homogeneous additive Gaussian noise in each colour component, with known inter-field correlation statistics. It defines a noise-decorrelating colour space transform which provides a simple implementation of the ML formulation. Results for noisy synthesised colour sequences with known motion and noise statistics demonstrate the superiority of the exact ML formulation over straightforward, unweighted three-component estimation, most noticeably in high noise conditions.

Three-dimensional video segmentation using a variational method

Spatio-temporal segmentation of streaming video is performed using a novel 3D extension of a 2D Mumford-Shah region-merging segmentation algorithm. By treating the video segmentation problem as a pure 3D segmentation problem, the algorithm can provide both intra-frame segmentation and interframe region correspondences in a single, unified framework. The Mumford-Shah functional formulation leads to improved segmentation results compared to existing approaches, and its theoretical foundations allow a simple graph-based implementation and elegant solutions to some important subproblems.

Optimal schemes for motion estimation on colour image sequences

This paper describes a method for incorporating the chrominance information when estimating the motion in a colour image sequence. It is based on a maximum likelihood formulation of the motion estimation problem which assumes homogeneous additive Gaussian noise in each colour component, with known inter-field correlation statistics. The formulation is applied to the complex-wavelet-domain matching algorithm of Magarey and Kingsbury (see Proc. IEEE Int. Conf. on Image Processing, p.969-72, 1996). We also define a noise-decorrelating colour space transform which provides a simple implementation of the ML formulation in the wavelet domain. Results for noisy synthesised colour sequences with known motion and noise statistics demonstrate the superiority of the exact ML formulation over straightforward, unweighted three-component estimation, most noticeably in high noise conditions.

Multiresolution Phase-Based Bidirectional Stereo Matching with Provision for Discontinuity and Occlusion

Stereo image matching is the most robust and domain-independent way of reconstructing object surfaces from perspective images. Dense feature space matching is desirable because it constructs a disparity field without arbitrary selection of interest points for individual matching. Multiresolution or coarse-to-fine strategies have been shown to be effective in overcoming the inherent ambiguities of dense feature space matching. We use Gabor phase as the basis for dense multiresolution matching, as it is a stable, ubiquitous feature of a signal. The use of Gabor-like complex wavelets enables the efficient transformation of images to Gabor-phase-based feature pyramids. Disparity discontinuity and occlusions are the major potential disruptions to coarse-to-fine image matching. Through perspective geometry, we show that depth discontinuities are the direct cause of the discontinuity of disparity field, and bidirectional matching is required to detect disparity discontinuities (and their dual counterpart, occlusions) in both views. To handle these, a global objective function incorporating feature similarity, disparity smoothness, and discontinuity/occlusions is established under the maximuma posterioriprobability criterion and equivalently transformed into a minimum description length criterion. A general procedure, using stochastic relaxation with special provision for occlusions, is developed for minimising the objective function. The result is a disparity field which is regularised in the continuous regions, while the discontinuities and occlusions are detected and preserved. Some results from an aerial terrain image pair indicate the applicability of this approach.

Robust motion estimation using complex wavelets

This paper describes a new approach to the problem of estimating motion in video image sequences. The algorithm operates on the complex coeecients obtained by applying the new Complex Discrete Wavelet Transform to each frame. This transform implements an eecient analysis by an ensemble of Gabor-like lters of diierent orientation and scales. The Gabor-like form of the lters means that local translations induce phase rotations in the corresponding complex coeecients. This relationship is used to estimate motion at each subpixel of each scale. The estimates are combined over all orientations and scales using a coarse-tone reenement strategy to produce a fractional -pel accurate motion eld with a directional conn-dence measure. Compared with intensity-based algorithms, the new algorithm is robust to simple image formation perturbations .