M. Dirk Robinson

Fundamental performance limits in image registration

The task of image registration is fundamental in image processing. It often is a critical preprocessing step to many modern image processing and computer vision tasks, and many algorithms and techniques have been proposed to address the registration problem. Often, the performances of these techniques have been presented using a variety of relative measures comparing different estimators, leaving open the critical question of overall optimality. In this paper, we present the fundamental performance limits for the problem of image registration as derived from the Cramer-Rao inequality. We compare the experimental performance of several popular methods with respect to this performance bound, and explain the fundamental tradeoff between variance and bias inherent to the problem of image registration. In particular, we derive and explore the bias of the popular gradient-based estimator showing how widely used multiscale methods for improving performance can be explained with this bias expression. Finally, we present experimental simulations showing the general rule-of-thumb performance limits for gradient-based image registration techniques.

Statistical performance analysis of super-resolution

Recently, there has been a great deal of work developing super-resolution algorithms for combining a set of low-quality images to produce a set of higher quality images. Either explicitly or implicitly, such algorithms must perform the joint task of registering and fusing the low-quality image data. While many such algorithms have been proposed, very little work has addressed the performance bounds for such problems. In this paper, we analyze the performance limits from statistical first principles using Crame/spl acute/r-Rao inequalities. Such analysis offers insight into the fundamental super-resolution performance bottlenecks as they relate to the subproblems of image registration, reconstruction, and image restoration.

Efficient Fourier-Wavelet Super-Resolution

Super-resolution (SR) is the process of combining multiple aliased low-quality images to produce a high-resolution high-quality image. Aside from registration and fusion of low-resolution images, a key process in SR is the restoration and denoising of the fused images. We present a novel extension of the combined Fourier-wavelet deconvolution and denoising algorithm ForWarD to the multiframe SR application. Our method first uses a fast Fourier-base multiframe image restoration to produce a sharp, yet noisy estimate of the high-resolution image. Our method then applies a space-variant nonlinear wavelet thresholding that addresses the nonstationarity inherent in resolution-enhanced fused images. We describe a computationally efficient method for implementing this space-variant processing that leverages the efficiency of the fast Fourier transform (FFT) to minimize complexity. Finally, we demonstrate the effectiveness of this algorithm for regular imagery as well as in digital mammography.

Optimal Registration Of Aliased Images Using Variable Projection With Applications To Super-Resolution

Accurate registration of images is the most important and challenging aspect of multiframe image restoration problems such as super-resolution. The accuracy of super-resolution algorithms is quite often limited by the ability to register a set of low-resolution images. The main challenge in registering such images is the presence of aliasing. In this paper, we analyse the problem of jointly registering a set of aliased images and its relationship to super-resolution. We describe a statistically optimal approach to multiframe registration which exploits the concept of variable projections to achieve very efficient algorithms. Finally, we demonstrate how the proposed algorithm offers accurate estimation under various conditions when standard approaches fail to provide sufficient accuracy for super-resolution.

Fast Local and Global Projection-Based Methods for Affine Motion Estimation

The demand for more effective compression, storage, and transmission of video data is ever increasing. To make the most effective use of bandwidth and memory, motion-compensated methods rely heavily on fast and accurate motion estimation from image sequences to compress not the full complement of frames, but rather a sequence of reference frames, along with “differences” between these frames which results from estimated frame-to-frame motion. Motivated by the need for fast and accurate motion estimation for compression, storage, and transmission of video as well as other applications of motion estimation, we present algorithms for estimating affine motion from video image sequences. Our methods utilize properties of the Radon transform to estimate image motion in a multiscale framework to achieve very accurate results. We develop statistical and computational models that motivate the use of such methods, and demonstrate that it is possible to improve the computational burden of motion estimation by more than an order of magnitude, while maintaining the degree of accuracy afforded by the more direct, and less efficient, 2-D methods.

Fast and robust super-resolution

In the last two decades, many papers have been published, proposing a variety methods of multiframe resolution enhancement. These methods are usually very sensitive to their assumed model of data and noise, which limits their utility. This paper reviews some of these methods and addresses their shortcomings. We propose a different implementation using L/sub 1/ norm minimization and robust regularization to deal with different data and noise models. This computationally inexpensive method is robust to errors in motion and blur estimation, and results in sharp edges. Simulation results confirm the effectiveness of our method and demonstrate its superiority to other robust super-resolution methods.

Theoretical foundations for joint digital-optical analysis of electro-optical imaging systems

We describe the mathematical and conceptual foundations for a novel methodology for jointly optimizing the design and analysis of the optics, detector, and digital image processing for imaging systems. Our methodology is based on the end-to-end merit function of predicted average pixel sum-squared error to find the optical and image processing parameters that minimize this merit function. Our approach offers several advantages over the traditional principles of optical design, such as improved imaging performance, expanded operating capabilities, and improved as-built performance.

Bias minimizing filter design for gradient-based image registration

Gradient-based image registration techniques represent a very popular class of approaches to registering pairs or sets of images. As the name suggests, these methods rely on image gradients to perform the task of registration. Very often, little attention is paid to the filters used to estimate image gradients. In this paper, we explore the relationship between such gradient filters and their effect on overall estimation performance in registering translated images. We propose a methodology for designing filters based on image content that minimize the estimator bias inherent to gradient-based image registration. We show that minimizing such bias improves the overall estimator performance in terms of mean square error (MSE) for high signal-to-noise ratio (SNR) scenarios. Finally, we propose a technique for designing such optimal gradient filters in the context of iterative multiscale image registration and verify their further improved performance.

Joint digital-optical design of superresolution multiframe imaging systems

Typical electro-optic imaging systems produce image aliasing artifacts. Superresolution algorithms process multiple aliased images to yield a single high-resolution image. We design imaging systems by jointly optimizing the optics and postprocessing to maximize such multiframe imaging performance. We describe efficient software methods that can be used to perform joint design by use of commercially available lens design software.

Joint Design of Lens Systems and Digital Image Processing

The traditional approach to designing an electro-optical imaging system involves first optimizing the lens subsystem using an optical measure of performance and second optimizing the image processing subsystem. Designing the system in this sequential fashion fails to exploit opportunities for efficient cooperation between the optical and digital systems. We introduce a novel framework for designing digital imaging systems and specifically an end-to-end merit function based on pixel-wise mean squared error. We describe how we adapt commercial ray tracing software to design the matched optical and image processing subsystems in a joint fashion while satisfying the constraints imposed on each of the subsystems.