Patrick Vandewalle

A frequency domain approach to registration of aliased images with application to super-resolution

Super-resolution algorithms reconstruct a high-resolution image from a set of low-resolution images of a scene. Precise alignment of the input images is an essential part of such algorithms. If the low-resolution images are undersampled and have aliasing artifacts, the performance of standard registration algorithms decreases. We propose a frequency domain technique to precisely register a set of aliased images, based on their low-frequency, aliasing-free part. A high-resolution image is then reconstructed using cubic interpolation. Our algorithm is compared to other algorithms in simulations and practical experiments using real aliased images. Both show very good visual results and prove the attractivity of our approach in the case of aliased input images. A possible application is to digital cameras where a set of rapidly acquired images can be used to recover a higher-resolution final image.

Reproducible research in signal processing

What should we do to raise the quality of signal processing publications to an even higher level? We believe it to be crucial to maintain the precision in describing our work in publications, ensured through a high-quality reviewing process. We also believe that if the experiments are performed on a large data set, the algorithm is compared to the state-of-the-art methods, the code and/or data are well documented and available online, we will all benefit and make it easier to build upon each others work. It is a clear win-win situation for our community: we will have access to more and more algorithms and can spend time inventing new things rather than recreating existing ones.

Super-Resolution From Unregistered and Totally Aliased Signals Using Subspace Methods

In many applications, the sampling frequency is limited by the physical characteristics of the components: the pixel pitch, the rate of the analog-to-digital (AID) converter, etc. A low- pass filter is usually applied before the sampling operation to avoid aliasing. However, when multiple copies are available, it is possible to use the information that is inherently present in the aliasing to reconstruct a higher resolution signal. If the different copies have unknown relative offsets, this is a nonlinear problem in the offsets and the signal coefficients. They are not easily separable in the set of equations describing the super-resolution problem. Thus, we perform joint registration and reconstruction from multiple unregistered sets of samples. We give a mathematical formulation for the problem when there are M sets of N samples of a signal that is described by L expansion coefficients. We prove that the solution of the registration and reconstruction problem is generically unique if MN ges L + M - 1. We describe two subspace-based methods to compute this solution. Their complexity is analyzed, and some heuristic methods are proposed. Finally, some numerical simulation results on one- and two-dimensional signals are given to show the performance of these methods.

Superresolution images reconstructed from aliased images

In this paper, we present a simple method to almost quadruple the spatial resolution of aliased images. From a set of four low resolution, undersampled and shifted images, a new image is constructed with almost twice the resolution in each dimension. The resulting image is aliasing-free. A small aliasing-free part of the frequency domain of the images is used to compute the exact subpixel shifts. When the relative image positions are known, a higher resolution image can be constructed using the Papoulis-Gerchberg algorithm. The proposed method is tested in a simulation where all simulation parameters are well controlled, and where the resulting image can be compared with its original. The algorithm is also applied to real, noisy images from a digital camera. Both experiments show very good results.

How to take advantage of aliasing in bandlimited signals

In signal processing systems, aliasing is normally treated as a disturbing signal. That motivates the need for effective analog, optical and digital anti-aliasing filters. However, aliasing also conveys valuable information on the signal above the Nyquist frequency. Hence, an effective processing of the samples, based on a model of the input signal, would virtually allow the sampling frequency to be increased using slower and cheaper converters. We present such an algorithm for bandlimited signals that are sampled below twice the maximum signal frequency. Using a subspace method in the frequency domain, we show that these signals can be reconstructed from multiple sets of samples. The offset between the sets is unknown and can have arbitrary values. This approach can be applied to the creation of super-resolution images from sets of low resolution images. In this application, registration parameters have to be computed from aliased images. We show that parameters and high resolution images can be computed precisely, even when high levels of aliasing are present on the low resolution images.

Super-resolution from highly undersampled images

Aliasing artifacts in images are visually very disturbing. Therefore, most imaging devices apply a low-pass filter before sampling. This removes all aliasing from the image, but it also creates a blurred image. Actually, all the image information above half the sampling frequency is removed. In this paper, we present a new method for the reconstruction of a high resolution image from a set of highly undersampled and thus aliased images. We use the information in the entire frequency spectrum, including the aliased part, to create a sharp, high resolution image. The unknown relative shifts between the images are computed using a subspace projection approach. We show that the projection can be decomposed into multiple projections onto smaller subspaces. This allows for a considerable reduction of the overall computational complexity of the algorithm. A high resolution image can then be reconstructed from the registered low resolution images. Simulation results show the validity of our algorithm.

Double resolution from a set of aliased images

In this paper, we present a super-resolution method to approximately double image resolution in both dimensions from a set of four low resolution, aliased images. The camera is shifted and rotated by small amounts between the different image captures. Only the low frequency, aliasing-free part of the images is used to find the shift and rotation parameters. When the images are registered, it is possible to reconstruct a higher resolution, aliasing-free image from the four low resolution images using cubic interpolation. We applied our algorithm in a simulation, where all parameters are known and controlled, as well as in a practical experiment using images taken with a real digital camera. The results obtained in both tests prove the validity of our method.

Joint Demosaicing and Super-Resolution Imaging from a Set of Unregistered Aliased Images

We present a new algorithm that performs demosaicing and super-resolution jointly from a set of raw images sampled with a color filter array. Such a combined approach allows us to compute the alignment parameters between the images on the raw camera data before interpolation artifacts are introduced. After image registration, a high resolution color image is reconstructed at once using the full set of images. For this, we use normalized convolution, an image interpolation method from a nonuniform set of samples. Our algorithm is tested and compared to other approaches in simulations and practical experiments.

Experiences with Reproducible Research in Various Facets of Signal Processing Research

How often have you been able to implement an algorithm as it is described in a paper? And when you did, were you confident that you had exactly the same parameter values and results as the authors of the paper? All too often, articles do not describe all the details of an algorithm and thus prohibit an implementation by someone else. In this paper, we describe our experience with reproducible research, a paradigm to allow other people to reproduce with minimal effort the results we have obtained. We discuss both the reproducibility of data and algorithms, and give examples for each of them. The effort required to make research reproducible is compensated by a higher visibility and impact of the results.

Subspace-based methods for image registration and super-resolution

Super-resolution algorithms combine multiple low resolution images into a single high resolution image. They have received a lot of attention recently in various application domains such as HDTV, satellite imaging, and video surveillance. These techniques take advantage of the aliasing present in the input images to reconstruct high frequency information of the resulting image. One of the major challenges in such algorithms is a good alignment of the input images: subpixel precision is required to enable accurate reconstruction. In this paper, we give an overview of some subspace techniques that address this problem. We first formulate super-resolution in a multichannel sampling framework with unknown offsets. Then, we present three registration methods: one approach using ideas from variable projections, one using a Fourier description of the aliased signals, and one using a spline description of the sampling kernel. The performance of the algorithms is evaluated in numerical simulations.