Philippe Thévenaz

A pyramid approach to subpixel registration based on intensity

We present an automatic subpixel registration algorithm that minimizes the mean square intensity difference between a reference and a test data set, which can be either images (two-dimensional) or volumes (three-dimensional). It uses an explicit spline representation of the images in conjunction with spline processing, and is based on a coarse-to-fine iterative strategy (pyramid approach). The minimization is performed according to a new variation (ML*) of the Marquardt-Levenberg algorithm for nonlinear least-square optimization. The geometric deformation model is a global three-dimensional (3-D) affine transformation that can be optionally restricted to rigid-body motion (rotation and translation), combined with isometric scaling. It also includes an optional adjustment of image contrast differences. We obtain excellent results for the registration of intramodality positron emission tomography (PET) and functional magnetic resonance imaging (fMRI) data. We conclude that the multiresolution refinement strategy is more robust than a comparable single-stage method, being less likely to be trapped into a false local optimum. In addition, our improved version of the Marquardt-Levenberg algorithm is faster.

Optimization of mutual information for multiresolution image registration

We propose a new method for the intermodal registration of images using a criterion known as mutual information. Our main contribution is an optimizer that we specifically designed for this criterion. We show that this new optimizer is well adapted to a multiresolution approach because it typically converges in fewer criterion evaluations than other optimizers. We have built a multiresolution image pyramid, along with an interpolation process, an optimizer, and the criterion itself, around the unifying concept of spline-processing. This ensures coherence in the way we model data and yields good performance. We have tested our approach in a variety of experimental conditions and report excellent results. We claim an accuracy of about a hundredth of a pixel under ideal conditions. We are also robust since the accuracy is still about a tenth of a pixel under very noisy conditions. In addition, a blind evaluation of our results compares very favorably to the work of several other researchers.

Interpolation revisited [medical images application]

Based on the theory of approximation, this paper presents a unified analysis of interpolation and resampling techniques. An important issue is the choice of adequate basis functions. The authors show that, contrary to the common belief, those that perform best are not interpolating. By opposition to traditional interpolation, the authors call their use generalized interpolation; they involve a prefiltering step when correctly applied. The authors explain why the approximation order inherent in any basis function is important to limit interpolation artifacts. The decomposition theorem states that any basis function endowed with approximation order ran be expressed as the convolution of a B spline of the same order with another function that has none. This motivates the use of splines and spline-based functions as a tunable way to keep artifacts in check without any significant cost penalty. The authors discuss implementation and performance issues, and they provide experimental evidence to support their claims.

Elastic registration of biological images using vector-spline regularization

We present an elastic registration algorithm for the alignment of biological images. Our method combines and extends some of the best techniques available in the context of medical imaging. We express the deformation field as a B-spline model, which allows us to deal with a rich variety of deformations. We solve the registration problem by minimizing a pixelwise mean-square distance measure between the target image and the warped source. The problem is further constrained by way of a vector-spline regularization which provides some control over two independent quantities that are intrinsic to the deformation: its divergence, and its curl. Our algorithm is also able to handle soft landmark constraints, which is particularly useful when parts of the images contain very little information or when its repartition is uneven. We provide an optimal analytical solution in the case when only landmarks and smoothness considerations are taken into account. We have applied our approach to perform the elastic registration of images such as electrophoretic gels and fly embryos. The validation of the results by experts has been favorable in all cases.

Convolution-based interpolation for fast, high-quality rotation of images

This paper focuses on the design of fast algorithms for rotating images and preserving high quality. The basis for the approach is a decomposition of a rotation into a sequence of one-dimensional translations. As the accuracy of these operations is critical, we introduce a general theoretical framework that addresses their design and performance. We also investigate the issue of optimality and present an improved least-square formulation of the problem. This approach leads to a separable three-pass implementation of a rotation using one-dimensional convolutions only. We provide explicit filter formulas for several continuous signal models including spline and bandlimited representations. Finally, we present rotation experiments and compare the currently standard techniques with the various versions of our algorithm. Our results indicate that the present algorithm in its higher-order versions outperforms all standard high-accuracy methods of which we are aware, both in terms of speed and quality. Its computational complexity increases linearly with the order of accuracy. The best-quality results are obtained with the sine-based algorithm, which can be implemented using simple one-dimensional FFTs.

Variational B-Spline Level-Set: A Linear Filtering Approach for Fast Deformable Model Evolution

In the field of image segmentation, most level-set-based active-contour approaches take advantage of a discrete representation of the associated implicit function. We present in this paper a different formulation where the implicit function is modeled as a continuous parametric function expressed on a B-spline basis. Starting from the active-contour energy functional, we show that this formulation allows us to compute the solution as a restriction of the variational problem on the space spanned by the B-splines. As a consequence, the minimization of the functional is directly obtained in terms of the B-spline coefficients. We also show that each step of this minimization may be expressed through a convolution operation. Because the B-spline functions are separable, this convolution may in turn be performed as a sequence of simple 1-D convolutions, which yields an efficient algorithm. As a further consequence, each step of the level-set evolution may be interpreted as a filtering operation with a B-spline kernel. Such filtering induces an intrinsic smoothing in the algorithm, which can be controlled explicitly via the degree and the scale of the chosen B-spline kernel. We illustrate the behavior of this approach on simulated as well as experimental images from various fields.

A pyramid approach to sub-pixel image fusion based on mutual information

We investigate aspects of multi-modal image registration based on a new criterion named mutual information (or sometimes Shannon information). This criterion is intensity-based and requires no landmarks; hence, its application can be automated without resorting to segmentation. We present a form amenable to derivation with respect to the geometric transformation parameters (affine transformation). This form involves Parzen windows; we explore the dependence of the registration accuracy on these windows and propose that they be tuned to each resolution level in a pyramid approach. We conduct experiments and show that both the window width and the number of windows is relevant. In addition, we show that it is beneficial to use a spline-based high-order interpolation scheme for applying the geometric transformation.

Iterative multi-scale registration without landmarks

We present an automatic sub-pixel registration algorithm that minimizes the mean square difference of intensities between a reference and a test data set (volumes or images). It uses spline processing, is based on a coarse-to-fine pyramid strategy, and performs minimization according to a variation of the iterative Marquardt-Levenberg (1963) scheme. The geometric deformation model is a general affine transformation that one may optionally restrict to a rigid-body (isometric scale, rotation and translation), procrustean (rotation and translation) or translational case; it also includes an optional parameter for the linear adaptation of intensity. We present several PET and fMRI experiments and show that this algorithm provides excellent results. We conclude that the multi-resolution refinement strategy is faster and more robust than a comparable single-scale one.

An efficient mutual information optimizer for multiresolution image registration

We propose a new optimizer in the context of multimodal image registration. The optimized criterion is the mutual information between the images to be align. This criterion requires that their joint histogram be available. For its computation, we introduce differentiable and separable Parzen windows that satisfy the partition of unity. Along with a continuous model of the images based on splines, this allows us to derive exact and tractable expressions for the gradient and the Hessian of the criterion. Then, we develop an optimizer based on the Marquardt-Levenberg (1963) strategy. Our new optimizer is specific to mutual information, in the same sense that Marquardt-Levenberg is specific to least-squares. We show that our optimizer is particularly well-adapted to an iterative coarse-to fine approach. We validate its accuracy by comparing its performance to that of several results available in the literature.

Registration and statistical analysis of PET images using the wavelet transform

We have described a general procedure for the processing and analysis of PET data. We have used the multiresolution framework of the wavelet transform to derive new solutions for the two main processing steps. The first task was to align the various brain images using a general affine deformation model. Our registration procedure uses a continuous polynomial spline image model and takes advantage of the multiresolution structure of the underlying function spaces. This method implements a nonlinear least squares optimization technique with a coarse-to-fine iteration strategy that substantially improves the overall performance of the algorithm. The second task was to analyze the series of registered images and to detect the between group differences in metabolic brain activity. We chose to take advantage of the orthogonality and localization properties of the wavelet transform. Our approach was to apply this transform to the group-difference image and identify the wavelet channels that are globally significantly different from noise. >