J. B. Antoine Maintz

A survey of medical image registration

The purpose of this paper is to present a survey of recent (published in 1993 or later) publications concerning medical image registration techniques. These publications will be classified according to a model based on nine salient criteria, the main dichotomy of which is extrinsic versus intrinsic methods. The statistics of the classification show definite trends in the evolving registration techniques, which will be discussed. At this moment, the bulk of interesting intrinsic methods is based on either segmented points or surfaces, or on techniques endeavouring to use the full information content of the images involved.

Interpolation Artefacts in Mutual Information-Based Image Registration

Image registration requires the transformation of one image to another so as to spatially align the two images. This involves interpolation to estimate gray values of one of the images at positions other than the grid points. When registering two images that have equal grid distances in one or more dimensions, the grid points can be aligned in those dimensions for certain geometric transformations. Consequently, the number of times interpolation is required to compute the registration measure of two images is dependent on the image transformation. When an entropy-based registration measure, such as mutual information, is plotted as a function of the transformation, it will show sudden changes in value for grid-aligning transformations. Such patterns of local extrema impede the registration optimization process. More importantly, they rule out subvoxel accuracy. In this paper, two frequently applied interpolation methods in mutual information-based image registration are analyzed, viz. linear interpolation and partial volume interpolation. It is shown how the registration function depends on the interpolation method and how a slight resampling of one of the images may drastically improve the smoothness of this function.

General multimodal elastic registration based on mutual information

Recent studies indicate that maximizing the mutual information of the joint histogram of two images is an accurate and robust way to rigidly register two mono- or multimodal images. Using mutual information for registration directly in a local manner is often not admissible owing to the weakened statistical power of the local histogram compared to a global one. We propose to use a global joint histogram based on optimized mutual information combined with a local registration measure to enable local elastic registration.

Comparison of edge-based and ridge-based registration of CT and MR brain images

In modern medicine, several different imaging techniques are frequently employed in the study of a single patient. This is useful, since different images show complementary information on the functionality and/or structure of the anatomy examined. This very difference between modalities, however, complicates the problem of proper registration of the images involved, and rules out the most basic approaches--like direct grey value correlation--to achieve registration. The observation that some common structures will always exist is supportive of the statement that registration may be feasible using edges or ridges present in the images. The existence of such structures defined in the binary sense is questionable, however, and their extraction from images requires a segmentation by definition. In this paper we propose to use fuzzy edgeness and ridgeness images, thus avoiding the need for segmentation and using more of the available information from the original images. We will show that such fuzzy images can be used to achieve accurate registration. Several ridgeness and edgeness computing operators were compared. The best registration results were obtained using a gradient magnitude operator.

Active-shape-model-based segmentation of abdominal aortic aneurysms in CTA images

An automated method for the segmentation of thrombus in abdominal aortic aneurysms from CTA data is presented. The method is based on Active Shape Model (ASM) fitting in sequential slices, using the contour obtained in one slice as the initialisation in the adjacent slice. The optimal fit is defined by maximum correlation of grey value profiles around the contour in successive slices, in contrast to the original ASM scheme as proposed by Cootes and Taylor, where the correlation with profiles from training data is maximised. An extension to the proposed approach prevents the inclusion of low-intensity tissue and allows the model to refine to nearby edges. The applied shape models contain either one or two image slices, the latter explicitly restricting the shape change from slice to slice. To evaluate the proposed methods a leave-one-out experiment was performed, using six datasets containing 274 slices to segment. Both adapted ASM schemes yield significantly better results than the original scheme (p<0.0001). The extended slice correlation fit of a one-slice model showed best overall performance. Using one manually delineated image slice as a reference, on average a number of 29 slices could be automatically segmented with an accuracy within the bounds of manual inter-observer variability.

Image Registration by Maximization of Combined Mututal Information and Gradient Information

Despite generally good performance, mutual information has also been shown by several researchers to lack robustness for certain registration problems. A possible cause may be the absence of spatial information in the measure. The present paper proposes to include spatial information by combining mutual information with a term based on the image gradient of the images to be registered. The gradient term not only seeks to align locations of high gradient magnitude, but also aims for a similar orientation of the gradients at these locations.

Image fusion using geometrical features

This paper describes a new approach to register images obtained from different modalities. Differential operators in scale space are used to extract geometric features from the images corresponding to similar structures. The resulting feature images may be matched by minimizing some function of the distances between the features in the respective images. Our first application concerns matching of brain images. We discuss a differential operator that produces ridge-like feature images from which the center curve of the cranium is easily extracted in CT and MRI. Results of the performance of these operators in 2-D matching tasks are presented. In addition, the potential of this approach for multimodality matching of 3-D medical images is illustrated by the striking similarity of the ridge images extracted from CT and MR images by the 3-D version of the operator.

Mutual information matching and interpolation artifacts

Registration algorithms often require the estimation of grey values at image locations that do not coincide with image grid points. Because of the intrinsic uncertainty, the estimation process will invariably be a source of error in the registration process. For measures based on entropy, such as mutual information, an interpolation method that changes the amount of dispersion in the probability distributions of the grey values of the images will influence the registration measure. With two images that have equal grid distances in one or more corresponding dimensions, a large number of grid points can be aligned for certain geometric transformations. As a result, the level of interpolation is dependent on the image transformation and hence, so is the interpolation-induced change in dispersion of the histograms. When an entropy based registration measure is plotted as a function of transformation, it will show sudden changes in value for the grid-aligning transformations. Such patterns of local extrema impede the optimization process. More importantly, they rule out subvoxel accuracy. Interpolation-induced artifacts are shown to occur in registration of clinical images, both for trilinear and partial volume interpolation. Furthermore, the results suggest that improved registration accuracy for scale-corrected MR images may be partly accounted for by the inequality of grid distances that is a result of scale correction.

Multimodality visualization of medical volume data

Abstract New developments in 3-D volume acquisitions are creating a rapidly increasing demand for integrating multimodality 3-D visualization. In order to accomplish routine clinical multimodality visualization, many issues have to be dealt with, such as techniques for accurate spatial registration, integrated representation, suitable graphical user interfaces, and obtaining adequate rendering speeds. The aim of this experience paper is 2-fold. First, it presents various results from our research on multimodality visualization/registration. Second, this paper explicitly addresses practical problems and findings related to software development and multimodality registration/visualization. We hope that this will give colleagues a better understanding in some of these issues based on our experience, including notably our mistakes.

INTEGRATION OF FUNCTIONAL AND ANATOMICAL BRAIN IMAGES

This article concerns the integration of functional and anatomical volumetric brain images. Integration consists of two steps: matching or registration, where the images are brought into spatial agreement, and fusion or simultaneous display where the registered multimodal image information is presented in an integrated fashion. Approaches to register multiple images are divided into extrinsic methods based on artificial markers, and intrinsic matching methods based solely on the patient related image data. The various methods are compared by a number of characteristics, which leads to a clear preference for one class of intrinsic methods, viz. voxel-based matching. Furthermore, two- and three-dimensional techniques to display multimodality image information are outlined.