Boklye Kim

Demonstration of accuracy and clinical versatility of mutual information for automatic multimodality image fusion using affine and thin-plate spline warped geometric deformations

This paper applies and evaluates an automatic mutual information-based registration algorithm across a broad spectrum of multimodal volume data sets. The algorithm requires little or no pre-processing, minimal user input and easily implements either affine, i.e. linear or thin-plate spline (TPS) warped registrations. We have evaluated the algorithm in phantom studies as well as in selected cases where few other algorithms could perform as well, if at all, to demonstrate the value of this new method. Pairs of multimodal gray-scale volume data sets were registered by iteratively changing registration parameters to maximize mutual information. Quantitative registration errors were assessed in registrations of a thorax phantom using PET/CT and in the National Library of Medicines Visible Male using MRI T2-/T1-weighted acquisitions. Registrations of diverse clinical data sets were demonstrated including rotate-translate mapping of PET/MRI brain scans with significant missing data, full affine mapping of thoracic PET/CT and rotate-translate mapping of abdominal SPECT/CT. A five-point thin-plate spline (TPS) warped registration of thoracic PET/CT is also demonstrated. The registration algorithm converged in times ranging between 3.5 and 31 min for affine clinical registrations and 57 min for TPS warping. Mean error vector lengths for rotate-translate registrations were measured to be subvoxel in phantoms. More importantly the rotate-translate algorithm performs well even with missing data. The demonstrated clinical fusions are qualitatively excellent at all levels. We conclude that such automatic, rapid, robust algorithms significantly increase the likelihood that multimodality registrations will be routinely used to aid clinical diagnoses and post-therapeutic assessment in the near future.

Mutual Information for Automated Unwarping of Rat Brain Autoradiographs

An automated multimodal warping based on mutual information metric (MI) as a mapping cost function is demonstrated. Mutual information (I) is calculated from a two-dimensional (2D) gray scale histogram of an image pair, and MI (= -I) provides a matching cost function which can be effective in registration of two- or three-dimensional data sets independent of modality. Most histological image data, though information rich and high resolution, present nonlinear deformations due to the specimen sectioning and need reconstitution into deformation-corrected volumes prior to geometric mapping to an anatomical volume for spatial analyses. Section alignment via automatic 2D registrations employing MI as a global cost function and thin-plate-spline (TPS) warping is applied to deoxy-D-[14C]glucose autoradiographic image slices of a rat brain with video reference images of the uncut block face to reconstitute a cerebral glucose metabolic volume data. Unlike the traditional feature-based TPS warping algorithms, initial control points are defined independently from feature landmarks. Registration quality using automated multimodal image warping is validated by comparing MIs of the image pair registered by automated affine registration and manual warping method. The MI proves to be a robust objective matching cost function effective for automatic multimodality warping for 2D data sets and can be readily applied to volume registrations.

Motion Correction in fMRI via Registration of Individual Slices Into an Anatomical Volume

An automated retrospective image registration based on mutual information is adapted to a multislice functional magnetic resonance imaging (fMRI) acquisition protocol to provide accurate motion correction. Motion correction is performed by mapping each slice to an anatomic volume data set acquired in the same fMRI session to accommodate inter‐slice head motion. Accuracy of the registration parameters was assessed by registration of simulated MR data of the known truth. The widely used rigid body volume registration approach based on stacked slices from the time series data may hinder statistical accuracy by introducing inaccurate assumptions of no motion between slices for multislice fMRI data. Improved sensitivity and specificity of the fMRI signal from mapping‐each‐slice‐to‐volume method is demonstrated in comparison with a stacked‐slice correction method by examining functional data from two normal volunteers. The data presented in a standard anatomical coordinate system suggest the reliability of the mapping‐each‐slice‐to‐volume method to detect the activation signals consistent between the two subjects. Magn Reson Med 41:964–972, 1999.

Semiautomatic registration of volumetric ultrasound scans

We demonstrate the ability to register easily and accurately volumetric ultrasound scans without significant data preprocessing or user intervention. Two volumetric ultrasound breast scan data sets were acquired from two different patients with breast cancer. Volumetric scan data were acquired by manually sweeping a linear array transducer mounted on a linear slider with a position encoder. The volumetric data set pairs consisted of color flow and/or power mode Doppler data sets acquired serially on the same patients. A previously described semiautomatic registration method based on maximizing mutual information was used to determine the transform between data sets. The results suggest that, even for the deformable breast, three-dimensional full affine transforms can be sufficient to obtain clinically useful registrations; warping may be necessary for increased registration accuracy. In conclusion, mutual information-based automatic registration as implemented on modern workstations is capable of yielding clinically useful registrations in times <35 min.

Accelerated Nonrigid Intensity-Based Image Registration Using Importance Sampling

Nonrigid image registration methods using intensity-based similarity metrics are becoming increasingly common tools to estimate many types of deformations. Nonrigid warps can be very flexible with a large number of parameters and gradient optimization schemes are widely used to estimate them. However, for large datasets, the computation of the gradient of the similarity metric with respect to these many parameters becomes very time consuming. Using a small random subset of image voxels to approximate the gradient can reduce computation time. This work focuses on the use of importance sampling to reduce the variance of this gradient approximation. The proposed importance sampling framework is based on an edge-dependent adaptive sampling distribution designed for use with intensity-based registration algorithms. We compare the performance of registration based on stochastic approximations with and without importance sampling to that using deterministic gradient descent. Empirical results, on simulated magnetic resonance brain data and real computed tomography inhale-exhale lung data from eight subjects, show that a combination of stochastic approximation methods and importance sampling accelerates the registration process while preserving accuracy.

Mutual Information for Automated Multimodal Image Warping

A quantitative assessment of mapping accuracy based on mutual information index (MI), calculated from gray scale 2D histogram, is applied to an automated multimodality (un)warping algorithm. Information rich histological image data, which present non-linear deformations due to the specimen sectioning, is reconstituted into deformation-corrected, 3D volumes for geometric mapping to anatomical data for spatial analyses. Thin-plate-spline (TPS) algorithm has been implemented for automatic unwarping of the distortions using a multivariate optimizer and MI as a global cost function. The MI proves to be a robust objective matching criterion effective for automatic multimodality warping for 2D data sets and can be readily applied to volumetric 3D registrations. The improved performance of TPS warping compared to full affine transformation is quantified by comparison of MIs of both methods.

Evaluation of Control Point Selection in Automatic, Mutual Information Driven, 3D Warping

We have implemented automatic 3D thin-plate spline warping as a geometric interpolant to map one dataset volume onto another. Homologous control points in one space are iteratively moved by an optimizer to maximize the global mutual information between the two data volumes. Given two different poses between highly deformed objects we desire to compute the relative geometric deformation using a minimal set of control points as determined by number and placement. The general solution to this problem is not known. In this paper we assess retrospective control point selection for the case of significant patient motion during MRI breast imaging.

Co-Registration of MRI and Autoradiography of Rat Brain in Three-Dimensions Following Automatic Reconstruction of 2D Data Set

Brain images obtained in 2DG-autoradiography have been reconstructed into 3D volumes for the purpose of accurate three dimensional registration with MRI data to obtain spatially registered, histologic “truth” data. Modalities were chosen to closely model the current clinical interest in correlation of MRI and FDG-PET imaging. An automatic 2D-registration algorithm that takes into account variations in sample orientation and shearing has been developed for accurate alignment of brain slices. It uses a multivariate optimization algorithm on the peak correlation between the gradient filtered autoradiograph image and the corresponding video image of the specimen’s block face. Registration of the reconstructed 2DG-autoradiography volume data with 3D reconstructed in vivo multislice MRI of rat brains was accomplished with a 3D registration algorithm utilizing user identified homologous feature pairs consisting of points, line segments, and planar patches.

Improved Motion Correction in fMRI by Joint Mapping of Slices into an Anatomical Volume

Motion correction in fMRI time series is essential for accurate statistical analyses. Typically motion correction is applied in a rigid fashion between time series volumes. Such corrections assume no relative motion between slices. An improved motion correction scheme, map-slice-to-volume (MSV), was developed previously using mutual information (MI) to register individual fMRI slices onto an anatomical volume to account for inter-slice motion [6]. As each slice’s orientation represents a statistically independent sampling of the patient’s motion at multiple intervals throughout the acquisition, a smoothed estimate of the patient’s trajectory in time can be computed by jointly estimating each slice’s orientation while minimizing the implied acceleration of the patient’s head subject to applying a priori, slice-related weights based on known registration reliabilities. The results of this joint mapping of slices into a volume (JMSV) show further substantial improvement in slice registration and subsequent motion estimation.

Probabilistic Brain Atlas Construction: Thin-Plate Spline Warping via Maximization of Mutual Information

The concepts of a probabilistic atlas are well known. The dispersions of the resulting atlas’ spatial probability distributions depend not only on the intrinsic variation of structures between subjects, but also on the ability of the intersubject mapping method to compensate for gross spatial variations. We demonstrate an automatic method of registering patients to an atlas by maximization the mutual information between the atlas and the patient’s gray scale data set. The global thin-plate spline (TPS) transformation for mapping each subject is computed by automatically optimizing the loci of 40 control points distributed within the atlas. The use of 40 control points, i.e. 3*40=120 degrees of freedom (DOF), is a compromise between viscous flow methods with huge DOF, and the 12 DOF affine mapping. We quantitatively compare the results between using a full affine transformation versus the MI-driven 40 control point thin-plate spline for the mean and standard deviation volume data sets computed over the gray scale volumes of 7 patients.