Andriy Myronenko

Point Set Registration: Coherent Point Drift

Point set registration is a key component in many computer vision tasks. The goal of point set registration is to assign correspondences between two sets of points and to recover the transformation that maps one point set to the other. Multiple factors, including an unknown nonrigid spatial transformation, large dimensionality of point set, noise, and outliers, make the point set registration a challenging problem. We introduce a probabilistic method, called the Coherent Point Drift (CPD) algorithm, for both rigid and nonrigid point set registration. We consider the alignment of two point sets as a probability density estimation problem. We fit the Gaussian mixture model (GMM) centroids (representing the first point set) to the data (the second point set) by maximizing the likelihood. We force the GMM centroids to move coherently as a group to preserve the topological structure of the point sets. In the rigid case, we impose the coherence constraint by reparameterization of GMM centroid locations with rigid parameters and derive a closed form solution of the maximization step of the EM algorithm in arbitrary dimensions. In the nonrigid case, we impose the coherence constraint by regularizing the displacement field and using the variational calculus to derive the optimal transformation. We also introduce a fast algorithm that reduces the method computation complexity to linear. We test the CPD algorithm for both rigid and nonrigid transformations in the presence of noise, outliers, and missing points, where CPD shows accurate results and outperforms current state-of-the-art methods.

Intensity-Based Image Registration by Minimizing Residual Complexity

Accurate definition of the similarity measure is a key component in image registration. Most commonly used intensity-based similarity measures rely on the assumptions of independence and stationarity of the intensities from pixel to pixel. Such measures cannot capture the complex interactions among the pixel intensities, and often result in less satisfactory registration performances, especially in the presence of spatially-varying intensity distortions. We propose a novel similarity measure that accounts for intensity nonstationarities and complex spatially-varying intensity distortions in mono-modal settings. We derive the similarity measure by analytically solving for the intensity correction field and its adaptive regularization. The final measure can be interpreted as one that favors a registration with minimum compression complexity of the residual image between the two registered images. One of the key advantages of the new similarity measure is its simplicity in terms of both computational complexity and implementation. This measure produces accurate registration results on both artificial and real-world problems that we have tested, and outperforms other state-of-the-art similarity measures in these cases.

Image registration by minimization of residual complexity

Accurate definition of similarity measure is a key component in image registration. Most commonly used intensity-based similarity measures rely on the assumptions of independence and stationarity of the intensities from pixel to pixel. Such measures cannot capture the complex interactions among the pixel intensities, and often result in less satisfactory registration performances, especially in the presence of nonstationary intensity distortions. We propose a novel similarity measure that accounts for intensity non-stationarities and complex spatially-varying intensity distortions. We derive the similarity measure by analytically solving for the intensity correction field and its adaptive regularization. The final measure can be interpreted as one that favors a registration with minimum compression complexity of the residual image between the two registered images. This measure produces accurate registration results on both artificial and real-world problems that we have tested, whereas many other state-of-the-art similarity measures have failed to do so.

LV motion tracking from 3D echocardiography using textural and structural information

Automated motion reconstruction of the left ventricle (LV) from 3D echocardiography provides insight into myocardium architecture and function. Low image quality and artifacts make 3D ultrasound image processing a challenging problem. We introduce a LV tracking method, which combines textural and structural information to overcome the image quality limitations. Our method automatically reconstructs the motion of the LV contour (endocardium and epicardium) from a sequence of 3D ultrasound images.

Defining Left Ventricular Apex-to-Base Twist Mechanics Computed From High-Resolution 3D Echocardiography: Validation Against Sonomicrometry

OBJECTIVES To compute left ventricular (LV) twist from 3-dimensional (3D) echocardiography. BACKGROUND LV twist is a sensitive index of cardiac performance. Conventional 2-dimensional based methods of computing LV twist are cumbersome and subject to errors. METHODS We studied 10 adult open-chest pigs. The pre-load to the heart was altered by temporary controlled occlusion of the inferior vena cava, and myocardial ischemia was produced by ligating the left anterior descending coronary artery. Full-volume 3D loops were reconstructed by stitching of pyramidal volumes acquired from 7 consecutive heart beats with electrocardiography gating on a Philips IE33 system (Philips Medical Systems, Andover, Massachusetts) at baseline and other steady states. Polar coordinate data of the 3D images were entered into an envelope detection program implemented in MatLab (The MathWorks, Inc., Natick, Massachusetts), and speckle motion was tracked using nonrigid image registration with spline-based transformation parameterization. The 3D displacement field was obtained, and rotation at apical and basal planes was computed. LV twist was derived as the net difference of apical and basal rotation. Sonomicrometry data of cardiac motion were also acquired from crystals anchored to epicardium in apical and basal planes at all states. RESULTS The 3D dense tracking slightly overestimated the LV twist, but detected changes in LV twist at different states and showed good correlation (r = 0.89) when compared with sonomicrometry-derived twist at all steady states. In open chest pigs, peak cardiac twist was increased with reduction of pre-load from inferior vena cava occlusion from 6.25 degrees +/- 1.65 degrees to 9.45 degrees +/- 1.95 degrees . With myocardial ischemia from left anterior descending coronary artery ligation, twist was decreased to 4.90 degrees +/- 0.85 degrees (r = 0.8759). CONCLUSIONS Despite lower spatiotemporal resolution of 3D echocardiography, LV twist and torsion can be computed accurately.

Maximum Likelihood Motion Estimation in 3D Echocardiography through Non-rigid Registration in Spherical Coordinates

Automated motion tracking of the myocardium from 3D echocardiography provides insight into hearts architecture and function. We present a method for 3D cardiac motion tracking using non-rigid image registration. Our contribution is two-fold. We introduce a new similarity measure derived from a maximum likelihood perspective taking into account physical properties of ultrasound image acquisition and formation. Second, we use envelope-detected 3D echo images in the raw spherical coordinates format, which preserves speckle statistics and represents a compromise between signal detail and data complexity. We derive mechanical measures such as strain and twist, and validate using sonomicrometry in open-chest piglets. The results demonstrate the accuracy and feasibility of our method for studying cardiac motion.

Global active contour-based image segmentation via probability alignment

Active contours is a popular technique for image segmentation. However, active contour tend to converge to the closest local minimum of its energy function and often requires a close boundary initialization. We introduce a new approach that overcomes the close boundary initialization problem by reformulating the external energy term. We treat the active contour as a mean curve of the probability density function p(x). It moves to minimize the Kullback-Leibler (KL) divergence between p(x) and the probability density function derived from the image. KL divergence forces p(x) to “cover all image areas” and the uncovered areas are heavily penalized, which allows the active contour to go over the edges. Also we use deterministic annealing on the width of p(x) to implement a coarse-to-fine search strategy. In the limit, when the width of p(x) goes to zero, the KL divergence function converges to the conventional external energy term (which can be seen a special case) of active contours. Our method produces robust segmentation results from arbitrary initialization positions.

Speckle Tracking in 3D Echocardiography with Motion Coherence

Tracking of speckles in echocardiography enables the study of myocardium deformation, and thus can provide insights about heart structure and function. Most of the current methods are based on 2D speckle tracking, which suffers from errors due to through-plane decorrelation. Speckle tracking in 3D overcomes such limitation. However, 3D speckle tracking is a challenging problem due to relatively low spatial and temporal resolution of 3D echocardiography. To ensure accurate and robust tracking, high level spatial and temporal constraints need to be incorporated. In this paper, we introduce a novel method for speckle tracking in 3D echocardiography. Instead of tracking each speckle independently, we enforce a motion coherence constraint, in conjunction with a dynamic model for the speckles. This method is validated on in vivo porcine hearts, and is proved to be accurate and robust.

Registration of microscopic iris image sequences using probabilistic mesh

This paper explores the use of deformable mesh for registration of microscopic iris image sequences. The registration, as an effort for stabilizing and rectifying images corrupted by motion artifacts, is a crucial step toward leukocyte tracking and motion characterization for the study of immune systems. The image sequences are characterized by locally nonlinear deformations, where an accurate analytical expression can not be derived through modeling of image formation. We generalize the existing deformable mesh and formulate it in a probabilistic framework, which allows us to conveniently introduce local image similarity measures, to model image dynamics and to maintain a well-defined mesh structure and smooth deformation through appropriate regularization. Experimental results demonstrate the effectiveness and accuracy of the algorithm.

Interpreting ultrasound elastography: Image registration of breast cancer ultrasound elastography to histopathology images

Ultrasound elastography is a promising technique for the detection of breast cancer. Despite being proven to be a useful method in clinical studies, no studies have looked at directly correlating information in in-vivo ultrasound elastography images with histopathology images (regarded as the gold standard) using image registration. In this paper, expert knowledge from clinicians is utilised as constraints to register 2D elastography and histopathology images based on corresponding features identified by clinicians such as tumours and fibrous structures. The recently proposed coherent point drift (CPD) algorithm by Myronenko and Song [11] was applied to align the corresponding feature points and the thin-plate splines method of Bookstein [1] is then used to warp the images. The registered images were then overlaid. It was found that in elastography images, the stiffness of malignant tumours tend to extend beyond the tumour boundaries identified in the histopathology images, and there were many stiff areas indicated in the elastography images where no corresponding features could be identified in the histopathology images. The study thus provides some new insight into the relationship of elastography and histopathology as well as suggests further work is needed to better understand how to interpret patterns in elastography images.