Steven C. Mitchell

3-D active appearance models: segmentation of cardiac MR and ultrasound images

A model-based method for three-dimensional image segmentation was developed and its performance assessed in segmentation of volumetric cardiac magnetic resonance (MR) images and echocardiographic temporal image sequences. Comprehensive design of a three-dimensional (3-D) active appearance model (AAM) is reported for the first time as an involved extension of the AAM framework introduced by Cootes et al. The models behavior is learned from manually traced segmentation examples during an automated training stage. Information about shape and image appearance of the cardiac structures is contained in a single model. This ensures a spatially and/or temporally consistent segmentation of three-dimensional cardiac images. The clinical potential of the 3-D AAM is demonstrated in short-axis cardiac MR images and four-chamber echocardiographic sequences. The methods performance was assessed by comparison with manually identified independent standards in 56 clinical MR and 64 clinical echo image sequences. The AAM method showed good agreement with the independent standard using quantitative indexes of border positioning errors, endo- and epicardial volumes, and left ventricular mass. In MR, the endocardial volumes, epicardial volumes, and left ventricular wall mass correlation coefficients between manual and AAM were R/sup 2/=0.94,0.97,0.82, respectively. For echocardiographic analysis, the area correlation was R/sup 2/=0.79. The AAM method shows high promise for successful application to MR and echocardiographic image analysis in a clinical setting.

Automatic segmentation of echocardiographic sequences by active appearance motion models

A novel extension of active appearance models (AAMs) for automated border detection in echocardiographic image sequences is reported. The active appearance motion model (AAMM) technique allows fully automated robust and time-continuous delineation of left ventricular (LV) endocardial contours over the full heart cycle with good results. Nonlinear intensity normalization was developed and employed to accommodate ultrasound-specific intensity distributions. The method was trained and tested on 16-frame phase-normalized transthoracic four-chamber sequences of 129 unselected infarct patients, split randomly into a training set (n=65) and a test set (n=64). Borders were compared to expert drawn endocardial contours. On the test set, fully automated AAMM performed well in 97% of the cases (average distance between manual and automatic landmark points was 3.3 mm, comparable to human interobserver variabilities). The ultrasound-specific intensity normalization proved to be of great value for good results in echocardiograms. The AAMM was significantly more accurate than an equivalent set of two-dimensional AAMs.

Time-Continuous Segmentation of Cardiac Image Sequences Using Active Appearance Motion Models

This paper describes a novel, 2D+time Active Appearance Motion Model (AAMM). Cootess 2D AAM framework was extended by considering a complete image sequence as a single shape/intensity sample. This way, the cardiac motion is modeled in combination with the shape and image appearance of the heart. The clinical potential of the AAMMs is demonstrated on two imaging modalities - cardiac MRI and echocardiography.

Segmentation of cardiac MR images: an active appearance model approach

Active Appearance Models (AAM), which have been recently introduced by Cootes et al., describe the shape of objects and gray level appearance from a set of example images. An AAM is created from user-placed contours defining the shape of objects of interest in each training image. The information about shape changes observed in the training set is used to model the shape variation. Principle component analysis (PCA) is utilized to model gray level variation observed in the training set. The resulting model describes objects as a linear combination of eigen vectors both in shape and gray levels applied to the mean image. The main purpose of this work is to investigate the clinical potential of AAMs for segmentation of cardiovascular MR images acquired in routine clinical practice. An AAM was constructed using 102 end- diastolic short-axis cardiac MR images at the papillary muscle level from normals and patients with varying pathologies. The resulting AAM is a compact representation consisting of a mean image and a limited number of coefficients of eigen vectors, representing 97% of shape and gray level variation observed in the training set. The segmentation performance is tested in 60 end-diastolic short-axis cardiac MR images from different patients.

Four-dimensional coronary morphology and computational hemodynamics

Conventional reconstructions from intravascular ultrasound (IVUS) stack the frames as acquired during the pullback of the catheter to form a straight three-dimensional volume, thus neglecting the vessel curvature and merging images from different heart phases. We are developing a comprehensive system for fusion of the IVUS data with the pullback path as determined from x-ray angiography, to create a geometrically accurate 4-D (3-D plus time) model of the coronary vasculature as basis for computational hemodynamics. The overall goal of our work is to correlate shear stress with plaque thickness. The IVUS data are obtained in a single pullback using an automated pullback device; the frames are afterwards assigned to their respective heart phases based upon the ECG signal. A set of 3-D models is reconstructed by fusion of IVUS and angiographic data corresponding to the same ECG-gated heart phase; methods of computational fluid dynamics (CFD) are applied to obtain important hemodynamic data. Combining these models yields the final 4-D reconstruction. Visualization is performed using the platform-independent VRML standard for a user-friendly manipulation of the scene. An extension for virtual angioscopy allows an easy assessment of the vessel features within their local context. Validation was successfully performed both in-vitro and in-vivo.

Shape- and appearance-based segmentation of volumetric medical images

A novel approach to three-dimensional segmentation, parametric surface representation, and interactive surface modification is reported. The combination of shape-based and appearance-based volumetric segmentation, parametric representation of 3D surfaces, and their interactive modification and editing forms a very powerful paradigm for a variety of volumetric segmentation tasks. Performance assessment of the method for automated segmentation of diaphragm surfaces in volumetric 3D CT images is ongoing.

Time-continuous segmentation of cardiac MR image sequences using active appearance motion models

Active Appearance Models (AAMs) are useful for segmentation of static cardiac MR images since they exploit prior knowledge about the cardiac shape and image appearance. However, applying 2D AAMs to full cardiac cycle segmentation would require multiple models for different phases of the cardiac cycle because traditional AAMs account only for the variations within image classes and not temporal classes. This paper presents a novel 2D+time Active Appearance Motion Model (AAMM) that represents the dynamics of the cardiac cycle in combination with shape and image appearance of the heart, ensuring a time-continuous segmentation of a complete cardiac MR sequence. In AAMM, single-beat sequences are phase-normalized into sets of 2D images and the shape points and gray intensities between frames are concatenated into a shape vector and intensity vector. Appearance variations over time are captured using Principal Component Analysis on both vectors in the training set. Time-continuous segmentation is achieved by minimizing the model appearance-to-target differences by adjusting the model eigen-coefficients using gradient descent approach. In matching tests, the model shows to be robust in initial position and approximates the true segmentation very well. Large-scale clinical validation in patients is ongoing.

Active appearance motion models for endocardial contour detection in time sequences of echocardiograms

Active Appearance Models (AAM) are suitable for segmenting 2D images, but for image sequences time-continuous results are desired. Active Appearance-Motion Models (AAMM) model shape and appearance of the heart over the full cardiac cycle. Single-beat sequences are phase-normalized into stacks of 16 2D images. In a training set, corresponding shape points on the endocard are defined for each image based on expert drawn contours. Shape (2D) and intensity vectors are derived similar to AAM. Intensities are normalized non-linearly to handle ultrasound-specific problems. For all time frames, shape vectors are simply concatenated, as well as and intensity vectors. Principal Component Analysis extracts appearance eigenvariations over the cycle, capturing typical motion patterns. AAMMs perform segmentation on complete sequences by minimizing model-to-target differences, adjusting AAMM eigenvariation coefficients using gradient descent minimization. This results in time-continuous segmentation. The method was trained and tested on echocardiographic 4-chamber sequences of 129 unselected patients split randomly into a training set (n=65) and a test set (n=64). In all sequences, an independent expert manually drew endocardial contours. On the test set, fully automated AAMM performed well in 97% of cases (average distance 3.3 mm, 9.3 pixels, comparable to human inter- and intraobserver variabilities).

Segmentation of cardiac MR volume data using 3D active appearance models

Active Appearance Models (AAMs) are useful for the segmentation of cardiac MR images since they exploit prior knowledge about the cardiac shape and image appearance. However, traditional AAMs only process 2D images, not taking into account the 3D data inherent to MR. This paper presents a novel, true 3D Active Appearance Model that models the intrinsic 3D shape and image appearance of the left ventricle in cardiac MR data. In 3D-AAM, shape and appearance of the Left Ventricle (LV) is modeled from a set of expert drawn contours. The contours are then resampled to a manually defined set of landmark points, and subsequently aligned. Appearance variations in both shape and texture are captured using Principal Component Analysis (PCA) on the training set. Segmentation is achieved by minimizing the model appearance-to-target differences by adjusting the model eigen-coefficients using a gradient descent approach. The clinical potential of the 3D-AAM is demonstrated in short-axis cardiac magnetic resonance (MR) images. The methods performance was assessed by comparison with manually-identified independent standards in 56 clinical MR sequences. The method showed good agreement with the independent standards using quantitative indices such as border positioning errors, endo- and epicardial volumes, and left ventricular mass. The 3D AAM method shows high promise for successful segmentation of three-dimensional images in MR.

Fully automated endocardial contour detection in time sequences of echocardiograms by three-dimensional active appearance models

A novel 3-D Active Appearance Model (3-D AAM) is applied to fully automated endocardial contour detection in 2-D + time (2DT) 4-chamber ultrasound sequences, without knowledge of cardiac phase (ED/ES frames). 2DT appearance of the heart is modeled in 3-D by converting the stack of 2-D time slices into a 3-D voxel space. In a training set, an expert defines corresponding endocardial contour points for one complete cardiac cycle (ED to ED). 2DT shape is represented as a 3-D surface. Image appearance is modeled as a vector of voxel intensities in a volume-patch spanned by the 3-D surface. Principal Component Analysis extracts eigenvariations of 3-D shape and appearance, capturing typical cardiac motion patterns. 3-D AAM segments the image volume by minimizing 3-D model-to-target intensity differences, adjusting eigenvariation coefficients and 3-D pose using gradient descent minimization. This provides time-continuous border localization for one beat in both time and space. The method was used on 3-beat sequences from 129 patients split randomly into a training (65) and a test set (64). An independent expert manually drew all endocardial contours. 3-D AAM converged well in 89% of test cases. Average absolute temporal error was 37.0 msec, spatial error 3.35 mm, comparable to human inter-observer variabilities.