Harriët W. Mulder

Segmentation of multiple heart cavities in 3-D transesophageal ultrasound images

Three-dimensional transesophageal echocardiography (TEE) is an excellent modality for real-time visualization of the heart and monitoring of interventions. To improve the usability of 3-D TEE for intervention monitoring and catheter guidance, automated segmentation is desired. However, 3-D TEE segmentation is still a challenging task due to the complex anatomy with multiple cavities, the limited TEE field of view, and typical ultrasound artifacts. We propose to segment all cavities within the TEE view with a multi-cavity active shape model (ASM) in conjunction with a tissue/blood classification based on a gamma mixture model (GMM). 3-D TEE image data of twenty patients were acquired with a Philips X7-2t matrix TEE probe. Tissue probability maps were estimated by a two-class (blood/tissue) GMM. A statistical shape model containing the left ventricle, right ventricle, left atrium, right atrium, and aorta was derived from computed tomography angiography (CTA) segmentations by principal component analysis. ASMs of the whole heart and individual cavities were generated and consecutively fitted to tissue probability maps. First, an average whole-heart model was aligned with the 3-D TEE based on three manually indicated anatomical landmarks. Second, pose and shape of the whole-heart ASM were fitted by a weighted update scheme excluding parts outside of the image sector. Third, pose and shape of ASM for individual heart cavities were initialized by the previous whole heart ASM and updated in a regularized manner to fit the tissue probability maps. The ASM segmentations were validated against manual outlines by two observers and CTA derived segmentations.

Multiframe registration of real-time three-dimensional echocardiography time series

Abstract. Mosaicing of real-time three-dimensional echocardiography (RT3-DE) images aims at extending the field-of-view of overlapping images. Currently available methods discard most of the temporal information available in the time series. We investigate the added value of simultaneous registration of multiple temporal frames using common similarity metrics. We combine RT3-DE images of the left and right ventricles by registration and fusion. The standard approach of registering single frames, either end-diastolic (ED) or end-systolic (ES), is compared with simultaneous registration of multiple time frames, to evaluate the effect of using the information from all images in the metric. A transformation estimating the protocol-specific misalignment is used to initialize the registration. It is shown that multiframe registration can be as accurate as alignment of the images based on manual annotations. Multiframe registration using normalized cross-correlation outperforms any of the single-frame methods. As opposed to expectations, extending the multiframe registration beyond simultaneous use of ED and ES frames does not further improve registration results.

Improved Segmentation of Multiple Cavities of the Heart in Wide-View 3-D Transesophageal Echocardiograms.

Minimally invasive interventions in the heart such as in electrophysiology are becoming more and more important in clinical practice. Currently, preoperative computed tomography angiography (CTA) is used to provide anatomic information during electrophysiology interventions, but this does not provide real-time feedback and burdens the patient with additional radiation and side effects of the contrast agent. Three-dimensional transesophageal echocardiography (TEE) is an excellent modality for visualization of anatomic structures and instruments in real time, but some cavities, especially the left atrium, suffer from the limited coverage of the 3-D TEE volumes. This leads to difficulty in segmenting the left atrium. We propose replacing or complementing pre-operative CTA imaging with wide-view TEE. We tested this proposal on 20 patients for which TEE image volumes covering the left atrium and CTA images were acquired. The TEE images were manually registered, and wide-view volumes were generated. Five heart cavities in single-view and wide-view TEE were segmented and compared with atlas based-segmentations derived from the CTA images. We found that the segmentation accuracy (Dice coefficients) improved relative to segmentation of single-view images by 5, 15 and 9 percentage points for the left atrium, right atrium and aorta, respectively. Average anatomic coverage was improved by 2, 29, 62 and 49 percentage points for the right ventricle, left atrium, right atrium and aorta, respectively. This finding confirms that wide-view 3-D TEE can be useful in supporting electrophysiology interventions.

Registration of multi-view apical 3D echocardiography images

Real-time three-dimensional echocardiography (RT3DE) is a non-invasive method to visualize the heart. Disadvantageously, it suffers from non-uniform image quality and a limited field of view. Image quality can be improved by fusion of multiple echocardiography images. Successful registration of the images is essential for prosperous fusion. Therefore, this study examines the performance of different methods for intrasubject registration of multi-view apical RT3DE images. A total of 14 data sets was annotated by two observers who indicated the position of the apex and four points on the mitral valve ring. These annotations were used to evaluate registration. Multi-view end-diastolic (ED) as well as end-systolic (ES) images were rigidly registered in a multi-resolution strategy. The performance of single-frame and multi-frame registration was examined. Multi-frame registration optimizes the metric for several time frames simultaneously. Furthermore, the suitability of mutual information (MI) as similarity measure was compared to normalized cross-correlation (NCC). For initialization of the registration, a transformation that describes the probe movement was obtained by manually registering five representative data sets. It was found that multi-frame registration can improve registration results with respect to single-frame registration. Additionally, NCC outperformed MI as similarity measure. If NCC was optimized in a multi-frame registration strategy including ED and ES time frames, the performance of the automatic method was comparable to that of manual registration. In conclusion, automatic registration of RT3DE images performs as good as manual registration. As registration precedes image fusion, this method can contribute to improved quality of echocardiography images.

Atlas-Based Mosaicing of Left Atrial 3-D Transesophageal Echocardiography Images

Transesophageal echocardiography (TEE) is a promising imaging modality used to guide cardiac interventions, such as catheter ablation for the treatment of cardiac arrhythmias. These procedures rely on good visualization of the left atrium and pulmonary veins. To visualize these structures in a single volume, the acquisition, registration and fusion of multiple TEE views of the left atrium are required. We introduce atlas-based mosaicing as a method for the registration of images that are acquired according to a standardized protocol. Inspired by atlas-based segmentation approaches, compounded data of other patients serve as atlases for the registration of new data. The performance of atlas-based mosaicing is studied on 3-D TEE data of the left atrium and compared with that of regular pairwise registration. This study indicates that improved registration robustness and smaller registration errors are achieved with atlas-based mosaicing compared with regular pairwise registration. This is an important step toward the use of TEE for interventional guidance of ablation procedures.

Atlas-based mosaicing of 3D transesophageal echocardiography images of the left atrium

3D transesophageal echocardiography (TEE) is routinely used for planning and guidance of cardiac interventions. However, the limited field-of-view dictates the compounding of multiple images for visualization of large structures, e.g. the left atrium (LA). Previously, we developed a TEE image acquisition protocol to capture the LA in six views. This paper proposes the use of fused TEE data of other patients as atlases to guide the registration of these views in a novel method, denoted atlas-based mosaicing (ABM). In this method, each TEE view is rigidly registered to a fused TEE atlas image of another patient, followed by pairwise registration of neighboring TEE views. Experiments on ten patient data sets and five atlas sets reveal the superior performance of ABM compared to regular pairwise registration. Similar to atlas-based segmentation approaches, ABM has the power of combining the outcome of multiple atlases resulting in accurate registration of images with little overlap.

Segmentation of multiple heart cavities in wide-view fused 3D transesophageal echocardiograms

Three-dimensional transesophageal echocardiography (3D TEE) is an excellent modality for real-time visualization of the heart and monitoring of interventions. However, 3D TEE segmentation is still a challenging task due to the complex anatomy, typical ultrasound artifacts, and the limited field of view. To improve the segmentation accuracy of the left atrium we created wide-view TEE images by fusing several individual recordings. For five patients, six individual 3D TEE volumes were acquired (Philips X7-2t) by manipulating (e.g. rotating) the TEE probe head in the esophagus. CTA images were also acquired in these patients and segmented automatically by an atlas-based method, which served as ground truth (GT). The TEE volumes were manually registered by aligning them first with the CTA volume and afterwards with each other. The individual TEE sets were fused using a minimum intensity projection. Five cavities, left and right ventricle, left and right atrium and aorta (LV, LA, RV, LA, Ao) were segmented using a three stage segmentation scheme utilizing an Active Shape Model (ASM) and tissue probability maps estimated by a two class Gamma Mixture Model. The multi-cavity shape model was generated by Principal Component Analysis from a large database of segmented CTA images. The ASM stages involved a rigid transform fitting of the model, a shape updating stage for the whole heart model and a refinement stage for each individual cavity model. The Dice coefficients for the individual cavities between the TEE segmentations and the GT CTA segmentations were computed. We compared the quality of the segmentation results on the fused TEE sets with those using only the central single TEE view. There was a considerable improvement of the Dice coefficients for the fused data sets. The increase of the median Dice coefficients was 2, 5, 9, and 12 percent points for RV, LA, RA, and Ao respectively. These results show that image fusion significantly improves the segmentation of the cavities close to the TEE probe head (LA, RA, and Ao).

Simultaneous pairwise registration for image mosaicing of TEE data

Due to the limited field-of-view of transesophageal echocardiography (TEE) images, mosaicing is required to visualize the entire left atrium in a single image. However, the small overlap between the images and the lack of a single reference image challenges the registration. Our approach is to exploit overlap of an image with multiple other images by simultaneous pairwise registration. Three images were registered to a floating common reference using a rigid transformation. The images iteratively serve as floating reference for the other images. Averaging the resulting transformations for each image will make the simultaneous registration converge to a common reference space. It was shown on randomly transformed MR brain and TEE images that the simultaneous method achieved higher success rates than regular pairwise registration. Initial results on TEE images of the left atrium demonstrated the ability of our method to register the images to a common space.

Selection strategies for atlas-based mosaicing of left atrial 3-D transesophageal echocardiography data

Three-dimensional transesophageal echocardiography (TEE) provides real-time soft tissue information, but its use is hampered by its limited field of view. The mosaicing of multiple TEE views makes it possible to visualize a large structure, like the left atrium, in a single volume. To this end, an automatic registration method is required. Similarly to atlas-based segmentation approaches, atlas-based mosaicing (ABM) uses a full volume atlas set to moderate the onerous registration of the individual TEE views. The performance of ABM depends both on the quality of the involved registrations and on the selection of the optimal transformation from the candidate transformations that result from the various atlases. The study described here explored the performance of different selection strategies on multiview TEE data of the left atrium. We found that by incorporating two stages of transformation selection, using the image similarity and the conformity between the candidate transformations as selection criteria, the average registration error dropped below 3 mm with respect to manual registration of these data. Finally, we used this method for the automatic construction of a wide-view TEE volume of the left atrium.