Primož Markelj

A review of 3D/2D registration methods for image-guided interventions

Registration of pre- and intra-interventional data is one of the key technologies for image-guided radiation therapy, radiosurgery, minimally invasive surgery, endoscopy, and interventional radiology. In this paper, we survey those 3D/2D data registration methods that utilize 3D computer tomography or magnetic resonance images as the pre-interventional data and 2D X-ray projection images as the intra-interventional data. The 3D/2D registration methods are reviewed with respect to image modality, image dimensionality, registration basis, geometric transformation, user interaction, optimization procedure, subject, and object of registration.

Validation for 2D/3D registration. I: A new gold standard data set.

PURPOSE In this article, the authors propose a new gold standard data set for the validation of two-dimensional/three-dimensional (2D/3D) and 3D/3D image registration algorithms. METHODS A gold standard data set was produced using a fresh cadaver pig head with attached fiducial markers. The authors used several imaging modalities common in diagnostic imaging or radiotherapy, which include 64-slice computed tomography (CT), magnetic resonance imaging using T1, T2, and proton density sequences, and cone beam CT imaging data. Radiographic data were acquired using kilovoltage and megavoltage imaging techniques. The image information reflects both anatomy and reliable fiducial marker information and improves over existing data sets by the level of anatomical detail, image data quality, and soft-tissue content. The markers on the 3D and 2D image data were segmented using ANALYZE 10.0 (AnalyzeDirect, Inc., Kansas City, KN) and an in-house software. RESULTS The projection distance errors and the expected target registration errors over all the image data sets were found to be less than 2.71 and 1.88 mm, respectively. CONCLUSIONS The gold standard data set, obtained with state-of-the-art imaging technology, has the potential to improve the validation of 2D/3D and 3D/3D registration algorithms for image guided therapy.

Validation for 2D/3D registration II: The comparison of intensity- and gradient-based merit functions using a new gold standard data set

PURPOSE A new gold standard data set for validation of 2D/3D registration based on a porcine cadaver head with attached fiducial markers was presented in the first part of this article. The advantage of this new phantom is the large amount of soft tissue, which simulates realistic conditions for registration. This article tests the performance of intensity- and gradient-based algorithms for 2D/3D registration using the new phantom data set. METHODS Intensity-based methods with four merit functions, namely, cross correlation, rank correlation, correlation ratio, and mutual information (MI), and two gradient-based algorithms, the backprojection gradient-based (BGB) registration method and the reconstruction gradient-based (RGB) registration method, were compared. Four volumes consisting of CBCT with two fields of view, 64 slice multidetector CT, and magnetic resonance-T1 weighted images were registered to a pair of kV x-ray images and a pair of MV images. A standardized evaluation methodology was employed. Targets were evenly spread over the volumes and 250 starting positions of the 3D volumes with initial displacements of up to 25 mm from the gold standard position were calculated. After the registration, the displacement from the gold standard was retrieved and the root mean square (RMS), mean, and standard deviation mean target registration errors (mTREs) over 250 registrations were derived. Additionally, the following merit properties were computed: Accuracy, capture range, number of minima, risk of nonconvergence, and distinctiveness of optimum for better comparison of the robustness of each merit. RESULTS Among the merit functions used for the intensity-based method, MI reached the best accuracy with an RMS mTRE down to 1.30 mm. Furthermore, it was the only merit function that could accurately register the CT to the kV x rays with the presence of tissue deformation. As for the gradient-based methods, BGB and RGB methods achieved subvoxel accuracy (RMS mTRE down to 0.56 and 0.70 mm, respectively). Overall, gradient-based similarity measures were found to be substantially more accurate than intensity-based methods and could cope with soft tissue deformation and enabled also accurate registrations of the MR-T1 volume to the kV x-ray image. CONCLUSIONS In this article, the authors demonstrate the usefulness of a new phantom image data set for the evaluation of 2D/3D registration methods, which featured soft tissue deformation. The authors evaluation shows that gradient-based methods are more accurate than intensity-based methods, especially when soft tissue deformation is present. However, the current nonoptimized implementations make them prohibitively slow for practical applications. On the other hand, the speed of the intensity-based method renders these more suitable for clinical use, while the accuracy is still competitive.

Standardized evaluation methodology for 3D/2D registration based on the Visible Human data set

PURPOSE A new image database with a reference-based standardized evaluation methodology for objective evaluation and comparison of three-dimensional/two-dimensional (3D/2D) registration methods has been introduced. METHODS Computed tomography (CT) images of a male and female from the Visible Human Project were used and 16 subvolumes, each containing one of vertebrae T3-T12 and L1-L5 and the pelvis, were defined from the CTs. Six pairs of 2D fluoroscopic x-ray images from different views, showing the thoracic, lumbar, and pelvic regions, were rendered from the CT data using a ray-casting algorithm with an energy conversion function. Furthermore, a single 13-gauge needle was analytically simulated and projected onto the 2D images. By the novel standardized evaluation methodology, a 3D/2D registration method is evaluated by four evaluation criteria: Accuracy, reliability, robustness, and algorithm complexity. RESULTS To demonstrate the usefulness of the proposed data set and the standardized evaluation methodology, a part of the data set was used in an evaluation study of two gradient-based 3D/2D registration methods. It was shown that the use of a failure criterion to calculate the registration accuracy and reliability is not required, since all the information about a registration method can be determined from the estimated distribution of registration errors. CONCLUSIONS The proposed simulated image data set with quite realistic synthetic 2D images, depicting soft tissues and outliers, is especially suitable for preliminary testing of 3D/2D registration algorithms. Since the aim of this article is to provide objective comparison and unbiased evaluation of 3D/2D registration methods, the standardized evaluation methodology is available upon request from the authors.

Efficient implementation of the rank correlation merit function for 2D/3D registration

A growing number of clinical applications using 2D/3D registration have been presented recently. Usually, a digitally reconstructed radiograph is compared iteratively to an x-ray image of the known projection geometry until a match is achieved, thus providing six degrees of freedom of rigid motion which can be used for patient setup in image-guided radiation therapy or computer-assisted interventions. Recently, stochastic rank correlation, a merit function based on Spearmans rank correlation coefficient, was presented as a merit function especially suitable for 2D/3D registration. The advantage of this measure is its robustness against variations in image histogram content and its wide convergence range. The considerable computational expense of computing an ordered rank list is avoided here by comparing randomly chosen subsets of the DRR and reference x-ray. In this work, we show that it is possible to omit the sorting step and to compute the rank correlation coefficient of the full image content as fast as conventional merit functions. Our evaluation of a well-calibrated cadaver phantom also confirms that rank correlation-type merit functions give the most accurate results if large differences in the histogram content for the DRR and the x-ray image are present.

Optimizing bone extraction in MR images for 3D/2D gradient based registration of MR and x-ray images

A number of intensity and feature based methods have been proposed for 3D to 2D registration. However, for multimodal 3D/2D registration of MR and X-ray images, only hybrid and reconstruction-based methods were shown to be feasible. In this paper we optimize the extraction of features in the form of bone edge gradients, which were proposed for 3D/2D registration of MR and X-ray images. The assumption behind such multimodal registration is that the extracted gradients in 2D X-ray images match well to the corresponding gradients extracted in 3D MR images. However, since MRI and X-rays are fundamentally different modalities, the corresponding bone edge gradients may not appear in the same position and the the above-mentioned assumption may thus not be valid. To test the validity of this assumption, we optimized the extraction of bone edges in 3D MR and also in CT images for the registration to 2D X-ray images. The extracted bone edges were systematically displaced in the direction of their gradients, i.e. in the direction of the normal to the bone surface, and corresponding effects on the accuracy and convergence of 3D/2D registration were evaluated. The evaluation was performed on two different sets of MR, CT and X-ray images of spine phantoms with known gold standard, first consisting of five and the other of eight vertebrae. The results showed that a better registration can be obtained if bone edges in MR images are optimized for each application-specific MR acquisition protocol.

Gradient-based 3D-2D registration of cerebral angiograms

Endovascular treatment of cerebral aneurysms and arteriovenous malformations (AVM) involves navigation of a catheter through the femoral artery and vascular system into the brain and into the aneurysm or AVM. Intra-interventional navigation utilizes digital subtraction angiography (DSA) to visualize vascular structures and X-ray fluoroscopy to localize the endovascular components. Due to the two-dimensional (2D) nature of the intra-interventional images, navigation through a complex three-dimensional (3D) structure is a demanding task. Registration of pre-interventional MRA, CTA, or 3D-DSA images and intra-interventional 2D DSA images can greatly enhance visualization and navigation. As a consequence of better navigation in 3D, the amount of required contrast medium and absorbed dose could be significantly reduced. In the past, development and evaluation of 3D-2D registration methods received considerable attention. Several validation image databases and evaluation criteria were created and made publicly available in the past. However, applications of 3D-2D registration methods to cerebral angiograms and their validation are rather scarce. In this paper, the 3D-2D robust gradient reconstruction-based (RGRB) registration algorithm is applied to CTA and DSA images and analyzed. For the evaluation purposes five image datasets, each comprised of a 3D CTA and several 2D DSA-like digitally reconstructed radiographs (DRRs) generated from the CTA, with accurate gold standard registrations were created. A total of 4000 registrations on these five datasets resulted in mean mTRE values between 0.07 and 0.59 mm, capture ranges between 6 and 11 mm and success rates between 61 and 88% using a failure threshold of 2 mm.

3D-2D registration of cerebral angiograms based on vessel directions and intensity gradients

Endovascular treatment of cerebral aneurysms and arteriovenous malformations (AVM) involves navigation of a catheter through the femoral artery and vascular system to the site of pathology. Intra-interventional navigation is done under the guidance of one or at most two two-dimensional (2D) X-ray fluoroscopic images or 2D digital subtracted angiograms (DSA). Due to the projective nature of 2D images, the interventionist needs to mentally reconstruct the position of the catheter in respect to the three-dimensional (3D) patient vasculature, which is not a trivial task. By 3D-2D registration of pre-interventional 3D images like CTA, MRA or 3D-DSA and intra-interventional 2D images, intra-interventional tools such as catheters can be visualized on the 3D model of patient vasculature, allowing easier and faster navigation. Such a navigation may consequently lead to the reduction of total ionizing dose and delivered contrast medium. In the past, development and evaluation of 3D-2D registration methods for endovascular treatments received considerable attention. The main drawback of these methods is that they have to be initialized rather close to the correct position as they mostly have a rather small capture range. In this paper, a novel registration method that has a higher capture range and success rate is proposed. The proposed method and a state-of-the-art method were tested and evaluated on synthetic and clinical 3D-2D image-pairs. The results on both databases indicate that although the proposed method was slightly less accurate, it significantly outperformed the state-of-the-art 3D-2D registration method in terms of robustness measured by capture range and success rate.

Robust 3-D/2-D registration of CT and MR to X-ray images based on gradient reconstruction

ObjectiveA novel 3-D/2-D registration method based on matching 3-D pre-interventional image gradients and coarsely reconstructed 3-D gradients from intra-interventional 2-D images is presented.Material and methodsThe novel method establishes correspondences between two sets of gradients by searching for correspondences along normals to anatomical structures in 3-D images, while the final correspondences are established in an iterative process, combining the robust random sample consensus algorithm (RANSAC) and a special gradient matching criterion function. The proposed method was evaluated by the publicly available standardized evaluation methodology for 3-D/2-D registration, consisting of 3-D rotational X-ray, computed tomography (CT), magnetic resonance (MR), and 2-D X-ray images of two spine segments, and evaluation criteria.ResultsPreliminary results show significant improve- ment in robustness (capture range and success rate) over three well established intensity-based, gradient-based, and reconstruction-based methods.ConclusionThe 3-D/2-D gradient reconstruction-based registration method efficiently combines the advantages of gradient and reconstruction-based methods, thereby enabling robust registration of CT and MR to only two X-ray images, while keeping the computational demands low.

A new image database for 3D/2D registration based on the visible human data set

Before an image registration method can be used in the medical theater a rigorous performance assessment of the registration method must be performed. In this paper, a new image database with a reference-based standardized evaluation methodology for objective evaluation and comparison of 3D/2D registration methods has been introduced. CT images of a female from the Visible Human Project® were used and 15 subvolumes each containing one of the vertebrae T3-T12 and L1-L5, and the pelvis were defined. Three pairs of lateral and anterior-posterior 2D fluoroscopic X-ray images were rendered from the CT data. Ray-casting algorithm with an energy conversion function was used to generate realistic fluoroscopic-like DRR images. Furthermore, outliers similar to medical intervention tools were also simulated on the 2D images. The assessment protocol to evaluate four criteria: accuracy, reliability, robustness and algorithm complexity, was defined. The proposed image database with the standardized evaluation methodology comprising ground truth registrations, displacements from the ground truth and target points is available upon request from the authors.