Dejan Tomaževič

A review of 3D/2D registration methods for image-guided interventions

Registration of pre- and intra-interventional data is one of the key technologies for image-guided radiation therapy, radiosurgery, minimally invasive surgery, endoscopy, and interventional radiology. In this paper, we survey those 3D/2D data registration methods that utilize 3D computer tomography or magnetic resonance images as the pre-interventional data and 2D X-ray projection images as the intra-interventional data. The 3D/2D registration methods are reviewed with respect to image modality, image dimensionality, registration basis, geometric transformation, user interaction, optimization procedure, subject, and object of registration.

Gold standard data for evaluation and comparison of 3D/2D registration methods.

Evaluation and comparison of registration techniques for image-guided surgery is an important problem that has received little attention in the literature. In this paper we address the challenging problem of generating reliable “gold standard” data for use in evaluating the accuracy of 3D/2D registrations. We have devised a cadaveric lumbar spine phantom with fiducial markers and established highly accurate correspondences between 3D CT and MR images and 18 2D X-ray images. The expected target registration errors for target points on the pedicles are less than 0.26 mm for CT-to-X-ray registration and less than 0.42 mm for MR-to-X-ray registration. As such, the “gold standard” data, which has been made publicly available on the Internet (http://lit.fe.uni-lj.si/Downloads/downloads.asp), is useful for evaluation and comparison of 3D/2D image registration methods.

Digital imaging as a process analytical technology tool for fluid-bed pellet coating process

Pellet coating processes are usually driven by fairly well optimized procedures, while coating suspension sprayed on pellets and adverse effects, such as agglomeration, can not be seen during coating process and are only detected at the very end of the process, when it is too late for any adjustments of the coating process. The aim of this study is to evaluate digital visual imaging as process analytical technology (PAT) tool for fluid-bed pellet coating processes. The method accurately estimates spherical diameter, coating thickness and adverse agglomeration of pellets by contactless measurements, classification and analysis of pellets based on digital imaging. Calibration and thorough assessment of the accuracy, precision, stability and speed of the proposed method was performed with high precision bearing balls. The obtained results on real pellets indicated that the method is feasible for real-time controlling, understanding, designing and optimizing of fluid-bed pellet coating processes according to PAT guidance.

Automated visual inspection of imprint quality of pharmaceutical tablets

Visual appearance is an important quality factor of pharmaceutical tablets. Moreover, it plays a key role in identification of tablets, which is needed to prevent mix-ups among various types of tablets. Since identification of tablets is most frequently done by imprints, good imprint quality, a property that makes the imprint readable, is of utmost importance in preventing mix-ups among the tablets. In this paper, we propose a novel method for automated visual inspection of tablets. Besides defect detection, imprint quality inspection is also considered. Performance of the method was evaluated on three different real tablet image databases of imprinted tablets. A “gold standard” was established by manually classifying tablets into a good and a defective class. The receiver operating characteristics (ROC) analysis indicated that the proposed method yields better sensitivity and specificity than the previous defect detection method.

Characterization of a spectrograph based hyperspectral imaging system

A significant part of the uniformity degradation in the acquired hyperspectral images can be attributed to the coregistration distortions and spectrally and spatially dependent resolution arising from the misalignments and the operation principle of the spectrograph based hyperspectral imaging system. The aim of this study was the development and validation of a practical method for characterization of the geometric coregistration distortions and position dependent resolution. The proposed method is based on modeling the imaging system response to several affordable reference objects. The results of the characterization can be used for calibration of the acquired images or as a tool for assessment of the expected errors in various hyperspectral imaging systems.

Standardized Evaluation of 2D-3D Registration

In the past few years a number of 2D-3D registration algorithms have been introduced. However, these methods have not been directly compared or only work for specific applications. Understanding and evaluating their performance is therefore an open and important issue. To address this challenge we introduce a standard evaluation method, which can be used for all types of methods and different applications. Our method uses the geometry of the 3D Rotational X-ray (3DRX) imaging system in combination with 3D-3D registration for attaining a highly accurate ground truth for 2D multiple X-ray to 3D MR/CT/3DRX registration. The data and ground truth transformations will be made available on the Internet. Furthermore, we propose starting positions and failure criteria to allow future researchers to directly compare their methods. As an illustration, the proposed method has been used to evaluate the performance of two 2D-3D registration techniques, viz. a gradient-based and an intensity-based method, in spinal applications.

3D/2D image registration: the impact of X-ray views and their number

An important part of image-guided radiation therapy or surgery is registration of a three-dimensional (3D) preoperative image to two-dimensional (2D) images of the patient. It is expected that the accuracy and robustness of a 3D/2D image registration method do not depend solely on the registration method itself but also on the number and projections (views) of intraoperative images. In this study, we systematically investigate these factors by using registered image data, comprising of CT and X-ray images of a cadaveric lumbar spine phantom and the recently proposed 3D/2D registration method. The results indicate that the proportion of successful registrations (robustness) significantly increases when more X-ray images are used for registration.

Optimizing bone extraction in MR images for 3D/2D gradient based registration of MR and x-ray images

A number of intensity and feature based methods have been proposed for 3D to 2D registration. However, for multimodal 3D/2D registration of MR and X-ray images, only hybrid and reconstruction-based methods were shown to be feasible. In this paper we optimize the extraction of features in the form of bone edge gradients, which were proposed for 3D/2D registration of MR and X-ray images. The assumption behind such multimodal registration is that the extracted gradients in 2D X-ray images match well to the corresponding gradients extracted in 3D MR images. However, since MRI and X-rays are fundamentally different modalities, the corresponding bone edge gradients may not appear in the same position and the the above-mentioned assumption may thus not be valid. To test the validity of this assumption, we optimized the extraction of bone edges in 3D MR and also in CT images for the registration to 2D X-ray images. The extracted bone edges were systematically displaced in the direction of their gradients, i.e. in the direction of the normal to the bone surface, and corresponding effects on the accuracy and convergence of 3D/2D registration were evaluated. The evaluation was performed on two different sets of MR, CT and X-ray images of spine phantoms with known gold standard, first consisting of five and the other of eight vertebrae. The results showed that a better registration can be obtained if bone edges in MR images are optimized for each application-specific MR acquisition protocol.

Gradient-based registration of 3D MR and 2D X-ray images

Abstract In this paper, we present a novel method for registering a 3D MR image to 2D X-ray images with the final goal to estimate the position and orientation of the patient during surgery or external beam radiotherapy. By using 3D MR preoperative images instead of CT images, we increase a soft tissue contrast and reduce the dose delivered to the patient. Such an approach, however, makes the registration difficult because the images of different modalities (MR and X-ray) and different domains (3D and 2D) need to be registered. We address this problem by finding the best match between 2D projection images of the 3D MR gradient image and the X-ray gradient image. The method was examined by using the Visible Human CT and MR data of the pelvis and hips. The experimental results show that in 60% of registrations a target registration error smaller than 10 mm was achieved.

Robust 3-D/2-D registration of CT and MR to X-ray images based on gradient reconstruction

ObjectiveA novel 3-D/2-D registration method based on matching 3-D pre-interventional image gradients and coarsely reconstructed 3-D gradients from intra-interventional 2-D images is presented.Material and methodsThe novel method establishes correspondences between two sets of gradients by searching for correspondences along normals to anatomical structures in 3-D images, while the final correspondences are established in an iterative process, combining the robust random sample consensus algorithm (RANSAC) and a special gradient matching criterion function. The proposed method was evaluated by the publicly available standardized evaluation methodology for 3-D/2-D registration, consisting of 3-D rotational X-ray, computed tomography (CT), magnetic resonance (MR), and 2-D X-ray images of two spine segments, and evaluation criteria.ResultsPreliminary results show significant improve- ment in robustness (capture range and success rate) over three well established intensity-based, gradient-based, and reconstruction-based methods.ConclusionThe 3-D/2-D gradient reconstruction-based registration method efficiently combines the advantages of gradient and reconstruction-based methods, thereby enabling robust registration of CT and MR to only two X-ray images, while keeping the computational demands low.