Matthias Mitschke

Merging visible and invisible: two Camera-Augmented Mobile C-arm (CAMC) applications

This paper presents the basic concept of CAMC and some of its applications. A CCD camera is attached to a mobile C-arm fluoroscopy X-ray system. Both optical and X-ray imaging systems are calibrated in the same coordinate system in an off-line process. The new system is able to provide X-ray and optical images simultaneously. The CAMC framework has great potential for medical augmented reality. We briefly introduce two new CAMC applications to the augmented reality research community. The first application aims at merging video images with a pre-computed tomographic reconstruction of the 3D volume of interest. This is a logical continuation of our work on 3D reconstruction using a CAMC (1999). The second approach is a totally new CAMC design where using a double mirror system and an appropriate calibration procedure the X-ray and optical images are merged in real-time. This new system enables the user to see an optical image, an X-ray image, or an augmented image where both visible and invisible are combined in real-time. The paper is organized in two independent sections describing each of the above. Experimental results are provided at the same time as the methods and apparatus are described for each section.

Respiratory correlated cone-beam computed tomography on an isocentric C-arm.

A methodology for 3D image reconstruction from retrospectively gated cone-beam CT projection data has been developed. A mobile x-ray cone-beam device consisting of an isocentric C-arm equipped with a flat panel detector was used to image a moving phantom. Frames for reconstruction were retrospectively selected from complete datasets based on the known rotation of the C-arm and a signal from a respiratory monitor. Different sizes of gating windows were tested. A numerical criterion for blur on the reconstructed image was suggested. The criterion is based on minimization of an Ising energy function, similar to approaches used in image segmentation or restoration. It is shown that this criterion can be used for the determination of the optimal gating window size. Images reconstructed from the retrospectively gated projection sequences using the optimal gating window data showed a significant improvement compared to images reconstructed from the complete projection datasets.

Improving 3D image quality of x-ray C-arm imaging systems by using properly designed pose determination systems for calibrating the projection geometry

C-arm volume reconstruction has become increasingly popular over the last years. These imaging systems generate 3D data sets for various interventional procedures such as endovascular treatment of aneurysms or orthopedic applications. Due to their open design and mechanical instability, C-arm imaging systems acquire projections along non-ideal scan trajectories. Volume reconstruction from filtered 2D X-ray projections requires a very precise knowledge of the imaging geometry. We show that the 3D image quality of C-arm cone beam imaging devices can be improved by proper design of the calibration phantom.

Recovering the X-ray projection geometry for three-dimensional tomographic reconstruction with additional sensors: Attached camera versus external navigation system

Three-dimensional tomographic reconstruction using intra-operative mobile C-arms could provide physicians with new and exciting tools for image-guided surgery. Recovery of the projection geometry of mobile X-ray systems is a crucial step for such reconstruction procedures. Recent work on medical imaging describes the use of optical or electro-magnetic sensor systems in order to navigate surgical instruments. These systems can also be used for the estimation of C-arm motion, and therefore for the recovery of the projection geometry of the X-ray C-arm. In this case, the mathematical problem that needs to be solved is equivalent to the hand-eye calibration well studied by both the computer vision and robotics community. We first study the recovery of the motion and projection geometry using five different hand-eye calibration methods proposed in the literature. The optical navigation system POLARIS from Northern Digital Inc. was used in our experiments. The results of the estimated motion and projection geometry using the five hand-eye calibration methods are compared with the same results obtained using an off-the-shelf CCD camera attached to the mobile C-arm. The experimental results include three-dimensional tomographic reconstruction results using our mobile C-arm. We show that even though the motion of the C-arm is more precisely recovered using the navigation system, the projection geometry is better estimated using the attached CCD camera.

Real-time face recognition using feature combination

We present an experimental setup for real time face identification in a cluttered scene. Color images of people are recorded with a static camera. A rough face detection is performed, and the resulting images are stored in a database. At a future time, a person standing in front of the camera (although against a different background) is identified, if their image was present in the database. In our experiments, the main variation of the faces is wide pose variation (out-of-image plane rotation of the head); some scale variation was also present. For real time ability, we use simple image features and a voting procedure for performing face recognition.

Camera-Augmented Mobile C-arm (CAMC) Application: 3D Reconstruction Using a Low-Cost Mobile C-arm

High-end X-ray C-arm gantries have recently been used for 3D reconstruction. Low-cost mobile C-arms enjoy the advantage of being readily available and are often used as interventional imaging device, but do not guarantee the reproducibility of their motion. The calibration and reconstruction process used for high-end C-arms cannot be applied to them. Camera-Augmented Mobile C-arm (CAMC) is the solution we propose. A CCD camera is attached to the (motorized) mobile C-arm in order to calibrate the C-arm’s projection geometry on-line. The relationship between X-ray and camera projection geometry is characterized in an off-line calibration process. We propose the notion of Virtual Detector (VD), which enables us to describe both optical and X-ray geometry as pinhole cameras with fixed intrinsic parameters. We have conducted experiments in order to compare the results of CAMC calibration with the calibration method used for high-end C-arms and using an optical tracking system (Polaris from Northern Digital, Inc.).

Interventions under Video-Augmented X-Ray Guidance: Application to Needle Placement

The camera augmented mobile C-arm (CAMC) has been introduced in [12] for the purpose of online geometrical calibration. Here, we propose its use for an augmented reality visualization. Introducing a double mirror system [11] the optical axes of both imaging systems (X-ray and optical) can be aligned. With this property both images can be merged or co-registered by only a planar transformation. This allows a real-time augmentation of X-ray and CCD camera images. We show that the needle placement procedure can be performed under this augmented reality visualization instead of fluoroscopy. Only two single X-ray images from different unknown C-arm positions are needed to align the needle to a target structure labeled by the surgeon in both X-ray images. The actual alignment is done by the surgeon while she/he sees the alignment of the needle to the target structure on video images co-registered with the X-ray image. Preliminary experimental results show the power of CAMC for medical augmented reality imaging. It also shows that this imaging system can provide surgeons with new possibilities for image guided surgery. In particular it reduces the X-ray exposures to both patient and physician.

Soft tissue visualization using a highly efficient megavoltage cone beam CT imaging system

Recent developments in two-dimensional x-ray detector technology have made volumetric Cone Beam CT (CBCT) a feasible approach for integration with conventional medical linear accelerators. The requirements of a robust image guidance system for radiation therapy include the challenging combination of soft tissue sensitivity with clinically reasonable doses. The low contrast objects may not be perceptible with MV energies due to the relatively poor signal to noise ratio (SNR) performance. We have developed an imaging system that is optimized for MV and can acquire Megavoltage CBCT images containing soft tissue contrast using a 6MV x-ray beam. This system is capable of resolving relative electron density as low as 1% with clinically acceptable radiation doses. There are many factors such as image noise, x-ray scatter, improper calibration and acquisitions that have a profound effect on the imaging performance of CBCT and in this study attempts were made to optimize these factors in order to maximize the SNR. A QC-3V phantom was used to determine the contrast to noise ratio (CNR) and f50 of a single 2-D projection. The computed f50 was 0.43 lp/mm and the CNR for a radiation dose of 0.02cGy was 43. Clinical Megavoltage CBCT images acquired with this system demonstrate that anatomical structures such as the prostate in a relatively large size patient are visible using radiation doses in range of 6 to 8cGy.

Retrieving Images by Content: The Surfimage System

Surfimage is a versatile content-based image retrieval system allowing both efficiency and flexibility, depending on the application. Surfimage uses the query-by-example approach for retrieving images and integrates advanced features such as image signature combination, multiple queries, query refinement, and partial queries. The classic and advanced features of Surfimage are detailed hereafter. Surfimage has been extensively tested on dozens of databases, demonstrating performance and robustness. Several experimental results are presented in the paper.

Cone-beam CT using a mobile C-arm: a registration solution for IGRT with an optical tracking system.

A method for registering images acquired from a prototype flat panel mobile C-arm, capable of kilovoltage (kV) cone-beam computed tomography (CT), to a linear accelerator (LINAC) isocenter is presented. A calibration procedure is performed which involves locating reflective markers placed on the C-arm and a phantom in two coordinate systems. A commercial optical tracking system locates the markers relative to the LINAC isocenter (room coordinates). The cone-beam imaging capabilities of the C-arm provide the location of the markers on the calibration phantom in image coordinates. A singular value decomposition (SVD) algorithm is used to determine the relationship between the C-arm, image coordinates and room coordinates. Once the calibration is completed, the position of the C-arm at any arbitrary location is accurately determined from the tracking system. A final transformation is calculated capable of mapping voxels in the reconstructed image set to their corresponding position in room coordinates. An evaluation to determine the accuracy of this method was performed by locating markers on a phantom. The position of the phantom markers in room coordinates was obtained directly using the optical tracking system and compared with that using the described method above. A mean absolute distance of 1.4+/-0.5 was observed for a completely transformed image set. This is comparable to that of systems routinely used for image-guided radiation therapy (IGRT).