Charles R. Hatt

Robust 5DOF Transesophageal Echo Probe Tracking at Fluoroscopic Frame Rates

Registration between transesophageal echocardiography TEE and x-ray fluoroscopy XRF has recently been introduced as a potentially useful tool for advanced image guidance of structural heart interventions. Algorithms for registration at fluoroscopic imaging frame rates 15-30 fps have yet to be reported, despite the fact that probe movement resulting from cardiorespiratory motion and physician manipulation can introduce non-trivial registration errors during untracked image frames. In this work, we present a novel algorithm for GPU-accelerated 2D/3D registration and apply it to the problem of TEE probe tracking in XRF sequences. Implementation in CUDA C resulted in an extremely fast similarity computation of < 80 μs, which in turn enabled registration frame rates ranging from 23.6-92.3 fps. The method was validated on simulated and clinical datasets and achieved target registration errors comparable to previously reported methods but at much faster registration speeds. Our results show, for the first time, the ability to accurately register TEE and XRF coordinate systems at fluoroscopic frame rates without the need for external hardware. The algorithm is generic and can potentially be applied to other 2D/3D registration problems where real-time performance is required.

MRI - 3D Ultrasound - X-ray Image Fusion with Electromagnetic Tracking for Transendocardial Therapeutic Injections: In-vitro Validation and In-vivo Feasibility

Myocardial infarction (MI) is one of the leading causes of death in the world. Small animal studies have shown that stem-cell therapy offers dramatic functional improvement post-MI. An endomyocardial catheter injection approach to therapeutic agent delivery has been proposed to improve efficacy through increased cell retention. Accurate targeting is critical for reaching areas of greatest therapeutic potential while avoiding a life-threatening myocardial perforation. Multimodal image fusion has been proposed as a way to improve these procedures by augmenting traditional intra-operative imaging modalities with high resolution pre-procedural images. Previous approaches have suffered from a lack of real-time tissue imaging and dependence on X-ray imaging to track devices, leading to increased ionizing radiation dose. In this paper, we present a new image fusion system for catheter-based targeted delivery of therapeutic agents. The system registers real-time 3D echocardiography, magnetic resonance, X-ray, and electromagnetic sensor tracking within a single flexible framework. All system calibrations and registrations were validated and found to have target registration errors less than 5 mm in the worst case. Injection accuracy was validated in a motion enabled cardiac injection phantom, where targeting accuracy ranged from 0.57 to 3.81 mm. Clinical feasibility was demonstrated with in-vivo swine experiments, where injections were successfully made into targeted regions of the heart.

Real-time 3D ultrasound guided interventional system for cardiac stem cell therapy with motion Compensation

This paper describes a clinically translatable interventional guidance platform to improve the accuracy and precision of stem cell injections into a beating heart. The proposed platform overlays live position of an injection catheter onto a fusion of a pre-procedural MR roadmap with real-time 3D transesophageal echocardiography (TEE). Electromagnetic (EM) tracking is used to initialize the fusion. The fusion is intra-operatively compensated for respiratory motion using a novel algorithm that uses peri-operative full volume ultrasound images. Validation of the system on a moving heart phantom produced a landmark registration accuracy of 2.8 +/- 1.45mm. Validation on animal in vivo data produced an average registration accuracy of 2.2 +/- 1.8 mm; indicating that it is feasible to reliably and robustly fuse the MR road-map with catheter position using 3D ultrasound in a clinical setting.

Real-time pose estimation of devices from x-ray images: Application to x-ray/echo registration for cardiac interventions

In recent years, registration between x-ray fluoroscopy (XRF) and transesophageal echocardiography (TEE) has been rapidly developed, validated, and translated to the clinic as a tool for advanced image guidance of structural heart interventions. This technology relies on accurate pose-estimation of the TEE probe via standard 2D/3D registration methods. It has been shown that latencies caused by slow registrations can result in errors during untracked frames, and a real-time ( > 15 hz) tracking algorithm is needed to minimize these errors. This paper presents two novel similarity metrics designed for accurate, robust, and extremely fast pose-estimation of devices from XRF images: Direct Splat Correlation (DSC) and Patch Gradient Correlation (PGC). Both metrics were implemented in CUDA C, and validated on simulated and clinical datasets against prior methods presented in the literature. It was shown that by combining DSC and PGC in a hybrid method (HYB), target registration errors comparable to previously reported methods were achieved, but at much higher speeds and lower failure rates. In simulated datasets, the proposed HYB method achieved a median projected target registration error (pTRE) of 0.33 mm and a mean registration frame-rate of 12.1 hz, while previously published methods produced median pTREs greater than 1.5 mm and mean registration frame-rates less than 4 hz. In clinical datasets, the HYB method achieved a median pTRE of 1.1 mm and a mean registration frame-rate of 20.5 hz, while previously published methods produced median pTREs greater than 1.3 mm and mean registration frame-rates less than 12 hz. The proposed hybrid method also had much lower failure rates than previously published methods.

Hough Forests for Real-Time, Automatic Device Localization in Fluoroscopic Images: Application to TAVR

A method for real-time localization of devices in fluoroscopic images is presented. Device pose is estimated using a Hough forest based detection framework. The method was applied to two types of devices used for transcatheter aortic valve replacement: a transesophageal echo TEE probe and prosthetic valve PV. Validation was performed on clinical datasets, where both the TEE probe and PV were successfully detected in 95.8% and 90.1% of images, respectively. TEE probe position and orientation errors were 1.42 ± 0.79 mm and 2.59i¾? ± 1.87i¾?, while PV position and orientation errors were 1.04 ± 0.77 mm and 2.90i¾? ± 2.37i¾?. The Hough forest was implemented in CUDA C, and was able to generate device location hypotheses in less than 50 ms for all experiments.

Efficient feature-based 2D/3D registration of transesophageal echocardiography to x-ray fluoroscopy for cardiac interventions

We present a novel 2D/ 3D registration algorithm for fusion between transesophageal echocardiography (TEE) and X-ray fluoroscopy (XRF). The TEE probe is modeled as a subset of 3D gradient and intensity point features, which facilitates efficient 3D-to-2D perspective projection. A novel cost-function, based on a combination of intensity and edge features, evaluates the registration cost value without the need for time-consuming generation of digitally reconstructed radiographs (DRRs). Validation experiments were performed with simulations and phantom data. For simulations, in silica XRF images of a TEE probe were generated in a number of different pose configurations using a previously acquired CT image. Random misregistrations were applied and our method was used to recover the TEE probe pose and compare the result to the ground truth. Phantom experiments were performed by attaching fiducial markers externally to a TEE probe, imaging the probe with an interventional cardiac angiographic x-ray system, and comparing the pose estimated from the external markers to that estimated from the TEE probe using our algorithm. Simulations found a 3D target registration error of 1.08(1.92) mm for biplane (monoplane) geometries, while the phantom experiment found a 2D target registration error of 0.69mm. For phantom experiments, we demonstrated a monoplane tracking frame-rate of 1.38 fps. The proposed feature-based registration method is computationally efficient, resulting in near real-time, accurate image based registration between TEE and XRF.

Depth-resolved registration of transesophageal echo to x-ray fluoroscopy using an inverse geometry fluoroscopy system

PURPOSE Image registration between standard x-ray fluoroscopy and transesophageal echocardiography (TEE) has recently been proposed. Scanning-beam digital x-ray (SBDX) is an inverse geometry fluoroscopy system designed for cardiac procedures. This study presents a method for 3D registration of SBDX and TEE images based on the tomosynthesis and 3D tracking capabilities of SBDX. METHODS The registration algorithm utilizes the stack of tomosynthetic planes produced by the SBDX system to estimate the physical 3D coordinates of salient key-points on the TEE probe. The key-points are used to arrive at an initial estimate of the probe pose, which is then refined using a 2D/3D registration method adapted for inverse geometry fluoroscopy. A phantom study was conducted to evaluate probe pose estimation accuracy relative to the ground truth, as defined by a set of coregistered fiducial markers. This experiment was conducted with varying probe poses and levels of signal difference-to-noise ratio (SDNR). Additional phantom and in vivo studies were performed to evaluate the correspondence of catheter tip positions in TEE and x-ray images following registration of the two modalities. RESULTS Target registration error (TRE) was used to characterize both pose estimation and registration accuracy. In the study of pose estimation accuracy, successful pose estimates (3D TRE < 5.0 mm) were obtained in 97% of cases when the SDNR was 5.9 or higher in seven out of eight poses. Under these conditions, 3D TRE was 2.32 ± 1.88 mm, and 2D (projection) TRE was 1.61 ± 1.36 mm. Probe localization error along the source-detector axis was 0.87 ± 1.31 mm. For the in vivo experiments, mean 3D TRE ranged from 2.6 to 4.6 mm and mean 2D TRE ranged from 1.1 to 1.6 mm. Anatomy extracted from the echo images appeared well aligned when projected onto the SBDX images. CONCLUSIONS Full 6 DOF image registration between SBDX and TEE is feasible and accurate to within 5 mm. Future studies will focus on real-time implementation and application-specific analysis.

Single-view geometric calibration for C-arm inverse geometry CT

Abstract. Accurate and artifact-free reconstruction of tomographic images requires precise knowledge of the imaging system geometry. A projection matrix-based calibration method to enable C-arm inverse geometry CT (IGCT) is proposed. The method is evaluated for scanning-beam digital x-ray (SBDX), a C-arm mounted inverse geometry fluoroscopic technology. A helical configuration of fiducials is imaged at each gantry angle in a rotational acquisition. For each gantry angle, digital tomosynthesis is performed at multiple planes and a composite image analogous to a cone-beam projection is generated from the plane stack. The geometry of the C-arm, source array, and detector array is determined at each angle by constructing a parameterized three-dimensional-to-two-dimensional projection matrix that minimizes the sum-of-squared deviations between measured and projected fiducial coordinates. Simulations were used to evaluate calibration performance with translations and rotations of the source and detector. The relative root-mean-square error in a reconstruction of a numerical thorax phantom was 0.4% using the calibration method versus 7.7% without calibration. In phantom studies, reconstruction of SBDX projections using the proposed method eliminated artifacts present in noncalibrated reconstructions. The proposed IGCT calibration method reduces image artifacts when uncertainties exist in system geometry.

A geometric calibration method for inverse geometry computed tomography using P-matrices

Accurate and artifact free reconstruction of tomographic images requires precise knowledge of the imaging system geometry. This work proposes a novel projection matrix (P-matrix) based calibration method to enable C-arm inverse geometry CT (IGCT). The method is evaluated for scanning-beam digital x-ray (SBDX), a C-arm mounted inverse geometry fluoroscopic technology. A helical configuration of fiducials is imaged at each gantry angle in a rotational acquisition. For each gantry angle, digital tomosynthesis is performed at multiple planes and a composite image analogous to a cone-beam projection is generated from the plane stack. The geometry of the C-arm, source array, and detector array is determined at each angle by constructing a parameterized 3D-to-2D projection matrix that minimizes the sum-of-squared deviations between measured and projected fiducial coordinates. Simulations were used to evaluate calibration performance with translations and rotations of the source and detector. In a geometry with 1 mm translation of the central ray relative to the axis-of-rotation and 1 degree yaw of the detector and source arrays, the maximum error in the recovered translational parameters was 0.4 mm and maximum error in the rotation parameter was 0.02 degrees. The relative rootmean- square error in a reconstruction of a numerical thorax phantom was 0.4% using the calibration method, versus 7.7% without calibration. Changes in source-detector-distance were the most challenging to estimate. Reconstruction of experimental SBDX data using the proposed method eliminated double contour artifacts present in a non-calibrated reconstruction. The proposed IGCT geometric calibration method reduces image artifacts when uncertainties exist in system geometry.

Dynamic tracking of prosthetic valve motion and deformation from bi-plane x-ray views: feasibility study

Transcatheter aortic valve replacement (TAVR) requires navigation and deployment of a prosthetic valve within the aortic annulus under fluoroscopic guidance. To support improved device visualization in this procedure, this study investigates the feasibility of frame-by-frame 3D reconstruction of a moving and expanding prosthetic valve structure from simultaneous bi-plane x-ray views. In the proposed method, a dynamic 3D model of the valve is used in a 2D/3D registration framework to obtain a reconstruction of the valve. For each frame, valve model parameters describing position, orientation, expansion state, and deformation are iteratively adjusted until forward projections of the model match both bi-plane views. Simulated bi-plane imaging of a valve at different signal-difference-to-noise ratio (SDNR) levels was performed to test the approach. 20 image sequences with 50 frames of valve deployment were simulated at each SDNR. The simulation achieved a target registration error (TRE) of the estimated valve model of 0.93 ± 2.6 mm (mean ± S.D.) for the lowest SDNR of 2. For higher SDNRs (5 to 50) a TRE of 0.04 mm ± 0.23 mm was achieved. A tabletop phantom study was then conducted using a TAVR valve. The dynamic 3D model was constructed from high resolution CT scans and a simple expansion model. TRE was 1.22 ± 0.35 mm for expansion states varying from undeployed to fully deployed, and for moderate amounts of inter-frame motion. Results indicate that it is feasible to use bi-plane imaging to recover the 3D structure of deformable catheter devices.