Thiago R. Dos Santos

Physics‐based shape matching for intraoperative image guidance

PURPOSE Soft-tissue deformations can severely degrade the validity of preoperative planning data during computer assisted interventions. Intraoperative imaging such as stereo endoscopic, time-of-flight or, laser range scanner data can be used to compensate these movements. In this context, the intraoperative surface has to be matched to the preoperative model. The shape matching is especially challenging in the intraoperative setting due to noisy sensor data, only partially visible surfaces, ambiguous shape descriptors, and real-time requirements. METHODS A novel physics-based shape matching (PBSM) approach to register intraoperatively acquired surface meshes to preoperative planning data is proposed. The key idea of the method is to describe the nonrigid registration process as an electrostatic-elastic problem, where an elastic body (preoperative model) that is electrically charged slides into an oppositely charged rigid shape (intraoperative surface). It is shown that the corresponding energy functional can be efficiently solved using the finite element (FE) method. It is also demonstrated how PBSM can be combined with rigid registration schemes for robust nonrigid registration of arbitrarily aligned surfaces. Furthermore, it is shown how the approach can be combined with landmark based methods and outline its application to image guidance in laparoscopic interventions. RESULTS A profound analysis of the PBSM scheme based on in silico and phantom data is presented. Simulation studies on several liver models show that the approach is robust to the initial rigid registration and to parameter variations. The studies also reveal that the method achieves submillimeter registration accuracy (mean error between 0.32 and 0.46 mm). An unoptimized, single core implementation of the approach achieves near real-time performance (2 TPS, 7-19 s total registration time). It outperforms established methods in terms of speed and accuracy. Furthermore, it is shown that the method is able to accurately match partial surfaces. Finally, a phantom experiment demonstrates how the method can be combined with stereo endoscopic imaging to provide nonrigid registration during laparoscopic interventions. CONCLUSIONS The PBSM approach for surface matching is fast, robust, and accurate. As the technique is based on a preoperative volumetric FE model, it naturally recovers the position of volumetric structures (e.g., tumors and vessels). It cannot only be used to recover soft-tissue deformations from intraoperative surface models but can also be combined with landmark data from volumetric imaging. In addition to applications in laparoscopic surgery, the method might prove useful in other areas that require soft-tissue registration from sparse intraoperative sensor data (e.g., radiation therapy).

Particle filtering for respiratory motion compensation during navigated bronchoscopy

Although the field of a navigated bronchoscopy gains increasing attention in the literature, robust guidance in the presence of respiratory motion and electromagnetic noise remains challenging. The robustness of a previously introduced motion compensation approach was increased by taking into account the already traveled trajectory of the instrument within the lung. To evaluate the performance of the method a virtual environment, which accounts for respiratory motion and electromagnetic noise was used. The simulation is based on a deformation field computed from human computed tomography data. According to the results, the proposed method outperforms the original method and is suitable for lung motion compensation during electromagnetically guided interventions.

Correspondences search for surface-based intra-operative registration

Intra-operative registration is one of the main challenges related to computer-assisted interventions. One common approach involves matching intra-operatively acquired surfaces (e.g. from a laser range scanner) to pre-operatively acquired planning data. In this paper, we propose a new method for correspondences search between surfaces, which can be used for the computation of an initial alignment. It generates graph representations and establishes correspondences by maximizing a global similarity measure. The method does not rely on landmarks or prominent surface characteristics and is independent on the initial pose of the surfaces relative to each other. According to an evaluation on a set of liver meshes, the method is able to correctly match small submeshes even in this presence of noise.

MITK-ToF—Range data within MITK

PurposeThe time-of-flight (ToF) technique is an emerging technique for rapidly acquiring distance information and is becoming increasingly popular for intra-operative surface acquisition. Using the ToF technique as an intra-operative imaging modality requires seamless integration into the clinical workflow. We thus aim to integrate ToF support in an existing framework for medical image processing.MethodsMITK-ToF was implemented as an extension of the open-source C++ Medical Imaging Interaction Toolkit (MITK) and provides the basic functionality needed for rapid prototyping and development of image-guided therapy (IGT) applications that utilize range data for intra-operative surface acquisition. This framework was designed with a module-based architecture separating the hardware-dependent image acquisition task from the processing of the range data.ResultsThe first version of MITK-ToF has been released as an open-source toolkit and supports several ToF cameras and basic processing algorithms. The toolkit, a sample application, and a tutorial are available from http://mitk.org.ConclusionsWith the increased popularity of time-of-flight cameras for intra-operative surface acquisition, integration of range data supports into medical image processing toolkits such as MITK is a necessary step. Handling acquisition of range data from different cameras and processing of the data requires the establishment and use of software design principles that emphasize flexibility, extendibility, robustness, performance, and portability. The open-source toolkit MITK-ToF satisfies these requirements for the image-guided therapy community and was already used in several research projects.

Adaptive bilateral filter for image denoising and its application to in-vitro Time-of-Flight data

Image-guided therapy systems generally require registration of pre-operative planning data with the patients anatomy. One common approach to achieve this is to acquire intra-operative surface data and match it to surfaces extracted from the planning image. Although increasingly popular for surface generation in general, the novel Time-of-Flight (ToF) technology has not yet been applied in this context. This may be attributed to the fact that the ToF range images are subject to considerable noise. The contribution of this study is two-fold. Firstly, we present an adaption of the well-known bilateral filter for denoising ToF range images based on the noise characteristics of the camera. Secondly, we assess the quality of organ surfaces generated from ToF range data with and without bilateral smoothing using corresponding high resolution CT data as ground truth. According to an evaluation on five porcine organs, the root mean squared (RMS) distance between the denoised ToF data points and the reference computed tomography (CT) surfaces ranged from 3.0 mm (lung) to 9.0 mm (kidney). This corresponds to an error-reduction of up to 36% compared to the error of the original ToF surfaces.

Robust multi-modal surface matching for intra-operative registration

Range imaging modalities, such as time-of-flight cameras (ToF), are becoming popular for the acquisition of intra-operative data, which can be used for registering the patients anatomy with pre-operative data (e.g. Computed Tomography or Magnetic Resonance Imaging). As pre- and intra-operative data are acquired with different modalities, and are subject to different sources of error (systematic errors and noise), matching them is a challenging problem. We present a method for the registration of multi-modal surfaces, which is robust to high noise levels and is applicable to incomplete surfaces. An evaluation with simulated and real data demonstrates that the presented method is very accurate and is fast enough for intra-operative purposes.

Minimally deformed correspondences between surfaces for intra-operative registration

Range imaging modalities, such as time-of-flight cameras (ToF), are becoming very popular for the acquisition of intra-operative data, which can be used for registering the patients anatomy with pre-operative data, such as 3D images generated by computed tomographies (CT) or magnetic resonance imaging (MRI). However, due to the distortions that appear because of the different acquisition principles of the input surfaces, the noise, and the deformations that may occur in the intra-operative environment, we face different surface properties for points lying on the same anatomical locations and unreliable feature points detection, which are crucial for most surface matching algorithms. In order to overcome these issues, we present a method for automatically finding correspondences between surfaces that searches for minimally deformed configurations. For this purpose, an error metric that expresses the reliability of a correspondence set based on its spatial configuration is employed. The registration error is minimized by a combinatorial analysis through search-trees. Our method was evaluated with real and simulated ToF and CT data, and showed to be reliable for the registration of partial multi-modal surfaces with noise and distortions.

Generation of Triangle Meshes from Time-of-Flight Data for Surface Registration

One approach to intra-operative registration in computerassisted medical interventions involves matching intra-operatively acquired organ surfaces with pre-operatively generated high resolution surfaces. The matching is based on so-called curvature descriptors assigned to the vertices of the two meshes. Therefore, high compliance of the input meshes with respect to curvature properties is essential. Time-of-Flight cameras can provide the required surface data during the intervention as a point cloud. Although different methods for generation of triangle meshes from range data have been proposed in the literature, their effect on the quality of the mesh with respect to curvature properties has not yet been investigated. In this paper, we evaluate six of these methods and derive application-specific recommendations for their usage.

Multi-modal surface comparison and its application to intra-operative range data

Time-of-flight (ToF) cameras are a novel, fast, and robust means for intra-operative 3D surface acquisition. They acquire surface information (range images) in real-time. In the intra-operative registration context, these surfaces must be matched to pre-operative CT or MR surfaces, using so called descriptors, which represent surface characteristics. We present a framework for local and global multi-modal comparison of surface descriptors and characterize the differences between ToF and CT data in an in vitro experiment. The framework takes into account various aspects related to the surface characteristics and does not require high resolution input data in order to establish appropriate correspondences. We show that the presentation of local and global comparison data allows for an accurate assessment of ToF-CT discrepancies. The information gained from our study may be used for developing ToF pre-processing and matching algorithms, or for improving calibration procedures for compensating systematic distance errors. The framework is available in the open-source platform Medical Imaging Interaction Toolkit (MITK).

Time-of-flight surface De-noising by spectral decomposition

An increasingly popular approach to the acquisition of intraoperative data is the novel Time-of-Flight (ToF) camera technique, which provides surface information with high update rates. This information can be used for intra-operative registration with pre-operative data through surface matching techniques. However, ToF data is subject to different systematic errors and noise, which must be eliminated for the purposes of matching with high-quality pre-operative data. While methods for de-noising of data concentrate on the processing of the range images, we focus directly on the surfaces. We decompose the frequency spectrum of the surface and use it for the computation of a low-pass filter, thus eliminating all the higher frequencies on the surface (noise). The low-pass filter was evaluated on in vitro data and was compared to a previously published method for ToF de-noising, which takes advantage of the fast data acquisition provided by the ToF technology. In almost all cases, the low-pass filter showed a better performance. Decomposition of the frequency spectrum of surfaces allows not only filtering and de-noising, but also the application of other valuable signal processing methods, such as enhancement or homogenization.