Dirk Fortmeier

Image-Based Palpation Simulation With Soft Tissue Deformations Using Chainmail on the GPU

Virtual reality surgery simulation can provide an environment for the safe training of medical interventions. In many of these interventions palpation of target organs is common to search for certain anatomical structures in a first step. We present a method for visuohaptic simulation with tissue deformation caused by palpation solely based on CT data of a patient. Generation of haptic force feedback involves a force parameter image based on distances to the patient’s skin and bone. To create a deformed version of the patient’s image data, the ChainMail method is applied; bone structures are considered to be undeformable. The simulation can be used to palpate the iliac crest and spinous processes for the preparation of a lumbar puncture or for palpation of the ribcage.

Evaluation of Direct Haptic 4D Volume Rendering of Partially Segmented Data for Liver Puncture Simulation

This work presents an evaluation study using a force feedback evaluation framework for a novel direct needle force volume rendering concept in the context of liver puncture simulation. PTC/PTCD puncture interventions targeting the bile ducts have been selected to illustrate this concept. The haptic algorithms of the simulator system are based on (1) partially segmented patient image data and (2) a non-linear spring model effective at organ borders. The primary aim is to quantitatively evaluate force errors caused by our patient modeling approach, in comparison to haptic force output obtained from using gold-standard, completely manually-segmented data. The evaluation of the force algorithms compared to a force output from fully manually segmented gold-standard patient models, yields a low mean of 0.12 N root mean squared force error and up to 1.6 N for systematic maximum absolute errors. Force errors were evaluated on 31,222 preplanned test paths from 10 patients. Only twelve percent of the emitted forces along these paths were affected by errors. This is the first study evaluating haptic algorithms with deformable virtual patients in silico. We prove haptic rendering plausibility on a very high number of test paths. Important errors are below just noticeable differences for the hand-arm system.

Direct Visuo-Haptic 4D Volume Rendering Using Respiratory Motion Models

This article presents methods for direct visuo-haptic 4D volume rendering of virtual patient models under respiratory motion. Breathing models are computed based on patient-specific 4D CT image data sequences. Virtual patient models are visualized in real-time by ray casting based rendering of a reference CT image warped by a time-variant displacement field, which is computed using the motion models at run-time. Furthermore, haptic interaction with the animated virtual patient models is provided by using the displacements computed at high rendering rates to translate the position of the haptic device into the space of the reference CT image. This concept is applied to virtual palpation and the haptic simulation of insertion of a virtual bendable needle. To this aim, different motion models that are applicable in real-time are presented and the methods are integrated into a needle puncture training simulation framework, which can be used for simulated biopsy or vessel puncture in the liver. To confirm real-time applicability, a performance analysis of the resulting framework is given. It is shown that the presented methods achieve mean update rates around 2,000 Hz for haptic simulation and interactive frame rates for volume rendering and thus are well suited for visuo-haptic rendering of virtual patients under respiratory motion.

GPU-Based Visualization of Deformable Volumetric Soft-Tissue for Real-Time Simulation of Haptic Needle Insertion

Virtual reality simulations can be used for training of surgery procedures such as needle insertion. Using a haptic force-feedback device a realistic virtual environment can be provided by computation of forces for specific patient data. This work presents an algorithm to calculate and visualize deformations of volumetric data representing softtissue inspired by the relaxation step of the ChainMail algorithm. It uses the coupling of haptic force-feedback computation and the deformation visualization algorithm to enhance the visual experience of our needle insertion training simulation. Real-time performance is achieved by implementing the relaxation on the GPU which outperforms a CPU-based implementation.

Patch-based label fusion using local confidence-measures and weak segmentations

A system for the fully automatic segmentation of the liver and spleen is presented. In a multi-atlas based segmentation framework, several existing segmentations are deformed in parallel to image intensity based registrations targeting the unseen patient. A new locally adaptive label fusion method is presented as the core of this paper. In a patch comparison approach, the transformed segmentations are compared to a weak segmentation of the target organ in the unseen patient. The weak segmentation roughly estimates the hidden truth. Traditional fusion approaches just rely on the deformed expert segmentations only. The result of patch comparison is a confidence weight for a neighboring voxel-label in the atlas label images to contribute to the voxel under study. Fusion is finally carried out in a weighted averaging scheme. The new contribution is the incorporation of locally determined confidence features of the unseen patient into the fusion process. For a small experimental set-up consisting of 12 patients, the proposed method performs favorable to standard classifier label fusion methods. In leave-one-out experiments, we obtain a mean Dice ratio of 0.92 for the liver and 0.82 for the spleen.

Random forest classification of large volume structures for visuo-haptic rendering in CT images

For patient-specific voxel-based visuo-haptic rendering of CT scans of the liver area, the fully automatic segmentation of large volume structures such as skin, soft tissue, lungs and intestine (risk structures) is important. Using a machine learning based approach, several existing segmentations from 10 segmented gold-standard patients are learned by random decision forests individually and collectively. The core of this paper is feature selection and the application of the learned classifiers to a new patient data set. In a leave-some-out cross-validation, the obtained full volume segmentations are compared to the gold-standard segmentations of the untrained patients. The proposed classifiers use a multi-dimensional feature space to estimate the hidden truth, instead of relying on clinical standard threshold and connectivity based methods. The result of our efficient whole-body section classification are multi-label maps with the considered tissues. For visuo-haptic simulation, other small volume structures would have to be segmented additionally. We also take a look into these structures (liver vessels). For an experimental leave-some-out study consisting of 10 patients, the proposed method performs much more efficiently compared to state of the art methods. In two variants of leave-some-out experiments we obtain best mean DICE ratios of 0.79, 0.97, 0.63 and 0.83 for skin, soft tissue, hard bone and risk structures. Liver structures are segmented with DICE 0.93 for the liver, 0.43 for blood vessels and 0.39 for bile vessels.

Efficient patient modeling for visuo-haptic VR simulation using a generic patient atlas

BACKGROUND AND OBJECTIVE This work presents a new time-saving virtual patient modeling system by way of example for an existing visuo-haptic training and planning virtual reality (VR) system for percutaneous transhepatic cholangio-drainage (PTCD). METHODS Our modeling process is based on a generic patient atlas to start with. It is defined by organ-specific optimized models, method modules and parameters, i.e. mainly individual segmentation masks, transfer functions to fill the gaps between the masks and intensity image data. In this contribution, we show how generic patient atlases can be generalized to new patient data. The methodology consists of patient-specific, locally-adaptive transfer functions and dedicated modeling methods such as multi-atlas segmentation, vessel filtering and spline-modeling. RESULTS Our full image volume segmentation algorithm yields median DICE coefficients of 0.98, 0.93, 0.82, 0.74, 0.51 and 0.48 regarding soft-tissue, liver, bone, skin, blood and bile vessels for ten test patients and three selected reference patients. Compared to standard slice-wise manual contouring time saving is remarkable. CONCLUSIONS Our segmentation process shows out efficiency and robustness for upper abdominal puncture simulation systems. This marks a significant step toward establishing patient-specific training and hands-on planning systems in a clinical environment.

Anisotropic Diffusion for Direct Haptic Volume Rendering in Lumbar Puncture Simulation

The segmentation of patient data often is mandatory for surgical simulations to enable realistic visual and haptic rendering. The necessary preparation time lies in the range from several hours to days. Here we augment a direct haptic volume rendering approach for lumbar punctures by edge-preserving smoothing preprocessing. Evaluation is carried out on user defined paths. Compared to our reference system force output can be improved over non-preprocessed image data.

Modellbasierte Simulation der Atembewegung für das Virtual-Reality-Training von Punktionseingriffen

Virtual-Reality-Simulatoren bieten Medizinern eine virtuelle Trainingsumgebung, in der Eingriffe kostengunstig trainiert und geplant werden konnen, ohne hierbei reale Patienten zu gefahrden. Eine Einschrankung der meisten VR-Trainingssimulatoren ist, dass sie von einem statischen Patienten ausgehen, dessen Anatomie im Bereich des simulierten Eingriffs wahrend der Simulation keiner durch die Atmung verursachten Bewegung unterliegt. In diesem Beitrag wird gezeigt, wie Methoden zur Modellierung und Schatzung der Atembewegung aus dem Bereich der Strahlentherapie bewegter Tumoren genutzt werden konnen, um eine realistische Simulation komplexer, variabler Atembewegungen in VR-Trainingssimulatoren zu erreichen. Die entwickelte Methodik erlaubt eine Visualisierung der Atembewegung in Echtzeit und ermoglicht eine haptische Interaktion mit dem atmenden virtuellen Korper. Dies wird exemplarisch fur das Szenario der Leberpunktion gezeigt.

Real-Time Resampling of Medical Images Based on Deformed Tetrahedral Structures for Needle Insertion VR-Simulation

To provide real-time visualization of deformed volumetric image data for virtual surgery simulation, a resampling algorithm based on deformed tetrahedral structures has been developed. Deformations of the tetrahedral mesh are computed by a soft-tissue simulation. The major advantage of this approach is the possibility to use the resampled image data in different rendering methods such as ray casting, simulated ultrasound or simulated X-ray imaging. To achieve real-time capability, the algorithm was parallelized on the GPU using Nvidia Cuda. Performance measurements have been done on different mesh resolutions. For a subset of 1688 tetrahedrons, short resamplings times of around 2.3 ms are measured on an Nvidia GTX 680, endorsing the algorithm for real-time application.