Igor Peterlik

Image-guided simulation of heterogeneous tissue deformation for augmented reality during hepatic surgery

This paper presents a method for real-time augmentation of vascular network and tumors during minimally invasive liver surgery. Internal structures computed from pre-operative CT scans can be overlaid onto the laparoscopic view for surgery guidance. Compared to state-of-the-art methods, our method uses a real-time biomechanical model to compute a volumetric displacement field from partial three-dimensional liver surface motion. This permits to properly handle the motion of internal structures even in the case of anisotropic or heterogeneous tissues, as it is the case for the liver and many anatomical structures. Real-time augmentation results are presented on in vivo and phantom data and illustrate the benefits of such an approach for minimally invasive surgery.

Constraint-Based Haptic Rendering of Multirate Compliant Mechanisms

The paper is dedicated to haptic rendering of complex physics-based environment in the context of surgical simulation. A new unified formalism for modeling the mechanical interactions between medical devices and anatomical structures and for computing accurately the haptic force feedback is presented. The approach deals with the mechanical interactions using appropriate force and/or motion transmission models named compliant mechanisms. These mechanisms are formulated as a constraint-based problem that is solved in two separate threads running at different frequencies. The first thread processes the whole simulation including the soft-tissue deformations, whereas the second one only deals with computer haptics. This method builds a bridge between the so-called virtual mechanisms (that were proposed for haptic rendering of rigid bodies) and intermediate representations (used for rendering of complex simulations). With this approach, it is possible to describe the specific behavior of various medical devices while relying on a unified method for solving the mechanical interactions between deformable objects and haptic rendering. The technique is demonstrated in interactive simulation of flexible needle insertion through soft anatomical structures with force feedback.

Modeling and real-time simulation of a vascularized liver tissue

In Europe only, about 100,000 deaths per year are related to cirrhosis or liver cancer. While surgery remains the option that offers the foremost success rate against such pathologies, several limitations still hinder its widespread development. Among the limiting factors is the lack of accurate planning systems, which has been a motivation for several recent works, aiming at better resection planning and training systems, relying on pre-operative imaging, anatomical and biomechanical modelling. While the vascular network in the liver plays a key role in defining the operative strategy, its influence at a biomechanical level has not been taken into account. In the paper we propose a real-time model of vascularized organs such as the liver. The model takes into account separate constitutive laws for the parenchyma and vessels, and defines a coupling mechanism between these two entities. In the evaluation section, we present results of in vitro porcine liver experiments that indicate a significant influence of vascular structures on the mechanical behaviour of tissue. We confirm the values obtained in the experiments by computer simulation using standard FEM. Finally, we show that the conventional modelling approach can be efficiently approximated with the proposed composite model capable of real-time calculations.

Technical Section: Real-time visio-haptic interaction with static soft tissue models having geometric and material nonlinearity

Realistic soft tissue models running in real-time are required for the development of computer-based surgical training systems. To construct a realistic soft tissue model, finite element (FE) modeling techniques are preferred over the particle-based techniques since the material properties can be integrated directly into the FE model to provide more accurate visual and haptic feedback to a user during the simulations. However, running even a static (time-independent) nonlinear FE model in real-time is a highly challenging task because the resulting stiffness matrix (K) is not constant and varies with the depth of penetration into the model. We propose a new computational approach allowing visio-haptic interaction with an FE model of a human liver having both nonlinear geometric and material properties. Our computational approach consists of two main steps: a pre-computation of the configuration space of all deformation configurations of the model, followed by the interpolation of the precomputed data for the calculation of the nodal displacements and reaction forces that are displayed to the user during the real-time interactions through a visual display and a haptic device, respectively. For the implementation of the proposed approach, no a priori assumptions or modeling simplifications about the mathematical complexity of the underlying soft tissue model, size and irregularity of the FE mesh are necessary. Moreover, it turns out that the deformation and force responses of the liver in the simulations are heavily influenced by the selected simulation parameters, such as the material model, boundary conditions and loading path, but the stability of the visual and haptic rendering in our approach does not depend on these parameters. In addition to showing the stability of our approach, the length of the precomputations as well as the accuracy of the interpolation scheme are evaluated for different interpolation functions and configuration space densities.

Distributed Construction of Configuration Spaces for Real-Time Haptic Deformation Modeling

Haptic rendering is an important area of research that enables the user to employ haptic perception in human-computer interaction. An important motivation here is to use the human touch to study the behavior of various models. However, the high refresh rate needed for a stable haptic interaction on the one hand and the high-computational-cost characteristic for the simulation of numerous phenomena on the other hand represent a big issue. In this paper, an approach based on the distributed construction of configuration spaces is presented. The main idea behind this approach is to profit from employing a high-performance environment (e.g., computational grid) to overcome or at least moderate the high-frequency issue. The approach is presented using nonlinear deformation models, which are essential for realistic modeling of soft tissues. A distributed algorithm is presented, and its properties are evaluated quantitatively.

Impact of Soft Tissue Heterogeneity on Augmented Reality for Liver Surgery

This paper presents a method for real-time augmented reality of internal liver structures during minimally invasive hepatic surgery. Vessels and tumors computed from pre-operative CT scans can be overlaid onto the laparoscopic view for surgery guidance. Compared to current methods, our method is able to locate the in-depth positions of the tumors based on partial three-dimensional liver tissue motion using a real-time biomechanical model. This model permits to properly handle the motion of internal structures even in the case of anisotropic or heterogeneous tissues, as it is the case for the liver and many anatomical structures. Experimentations conducted on phantom liver permits to measure the accuracy of the augmentation while real-time augmentation on in vivo human liver during real surgery shows the benefits of such an approach for minimally invasive surgery.

An Algorithm of State-Space Precomputation Allowing Non-linear Haptic Deformation Modelling Using Finite Element Method

After presenting the mathematical background of modelling elastic deformations together with finite element formulation, we propose a new algorithm allowing haptic interaction with soft tissues having both non-linear geometric and physical properties. The algorithm consists of two phases -first the configuration space is precomputed in a high-performance possibly distributed environment, then the precomputed data are used during the haptic interaction. In the paper, we focus mainly on sequential description of the first phase of the algorithm. We present some preliminary experimental results concerning the accuracy of the proposed method as well and the sketch of further extensions

Atlas-Based Transfer of Boundary Conditions for Biomechanical Simulation

An environment composed of different types of living tissues (such as the abdominal cavity) reveals a high complexity of boundary conditions, which are the attachments (e.g. connective tissues, ligaments) connecting different anatomical structures. Together with the material properties, the boundary conditions have a significant influence on the mechanical response of the organs, however corresponding correct mechanical modeling remains a challenging task, as the connective structures are difficult to identify in certain standard imaging modalities. In this paper, we present a method for automatic modeling of boundary conditions in deformable anatomical structures, which is an important step in patient-specific biomechanical simulations. The method is based on a statistical atlas which gathers data defining the connective structures attached to the organ of interest. In order to transfer the information stored in the atlas to a specific patient, the atlas is registered to the patient data using a physics-based technique and the resulting boundary conditions are defined according to the mean position and variance available in the atlas. The method is evaluated using abdominal scans of ten patients. The results show that the atlas provides a sufficient information about the boundary conditions which can be reliably transferred to a specific patient. The boundary conditions obtained by the atlas-based transfer show a good match both with actual segmented boundary conditions and in terms of mechanical response of deformable organs.

Preoperative trajectory planning for percutaneous procedures in deformable environments

In image-guided percutaneous interventions, a precise planning of the needle path is a key factor to a successful intervention. In this paper we propose a novel method for computing a patient-specific optimal path for such interventions, accounting for both the deformation of the needle and soft tissues due to the insertion of the needle in the body. To achieve this objective, we propose an optimization method for estimating preoperatively a curved trajectory allowing to reach a target even in the case of tissue motion and needle bending. Needle insertions are simulated and regarded as evaluations of the objective function by the iterative planning process. In order to test the planning algorithm, it is coupled with a fast needle insertion simulation involving a flexible needle model and soft tissue finite element modeling, and experimented on the use-case of thermal ablation of liver tumors. Our algorithm has been successfully tested on twelve datasets of patient-specific geometries. Fast convergence to the actual optimal solution has been shown. This method is designed to be adapted to a wide range of percutaneous interventions.

Towards an accurate tracking of liver tumors for augmented reality in robotic assisted surgery

This article introduces a method for tracking the internal structures of the liver during robot-assisted procedures. Vascular network, tumors and cut planes, computed from pre-operative data, can be overlaid onto the laparoscopic view for image-guidance, even in the case of large motion or deformation of the organ. Compared to current methods, our method is able to precisely propagate surface motion to the internal structures. This is made possible by relying on a fast yet accurate biomechanical model of the liver combined with a robust visual tracking approach designed to properly constrain the model. Augmentation results are demonstrated on in-vivo sequences of a human liver during robotic surgery, while quantitative validation is performed on an ex-vivo porcine liver experimentation. Validation results show that our approach gives an accurate surface registration with an error of less than 6mm on the position of the tumor.