Davide Zerbato

Simulation of deformable environment with haptic feedback on GPU

Interactive simulations of deformable bodies are a growing research area with possible applications in several fields, i.e. computer aided surgery. The main implementation issue is to mimic the real behavior of the body at the extremely high rates required by haptic devices. Since even high-end computers have inadequate performance, one possible solution is to exploit the parallelism of modern Graphics Processing Units. In this paper we present our research aiming at moving the whole computational process from the CPU to the GPU taking advantage of the computational power of the graphics hardware. We use a mass-spring model, augmented with local damping coefficients and volume preservation forces. Collision detection is performed against external rigid bodies with high complexity mesh, such as the skeletonpsilas one. The user interacts with the model by controlling virtual tools, i.e. probes or tweezers. Haptic forces are computed on GPU and the results are asyncronously transferred to the CPU. Our approach can simulate the deformation of complex models with gravity and interaction with environment and tools at a frame rate higher than 1 KHz, making it suitable for visual rendering and haptic feedback.

GPU-based physical cut in interactive haptic simulations

PurposeInteractive, physics based, simulations of deformable bodies are a growing research area with possible applications to computer-aided surgery. Their aim is to create virtual environments where surgeons are free to practice. To ensure the needed realism, the simulations must be performed with deformable bodies. The goal of this paper is to describe the approach to the development of a physics-based surgical simulator with haptic feedback.MethodThe main development issue is the representation of the organ behavior at the high rates required by haptic realism. Since even high-end computers have inadequate performance, our approach exploits the parallelism of modern Graphics Processing Units (GPU). Particular attention is paid to the simulation of cuts because of their great importance in the surgical practice and the difficulty in handling topological changes in real time.ResultsTo prove the correctness of our approach, we simulated an interactive, physically based, virtual abdomen. The simulation allows the user to interact with deformable models. Deformable models are updated in real time, thus allowing the rendering of force feedback to the user. The method is optimized to handle high quality scenes: we report results of interactive simulation of two virtual tools interacting with a complex model.ConclusionsThe integration of physics-based deformable models in simulations greatly increases the realism of the virtual environment, taking into account real tissue properties and allowing the user to feel the actual forces exerted by organs on virtual tools. Our method proves the feasibility of exploiting GPU to simulate deformable models in interactive virtual environments.

Improving the development of surgical skills with virtual fixtures in simulation

This paper focuses on the use of virtual fixtures to improve the learning of basic skills for laparoscopic surgery. Five virtual fixtures are defined, integrated into a virtual surgical simulator and used to define an experimental setup based on a trajectory following task. 46 subjects among surgeons and residents underwent a training session based on the proposed setup. Their performance has been logged and used to identify the effect of virtual fixtures on the learning curve from the point of view of accuracy and completion time. Virtual fixtures prove to be effective in improving the learning and affect differently accuracy and completion time. This suggests the possibility to tailor virtual fixtures on the specific task requirements.

A unified representation to interact with simulated deformable objects in virtual environments

Deformations are an essential aspect of our interaction with real bodies, prompting the development of many modelling and simulation methods for virtual environments. Some of the methods address specific classes of interaction, e.g. pushing, grabbing, cutting or needle insertion, whereas hybrid approaches have been proposed to deal with more complex scenarios. However, a general strategy to combine different simulation methods is still not available. This paper presents a unified approach to combine different methods, each optimised for a specific interaction and object type. Our approach is tailored to the needs of simulating haptic Human-Robot Interactions and allows abstracting from the implementation details of different methods for modelling deformable objects. It has been integrated with collision detection and friction models, ported to a graphic processing unit (GPU), and demonstrated with realistic simulations and experiments.

Interactive constrained dynamics for rigid and deformable objects

Following the continuous increase in computational power of consumer hardware, interactive virtual environments have been recently enriched with more and more complex deformable objects. However, many physics engines are still very limited in the way they handle interacting rigid and deformable objects. This paper proposes a constraint‐based approach to real‐time simulation of coupled rigid and deformable objects capable of providing two‐way interactions. Similar techniques have seen widespread usage for either rigid or deformable objects, but not for the simultaneous simulation of both. By extending such approaches, we show not only how interaction is possible but also how it can be performed at real‐time rates. We address contact response and also show how to implement typical constraints to enforce limitations in the degrees of freedom and to enhance the dynamical properties of deformable objects. The method is easily integrated into existing physics engines that use similar constraint solvers and is independent on the kind of deformable object paradigm chosen. The provided simulation results show that the method is fast and effective in handling contacts between rigid and deformable objects and in simulating friction and other kinds of constraints. Copyright

Dynamics simulation for the training of teleoperated retrieval of spent nuclear fuel

This paper addresses the problem of training of operators for telemanipulation tasks. In particular, it describes the development of a physics based virtual environment that allows a user to train in the control of an innovative robotic tool designed for the retrieval of spent nuclear fuels. The robotic device is designed to adapt to very different environments, at the cost of an increased complexity in its control. The virtual environment provides realistic simulation of robot dynamics. The two most challenging tasks related to robot control have been identified and implemented in the simulation, leading to an effective tool for the training. The developed application is described in details and the outcome of one simulated intervention is proposed and analyzed in terms of user interaction and realism.

Integration of new features for telerobotic surgery into the MiroSurge system

Minimally invasive robotic surgery has gained wide acceptance recently. Computer-aided features to assist the surgeon during these interventions may help to develop safer, faster, and more accurate procedures. Especially physiological motion compensation of the beating heart and online soft tissue modelling are promising features that were developed recently. This paper presents the integration of these new features into the minimally invasive robotic surgery platform MiroSurge. A central aim of this research platform is to enable evaluation and comparison of new functionalities for minimally invasive robotic surgery. The system structure of MiroSurge is presented as well as the interfaces for the new functionalities. Some details about the modules for motion tracking and for soft tissue simulation are given. Results are shown with an experimental setup that includes a heart motion simulator and dedicated silicone organ models. Both features are integrated seamlessly and work reliably in the chosen setup. The MiroSurge platform thus shows the potential to provide valuable results in evaluating new functionalities for minimally invasive robotic surgery.

Preoperative workflow for lymph nodes staging

PurposeAccurate staging of lymph nodes relies mainly on surgical exploration and manual palpation. We present a new non-invasive diagnostic approach: simulated palpation through virtual laparoscopic instruments.MethodsWe set up a diagnostic process to extract lymph nodes shape and position from CTs and to analyze the trend of pixels intensities to determine tissue properties in order to feedback the force information.ResultsWe have integrated the model, obtained from both the morphological information and stiffness values, in our laparoscopy simulator and surgeons can virtually palpate, with a haptic device, the lymph nodes. We evaluated the workflow extracting lymph nodes from a case study: the feedback provided through the simulator greatly helps the surgeon in the correct staging.ConclusionsResults show the feasibility of the approach and in the future we will clinically evaluate this new diagnostic methodology. We are studying the possibility to integrate CTs with other imaging systems to increase the accuracy.

Performance enhancement with remote rendering for GPU based haptic simulation

Physical based simulations of deformable scenes require very high computational power but ensure realism and, if interactive, allow the haptic rendering to the user. Modern graphics processing units provide enough power to simulate complex models. Thanks to this achievement GPU based haptic simulators are gaining popularity, however, the correct timing of simulations and rendering still represents a neglected aspect of interactive physical simulations. In this work we propose a new method that enhances the computation of interactive deformable scenes on GPU by introducing remote rendering. The method improves the timings of physics simulation, thus leading to more realistic and stable interactive simulations. Moreover, the proposed method allows to increase the complexity of the graphic rendering engine used without any side effects on the physical simulation. Stereoscopic rendering and scene broadcasting can take advantage form this new technique. We applied the proposed method to our GPU based surgical simulator: analysis and preliminary results are presented.