Francois Conti

Virtual reality simulation in neurosurgery: technologies and evolution.

Neurosurgeons are faced with the challenge of learning, planning, and performing increasingly complex surgical procedures in which there is little room for error. With improvements in computational power and advances in visual and haptic display technologies, virtual surgical environments can now offer potential benefits for surgical training, planning, and rehearsal in a safe, simulated setting. This article introduces the various classes of surgical simulators and their respective purposes through a brief survey of representative simulation systems in the context of neurosurgery. Many technical challenges currently limit the application of virtual surgical environments. Although we cannot yet expect a digital patient to be indistinguishable from reality, new developments in computational methods and related technology bring us closer every day. We recognize that the design and implementation of an immersive virtual reality surgical simulator require expert knowledge from many disciplines. This article highlights a selection of recent developments in research areas related to virtual reality simulation, including anatomic modeling, computer graphics and visualization, haptics, and physics simulation, and discusses their implication for the simulation of neurosurgery.

Spanning large workspaces using small haptic devices

Exploring large virtual environments using a small haptic device with limited-workspace capabilities is a challenging task because the user very quickly reaches the borders of the physical workspace of the device. In the case of a computer mouse, this problem is solved by lifting the device off the table and repositioning it at a different location. With most ground-based haptic devices such indexing procedure is not possible and requires the use of an additional switch to decouple the device from the cursor and allow the user to relocate the end-effector at the center of the physical workspace. Below certain physical workspace dimensions such indexing methods become cumbersome to the operator and therefore different control paradigms are required. This paper presents a new approach referred to as workspace drift control which progressively relocates the physical workspace of the device mapped inside the virtual environment towards the area of interest of the operator without disturbing his or her perception of the environment. This technique uses the fact that people are greatly influenced by what they perceive visually and often do not notice small deviations of their hand unless that small deviation has a corresponding visual component.

Robotics and interactive simulation

As applications of robots extend into everyday human life, new approaches to simulating interactions between them and their environments are emerging at the intersection of the physical and virtual worlds.

The sigma.7 haptic interface for MiroSurge: A new bi-manual surgical console

This paper presents the design and control of the sigma.7 haptic device and the new surgical console of the MiroSurge robotic system. The console and the haptic devices are designed with respect to requirements in minimally invasive robotic surgery. Dedicated left and right handed devices are integrated in an operator console in an ergonomic configuration. The height of the whole console is adjustable, allowing the surgeon seated and standed operation. Each of the devices is fully actuated in seven degrees of freedom (DoF). A parallel mechanism with 3 DoF actuates the translational motion and an attached wrist with 3 intersecting axis drives the rotations of the grasping unit. This advantageous design leads to inherently decoupled kinematics and dynamics. Cartesian forces are 20 N within the translational workspace, which is a sphere of about 120 mm diameter for each device. The rotational wrist of the device covers the whole workspace of the human hand and provides maximum torques of about 0.4 Nm. The grasping unit can display forces up to 8 N. An integrated force/torque sensor is used to increase the transparency of the devices by reducing inertia and friction. It is theoretically shown that the non-linear closed loop system behaves like a passive system and experimental results validate the approach. The sigma.7 haptic devices are designed by Force Dimension in cooperation with the German Aerospace Center (DLR). DLR designed the surgical console and integrated the haptic devices in the MiroSurge system.

Interactive rendering of deformable objects based on a filling sphere modeling approach

Mass-spring systems have widely and effectively been used for modeling in real-time deformable objects. Easier to implement and faster than finite elements, these systems, on the other side, suffer from several drawbacks when coming to render physically believable behaviors. Neither isotropic or anisotropic materials can be controlled easily and the large number of springs and mass points composing the model makes it fastidious to define parameters to control elongation, flexion and torsion at a macroscopic level. Another weakness is that most of the materials found in nature maintain a constant or quasi-constant volume during deformations; unfortunately, mass-spring models do not have this property. In this paper, we extend the current state-of-the-art in soft tissue simulation by introducing a six-degree of freedom macroscopic elastic sphere described by mass, inertia and volumetric properties. Spheres are placed along the medial axis transform of the object whose centers are connected by a skeleton composed of a set of three-dimensional elastic links. Spheres represent internal mass, volume and control the global deformation of the object. The surface is modeled by setting point masses on the mesh nodes and damped springs on the mesh edges. These nodes are connected to the skeleton by individual elastic links, which control volume conservation and transfer forces between the surface and volumetric model. Using this framework we also present an efficient method to approximate collision detection between multiple bodies in real-time.

Interface Design and Control Strategies for a Robot Assisted Ultrasonic Examination System

This paper presents a new robotic system designed to assist sonographers in performing ultrasound examinations by addressing common limitations of sonography, namely the physical fatigue that can result from performing the examination, and the difficulty in interpreting ultrasound data. The proposed system comprises a robot manipulator that operates the transducer, and an integrated user interface that offers 3D visualization and a haptic device as the main user interaction tool. The sonographer controls the slave robot movements either haptically (collaborative tele-operation mode), or by prior programming of a desired path (semi-automatic mode). A force controller maintains a constant contact force between the transducer and the patient’s skin while the robot drives the transducer to the desired anatomical locations. The ultrasound imaging system is connected to a 3D visualization application which registers in real time the streaming 2D images generated by the transducer and displays the resulting data as 3D volumetric representation which can be further examined off-line.

Constraint-based six degree-of-freedom haptic rendering of volume-embedded isosurfaces

A method for 6-DOF haptic rendering of isosurface geometry embedded within sampled volume data is presented. The algorithm uses a quasi-static formulation of motion constrained by multiple contacts to simulate rigid-body interaction between a haptically controlled virtual tool (proxy), represented as a point-sampled surface, and volumetric isosurfaces. Unmodified volume data, such as computed tomography or magnetic resonance images, can be rendered directly with this approach, making it particularly suitable for applications in medical or surgical simulation. The algorithm was implemented and tested on a variety of volume data sets using several virtual tools with different geometry. As the constraint-based algorithm permits simulation of a massless proxy, no artificial mass or inertia were needed nor observed. The speed and transparency of the algorithm allowed motion to be responsive to extremely stiff contacts with complex virtualized geometry. Despite rendering stiffnesses that approach the physical limits of the interfaces used, the simulation remained stable through haptic interactions that typically present a challenge to other rendering methods, including wedging, prying, and hooking.

A New Actuation Approach for Haptic Interface Design

Traditional haptic interfaces available today use motors to generate forces, while a more recent class of devices uses passive elements to constrain movement. This paper presents a new hybrid actuation approach that combines the use of brakes, springs and motors. The proposed actuation design is potentially safer and more energy efficient than haptic devices that only rely on motors for actuation, and also overcomes many of the rendering limitations displayed by existing passive haptic displays. Applications of this new technology range from devices where safety and reliability are of prime concerns (e.g. large force-feedback interfaces) to devices which can only be powered by limited energy sources such as small batteries (e.g. portable haptic interfaces).

The Delta Haptic Device as a Nanomanipulator

At the EPFL, we have developed a force-feedback device and control architecture for high-end research and industrial applications. The Delta Haptic Device (DHD) consists of a 6 degrees-of-freedom (DOF) mecatronic device driven by a PC. Several experiments have been carried out in the fields of manipulation and simulation to assess the dramatic improvement haptic information brings to manipulation. This system is particularly well suited for scaled manipulation such as micro-, nano- and biomanipulation. Not only can it perform geometric and force scaling, but it can also include fairly complex physical models into the control loop to assist manipulation and enhance human understanding of the environment. To demonstrate this ability, we are currently interfacing our DHD with an atomic force microscope (AFM). In a first stage, we will be able to feel in real-time the topology of a given sample while visualizing it in 3D. The aim of the project is to make manipulation of carbon nanotubes possible by including physical models of such nanotubes behavior into the control loop, thus allowing humans to control complex structures. In this paper, we give a brief description of our device and present preliminary results of its interfacing with the AFM.

A Hybrid Actuation Approach for Haptic Devices

This paper presents a new actuation approach which combines the use of brakes, springs and mini motors to produce a safer and more energy efficient way to drive haptic devices. The applications which can greatly benefit from this new technology include force-feedback interfaces which operate medical robots, an area where safety and reliability are of prime concerns, and small portable devices which can only be powered by limited energy sources such as small batteries. This work also addresses the problems of limited rendering capabilities which today are present on most passive haptic displays