Marie-Aude Vitrani

Automatic Guidance of a Surgical Instrument with Ultrasound Based Visual Servoing

Visual servoing is a possible solution to assist the surgeon in performing tasks under ultrasound (US) imaging. To this aim, a system was developed that allows the surgeon to select a desired instrument location on a US image. Then a robot is programmed to automatically move the instrument towards the selected location. This approach requires robust tracking of the instrument in the US image, together with modeling of the overall system and implementation of a visual servoing loop. This paper presents geometrical and kinematic models of the system, as well as the control loop design, which is validated through both numerical simulations, and results of in vitro experiments.

Robust Ultrasound-Based Visual Servoing for Beating Heart Intracardiac Surgery

This paper presents a robust visual servoing approach for automatic guidance of an instrument. The visual sensor is an ultrasound probe that observes an instrument which is inserted into the beating heart of a patient and manipulated by a robot. The present paper provides stability analysis, robustification of the control law and an in vivo experiment.

Robust Real-Time Instrument Tracking in Ultrasound Images for Visual Servoing

Minimally invasive surgery in combination with ultrasound (US) imaging imposes high demands on the sur geon’s hand-eye-coordination capabilities. A possible solution to reduce these requirements is minimally invasive robotic surgery in which the instrument is guided by visual servoing towards the goal defined by the surgeon in the US image. This approach requires robust tracking of the instrument in the US image sequences which is known to be difficult due to poor image quality. This paper presents computer vision algorithms and results of visual servoing experiments. Adaptive thresholding according to Otsu’s method allows to cope with large intensity variations of the instrument echo. Subsequently applied morphological operations suppress noise and echo artefacts. A fast labelling algorithm based on run length coding allows for realtime labelling of the regions. A heuristic exploiting region size and region velocity helps to overcome ambiguities. The overall computation time is less than 10 ms per frame on a standard PC. The tracking algorithm requires no information about texture and shape which are known to be very unreliable in US image sequences. Experimental results for different instrument materials (polyvinyl chloride, polyurethane, nylon, and plexiglas) are given, illustrating the performance of the proposed approach: when chosing the appropriate material the reconstructed trajectories are smooth and only few outliers occur. As a consequence, the visual servoing loop showed to be robust and stable.

An Endorectal Ultrasound Probe Comanipulator With Hybrid Actuation Combining Brakes and Motors

A robotic device, aimed at assisting a urologist in positioning an endorectal ultrasound probe to perform prostate biopsies, is presented. The proposed system is a comanipulator that holds the probe simultaneously with the urologist. This robot supports two modes of operation: the free mode, where the entire movement control is left to the urologist when he/she positions the probe with respect to the prostate thanks to the feedback provided by the ultrasound images; the locked mode, where the robots role is to precisely maintain the targeted biopsy site at a given location, while the urologist can insert a needle through a guide mounted on the probe and proceed to biopsy. The device combines three brakes and three motors. This allows both transparent comanipulation in the free mode with six degrees of freedom liberated and stabilization of the probe in the locked mode. At the control level, a main challenge in the locked mode raises from antagonistic constraints: the needle placement shall be precise in spite of unknown external forces due to the contact between the probe and the rectum; the robot apparent impedance shall be low due to security constraints. This is solved by an inner low stiffness controller and an outer slow integration for canceling steady-state errors. Both in vitro and in cadavero experimental results show the efficiency of the system in the two modes of operation.

A robust ultrasound-based visual servoing approach for automatic guidance of a surgical instrument with in vivo experiments

This paper presents a visual servoing approach for automatic guidance of an instrument. In this approach, the visual sensor is an ultrasound probe that observes an instrument inserted inside the beating heart of a patient and manipulated by a robot. Following the preliminary results presented, the present paper provides a new parameterization of the visual feature signal that allows for enhanced robustness, and shows the first in vivo results on this ongoing research

A novel comanipulation device for assisting needle placement in ultrasound guided prostate biopsies

A novel robotic device, aimed at assisting a urologist in performing prostate biopsies guided by an endorectal ultrasound probe, is described. The paper describes the robot kinematics and the actuation system. The actuation system combines electromagnetic brakes, balancing springs, and electrical motors with cable transmissions. The robot supports two modes of operation: the free mode, where the entire control of the probe movements is left to the urologist, and the blocked mode, where the robot precisely maintains the probe at a given position and orientation with respect to the prostate. For the blocked mode, the set of specifications is antagonistic: firstly, a security constraint requires a low robot stiffness to allow to compliantly adapt to potential movements from the patient; secondly, a precision constraint requires a high robot stiffness in order to maintain the position and orientation of the probe in the presence of unmodeled external forces, when the robot is switched from the free mode to the blocked mode. A control strategy is developed to obtain this behavior. It combines an inner impedance controller with a relatively low stiffness and an outer intelligent position integrator that operates only during a limited period of time, when switching from the free mode to the blocked mode. Both in vitro and in cadavero experimental results show the efficiency of this approach.

Harp plucking robotic finger

This paper describes results about the development of a repeatable and configurable robotic finger designed to pluck harp strings. Eventually, this device will be a tool to study string instruments in playing conditions. We use a classical robot with two degrees of freedom enhanced with silicone fingertips. The validation method requires a comparison with real harpist performance. A specific experimental setup using a high-speed camera combined with an accelerometer was carried out. It provides finger and string trajectories during the whole plucking action and the soundboard vibrations during the string oscillations. A set of vibrational features are then extracted from these signals to compare robotic finger to harpist plucking actions. These descriptors have been analyzed on six fingertips of various shapes and hardnesses. Results allow to select the optimal shape and hardness among the silicone fingertips according to vibrational features.

Torque control of electrorheological fluidic actuators for haptic vehicular instrument controls

Force-feedback mechanisms have been designed to simplify and enhance the human-vehicle interface. The increase in secondary controls within vehicle cockpits has created a desire for a simpler, more efficient human-vehicle interface. By consolidating various controls into a single, haptic feedback control device, information can be transmitted to the operator, without requiring the drivers visual attention. In this work, the experimental closed loop torque control of electro-rheological fluids (ERF) based actuators for haptic application is performed. ERFs are liquids that respond mechanically to electric fields by changing their properties, such as viscosity and shear stress, electroactively. Using the electrically controlled rheological properties of ERFs, we developed actuators for haptic devices that can resist human operator forces in a controlled and tunable fashion. In this study, the ERF actuator analytical model is derived and experimentally verified and accurate closed loop torque control is experimentally achieved using a non-linear proportional integral controller with a feed-forward loop.

Prostate biopsies assisted by comanipulated probe-holder: first in man

PurposeA comanipulator for assisting endorectal prostate biopsies is evaluated through a first-in-man clinical trial. This lightweight system, based on conventional robotic components, possesses six degrees of freedom. It uses three electric motors and three brakes. It features a free mode, where its low friction and inertia allow for natural manipulation of the probe and a locked mode, exhibiting both a very low stiffness and a high steady-state precision.MethodsClinical trials focusing on the free mode and the locked mode of the robot are presented. The objective was to evaluate the practical usability and performance of the robot during clinical procedures. A research protocol for a prospective randomized clinical trial has been designed. Its specific goal was to compare the accuracy of biopsies performed with and without the assistance of the comanipulator.ResultsThe accuracy is compared between biopsies performed with and without the assistance of the comanipulator, across the 10 first patients included in the trial. Results show a statistically significant increase in the precision.

Hand-eye self-calibration of an ultrasound image-based robotic system

In recent years, there has been an increasing interest in developing systems that couple a robotic device with an ultrasound imager. Applications range from automatic probe positioning to ultrasound image-based guidance of instruments. One issue in these systems is to determine, prior to the intervention, the localization of the probe with respect to the robot. Literature suggests using external localizers, but they add to the complexity of the system, and the resulting precision is usually not good due to the addition of errors in the kinematic chain. In this paper, we study the hand-eye calibration problem without using any additional localizers. A generic system consisting of a fixed probe observing an instrument manipulated by a robot is used. We first derive a simplified model for mapping the image of the instrument with its 3D location w.r.t. the probe and then propose a calibration procedure based on a minimization algorithm. Results show that although very simple models were used for the imaging device, the localization is quite precise, as it results in errors of less than 2 mm, which is enough of a number of ultrasound guided interventions.