Ali Talasaz

Integration of Force Reflection with Tactile Sensing for Minimally Invasive Robotics-Assisted Tumor Localization

Tactile sensing and force reflection have been the subject of considerable research for tumor localization in soft-tissue palpation. The work presented in this paper investigates the relevance of force feedback (presented visually as well as directly) during tactile sensing (presented visually only) for tumor localization using an experimental setup close to one that could be applied for real robotics-assisted minimally invasive surgery. The setup is a teleoperated (master-slave) system facilitated with a state-of-the-art minimally invasive probe with a rigidly mounted tactile sensor at the tip and an externally mounted force sensor at the base of the probe. The objective is to capture the tactile information and measure the interaction forces between the probe and tissue during palpation and to explore how they can be integrated to improve the performance of tumor localization. To quantitatively explore the effect of force feedback on tactile sensing tumor localization, several experiments were conducted by human subjects to locate artificial tumors embedded in the ex vivo bovine livers. The results show that using tactile sensing in a force-controlled environment can realize, on average, 57 percent decrease in the maximum force and 55 percent decrease in the average force applied to tissue while increasing the tumor detection accuracy by up to 50 percent compared to the case of using tactile feedback alone. The results also show that while visual presentation of force feedback gives straightforward quantitative measures, improved performance of tactile sensing tumor localization is achieved at the expense of longer times for the user. Also, the quickness and intuitive data mapping of direct force feedback makes it more appealing to experienced users.

Design and characterization of a 7-DOF haptic interface for a minimally invasive surgery test-bed

In this paper, we present the design of a 7 degrees-of-freedom (DOF) Haptic Interface for applications in Minimally Invasive Surgery (MIS). The design of the interface is based on an existing dual-panthograph Haptic Wand and is capable of position and force reflection in three translational, three rotational DOF and grasping motion. The paper presents the implementation of a novel cable driven differential transmission to include the yaw and grasping force reflection to the interface. The kinematic and dynamic properties of the interface are characterized and presented. Experimental results demonstrate that the device is capable of high-force reflection with good transparency.

Haptics-enabled teleoperation for robot-assisted tumor localization

This paper focuses on the problem of incorporating haptics-enabled teleoperation in minimally invasive tumor localization. Since the stiffness of a tumor is higher than that of the surrounding tissue, it can be identified as a hard nodule when palpated. Using a Tactile Sensing Instrument (TSI) developed at CSTAR, the distributed pressure profiles along the contacting surface can be measured during remote tissue palpation. The tumor can be detected by using a visualization software that creates a color contour map based on the magnitude of the pressure over the palpated area. The accuracy of this method depends on the uniformity of the force applied to the tissue. A haptics-enabled teleoperation system provides the surgeon with the opportunity to feel the interaction force between the instrument and tissue during Minimally Invasive Surgery (MIS). The objective of this research was to assess the feasibility of combining force feedback with tactile feedback in order to increase the overall performance of tumor localization. The teleoperation system used in this work consists of a Mitsubishi PA10 robot as the slave that is remotely controlled (over a dedicated network) through a 7 Degree-Of-Freedom (DOF) haptic interface. A two-channel architecture, along with hybrid impedance control was utilized to form a bilateral teleoperation system in which the master is under force control and the slave is under position control. The experimental results confirm the effectiveness of using force feedback in robot-assisted tactile sensing for tumor detection.

Cooperative Teleoperation With Projection-Based Force Reflection for MIS

Implementation of haptic feedback in minimally invasive surgical teleoperator systems may lead to improved performance in many common surgical procedures; however, most of the currently available surgical teleoperators do not provide force feedback, mainly because of the associated stability issues. In this paper, we study the effect of a special type of the force reflection algorithms, called projection-based force reflection (PBFR) algorithms, on the stability and performance of a dual-arm haptic-enabled teleoperator system for minimally invasive surgical applications. The performance of different algorithms is experimentally compared in the presence of negligible as well as nonnegligible communication delays. In particular, the teleoperator system’s performance is experimentally evaluated in three common surgical tasks, which are knot tightening, pegboard transfer, and object manipulation. The results obtained indicate that, in almost all cases, the PBFR algorithms demonstrate statistically significant improvement of performance in comparison with the conventional direct force feedback.

Telerobotic palpation for tumor localization with depth estimation

This work is aimed at developing a new minimally invasive approach to characterize tissue properties in real time during telerobotic palpation and to localize tissue abnormality while estimating its depth. This method relies on using a minimally invasive probe with a rigidly mounted tactile sensor at the tip to capture the force distribution map and the indentation depth by each tactile element and thereby generating a stiffness map for the palpated tissue. The hybrid impedance control technique is used for this approach to enable the operator to switch between position control and force control and thereby to autonomously obtain the required information from the remote tissue. The operator would then be able to localize tissue abnormality based on the force distribution map, the tissue stiffness map and the indentation depth which are visually presented to him/her in real time. This method also enables the operator to estimate the depth at which the tissue abnormality is located. Our results show that tactile sensing alone may be unable to detect tumors embedded deep inside tissue and may also not be a good alternative for palpation on uneven tissue surfaces.

Remote palpation to localize tumors in robot-assisted minimally invasive approach

This paper presents a new tactile-force integrated method to localize tumors minimally invasively using robotic assistance. This method relies on using a capacitive sensor at the tip of a Tactile Sensing Instrument (TSI) which can be inserted into a patients body in a minimally invasive manner. In this work, the operator palpates tissue containing tumors in a minimally invasive surgical (MIS) training box, representing the patients body, through a master-slave teleoperation system which consists of a 7 degrees-of-freedom (DOF) haptic interface, used as the master, and a Mitsubishi PA10-7C robot as the slave. Using the proposed method, the operator would be able to palpate the tissue consistently, observe the pressure distribution over the tissue by a color contour map on a screen and feel the tumor on his/her fingers through a grasping mechanism of the haptic interface as a result of higher stiffness of the tumor. The tissue used for the experiments was ex vivo bovine lung and seven participants were asked to locate artificial tumors embedded in the lungs. The results show an accuracy of 93% in tumor localization using the proposed method while the average force applied to the tissue was 3.42N and the force never exceeded 6N.

A Dual-Arm 7-Degrees-of-Freedom Haptics-Enabled Teleoperation Test Bed for Minimally Invasive Surgery

This paper describes a dual-arm teleoperation (master-slave) system which has been developed to explore the effect of haptics in robotics-assisted minimally invasive surgery (RAMIS). This setup is capable of measuring forces in 7 degrees of freedom (DOF) and fully reflecting them to the operator through two 7-DOF haptic interfaces. An application of the test bed is in enabling the evaluation of the effect of replacing haptic feedback by other sensory cues such as visual representation of haptic information (sensory substitution). This paper discusses the design rationale, kinematic analysis and dynamic modeling of the robot manipulators, and the control system developed for the setup. Using the accurate model developed in this paper, a highly transparent haptics-enabled system can be achieved and used in robot-assisted telesurgery. Validation results obtained through experiments are presented and demonstrate the correctness and effectiveness of the developed models. The application of the setup for two RAMIS surgical tasks, a suture manipulation task and a tumor localization task, is described with different haptics modalities available through the developed haptics-enabled system for each application.

The Role of Direct and Visual Force Feedback in Suturing Using a 7-DOF Dual-Arm Teleoperated System

The lack of haptic feedback in robotics-assisted surgery can result in tissue damage or accidental tool-tissue hits. This paper focuses on exploring the effect of haptic feedback via direct force reflection and visual presentation of force magnitudes on performance during suturing in robotics-assisted minimally invasive surgery (RAMIS). For this purpose, a haptics-enabled dual-arm master-slave teleoperation system capable of measuring tool-tissue interaction forces in all seven Degrees-of-Freedom (DOFs) was used. Two suturing tasks, tissue puncturing and knot-tightening, were chosen to assess user skills when suturing on phantom tissue. Sixteen subjects participated in the trials and their performance was evaluated from various points of view: force consistency, number of accidental hits with tissue, amount of tissue damage, quality of the suture knot, and the time required to accomplish the task. According to the results, visual force feedback was not very useful during the tissue puncturing task as different users needed different amounts of force depending on the penetration of the needle into the tissue. Direct force feedback, however, was more useful for this task to apply less force and to minimize the amount of damage to the tissue. Statistical results also reveal that both visual and direct force feedback were required for effective knot tightening: direct force feedback could reduce the number of accidental hits with the tissue and also the amount of tissue damage, while visual force feedback could help to securely tighten the suture knots and maintain force consistency among different trials/users. These results provide evidence of the importance of 7-DOF force reflection when performing complex tasks in a RAMIS setting.

A 7-DOF haptics-enabled teleoperated robotic system: Kinematic modeling and experimental verification

The purpose of this paper is to demonstrate the development of a novel 7-degree-of-freedom teleoperated robotic system that provides force reflection to the surgeons hands while performing minimally invasive surgery and therapy (MIST). An endoscopic tool has been sensorized and modified for use as the end effector of this haptics-enabled system. In this paper, mathematical models required for controlling the main components of the system have been determined and experimentally validated. Several experiments have been performed with this MIST robotic system in order to compare its force exertion capabilities with those of the da Vinci system from Intuitive Surgical Inc. Force reflection from the slave to the master in the new system is also demonstrated experimentally.

Computer Vision Based Autonomous Robotic System for 3D Plant Growth Measurement

Research on increasing the production of crops is increasingly important these days. This research needs a way to quantitatively measure the 3D growth of plants under controlled environments to allow a cost versus benefits analysis. Plant scientists need a non-invasive, non-destructive method to quantitatively measure the 3D growth of plants. Traditional methods, for example, measuring weight, area or volume, often negatively affects the future plant growth. Also the manual nature of this measurement can be quite time consuming, tedious and error prone. Some recent effort have been reported in the literature about the construction of autonomous systems for plant phenotype, but these are not practical for large scale accurate 3D plant growth computation. To the best of our knowledge, we are the first in the world to attempt truly 3D approach via robot assisted plant growth analysis using 3D imaging and laser scanning technology. We describe an automated system to perform 3D plant modelling using a laser scanner mounted on a robot arm to capture 3D plant data. We present a detailed overview of the system integration, including the robotic arm, laser scanner and a programmable growth chamber. We also show some results on reconstructing the 3Dmodel of a growing plant which is better than the current state of the art.