Jelizaveta Konstantinova

Implementation of Tactile Sensing for Palpation in Robot-Assisted Minimally Invasive Surgery: A Review

Robot-assisted minimally invasive surgery (RMIS) made it possible to perform a number of medical manipulations with reduced patient trauma and better accuracy. Various devices, including tactile sensors, have been developed in recent years to enhance the quality of this procedure. The objective of this paper is to review the latest advancements and challenges in the development of tactile sensing devices designed for surgical applications. In particular, the focus is on palpation and probing devices that can be potentially used in RMIS. In addition, we explore the aspects that should be taken into account when designing tactile sensors for RMIS, incorporating biological inspiration of tactile sensing, features of manual palpation, requirements of RMIS. We provide an overview of recommendations for the development of tactile sensing devices, especially in the context of RMIS.

Palpation force modulation strategies to identify hard regions in soft tissue organs.

This paper presents experimental evidence for the existence of a set of unique force modulation strategies during manual soft tissue palpation to locate hard abnormalities such as tumors. We explore the active probing strategies of defined local areas and outline the role of force control. In addition, we investigate whether the applied force depends on the non-homogeneity of the soft tissue. Experimental results on manual palpation of soft silicone phantoms show that humans have a well defined force control pattern of probing that is used independently of the non-homogeneity of the soft tissue. We observed that the modulations of lateral forces are distributed around the mean frequency of 22.3 Hz. Furthermore, we found that the applied normal pressure during probing can be modeled using a second order reactive autoregressive model. These mathematical abstractions were implemented and validated for the autonomous palpation for different stiffness parameters using a robotic probe with a rigid spherical indentation tip. The results show that the autonomous robotic palpation strategy abstracted from human demonstrations is capable of not only detecting the embedded nodules, but also enhancing the stiffness perception compared to static indentation of the probe.

Force and proximity fingertip sensor to enhance grasping perception

It is well known that tactile information can be used to enhance the quality of grasping. Therefore, new technological solutions for sensing in grasping are needed. This paper presents an optical based fingertip sensor that measures both interaction forces and proximity between fingertip and environment. The combination of multiple sensing modalities in the tip of a finger can significantly improve grasping and manipulation capabilities. In this work we present the design and the required calibration of individual sensing elements, and of the integrated fingertip sensor developed for a 3-fingered metamorphic robotic hand. Emulated grasping experiments, using a pinch grip, were performed to illustrate the concept and validate the performance of the developed sensing system. As a result, it was possible to determine the sensor position with respect to an object during approach, contact and grasp.

Force-velocity modulation strategies for soft tissue examination

Advanced tactile tools in minimally invasive surgery have become a pressing need in order to reduce time and improve accuracy in localizing potential tissue abnormalities. In this regard, one of the main challenges is to be able to estimate tissue parameters in real time. In palpation, tactile information felt at a given location is identified by the viscoelastic dynamics of the neighboring tissue. Due to this reason the tissue examination behavior and the distribution of viscoelastic parameters in tissue should be considered in conjunction. This paper investigates the salient features of palpation behavior on soft tissue determining the effectiveness of localizing hard nodules. Experimental studies involving human participants, and validation tests using finite element simulations and a tele-manipulator, were carried out. Two distinctive tissue examination strategies in force-velocity modulation for the given properties of target tissue were found. Experimental results suggest that force-velocity modulations during continuous path measurements are playing an important role in the process of mechanical soft tissue examination. These behavioral insights, validated by detailed numerical models and robotic experimentations shed light on future designs of optimal robotic palpation.

Fingertip Fiber Optical Tactile Array with Two-Level Spring Structure

Tactile perception is a feature benefiting reliable grasping and manipulation. This paper presents the design of an integrated fingertip force sensor employing an optical fiber based approach where applied forces modulate light intensity. The proposed sensor system is developed to support grasping of a broad range of objects, including those that are hard as well those that are soft. The sensor system is comprised of four sensing elements forming a tactile array integrated with the tip of a finger. We investigate the design configuration of a separate force sensing element with the aim to improve its measurement range. The force measurement of a single tactile element is based on a two-level displacement that is achieved thanks to a hybrid sensing structure made up of a stiff linear and flexible ortho-planar spring. An important outcome of this paper is a miniature tactile fingertip sensor that is capable of perceiving light contact, typically occurring during the initial stages of a grasp, as well as measuring higher forces, commonly present during tight grasps.

Fingertip proximity sensor with realtime visual-based calibration

Proximity and distance estimation sensors are broadly used in robotic hands to enhance the quality of grasping during grasp planning, grasp correction and in-hand manipulation. This paper presents a fiber optical proximity sensor that is integrated with a tactile sensing fingertip of a robotic hand of a mobile robot. The distance estimation of proximity sensors are typically influenced by the reflective properties of an object, such as color or surface roughness. With the approach proposed in this paper, the accuracy of the proximity sensor is enhanced using the information collected by the vision system of the robot. A camera is employed to obtain RGB values of the object to be grasped. Further on, the data obtained from the camera is used to obtain the correct calibration for the proximity sensor. Based on the experimental evidence, it is shown that our approach can be effectively used to reduce the distance estimation error.

Evaluating Manual Palpation Trajectory Patterns in Tele-manipulation for Soft Tissue Examination

Robot-assisted minimal invasive surgery made it possible to improve the quality of surgical procedures and to enhance clinical outcomes. However, the need to palpate soft tissue organs with the aim to localize potential sites of abnormalities in real time has been recognized. For this work, ten subjects were recruited to perform a remote palpation procedure on a silicone phantom utilizing a tele-manipulation setup, to study their behavior when remotely palpating soft tissue. The stiffness values acquired during the remote palpation of a silicone phantom were transferred to the subjects by means of haptic and visual feedback. Participating subjects were asked to detect hard nodules in the silicone tissue using two distinct strategies: a) randomly chosen movements, and b) trajectory pattern, based on manual palpation techniques for clinical breast examination. We have compared relevant parameters, defining patterns observed during manual palpation, with the counterpart patterns occurring during remote palpation. The results show the effectiveness of applying palpation trajectory pattern used during manual soft tissue examination to tele-manipulation palpation.

Evaluation of stiffness feedback for hard nodule identification on a phantom silicone model

Haptic information in robotic surgery can significantly improve clinical outcomes and help detect hard soft-tissue inclusions that indicate potential abnormalities. Visual representation of tissue stiffness information is a cost-effective technique. Meanwhile, direct force feedback, although considerably more expensive than visual representation, is an intuitive method of conveying information regarding tissue stiffness to surgeons. In this study, real-time visual stiffness feedback by sliding indentation palpation is proposed, validated, and compared with force feedback involving human subjects. In an experimental tele-manipulation environment, a dynamically updated color map depicting the stiffness of probed soft tissue is presented via a graphical interface. The force feedback is provided, aided by a master haptic device. The haptic device uses data acquired from an F/T sensor attached to the end-effector of a tele-manipulated robot. Hard nodule detection performance is evaluated for 2 modes (force feedback and visual stiffness feedback) of stiffness feedback on an artificial organ containing buried stiff nodules. From this artificial organ, a virtual-environment tissue model is generated based on sliding indentation measurements. Employing this virtual-environment tissue model, we compare the performance of human participants in distinguishing differently sized hard nodules by force feedback and visual stiffness feedback. Results indicate that the proposed distributed visual representation of tissue stiffness can be used effectively for hard nodule identification. The representation can also be used as a sufficient substitute for force feedback in tissue palpation.

Camera-based force and tactile sensor

Tactile information has become a topic of great interest in the design of devices that explore the physical interaction with the external environment. For instance, it is important for a robot hand to perform manipulation tasks, such as grasping and active touching, using tactile sensors mounted on the finger pad to provide feedback information. In this research we present a novel device that obtains both force and tactile information in a single integrated elastomer. The proposed elastomer consists of two parts, one of which is transparent and is wrapped in another translucent one that has eight conical sensing elements underneath. Two parts are merged together via a mould. A CCD camera is mounted at the bottom of the device to record the images of two elastomer mediums illuminated by the LED arrays set inside of the device. The method consists of evaluating the state of the contact surface based on analysis of the image of two elastomers. The external deformation of the elastomer is used to measure three force components Fz, Mx and My. The measurement is based on the area changes of the conical sensing elements under different loads, while the image of the inner transparent elastomer captures the surface pattern, which can used to obtain tactile information.

Correction: Palpation force modulation strategies to identify hard regions in soft tissue organs

[This corrects the article DOI: 10.1371/journal.pone.0171706.].