Angelica I. Aviles

A recurrent neural network approach for 3D vision-based force estimation

Robotic-assisted minimally invasive surgery has demonstrated its benefits in comparison with traditional procedures. However, one of the major drawbacks of current robotic system approaches is the lack of force feedback. Apart from space restrictions, the main problems of using force sensors are their high cost and the biocompatibility. In this work a proposal based on Vision Based Force Measurement is presented, in which the deformation mapping of the tissue is obtained using the ℓ2 - Regularized Optimization class, and the force is estimated via a recurrent neural network that has as inputs the kinematic variables and the deformation mapping. Moreover, the capability of RNN for predicting time series is used in order to deal with tool occlusions. The highlights of this proposal, according to the results, are: knowledge of material properties are not necessary, there is no need of adding extra sensors and a good trade-off between accuracy and efficiency has been achieved.

Towards Retrieving Force Feedback in Robotic-Assisted Surgery: A Supervised Neuro-Recurrent-Vision Approach

Robotic-assisted minimally invasive surgeries have gained a lot of popularity over conventional procedures as they offer many benefits to both surgeons and patients. Nonetheless, they still suffer from some limitations that affect their outcome. One of them is the lack of force feedback which restricts the surgeons sense of touch and might reduce precision during a procedure. To overcome this limitation, we propose a novel force estimation approach that combines a vision based solution with supervised learning to estimate the applied force and provide the surgeon with a suitable representation of it. The proposed solution starts with extracting the geometry of motion of the hearts surface by minimizing an energy functional to recover its 3D deformable structure. A deep network, based on a LSTM-RNN architecture, is then used to learn the relationship between the extracted visual-geometric information and the applied force, and to find accurate mapping between the two. Our proposed force estimation solution avoids the drawbacks usually associated with force sensing devices, such as biocompatibility and integration issues. We evaluate our approach on phantom and realistic tissues in which we report an average root-mean square error of 0.02 N.

Force-feedback sensory substitution using supervised recurrent learning for robotic-assisted surgery

The lack of force feedback is considered one of the major limitations in Robot Assisted Minimally Invasive Surgeries. Since add-on sensors are not a practical solution for clinical environments, in this paper we present a force estimation approach that starts with the reconstruction of a 3D deformation structure of the tissue surface by minimizing an energy functional. A Recurrent Neural Network-Long Short Term Memory (RNN-LSTM) based architecture is then presented to accurately estimate the applied forces. According to the results, our solution offers long-term stability and shows a significant percentage of accuracy improvement, ranging from about 54% to 78%, over existing approaches.

Sensorless force estimation using a neuro-vision-based approach for robotic-assisted surgery

This paper addresses the issue of lack of force feedback in robotic-assisted minimally invasive surgeries. Force is an important measure for surgeons in order to prevent intra-operative complications and tissue damage. Thus, an innovative neuro-vision based force estimation approach is proposed. Tissue surface displacement is first measured via minimization of an energy functional. A neuro approach is then used to establish a geometric-visual relation and estimate the applied force. The proposed approach eliminates the need of add-on sensors, carrying out biocompatibility studies and is applicable to tissues of any shape. Moreover, we provided an improvement from 15.14% to 56.16% over other approaches which demonstrate the potential of our proposal.

Unconstrained L1-regularized minimization with interpolated transformations for heart motion compensation

Motion compensation constitutes a challenging issue in minimally invasive beating heart surgery. Since the zone to be repaired has a dynamic behaviour, precision and surgeons dexterity decrease. In order to solve this problem, various proposals have been presented using ℓ2-norm. However, as they present some limitations in terms of robustness and efficiency, motion compensation is still considered an open problem. In this work, a solution based on the class of ℓ1 Regularized Optimization is proposed. It has been selected due to its mathematical properties and practical benefits. In particular, deformation is characterized by cubic B-splines since they offer an excellent balance between computational cost and accuracy. Moreover, due to the non-differentiability of the functional, the logarithmic barrier function is used for generating a standard optimization problem. Results have provided a very good tradeoff between accuracy and efficiency, indicating the potential of the proposed approach and proving its stability even under complex deformations.

Exploring the effects of dimensionality reduction in deep networks for force estimation in robotic-assisted surgery

Robotic-Assisted Surgery approach overcomes the limitations of the traditional laparoscopic and open surgeries. However, one of its major limitations is the lack of force feedback. Since there is no direct interaction between the surgeon and the tissue, there is no way of knowing how much force the surgeon is applying which can result in irreversible injuries. The use of force sensors is not practical since they impose different constraints. Thus, we make use of a neuro-visual approach to estimate the applied forces, in which the 3D shape recovery together with the geometry of motion are used as input to a deep network based on LSTM-RNN architecture. When deep networks are used in real time, pre-processing of data is a key factor to reduce complexity and improve the network performance. A common pre-processing step is dimensionality reduction which attempts to eliminate redundant and insignificant information by selecting a subset of relevant features to use in model construction. In this work, we show the effects of dimensionality reduction in a real-time application: estimating the applied force in Robotic-Assisted Surgeries. According to the results, we demonstrated positive effects of doing dimensionality reduction on deep networks including: faster training, improved network performance, and overfitting prevention. We also show a significant accuracy improvement, ranging from about 33% to 86%, over existing approaches related to force estimation.

Automatic and robust single-camera specular highlight removal in cardiac images

In computer-assisted beating heart surgeries, accurate tracking of the hearts motion is of huge importance and there is a continuous need to eliminate any source of error that might disturb the tracking process. One source of error is the specular reflection that appears on the glossy surface of the heart. In this paper, we propose a robust solution for the detection and removal of specular highlights. A hybrid color attributes and wavelet based edge projection approach is applied to accurately identify the affected regions. These regions are then recovered using a dynamic search-based inpainting with adaptive windowing. Experimental results demonstrate the precision and efficiency of the proposed method. Moreover, it has a real-time performance and can be generalized to various other applications.

Robust cardiac motion estimation using ultrafast ultrasound data: a low-rank topology-preserving approach

Cardiac motion estimation is an important diagnostic tool for detecting heart diseases and it has been explored with modalities such as MRI and conventional ultrasound (US) sequences. US cardiac motion estimation still presents challenges because of complex motion patterns and the presence of noise. In this work, we propose a novel approach to estimate cardiac motion using ultrafast ultrasound data. Our solution is based on a variational formulation characterized by the L 2-regularized class. Displacement is represented by a lattice of b-splines and we ensure robustness, in the sense of eliminating outliers, by applying a maximum likelihood type estimator. While this is an important part of our solution, the main object of this work is to combine low-rank data representation with topology preservation. Low-rank data representation (achieved by finding the k-dominant singular values of a Casorati matrix arranged from the data sequence) speeds up the global solution and achieves noise reduction. On the other hand, topology preservation (achieved by monitoring the Jacobian determinant) allows one to radically rule out distortions while carefully controlling the size of allowed expansions and contractions. Our variational approach is carried out on a realistic dataset as well as on a simulated one. We demonstrate how our proposed variational solution deals with complex deformations through careful numerical experiments. The low-rank constraint speeds up the convergence of the optimization problem while topology preservation ensures a more accurate displacement. Beyond cardiac motion estimation, our approach is promising for the analysis of other organs that exhibit motion.

A Deep-Neuro-Fuzzy approach for estimating the interaction forces in Robotic surgery

Fuzzy theory was motivated by the need to create human-like solutions that allow representing vagueness and uncertainty that exist in the real-world. These capabilities have been recently further enhanced by deep learning since it allows converting complex relation between data into knowledge. In this paper, we present a novel Deep-Neuro-Fuzzy strategy for unsupervised estimation of the interaction forces in Robotic Assisted Minimally Invasive scenarios. In our approach, the capability of Neuro-Fuzzy systems for handling visual uncertainty, as well as the inherent imprecision of real physical problems, is reinforced by the advantages provided by Deep Learning methods. Experiments conducted in a realistic setting have demonstrated the superior performance of the proposed approach over existing alternatives. More precisely, our method increased the accuracy of the force estimation and compared favorably to existing state of the art approaches, offering a percentage of improvement that ranges from about 35% to 85%.

On Genetic Algorithms Optimization for Heart Motion Compensation

Heart motion compensation is a challenging problem within medical robotics and it is still considered an open research area due to the lack of robustness. As it can be formulated as an energy minimization problem, an optimization technique is needed. The selection of an adequate method has a significant impact over the global solution. For this reason, a new methodology is presented here for solving heart motion compensation in which the central topic is oriented to increase robustness with the goal of achieving a balance between efficiency and efficacy. Particularly, genetic algorithms are used as optimization technique since they can be adapted to any real application, complex and oriented to work in real-time problems.