Antônio Padilha Lanari Bó

Towards robust 3D visual tracking for motion compensation in beating heart surgery

In the context of minimally invasive cardiac surgery, active vision-based motion compensation schemes have been proposed for mitigating problems related to physiological motion. However, robust and accurate visual tracking remains a difficult task. The purpose of this paper is to present a robust visual tracking method that estimates the 3D temporal and spatial deformation of the heart surface using stereo endoscopic images. The novelty is the combination of a visual tracking method based on a Thin-Plate Spline (TPS) model for representing the heart surface deformations with a temporal heart motion model based on a time-varying dual Fourier series for overcoming tracking disturbances or failures. The considerable improvements in tracking robustness facing specular reflections and occlusions are demonstrated through experiments using images of in vivo porcine and human beating hearts.

Beating heart motion prediction for robust visual tracking

In the context of minimally invasive cardiac surgery, robotic assistance has significantly helped surgeons to overcome difficulties related to the minimally invasive procedure. Recently, techniques have been proposed for active canceling the beating heart motion for improving the accuracy of the surgical gestures. In this scenario, computer vision techniques can be applied for estimating the heart motion based solely on natural structures on the heart surface. However, visual tracking is complicated by the particular lighting conditions and clutter (smoke, liquids, etc) during surgery. Another challenging problem are the occasional occlusions by surgical instruments. In order to overcome these problems, we exploit the quasi-periodicity of the beating heart motion for increasing the robustness of the visual tracking task. In this paper, a novel time-varying dual Fourier series for modeling the quasi-periodic beating heart motion is proposed. For estimating the series parameters, an Extended Kalman Filter (EKF) is used. The proposed method is applied in a visual tracking task for bridging tracking disturbances and automatically reestablish tracking in cases of occlusions. The efficiency of the prediction method and the sensible improvements in the visual tracking task are demonstrated through in vivo experiments.

Robust 3d visual tracking for robotic-assisted cardiac interventions

In the context of minimally invasive cardiac surgery, active vision-based motion compensation schemes have been proposed for mitigating problems related to physiological motion. However, robust and accurate visual tracking is a difficult task. The purpose of this paper is to present a hybrid tracker that estimates the heart surface deformation using the outputs of multiple visual tracking techniques. In the proposed method, the failure of an individual technique can be circumvented by the success of others, enabling the robust estimation of the heart surface deformation with increased spatial resolution. In addition, for coping with the absence of visual information due to motion blur or occlusions, a temporal heart motion model is incorporated as an additional support for the visual tracking task. The superior performance of the proposed technique compared to existing techniques individually is demonstrated through experiments conducted on recorded images of an in vivo minimally invasive CABG using the DaVinci robotic platform.

Exploring Peripheral Mechanism of Tremor on Neuromusculoskeletal Model: A General Simulation Study

This paper provides a general simulation study on tremor based on a modular neuromusculoskeletal model. It focuses on the peripheral mechanism. It is known that the reflex loops in the peripheral nervous system have influences on the tremor. A neuromusculoskeletal model with several reflex loops is developed to explore the dynamics of tremor. The muscle model is derived from a Hill-type muscle model. The reflex loops include the spindle organ, Golgi tendon organ, and Renshaw cell. Their effects are investigated quantitatively in detail. A two-muscle (agonist/antagonist) system with interaction is further studied. Moreover, a model in combination with the central oscillation and peripheral system is developed. Some results are in accordance with the previous research, whereas some new findings are proposed according to the simulation study.

Motion prediction for tracking the beating heart

In the past few years, several research groups have worked on the design of efficient motion compensation systems for cardiac robotic-assisted Minimally Invasive Surgery (MIS). By providing surgeons with a stabilized work environment, significant improvements of the precision and repeatability of their gestures can be achieved. The design of a motion compensation system requires the accurate measurement of the heart motion, which can be achieved using computer vision techniques for tracking cardiac structures on the heart surface. However, most works in the literature focus on the representation and localization of cardiac structures while few explore their motion dynamics. In this paper we study and implement different adaptive methods for predicting the future heart motion using Kalman filtering. By exploiting the quasi-periodic nature of the heart motion, we are able to increase tracking robustness and computational efficiency. The experimental results indicate the significant increase in tracking performance when heart motion prediction is employed.

FES-controlled co-contraction strategies for pathological tremor compensation

In this paper, a strategy for pathological tremor compensation based on co-contraction of antagonist muscles induced by Functional Electrical Stimulation (FES) is presented. Although one of the simplest alternatives to apply FES for reducing the effects of tremor, the contribution of different co- contraction levels for joint motion and impedance must be accurately estimated, specially since tremor itself is highly time-varying. In this work, a detailed musculoskeletalmodel of the human wrist actuated by flexor and extensor muscles is used for this purpose. The model takes into account different properties that affect muscle dynamics, such as proprioceptive feed- back and combined natural and artificial activation. The model, analysis of stiffness modulation due to FES-controlled co-contraction and simulation results are presented in the paper.

Towards a cooperative framework for interactive manipulation involving a human and a humanoid

In this paper we propose a novel approach for interactive manipulation involving a human and a humanoid. The interaction is represented by means of the relative configuration between the humans and the robots hands. Based on this principle and a set of mathematical tools also proposed in the paper, a large set of tasks can be represented intuitively. We also introduce the concept of simultaneous handling using mirrored movements, where the human controls the robot and simultaneously interacts with it by means of a common manipulated object. Illustrative experiments are performed to validate the proposed techniques.

Online pathological tremor characterization using extended Kalman filtering

This paper describes different algorithms that perform online pathological tremor characterization in terms of acceleration. Two distinct parametricmodels are used, an Auto-Regressive (AR) model and an harmonic model. Both models are recursively estimated with Extended Kalman Filters (EKFs). Experimental data was obtained with low cost sensors and the results are compared in terms of spectrogram estimation and prediction performance.

Kinematic modeling and control for human-robot cooperation considering different interaction roles

This paper presents a novel approach for the description of physical human-robot interaction (pHRI) tasks that involve two-arm coordination, and where tasks are described by the relative pose between the human hand and the robot hand. We develop a unified kinematic model that takes into account the human-robot system from a holistic point of view, and we also propose a kinematic control strategy for pHRI that comprises different levels of shared autonomy. Since the kinematic model takes into account the complete human-robot interaction system and the kinematic control law is closed loop at the interaction level, the kinematic constraints of the task are enforced during its execution. Experiments are performed in order to validate the proposed approach, including a particular case where the robot controls the human arm by means of functional electrical stimulation (FES), which may potentially provide useful solutions for the interaction between assistant robots and impaired individuals (e.g., quadriplegics and hemiplegics).

On the Use of Fixed‐Intensity Functional Electrical Stimulation for Attenuating Essential Tremor

A great proportion of essential tremor (ET) patients have not so far been able to receive functional benefits from traditional therapies. In this regard, the use of functional electrical stimulation (FES) has been proposed for reducing tremor amplitude by stimulating muscles in antiphase with respect to the trembling motion. Although some studies have reported success in terms of tremor attenuation, drawbacks still exist that prevent the method from being used in real-life applications. In this article, we explore an alternative approach: a strategy based on the hypothesis that FES-induced constant muscle contraction may provide functional benefit for tremor patients. To evaluate the proposed strategy, experiments were conducted in which stimulation was intermittently turned on and off while the subjects performed a static motor task. The results of the proposed experimental protocol indicate that tremor attenuation using this strategy is feasible, as consistent tremor attenuation levels were obtained in eight out of 10 ET patients. Nonetheless, tremor reduction was not instantaneous for all successful trials, indicating that prior training with FES may improve the overall response. Furthermore, although simpler assistive devices may potentially be designed based on this technique, some experimental difficulties still exist, which suggests that further studies are necessary.