Tapomayukh Bhattacharjee

Haptic classification and recognition of objects using a tactile sensing forearm

In this paper, we demonstrate data-driven inference of mechanical properties of objects using a tactile sensor array (skin) covering a robots forearm. We focus on the mobility (sliding vs. fixed), compliance (soft vs. hard), and identity of objects in the environment, as this information could be useful for efficient manipulation and search. By using the large surface area of the forearm, a robot could potentially search and map a cluttered volume more efficiently, and be informed by incidental contact during other manipulation tasks. Our approach tracks a contact region on the forearm over time in order to generate time series of select features, such as the maximum force, contact area, and contact motion. We then process and reduce the dimensionality of these time series to generate a feature vector to characterize the contact. Finally, we use the k-nearest neighbor algorithm (k-NN) to classify a new feature vector based on a set of previously collected feature vectors. Our results show a high cross-validation accuracy in both classification of mechanical properties and object recognition. In addition, we analyze the effect of taxel resolution, duration of observation, feature selection, and feature scaling on the classification accuracy.

Gain-Scheduling Control of Teleoperation Systems Interacting With Soft Tissues

Surgical teleoperation systems are being increasingly deployed recently. There are, however, some unsolved issues such as nonlinear characteristics of the interaction between the slave robot and soft tissues and difficulty in employing force sensors in the surgical end-effectors of the slave. These issues make it difficult to generalize any approach to develop a control for the system. This paper addresses these issues by proposing a H∞ suboptimal controller preserving robust stability and performance. The environment, i.e., soft tissues, is characterized with the nonlinear Hunt-Crossley model. This nonlinear characteristics of soft tissues are expressed with an affine combination of linear models within a predefined parameter polytope. For this linear parameter-varying system, a gain-scheduling control scheme is employed to design a suboptimal controller while guaranteeing its stability. To avoid using any force measurement in slave, we used position-position (PP) control architecture. The developed gain-scheduling control is validated with quantitative experimental results. The developed gain-scheduling PP control scheme shows good tracking capacity and high transparency for varied experimental conditions. Error of the transmitted impedance is significantly lower compared with other conventional control schemes for frequencies less than 2 Hz, which is frequently recommended for surgical teleoperation.

Enhancement in Operator's Perception of Soft Tissues and Its Experimental Validation for Scaled Teleoperation Systems

This paper focuses on scaled teleoperation systems interacting with soft tissues and presents an optimal control scheme to maximize the operators kinesthetic perception of remote soft environments while maintaining the stability in macro-micro interactions. Two performance metrics are defined to quantify the kinesthetic perception of the surgeons and the position tracking ability of the master-slave system. Kinesthetic perception is defined based on psychophysics by using two metrics, which relate to the detection and discrimination of stimulus. This paper then employs a multiconstrained optimization approach to get an optimal solution in the presence of the stability-performance tradeoff wherein the objective is to enhance the kinesthetic perception while maintaining the tracking and robust stability for interactions between macro and microworlds. Simplified stability constraints for scaled teleoperation systems are designed based on Llewellyns absolute stability criterion for the optimization procedure, which provides easy and effective design guidelines for selecting control gains. Experiments with phantom soft tissues have been conducted using scaled force-position control architecture, scaled position-position control architecture, and scaled four-channel control architecture to verify the proposed control scheme. Results prove the effectiveness of this algorithm in enhancing the kinesthetic perception of surgeons for scaled teleoperation systems. Psychophysical experiments were then performed to compare our approach with similar contemporary research methods that further validated its efficacy.

Estimation of environmental force for the haptic interface of robotic surgery

The success of a telerobotic surgery system with haptic feedback requires accurate force‐tracking and position‐tracking capacity of the slave robot. The two‐channel force‐position control architecture is widely used in teleoperation systems with haptic feedback for its better force‐tracking characteristics and superior position‐tracking capacity for the maximum stability margin. This control architecture, however, requires force sensors at the end‐effector of the slave robot to measure the environment force. However, it is difficult to attach force sensors to slave robots, mainly due to their large size, insulation issues and also large currents often flowing through the end‐effector for incision or cautery of tissues.

Tactile sensing over articulated joints with stretchable sensors

Biological organisms benefit from tactile sensing across the entire surfaces of their bodies. Robots may also be able to benefit from this type of sensing, but fully covering a robot with robust and capable tactile sensors entails numerous challenges. To date, most tactile sensors for robots have been used to cover rigid surfaces. In this paper, we focus on the challenge of tactile sensing across articulated joints, which requires sensing across a surface whose geometry varies over time. We first demonstrate the importance of sensing across joints by simulating a planar arm reaching in clutter and finding the frequency of contact at the joints. We then present a simple model of how much a tactile sensor would need to stretch in order to cover a 2 degree-of-freedom (DoF) wrist joint. Next, we describe and characterize a new tactile sensor made with stretchable fabrics. Finally, we present results for a stretchable sleeve with 25 tactile sensors that covers the forearm, 2 DoF wrist, and end effector of a humanoid robot. This sleeve enabled the robot to reach a target in instrumented clutter and reduce contact forces.

Rapid categorization of object properties from incidental contact with a tactile sensing robot arm

We demonstrate that data-driven methods can be used to rapidly categorize objects encountered through incidental contact on a robot arm. Allowing incidental contact with surrounding objects has benefits during manipulation such as increasing the workspace during reaching tasks. The information obtained from such contact, if available online, can potentially be used to map the environment and help in manipulation tasks. In this paper, we address this problem of online categorization using incidental contact during goal-oriented motion. In cluttered environments, the detailed internal structure of clutter can be difficult to infer, but the environment type is often apparent. In a randomized cluttered environment of known object types and “outliers”, our approach uses Hidden Markov Models to capture the dynamic robot-environment interactions and to categorize objects based on the interactions. We combined leaf and trunk objects to create artificial foliage as a test environment. We collected data using a skin-sensor on the robots forearm while it reached into clutter. Our algorithm classifies the objects rapidly with low computation time and few data-samples. Using a taxel-by-taxel classification approach, we can successfully categorize simultaneous contacts with multiple objects and can also identify outlier objects in the environment based on the prior associated with an objects likelihood in the given environment.

Effect of Impedance-Shaping on Perception of Soft Tissues in Macro-Micro Teleoperation

This paper aims at analyzing the effect of widely known impedance-shaping (IS) control method on the perception of soft tissues in telemicrosurgical applications. The generalized teleoperation control architecture has been modified to include the IS term. New performance index has been defined based on the two proposed indices for the detection and the discrimination of the soft environments to analyze the effect of this modified control on the kinesthetic perception of soft tissues. The effect is then theoretically analyzed on the conventional position-position, force-position, and four-channel control architectures based on the newly defined index. The effectiveness of this newly proposed kinesthetic perception index is also verified using psychophysics experiments. The theoretical analysis of the effects of the IS method on the perception of soft tissues is then validated using the proposed index by experiments with phantom soft tissues for conventional teleoperation architectures.

Material Recognition from Heat Transfer given Varying Initial Conditions and Short-Duration Contact

When making contact with an object, a robot can use a tactile sensor consisting of a heating element and a temperature sensor to recognize the object’s material based on conductive heat transfer from the tactile sensor to the object. When this type of tactile sensor has time to fully reheat prior to contact and the duration of contact is long enough to achieve a thermal steady state, numerous methods have been shown to perform well. In order to enable robots to more efficiently sense their environments and take advantage of brief contact events over which they lack control, we focus on the problem of material recognition from heat transfer given varying initial conditions and short-duration contact. We present both modelbased and data-driven methods. For the model-based method, we modeled the thermodynamics of the sensor in contact with a material as contact between two semi-infinite solids. For the data-driven methods, we used three machine learning algorithms (SVM+PCA, k-NN+PCA, HMMs) with time series of raw temperature measurements and temperature change estimates. When recognizing 11 materials with varying initial conditions and 3fold cross-validation, SVM+PCA outperformed all other methods, achieving 84% accuracy with 0.5 s of contact and 98% accuracy with 1.5 s of contact.

Analytical and Psychophysical Comparison of Bilateral Teleoperators for Enhanced Perceptual Performance

This paper focuses on the human perception capabilities for haptic interaction with remote environments. The perception capabilities are compared for two well-known control methods with two kinds of haptic cues. Analytical and psychophysical methods are used to analyze the performance. The first control method aims at maximizing the transparency of the remote interactions (i.e., transparency-based method), whereas the second one aims at maximizing the detection and discrimination abilities of the human operator (i.e., perception-based method). For each of these two control methods, two kinds of haptic cues are studied, which use position and force cues from remote environments. Hybrid matrix formulation is employed, and it is analyzed in the frequency domain for these studies. Psychophysical experiments are then conducted for human-centered evaluation and comparison of the control methods. Analytical and experimental results clearly show that the perception-based method, when compared with the transparency-based method, enhances the human operators perceptual capabilities of remote environments irrespective of force cues. For each of the two control methods, the force cues always contribute more to the increase in perceptual sensitivity when compared with the case of position cues.

Small forces that differ with prior motor experience can communicate movement goals during human-human physical interaction

BackgroundPhysical interactions between two people are ubiquitous in our daily lives, and an integral part of many forms of rehabilitation. However, few studies have investigated forces arising from physical interactions between humans during a cooperative motor task, particularly during overground movements. As such, the direction and magnitude of interaction forces between two human partners, how those forces are used to communicate movement goals, and whether they change with motor experience remains unknown. A better understanding of how cooperative physical interactions are achieved in healthy individuals of different skill levels is a first step toward understanding principles of physical interactions that could be applied to robotic devices for motor assistance and rehabilitation.MethodsInteraction forces between expert and novice partner dancers were recorded while performing a forward-backward partnered stepping task with assigned “leader” and “follower” roles. Their position was recorded using motion capture. The magnitude and direction of the interaction forces were analyzed and compared across groups (i.e. expert-expert, expert-novice, and novice-novice) and across movement phases (i.e. forward, backward, change of direction).ResultsAll dyads were able to perform the partnered stepping task with some level of proficiency. Relatively small interaction forces (10–30N) were observed across all dyads, but were significantly larger among expert-expert dyads. Interaction forces were also found to be significantly different across movement phases. However, interaction force magnitude did not change as whole-body synchronization between partners improved across trials.ConclusionsRelatively small interaction forces may communicate movement goals (i.e. “what to do and when to do it”) between human partners during cooperative physical interactions. Moreover, these small interactions forces vary with prior motor experience, and may act primarily as guiding cues that convey information about movement goals rather than providing physical assistance. This suggests that robots may be able to provide meaningful physical interactions for rehabilitation using relatively small force levels.