Veronica J. Santos

Biomimetic Tactile Sensor Array

The performance of robotic and prosthetic hands in unstructured environments is severely limited by their having little or no tactile information compared to the rich tactile feedback of the human hand. We are developing a novel, robust tactile sensor array that mimics the mechanical properties and distributed touch receptors of the human fingertip. It consists of a rigid core surrounded by a weakly conductive fluid contained within an elastomeric skin. The sensor uses the deformable properties of the finger pad as part of the transduction process. Multiple electrodes are mounted on the surface of the rigid core and connected to impedance-measuring circuitry safely embedded within the core. External forces deform the fluid path around the electrodes, resulting in a distributed pattern of impedance changes containing information about those forces and the objects that applied them. Here we describe means to optimize the dynamic range of individual electrode sensors by texturing the inner surface of the silicone skin. Forces ranging from 0.1 to 30 N produced impedances ranging from 5 to 1000 kΩ. Spatial resolution (below 2 mm) and frequency response (above 50 Hz) appeared to be limited only by the viscoelastic properties of the silicone elastomeric skin.

Biomimetic Tactile Sensor for Control of Grip

We are developing a novel, robust tactile sensor array that mimics the human fingertip and its distributed set of touch receptors. The mechanical components are similar to a fingertip, with a rigid core surrounded by a weakly conductive fluid contained within an elastomeric skin. It uses the deformable properties of the finger pad as part of the transduction process. Multiple electrodes are mounted on the surface of the rigid core and connected to impedance measuring circuitry within the core. External forces deform the fluid path around the electrodes, resulting in a distributed pattern of impedance changes containing information about those forces and the objects that applied them. Here we report preliminary results with prototypes of the sensor, and we propose strategies for extracting features related to the mechanical inputs and using this information for reflexive grip control.

A robust micro-vibration sensor for biomimetic fingertips

Controlling grip force in a prosthetic or robotic hand requires detailed sensory feedback information about microslips between the artificial fingertips and the object. In the biological hand this is accomplished with neural transducers capable of measuring micro-vibrations in the skin due to sliding friction. For prosthetic tactile sensors, emulating these biological transducers is a difficult challenge due to the fragility associated with highly sensitive devices. Incorporating a pressure sensor into a fluid-filled fingertip provides a novel solution to this problem by effectively creating a device similar to a hydrophone, capable of recording vibrations from lateral movements. The fluid conducts these acoustic signals well and with little attenuation, permitting the pressure sensing elements to be located in a protected region inside the core of the sensor and removing them from harmpsilas way. Preliminary studies demonstrate that high frequency vibrations (50-400 Hz) can be readily detected when such a fingertip slides across a ridged surface.

Deformable skin design to enhance response of a biomimetic tactile sensor

Grasping of objects by robotic hands in unstructured environments demands a sensor surface that is durable, compliant, and responsive to various force and slip conditions. A compliant and robust skin can be as critical to grasping objects as the sensor it protects. In an effort to combine compliant mechanics and robust sensing, a biomimetic tactile sensor is being developed. Deformations of its skin can be detected by displacing a conductive fluid from the vicinity of electrodes on a rigid core. In this study, we used simplified finite element models to understand the effects of various textures for the inner surface of the skin and then produced the more promising textures by molding the elastomeric skin material against negatives made by stereolithography. The impedance vs. force relationships obtained with these molded skins had the predicted and desired wide dynamic range. By selecting the appropriate materials for the skin and fluid, previously described problems with hysteresis and diffusion losses have been greatly reduced.

A robot hand testbed designed for enhancing embodiment and functional neurorehabilitation of body schema in subjects with upper limb impairment or loss

Many upper limb amputees experience an incessant, post-amputation “phantom limb pain” and report that their missing limbs feel paralyzed in an uncomfortable posture. One hypothesis is that efferent commands no longer generate expected afferent signals, such as proprioceptive feedback from changes in limb configuration, and that the mismatch of motor commands and visual feedback is interpreted as pain. Non-invasive therapeutic techniques for treating phantom limb pain, such as mirror visual feedback (MVF), rely on visualizations of postural changes. Advances in neural interfaces for artificial sensory feedback now make it possible to combine MVF with a high-tech “rubber hand” illusion, in which subjects develop a sense of embodiment with a fake hand when subjected to congruent visual and somatosensory feedback. We discuss clinical benefits that could arise from the confluence of known concepts such as MVF and the rubber hand illusion, and new technologies such as neural interfaces for sensory feedback and highly sensorized robot hand testbeds, such as the “BairClaw” presented here. Our multi-articulating, anthropomorphic robot testbed can be used to study proprioceptive and tactile sensory stimuli during physical finger–object interactions. Conceived for artificial grasp, manipulation, and haptic exploration, the BairClaw could also be used for future studies on the neurorehabilitation of somatosensory disorders due to upper limb impairment or loss. A remote actuation system enables the modular control of tendon-driven hands. The artificial proprioception system enables direct measurement of joint angles and tendon tensions while temperature, vibration, and skin deformation are provided by a multimodal tactile sensor. The provision of multimodal sensory feedback that is spatiotemporally consistent with commanded actions could lead to benefits such as reduced phantom limb pain, and increased prosthesis use due to improved functionality and reduced cognitive burden.

Haptic exploration of fingertip-sized geometric features using a multimodal tactile sensor

Haptic perception remains a grand challenge for artificial hands. Dexterous manipulators could be enhanced by “haptic intelligence” that enables identification of objects and their features via touch alone. Haptic perception of local shape would be useful when vision is obstructed or when proprioceptive feedback is inadequate, as observed in this study. In this work, a robot hand outfitted with a deformable, bladder-type, multimodal tactile sensor was used to replay four human-inspired haptic “exploratory procedures” on fingertip-sized geometric features. The geometric features varied by type (bump, pit), curvature (planar, conical, spherical), and footprint dimension (1.25 - 20 mm). Tactile signals generated by active fingertip motions were used to extract key parameters for use as inputs to supervised learning models. A support vector classifier estimated order of curvature while support vector regression models estimated footprint dimension once curvature had been estimated. A distal-proximal stroke (along the long axis of the finger) enabled estimation of order of curvature with an accuracy of 97%. Best-performing, curvature-specific, support vector regression models yielded R2 values of at least 0.95. While a radial-ulnar stroke (along the short axis of the finger) was most helpful for estimating feature type and size for planar features, a rolling motion was most helpful for conical and spherical features. The ability to haptically perceive local shape could be used to advance robot autonomy and provide haptic feedback to human teleoperators of devices ranging from bomb defusal robots to neuroprostheses.

Interactions Between Tactile and Proprioceptive Representations in Haptics

ABSTRACT Neuroprosthetic limbs, regardless of their sophisticated motor control, require sensory feedback to viably interact with the environment. Toward that aim, the authors examined interrelationships between tactile and proprioceptive sensations. Through human psychophysics experiments, they evaluated error patterns of subjects estimating hand location in a horizontal 2-dimensional workspace under 3 tactile conditions. While tactile cues did not significantly affect the structure of the pattern of errors, touching the workspace reduced estimation errors. During neurophysiological experiments, a macaque grasped textured objects using 2 hand postures. Sensory coding showed dependence on both roughness of the manipulandum and posture. In summary, the authors suggest that tactile sensations underlying haptics are processed in a stable spatial reference frame provided by a proprioceptive system, and that tactile and proprioceptive inputs can be encoded simultaneously by individual cells. Such insights will be useful for providing stable, adaptive sensory feedback for neuroprosthetics.

Medical robotics—Regulatory, ethical, and legal considerations for increasing levels of autonomy

A proposed framework for regulatory, ethical, and legal discussions identifies six levels of autonomy for medical robotics. The regulatory, ethical, and legal barriers imposed on medical robots necessitate careful consideration of different levels of autonomy, as well as the context for use.

Spatial Asymmetry in Tactile Sensor SkinDeformation Aids Perception of EdgeOrientation During Haptic Exploration

Upper-limb amputees rely primarily on visual feedback when using their prostheses to interact with others or objects in their environment. A constant reliance upon visual feedback can be mentally exhausting and does not suffice for many activities when line-of-sight is unavailable. Upper-limb amputees could greatly benefit from the ability to perceive edges, one of the most salient features of 3D shape, through touch alone. We present an approach for estimating edge orientation with respect to an artificial fingertip through haptic exploration using a multimodal tactile sensor on a robot hand. Key parameters from the tactile signals for each of four exploratory procedures were used as inputs to a support vector regression model. Edge orientation angles ranging from -90 to 90 degrees were estimated with an 85-input model having an R 2 of 0.99 and RMS error of 5.08 degrees. Electrode impedance signals provided the most useful inputs by encoding spatially asymmetric skin deformation across the entire fingertip. Interestingly, sensor regions that were not in direct contact with the stimulus provided particularly useful information. Methods described here could pave the way for semi-autonomous capabilities in prosthetic or robotic hands during haptic exploration, especially when visual feedback is unavailable.

Human Grip Responses to Perturbations of Objects During Precision Grip

Grasp stability of a precision grip requires fine control of three-dimensional fingertip forces. This chapter begins with a review of the literature on how precision grip forces are affected by intrinsic object properties, anticipation, load direction, and sensory feedback. Previous studies have established that reactive, initial increases in grip forces (pulse-like “catch-up responses” in grip force rates) are elicited by unexpected translational perturbations and that response latency and strength scale with the direction of linear slip relative to the hand as well as gravity. To determine if catch-up responses are elicited by unexpected rotational perturbations and are strength-, axis-, and/or direction- dependent, we imposed step torque loads about each of two axes which were defined relative to the hand: the distal-proximal axis away from and towards the palm, and the grip axis which connects the two fingertips. First dorsal interosseous activity, marking the start of the catch-up response, began 71–89 ms after the onset of perturbation. Onset latency, shape, and duration (217–231 ms) of the catch-up response were not affected by axis, direction, or magnitude of the rotational perturbation, while strength scaled with axis of rotation and slip conditions. Rotations about the grip axis induced rotational slip at the fingerpads and elicited stronger catch-up responses than rotations about the distal-proximal axis. The chapter concludes with a discussion of this study that, to our knowledge, is the first to investigate grip responses to unexpected torque loads and to show characteristic, yet axis-dependent, catch-up responses for conditions other than pure linear slip.