Francesco Clemente

Humans can integrate feedback of discrete events in their sensorimotor control of a robotic hand

Abstract Providing functionally effective sensory feedback to users of prosthetics is a largely unsolved challenge. Traditional solutions require high band-widths for providing feedback for the control of manipulation and yet have been largely unsuccessful. In this study, we have explored a strategy that relies on temporally discrete sensory feedback that is technically simple to provide. According to the Discrete Event-driven Sensory feedback Control (DESC) policy, motor tasks in humans are organized in phases delimited by means of sensory encoded discrete mechanical events. To explore the applicability of DESC for control, we designed a paradigm in which healthy humans operated an artificial robot hand to lift and replace an instrumented object, a task that can readily be learned and mastered under visual control. Assuming that the central nervous system of humans naturally organizes motor tasks based on a strategy akin to DESC, we delivered short-lasting vibrotactile feedback related to events that are known to forcefully affect progression of the grasp-lift-and-hold task. After training, we determined whether the artificial feedback had been integrated with the sensorimotor control by introducing short delays and we indeed observed that the participants significantly delayed subsequent phases of the task. This study thus gives support to the DESC policy hypothesis. Moreover, it demonstrates that humans can integrate temporally discrete sensory feedback while controlling an artificial hand and invites further studies in which inexpensive, noninvasive technology could be used in clever ways to provide physiologically appropriate sensory feedback in upper limb prosthetics with much lower band-width requirements than with traditional solutions.

Vibrotactile Stimulation Promotes Embodiment of an Alien Hand in Amputees With Phantom Sensations

Tactile feedback is essential to intuitive control and to promote the sense of self-attribution of a prosthetic limb. Recent findings showed that amputees can be tricked to experience this embodiment, when synchronous and modality-matched stimuli are delivered to biological afferent structures and to an alien rubber hand. Hence, it was suggested to exploit this effect by coupling touch sensors in a prosthesis to an array of haptic tactile stimulators in the prosthetic socket. However, this approach is not clinically viable due to physical limits of current haptic devices. To address this issue we have proposed modality-mismatched stimulation and demonstrated that this promotes self-attribution of an alien hand on normally limbed subjects. In this work we investigated whether similar effects could be induced in transradial amputees with referred phantom sensations in a series of experiments fashioned after the Rubber Hand Illusion using vibrotactile stimulators. Results from three independent measures of embodiment demonstrated that vibrotactile sensory substitution elicits body-ownership of a rubber hand in transradial amputees. These results open up promising possibilities in this field; indeed miniature, safe and inexpensive vibrators could be fitted into commercially available prostheses and sockets to induce the illusion every time the prosthesis manipulates an object.

Non-invasive, temporally discrete feedback of object contact and release improves grasp control of closed-loop myoelectric transradial prostheses

Human grasping and manipulation control critically depends on tactile feedback. Without this feedback, the ability for fine control of a prosthesis is limited in upper limb amputees. Although various approaches have been investigated in the past, at present there is no commercially available device able to restore tactile feedback in upper limb amputees. Based on the Discrete Event-driven Sensory feedback Control (DESC) policy we present a device able to deliver short-lasting vibrotactile feedback to transradial amputees using commercially available myoelectric hands. The device (DESC-glove) comprises sensorized thimbles to be placed on the prosthesis digits, a battery-powered electronic board, and vibrating units embedded in an arm-cuff being transiently activated when the prosthesis makes and breaks contact with objects. The consequences of using the DESC-glove were evaluated in a longitudinal study. Five transradial amputees were equipped with the device for one month at home. Through a simple test proposed here for the first time-the virtual eggs test-we demonstrate the effectiveness of the device for prosthetic control in daily life conditions. In the future the device could be easily exploited as an add-on to complement myoelectric prostheses or even embedded in prosthetic sockets to enhance their control by upper limb amputees.

The SSSA-MyHand: A Dexterous Lightweight Myoelectric Hand Prosthesis

The replacement of a missing hand by a prosthesis is one of the most fascinating challenges in rehabilitation engineering. State of art prostheses are curtailed by the physical features of the hand, like poor functionality and excessive weight. Here we present a new multi-grasp hand aimed at overcoming such limitations. The SSSA-MyHand builds around a novel transmission mechanism that implements a semi-independent actuation of the abduction/adduction of the thumb and of the flexion/extension of the index, by means of a single actuator. Thus, with only three electric motors the hand is capable to perform most of the grasps and gestures useful in activities of daily living, akin commercial prostheses with up to six actuators, albeit it is as lightweight as conventional 1-Degrees of Freedom prostheses. The hand integrates position and force sensors and an embedded controller that implements automatic grasps and allows inter-operability with different human-machine interfaces. We present the requirements, the design rationale of the first prototype and the evaluation of its performance. The weight (478 g), force (31 N maximum force at the thumb fingertip) and speed of the hand (closing time: <370 ms), make this new design an interesting alternative to clinically available multi-grasp prostheses.

Humans Can Integrate Augmented Reality Feedback in Their Sensorimotor Control of a Robotic Hand

Tactile feedback is pivotal for grasping and manipulation in humans. Providing functionally effective sensory feedback to prostheses users is an open challenge. Past paradigms were mostly based on vibro- or electrotactile stimulations. However, the tactile sensitivity on the targeted body parts (usually the forearm) is greatly less than that of the hand/fingertips, restricting the amount of information that can be provided through this channel. Visual feedback is the most investigated technique in motor learning studies, where it showed positive effects in learning both simple and complex tasks; however, it was not exploited in prosthetics due to technological limitations. Here, we investigated if visual information provided in the form of augmented reality (AR) feedback can be integrated by able-bodied participants in their sensorimotor control of a pick-and-lift task while controlling a robotic hand. For this purpose, we provided visual continuous feedback related to grip force and hand closure to the participants. Each variable was mapped to the length of one of the two ellipse axes visualized on the screen of wearable single-eye display AR glasses. We observed changes in behavior when subtle (i.e., not announced to the participants) manipulation of the AR feedback was introduced, which indicated that the participants integrated the artificial feedback within the sensorimotor control of the task. These results demonstrate that it is possible to deliver effective information through AR feedback in a compact and wearable fashion. This feedback modality may be exploited for delivering sensory feedback to amputees in a clinical scenario.

Touch and Hearing Mediate Osseoperception

Osseoperception is the sensation arising from the mechanical stimulation of a bone-anchored prosthesis. Here we show that not only touch, but also hearing is involved in this phenomenon. Using mechanical vibrations ranging from 0.1 to 6 kHz, we performed four psychophysical measures (perception threshold, sensation discrimination, frequency discrimination and reaction time) on 12 upper and lower limb amputees and found that subjects: consistently reported perceiving a sound when the stimulus was delivered at frequencies equal to or above 400 Hz; were able to discriminate frequency differences between stimuli delivered at high stimulation frequencies (~1500 Hz); improved their reaction time for bimodal stimuli (i.e. when both vibration and sound were perceived). Our results demonstrate that osseoperception is a multisensory perception, which can explain the improved environment perception of bone-anchored prosthesis users. This phenomenon might be exploited in novel prosthetic devices to enhance their control, thus ultimately improving the amputees’ quality of life.

A novel device for multi-modal sensory feedback in hand prosthetics: Design and preliminary prototype

This paper presents a new multi-modal haptic device designed to deliver sensory feedback to upper limb amputees using a myoelectric prosthesis. In particular this device conveys to the individual information about (i) contact between digits and the environment through a vibrotactile stimulus and (ii) grip force applied by the prosthesis on objects via force/pressure feedback. The device described in this work may be used both in research and clinical scenarios in the near future due to its limited dimensions, weight and power consumption.

The preload force affects the perception threshold of muscle vibration-induced movement illusions

The control and the execution of motor tasks are largely influenced by proprioceptive feedback, i.e. the information about the position and movement of the body. In 1972, it was discovered that a vibratory stimulation applied non-invasively to a muscle or a tendon induces a movement illusion consistent with the elongation of the vibrated muscle/tendon. Although this phenomenon was reported by several studies, it is still unclear how to reliably reproduce it because of the many different features of the stimulation altering the sensation (e.g. frequency, duration, location). By performing a psychophysical test, we analysed the effects of the stimulation point and the preload force on the minimum stimulation amplitude needed to elicit an illusion of movement. In particular, we stimulated two groups of healthy subjects on three target regions of the biceps brachii muscle (the distal tendon, the muscle belly and one of the proximal tendons) applying three preload force ranges (0.5–0.75N, 1–2N and 3–4N). Our results showed that the minimum stimulation amplitude eliciting a sensation is affected by the preload force. On the contrary, it did not change significantly among the three stimulated regions. Nevertheless, the reported vividness of the illusion of movement changed across the stimulated points decreasing while moving from the distal to the proximal tendons. Overall, these outcomes contribute to the scientific debate on the features that modulate the vibration-induced movement illusion proposing ways to increase the reliability of the procedure in basic and applied research studies.

The myokinetic control interface: tracking implanted magnets as a means for prosthetic control

Upper limb amputation deprives individuals of their innate ability to manipulate objects. Such disability can be restored with a robotic prosthesis linked to the brain by a human-machine interface (HMI) capable of decoding voluntary intentions, and sending motor commands to the prosthesis. Clinical or research HMIs rely on the interpretation of electrophysiological signals recorded from the muscles. However, the quest for an HMI that allows for arbitrary and physiologically appropriate control of dexterous prostheses, is far from being completed. Here we propose a new HMI that aims to track the muscles contractions with implanted permanent magnets, by means of magnetic field sensors. We called this a myokinetic control interface. We present the concept, the features and a demonstration of a prototype which exploits six 3-axis sensors to localize four magnets implanted in a forearm mockup, for the control of a dexterous hand prosthesis. The system proved highly linear (R2 = 0.99) and precise (1% repeatability), yet exhibiting short computation delay (45 ms) and limited cross talk errors (10% the mean stroke of the magnets). Our results open up promising possibilities for amputees, demonstrating the viability of the myokinetic approach in implementing direct and simultaneous control over multiple digits of an artificial hand.

A MyoKinetic HMI for the Control of Hand Prostheses: A Feasibility Study

In an attempt to overcome the several limitations of currently available/investigated human-machine interfaces (HMI) for the control of robotic hand prostheses, we propose a new HMI exploiting the magnetic field produced by magnets implanted in the muscles. As a magnet is implanted in a muscle it will travel with it, and its localization could provide a direct measure of the contraction/elongation of that muscle, which is voluntarily controlled by the individual. Here we present a proof of concept of a single magnet localizer, which computes on-line the position of a magnet in a certain workspace. In particular, the system comprises a pair of magnetic field sensors mounted on custom printed circuit boards, and an algorithm that resolves the inverse magnetic problem using the magnetic dipole model. The accuracy and the repeatability of our system were evaluated using six miniature magnets. Ongoing results suggest that the envisioned system is viable.