Christian Cipriani

Restoring Natural Sensory Feedback in Real-Time Bidirectional Hand Prostheses

A multigrasp, bidirectional hand prosthesis delivers dynamic sensory feedback, allowing a user with a hand amputation to achieve fine grasping force control and realistic object sensing. An Artificial Hand’s Sense of Touch To feel the hard curvature of a baseball or the soft cylinder that is a soda can—these sensations we often take for granted. But amputees with a prosthetic arm know only that they are holding an object, the shape and stiffness discernible only by eye or from experience. Toward a more sophisticated prosthetic that can “feel” an object, Raspopovic and colleagues incorporated a feedback system connected to the amputee’s arm nerves, which delivers sensory information in real time. The authors connected electrodes in the arm nerves to sensors in two fingers of the prosthetic hand. To “feel” an object, the electrodes delivered electrical stimuli to the nerves that were proportional to the finger sensor readouts. To grasp an object and perform other motor commands, muscle signals were decoded. This bidirectional hand prosthetic was tested in a single amputee who was blindfolded and acoustically shielded to assure that sound and vision were not being used to manipulate objects. In more than 700 trials, the subject showed that he could modulate force and grasp and identify physical characteristics of different types of objects, such as cotton balls, an orange, and a piece of wood. Such sensory feedback with precise control over a hand prosthetic would allow amputees to more freely and naturally explore their environments. Hand loss is a highly disabling event that markedly affects the quality of life. To achieve a close to natural replacement for the lost hand, the user should be provided with the rich sensations that we naturally perceive when grasping or manipulating an object. Ideal bidirectional hand prostheses should involve both a reliable decoding of the user’s intentions and the delivery of nearly “natural” sensory feedback through remnant afferent pathways, simultaneously and in real time. However, current hand prostheses fail to achieve these requirements, particularly because they lack any sensory feedback. We show that by stimulating the median and ulnar nerve fascicles using transversal multichannel intrafascicular electrodes, according to the information provided by the artificial sensors from a hand prosthesis, physiologically appropriate (near-natural) sensory information can be provided to an amputee during the real-time decoding of different grasping tasks to control a dexterous hand prosthesis. This feedback enabled the participant to effectively modulate the grasping force of the prosthesis with no visual or auditory feedback. Three different force levels were distinguished and consistently used by the subject. The results also demonstrate that a high complexity of perception can be obtained, allowing the subject to identify the stiffness and shape of three different objects by exploiting different characteristics of the elicited sensations. This approach could improve the efficacy and “life-like” quality of hand prostheses, resulting in a keystone strategy for the near-natural replacement of missing hands.

Design of a cybernetic hand for perception and action

Strong motivation for developing new prosthetic hand devices is provided by the fact that low functionality and controllability—in addition to poor cosmetic appearance—are the most important reasons why amputees do not regularly use their prosthetic hands. This paper presents the design of the CyberHand, a cybernetic anthropomorphic hand intended to provide amputees with functional hand replacement. Its design was bio-inspired in terms of its modular architecture, its physical appearance, kinematics, sensorization, and actuation, and its multilevel control system. Its underactuated mechanisms allow separate control of each digit as well as thumb–finger opposition and, accordingly, can generate a multitude of grasps. Its sensory system was designed to provide proprioceptive information as well as to emulate fundamental functional properties of human tactile mechanoreceptors of specific importance for grasp-and-hold tasks. The CyberHand control system presumes just a few efferent and afferent channels and was divided in two main layers: a high-level control that interprets the user’s intention (grasp selection and required force level) and can provide pertinent sensory feedback and a low-level control responsible for actuating specific grasps and applying the desired total force by taking advantage of the intelligent mechanics. The grasps made available by the high-level controller include those fundamental for activities of daily living: cylindrical, spherical, tridigital (tripod), and lateral grasps. The modular and flexible design of the CyberHand makes it suitable for incremental development of sensorization, interfacing, and control strategies and, as such, it will be a useful tool not only for clinical research but also for addressing neuroscientific hypotheses regarding sensorimotor control.

Double nerve intraneural interface implant on a human amputee for robotic hand control

OBJECTIVES The principle underlying this project is that, despite nervous reorganization following upper limb amputation, original pathways and CNS relays partially maintain their function and can be exploited for interfacing prostheses. Aim of this study is to evaluate a novel peripheral intraneural multielectrode for multi-movement prosthesis control and for sensory feed-back, while assessing cortical reorganization following the re-acquired stream of data. METHODS Four intrafascicular longitudinal flexible multielectrodes (tf-LIFE4) were implanted in the median and ulnar nerves of an amputee; they reliably recorded output signals for 4 weeks. Artificial intelligence classifiers were used off-line to analyse LIFE signals recorded during three distinct hand movements under voluntary order. RESULTS Real-time control of motor output was achieved for the three actions. When applied off-line artificial intelligence reached >85% real-time correct classification of trials. Moreover, different types of current stimulation were determined to allow reproducible and localized hand/fingers sensations. Cortical organization was observed via TMS in parallel with partial resolution of symptoms due to the phantom-limb syndrome (PLS). CONCLUSIONS tf-LIFE4s recorded output signals in human nerves for 4 weeks, though the efficacy of sensory stimulation decayed after 10 days. Recording from a number of fibres permitted a high percentage of distinct actions to be classified correctly. Reversal of plastic changes and alleviation of PLS represent corollary findings of potential therapeutic benefit. SIGNIFICANCE This study represents a breakthrough in robotic hand use in amputees.

The SmartHand transradial prosthesis

BackgroundProsthetic components and control interfaces for upper limb amputees have barely changed in the past 40 years. Many transradial prostheses have been developed in the past, nonetheless most of them would be inappropriate if/when a large bandwidth human-machine interface for control and perception would be available, due to either their limited (or inexistent) sensorization or limited dexterity. SmartHand tackles this issue as is meant to be clinically experimented in amputees employing different neuro-interfaces, in order to investigate their effectiveness. This paper presents the design and on bench evaluation of the SmartHand.MethodsSmartHand design was bio-inspired in terms of its physical appearance, kinematics, sensorization, and its multilevel control system. Underactuated fingers and differential mechanisms were designed and exploited in order to fit all mechatronic components in the size and weight of a natural human hand. Its sensory system was designed with the aim of delivering significant afferent information to the user through adequate interfaces.ResultsSmartHand is a five fingered self-contained robotic hand, with 16 degrees of freedom, actuated by 4 motors. It integrates a bio-inspired sensory system composed of 40 proprioceptive and exteroceptive sensors and a customized embedded controller both employed for implementing automatic grasp control and for potentially delivering sensory feedback to the amputee. It is able to perform everyday grasps, count and independently point the index. The weight (530 g) and speed (closing time: 1.5 seconds) are comparable to actual commercial prostheses. It is able to lift a 10 kg suitcase; slippage tests showed that within particular friction and geometric conditions the hand is able to stably grasp up to 3.6 kg cylindrical objects.ConclusionsDue to its unique embedded features and human-size, the SmartHand holds the promise to be experimentally fitted on transradial amputees and employed as a bi-directional instrument for investigating -during realistic experiments- different interfaces, control and feedback strategies in neuro-engineering studies.

Sensory feedback in upper limb prosthetics

One of the challenges facing prosthetic designers and engineers is to restore the missing sensory function inherit to hand amputation. Several different techniques can be employed to provide amputees with sensory feedback: sensory substitution methods where the recorded stimulus is not only transferred to the amputee, but also translated to a different modality (modality-matched feedback), which transfers the stimulus without translation and direct neural stimulation, which interacts directly with peripheral afferent nerves. This paper presents an overview of the principal works and devices employed to provide upper limb amputees with sensory feedback. The focus is on sensory substitution and modality matched feedback; the principal features, advantages and disadvantages of the different methods are presented.

Online Myoelectric Control of a Dexterous Hand Prosthesis by Transradial Amputees

A real-time pattern recognition algorithm based on k-nearest neighbors and lazy learning was used to classify, voluntary electromyography (EMG) signals and to simultaneously control movements of a dexterous artificial hand. EMG signals were superficially recorded by eight pairs of electrodes from the stumps of five transradial amputees and forearms of five able-bodied participants and used online to control a robot hand. Seven finger movements (not involving the wrist) were investigated in this study. The first objective was to understand whether and to which extent it is possible to control continuously and in real-time, the finger postures of a prosthetic hand, using superficial EMG, and a practical classifier, also taking advantage of the direct visual feedback of the moving hand. The second objective was to calculate statistical differences in the performance between participants and groups, thereby assessing the general applicability of the proposed method. The average accuracy of the classifier was 79% for amputees and 89% for able-bodied participants. Statistical analysis of the data revealed a difference in control accuracy based on the aetiology of amputation, type of prostheses regularly used and also between able-bodied participants and amputees. These results are encouraging for the development of noninvasive EMG interfaces for the control of dexterous prostheses.

Roughness Encoding for Discrimination of Surfaces in Artificial Active-Touch

A 2 × 2 array of four microelectromechanical system (MEMS) tactile microsensors is integrated with readout electronics in the distal phalanx of an anthropomorphic robotic finger. A total of 16 sensing elements are available in a 22.3-mm area (i.e., 72 units/cm ) of the artificial finger, thus achieving a density comparable with human Merkel mechanoreceptors. The MEMS array is covered by a polymeric packaging with biomimetic fingerprints enhancing the sensitivity in roughness encoding. This paper shows the ability of the sensor array to encode roughness for discrimination of surfaces, without requiring dedicated proprioceptive sensors for end-effector velocity. Three fine surfaces with 400-, 440-, and 480- μm spatial periods are quantitatively evaluated. Core experiments consisted in active-touch exploration of surfaces by the finger executing a stereotyped human-like movement. A time-frequency analysis on pairs of tactile array outputs shows a clustering of the fundamental frequency, thus yielding 97.6% worst-case discrimination accuracy with a k -nearest-neighbor (k-NN) classifier. Hence, surfaces differing down to 40 μm are identified in active-touch by both hardware and processing methods based on exteroceptive tactile information. Finally, active-touch results with five textiles (which differ in texture or orientation) are shown as a preliminary qualitative assessment of discrimination in a more realistic tactile-stimulation scenario.

A Miniature Vibrotactile Sensory Substitution Device for Multifingered Hand Prosthetics

A multisite, vibrotactile sensory substitution system, that could be used in conjunction with artificial touch sensors in multifingered prostheses, to deliver sensory feedback to upper limb amputees is presented. The system is based on a low cost/power/size smart architecture of off-the-shelf miniaturized vibration motors; the main novelty is that it is able to generate stimuli where both vibration amplitude and frequency as well as beat interference can be modulated. This paper is aimed at evaluating this system by investigating the capability of healthy volunteers to perceive-on their forearms-vibrations with different amplitudes and/or frequencies. In addition, the ability of subjects in spatially discriminating stimulations on three forearm sites and recognizing six different combinations of stimulations was also addressed. Results demonstrate that subjects were able to discriminate different force amplitudes exerted by the device (accuracies greater than 75%); when both amplitude and frequency were simultaneously varied, the pure discrimination of amplitude/frequency variation was affected by the variation of the other. Subjects were also able to discriminate with an accuracy of 93% three different sites and with an accuracy of 78% six different stimulation patterns.

Referral of sensation to an advanced humanoid robotic hand prosthesis.

Hand prostheses that are currently available on the market are used by amputees to only a limited extent, partly because of lack of sensory feedback from the artificial hand. We report a pilot study that showed how amputees can experience a robot-like advanced hand prosthesis as part of their own body. We induced a perceptual illusion by which touch applied to the stump of the arm was experienced from the artificial hand. This illusion was elicited by applying synchronous tactile stimulation to the hidden amputation stump and the robotic hand prosthesis in full view. In five people who had had upper limb amputations this stimulation caused referral touch sensation from the stump to the artificial hand, and the prosthesis was experienced more like a real hand. We also showed that this illusion can work when the amputee controls the movements of the artificial hand by recordings of the arm muscle activity with electromyograms. These observations indicate that the previously described “rubber hand illusion” is also valid for an advanced hand prosthesis, even when it has a robotic-like appearance.

Artificial Redirection of Sensation From Prosthetic Fingers to the Phantom Hand Map on Transradial Amputees: Vibrotactile Versus Mechanotactile Sensory Feedback

This work assesses the ability of transradial amputees to discriminate multi-site tactile stimuli in sensory discrimination tasks. It compares different sensory feedback modalities using an artificial hand prosthesis in: 1) a modality matched paradigm where pressure recorded on the five fingertips of the hand was fed back as pressure stimulation on five target points on the residual limb; and 2) a modality mismatched paradigm where the pressures were transformed into mechanical vibrations and fed back. Eight transradial amputees took part in the study and were divided in two groups based on the integrity of their phantom map; group A had a complete phantom map on the residual limb whereas group B had an incomplete or nonexisting map. The ability in localizing stimuli was compared with that of 10 healthy subjects using the vibration feedback and 11 healthy subjects using the pressure feedback (in a previous study), on their forearms, in similar experiments. Results demonstrate that pressure stimulation surpassed vibrotactile stimulation in multi-site sensory feedback discrimination. Furthermore, we demonstrate that subjects with a detailed phantom map had the best discrimination performance and even surpassed healthy participants for both feedback paradigms whereas group B had the worst performance overall. Finally, we show that placement of feedback devices on a complete phantom map improves multi-site sensory feedback discrimination, independently of the feedback modality.