Jacopo Carpaneto

Restoring Natural Sensory Feedback in Real-Time Bidirectional Hand Prostheses

A multigrasp, bidirectional hand prosthesis delivers dynamic sensory feedback, allowing a user with a hand amputation to achieve fine grasping force control and realistic object sensing. An Artificial Hand’s Sense of Touch To feel the hard curvature of a baseball or the soft cylinder that is a soda can—these sensations we often take for granted. But amputees with a prosthetic arm know only that they are holding an object, the shape and stiffness discernible only by eye or from experience. Toward a more sophisticated prosthetic that can “feel” an object, Raspopovic and colleagues incorporated a feedback system connected to the amputee’s arm nerves, which delivers sensory information in real time. The authors connected electrodes in the arm nerves to sensors in two fingers of the prosthetic hand. To “feel” an object, the electrodes delivered electrical stimuli to the nerves that were proportional to the finger sensor readouts. To grasp an object and perform other motor commands, muscle signals were decoded. This bidirectional hand prosthetic was tested in a single amputee who was blindfolded and acoustically shielded to assure that sound and vision were not being used to manipulate objects. In more than 700 trials, the subject showed that he could modulate force and grasp and identify physical characteristics of different types of objects, such as cotton balls, an orange, and a piece of wood. Such sensory feedback with precise control over a hand prosthetic would allow amputees to more freely and naturally explore their environments. Hand loss is a highly disabling event that markedly affects the quality of life. To achieve a close to natural replacement for the lost hand, the user should be provided with the rich sensations that we naturally perceive when grasping or manipulating an object. Ideal bidirectional hand prostheses should involve both a reliable decoding of the user’s intentions and the delivery of nearly “natural” sensory feedback through remnant afferent pathways, simultaneously and in real time. However, current hand prostheses fail to achieve these requirements, particularly because they lack any sensory feedback. We show that by stimulating the median and ulnar nerve fascicles using transversal multichannel intrafascicular electrodes, according to the information provided by the artificial sensors from a hand prosthesis, physiologically appropriate (near-natural) sensory information can be provided to an amputee during the real-time decoding of different grasping tasks to control a dexterous hand prosthesis. This feedback enabled the participant to effectively modulate the grasping force of the prosthesis with no visual or auditory feedback. Three different force levels were distinguished and consistently used by the subject. The results also demonstrate that a high complexity of perception can be obtained, allowing the subject to identify the stiffness and shape of three different objects by exploiting different characteristics of the elicited sensations. This approach could improve the efficacy and “life-like” quality of hand prostheses, resulting in a keystone strategy for the near-natural replacement of missing hands.

Double nerve intraneural interface implant on a human amputee for robotic hand control

OBJECTIVES The principle underlying this project is that, despite nervous reorganization following upper limb amputation, original pathways and CNS relays partially maintain their function and can be exploited for interfacing prostheses. Aim of this study is to evaluate a novel peripheral intraneural multielectrode for multi-movement prosthesis control and for sensory feed-back, while assessing cortical reorganization following the re-acquired stream of data. METHODS Four intrafascicular longitudinal flexible multielectrodes (tf-LIFE4) were implanted in the median and ulnar nerves of an amputee; they reliably recorded output signals for 4 weeks. Artificial intelligence classifiers were used off-line to analyse LIFE signals recorded during three distinct hand movements under voluntary order. RESULTS Real-time control of motor output was achieved for the three actions. When applied off-line artificial intelligence reached >85% real-time correct classification of trials. Moreover, different types of current stimulation were determined to allow reproducible and localized hand/fingers sensations. Cortical organization was observed via TMS in parallel with partial resolution of symptoms due to the phantom-limb syndrome (PLS). CONCLUSIONS tf-LIFE4s recorded output signals in human nerves for 4 weeks, though the efficacy of sensory stimulation decayed after 10 days. Recording from a number of fibres permitted a high percentage of distinct actions to be classified correctly. Reversal of plastic changes and alleviation of PLS represent corollary findings of potential therapeutic benefit. SIGNIFICANCE This study represents a breakthrough in robotic hand use in amputees.

Control of Hand Prostheses Using Peripheral Information

Several efforts have been carried out to enhance dexterous hand prosthesis control by impaired individuals. Choosing which voluntary signal to use for control purposes is a critical element to achieve this goal. This review presents and discusses the recent results achieved by using electromyographic signals, recorded either with surface (sEMG) or intramuscular (iEMG) electrodes, and electroneurographic (ENG) signals. The potential benefits and shortcomings of the different approaches are described with a particular attention to the definition of all the steps required to achieve an effective hand prosthesis control in the different cases. Finally, a possible roadmap in the field is also presented.

On the Use of Longitudinal Intrafascicular Peripheral Interfaces for the Control of Cybernetic Hand Prostheses in Amputees

Significant strides have been recently made to develop highly sensorized cybernetic prostheses aimed at restoring sensorimotor limb functions to those who have lost them because of a traumatic event (amputation). In these cases, one of the main goals is to create a bidirectional link between the artificial devices (e.g., robotic hands, arms, or legs) and the nervous system. Several human-machine interfaces (HMIs) are currently used to this aim. Among them, interfaces with the peripheral nervous system and in particular longitudinal intrafascicular electrodes can be a promising solution able to improve the current situation. In this paper, the potentials and limits of the use of this interface to control robotic devices are presented. Specific information is provided on: 1) the neurophysiological bases for the use peripheral nerve interfaces; 2) a comparison of the potentials of the different peripheral neural interfaces; 3) the possibility of extracting and appropriately interpreting the neural code for motor commands and of delivering sensory feedback by stimulating afferent fibers by using longitudinal intrafascicular electrodes; 4) a preliminary comparative analysis of the performance of this approach with the ones of others HMIs; 5) the open issues which have to be addressed for a chronic usability of this approach.

On the use of wavelet denoising and spike sorting techniques to process electroneurographic signals recorded using intraneural electrodes.

Among the possible interfaces with the peripheral nervous system (PNS), intraneural electrodes represent an interesting solution for their potential advantages such as the possibility of extracting spikes from electroneurographic (ENG) signals. Their use could increase the precision and the amount of information which can be detected with respect to other processing methods. In this study, in order to verify this assumption, thin-film longitudinal intrafascicular electrodes (tfLIFE) were implanted in the sciatic nerve of rabbits. Various sensory stimuli were applied to the hind limb of the animal and the elicited ENG signals were recorded using the tfLIFEs. These signals were processed to determine whether the different types of information can be decoded. Signals were wavelet denoised and spike sorted. Support vector machines were trained to use the spike waveforms found to infer the stimulus applied to the rabbit. This approach was also compared with previously used ENG-processing methods. The results indicate that the combination of wavelet denoising and spike sorting techniques can increase the amount of information extractable from ENG signals recorded with intraneural electrodes. This strategy could allow the development of more effective closed-loop neuroprostheses and hybrid bionic systems connecting the human nervous system with artificial devices.

Decoding Information From Neural Signals Recorded Using Intraneural Electrodes: Toward the Development of a Neurocontrolled Hand Prosthesis

The possibility of controlling dexterous hand prostheses by using a direct connection with the nervous system is particularly interesting for the significant improvement of the quality of life of patients, which can derive from this achievement. Among the various approaches, peripheral nerve based intrafascicular electrodes are excellent neural interface candidates, representing an excellent compromise between high selectivity and relatively low invasiveness. Moreover, this approach has undergone preliminary testing in human volunteers and has shown promise. In this paper, we investigate whether the use of intrafascicular electrodes can be used to decode multiple sensory and motor information channels with the aim to develop a finite state algorithm that may be employed to control neuroprostheses and neurocontrolled hand prostheses. The results achieved both in animal and human experiments show that the combination of multiple sites recordings and advanced signal processing techniques (such as wavelet denoising and spike sorting algorithms) can be used to identify both sensory stimuli (in animal models) and motor commands (in a human volunteer). These findings have interesting implications, which should be investigated in future experiments.

Development and Quantitative Performance Evaluation of a Noninvasive EMG Computer Interface

This paper describes a noninvasive electromyography (EMG) signal-based computer interface and a performance evaluation method based on Fittspsila law. The EMG signals induced by volitional wrist movements were acquired from four sites in the lower arm to extract userspsila intentions, and six classes of wrist movements were distinguished using an artificial neural network. Using the developed interface, a user can move the cursor, click buttons, and type text on a computer. The test setup was built to evaluate the developed interface, and the mouse was tested by five volunteers with intact limbs. The performance of the developed computer interface and the mouse was tested at 1.299 and 7.733 b/s, respectively, and these results were compared with the performance of a commercial noninvasive brain signal interface (0.386 b/s). The results show that the developed interface performed better than the commercial interface, but less satisfactorily than a computer mouse. Although some issues remain to be resolved, the developed EMG interface has the potential to help people with motor disabilities to access computers and Internet environments in a natural and intuitive manner.

Decoding of grasping information from neural signals recorded using peripheral intrafascicular interfaces

BackgroundThe restoration of complex hand functions by creating a novel bidirectional link between the nervous system and a dexterous hand prosthesis is currently pursued by several research groups. This connection must be fast, intuitive, with a high success rate and quite natural to allow an effective bidirectional flow of information between the users nervous system and the smart artificial device. This goal can be achieved with several approaches and among them, the use of implantable interfaces connected with the peripheral nervous system, namely intrafascicular electrodes, is considered particularly interesting.MethodsThin-film longitudinal intra-fascicular electrodes were implanted in the median and ulnar nerves of an amputees stump during a four-week trial. The possibility of decoding motor commands suitable to control a dexterous hand prosthesis was investigated for the first time in this research field by implementing a spike sorting and classification algorithm.ResultsThe results showed that motor information (e.g., grip types and single finger movements) could be extracted with classification accuracy around 85% (for three classes plus rest) and that the user could improve his ability to govern motor commands over time as shown by the improved discrimination ability of our classification algorithm.ConclusionsThese results open up new and promising possibilities for the development of a neuro-controlled hand prosthesis.

On the identification of sensory information from mixed nerves by using single-channel cuff electrodes

BackgroundSeveral groups have shown that the performance of motor neuroprostheses can be significantly improved by detecting specific sensory events related to the ongoing motor task (e.g., the slippage of an object during grasping). Algorithms have been developed to achieve this goal by processing electroneurographic (ENG) afferent signals recorded by using single-channel cuff electrodes. However, no efforts have been made so far to understand the number and type of detectable sensory events that can be differentiated from whole nerve recordings using this approach.MethodsTo this aim, ENG afferent signals, evoked by different sensory stimuli were recorded using single-channel cuff electrodes placed around the sciatic nerve of anesthetized rats. The ENG signals were digitally processed and several features were extracted and used as inputs for the classification. The work was performed on integral datasets, without eliminating any noisy parts, in order to be as close as possible to real application.ResultsThe results obtained showed that single-channel cuff electrodes are able to provide information on two to three different afferent (proprioceptive, mechanical and nociceptive) stimuli, with reasonably good discrimination ability. The classification performances are affected by the SNR of the signal, which in turn is related to the diameter of the fibers encoding a particular type of neurophysiological stimulus.ConclusionsOur findings indicate that signals of acceptable SNR and corresponding to different physiological modalities (e.g. mediated by different types of nerve fibers) may be distinguished.

A sensorized thumb for force closed-loop control of hand neuroprostheses

In this paper, we presented a sensorized thumb based on a matrix of piezoresistive force sensors, with an acquisition unit and a special wearing support. The sensor was calibrated and then the device was tested during different tasks simulating activities of daily living performed by seven able-bodied subjects. By means of these experiments, we verified that the device proposed can be used to extract force information during grasp. In fact, the device was able to provide useful force information in the 98% of the trials with a good repeatability during all the different conditions. Moreover, we evaluated the patterns obtained during the different grasping tasks. The palmar grasps were performed in a similar manner, whereas the lateral pinch and the spherical volar grip were more different. This device can provide force information with good performance and acceptability and it can be used for force closed-loop control of hand neuroprostheses.