Luca Citi

Restoring Natural Sensory Feedback in Real-Time Bidirectional Hand Prostheses

A multigrasp, bidirectional hand prosthesis delivers dynamic sensory feedback, allowing a user with a hand amputation to achieve fine grasping force control and realistic object sensing. An Artificial Hand’s Sense of Touch To feel the hard curvature of a baseball or the soft cylinder that is a soda can—these sensations we often take for granted. But amputees with a prosthetic arm know only that they are holding an object, the shape and stiffness discernible only by eye or from experience. Toward a more sophisticated prosthetic that can “feel” an object, Raspopovic and colleagues incorporated a feedback system connected to the amputee’s arm nerves, which delivers sensory information in real time. The authors connected electrodes in the arm nerves to sensors in two fingers of the prosthetic hand. To “feel” an object, the electrodes delivered electrical stimuli to the nerves that were proportional to the finger sensor readouts. To grasp an object and perform other motor commands, muscle signals were decoded. This bidirectional hand prosthetic was tested in a single amputee who was blindfolded and acoustically shielded to assure that sound and vision were not being used to manipulate objects. In more than 700 trials, the subject showed that he could modulate force and grasp and identify physical characteristics of different types of objects, such as cotton balls, an orange, and a piece of wood. Such sensory feedback with precise control over a hand prosthetic would allow amputees to more freely and naturally explore their environments. Hand loss is a highly disabling event that markedly affects the quality of life. To achieve a close to natural replacement for the lost hand, the user should be provided with the rich sensations that we naturally perceive when grasping or manipulating an object. Ideal bidirectional hand prostheses should involve both a reliable decoding of the user’s intentions and the delivery of nearly “natural” sensory feedback through remnant afferent pathways, simultaneously and in real time. However, current hand prostheses fail to achieve these requirements, particularly because they lack any sensory feedback. We show that by stimulating the median and ulnar nerve fascicles using transversal multichannel intrafascicular electrodes, according to the information provided by the artificial sensors from a hand prosthesis, physiologically appropriate (near-natural) sensory information can be provided to an amputee during the real-time decoding of different grasping tasks to control a dexterous hand prosthesis. This feedback enabled the participant to effectively modulate the grasping force of the prosthesis with no visual or auditory feedback. Three different force levels were distinguished and consistently used by the subject. The results also demonstrate that a high complexity of perception can be obtained, allowing the subject to identify the stiffness and shape of three different objects by exploiting different characteristics of the elicited sensations. This approach could improve the efficacy and “life-like” quality of hand prostheses, resulting in a keystone strategy for the near-natural replacement of missing hands.

Double nerve intraneural interface implant on a human amputee for robotic hand control

OBJECTIVES The principle underlying this project is that, despite nervous reorganization following upper limb amputation, original pathways and CNS relays partially maintain their function and can be exploited for interfacing prostheses. Aim of this study is to evaluate a novel peripheral intraneural multielectrode for multi-movement prosthesis control and for sensory feed-back, while assessing cortical reorganization following the re-acquired stream of data. METHODS Four intrafascicular longitudinal flexible multielectrodes (tf-LIFE4) were implanted in the median and ulnar nerves of an amputee; they reliably recorded output signals for 4 weeks. Artificial intelligence classifiers were used off-line to analyse LIFE signals recorded during three distinct hand movements under voluntary order. RESULTS Real-time control of motor output was achieved for the three actions. When applied off-line artificial intelligence reached >85% real-time correct classification of trials. Moreover, different types of current stimulation were determined to allow reproducible and localized hand/fingers sensations. Cortical organization was observed via TMS in parallel with partial resolution of symptoms due to the phantom-limb syndrome (PLS). CONCLUSIONS tf-LIFE4s recorded output signals in human nerves for 4 weeks, though the efficacy of sensory stimulation decayed after 10 days. Recording from a number of fibres permitted a high percentage of distinct actions to be classified correctly. Reversal of plastic changes and alleviation of PLS represent corollary findings of potential therapeutic benefit. SIGNIFICANCE This study represents a breakthrough in robotic hand use in amputees.

P300-Based BCI Mouse With Genetically-Optimized Analogue Control

In this paper we propose a brain-computer interface (BCI) mouse based on P300 waves in electroencephalogram (EEG) signals. The system is analogue in that at no point a binary decision is made as to whether or not a P300 was actually produced in response to the stimuli. Instead, the 2D motion of the pointer on the screen, using a novel BCI paradigm, is controlled by directly combining the amplitudes of the output produced by a filter in the presence of different stimuli. This filter and the features to be combined within it are optimised by an evolutionary algorithm.

On the Use of Longitudinal Intrafascicular Peripheral Interfaces for the Control of Cybernetic Hand Prostheses in Amputees

Significant strides have been recently made to develop highly sensorized cybernetic prostheses aimed at restoring sensorimotor limb functions to those who have lost them because of a traumatic event (amputation). In these cases, one of the main goals is to create a bidirectional link between the artificial devices (e.g., robotic hands, arms, or legs) and the nervous system. Several human-machine interfaces (HMIs) are currently used to this aim. Among them, interfaces with the peripheral nervous system and in particular longitudinal intrafascicular electrodes can be a promising solution able to improve the current situation. In this paper, the potentials and limits of the use of this interface to control robotic devices are presented. Specific information is provided on: 1) the neurophysiological bases for the use peripheral nerve interfaces; 2) a comparison of the potentials of the different peripheral neural interfaces; 3) the possibility of extracting and appropriately interpreting the neural code for motor commands and of delivering sensory feedback by stimulating afferent fibers by using longitudinal intrafascicular electrodes; 4) a preliminary comparative analysis of the performance of this approach with the ones of others HMIs; 5) the open issues which have to be addressed for a chronic usability of this approach.

On the use of wavelet denoising and spike sorting techniques to process electroneurographic signals recorded using intraneural electrodes.

Among the possible interfaces with the peripheral nervous system (PNS), intraneural electrodes represent an interesting solution for their potential advantages such as the possibility of extracting spikes from electroneurographic (ENG) signals. Their use could increase the precision and the amount of information which can be detected with respect to other processing methods. In this study, in order to verify this assumption, thin-film longitudinal intrafascicular electrodes (tfLIFE) were implanted in the sciatic nerve of rabbits. Various sensory stimuli were applied to the hind limb of the animal and the elicited ENG signals were recorded using the tfLIFEs. These signals were processed to determine whether the different types of information can be decoded. Signals were wavelet denoised and spike sorted. Support vector machines were trained to use the spike waveforms found to infer the stimulus applied to the rabbit. This approach was also compared with previously used ENG-processing methods. The results indicate that the combination of wavelet denoising and spike sorting techniques can increase the amount of information extractable from ENG signals recorded with intraneural electrodes. This strategy could allow the development of more effective closed-loop neuroprostheses and hybrid bionic systems connecting the human nervous system with artificial devices.

Revealing Real-Time Emotional Responses: a Personalized Assessment based on Heartbeat Dynamics

Emotion recognition through computational modeling and analysis of physiological signals has been widely investigated in the last decade. Most of the proposed emotion recognition systems require relatively long-time series of multivariate records and do not provide accurate real-time characterizations using short-time series. To overcome these limitations, we propose a novel personalized probabilistic framework able to characterize the emotional state of a subject through the analysis of heartbeat dynamics exclusively. The study includes thirty subjects presented with a set of standardized images gathered from the international affective picture system, alternating levels of arousal and valence. Due to the intrinsic nonlinearity and nonstationarity of the RR interval series, a specific point-process model was devised for instantaneous identification considering autoregressive nonlinearities up to the third-order according to the Wiener-Volterra representation, thus tracking very fast stimulus-response changes. Features from the instantaneous spectrum and bispectrum, as well as the dominant Lyapunov exponent, were extracted and considered as input features to a support vector machine for classification. Results, estimating emotions each 10 seconds, achieve an overall accuracy in recognizing four emotional states based on the circumplex model of affect of 79.29%, with 79.15% on the valence axis, and 83.55% on the arousal axis.

Defining brain-machine interface applications by matching interface performance with device requirements

Interaction with machines is mediated by human-machine interfaces (HMIs). Brain-machine interfaces (BMIs) are a particular class of HMIs and have so far been studied as a communication means for people who have little or no voluntary control of muscle activity. In this context, low-performing interfaces can be considered as prosthetic applications. On the other hand, for able-bodied users, a BMI would only be practical if conceived as an augmenting interface. In this paper, a method is introduced for pointing out effective combinations of interfaces and devices for creating real-world applications. First, devices for domotics, rehabilitation and assistive robotics, and their requirements, in terms of throughput and latency, are described. Second, HMIs are classified and their performance described, still in terms of throughput and latency. Then device requirements are matched with performance of available interfaces. Simple rehabilitation and domotics devices can be easily controlled by means of BMI technology. Prosthetic hands and wheelchairs are suitable applications but do not attain optimal interactivity. Regarding humanoid robotics, the head and the trunk can be controlled by means of BMIs, while other parts require too much throughput. Robotic arms, which have been controlled by means of cortical invasive interfaces in animal studies, could be the next frontier for non-invasive BMIs. Combining smart controllers with BMIs could improve interactivity and boost BMI applications.

Decoding Information From Neural Signals Recorded Using Intraneural Electrodes: Toward the Development of a Neurocontrolled Hand Prosthesis

The possibility of controlling dexterous hand prostheses by using a direct connection with the nervous system is particularly interesting for the significant improvement of the quality of life of patients, which can derive from this achievement. Among the various approaches, peripheral nerve based intrafascicular electrodes are excellent neural interface candidates, representing an excellent compromise between high selectivity and relatively low invasiveness. Moreover, this approach has undergone preliminary testing in human volunteers and has shown promise. In this paper, we investigate whether the use of intrafascicular electrodes can be used to decode multiple sensory and motor information channels with the aim to develop a finite state algorithm that may be employed to control neuroprostheses and neurocontrolled hand prostheses. The results achieved both in animal and human experiments show that the combination of multiple sites recordings and advanced signal processing techniques (such as wavelet denoising and spike sorting algorithms) can be used to identify both sensory stimuli (in animal models) and motor commands (in a human volunteer). These findings have interesting implications, which should be investigated in future experiments.

Point-Process Nonlinear Models With Laguerre and Volterra Expansions: Instantaneous Assessment of Heartbeat Dynamics

In the last decades, mathematical modeling and signal processing techniques have played an important role in the study of cardiovascular control physiology and heartbeat nonlinear dynamics. In particular, nonlinear models have been devised for the assessment of the cardiovascular system by accounting for short-memory second-order nonlinearities. In this paper, we introduce a novel inverse Gaussian point process model with Laguerre expansion of the nonlinear Volterra kernels. Within the model, the second-order nonlinearities also account for the long-term information given by the past events of the nonstationary non-Gaussian time series. In addition, the mathematical link to an equivalent cubic input-output Wiener-Volterra model allows for a novel instantaneous estimation of the dynamic spectrum, bispectrum and trispectrum of the considered inter-event intervals. The proposed framework is tested with synthetic simulations and two experimental heartbeat interval datasets. Applications on further heterogeneous datasets such as milling inserts, neural spikes, gait from short walks, and geyser geologic events are also reported. Results show that our model improves on previously developed models and, at the same time, it is able to provide a novel instantaneous characterization and tracking of the inherent nonlinearity of heartbeat dynamics.

Characterization of Depressive States in Bipolar Patients Using Wearable Textile Technology and Instantaneous Heart Rate Variability Assessment

The analysis of cognitive and autonomic responses to emotionally relevant stimuli could provide a viable solution for the automatic recognition of different mood states, both in normal and pathological conditions. In this study, we present a methodological application describing a novel system based on wearable textile technology and instantaneous nonlinear heart rate variability assessment, able to characterize the autonomic status of bipolar patients by considering only electrocardiogram recordings. As a proof of this concept, our study presents results obtained from eight bipolar patients during their normal daily activities and being elicited according to a specific emotional protocol through the presentation of emotionally relevant pictures. Linear and nonlinear features were computed using a novel point-process-based nonlinear autoregressive integrative model and compared with traditional algorithmic methods. The estimated indices were used as the input of a multilayer perceptron to discriminate the depressive from the euthymic status. Results show that our system achieves much higher accuracy than the traditional techniques. Moreover, the inclusion of instantaneous higher order spectra features significantly improves the accuracy in successfully recognizing depression from euthymia.