Ivan Vujaklija

Intuitive, Online, Simultaneous, and Proportional Myoelectric Control Over Two Degrees-of-Freedom in Upper Limb Amputees

We propose an approach for online simultaneous and proportional myoelectric control of two degrees-of-freedom (DoF) of the wrist, using surface electromyographic signals. The method is based on the nonnegative matrix factorization (NMF) of the wrist muscle activation to extract low-dimensional control signals translated by the user into kinematic variables. This procedure does not need a training set of signals for which the kinematics is known (labeled dataset) and is thus unsupervised (although it requires an initial calibration without labeled signals). The estimated control signals using NMF are used to directly control two DoFs of wrist. The method was tested on seven subjects with upper limb deficiency and on seven able-bodied subjects. The subjects performed online control of a virtual object with two DoFs to achieve goal-oriented tasks. The performance of the two subject groups, measured as the task completion rate, task completion time, and execution efficiency, was not statistically different. The approach was compared, and demonstrated to be superior to the online control by the industrial state-of-the-art approach. These results show that this new approach, which has several advantages over the previous myoelectric prosthetic control systems, has the potential of providing intuitive and dexterous control of artificial limbs for amputees.

High-Density Electromyography and Motor Skill Learning for Robust Long-Term Control of a 7-DoF Robot Arm

Myoelectric control offers a direct interface between human intent and various robotic applications through recorded muscle activity. Traditional control schemes realize this interface through direct mapping or pattern recognition techniques. The former approach provides reliable control at the expense of functionality, while the latter increases functionality at the expense of long-term reliability. An alternative approach, using concepts of motor learning, provides session-independent simultaneous control, but previously relied on consistent electrode placement over biomechanically independent muscles. This paper extends the functionality and practicality of the motor learning-based approach, using high-density electrode grids and muscle synergy-inspired decomposition to generate control inputs with reduced constraints on electrode placement. The method is demonstrated via real-time simultaneous and proportional control of a 4-DoF myoelectric interface over multiple days. Subjects showed learning trends consistent with typical motor skill learning without requiring any retraining or recalibration between sessions. Moreover, they adjusted to physical constraints of a robot arm after learning the control in a constraint-free virtual interface, demonstrating robust control as they performed precision tasks. The results demonstrate the efficacy of the proposed man-machine interface as a viable alternative to conventional control schemes for myoelectric interfaces designed for long-term use.

Noninvasive, Accurate Assessment of the Behavior of Representative Populations of Motor Units in Targeted Reinnervated Muscles

Targeted muscle reinnervation (TMR) redirects nerves that have lost their target, due to amputation, to remaining muscles in the region of the stump with the intent of establishing intuitive myosignals to control a complex prosthetic device. In order to directly recover the neural code underlying an attempted limb movement, in this paper, we present the decomposition of high-density surface electromyographic (EMG) signals detected from three TMR patients into the individual motor unit spike trains. The aim was to prove, for the first time, the feasibility of decoding the neural drive that would reach muscles of the missing limb in TMR patients, to show the accuracy of the decoding, and to demonstrate the representativeness of the pool of extracted motor units. Six to seven flexible EMG electrode grids of 64 electrodes each were mounted over the reinnervated muscles of each patient, resulting in up to 448 EMG signals. The subjects were asked to attempt elbow extension and flexion, hand open and close, wrist extension and flexion, wrist pronation and supination, of their missing limb. The EMG signals were decomposed using the Convolution Kernel Compensation technique and the decomposition accuracy was evaluated with a signal-based index of accuracy, called pulse-to-noise ratio (PNR). The results showed that the spike trains of 3 to 27 motor units could be identified for each task, with a sensitivity of the decomposition >90%, as revealed by PNR. The motor unit discharge rates were within physiological values of normally innervated muscles. Moreover, the detected motor units showed a high degree of common drive so that the set of extracted units per task was representative of the behavior of the population of active units. The results open a path for a new generation of human-machine interfaces in which the control signals are extracted from noninvasive recordings and the obtained neural information is based directly on the spike trains of motor neurons.

Context-Dependent Upper Limb Prosthesis Control for Natural and Robust Use

Pattern recognition and regression methods applied to the surface EMG have been used for estimating the user intended motor tasks across multiple degrees of freedom (DOF), for prosthetic control. While these methods are effective in several conditions, they are still characterized by some shortcomings. In this study we propose a methodology that combines these two approaches for mutually alleviating their limitations. This resulted in a control method capable of context-dependent movement estimation that switched automatically between sequential (one DOF at a time) or simultaneous (multiple DOF) prosthesis control, based on an online estimation of signal dimensionality. The proposed method was evaluated in scenarios close to real-life situations, with the control of a physical prosthesis in applied tasks of varying difficulties. Test prostheses were individually manufactured for both able-bodied and transradial amputee subjects. With these prostheses, two amputees performed the Southampton Hand Assessment Procedure test with scores of 58 and 71 points. The five able-bodied individuals performed standardized tests, such as the box&block and clothes pin test, reducing the completion times by up to 30%, with respect to using a state-of-the-art pure sequential control algorithm. Apart from facilitating fast simultaneous movements, the proposed control scheme was also more intuitive to use, since human movements are predominated by simultaneous activations across joints. The proposed method thus represents a significant step towards intelligent, intuitive and natural control of upper limb prostheses.

New developments in prosthetic arm systems

Absence of an upper limb leads to severe impairments in everyday life, which can further influence the social and mental state. For these reasons, early developments in cosmetic and body-driven prostheses date some centuries ago, and they have been evolving ever since. Following the end of the Second World War, rapid developments in technology resulted in powered myoelectric hand prosthetics. In the years to come, these devices were common on the market, though they still suffered high user abandonment rates. The reasons for rejection were trifold – insufficient functionality of the hardware, fragile design, and cumbersome control. In the last decade, both academia and industry have reached major improvements concerning technical features of upper limb prosthetics and methods for their interfacing and control. Advanced robotic hands are offered by several vendors and research groups, with a variety of active and passive wrist options that can be articulated across several degrees of freedom. Nowadays, elbow joint designs include active solutions with different weight and power options. Control features are getting progressively more sophisticated, offering options for multiple sensor integration and multi-joint articulation. Latest developments in socket designs are capable of facilitating implantable and multiple surface electromyography sensors in both traditional and osseointegration-based systems. Novel surgical techniques in combination with modern, sophisticated hardware are enabling restoration of dexterous upper limb functionality. This article is aimed at reviewing the latest state of the upper limb prosthetic market, offering insights on the accompanying technologies and techniques. We also examine the capabilities and features of some of academia’s flagship solutions and methods.

Bayesian Filtering of Surface EMG for Accurate Simultaneous and Proportional Prosthetic Control.

The amplitude of the surface EMG (sEMG) is commonly estimated by rectification or other nonlinear transformations, followed by smoothing (low-pass linear filtering). Although computationally efficient, this approach leads to an estimation accuracy with a limited theoretical signal-to-noise ratio (SNR). Since sEMG amplitude is one of the most relevant features for myoelectric control, its estimate has become one of the limiting factors for the performance of myoelectric control applications, such as powered prostheses. In this study, we present a recursive nonlinear estimator of sEMG amplitude based on Bayesian filtering. Furthermore, we validate the advantage of the proposed Bayesian filter over the conventional linear filters through an online simultaneous and proportional control (SPC) task, performed by eight able-bodied subjects and three below-elbow limb deficient subjects. The results demonstrated that the proposed Bayesian filter provides significantly more accurate SPC, particularly for the patients, when compared with conventional linear filters. This result presents a major step toward accurate prosthetic control for advanced multi-function prostheses.

Translating Research on Myoelectric Control into Clinics—Are the Performance Assessment Methods Adequate?

Missing an upper limb dramatically impairs daily-life activities. Efforts in overcoming the issues arising from this disability have been made in both academia and industry, although their clinical outcome is still limited. Translation of prosthetic research into clinics has been challenging because of the difficulties in meeting the necessary requirements of the market. In this perspective article, we suggest that one relevant factor determining the relatively small clinical impact of myocontrol algorithms for upper limb prostheses is the limit of commonly used laboratory performance metrics. The laboratory conditions, in which the majority of the solutions are being evaluated, fail to sufficiently replicate real-life challenges. We qualitatively support this argument with representative data from seven transradial amputees. Their ability to control a myoelectric prosthesis was tested by measuring the accuracy of offline EMG signal classification, as a typical laboratory performance metrics, as well as by clinical scores when performing standard tests of daily living. Despite all subjects reaching relatively high classification accuracy offline, their clinical scores varied greatly and were not strongly predicted by classification accuracy. We therefore support the suggestion to test myocontrol systems using clinical tests on amputees, fully fitted with sockets and prostheses highly resembling the systems they would use in daily living, as evaluation benchmark. Agreement on this level of testing for systems developed in research laboratories would facilitate clinically relevant progresses in this field.

A Structured Rehabilitation Protocol for Improved Multifunctional Prosthetic Control: A Case Study.

Advances in robotic systems have resulted in prostheses for the upper limb that can produce multifunctional movements. However, these sophisticated systems require upper limb amputees to learn complex control schemes. Humans have the ability to learn new movements through imitation and other learning strategies. This protocol describes a structured rehabilitation method, which includes imitation, repetition, and reinforcement learning, and aims to assess if this method can improve multifunctional prosthetic control. A left below elbow amputee, with 4 years of experience in prosthetic use, took part in this case study. The prosthesis used was a Michelangelo hand with wrist rotation, and the added features of wrist flexion and extension, which allowed more combinations of hand movements. The participants Southampton Hand Assessment Procedure score improved from 58 to 71 following structured training. This suggests that a structured training protocol of imitation, repetition and reinforcement may have a role in learning to control a new prosthetic hand. A larger clinical study is however required to support these findings.

Elective amputation and bionic substitution restore functional hand use after critical soft tissue injuries.

Critical soft tissue injuries may lead to a non-functional and insensate limb. In these cases standard reconstructive techniques will not suffice to provide a useful outcome, and solutions outside the biological arena must be considered and offered to these patients. We propose a concept which, after all reconstructive options have been exhausted, involves an elective amputation along with a bionic substitution, implementing an actuated prosthetic hand via a structured tech-neuro-rehabilitation program. Here, three patients are presented in whom this concept has been successfully applied after mutilating hand injuries. Clinical tests conducted before, during and after the procedure, evaluating both functional and psychometric parameters, document the benefits of this approach. Additionally, in one of the patients, we show the possibility of implementing a highly functional and natural control of an advanced prosthesis providing both proportional and simultaneous movements of the wrist and hand for completing tasks of daily living with substantially less compensatory movements compared to the traditional systems. It is concluded that the proposed procedure is a viable solution for re-gaining highly functional hand use following critical soft tissue injuries when existing surgical measures fail. Our results are clinically applicable and can be extended to institutions with similar resources.

Simultaneous myoelectric control of a robot arm using muscle synergy-inspired inputs from high-density electrode grids

Myoelectric control has seen decades of research as a potential interface between human and machines. High-density surface electromyography (HDsEMG) non-invasively provides a rich set of signals representing underlying muscle contractions and, at a higher level, human motion intent. Many pattern recognition techniques have been proposed to predict motions based on these signals. However, control schemes incorporating pattern recognition struggle with long-term reliability due to signal stochasticity and transient changes. This study proposes an alternative approach for HDsEMG-based interfaces using concepts of motor skill learning and muscle synergies to address long-term reliability. Muscle synergy-inspired decomposition reduces HDsEMG into control inputs robust to small electrode displacements. The novel control scheme provides simultaneous and proportional control, and is learned by the subject simply by interacting with the device. In a multiple-day experiment, subjects learned to control a virtual 7-DoF myoelectric interface, displaying performance learning curves consistent with motor skill learning. On a separate day, subjects intuitively transferred this learning to demonstrate precision tasks with a 7-DoF robot arm, without requiring any recalibration. These results suggest that the proposed method may be a practical alternative to pattern recognition-based control for long-term use of myoelectric interfaces.