Luke Osborn

Neuromimetic Event-Based Detection for Closed-Loop Tactile Feedback Control of Upper Limb Prostheses

Upper limb amputees lack the valuable tactile sensing that helps provide context about the surrounding environment. Here, we utilize tactile information to provide active touch feedback to a prosthetic hand. First, we developed fingertip tactile sensors for producing biomimetic spiking responses for monitoring contact, release, and slip of an object grasped by a prosthetic hand. We convert the sensor output into pulses, mimicking the rapid and slowly adapting spiking responses of receptor afferents found in the human body. Second, we designed and implemented two neuromimetic event-based algorithms, Compliant Grasping and Slip Prevention, on a prosthesis to create a local closed-loop tactile feedback control system (i.e., tactile information is sent to the prosthesis). Grasping experiments were designed to assess the benefit of this biologically inspired neuromimetic tactile feedback to a prosthesis. Results from able-bodied and amputee subjects show the average number of objects that broke or slipped during grasping decreased by over 50 percent and the average time to complete a grasping task decreased by at least 10 percent for most trials when comparing neuromimetic tactile feedback with no feedback on a prosthesis. Our neuromimetic method of closed-loop tactile sensing is a novel approach to improving the function of upper limb prostheses.

Tactile feedback in upper limb prosthetic devices using flexible textile force sensors

Many upper limb amputees are faced with the difficult challenge of using a prosthesis that lacks tactile sensing. State of the art research caliber prosthetic hands are often equipped with sophisticated sensors that provide valuable information regarding the prosthesis and its surrounding environment. Unfortunately, most commercial prosthetic hands do not contain any tactile sensing capabilities. In this paper, a textile based tactile sensor system was designed, built, and evaluated for use with upper limb prosthetic devices. Despite its simplicity, we demonstrate the ability of the sensors to determine object contact and perturbations due to slip during a grasping task with a prosthetic hand. This suggests the use of low-cost, customizable, textile sensors as part of a closed-loop tactile feedback system for monitoring grasping forces specifically in an upper limb prosthetic device.

Dynamic training protocol improves the robustness of PR-based myoelectric control

Abstract In pattern recognition (PR)-based myoelectric control schemes, the classifier is generally trained in ideal laboratory conditions, due to which the classification accuracy might be affected by confounding factors such as force variations, limb positions, and inadvertent electromyography (EMG) activation. Many endeavors have been put forward to mitigate this effect by adopting new training protocols that consider only quite a few independent factors. In this note, we propose a dynamic protocol, which embraces multiple EMG variations in data collection, to train a classifier with improved generalization ability. A total of four training protocols are examined, wherein affecting factors like upper-limb movements, contraction levels and inadvertent EMG activations are differently considered. Based on receiver operating characteristic (ROC) analysis, we came up with a new performance metric, ROC area rate (RAR), to directly inspect the accuracy and robustness of the classifiers obtained through different training protocols. Our results show that, compared with the other three protocols, the protocol with dynamic limb postures and dynamic muscle contractions (termed as DPDE) obtains the highest RAR (73.3%, on-way analysis of variance, p 0.005 ). Our results suggest that there is no need to integrate every EMG variation in the training protocol for receiving a robust EMG pattern recognition. Online control experiments with three amputees manipulating a multiple-DOF prosthetic hand also verify our findings.

Prosthesis with neuromorphic multilayered e-dermis perceives touch and pain

A multilayered e-dermis allows a prosthesis and an amputee to perceive a range of innocuous and noxious tactile stimuli. The human body is a template for many state-of-the-art prosthetic devices and sensors. Perceptions of touch and pain are fundamental components of our daily lives that convey valuable information about our environment while also providing an element of protection from damage to our bodies. Advances in prosthesis designs and control mechanisms can aid an amputee’s ability to regain lost function but often lack meaningful tactile feedback or perception. Through transcutaneous electrical nerve stimulation (TENS) with an amputee, we discovered and quantified stimulation parameters to elicit innocuous (nonpainful) and noxious (painful) tactile perceptions in the phantom hand. Electroencephalography (EEG) activity in somatosensory regions confirms phantom hand activation during stimulation. We invented a multilayered electronic dermis (e-dermis) with properties based on the behavior of mechanoreceptors and nociceptors to provide neuromorphic tactile information to an amputee. Our biologically inspired e-dermis enables a prosthesis and its user to perceive a continuous spectrum from innocuous to noxious touch through a neuromorphic interface that produces receptor-like spiking neural activity. In a pain detection task (PDT), we show the ability of the prosthesis and amputee to differentiate nonpainful or painful tactile stimuli using sensory feedback and a pain reflex feedback control system. In this work, an amputee can use perceptions of touch and pain to discriminate object curvature, including sharpness. This work demonstrates possibilities for creating a more natural sensation spanning a range of tactile stimuli for prosthetic hands.

Utilizing tactile feedback for biomimetic grasping control in upper limb prostheses

A biomimetic system for enhancing the control and reliability of grasping with prosthetic hands was designed and experimentally evaluated. Barometric pressure sensors as well as a force-sensitive resistor (FSR) were placed on a prosthetic hand to provide valuable tactile feedback. Contact and slip detection grip control algorithms were developed to interpret force signals for enhancing stable grasping. Recent advances in radio-frequency identification (RFID) technology enable the amputee to select between grip control strategies based on the desired object to be grasped. Experimental results indicate that the control algorithms are capable of utilizing real-time force responses to detect object contact as well as slip. By allowing the user to act as a high-level controller with RFID technology, a multi-faceted low-level controller that responds to tactile feedback can be developed for enhancing grasping functionality in prosthetic hands.

Limb-position robust classification of myoelectric signals for prosthesis control using sparse representations

The fundamental objective in non-invasive myoelectric prosthesis control is to determine the users intended movements from corresponding skin-surface recorded electromyographic (sEMG) activation signals as quickly and accurately as possible. Linear Discriminant Analysis (LDA) has emerged as the de facto standard for real-time movement classification due to its ease of use, calculation speed, and remarkable classification accuracy under controlled training conditions. However, performance of cluster-based methods like LDA for sEMG pattern recognition degrades significantly when real-world testing conditions do not resemble the trained conditions, limiting the utility of myoelectrically controlled prosthesis devices. We propose an enhanced classification method that is more robust to generic deviations from training conditions by constructing sparse representations of the input data dictionary comprised of sEMG time-frequency features. We apply our method in the context of upper-limb position changes to demonstrate pattern recognition robustness and improvement over LDA across discrete positions not explicitly trained. For single position training we report an accuracy improvement in untrained positions of 7.95%, p ≪ .001, in addition to significant accuracy improvements across all multiposition training conditions, p <; .001.The fundamental objective in non-invasive myoelectric prosthesis control is to determine the users intended movements from corresponding skin-surface recorded electromyographic (sEMG) activation signals as quickly and accurately as possible. Linear Discriminant Analysis (LDA) has emerged as the de facto standard for real-time movement classification due to its ease of use, calculation speed, and remarkable classification accuracy under controlled training conditions. However, performance of cluster-based methods like LDA for sEMG pattern recognition degrades significantly when real-world testing conditions do not resemble the trained conditions, limiting the utility of myoelectrically controlled prosthesis devices. We propose an enhanced classification method that is more robust to generic deviations from training conditions by constructing sparse representations of the input data dictionary comprised of sEMG time-frequency features. We apply our method in the context of upper-limb position changes to demonstrate pattern recognition robustness and improvement over LDA across discrete positions not explicitly trained. For single position training we report an accuracy improvement in untrained positions of 7.95%, p ≪ .001, in addition to significant accuracy improvements across all multiposition training conditions, p <; .001.

Real-time arm tracking for HMI applications

Limb tracking is an important aspect of human-machine interfaces (HMI). These systems, however, can often be limited by complex algorithms requiring significant processing power, obtrusive and immobile sensing techniques, and high costs. In this work, we utilize a sensor fusion algorithm implemented in commercial inertial measurement units (IMU) to combine accelerometer and gyroscope measurements in an effort to minimize computational requirements of the limb tracking system. In addition, previously developed methods were implemented to eliminate sensor drift by including information from a magnetometer. We tested the accuracy of our system by computing the root mean squared error (RMSE) of the true angle between the headings of two sensors and the estimate of that angle through quaternion-vector manipulations. An average RMSE of approximately 2.9° was achieved. Our limb tracking system is wearable, minimally complex, low-cost, and simple to use which has proven useful in multiple HMI applications discussed herein.

Live demonstration — An adaptable prosthetic socket: Regulating independent air bladders through closed-loop control

This is a live demonstration of the work described in [l]. The paper ID of this submission is 1292. The goal of this work is to maintain specific pressures on the model residual limb (MRL) to counteract the pressure changes caused by loading/limb movement. Custom textile force sensors are embedded in between the air bladders and the socket. These force sensors communicate with the fluidic control board, which based on a proportional algorithm maintains airflow to the bladders, in response to the changing loads on the socket.

Live demonstration: Prosthesis grip force modulation using neuromorphic tactile sensing

This is a live demonstration of the work described in [1]. The paper ID of this submission is 1634. The goal of this work is to use a neuromorphic model for providing tactile feedback to a prosthetic hand to improve grasping functionality. Custom force sensors are placed on the fingertips of a bebionc3 (Steeper, Leeds, UK) prosthetic hand and communicate with the prosthesis controller (Infinite Biomedical Technologies, Baltimore, USA). The prosthesis grip force is used as the input to a leaky integrate and fire (LIF) with spike rate adaption neuron model to produce a tactile signal represented by spiking information, which is similar to the behavior of mechanoreceptors found in humans. The prosthesis controller uses the spiking information to modulate the grip force and allow the hand to grasp a delicate object.

An adaptable prosthetic socket: Regulating independent air bladders through closed-loop control

During grasping or natural movement a prosthesis experiences varying loads, which can directly impact the comfort and fit of a socket on the users residual limb. To alleviate this issue, four independent air bladders were integrated in a custom prosthetic socket, which contacted a model residual limb. The purpose of the bladders was to maintain specific pressures on the model residual limb to counteract pressure changes caused by an applied load. To sense pressure, calibrated custom piezoresistive sensors were placed between the air bladder and the model residual limb. A closed-loop algorithm was implemented to utilize sensor feedback and update airflow into each bladder to achieve a pressure equilibrium that maintains a static socket-limb system. The RMS error of internal pressure using pulse width modulation (PWM) bladder regulation decreased by 33% when compared no pressure regulation on the top bladder. We showed that this adaptive biomedical system for the human machine interface between a residual limb and socket can improve stability for prosthesis users using real-time pressure feedback.