Matija Štrbac

Multichannel Electrotactile Feedback With Spatial and Mixed Coding for Closed-Loop Control of Grasping Force in Hand Prostheses

Providing somatosensory feedback to the user of a myoelectric prosthesis is an important goal since it can improve the utility as well as facilitate the embodiment of the assistive system. Most often, the grasping force was selected as the feedback variable and communicated through one or more individual single channel stimulation units (e.g., electrodes, vibration motors). In the present study, an integrated, compact, multichannel solution comprising an array electrode and a programmable stimulator was presented. Two coding schemes (15 levels), spatial and mixed (spatial and frequency) modulation, were tested in able-bodied subjects, psychometrically and in force control with routine grasping and force tracking using real and simulated prosthesis. The results demonstrated that mixed and spatial coding, although substantially different in psychometric tests, resulted in a similar performance during both force control tasks. Furthermore, the ideal, visual feedback was not better than the tactile feedback in routine grasping. To explain the observed results, a conceptual model was proposed emphasizing that the performance depends on multiple factors, including feedback uncertainty, nature of the task and the reliability of the feedforward control. The study outcomes, specific conclusions and the general model, are relevant for the design of closed-loop myoelectric prostheses utilizing tactile feedback.

Kinect in neurorehabilitation: Computer vision system for real time hand and object detection and distance estimation

This paper presents image processing and scene analysis methods that can provide artificial vision that is of interest for automatic selection of hand trajectory and prehension. The new algorithm, which uses data from the Kinect sensor, allows real-time detection of the hand of the person grasping an object at working table in front of that person. The outputs are real world coordinates of the hand and the object. The image processing is done in Matlab over the depth image stream taken from the Microsoft Kinect as a sensory input. Results show that in the presented system setup our program is capable of tracking hand movements in the transverse plane and estimating hand and object position in real-time with tolerable estimation error for the selection of stimulation paradigm that could control hand trajectory.

Integrated and flexible multichannel interface for electrotactile stimulation

OBJECTIVE The aim of the present work was to develop and test a flexible electrotactile stimulation system to provide real-time feedback to the prosthesis user. The system requirements were to accommodate the capabilities of advanced multi-DOF myoelectric hand prostheses and transmit the feedback variables (proprioception and force) using intuitive coding, with high resolution and after minimal training. APPROACH We developed a fully-programmable and integrated electrotactile interface supporting time and space distributed stimulation over custom designed flexible array electrodes. The system implements low-level access to individual stimulation channels as well as a set of high-level mapping functions translating the state of a multi-DoF prosthesis (aperture, grasping force, wrist rotation) into a set of predefined dynamic stimulation profiles. The system was evaluated using discrimination tests employing spatial and frequency coding (10 able-bodied subjects) and dynamic patterns (10 able-bodied and 6 amputee subjects). The outcome measure was the success rate (SR) in discrimination. MAIN RESULTS The more practical electrode with the common anode configuration performed similarly to the more usual concentric arrangement. The subjects could discriminate six spatial and four frequency levels with SR >90% after a few minutes of training, whereas the performance significantly deteriorated for more levels. The dynamic patterns were intuitive for the subjects, although amputees showed lower SR than able-bodied individuals (86% ± 10% versus 99% ± 3%). SIGNIFICANCE The tests demonstrated that the system was easy to setup and apply. The design and resolution of the multipad electrode was evaluated. Importantly, the novel dynamic patterns, which were successfully tested, can be superimposed to transmit multiple feedback variables intuitively and simultaneously. This is especially relevant for closing the loop in modern multifunction prostheses. Therefore, the proposed system is convenient for practical applications and can be used to implement sensory perception training and/or closed-loop control of myoelectric prostheses, providing grasping force and proprioceptive feedback.

Microsoft Kinect-Based Artificial Perception System for Control of Functional Electrical Stimulation Assisted Grasping

We present a computer vision algorithm that incorporates a heuristic model which mimics a biological control system for the estimation of control signals used in functional electrical stimulation (FES) assisted grasping. The developed processing software acquires the data from Microsoft Kinect camera and implements real-time hand tracking and object analysis. This information can be used to identify temporal synchrony and spatial synergies modalities for FES control. Therefore, the algorithm acts as artificial perception which mimics human visual perception by identifying the position and shape of the object with respect to the position of the hand in real time during the planning phase of the grasp. This artificial perception used within the heuristically developed model allows selection of the appropriate grasp and prehension. The experiments demonstrate that correct grasp modality was selected in more than 90% of tested scenarios/objects. The system is portable, and the components are low in cost and robust; hence, it can be used for the FES in clinical or even home environment. The main application of the system is envisioned for functional electrical therapy, that is, intensive exercise assisted with FES.

Electrotactile feedback improves performance and facilitates learning in the routine grasping task

Aim of this study was to investigate the feasibility of electrotactile feedback in closed loop training of force control during the routine grasping task. The feedback was provided using an array electrode and a simple six-level spatial coding, and the experiment was conducted in three amputee subjects. The psychometric tests confirmed that the subjects could perceive and interpret the electrotactile feedback with a high success rate. The subjects performed the routine grasping task comprising 4 blocks of 60 grasping trials. In each trial, the subjects employed feedforward control to close the hand and produce the desired grasping force (four levels). First (baseline) and the last (validation) session were performed in open loop, while the second and the third session (training) included electrotactile feedback. The obtained results confirmed that using the feedback improved the accuracy and precision of the force control. In addition, the subjects performed significantly better in the validation vs. baseline session, therefore suggesting that electrotactile feedback can be used for learning and training of myoelectric control.

Software Tool for the Prosthetic Foot Modeling and Stiffness Optimization

We present the procedure for the optimization of the stiffness of the prosthetic foot. The procedure allows the selection of the elements of the foot and the materials used for the design. The procedure is based on the optimization where the cost function is the minimization of the difference between the knee joint torques of healthy walking and the walking with the transfemural prosthesis. We present a simulation environment that allows the user to interactively vary the foot geometry and track the changes in the knee torque that arise from these adjustments. The software allows the estimation of the optimal prosthetic foot elasticity and geometry. We show that altering model attributes such as the length of the elastic foot segment or its elasticity leads to significant changes in the estimated knee torque required for a given trajectory.

Dynamic Stimulation Patterns for Conveying Proprioceptive Information from Multi-DOF Prosthesis

The aim of this study was to investigate the ability of the amputees to understand and identify proprioceptive feedback information presented by a set of dynamic stimulation patterns. The feedback relied on spatial coding of electrotactile stimuli, provided by a multichannel electrical stimulator, over custom designed array electrodes. Four stimulation patterns representing opening, closing, pronation and supination of a prosthetic hand were defined to mimic the change of the corresponding prosthesis degree of freedom. The psychometric evaluation on three amputee subjects confirmed that the four proposed dynamic stimulation patterns can be distinguished successfully after a short training.

Evolution of Surface Motor Activation Zones in Hemiplegic Patients During 20 Sessions of FES Therapy with Multi-pad Electrodes.

The purpose of this study was to examine surface motor activation zones for wrist, fingers and thumb extension movements and their temporal change during 20 therapy sessions using advanced multi-pad functional electrical stimulation system. Results from four hemiplegic patients indicate that certain zones have higher probability of eliciting each of the target movements. However, mutual overlap and variations of the zones are present not just between the subjects, but also on the intrasubject level, reflected through these session to session transformations of the selected virtual electrodes. The obtained results could be used as a priori knowledge for semi-automated optimization algorithm and could shorten the time required for calibration of the multi-pad electrode.

Wireless camera network system: Test of concept

In this manuscript a modular wireless system with low power consumption for image acquisition is presented. This system consists out of NXP LPC1786 microcontroller, Linksprite camera LSY201 with JPEG compression and UART interface, and Wifly RN131G Wi-Fi module. System is setup to be a part of Wi-Fi based wireless camera network and communicate with a distant control and acquisition system over a TCP/IP protocol. The aggregate speed of image acquisition and data transfer for various compression ratios are compared in order to explore further applicability of this system and possible technical advances in this field.