Miloš Kostić

Cognitive vision system for control of dexterous prosthetic hands: Experimental evaluation

BackgroundDexterous prosthetic hands that were developed recently, such as SmartHand and i-LIMB, are highly sophisticated; they have individually controllable fingers and the thumb that is able to abduct/adduct. This flexibility allows implementation of many different grasping strategies, but also requires new control algorithms that can exploit the many degrees of freedom available. The current study presents and tests the operation of a new control method for dexterous prosthetic hands.MethodsThe central component of the proposed method is an autonomous controller comprising a vision system with rule-based reasoning mounted on a dexterous hand (CyberHand). The controller, termed cognitive vision system (CVS), mimics biological control and generates commands for prehension. The CVS was integrated into a hierarchical control structure: 1) the user triggers the system and controls the orientation of the hand; 2) a high-level controller automatically selects the grasp type and size; and 3) an embedded hand controller implements the selected grasp using closed-loop position/force control. The operation of the control system was tested in 13 healthy subjects who used Cyberhand, attached to the forearm, to grasp and transport 18 objects placed at two different distances.ResultsThe system correctly estimated grasp type and size (nine commands in total) in about 84% of the trials. In an additional 6% of the trials, the grasp type and/or size were different from the optimal ones, but they were still good enough for the grasp to be successful. If the control task was simplified by decreasing the number of possible commands, the classification accuracy increased (e.g., 93% for guessing the grasp type only).ConclusionsThe original outcome of this research is a novel controller empowered by vision and reasoning and capable of high-level analysis (i.e., determining object properties) and autonomous decision making (i.e., selecting the grasp type and size). The automatic control eases the burden from the user and, as a result, the user can concentrate on what he/she does, not on how he/she should do it. The tests showed that the performance of the controller was satisfactory and that the users were able to operate the system with minimal prior training.

Multi-Pad Electrode for Effective Grasping: Design

We designed a new surface multi-pad electrode for the electrical stimulation of the forearm that is effective in controlling functional grasp in hemiplegic patients. The electrode shape and size were designed on the basis of the surface stimulation map of the forearm, determined from measurements in seven hemiplegic patients who had limited or absent voluntary movements of the fingers, thumb and wrist. The stimulation map for each patient was assessed with a conventional set of single pad Pals Platinum electrodes. Since the sites for the stimulation varied greatly between patients, the end result was a rather large multi-pad electrode. Modulating multi-pad electrode size, shape, position and individual pad stimulation parameters allows us to accommodate the diversity of the neural tissues in patients that need to be activated for functional grasp. This also allows asynchronous activation of different portions of the muscle and dynamic adaptation of the stimulation sites to appropriate underlying tissues during functional use. The validity of the determined stimulation map was tested in the same group of hemiplegic patients. The selected set of active pads resulted in fully functional and reproducible palmar and lateral grasps similar to healthy-like grasps.

Feedback-Mediated Upper Extremities Exercise: Increasing Patient Motivation in Poststroke Rehabilitation

Purpose. This proof-of-concept study investigated whether feedback-mediated exercise (FME) of the affected arm of hemiplegic patients increases patient motivation and promotes greater improvement of motor function, compared to no-feedback exercise (NFE). Method. We developed a feedback-mediated treatment that uses gaming scenarios and allows online and offline monitoring of both temporal and spatial characteristics of planar movements. Twenty poststroke hemiplegic inpatients, randomly assigned to the FME and NFE group, received therapy five days a week for three weeks. The outcome measures were evaluated from the following: (1) the modified drawing test (mDT), (2) received therapy time—RTT, and (3) intrinsic motivation inventory—IMI. Results. The FME group patients showed significantly higher improvement in the speed metric (P < 0.01), and smoothness metric (P < 0.01), as well as higher RTT (P < 0.01). Significantly higher patient motivation is observed in the FME group (interest/enjoyment subscale (P < 0.01) and perceived competence subscale (P < 0.01)). Conclusion. Prolonged endurance in training and greater improvement in certain areas of motor function, as well as very high patient motivation and strong positive impressions about the treatment, suggest the positive effects of feedback-mediated treatment and its high level of acceptance by patients.

Influence of planar manipulandum to the hand trajectory during point to point movement

We present the analysis of the planar manipulandum effects to the trajectory of point to point movements in horizontal plane. This analysis is of significance for the control of a haptic robot that can be used for the rehabilitation of hemiplegic patients. The effects were assessed by comparing data collected in experiments with healthy subjects when performing simple movements that are used in the therapy of stroke patients. We found significant differences between the preferred trajectories and the deviations from the preferred trajectories (p<0.01) when moving with and without the manipulandum. This result suggests that for the design of the controller of a robot assistant inertial properties of the robot mechanism must be considered even in the case that it is used only for the assessment (passive) or within the bio-feedback.

Control of Prehension for the Transradial Prosthesis: Natural-like Image Recognition System

We describe the hardware and software for the control of prehension for a dexterous transradial prosthesis. The prehension process comprises hand orientation (three degrees of freedom) and the opening of the hand in a manner that is appropriate for the shape and size of the object. The hardware consists of a standard web camera, accelerometer, ultrasound distance sensor, laser pointer and an LED illumination system. Software operating in real time estimates the shape and size of the object as well as the relative orientation of the hand with respect to the object. Based on this data, the controller generates signals that are sent to the three-dimensional (3D) wrist rotator, and drives which control fingers and thumb of the transradial prosthesis, thereby preparing the hand for palmar, lateral, or precision (2-digit or 3-digit) grasps. The choice of the grasp follows heuristics captured from healthy humans when grasping and expressed in the form of IF-THEN rules.

A method for assessing the arm movement performance: probability tube

Quantification of motor performance is an important component of the rehabilitation of humans with sensory-motor disability. We developed a method for assessing arm movement performance of trainees (patients) termed “probability tube” (PT). PT captures the stochastic characteristics of a desired movement when repeated by an expert (therapist). The PT is being generated automatically from data recorded during point-to-point movement executed not more than 15 repetitions by the clinician and/or other non-expert programmer in just a few minutes. We introduce the index, termed probability tube score (PTS), as a single “goodness-of-fit” value allowing quantified analysis of the recovery and effects of the therapy. This index in fact scores the difference between the movement (velocity profile) executed by the trainee and the velocity profile of the desired movement (executed by the expert). We document the goodness of the automatic method with results from studies which included healthy subjects and show the use of the PTS in healthy and post-stroke hemiplegic subjects.

WiiMote control: Gaming feedback for motivational training of the arm movements

In this paper we present the method for WiiMote position control in order to gain motivational feedback in stroke patients who need to train their arm movements. Developed software acts as interface between input system, such as a computer mouse, which patients could use in rehabilitation, and custom made robotized system for controlling of WiiMote. With this software it is possible to transform any therapy prescribed movements into controls of WiiMote, so Nintendo Wii game could be played successfully.

Action representation of point to point movements: Classification with probability tube

Use of haptic robots in neurorehabilitation is gaining interest in the scientific community. One of the most favorable features of such systems is the possibility to integrate biomimetic features in the control. The control requires an action representation which is not available directly. We developed a method for arm movement action representation which considers expert movements as a stochastic process. This approach allows capturing of the variability of movement dynamics during skilled movement and later use of this action representation for the control of the haptic robot. We present here the method applied to the velocity space and introduce the probability tube as the tunnel where the width of the tunnel considers the variability.

Action representation for Wii bowling: Classification

We present the method for classifying kinematical data required for control of a rehabilitation robot for upper extremities. The classification to two cases (success, no-success) was analyzed by two methods: Bayes estimation and artificial neural network (ANN). The results are presented for an example being envisioned for rehabilitation: playing the Wii bowling with the specially constructed pantograph. The pantograph transforms the pointing-like movement into the appropriate motion of the WiiMote (hand held controller for Wii game); thereby, the user is playing Wii bowling with greatly simplified movement of the hand (range and speed) compared with normal play. The data analysis reduced the information to two key parameters for distinction of success vs. no-success: 1) maximal acceleration of WiiMote and 2) the acceleration of the WiiMote at the ball release time. The Bayes estimation resulted with 82% of correct classification, while the ANN reached the level of 90%.

The Robot that Learns from the Therapist How to Assist Stroke Patients

Results from clinical studies suggest that assisted training is beneficial for the recovery of functioning in patients with stroke and other central nervous system injuries. The training consists of the repetition of movements, which change the excitability of the brain, and due to cortical plasticity have carry-over effects. We are developing a 3D arm assistant that interfaces the patient at the hand/wrist. The development addresses three major issues: (1) the selection of the tasks that are appropriate for the training based on the level of motor abilities (2) the design of the visual feedback that enhances the motivation to train, and (3) the assessment of the performance. Therefore, our design integrates the new 3D robot assistant, various gaming based visual feedback, and software that acquires data on-line from sensors (position of the hand and force between the robot and the hand). The major novelties that the 3D arm assistant brings are the following: an automatic method of capturing movements presented by the therapist (expert), the use of the probabilistic movement representation for control of the robot, the incorporation of simple gaming with adjustable levels of difficulty, and finally, the assessment of differences between the achieved and target movements (kinematics) and interface force measured by a special handle with multiple sensors. The components of the new arm assistant in 2D have been tested and proved to work effectively in the clinical trials with stroke patients.