Felix C. Huang

Visual and haptic feedback contribute to tuning and online control during object manipulation.

The authors employed a virtual environment to investigate how humans use haptic and visual feedback in a simple, rhythmic object-manipulation task. The authors hypothesized that feedback would help participants identify the appropriate resonant frequency and perform online control adjustments. The 1st test was whether sensory feedback is needed at all; the 2nd was whether the motor system combines visual and haptic feedback to improve performance. Task performance was quantified in terms of work performed on the virtual inertia, ability to identify the correct rhythm, and variability of movement. Strict feedforward control was found to be ineffective for this task, even when participants had previous knowledge of the rhythm. Participants (N = 11) performed far better when feedback was available (11 times more work, 2.2 times more precise frequency, 30% less variability; p < .05 for all 3 performance measures). Using sensory feedback, participants were able to rapidly identify 4 different spring-inertia systems without foreknowledge of the corresponding resonant frequencies. They performed over 20% more work with 24% less variability when provided with both visual and haptic feedback than they did with either feedback channel alone (p < .05), providing evidence that they integrated online sensory channels. Whereas feedforward control alone led to poor performance, feedback control led to fast tuning or calibration of control according to the resonant frequency of the object, and to better control of the rhythmic movement itself.

Manual Skill Generalization Enhanced by Negative Viscosity

Recent human-machine interaction studies have suggested that movement augmented with negative viscosity can enhance performance and can even promote better motor learning. To test this, we investigated how negative viscosity influences motor adaptation to an environment where forces acted only in one axis of motion. Using a force-feedback device, subjects performed free exploratory movements with a purely inertia generating forces proportional to hand acceleration, negative viscosity generating destabilizing forces proportional to hand velocity, or a combination of the acceleration and velocity fields. After training, we evaluated each subjects ability to perform circular movements in only the inertial field. Combined training resulted in lowest error and revealed similar responses as inertia training in catch trials. These findings are remarkable because negative viscosity, available only during training, evidently enhanced learning when combined with inertia. This success in generalization is consistent with the ability of the nervous system to decompose the perturbing forces into velocity and acceleration dependent components. Compared with inertia, the combined group exhibited a broader range of speeds along the direction of maximal perturbing force. Broader exploration was also correlated with better performance in subsequent evaluation trials; this suggests that negative viscosity improved performance by enhancing identification of each force field. These findings shed light on a new way to enhance sensorimotor adaptation through robot-applied augmentation of mechanics.

Haptic feedback and human performance in a dynamic task

This study explores the effects of haptic feedback on performance and learning by human subjects executing a dynamic task. We present the results of experiments involving the control of a ball and beam. Human subjects perform position targeting of the ball through hand operation of the beam angle. In our dynamic analysis we discuss how this prototype task may be used to test the efficacy of various haptic feedback conditions. We obtain results for two conditions of haptic feedback, produced using two ball sizes, and apply various metrics to analyze performance. We also examine ordering effects that occur in the presentation of these haptic conditions. Our analysis and experimental findings indicate that the performance of a dynamic task is governed by the complexity of system dynamics and the magnitude of haptic feedback. Our results provide modest support to recommend exposure to a more complex, higher force-feedback task prior to the execution of a simpler lower feedback task.

Augmented Dynamics and Motor Exploration as Training for Stroke

With chronic stroke survivors (n = 30), we investigated how upper extremity training with negative viscosity affects coordination under unperturbed conditions. Subjects trained with a planar robotic interface simulating 1) negative viscosity augmented to elbow and shoulder joints; 2) negative viscosity combined with inertia; or 3) a null-field condition. Two treatment groups practiced with both force conditions (cross-over design), while a control group practiced with a null-field condition. Training (exploratory movement) and evaluations (prescribed circular movement) alternated in several phases to facilitate transfer from forces to the null field. Negative viscosity expanded exploration especially in the sagittal axis, and resulted in significant within-day improvements. Both treatment groups exhibited next day retention unobserved in the control. Our results suggest enhanced learning from forces that induce a broader range of kinematics. This study supports the use of robot-assisted training that encourages active patient involvement by preserving efferent commands for driving movement.

Human Adaptation to Interaction Forces in Visuo-Motor Coordination

We tested whether humans can learn to sense and compensate for interaction forces in contact tasks. Many tasks, such as use of hand tools, involve significant interaction forces between hand and environment. One control strategy would be to use high hand impedance to reduce sensitivity to these forces. But an alternative would be to learn feedback compensation for the extrinsic dynamics and associated interaction forces, with the potential for lower control effort. We observed subjects as they learned control of a ball-and-beam system, a visuo-motor task where the goal was to quickly position a ball rolling atop a rotating beam, through manual rotation of the beam alone. We devised a ball-and-beam apparatus that could be operated in a real mode, where a physical ball was present; or in a virtual training mode, where the balls dynamics were simulated in real time. The apparatus presented the same visual feedback in all cases, and optionally produced haptic feedback of the interaction forces associated with the balls motion. Two healthy adult subject groups, vision-only and vision-haptics (each n=10), both trained for 80 trials on the simulated system, and then were evaluated on the real system to test for skill transfer effects. If humans incorporate interaction forces in their learning, the vision-haptics group would be expected to exhibit a smoother transfer, as quantified by changes in completion time of a ball-positioning task. During training, both groups adapted well to the task, with reductions of 64%-70% in completion time. At skill transfer to the real system, the vision-only group had a significant 35% increase in completion time (p<0.05). There was no significant change in the vision-haptics group, indicating that subjects had learned to compensate for interaction forces. These forces could potentially be incorporated in virtual environments to assist with motor training or rehabilitation

Perception of stiffness in laparoscopy - the fulcrum effect.

We explored how the perception of stiffness can be distorted in Minimally Invasive Surgery. We combined a mechanical simulator with a haptic device, and implemented linear springs at the tip of the simulated laparoscopic device. To explore the influence of mechanical advantage on perception, we set different values of the ratio between internal and external length of the tool. We found that a nonsymmetrical ratio causes bias in the perceived stiffness when novice tangential probing is compared to radial probing. In contrast, haptic experts did not show similar perceptual bias.

Evaluation of negative viscosity as upper extremity training for stroke survivors

With stroke survivors (n=30) as the test population, we investigated how upper extremity training with negative viscosity affects coordination in unassisted conditions. Using a planar force-feedback device, subjects performed exploratory movements within an environment that simulated 1) negative viscosity added to elbow and shoulder joints 2) augmented inertia to the upper and lower arm combined with negative viscosity, or 3) a null force field (control). After training, we evaluated each subjects ability to perform circular movements in the null field. Negative viscosity training resulted in greater within-day reductions in error compared with the combined field training. Negative viscosity promoted greater distributions of accelerations during free exploration, especially in the sagittal axis, while combined field training diminished overall activity. Both force field training groups exhibited next day retention, while this was not observed for the control group. The improvement in performance suggests that greater range of kinematic experiences contribute to learning, even despite novel force field environments. These findings provide support for the use of movement amplifying environments for upper extremity rehabilitation, allowing greater access to training while maintaining user engagement.

Movement distributions of stroke survivors exhibit distinct patterns that evolve with training

BackgroundWhile clinical assessments provide tools for characterizing abilities in motor-impaired individuals, concerns remain over their repeatability and reliability. Typical robot-assisted training studies focus on repetition of prescribed actions, yet such movement data provides an incomplete account of abnormal patterns of coordination. Recent studies have shown positive effects from self-directed movement, yet such a training paradigm leads to challenges in how to quantify and interpret performance.MethodsWith data from chronic stroke survivors (n = 10, practicing for 3 days), we tabulated histograms of the displacement, velocity, and acceleration for planar motion, and examined whether modeling of distributions could reveal changes in available movement patterns. We contrasted these results with scalar measures of the range of motion. We performed linear discriminant analysis (LDA) classification with selected histogram features to compare predictions versus actual subject identifiers. As a basis of comparison, we also present an age-matched control group of healthy individuals (n = 10, practicing for 1 day).ResultsAnalysis of range of motion did not show improvement from self-directed movement training for the stroke survivors in this study. However, examination of distributions indicated that increased multivariate normal components were needed to accurately model the patterns of movement after training. Stroke survivors generally exhibited more complex distributions of motor exploration compared to the age-matched control group. Classification using linear discriminant analysis revealed that movement patterns were identifiable by individual. Individuals in the control group were more difficult to identify using classification methods, consistent with the idea that motor deficits contribute significantly to unique movement signatures.ConclusionsDistribution analysis revealed individual patterns of abnormal coordination in stroke survivors and changes in these patterns with training. These findings were not apparent from scalar metrics that simply summarized properties of motor exploration. Our results suggest new methods for characterizing motor capabilities, and could provide the basis for powerful tools for designing customized therapy.

Evidence of multiple coordinate representations during generalization of motor learning

Abstract Several studies have suggested that the motor system takes advantage of a coordinate system when learning a novel sensorimotor environment. Such investigations, however, have not distinguished between initial preferences of a coordinate system versus possible changes due to learning. Here, we present experimental methods that specifically entertain the possibility of multiple coordinate systems during generalization. Subjects trained with their right arm on a viscous force field. We evaluated their performances for both arms in an untrained workspace before and after training using three fields, each representing extrapolation with a candidate coordinate system. Surprisingly, our results showed evidence of improvement (pre to post) in all fields for both limbs. These findings are consistent with the hypothesis of multiple, simultaneous coordinate systems involved in generalization. We also investigated how feedback might affect the results and found in several cases that performance was better for visual displays that were aligned with the limb (in first person) versus non-aligned.

Simultaneous coordinate representations are influenced by visual feedback in a motor learning task

It has been widely accepted that the CNS develops a representation (model) of the environment, but what is not entirely clear is the coordinate reference frame used. We explored how visual feedback influenced the coordinate frame in which the CNS stores and recalls these memories of learned skills in a reaching-generalization task. Four groups of subjects trained to perform reaching movements in a perturbing force field, two with aligned (first person) visual feedback and two with non-aligned (vertical screen). After 170 trials of practice, we asked subjects to extrapolate (generalize) what they learned to a new part of the workspace in novel force environments (endpoint-based versus joint-based extrapolated force fields). Regardless of the test condition, all subjects improved their ability to generalize skills to the new workspace. There was evidence that the extrapolation of their learned skills was based on both object-centered and joint-based coordinates. Consistent with previous studies, subjects performed significantly better in joint-extrapolated force field, but only if the visual feedback was vertical. Subjects performed equivalently in both force fields with aligned (first person) feedback. These findings suggest that the type of visual feedback biases the way subjects perform, and that learning results can be significantly influenced by feedback.