Qiushi Fu

Anticipatory planning and control of grasp positions and forces for dexterous two-digit manipulation

Dexterous object manipulation requires anticipatory control of digit positions and forces. Despite extensive studies on sensorimotor learning of digit forces, how humans learn to coordinate digit positions and forces has never been addressed. Furthermore, the functional role of anticipatory modulation of digit placement to object properties remains to be investigated. We addressed these questions by asking human subjects (12 females, 12 males) to grasp and lift an inverted T-shaped object using precision grip at constrained or self-chosen locations. The task requirement was to minimize object roll during lift. When digit position was not constrained, subjects could have implemented many equally valid digit position-force coordination patterns. However, choice of digit placement might also have resulted in large trial-to-trial variability of digit position, hence challenging the extent to which the CNS could have relied on sensorimotor memories for anticipatory control of digit forces. We hypothesized that subjects would modulate digit placement for optimal force distribution and digit forces as a function of variable digit positions. All subjects learned to minimize object roll within the first three trials, and the unconstrained device was associated with significantly smaller grip forces but larger variability of digit positions. Importantly, however, digit load force modulation compensated for position variability, thus ensuring consistent object roll minimization on each trial. This indicates that subjects learned object manipulation by integrating sensorimotor memories with sensory feedback about digit positions. These results are discussed in the context of motor equivalence and sensorimotor integration of grasp kinematics and kinetics.

Manipulation After Object Rotation Reveals Independent Sensorimotor Memory Representations of Digit Positions and Forces

Planning of object manipulations is dependent on the ability to generate, store, and retrieve sensorimotor memories of previous actions associated with grasped objects. However, the sensorimotor memory representations linking object properties to the planning of grasp are not well understood. Here we use an object rotation task to gain insight into the mechanisms underlying the nature of these sensorimotor memories. We asked subjects to grasp a grip device with an asymmetrical center of mass (CM) anywhere on its vertical surfaces and lift it while minimizing object roll. After subjects learned to minimize object roll by generating a compensatory moment, they were asked to rotate the object 180 degrees about a vertical axis and lift it again. The rotation resulted in changing the direction of external moment opposite to that experienced during the prerotation block. Anticipatory grasp control was quantified by measuring the compensatory moment generated at object lift onset by thumb and index finger forces through their respective application points. On the first postrotation trial, subjects failed to generate a compensatory moment to counter the external moment caused by the new CM location, thus resulting in a large object roll. Nevertheless, after several object rotations subjects reduced object roll on the initial postrotation trials by anticipating the new CM location through the modulation of digit placement but not tangential forces. The differential improvement in modulating these two variables supports the notion of independent memory representations of kinematics and kinetics and is discussed in relation to neural mechanisms underlying visuomotor transformations.

Transfer of Learned Manipulation following Changes in Degrees of Freedom

The present study was designed to determine whether manipulation learned with a set of digits can be transferred to grips involving a different number of digits, and possible mechanisms underlying such transfer. The goal of the task was to exert a torque and vertical forces on a visually symmetrical object at object lift onset to balance the external torque caused by asymmetrical mass distribution. Subjects learned this manipulation through consecutive practice using one grip type (two or three digits), after which they performed the same task but with another grip type (e.g., after adding or removing one digit, respectively). Subjects were able to switch grip type without compromising the behavioral outcome (i.e., the direction, timing, and magnitude of the torque exerted on the object was unchanged), despite the use of significantly different digit force-position coordination patterns in the two grip types. Our results support the transfer of learning for anticipatory control of manipulation and indicate that the CNS forms an internal model of the manipulation task independent of the effectors that are used to learn it. We propose that sensory information about the new digit placement—resulting from adding or removing a digit immediately after the switch in grip type—plays an important role in the accurate modulation of new digit force distributions. We discuss our results in relation to studies of manipulation reporting lack of learning transfer and propose a theoretical framework that accounts for failure or success of motor learning generalization.

Learned manipulation at unconstrained contacts does not transfer across hands.

Recent studies about sensorimotor control of the human hand have focused on how dexterous manipulation is learned and generalized. Here we address this question by testing the extent to which learned manipulation can be transferred when the contralateral hand is used and/or object orientation is reversed. We asked subjects to use a precision grip to lift a grip device with an asymmetrical mass distribution while minimizing object roll during lifting by generating a compensatory torque. Subjects were allowed to grasp anywhere on the object’s vertical surfaces, and were therefore able to modulate both digit positions and forces. After every block of eight trials performed in one manipulation context (i.e., using the right hand and at a given object orientation), subjects had to lift the same object in the second context for one trial (transfer trial). Context changes were made by asking subjects to switch the hand used to lift the object and/or rotate the object 180° about a vertical axis. Therefore, three transfer conditions, hand switch (HS), object rotation (OR), and both hand switch and object rotation (HS+OR), were tested and compared with hand matched control groups who did not experience context changes. We found that subjects in all transfer conditions adapted digit positions across multiple transfer trials similar to the learning of control groups, regardless of different changes of contexts. Moreover, subjects in both HS and HS+OR group also adapted digit forces similar to the control group, suggesting independent learning of the left hand. In contrast, the OR group showed significant negative transfer of the compensatory torque due to an inability to adapt digit forces. Our results indicate that internal representations of dexterous manipulation tasks may be primarily built through the hand used for learning and cannot be transferred across hands.

Tracking whole hand kinematics using extended Kalman filter

This paper describes the general procedure, model construction, and experimental results of tracking whole hand kinematics using extended Kalman filter (EKF) based on data recorded from active surface markers. We used a hand model with 29 degrees of freedom that consists of hand global posture, wrist, and digits. The marker protocol had 4 markers on the distal forearm and 20 markers on the dorsal surface of the joints of the digits. To reduce computational load, we divided the state space into four sub-spaces, each of which were estimated with an EKF in a specific order. We tested our framework and found reasonably accurate results (2–4 mm tip position error) when sampling tip to tip pinch at 120 Hz.

Retention and interference of learned dexterous manipulation: interaction between multiple sensorimotor processes

An object can be used in multiple contexts, each requiring different hand actions. How the central nervous system builds and maintains memory of such dexterous manipulations remains unclear. We conducted experiments in which human subjects had to learn and recall manipulations performed in two contexts, A and B. Both contexts involved lifting the same L-shaped object whose geometry cued its asymmetrical mass distribution. Correct performance required producing a torque on the vertical handle at object lift onset to prevent it from tilting. The torque direction depended on the context, i.e., object orientation, which was changed by 180° object rotation about a vertical axis. With an A1B1A2 context switching paradigm, subjects learned A1 in the first block of eight trials as indicated by a torque approaching the required one. However, subjects made large errors in anticipating the required torque when switching to B1 immediately after A1 (negative transfer), as well as when they had to recall A1 when switching to A2 after learning B through another block of eight lifts (retrieval interference). Classic sensorimotor learning theories attribute such interferences to multi-rate, multi-state error-driven updates of internal models. However, by systematically changing the interblock break duration and within-block number of trials, our results suggest an alternative explanation underlying interference and retention of dexterous manipulation. Specifically, we identified and quantified through a novel computational model the nonlinear interaction between two sensorimotor mechanisms: a short-lived, context-independent, use-dependent sensorimotor memory and a context-sensitive, error-based learning process.

Extraction of Time and Frequency Features From Grip Force Rates During Dexterous Manipulation

The time course of grip force from object contact to onset of manipulation has been extensively studied to gain insight into the underlying control mechanisms. Of particular interest to the motor neuroscience and clinical communities is the phenomenon of bell-shaped grip force rate (GFR) that has been interpreted as indicative of feedforward force control. However, this feature has not been assessed quantitatively. Furthermore, the time course of grip force may contain additional features that could provide insight into sensorimotor control processes. In this study, we addressed these questions by validating and applying two computational approaches to extract features from GFR in humans: 1) fitting a Gaussian function to GFR and quantifying the goodness of the fit [root-mean-square error, (RMSE)]; and 2) continuous wavelet transform (CWT), where we assessed the correlation of the GFR signal with a Mexican Hat function. Experiment 1 consisted of a classic pseudorandomized presentation of object mass (light or heavy), where grip forces developed to lift a mass heavier than expected are known to exhibit corrective responses. For Experiment 2, we applied our two techniques to analyze grip force exerted for manipulating an inverted T-shaped object whose center of mass was changed across blocks of consecutive trials. For both experiments, subjects were asked to grasp the object at either predetermined or self-selected grasp locations (“constrained” and “unconstrained” task, respectively). Experiment 1 successfully validated the use of RMSE and CWT as they correctly distinguished trials with versus without force corrective responses. RMSE and CWT also revealed that grip force is characterized by more feedback-driven corrections when grasping at self-selected contact points. Future work will examine the application of our analytical approaches to a broader range of tasks, e.g., assessment of recovery of sensorimotor function following clinical intervention, interlimb differences in force control, and force coordination in human-machine interactions.

Cable-driven elastic parallel humanoid head with face tracking for Autism Spectrum Disorder interventions

This paper presents the development of new prismatic actuation approach and its application in human-safe humanoid head design. To reduce actuator output impedance and mitigate unexpected external shock, the prismatic actuation method uses cables to drive a piston with preloaded spring. By leveraging the advantages of parallel manipulator and cable-driven mechanism, the developed neck has a parallel manipulator embodiment with two cable-driven limbs embedded with preloaded springs and one passive limb. The eye mechanism is adapted for low-cost webcam with succinct “ball-in-socket” structure. Based on human head anatomy and biomimetics, the neck has 3 degree of freedom (DOF) motion: pan, tilt and one decoupled roll while each eye has independent pan and synchronous tilt motion (3 DOF eyes). A Kalman filter based face tracking algorithm is implemented to interact with the human. This neck and eye structure is translatable to other human-safe humanoid robots. The robots appearance reflects a non-threatening image of a penguin, which can be translated into a possible therapeutic intervention for children with Autism Spectrum Disorders.

Towards a complete description of grasping kinematics: A framework for quantifying human grasping and manipulation

This paper proposes a framework for tracking both human hand kinematics and object contour during grasping task. The framework is based on modeling the object as point cloud and the use of marker-based tracking. We introduce how to estimate contact sites on both the hand and object, hand enclosing space, and graspable features from recorded data. Two experiments were performed to 1) verify the accuracy of contact site estimation (less than 5 mm), and 2) validate the feature extraction. Our approach can provide significant insight into how humans plan grasping and manipulation based on object recognition.

Human reach-to-grasp compensation with object pose uncertainty

This paper examined how humans alter reach-to-grasp behavior to compensate for environmentally-induced object orientation uncertainty. We used a novel motion tracking framework to capture hand-object interactions, as well as a custom cylindrical object to detect contacts. Subjects were instructed to reach, grasp, and lift the object with or without vision. The orientation of the object was randomly changed on each trial. We hypothesized subjects would use a reach-to-grasp strategy that minimizes post-contact adjustments. However, our results indicate that (1) subjects are more likely to use the hand as a sensing apparatus prior to contact, and (2) the reach-to-grasp kinematics may be optimized for efficient sensing of object orientation. Our findings could provide potential solution to efficient tactile sensing for robotic hand in unstructured environment.