Hendrik Reimann

A Neural Dynamic Architecture for Reaching and Grasping Integrates Perception and Movement Generation and Enables On-Line Updating

Reaching for objects and grasping them is a fundamental skill for any autonomous robot that interacts with its environment. Although this skill seems trivial to adults, who effortlessly pick up even objects they have never seen before, it is hard for other animals, for human infants, and for most autonomous robots. Any time during movement preparation and execution, human reaching movement are updated if the visual scene changes (with a delay of about 100 ms). The capability for online updating highlights how tightly perception, movement planning, and movement generation are integrated in humans. Here, we report on an effort to reproduce this tight integration in a neural dynamic process model of reaching and grasping that covers the complete path from visual perception to movement generation within a unified modeling framework, Dynamic Field Theory. All requisite processes are realized as time-continuous dynamical systems that model the evolution in time of neural population activation. Population level neural processes bring about the attentional selection of objects, the estimation of object shape and pose, and the mapping of pose parameters to suitable movement parameters. Once a target object has been selected, its pose parameters couple into the neural dynamics of movement generation so that changes of pose are propagated through the architecture to update the performed movement online. Implementing the neural architecture on an anthropomorphic robot arm equipped with a Kinect sensor, we evaluate the model by grasping wooden objects. Their size, shape, and pose are estimated from a neural model of scene perception that is based on feature fields. The sequential organization of a reach and grasp act emerges from a sequence of dynamic instabilities within a neural dynamics of behavioral organization, that effectively switches the neural controllers from one phase of the action to the next. Trajectory formation itself is driven by a dynamical systems version of the potential field approach. We highlight the emergent capacity for online updating by showing that a shift or rotation of the object during the reaching phase leads to the online adaptation of the movement plan and successful completion of the grasp.

A neural dynamics architecture for grasping that integrates perception and movement generation and enables on-line updating.

We present a neural dynamics architecture for grasping that integrates perceptual processes of scene exploration, object selection and classification, and grasp pose estimation with motor processes such as planning and controlling reach and grasp movements. Inspired by theories of human embodied cognition, the entire architecture is essentially one big dynamical system from which discrete events such as initiating and terminating reaches and grasps emerge through dynamical instabilities. Using a Kinect sensor as input, we implement the architecture on a Kuka light weight arm with a Schunk Dextrous Hand and demonstrate grasping movements that are updated on-line when the object is shifted or rotated during movement planning or execution.

Complementary mechanisms for upright balance during walking

Lateral balance is a critical factor in keeping the human body upright during walking. Two important mechanisms for balance control are the stepping strategy, in which the foot placement is changed in the direction of a sensed fall to modulate how the gravitational force acts on the body, and the lateral ankle strategy, in which the body mass is actively accelerated by an ankle torque. Currently, there is minimal evidence about how these two strategies complement one another to achieve upright balance during locomotion. We use Galvanic vestibular stimulation (GVS) to induce the sensation of a fall at heel-off during gait initiation. We found that young healthy adults respond to the illusory fall using both the lateral ankle strategy and the stepping strategy. The stance foot center of pressure (CoP) is shifted in the direction of the perceived fall by ≈2.5 mm, starting ≈247 ms after stimulus onset. The foot placement of the following step is shifted by ≈15 mm in the same direction. The temporal delay between these two mechanisms suggests that they independently contribute to upright balance during locomotion, potentially in a serially coordinated manner. Modeling results indicate that without the lateral ankle strategy, a much larger step width is required to maintain upright balance, suggesting that the small but early CoP shift induced by the lateral ankle strategy is critical for upright stability during locomotion. The relative importance of each mechanism and how neurological disorders may affect their implementation remain an open question.

Coordination of muscle torques stabilizes upright standing posture: an UCM analysis.

The control of upright stance is commonly explained on the basis of the single inverted pendulum model (ankle strategy) or the double inverted pendulum model (combination of ankle and hip strategy). Kinematic analysis using the uncontrolled manifold (UCM) approach suggests, however, that stability in upright standing results from coordinated movement of multiple joints. This is based on evidence that postural sway induces more variance in joint configurations that leave the body position in space invariant than in joint configurations that move the body in space. But does this UCM structure of kinematic variance truly reflect coordination at the level of the neural control strategy or could it result from passive biomechanical factors? To address this question, we applied the UCM approach at the level of muscle torques rather than joint angles. Participants stood on the floor or on a narrow base of support. We estimated torques at the ankle, knee, and hip joints using a model of the body dynamics. We then partitioned the joint torques into contributions from net, motion-dependent, gravitational, and generalized muscle torques. A UCM analysis of the structure of variance of the muscle torque revealed that postural sway induced substantially more variance in directions in muscle torque space that leave the Center of Mass (COM) force invariant than in directions that affect the force acting on the COM. This difference decreased when we decorrelated the muscle torque data by randomizing across time. Our findings show that the UCM structure of variance exists at the level of muscle torques and is thus not merely a by-product of biomechanical coupling. Because muscle torques reflect neural control signals more directly than joint angles do, our results suggest that the control strategy for upright stance involves the task-specific coordination of multiple degrees of freedom.

Development of infant leg coordination: Exploiting passive torques

Leg joint coordination systematically changes over the first months of life, yet there is minimal data on the underlying change in muscle torques that might account for this change in coordination. The purpose of this study is to investigate the contribution of torque changes to early changes in leg joint coordination. Kicking actions were analyzed of 10 full-term infants between 6 and 15-weeks of age using three-dimensional kinematics and kinetics. We found 11 of 15 joint angle pairs demonstrated a change from more in-phase intralimb coordination at 6-weeks to less in-phase coordination at 15-weeks. Although the magnitude of joint torques normalized to the mass of the leg remained relatively consistent, we noted more complex patterns of torque component contribution across ages. By focusing on the change in torques associated with hip-knee joint coordination, we found that less in-phase hip-knee joint coordination at 15-weeks was associated with decreased influence of knee muscle torque and increased influence of knee gravitational and motion-dependent torques, supporting that infants coordinate hip muscle torque with passive knee gravitational and motion-dependent torques to generate kicks with reduced active knee muscle torque. We propose that between 6 and 15-weeks of age less in-phase hip-knee coordination emerges as infants exploit passive dynamics in the coordination of hip and knee motions.

Carry-over coarticulation in joint angles

AbstractCoarticulation indicates a dependence of a movement segment on a preceding segment (carry-over coarticulation) or on the segment that follows (anticipatory coarticulation). Here we study coarticulation in multidegrees of freedom human arm movements. We asked participants to transport a cylinder from a starting position to a center target and on to a final target. In this naturalistic setting, the human arm has ten degrees of freedom and is thus comfortably redundant for the task. We studied coarticulation by comparing movements between the same spatial locations that were either preceded by different end-effector paths (carry-over coarticulation) or followed by different end-effector paths (anticipatory coarticulation). We found no evidence for coarticulation at the level of the end-effector. We found very clear evidence, however, for carry-over, not for anticipatory coarticulation at the joint level. We used the concept of the uncontrolled manifold to systematically establish coarticulation as a form of motor equivalence, in which most of the difference between different movement contexts lies within the uncontrolled manifold that leaves the end-effector invariant. The findings are consistent with movement planning occurring at the level of the end-effector, and those movement plans being transformed to the joint level by a form of inverse kinematics. The observation of massive self-motion excludes an account that is solely based on a kinematic pseudo-inverse.

Spatiotemporal gait changes with use of an arm swing cueing device in people with Parkinson’s disease

Impaired arm swing is a common motor symptom of Parkinsons disease (PD), and correlates with other gait impairments and increased risk of falls. Studies suggest that arm swing is not merely a passive consequence of trunk rotation during walking, but an active component of gait. Thus, techniques to enhance arm swing may improve gait characteristics. There is currently no portable device to measure arm swing and deliver immediate cues for larger movement. Here we test report pilot testing of such a device, ArmSense (patented), using a crossover repeated-measures design. Twelve people with PD walked in a video-recorded gym space at self-selected comfortable and fast speeds. After baseline, cues were given either visually using taped targets on the floor to increase step length or through vibrations at the wrist using ArmSense to increase arm swing amplitude. Uncued walking then followed, to assess retention. Subjects successfully reached cueing targets on >95% of steps. At a comfortable pace, step length increased during both visual cueing and ArmSense cueing. However, we observed increased medial-lateral trunk sway with visual cueing, possibly suggesting decreased gait stability. In contrast, no statistically significant changes in trunk sway were observed with ArmSense cues compared to baseline walking. At a fast pace, changes in gait parameters were less systematic. Even though ArmSense cues only specified changes in arm swing amplitude, we observed changes in multiple gait parameters, reflecting the active role arm swing plays in gait and suggesting a new therapeutic path to improve mobility in people with PD.

Neural Control of Balance During Walking

Neural control of standing balance has been extensively studied. However, most falls occur during walking rather than standing, and findings from standing balance research do not necessarily carry over to walking. This is primarily due to the constraints of the gait cycle: Body configuration changes dramatically over the gait cycle, necessitating different responses as this configuration changes. Notably, certain responses can only be initiated at specific points in the gait cycle, leading to onset times ranging from 350 to 600 ms, much longer than what is observed during standing (50–200 ms). Here, we investigated the neural control of upright balance during walking. Specifically, how the brain transforms sensory information related to upright balance into corrective motor responses. We used visual disturbances of 20 healthy young subjects walking in a virtual reality cave to induce the perception of a fall to the side and analyzed the muscular responses, changes in ground reaction forces and body kinematics. Our results showed changes in swing leg foot placement and stance leg ankle roll that accelerate the body in the direction opposite of the visually induced fall stimulus, consistent with previous results. Surprisingly, ankle musculature activity changed rapidly in response to the stimulus, suggesting the presence of a direct reflexive pathway from the visual system to the spinal cord, similar to the vestibulospinal pathway. We also observed systematic modulation of the ankle push-off, indicating the discovery of a previously unobserved balance mechanism. Such modulation has implications not only for balance but plays a role in modulation of step width and length as well as cadence. These results indicated a temporally-coordinated series of balance responses over the gait cycle that insures flexible control of upright balance during walking.

Infant intralimb coordination and torque production: Influence of prematurity

The purpose of this study is to investigate changes in leg joint coordination, intersegmental dynamics, and their relation in infants born preterm (PT) during the first months of life. Kicking actions were analyzed of 11 infants born PT at 6 and 15-weeks corrected age (CA) using three-dimensional kinematics and kinetics; results were compared to the kicking actions of 10 infants born full-term (FT). Both groups changed from a predominately in-phase coordination at 6-weeks CA to a less in-phase coordination at 15-weeks CA, however, at 6-weeks CA, infants born PT demonstrated less in-phase coordination of their ankle joints with their hip and knee joints. Between groups and across ages, both groups demonstrated consistent net and partitioned joint torque profiles, however, at 6-weeks CA infants born PT demonstrated more complex patterns of torque components. In both groups, less in-phase hip-knee coordination was associated with reduced active knee muscle torque and increased passive knee torques, however, passive knee torques had a greater influence on the kicks of infants born PT at 6-weeks CA. At 6-weeks CA, infants born PT, compared to FT, generated kicks with less in-phase hip-knee coordination, hip excursion, hip angular velocity, and hip muscle torque impulse. By 15-weeks CA, differences resolved in all variables except hip muscle torque impulse. These results highlight a different trajectory of leg joint coordination and torque production for infants born PT compared to FT.