Jason P. Gallivan

Decoding Effector-Dependent and Effector-Independent Movement Intentions from Human Parieto-Frontal Brain Activity

Our present understanding of the neural mechanisms and sensorimotor transformations that govern the planning of arm and eye movements predominantly come from invasive parieto-frontal neural recordings in nonhuman primates. While functional MRI (fMRI) has motivated investigations on much of these same issues in humans, the highly distributed and multiplexed organization of parieto-frontal neurons necessarily constrain the types of intention-related signals that can be detected with traditional fMRI analysis techniques. Here we employed multivoxel pattern analysis (MVPA), a multivariate technique sensitive to spatially distributed fMRI patterns, to provide a more detailed understanding of how hand and eye movement plans are coded in human parieto-frontal cortex. Subjects performed an event-related delayed movement task requiring that a reach or saccade be planned and executed toward one of two spatial target positions. We show with MVPA that, even in the absence of signal amplitude differences, the fMRI spatial activity patterns preceding movement onset are predictive of upcoming reaches and saccades and their intended directions. Within certain parieto-frontal regions we show that these predictive activity patterns reflect a similar spatial target representation for the hand and eye. Within some of the same regions, we further demonstrate that these preparatory spatial signals can be discriminated from nonspatial, effector-specific signals. In contrast to the largely graded effector- and direction-related planning responses found with fMRI subtraction methods, these results reveal considerable consensus with the parieto-frontal network organization suggested from primate neurophysiology and specifically show how predictive spatial and nonspatial movement information coexists within single human parieto-frontal areas.

Reaching for the unknown: Multiple target encoding and real-time decision-making in a rapid reach task

Decision-making is central to human cognition. Fundamental to every decision is the ability to internally represent the available choices and their relative costs and benefits. The most basic and frequent decisions we make occur as our motor system chooses and executes only those actions that achieve our current goals. Although these interactions with the environment may appear effortless, this belies what must be incredibly sophisticated visuomotor decision-making processes. In order to measure how visuomotor decisions unfold in real-time, we used a unique reaching paradigm that forced participants to initiate rapid hand movements toward multiple potential targets, with only one being cued after reach onset. We show across three experiments that, in cases of target uncertainty, trajectories are spatially sensitive to the probabilistic distribution of targets within the display. Specifically, when presented with two or three target displays, subjects initiate their reaches toward an intermediary or averaged location before correcting their trajectory in-flight to the cued target location. A control experiment suggests that our effect depends on the targets acting as potential reach locations and not as distractors. This study is the first to show that the averaging of target-directed reaching movements depends not only on the spatial position of the targets in the display but also the probability of acting at each target location.

fMRI Repetition Suppression for Familiar But Not Arbitrary Actions with Tools

For humans, daily life is characterized by routine interaction with many different tools for which corresponding actions are specified and performed according to well-learned procedures. The current study used functional MRI (fMRI) repetition suppression (RS) to identify brain areas underlying the transformation of visually defined tool properties to corresponding motor programs for conventional use. Before grasping and demonstrating how to use a specific tool, participants passively viewed either the same (repeated) tool or a different (non-repeated) tool. Repetition of tools led to reduced fMRI signals (RS) within a selective network of parietal and premotor areas. Comparison with newly learned, arbitrarily defined control actions revealed specificity of RS for tool use, thought to reflect differences in the extent of previous sensorimotor experience. The findings indicate that familiar tools are visually represented within the same sensorimotor areas underlying their dexterous use according to learned properties defined by previous experience. This interpretation resonates with the broader concept of affordance specification considered fundamental to action planning and execution whereby action-relevant object properties (affordances) are visually represented in sensorimotor areas. The current findings extend this view to reveal that affordance specification in humans includes learned object properties defined by previous sensorimotor experience. From an evolutionary perspective, the neural mechanisms identified in the current study offer clear survival advantage, providing fast efficient transformation of visual information to appropriate motor responses based on previous experience.

To use or to move: goal-set modulates priming when grasping real tools

How we interact with objects depends on what we intend to do with them. In the current work, we show that priming and the kinematics of grasping depend on the goals of grasping, as well as the context in which tasks are presented. We asked participants to grasp familiar kitchen tools in order to either move them, grasp-to-move (GTM), or to demonstrate their common use, grasp-to-use (GTU). When tasks were blocked separately (Experiment 1), we found that priming was only evident for the GTU task. However, when tasks were presented in the same block of trials (Experiment 2), we observed priming for both tasks. Independent of priming, differences in kinematics and reaction times according to task were evident for both Experiments. Longer reaction times for the GTU task indicate more extensive planning, and differences in grasping reflect the characteristics of subsequent actions. Priming of real grasping is determined by task goals as well as task setting, both of which are likely to modulate how object features (affordances) are perceived and influence the planning of future actions.

Mental blocks: fMRI reveals top-down modulation of early visual cortex when obstacles interfere with grasp planning.

When grasping an object, the fingers, hand and arm rarely collide with other non-target objects in the workspace. Kinematic studies of neurological patients (Schindler et al., 2004) and healthy participants (Chapman and Goodale, 2010a) suggest that the location of potential obstacles and the degree of interference they pose are encoded by the dorsal visual stream during action planning. Here, we used a slow event-related paradigm in functional magnetic resonance imaging (fMRI) to examine the neural encoding of obstacles in normal participants. Fifteen right-handed participants grasped a square target object with a thumb-front or thumb-side wrist-posture with (1) no obstacle present, (2) an obstacle behind the target object (interfering with the thumb-front grasp), or (3) an obstacle beside the target object (interfering with the thumb-side grasp). Within a specified network of areas involved in planning, a group voxelwise analysis revealed that one area in the left posterior intraparietal sulcus (pIPS) and one in early visual cortex were modulated by the degree of obstacle interference, and that this modulation occurred prior to movement execution. Given previous reports of a functional link between IPS and early visual cortex, we suggest that the increasing activity in the IPS with obstacle interference provides the top-down signal to suppress the corresponding obstacle coding in early visual areas, where we observed that activity decreased with interference. This is the first concrete evidence that the planning of a grasping movement can modulate early visual cortex and provides a unifying framework for understanding the dual role played by the IPS in motor planning and attentional orienting.

Visual salience dominates early visuomotor competition in reaching behavior

In this study, we investigated whether visual salience influences the competition between potential targets during reach planning. Participants initiated rapid pointing movements toward multiple potential targets, with the final target being cued only after the reach was initiated. We manipulated visual salience by varying the luminance of potential targets. Across two separate experiments, we demonstrate that initial reach trajectories are directed toward more salient targets, even when there are twice as many targets (and therefore twice the likelihood of the final target appearing) on the opposite side of space. We also show that this salience bias is time-dependent, as evidenced by the return of spatially averaged reach trajectories when participants were given an additional 500-ms preview of the target display prior to the cue to move. This study shows both when and to what extent task-irrelevant luminance differences affect the planning of reaches to multiple potential targets.

Neuroimaging reveals enhanced activation in a reach-selective brain area for objects located within participants' typical hand workspaces.

In recent years, there has been growing excitement within cognitive neuroscience about the concept of embodiment: How do the capabilities and limitations of our physical bodies affect neural representations in the brain? Neuropsychological and neurophysiological studies show clear evidence that short-term visuomotor experience can influence the encoding of the space around the body in parietal cortex. For example, tool-use may expand the neural representation of peripersonal space. But how is this initial spatial representation influenced by a lifetime of object-related interactions? To examine this question we used functional magnetic resonance imaging (fMRI) to investigate the neural effects of an individuals hand preferences for acting within peripersonal space. Left- and right-handed participants viewed real-world objects at different locations accessible by either the left hand, right hand, or neither hand. The superior parieto-occipital cortex (SPOC), an area most often implicated in reaching actions, showed enhanced visual responses for objects located within the range of space in which each group typically acts. Specifically, in right-handers, who strongly prefer grasping with the right hand, SPOC showed strongest activation for objects located within the range of space for the right hand only. In contrast, in left-handers, who use their two hands comparably often in visuomotor tasks, SPOC showed strongest activation for objects located within the range of space of either hand. These findings show that, even in the absence of overt responses, real 3D objects located in the individuals typical workspace for hand actions automatically invoke enhanced responses in associated visuomotor areas of the brain.

Evaluation of preprocessing steps to compensate for magnetic field distortions due to body movements in BOLD fMRI

Blood oxygenation level-dependent (BOLD) functional magnetic resonance imaging (fMRI) is currently the dominant technique for non-invasive investigation of brain functions. One of the challenges with BOLD fMRI, particularly at high fields, is compensation for the effects of spatiotemporally varying magnetic field inhomogeneities (DeltaB(0)) caused by normal subject respiration and, in some studies, movement of the subject during the scan to perform tasks related to the functional paradigm. The presence of DeltaB(0) during data acquisition distorts reconstructed images and introduces extraneous fluctuations in the fMRI time series that decrease the BOLD contrast-to-noise ratio. Optimization of the fMRI data-processing pipeline to compensate for geometric distortions is of paramount importance to ensure high quality of fMRI data. To investigate DeltaB(0) caused by subject movement, echo-planar imaging scans were collected with and without concurrent motion of a phantom arm. The phantom arm was constructed and moved by the experimenter to emulate forearm motions while subjects remained still and observed a visual stimulation paradigm. These data were then subjected to eight different combinations of preprocessing steps. The best preprocessing pipeline included navigator correction, a complex phase regressor and spatial smoothing. The synergy between navigator correction and phase regression reduced geometric distortions better than either step in isolation and preconditioned the data to make them more amenable to the benefits of spatial smoothing. The combination of these steps provided a 10% increase in t-statistics compared to only navigator correction and spatial smoothing and reduced the noise and false activations in regions where no legitimate effects would occur.

Short-term motor plasticity revealed in a visuomotor decision-making task

Selecting and executing an action toward only one object in our complex environments presents the visuomotor system with a significant challenge. To overcome this problem, the motor system is thought to simultaneously encode multiple motor plans, which then compete for selection. The decision between motor plans is influenced both by incoming sensory information and previous experience-which itself is comprised of long-term (e.g. weeks, months) and recent (seconds, minutes, hours) information. In this study, we were interested in how the recent trial-to-trial visuomotor experience would be factored into upcoming movement decisions made between competing potential targets. To this aim, we used a unique rapid reaching task to investigate how reach trajectories would be spatially influenced by previous decisions. Our task required subjects to initiate speeded reaches toward multiple potential targets before one was cued in-flight. A novel statistical analysis of the reach trajectories revealed that in cases of target uncertainty, subjects initiated a spatially averaged trajectory toward the midpoint of potential target locations before correcting to the selected target location. Interestingly, when the same target location was consecutively cued, reaches were biased toward that location on the next trial and this effect accumulated across trials. Beyond providing supporting evidence that potential reach locations are encoded and compete in parallel, our results strongly suggest that this motor competition is biased by recent trial history.