Martin A. Giese

Neural mechanisms for the recognition of biological movements

The visual recognition of complex movements and actions is crucial for the survival of many species. It is important not only for communication and recognition at a distance, but also for the learning of complex motor actions by imitation. Movement recognition has been studied in psychophysical, neurophysiological and imaging experiments, and several cortical areas involved in it have been identified. We use a neurophysiologically plausible and quantitative model as a tool for organizing and making sense of the experimental data, despite their growing size and complexity. We review the main experimental findings and discuss possible neural mechanisms, and show that a learning-based, feedforward model provides a neurophysiologically plausible and consistent summary of many key experimental results.

Nonvisual Motor Training Influences Biological Motion Perception

Experimental evidence suggests a link between perception and the execution of actions . In particular, it has been proposed that motor programs might directly influence visual action perception . According to this hypothesis, the acquisition of novel motor behaviors should improve their visual recognition, even in the absence of visual learning. We tested this prediction by using a new experimental paradigm that dissociates visual and motor learning during the acquisition of novel motor patterns. The visual recognition of gait patterns from point-light stimuli was assessed before and after nonvisual motor training. During this training, subjects were blindfolded and learned a novel coordinated upper-body movement based only on verbal and haptic feedback. The learned movement matched one of the visual test patterns. Despite the absence of visual stimulation during training, we observed a selective improvement of the visual recognition performance for the learned movement. Furthermore, visual recognition performance after training correlated strongly with the accuracy of the execution of the learned motor pattern. These results prove, for the first time, that motor learning has a direct and highly selective influence on visual action recognition that is not mediated by visual learning.

Norm-based face encoding by single neurons in the monkey inferotemporal cortex

The rich and immediate perception of a familiar face, including its identity, expression and even intent, is one of the most impressive shared faculties of human and non-human primate brains. Many visually responsive neurons in the inferotemporal cortex of macaque monkeys respond selectively to faces, sometimes to only one or a few individuals, while showing little sensitivity to scale and other details of the retinal image. Here we show that face-responsive neurons in the macaque monkey anterior inferotemporal cortex are tuned to a fundamental dimension of face perception. Using a norm-based caricaturization framework previously developed for human psychophysics, we varied the identity information present in photo-realistic human faces, and found that neurons of the anterior inferotemporal cortex were most often tuned around the average, identity-ambiguous face. These observations are consistent with face-selective responses in this area being shaped by a figural comparison, reflecting structural differences between an incoming face and an internal reference or norm. As such, these findings link the tuning of neurons in the inferotemporal cortex to psychological models of face identity perception.

Critical features for the perception of emotion from gait

Human observers readily recognize emotions expressed in body movement. Their perceptual judgments are based on simple movement features, such as overall speed, but also on more intricate posture and dynamic cues. The systematic analysis of such features is complicated due to the difficulty of considering the large number of potentially relevant kinematic and dynamic parameters. To identify emotion-specific features we motion-captured the neutral and emotionally expressive (anger, happiness, sadness, fear) gaits of 25 individuals. Body posture was characterized by average flexion angles, and a low-dimensional parameterization of the spatio-temporal structure of joint trajectories was obtained by approximation with a nonlinear mixture model. Applying sparse regression, we extracted critical emotion-specific posture and movement features, which typically depended only on a small number of joints. The features we extracted from the motor behavior closely resembled features that were critical for the perception of emotion from gait, determined by a statistical analysis of classification and rating judgments of 21 observers presented with avatars animated with the recorded movements. The perceptual relevance of these features was further supported by another experiment showing that artificial walkers containing only the critical features induced high-level after-effects matching those induced by adaptation with natural emotional walkers.

Morphable Models for the Analysis and Synthesis of Complex Motion Patterns

The linear combination of prototypical views provides a powerful approach for the recognition and the synthesis of images of stationary three-dimensional objects. In this article, we present initial results that demonstrate that similar ideas can be developed for the recognition and synthesis of complex motion patterns. We present a technique that permits to represent complex motion or action patterns by linear combinations of a small number of prototypical image sequences. We demonstrate the applicability of this new approach for the synthesis and analysis of biological motion using simulated and real video data from different locomotion patterns. Our results show that complex motion patterns are embedded in pattern spaces with a defined topological structure, which can be uncovered with our methods. The underlying pattern space seems to have locally, but not globally, the properties of a linear vector space. We show how the knowledge about the topology of the pattern space can be exploited during pattern recognition. Our method may provide a new interesting approach for the analysis and synthesis of video sequences and complex movements.

Critical features for the recognition of biological motion

Humans can perceive the motion of living beings from very impoverished stimuli like point-light displays. How the visual system achieves the robust generalization from normal to point-light stimuli remains an unresolved question. We present evidence on multiple levels demonstrating that this generalization might be accomplished by an extraction of simple mid-level optic flow features within coarse spatial arrangement, potentially exploiting relatively simple neural circuits: (1) A statistical analysis of the most informative mid-level features reveals that normal and point-light walkers share very similar dominant local optic flow features. (2) We devise a novel point-light stimulus (critical features stimulus) that contains these features, and which is perceived as a human walker even though it is inconsistent with the skeleton of the human body. (3) A neural model that extracts only these critical features accounts for substantial recognition rates for strongly degraded stimuli. We conclude that recognition of biological motion might be accomplished by detecting mid-level optic flow features with relatively coarse spatial localization. The computationally challenging reconstruction of precise position information from degraded stimuli might not be required.

Flexible coding for categorical decisions in the human brain.

Despite the importance of visual categorization for interpreting sensory experiences, little is known about the neural representations that mediate categorical decisions in the human brain. Here, we used psychophysics and pattern classification for the analysis of functional magnetic resonance imaging data to predict the features critical for categorical decisions from brain activity when observers categorized the same stimuli using different rules. Although a large network of cortical and subcortical areas contain information about visual categories, we show that only a subset of these areas shape their selectivity to reflect the behaviorally relevant features rather than simply physical similarity between stimuli. Specifically, temporal and parietal areas show selectivity for the perceived form and motion similarity, respectively. In contrast, frontal areas and the striatum represent the conjunction of spatiotemporal features critical for complex and adaptive categorization tasks and potentially modulate selectivity in temporal and parietal areas. These findings provide novel evidence for flexible neural coding in the human brain that translates sensory experiences to categorical decisions by shaping neural representations across a network of areas with dissociable functional roles in visual categorization.

Predicting point-light actions in real-time

There is convincing evidence for a mirror system in humans which simulates actions of conspecifics. One possible purpose of such a simulation system is to support action prediction in real-time. Our goal was to study whether the prediction of actions involves a real-time simulation process. We motion-captured a number of human actions and rendered them as point-light action sequences. Observers perceived brief videos of these actions, followed by an occluder and a static test posture. We independently varied the occluder time and the movement gap (i.e., the time between the endpoint of the action and the test posture). Observers were required to judge whether the test stimulus depicted a continuation of the action in the same depth orientation. Prediction performance was best when occluder time and movement gap corresponded, i.e., when the test posture was a continuation of the sequence that matched the occluder duration (Experiments 1, 2 and 4). This pattern of results was destroyed when the sequences and test images were flipped around the horizontal axis (Experiment 3). Overall, our findings suggest that action prediction involves a simulation process that operates in real-time. This process can break down when the actions are presented under viewing conditions for which observers have little experience.

Neural representations of kinematic laws of motion: Evidence for action-perception coupling

Behavioral and modeling studies have established that curved and drawing human hand movements obey the 2/3 power law, which dictates a strong coupling between movement curvature and velocity. Human motion perception seems to reflect this constraint. The functional MRI study reported here demonstrates that the brains response to this law of motion is much stronger and more widespread than to other types of motion. Compliance with this law is reflected in the activation of a large network of brain areas subserving motor production, visual motion processing, and action observation functions. Hence, these results strongly support the notion of similar neural coding for motion perception and production. These findings suggest that cortical motion representations are optimally tuned to the kinematic and geometrical invariants characterizing biological actions.

Long-term effects of coordinative training in degenerative cerebellar disease†

Few clinical studies have evaluated physiotherapeutic interventions for patients with degenerative cerebellar disease. In particular, evidence for long‐term effects and transfer to activities of daily life is rare. We have recently shown that coordinative training leads to short‐term improvements in motor performance. To evaluate long‐term benefits and translation to real world function, we here assessed motor performance and achievements in activities of daily life 1 year after a 4 week intensive coordinative training, which was followed by a home training program. Effects were assessed by clinical rating scales, a goal attainment score and quantitative movement analysis. Despite gradual decline of motor performance and gradual increase of ataxia symptoms due to progression of disease after 1 year, improvements in motor performance and achievements in activities of daily life persisted. Thus, also in patients with degenerative cerebellar disease, continuous coordinative training leads to long‐term improvements, which translate to real world function.