Tjeerd Jellema

Perceptual History Influences Neural Responses to Face and Body Postures

We show that under natural viewing, the responses of cells in the temporal lobe of the macaque to the sight of static head and body postures is controlled by the sight of immediately preceding actions. Cells in the anterior part of the superior temporal sulcus responded vigorously to the sight of a face or body posture that followed a particular body action, but not when it followed other actions. The effective action or posture presented in isolation or in different sequences failed to produce a response. Our results demonstrate that cells in the temporal cortex could support the formation of expectations about impending behavior of others.

Neural representations of perceived bodily actions using a categorical frame of reference

In object-centered, or categorical, visual representations of an agents actions, the spatial positions of (parts of) the body action are defined with respect to the principal axis of the agent, rather than in relation to the observer. Some cells in the superior temporal sulcus (STS) of the macaque monkey have been reported to use such a categorical frame of reference to code for bodily postures and actions, but their small numbers prevented a detailed investigation. Here we report for the first time that anterior sites in the STS contain a relatively large number of cells that use an object-centered frame of reference to code for animate objects and their actions. We further show that these cells are selectively responsive to much more restricted sections of an action trajectory than previously thought, and that they generalize over highly unusual, hitherto untested, orientations (such as upside-down walking). Quite remarkably, some cells coded for the position of a body part not only with respect to the objects body structure, but also with respect to objects in the immediate surrounding, such as the supporting substrate the agent was standing on. We illustrate these properties using three cell populations: one responding to rotations of the upper body with respect to the lower body, one responding to forward or backward walking actions, and one responding to flexions of the knees. We discuss a possible role for the object-centered STS cell populations in representing the abstract object structure of flexible, animate objects, which could enable their recognition even when presented in highly unusual poses. We further discuss their possible role in representing goal-directed or intentional actions, and in imitation.

EFFECT OF IMAGE ORIENTATION AND SIZE ON OBJECT RECOGNITION: RESPONSES OF SINGLE UNITS IN THE MACAQUE MONKEY TEMPORAL CORTEX

This study examined how cells in the temporal cortex code orientation and size of a complex object. The study focused on cells selectively responsive to the sight of the head and body but unresponsive to control stimuli. The majority of cells tested (19/26, 73%) were selectively responsive to a particular orientation in the picture plane of the static whole body stimulus, 7/26 cells showed generalisation responding to all orientations (three cells with orientation tuning superimposed on a generalised response). Of all cells sensitive to orientation, the majority (15/22, 68%) were tuned to the upright image. The majority of cells tested (81%, 13/16) were selective for stimulus size. The remaining cells (3/16) showed generalisation across four-fold decrease in size from life-sized. All size-sensitive cells were tuned to life-sized stimuli with decreasing responses to stimuli reduced from life-size. These results do not support previous suggestions that cells responsive to the head and body are selective to view but generalise across orientation and size. Here, extensive selectivity for size and orientation is reported. It is suggested that object orientation and size-specific responses might be pooled to obtain cell responses that generalise across size and orientation. The results suggest that experience affects neuronal coding of objects in that cells become tuned to views, orientation, and image sizes that are commonly experienced. Models of object recognition are discussed.

Delayed Response to Animate Implied Motion in Human Motion Processing Areas

Viewing static photographs of objects in motion evokes higher fMRI activation in the human medial temporal complex (MT+) than looking at similar photographs without this implied motion. As MT+ is traditionally thought to be involved in motion perception (and not in form perception), this finding suggests feedback from object-recognition areas onto MT+. To investigate this hypothesis, we recorded extracranial potentials evoked by the sight of photographs of biological agents with and without implied motion. The difference in potential between responses to pictures with and without implied motion was maximal between 260 and 400 msec after stimulus onset. Source analysis of this difference revealed one bilateral, symmetrical dipole pair in the occipital lobe. This area also showed a response to real motion, but approximately 100 msec earlier than the implied motion response. The longer latency of the implied motion response in comparison to the real motion response is consistent with a feedback projection onto MT+ following object recognition in higher-level temporal areas.

Involuntary interpretation of social cues is compromised in autism spectrum disorders

A new social distance judgment task was used to measure quantitatively the extent to which social cues are immediately and involuntary interpreted by typically developing (TD) individuals and by individuals with autism spectrum disorders (ASD). The task thus tapped into the ability to involuntary “pick up” the meaning of social cues. The cues tested were social attention and implied biological motion. Task performance of the ASD and TD groups was similarly affected by a perceptual low‐level illusion induced by physical characteristics of the stimuli. In contrast, a high‐level illusion induced by the implications of the social cues affected only the TD individuals; the ASD individuals remained unaffected (causing them to perform superior to TD controls). The results indicate that despite intact perceptual processing, the immediate involuntary interpretation of social cues can be compromised. We propose that this type of social cue understanding is a distinct process that should be differentiated from reflective social cue understanding and is specifically compromised in ASD. We discuss evidence for an underpinning neural substrate.

Deficits in implicit attention to social signals in schizophrenia and high risk groups: Behavioural evidence from a new illusion.

Background An increasing body of evidence suggests that the apparent social impairments observed in schizophrenia may arise from deficits in social cognitive processing capacities. The ability to process basic social cues, such as gaze direction and biological motion, effortlessly and implicitly is thought to be a prerequisite for establishing successful social interactions and for construing a sense of “social intuition.” However, studies that address the ability to effortlessly process basic social cues in schizophrenia are lacking. Because social cognitive processing deficits may be part of the genetic vulnerability for schizophrenia, we also investigated two groups that have been shown to be at increased risk of developing schizophrenia-spectrum pathology: first-degree relatives of schizophrenia patients and men with Klinefelter syndrome (47,XXY). Results We compared 28 patients with schizophrenia, 29 siblings of patients with schizophrenia, and 29 individuals with Klinefelter syndrome with 46 matched healthy control subjects on a new paradigm. This paradigm measures ones susceptibility for a bias in distance estimation between two agents that is induced by the implicit processing of gaze direction and biological motion conveyed by these agents. Compared to control subjects, patients with schizophrenia, as well as siblings of patients and Klinefelter men, showed a lack of influence of social cues on their distance judgments. Conclusions We suggest that the insensitivity for social cues is a cognitive aspect of schizophrenia that may be seen as an endophenotype as it appears to be present both in relatives who are at increased genetic risk and in a genetic disorder at risk for schizophrenia-spectrum psychopathology. These social cue–processing deficits could contribute, in part, to the difficulties in higher order social cognitive tasks and, hence, to decreased social competence that has been observed in these groups.

Adaptation to Real Motion Reveals Direction-selective Interactions between Real and Implied Motion Processing

Viewing static pictures of running humans evokes neural activity in the dorsal motion-sensitive cortex. To establish whether this response arises from direction-selective neurons that are also involved in real motion processing, we measured the visually evoked potential to implied motion following adaptation to static or moving random dot patterns. The implied motion response was defined as the difference between evoked potentials to pictures with and without implied motion. Interaction between real and implied motion was found as a modulation of this difference response by the preceding motion adaptation. The amplitude of the implied motion response was significantly reduced after adaptation to motion in the same direction as the implied motion, compared to motion in the opposite direction. At 280 msec after stimulus onset, the average difference in amplitude reduction between opposite and same adapted direction was 0.5 V on an average implied motion amplitude of 2.0 V. These results indicate that the response to implied motion arises from direction-selective motion-sensitive neurons. This is consistent with interactions between real and implied motion processing at a neuronal level.

Biases in the perception and affective valence of neutral facial expressions induced by the immediate perceptual history

We report a new perceptual distortion of neutral facial expressions induced by the immediate dynamic perceptual history. In Experiment 1, participants evaluated the facial expression on the last frame of videoclips showing morphs from a happy or angry expression to neutral, and consistently judged these as slightly angry in the happy-to-neutral condition, and as slightly happy in the angry-to-neutral condition. To exclude the possibility of a cognitive response strategy, in Experiments 2 and 3 the sequences were used in an affective priming paradigm as task-irrelevant distractors, with positive or negative words superimposed on the last frame as targets. Participants made speeded evaluations of word valence. In Experiment 2, neutral-to-intense sequences produced clear affective congruence effects. In Experiment 3, intense-to-neutral sequences again produced affective congruence effects, but in opposite direction. This suggests that the perceptual biases found in Experiment 1 were not due to a cognitive response strategy but reflected a genuine change in the observers judgement. We speculate that the bias in perceptual report reflected the influence of top-down emotional anticipation processes.

Visual Aftereffects for Walking Actions Reveal Underlying Neural Mechanisms for Action Recognition

The results of this study illustrate a new high-level visual aftereffect: Observing actors walking forward, without horizontal translation, makes subsequent actors appear to walk backward, and the opposite effect is obtained after observing backward walking. We used this aftereffect, which cannot be explained by simple low-level adaptation to motion direction, to investigate the properties of neural mechanisms underlying recognition of walking actions. Our results suggest that the perception of walking and the perception of static images of actors in walking postures rely on common brain mechanisms that are primarily object centered, rather than viewer centered, and that are blind to the identity of the actor. These results, obtained with human psychophysical adaptation techniques, support previous evidence accumulated using single-unit recording in nonhuman primates. In addition, these results provide evidence that current models of human action recognition require an object-centered processing stage.

Unequal impairment in the recognition of positive and negative emotions after right hemisphere lesions: A left hemisphere bias for happy faces

The processing of several important aspects of a human face was investigated in a single patient (LZ), who had a large infarct of the right hemisphere involving the parietal, and temporal lobes with extensions into the frontal region. LZ showed selective problems with recognizing emotional expressions, whereas she was flawless in recognizing gender, familiarity, and identity. She was very poor in recognizing negative facial expressions (fear, disgust, anger, sadness), but scored as well as the controls on the positive facial expression of happiness. However, in two experiments using both static and dynamic face stimuli, we showed that LZ also did not have a proper notion of what a facial expression of happiness looks like, and could not adequately apply this label. We conclude that the proper recognition of both negative and positive facial expressions relies on the right hemisphere, and that the left hemisphere produces a default state resulting in a bias towards evaluating expressions as happy. We discuss the implications of the current findings for the main models that aim to explain hemispheric specializations for processing of positive and negative emotions.