David J. Freedman

Experience-dependent representation of visual categories in parietal cortex.

Categorization is a process by which the brain assigns meaning to sensory stimuli. Through experience, we learn to group stimuli into categories, such as ‘chair’, ‘table’ and ‘vehicle’, which are critical for rapidly and appropriately selecting behavioural responses. Although much is known about the neural representation of simple visual stimulus features (for example, orientation, direction and colour), relatively little is known about how the brain learns and encodes the meaning of stimuli. We trained monkeys to classify 360° of visual motion directions into two discrete categories, and compared neuronal activity in the lateral intraparietal (LIP) and middle temporal (MT) areas, two interconnected brain regions known to be involved in visual motion processing. Here we show that neurons in LIP—an area known to be centrally involved in visuo-spatial attention, motor planning and decision-making—robustly reflect the category of motion direction as a result of learning. The activity of LIP neurons encoded directions of motion according to their category membership, and that encoding shifted after the monkeys were retrained to group the same stimuli into two new categories. In contrast, neurons in area MT were strongly direction selective but carried little, if any, explicit category information. This indicates that LIP might be an important nexus for the transformation of visual direction selectivity to more abstract representations that encode the behavioural relevance, or meaning, of stimuli.

A hierarchy of intrinsic timescales across primate cortex

Specialization and hierarchy are organizing principles for primate cortex, yet there is little direct evidence for how cortical areas are specialized in the temporal domain. We measured timescales of intrinsic fluctuations in spiking activity across areas and found a hierarchical ordering, with sensory and prefrontal areas exhibiting shorter and longer timescales, respectively. On the basis of our findings, we suggest that intrinsic timescales reflect areal specialization for task-relevant computations over multiple temporal ranges.

Neural correlates of categories and concepts

The ability to readily adapt to novel situations requires something beyond storing specific stimulus-response associations. Instead, many animals can detect basic characteristics of events and store them as generalized classes. Because these representations are abstracted beyond specific details of sensory inputs and motor outputs, they can be easily generalized and adapted to new circumstances. Explorations of neural mechanisms of sensory processing and motor output have progressed to the point where studies can begin to address the neural basis of abstract, categorical representations. Recent studies have revealed their neural correlates in various cortical areas of the non-human primate brain.

A proposed common neural mechanism for categorization and perceptual decisions

One of the most fascinating issues in neuroscience is how the brain makes decisions. Recent evidence points to the parietal cortex as an important locus for certain kinds of decisions. Because parietal neurons are also involved in movements, it has been proposed that decisions are encoded in an intentional, action-based framework based on the movements used to report decisions. An alternative or complementary view is that decisions represent more abstract information not linked to movements per se. Parallel experiments on categorization suggest that parietal neurons can indeed represent abstract categorical outcomes that are not linked to movements. This could provide a unified or complementary view of how the brain decides and categorizes.

Generalized associative representations in parietal cortex

Making associations between sensory stimuli is a critical aspect of behavior. We previously found that neurons in the lateral intraparietal area (LIP) of Macaca mulatta reflect learned associations between directions of moving visual stimuli. Individual LIP neurons might encode associations only for specific stimuli, such as motion directions; alternatively, they may encode more general associations whenever animals must decide between discrete alternatives. To test this, we asked whether LIP neurons encode learned associations between pairs of arbitrarily chosen static shapes and, in a separate task, whether the same neurons also encode associations between motion directions. Our experimental design dissociated the visual associations from the movements used to report those associations. We found robust encoding of the learned pair associations between shapes, and shape-pair–selective neurons tended to be selective for direction associations. These findings suggest that representing generic categorical outcomes may be a fundamental role of parietal neurons.

Independent Category and Spatial Encoding in Parietal Cortex

The posterior parietal cortex plays a central role in spatial functions, such as spatial attention and saccadic eye movements. However, recent work has increasingly focused on the role of parietal cortex in encoding nonspatial cognitive factors such as visual categories, learned stimulus associations, and task rules. The relationship between spatial encoding and nonspatial cognitive signals in parietal cortex, and whether cognitive signals are robustly encoded in the presence of strong spatial neuronal responses, is unknown. We directly compared nonspatial cognitive and spatial encoding in the lateral intraparietal (LIP) area by training monkeys to perform a visual categorization task during which they made saccades toward or away from LIP response fields (RFs). Here we show that strong saccade-related responses minimally influence robustly encoded category signals in LIP. This suggests that cognitive and spatial signals are encoded independently in LIP and underscores the role of parietal cortex in nonspatial cognitive functions.

Task-specific versus generalized mnemonic representations in parietal and prefrontal cortices

Our ability to learn a wide range of behavioral tasks is essential for responding appropriately to sensory stimuli according to behavioral demands, but the underlying neural mechanism has been rarely examined by neurophysiological recordings in the same subjects across learning. To understand how learning new behavioral tasks affects neuronal representations, we recorded from posterior parietal cortex (PPC) before and after training on a visual motion categorization task. We found that categorization training influenced cognitive encoding in PPC, with a marked enhancement of memory-related delay-period encoding during the categorization task that was absent during a motion discrimination task before categorization training. In contrast, the prefrontal cortex (PFC) exhibited strong delay-period encoding during both discrimination and categorization tasks. This reveals a dissociation between PFCs and PPCs roles in working memory, with general engagement of PFC across multiple tasks, in contrast with more task-specific mnemonic encoding in PPC.

Task Dependence of Visual and Category Representations in Prefrontal and Inferior Temporal Cortices

Visual categorization is an essential perceptual and cognitive process for assigning behavioral significance to incoming stimuli. Categorization depends on sensory processing of stimulus features as well as flexible cognitive processing for classifying stimuli according to the current behavioral context. Neurophysiological studies suggest that the prefrontal cortex (PFC) and the inferior temporal cortex (ITC) are involved in visual shape categorization. However, their precise roles in the perceptual and cognitive aspects of the categorization process are unclear, as the two areas have not been directly compared during changing task contexts. To address this, we examined the impact of task relevance on categorization-related activity in PFC and ITC by recording from both areas as monkeys alternated between a shape categorization and passive viewing tasks. As monkeys viewed the same stimuli in both tasks, the impact of task relevance on encoding in each area could be compared. While both areas showed task-dependent modulations of neuronal activity, the patterns of results differed markedly. PFC, but not ITC, neurons showed a modest increase in firing rates when stimuli were task relevant. PFC also showed significantly stronger category selectivity during the task compared with passive viewing, while task-dependent modulations of category selectivity in ITC were weak and occurred with a long latency. Finally, both areas showed an enhancement of stimulus selectivity during the task compared with passive viewing. Together, this suggests that the ITC and PFC show differing degrees of task-dependent flexibility and are preferentially involved in the perceptual and cognitive aspects of the categorization process, respectively.

Choice-correlated activity fluctuations underlie learning of neuronal category representation

The ability to categorize stimuli into discrete behaviourally relevant groups is an essential cognitive function. To elucidate the neural mechanisms underlying categorization, we constructed a cortical circuit model that is capable of learning a motion categorization task through reward-dependent plasticity. Here we show that stable category representations develop in neurons intermediate to sensory and decision layers if they exhibit choice-correlated activity fluctuations (choice probability). In the model, choice probability and task-specific interneuronal correlations emerge from plasticity of top-down projections from decision neurons. Specific model predictions are confirmed by analysis of single-neuron activity from the monkey parietal cortex, which reveals a mixture of directional and categorical tuning, and a positive correlation between category selectivity and choice probability. Beyond demonstrating a circuit mechanism for categorization, the present work suggests a key role of plastic top-down feedback in simultaneously shaping both neural tuning and correlated neural variability.

Dynamic Integration of Task-Relevant Visual Features in Posterior Parietal Cortex

The primate visual system consists of multiple hierarchically organized cortical areas, each specialized for processing distinct aspects of the visual scene. For example, color and form are encoded in ventral pathway areas such as V4 and inferior temporal cortex, while motion is preferentially processed in dorsal pathway areas such as the middle temporal area. Such representations often need to be integrated perceptually to solve tasks that depend on multiple features. We tested the hypothesis that the lateral intraparietal area (LIP) integrates disparate task-relevant visual features by recording from LIP neurons in monkeys trained to identify target stimuli composed of conjunctions of color and motion features. We show that LIP neurons exhibit integrative representations of both color and motion features when they are task relevant and task-dependent shifts of both direction and color tuning. This suggests that LIP plays a role in flexibly integrating task-relevant sensory signals.