Samuel A. Nastase

Uncertainty in visual and auditory series is coded by modality-general and modality-specific neural systems.

Coding for the degree of disorder in a temporally unfolding sensory input allows for optimized encoding of these inputs via information compression and predictive processing. Prior neuroimaging work has examined sensitivity to statistical regularities within single sensory modalities and has associated this function with the hippocampus, anterior cingulate, and lateral temporal cortex. Here we investigated to what extent sensitivity to input disorder, quantified by Markov entropy, is subserved by modality‐general or modality‐specific neural systems when participants are not required to monitor the input. Participants were presented with rapid (3.3 Hz) auditory and visual series varying over four levels of entropy, while monitoring an infrequently changing fixation cross. For visual series, sensitivity to the magnitude of disorder was found in early visual cortex, the anterior cingulate, and the intraparietal sulcus. For auditory series, sensitivity was found in inferior frontal, lateral temporal, and supplementary motor regions implicated in speech perception and sequencing. Ventral premotor and central cingulate cortices were identified as possible candidates for modality‐general uncertainty processing, exhibiting marginal sensitivity to disorder in both modalities. The right temporal pole differentiated the highest and lowest levels of disorder in both modalities, but did not show general sensitivity to the parametric manipulation of disorder. Our results indicate that neural sensitivity to input disorder relies largely on modality‐specific systems embedded in extended sensory cortices, though uncertainty‐related processing in frontal regions may be driven by both input modalities. Hum Brain Mapp 35:1111–1128, 2014.

How the Human Brain Represents Perceived Dangerousness or "Predacity" of Animals.

Common or folk knowledge about animals is dominated by three dimensions: (1) level of cognitive complexity or “animacy;” (2) dangerousness or “predacity;” and (3) size. We investigated the neural basis of the perceived dangerousness or aggressiveness of animals, which we refer to more generally as “perception of threat.” Using functional magnetic resonance imaging (fMRI), we analyzed neural activity evoked by viewing images of animal categories that spanned the dissociable semantic dimensions of threat and taxonomic class. The results reveal a distributed network for perception of threat extending along the right superior temporal sulcus. We compared neural representational spaces with target representational spaces based on behavioral judgments and a computational model of early vision and found a processing pathway in which perceived threat emerges as a dominant dimension: whereas visual features predominate in early visual cortex and taxonomy in lateral occipital and ventral temporal cortices, these dimensions fall away progressively from posterior to anterior temporal cortices, leaving threat as the dominant explanatory variable. Our results suggest that the perception of threat in the human brain is associated with neural structures that underlie perception and cognition of social actions and intentions, suggesting a broader role for these regions than has been thought previously, one that includes the perception of potential threat from agents independent of their biological class. SIGNIFICANCE STATEMENT For centuries, philosophers have wondered how the human mind organizes the world into meaningful categories and concepts. Today this question is at the core of cognitive science, but our focus has shifted to understanding how knowledge manifests in dynamic activity of neural systems in the human brain. This study advances the young field of empirical neuroepistemology by characterizing the neural systems engaged by an important dimension in our cognitive representation of the animal kingdom ontological subdomain: how the brain represents the perceived threat, dangerousness, or “predacity” of animals. Our findings reveal how activity for domain-specific knowledge of animals overlaps the social perception networks of the brain, suggesting domain-general mechanisms underlying the representation of conspecifics and other animals.

Attention Selectively Reshapes the Geometry of Distributed Semantic Representation

Abstract Humans prioritize different semantic qualities of a complex stimulus depending on their behavioral goals. These semantic features are encoded in distributed neural populations, yet it is unclear how attention might operate across these distributed representations. To address this, we presented participants with naturalistic video clips of animals behaving in their natural environments while the participants attended to either behavior or taxonomy. We used models of representational geometry to investigate how attentional allocation affects the distributed neural representation of animal behavior and taxonomy. Attending to animal behavior transiently increased the discriminability of distributed population codes for observed actions in anterior intraparietal, pericentral, and ventral temporal cortices. Attending to animal taxonomy while viewing the same stimuli increased the discriminability of distributed animal category representations in ventral temporal cortex. For both tasks, attention selectively enhanced the discriminability of response patterns along behaviorally relevant dimensions. These findings suggest that behavioral goals alter how the brain extracts semantic features from the visual world. Attention effectively disentangles population responses for downstream read-out by sculpting representational geometry in late-stage perceptual areas.

Connectivity in the human brain dissociates entropy and complexity of auditory inputs

Complex systems are described according to two central dimensions: (a) the randomness of their output, quantified via entropy; and (b) their complexity, which reflects the organization of a systems generators. Whereas some approaches hold that complexity can be reduced to uncertainty or entropy, an axiom of complexity science is that signals with very high or very low entropy are generated by relatively non-complex systems, while complex systems typically generate outputs with entropy peaking between these two extremes. In understanding their environment, individuals would benefit from coding for both input entropy and complexity; entropy indexes uncertainty and can inform probabilistic coding strategies, whereas complexity reflects a concise and abstract representation of the underlying environmental configuration, which can serve independent purposes, e.g., as a template for generalization and rapid comparisons between environments. Using functional neuroimaging, we demonstrate that, in response to passively processed auditory inputs, functional integration patterns in the human brain track both the entropy and complexity of the auditory signal. Connectivity between several brain regions scaled monotonically with input entropy, suggesting sensitivity to uncertainty, whereas connectivity between other regions tracked entropy in a convex manner consistent with sensitivity to input complexity. These findings suggest that the human brain simultaneously tracks the uncertainty of sensory data and effectively models their environmental generators.

Cross-modal searchlight classification: methodological challenges and recommended solutions

Multivariate cross-classification is a powerful tool for decoding abstract or supramodal representations from distributed neural populations. However, this approach introduces several methodological challenges not encountered in typical multivariate pattern analysis and information-based brain mapping. In the current report, we review these challenges, recommend solutions, and evaluate alternative approaches where possible. We address these challenges with reference to an example fMRI data set where participants were presented with brief series of auditory and visual stimuli of varying predictability with the aim of decoding predictability across auditory and visual modalities. In analyzing this data set, we highlight four particular challenges: response normalization, cross-validation, direction of cross-validation, and permutation testing.

Reliable individual differences in fine-grained cortical functional architecture

&NA; Fine‐grained functional organization of cortex is not well‐conserved across individuals. As a result, individual differences in cortical functional architecture are confounded by topographic idiosyncrasies—i.e., differences in functional–anatomical correspondence. In this study, we used hyperalignment to align information encoded in topographically variable patterns to study individual differences in fine‐grained cortical functional architecture in a common representational space. We characterized the structure of individual differences using three common functional indices, and assessed the reliability of this structure across independent samples of data in a natural vision paradigm. Hyperalignment markedly improved the reliability of individual differences across all three indices by resolving topographic idiosyncrasies and accommodating information encoded in spatially fine‐grained response patterns. Our results demonstrate that substantial individual differences in cortical functional architecture exist at fine spatial scales, but are inaccessible with anatomical normalization alone.

Cross-modal and non-monotonic representations of statistical regularity are encoded in local neural response patterns

&NA; Current neurobiological models assign a central role to predictive processes calibrated to environmental statistics. Neuroimaging studies examining the encoding of stimulus uncertainty have relied almost exclusively on manipulations in which stimuli were presented in a single sensory modality, and further assumed that neural responses vary monotonically with uncertainty. This has left a gap in theoretical development with respect to two core issues: (i) are there cross‐modal brain systems that encode input uncertainty in way that generalizes across sensory modalities, and (ii) are there brain systems that track input uncertainty in a non‐monotonic fashion? We used multivariate pattern analysis to address these two issues using auditory, visual and audiovisual inputs. We found signatures of cross‐modal encoding in frontoparietal, orbitofrontal, and association cortices using a searchlight cross‐classification analysis where classifiers trained to discriminate levels of uncertainty in one modality were tested in another modality. Additionally, we found widespread systems encoding uncertainty non‐monotonically using classifiers trained to discriminate intermediate levels of uncertainty from both the highest and lowest uncertainty levels. These findings comprise the first comprehensive report of cross‐modal and non‐monotonic neural sensitivity to statistical regularities in the environment, and suggest that conventional paradigms testing for monotonic responses to uncertainty in a single sensory modality may have limited generalizability. HighlightsParticipants were presented with auditory, visual and audiovisual series varying across four levels of regularity.For some brain regions, classifiers discriminated high from low regularity levels.For other regions, classifiers discriminated intermediate levels of regularity from both the highest and lowest regularity levels.Multivariate classifiers identify regions exhibiting cross‐modal decoding of input regularity.

Modeling semantic encoding in a common neural representational space

Encoding models for mapping voxelwise semantic tuning are typically estimated separately for each individual, limiting their generalizability. In the current report, we develop a method for estimating semantic encoding models that generalize across individuals. Functional MRI was used to measure brain responses while participants freely viewed a naturalistic audiovisual movie. Word embeddings capturing agent-, action-, object-, and scene-related semantic content were assigned to each imaging volume based on an annotation of the film. We constructed both conventional within-subject semantic encoding models and between-subject models where the model was trained on a subset of participants and validated on a left-out participant. Between-subject models were trained using cortical surface-based anatomical normalization or surface-based whole-cortex hyperalignment. We used hyperalignment to project group data into an individual’s unique anatomical space via a common representational space, thus leveraging a larger volume of data for out-of-sample prediction while preserving the individual’s fine-grained functional–anatomical idiosyncrasies. Our findings demonstrate that anatomical normalization degrades the spatial specificity of between-subject encoding models relative to within-subject models. Hyperalignment, on the other hand, recovers the spatial specificity of semantic tuning lost during anatomical normalization, and yields model performance exceeding that of within-subject models.

Structural Basis of Semantic Memory

Semantic memory comprises our conceptual knowledge of the world and provides a critical interface between perception, action, and language. Acquiring and later deploying semantic knowledge in service of behavior relies on the coordinated function of distributed cortical and hippocampal circuitry. Semantic memory is typically delineated from episodic memory in that its content is divorced from the autobiographical or experiential context at acquisition. Neuropsychological work has related this distinction to gross anatomical substrates: the hippocampus is required during context-rich acquisition and subsequent consolidation, but is over time superseded by more stable, context-free cortical encoding. Neuroimaging research has since proceeded to comprehensively map the cortical organization of semantic memory, demonstrating that the macroanatomical substrates parallel the sensorimotor systems most relevant to particular domains of knowledge. Finally, methodological advances in neuroimaging have enabled us to leverage sophisticated computational models of semantic representation and decode fine-grained neural representations of semantic content from distributed patterns of brain activity.