Nicholas C. Hindy

Linking pattern completion in the hippocampus to predictive coding in visual cortex

Models of predictive coding frame perception as a generative process in which expectations constrain sensory representations. These models account for expectations about how a stimulus will move or change from moment to moment, but do not address expectations about what other, distinct stimuli are likely to appear based on prior experience. We show that such memory-based expectations in human visual cortex are related to the hippocampal mechanism of pattern completion.

The Effect of Object State-Changes on Event Processing: Do Objects Compete with Themselves?

When an object is described as changing state during an event, do the representations of those states compete? The distinct states they represent cannot coexist at any one moment in time, yet each representation must be retrievable at the cost of suppressing the other possible object states. We used functional magnetic resonance imaging of human participants to test whether such competition does occur, and whether this competition between object states recruits brain areas sensitive to other forms of conflict. In Experiment 1, the same object was changed either substantially or minimally by one of two actions. In Experiment 2, the same action either substantially or minimally changed one of two objects. On a subject-specific basis, we identified voxels most responsive to conflict in a Stroop color-word interference task. Voxels in left posterior ventrolateral prefrontal cortex most responsive to Stroop conflict were also responsive to our object state-change manipulation, and were not responsive to the imageability of the described action. In contrast, voxels in left middle frontal gyrus responsive to Stroop conflict were not responsive even to language, and voxels in left middle temporal gyrus that were responsive to language and imageability were not responsive to object state-change. Results suggest that, when representing object state-change, multiple incompatible representations of an object compete, and the greater the difference between the initial state and the end state of an object, the greater the conflict.

A Cortical Network for the Encoding of Object Change

Understanding events often requires recognizing unique stimuli as alternative, mutually exclusive states of the same persisting object. Using fMRI, we examined the neural mechanisms underlying the representation of object states and object-state changes. We found that subjective ratings of visual dissimilarity between a depicted object and an unseen alternative state of that object predicted the corresponding multivoxel pattern dissimilarity in early visual cortex during an imagery task, while late visual cortex patterns tracked dissimilarity among distinct objects. Early visual cortex pattern dissimilarity for object states in turn predicted the level of activation in an area of left posterior ventrolateral prefrontal cortex (pVLPFC) most responsive to conflict in a separate Stroop color-word interference task, and an area of left ventral posterior parietal cortex (vPPC) implicated in the relational binding of semantic features. We suggest that when visualizing object states, representational content instantiated across early and late visual cortex is modulated by processes in left pVLPFC and left vPPC that support selection and binding, and ultimately event comprehension.

Action-Based Learning of Multistate Objects in the Medial Temporal Lobe

Actions constrain perception by changing the appearance of objects in the environment. As such, they provide an interactive basis for learning the structure of visual input. If an action systematically transforms one stimulus into another, then these stimuli are more likely to reflect different states of the same persisting object over time. Here we show that such multistate objects are represented in the human medial temporal lobe--the result of a mechanism in which actions influence associative learning of how objects transition between states. We further demonstrate that greater recruitment of these action-based representations during object perception is accompanied by attenuated activity in stimulus-selective visual cortex. In this way, our interactions with the environment help build visual knowledge that predictively facilitates perceptual processing.

Competition between mutually exclusive object states in event comprehension

Successful language comprehension requires one to correctly match symbols in an utterance to referents in the world, but the rampant ambiguity present in that mapping poses a challenge. Sometimes the ambiguity lies in which of two (or more) types of things in the world are under discussion (i.e., lexical ambiguity); however, even a word with a single sense can have an ambiguous referent. This ambiguity occurs when an object can exist in multiple states. Here, we consider two cases in which the presence of multiple object states may render a single-sense word ambiguous. In the first case, one must disambiguate between two states of a single object token in a short discourse. In the second case, the discourse establishes two different tokens of the object category. Both cases involve multiple object states: These states are mutually exclusive in the first case, whereas in the second case, these states can logically exist at the same time. We use fMRI to contrast same-token and different-token discourses, using responses in left posterior ventrolateral prefrontal cortex (pVLPFC) as an indicator of conflict. Because the left pVLPFC is sensitive to competition between multiple, incompatible representations, we predicted that state ambiguity should engender conflict only when those states are mutually exclusive. Indeed, we find evidence of conflict in same-token, but not different-token, discourses. Our data support a theory of left pVLPFC function in which general conflict resolution mechanisms are engaged to select between multiple incompatible representations that arise in many kinds of ambiguity present in language.