Sean M. Polyn

Beyond mind-reading: multi-voxel pattern analysis of fMRI data.

A key challenge for cognitive neuroscience is determining how mental representations map onto patterns of neural activity. Recently, researchers have started to address this question by applying sophisticated pattern-classification algorithms to distributed (multi-voxel) patterns of functional MRI data, with the goal of decoding the information that is represented in the subjects brain at a particular point in time. This multi-voxel pattern analysis (MVPA) approach has led to several impressive feats of mind reading. More importantly, MVPA methods constitute a useful new tool for advancing our understanding of neural information processing. We review how researchers are using MVPA methods to characterize neural coding and information processing in domains ranging from visual perception to memory search.

Category-Specific Cortical Activity Precedes Retrieval During Memory Search

Here we describe a functional magnetic resonance imaging study of humans engaged in memory search during a free recall task. Patterns of cortical activity associated with the study of three categories of pictures (faces, locations, and objects) were identified by a pattern-classification algorithm. The algorithm was used to track the reappearance of these activity patterns during the recall period. The reappearance of a given categorys activity pattern correlates with verbal recalls made from that category and precedes the recall event by several seconds. This result is consistent with the hypothesis that category-specific activity is cueing the memory system to retrieve studied items.

A context maintenance and retrieval model of organizational processes in free recall

The authors present the context maintenance and retrieval (CMR) model of memory search, a generalized version of the temporal context model of M. W. Howard and M. J. Kahana (2002a), which proposes that memory search is driven by an internally maintained context representation composed of stimulus-related and source-related features. In the CMR model, organizational effects (the tendency for related items to cluster during the recall sequence) arise as a consequence of associations between active context elements and features of the studied material. Semantic clustering is due to longstanding context-to-item associations, whereas temporal clustering and source clustering are both due to associations formed during the study episode. A behavioral investigation of the three forms of organization provides data to constrain the CMR model, revealing interactions between the organizational factors. Finally, the authors discuss the implications of CMR for their understanding of a broad class of episodic memory phenomena and suggest ways in which this theory may guide exploration of the neural correlates of memory search.

Memory search and the neural representation of context

A challenge for theories of episodic memory is to determine how we focus memory search on a set of recently learned items. Cognitive theories suggest that the recall of an item representation is driven by an internally maintained context representation that integrates incoming information with a long time-scale. Neural investigations have shown that recalling an item revives the pattern of brain activity present during its study. To link these neural and cognitive approaches, we propose a framework in which context is maintained and updated in prefrontal cortex, and is associated with item information through hippocampal projections. The proposed framework is broadly consistent with neurobiological studies of temporal integration and with studies of memory deficits in individuals with prefrontal damage.

Neural Activity in Human Hippocampal Formation Reveals the Spatial Context of Retrieved Memories

Remembrance of Places Past The hippocampus has two major roles in cognition. Place-responsive neurons form a context-sensitive cognitive map, firing more strongly when an animal traverses specific regions of its environment. Both humans and animals thus need the hippocampus to learn their way around novel environments. Similarly, the hippocampus is critical for our ability to remember a specific event in space and time. It has thus been suggested that the spatial and memory functions of the hippocampus reflect a common architecture. Recording from neurosurgical patients playing a virtual reality memory game, Miller et al. (p. 1111) found that the recall of events was indeed associated with reinstatement of the place-firing of neurons activated as the subjects navigated through the environment. Place cells in the human brain that fired at an object’s location are reactivated during spontaneous recall. In many species, spatial navigation is supported by a network of place cells that exhibit increased firing whenever an animal is in a certain region of an environment. Does this neural representation of location form part of the spatiotemporal context into which episodic memories are encoded? We recorded medial temporal lobe neuronal activity as epilepsy patients performed a hybrid spatial and episodic memory task. We identified place-responsive cells active during virtual navigation and then asked whether the same cells activated during the subsequent recall of navigation-related memories without actual navigation. Place-responsive cell activity was reinstated during episodic memory retrieval. Neuronal firing during the retrieval of each memory was similar to the activity that represented the locations in the environment where the memory was initially encoded.

Oscillatory patterns in temporal lobe reveal context reinstatement during memory search

Psychological theories of memory posit that when people recall a past event, they not only recover the features of the event itself, but also recover information associated with other events that occurred nearby in time. The events surrounding a target event, and the thoughts they evoke, may be considered to represent a context for the target event, helping to distinguish that event from similar events experienced at different times. The ability to reinstate this contextual information during memory search has been considered a hallmark of episodic, or event-based, memory. We sought to determine whether context reinstatement may be observed in electrical signals recorded from the human brain during episodic recall. Analyzing electrocorticographic recordings taken as 69 neurosurgical patients studied and recalled lists of words, we uncovered a neural signature of context reinstatement. Upon recalling a studied item, we found that the recorded patterns of brain activity were not only similar to the patterns observed when the item was studied, but were also similar to the patterns observed during study of neighboring list items, with similarity decreasing reliably with positional distance. The degree to which individual patients displayed this neural signature of context reinstatement was correlated with their tendency to recall neighboring list items successively. These effects were particularly strong in temporal lobe recordings. Our findings show that recalling a past event evokes a neural signature of the temporal context in which the event occurred, thus pointing to a neural basis for episodic memory.

Associative Retrieval Processes in Episodic Memory

Association and context constitute two of the central ideas in the history of episodic memory research. Following a brief discussion of the history of these ideas, we review data that demonstrate the complementary roles of temporal contiguity and semantic relatedness in determining the order in which subjects recall lists of items and the timing of their successive recalls. These analyses reveal that temporal contiguity effects normally studied on short timescales persist over very long timescales, a result that challenges traditional psychological and neuroscientific models of association. The form of the temporal contiguity effect is conserved across all of the major recall tasks and even appears in item recognition when subjects respond with high confidence. The near-universal form of the contiguity effect and its appearance at diverse timescales is shown to place tight constraints on the major theories of association.

Human memory reconsolidation can be explained using the temporal context model

Recent work by Hupbach, Gomez, Hardt, and Nadel (Learning & Memory, 14, 47–53, 2007) and Hupbach, Gomez, and Nadel (Memory, 17, 502–510, 2009) suggests that episodic memory for a previously studied list can be updated to include new items, if participants are reminded of the earlier list just prior to learning a new list. The key finding from the Hupbach studies was an asymmetric pattern of intrusions, whereby participants intruded numerous items from the second list when trying to recall the first list, but not viceversa. Hupbach et al. (2007; 2009) explained this pattern in terms of a cellular reconsolidation process, whereby first-list memory is rendered labile by the reminder and the labile memory is then updated to include items from the second list. Here, we show that the temporal context model of memory, which lacks a cellular reconsolidation process, can account for the asymmetric intrusion effect, using well-established principles of contextual reinstatement and item–context binding.

Task context and organization in free recall

Prior work on organization in free recall has focused on the ways in which semantic and temporal information determine the order in which material is retrieved from memory. Tulvings theory of ecphory suggests that these organizational effects arise from the interaction of a retrieval cue with the contents of memory. Using the continual-distraction free-recall paradigm [Bjork, R. A., & Whitten, W. B. (1974). Recency-sensitive retrieval processes in long-term free recall. Cognitive Psychology, 6, 173-189] to minimize retrieval during the study period, we show that encoding task context can organize recall, suggesting that task-related information is part of the retrieval cue. We interpret these results in terms of the Context Maintenance and Retrieval model (CMR; [Polyn, S. M., Norman, K. A., & Kahana, M. J. (2009). A context maintenance and retrieval model of organizational processes in free recall. Psychological Review, 116 (1), 129-156]), in which an internal contextual representation, containing semantic, temporal, and source-related information, serves as the retrieval cue and organizes the retrieval of information from memory. We discuss these results in terms of the guided activation theory [Miller, E. K., & Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. Annual Review of Neuroscience, 24, 167-202] of the role of prefrontal cortex in task performance, as well as the rich neuropsychological literature implicating prefrontal cortex in memory search (e.g., Schacter (1987). Memory, amnesia, and frontal lobe dysfunction. Psychobiology, 15, 21-36).

Functional Interactions Between Large-Scale Networks During Memory Search

Neuroimaging studies have identified two major large-scale brain networks, the default mode network (DMN) and the dorsal attention network (DAN), which are engaged for internally and externally directed cognitive tasks respectively, and which show anticorrelated activity during cognitively demanding tests and at rest. We identified these brain networks using independent component analysis (ICA) of functional magnetic resonance imaging data, and examined their interactions during the free-recall task, a self-initiated memory search task in which retrieval is performed in the absence of external cues. Despite the internally directed nature of the task, the DAN showed transient engagement in the seconds leading up to successful retrieval. ICA revealed a fractionation of the DMN into 3 components. A posteromedial network increased engagement during memory search, while the two others showed suppressed activity during memory search. Cooperative interactions between this posteromedial network, a right-lateralized frontoparietal control network, and a medial prefrontal network were maintained during memory search. The DAN demonstrated heterogeneous task-dependent shifts in functional coupling with various subnetworks within the DMN. This functional reorganization suggests a broader role of the DAN in the absence of externally directed cognition, and highlights the contribution of the posteromedial network to episodic retrieval.