Stéphane Rousset

Neural networks with a self-refreshing memory: Knowledge transfer in sequential learning tasks without catastrophic forgetting

We explore a dual-network architecture with self-refreshing memory (Ans and Rousset 1997) which overcomes catastrophic forgetting in sequential learning tasks. Its principle is that new knowledge is learned along with an internally generated activity reflecting the network history. What mainly distinguishes this model from others using pseudorehearsal in feedforward multilayer networks is a reverberating process used for generating pseudoitems. This process, which tends to go up to network attractors from random activation, is more suitable for capturing optimally the deep structure of previously learned knowledge than a single feedforward pass of activity. The proposed mechanism for ?transporting memory? without loss of information between two different brain structures could be viewed as a neurobiologically plausible means for consolidation in long-term memory. Knowledge transfer is explored with regard to learning speed, ability to generalize and vulnerability to network damages. We show that transfer is more efficient when two related tasks are sequentially learned than when they are learned concurrently. With a self-refreshing memory network knowledge can be saved for a long time and therefore reused in subsequent acquisitions.

Self-refreshing memory in artificial neural networks: learning temporal sequences without catastrophic forgetting

While humans forget gradually, highly distributed connectionist networks forget catastrophically: newly learned information often completely erases previously learned information. This is not just implausible cognitively, but disastrous practically. However, it is not easy in connectionist cognitive modelling to keep away from highly distributed neural networks, if only because of their ability to generalize. A realistic and effective system that solves the problem of catastrophic interference in sequential learning of ‘static’ (i.e. non-temporally ordered) patterns has been proposed recently (Robins 1995, Connection Science, 7: 123–146, 1996, Connection Science, 8: 259–275, Ans and Rousset 1997, CR Académie des Sciences Paris, Life Sciences, 320: 989–997, French 1997, Connection Science, 9: 353–379, 1999, Trends in Cognitive Sciences, 3: 128–135, Ans and Rousset 2000, Connection Science, 12: 1–19). The basic principle is to learn new external patterns interleaved with internally generated ‘pseudopatterns’ (generated from random activation) that reflect the previously learned information. However, to be credible, this self-refreshing mechanism for static learning has to encompass our human ability to learn serially many temporal sequences of patterns without catastrophic forgetting. Temporal sequence learning is arguably more important than static pattern learning in the real world. In this paper, we develop a dual-network architecture in which self-generated pseudopatterns reflect (non-temporally) all the sequences of temporally ordered items previously learned. Using these pseudopatterns, several self-refreshing mechanisms that eliminate catastrophic forgetting in sequence learning are described and their efficiency is demonstrated through simulations. Finally, an experiment is presented that evidences a close similarity between human and simulated behaviour.

Facenet: A connectionist model of face identification in context

Abstract The role of contexts in face identification constitutes a weak point of existing cognitive models of face recognition. A connectionist system (Face-net) based on a layered network has been specified and implemented to investigate the processes underlying identification. The architecture of the Face net system takes contextual information explicitly into account in the construction of identity representations, and is provided with a reinjection mechanism which gives it dynamic properties. The model proposes that three indicators are extracted in parallel in person identification from a face: familiarity feeling (feeling of deja-vu of the face stimulus), identity feeling (feeling that we know the person) and identity content (information about the person resulting from the integration of the contexts). Face net underwent an experimental procedure to study the structuring of identity representations in various learning conditions defined by the specificity and the variability of the encoding context...

Global precedence effect in audition and vision: evidence for similar cognitive styles across modalities.

This study aimed to provide evidence for a Global Precedence Effect (GPE) in both vision and audition modalities. In order to parallel Navons paradigm, a novel auditory task was designed in which hierarchical auditory stimuli were used to involve local and global processing. Participants were asked to process auditory and visual hierarchical patterns at the local or global level. In both modalities, a global-over-local advantage and a global interference on local processing were found. The other compelling result is a significant correlation between these effects across modalities. Evidence that the same participants exhibit similar processing style across modalities strongly supports the idea of a cognitive style to process information and common processing principle in perception.

Pseudopatterns and dual-network memory models: Advantages and shortcomings

The dual-network memory model is designed to be a neurobiologically plausible manner of avoiding catastrophic interference. We discuss a number of advantages of this model and potential clues that the model has provided in the areas of memory consolidation, category-specific deficits, anterograde and retrograde amnesia. We discuss a surprising result about how this class of models handles episodic (“snap-shot”) memory — namely, that they seem to be able to handle both episodic and abstract memory — and discuss two other promising areas of research involving these models.

Confrontation of PDP models and dual-route models through the analysis of a case of deep dysphasia

Abstract A case study is presented of a patient, EA, who demonstrated all the defining features of deep dysphasia. His repetition disorder was associated with surface dyslexia and deep dysgraphia. EA also showed a severely restricted phonological STM. His performance in both picture confrontation naming and writing-to-dictation paralleled his performance in repetition, whereas reading aloud and oral lexical decision were not influenced by the imageability of the word input. Further testing indicated that EA did not have difficulty in either perceiving or semantically processing spoken words. An exhaustive investigation of EAs cognitive functioning was first conducted by reference to Patterson and Shewells model (1987). Such a triple-route model can account for EAs overall performance by postulating multiple functional lesions. We alternatively show that EAs language profile could be accounted for within a highly interactive model of language processing incorporating most basic principles of connection...

Spatial deficits in an amnesic patient with hippocampal damage: Questioning the multiple trace theory

Mediotemporal lobe structures are involved in both spatial processing and long‐term memory. Patient M.R. suffers from amnesia, due to bilateral hippocampal lesion and temporoparietal atrophy following carbon monoxide poisoning. We compared his performance in immediate spatial memory tasks with the performance of ten healthy matched participants. Using an immediate reproduction of path, we observed a dissociation between his performance in three allocentric tasks and in five egocentric‐updating tasks. His performance was always impaired on tasks requiring the use of an egocentric‐updating representation but remained preserved on allocentric tasks. These results fit with the hypothesis that the hippocampus plays a role in spatial memory, but they also suggest that allocentric deficits previously observed in amnesia might actually reflect deficits in egocentric‐updating processes. Furthermore, the cooccurrence of deficits in episodic long‐term memory and short‐term egocentric‐updating representation without any short‐term allocentric deficit suggests a new link between the mnemonic and navigational roles of the hippocampus. The Cognitive Map theory, the Multiple Trace theory, as well as further models linking spatial and nonspatial functions of the hippocampus are discussed.

Sensory–motor properties of past actions bias memory in a recognition task

The aim of this study was to show that sensory–motor consequences of past actions form part of memory trace components cued by current experience. In a first task participants had to learn a list of words. Then in a guessing task they played against the computer. Finally, in a recognition task, they had to judge if the words were or were not present in the learning task. Words appeared either in the colour associated with success or failure in the guessing task, or in a non-informative colour. In the first experiment, results show that when the words to be judged were in the colour associated with success, participants answered faster and produced more “old” responses than when the words to be judged were in the colour associated with failure in the previous task. Moreover, when the words to be judged were in the colour associated with failure, participants were slower and produced less “old” responses than when the words were in a colour not informative of success or failure. The second experiment confirms that the results obtained in Experiment 1 were linked to the sensory–motor consequences of past actions associated with the colour and not to the colour itself.

Deficits in egocentric-updating and spatial context memory in a case of developmental amnesia

Patients with developmental amnesia usually suffer from both episodic and spatial memory deficits. DM, a developmental amnesic, was impaired in her ability to process self-motion (i.e., idiothetic) information while her ability to process external stable landmarks (i.e., allothetic) was preserved when no self-motion processing was required. On a naturalistic and incidental episodic task, DM was severely and predictably impaired on both free and cued recall tasks. Interestingly, when cued, she was more impaired at recalling spatial context than factual or temporal information. Theoretical implications of that co-occurrence of deficits and those dissociations are discussed and testable cerebral hypothesis are proposed.

Ongoing egocentric spatial processing during learning of non-spatial information results in temporal-parietal activity during retrieval

Deficits in amnesic patients suggest that spatial cognition and episodic memory are intimately related. Among the different types of spatial processing, the allocentric, relying on the hippocampal formation, and the egocentric-updated, relying on parieto-temporal connections have both been considered to functionally underlie episodic memory encoding and retrieval. We explore the cerebral correlates underlying the episodic retrieval of words previously learnt outside the magnet while performing different spatial processes, allocentric and egocentric-updated. Subsequently and during fMRI, participants performed an episodic word recognition task. Data processing revealed that the correct recognition of words learnt in egocentric-updated condition enhanced activity of the medial and lateral parietal, as well as temporal cortices. No additional regions were activated in the present study by retrieving words learnt in allocentric condition. This study sheds new light on the functional links between episodic memory and spatial processing: The temporo-parietal network is confirmed to be crucial in episodic memory in healthy participants and could be linked to the egocentric-updated process.