James L. McClelland

A Distributed, Developmental Model of Word Recognition and Naming

A parallel distributed processing model of visual word recognition and pronunciation is described. The model consists of sets of orthographic and phonological units and an interlevel of hidden units. Weights on connections between units were modified during a training phase using the back-propagation learning algorithm. The model simulates many aspects of human performance, including (a) differences between words in terms of processing difficulty, (b) pronunciation of novel items, (c) differences between readers in terms of word recognition skill, (d) transitions from beginning to skilled reading, and (e) differences in performance on lexical decision and naming tasks. The models behavior early in the learning phase corresponds to that of children acquiring word recognition skills. Training with a smaller number of hidden units produces output characteristic of many dyslexic readers. Naming is simulated without pronunciation rules, and lexical decisions are simulated without accessing word-level representations. The performance of the model is largely determined by three factors: the nature of the input, a significant fragment of written English; the learning rule, which encodes the implicit structure of the orthography in the weights on connections; and the architecture of the system, which influences the scope of what can be learned.

Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory.

Damage to the hippocampal system disrupts recent memory but leaves remote memory intact. The account presented here suggests that memories are first stored via synaptic changes in the hippocampal system, that these changes support reinstatement of recent memories in the neocortex, that neocortical synapses change a little on each reinstatement, and that remote memory is based on accumulated neocortical changes. Models that learn via changes to connections help explain this organization. These models discover the structure in ensembles of items if learning of each item is gradual and interleaved with learning about other items. This suggests that the neocortex learns slowly to discover the structure in ensembles of experiences. The hippocampal system permits rapid learning of new items without disrupting this structure, and reinstatement of new memories interleaves them with others to integrate them into structured neocortical memory systems.

Understanding normal and impaired word reading: Computational principles in quasi-regular domains.

A connectionist approach to processing in quasi-regular domains, as exemplified by English word reading, is developed. Networks using appropriately structured orthographic and phonological representations were trained to read both regular and exception words, and yet were also able to read pronounceable nonwords as well as skilled readers. A mathematical analysis of a simplified system clarifies the close relationship of word frequency and spelling-sound consistency in influencing naming latencies. These insights were verified in subsequent simulations, including an attractor network that accounted for latency data directly in its time to settle on a response. Further analyses of the ability of networks to reproduce data on acquired surface dyslexia support a view of the reading system that incorporates a graded division of labor between semantic and phonological processes, and contrasts in important ways with the standard dual-route account.

The TRACE model of speech perception

Abstract We describe a model called the TRACE model of speech perception. The model is based on the principles of interactive activation. Information processing takes place through the excitatory and inhibitory interactions of a large number of simple processing units, each working continuously to update its own activation on the basis of the activations of other units to which it is connected. The model is called the TRACE model because the network of units forms a dynamic processing structure called “the Trace,” which serves at once as the perceptual processing mechanism and as the systems working memory. The model is instantiated in two simulation programs. TRACE I, described in detail elsewhere, deals with short segments of real speech, and suggests a mechanism for coping with the fact that the cues to the identity of phonemes vary as a function of context. TRACE II, the focus of this article, simulates a large number of empirical findings on the perception of phonemes and words and on the interactions of phoneme and word perception. At the phoneme level, TRACE II simulates the influence of lexical information on the identification of phonemes and accounts for the fact that lexical effects are found under certain conditions but not others. The model also shows how knowledge of phonological constraints can be embodied in particular lexical items but can still be used to influence processing of novel, nonword utterances. The model also exhibits categorical perception and the ability to trade cues off against each other in phoneme identification. At the word level, the model captures the major positive feature of Marslen-Wilsons COHORT model of speech perception, in that it shows immediate sensitivity to information favoring one word or set of words over others. At the same time, it overcomes a difficulty with the COHORT model: it can recover from underspecification or mispronunciation of a words beginning. TRACE II also uses lexical information to segment a stream of speech into a sequence of words and to find word beginnings and endings, and it simulates a number of recent findings related to these points. The TRACE model has some limitations, but we believe it is a step toward a psychologically and computationally adequate model of the process of speech perception.

On the control of automatic processes: A parallel distributed processing account of the Stroop effect.

Traditional views of automaticity are in need of revision. For example, automaticity often has been treated as an all-or-none phenomenon, and traditional theories have held that automatic processes are independent of attention. Yet recent empirical data suggest that automatic processes are continuous, and furthermore are subject to attentional control. A model of attention is presented to address these issues. Within a parallel distributed processing framework, it is proposed that the attributes of automaticity depend on the strength of a processing pathway and that strength increases with training. With the Stroop effect as an example, automatic processes are shown to be continuous and to emerge gradually with practice. Specifically, a computational model of the Stroop task simulates the time course of processing as well as the effects of learning. This was accomplished by combining the cascade mechanism described by McClelland (1979) with the backpropagation learning algorithm (Rumelhart, Hinton, & Williams, 1986). The model can simulate performance in the standard Stroop task, as well as aspects of performance in variants of this task that manipulate stimulus-onset asynchrony, response set, and degree of practice. The model presented is contrasted against other models, and its relation to many of the central issues in the literature on attention, automaticity, and interference is discussed.

The time course of perceptual choice: The leaky, competing accumulator model

The time course of perceptual choice is discussed in a model of gradual, leaky, stochastic, and competitive information accumulation in nonlinear decision units. Special cases of the model match a classical diffusion process, but leakage and competition work together to address several challenges to existing diffusion, random walk, and accumulator models. The model accounts for data from choice tasks using both time-controlled (e.g., response signal) and standard reaction time paradigms and its adequacy compares favorably with other approaches. A new paradigm that controls the time of arrival of information supporting different choice alternatives provides further support. The model captures choice behavior regardless of the number of alternatives, accounting for the log-linear relation between reaction time and number of alternatives (Hicks law) and explains a complex pattern of visual and contextual priming in visual word identification.

On the Time Relations of Mental Processes: An Examination of Systems of Processes in Cascade.

This article examines the possibility that the components of an informationprocessing system all operate continuously, passing information from one to the next as it becomes available. A model called the cascade model is presented and it is shown to be compatible with the general form of the relation between time and accuracy in speed-accuracy trade-off experiments. In the model , experimentlLlmanipulations may have either or both of two effects on a processing level: They may alter the rate of response or the asymptotic quality pf the output. The effects of such manipulations on the output of a system of proessesare described. The model is then used to reexamine, the subtraction and additive factors methods for analyzing the composition of systems of processes. The examination of the additive factors method yields particularly interesting results. Among them is the finding that factors that affect the rates of two different processes would be expected to have additive effects on reaction times under the cascade model, whereas factors that both affect the rate of the same process would tend to interact, just as in the case in which the manipulations affect the durations of discrete stages. On the other hand, factors that affect asymptotic output tend to interact whether they affect the same or different processes. In light of this observation, the conclusions drawn from several studies about the locus of perceptual and attentional effects on processing are reexamined. Finally, an outline is presented of a new method for analyzing processes in cascade. The method extends the additive factors method to an analysis of the parameters of the function relating response time and accuracy.

An interactive activation model of context effects in letter perception: II. The contextual enhancement effect and some tests and extensions of the model

The interactive activation model of context effects in letter perception is reviewed, elaborated, and tested. According to the model context aids the perception of target letters as they are processed in the perceptual system. The implication that the duration and timing of the context in which a letter occurs should greatly influence the perceptibility of the target is confirmed by a series of experiments demonstrating that early or enhanced presentations of word and pronounceablepseudoword contexts greatly increase the perceptibility of target letters. Also according to the model, letters in strings that share several letters with words should be equally perceptible whether they are orthographically regular and pronounceable (SLET) or irregular (SLNT) and should be much more perceptible than letters in contexts that share few letters with any word (XLQJ). This prediction is tested and confirmed. The basic results of all the experiments are accounted for, with some modification of parameters, although there are some discrepancies in detail. Several recent findings that seem to challenge the model are considered and a number of extensions are proposed.

Distributed memory and the representation of general and specific information.

We describe a distributed model of information processing and memory and apply it to the representation of general and specific information. The model consists of a large number of simple processing elements which send excitatory and inhibitory signals to each other via modifiable connections. Information processing is thought of as the process whereby patterns of activation are formed over the units in the model through their excitatory and inhibitory interactions. The memory trace of a processing event is the change or increment to the strengths of the interconnections that results from the processing event. The traces of separate events are superimposed on each other in the values of the connection strengths that result from the entire set of traces stored in the memory. The model is applied to a number of findings related to the question of whether we store abstract representations or an enumeration of specific experiences in memory. The model simulates the results of a number of important experiments which have been taken as evidence for the enumeration of specific experiences. At the same time, it shows how the functional equivalent of abstract representations--prototypes, logogens, and even rules--can emerge from the superposition of traces of specific experiences, when the conditions are right for this to happen. In essence, the model captures the structure present in a set of input patterns; thus, it behaves as though it had learned prototypes or rules, to the extent that the structure of the environment it has learned about can be captured by describing it in terms of these abstractions.

A computational model of semantic memory impairment : modality specificity and emergent category specificity

It is demonstrated how a modality-specific semantic memory system can account for category-specific impairments after brain damage. In Experiment 1, the hypothesis that visual and functional knowledge play different roles in the representation of living things and nonliving things is tested and confirmed. A parallel distributed processing model of semantic memory in which knowledge is subdivided by modality into visual and functional components is described. In Experiment 2, the model is lesioned, and it is confirmed that damage to visual semantics primarily impairs knowledge of living things, and damage to functional semantics primarily impairs knowledge of nonliving things. In Experiment 3, it is demonstrated that the model accounts naturally for a finding that had appeared problematic for a modality-specific architecture, namely, impaired retrieval of functional knowledge about living things. Finally, in Experiment 4, it is shown how the model can account for a recent observation of impaired knowledge of living things only when knowledge is probed verbally.