David C. Plaut

Understanding normal and impaired word reading: Computational principles in quasi-regular domains.

A connectionist approach to processing in quasi-regular domains, as exemplified by English word reading, is developed. Networks using appropriately structured orthographic and phonological representations were trained to read both regular and exception words, and yet were also able to read pronounceable nonwords as well as skilled readers. A mathematical analysis of a simplified system clarifies the close relationship of word frequency and spelling-sound consistency in influencing naming latencies. These insights were verified in subsequent simulations, including an attractor network that accounted for latency data directly in its time to settle on a response. Further analyses of the ability of networks to reproduce data on acquired surface dyslexia support a view of the reading system that incorporates a graded division of labor between semantic and phonological processes, and contrasts in important ways with the standard dual-route account.

Deep dyslexia: A case study of connectionist neuropsychology

Abstract Deep dyslexia is an acquired reading disorder marked by the Occurrence of semantic errors (e.g. reading RIVER as “ocean”). In addition, patients exhibit a number of other symptoms, including visual and morphological effects in their errors, a part-of-speech effect, and an advantage for concrete over abstract words. Deep dyslexia poses a distinct challenge for cognitive neuropsychology because there is little understanding of why such a variety of symptoms should co-occur in virtually all known patients. Hinton and Shallice (1991) replicated the co-occurrence of visual and semantic errors by lesioning a recurrent connectionist network trained to map from orthography to semantics. Although the success of their simulations is encouraging. there is little understanding of what underlying principles are responsible for them. In this paper we evaluate and, where possible, improve on the most important design decisions made by Hinton and Shallice, relating to the task, the network architecture, the trai...

Individual and developmental differences in semantic priming: empirical and computational support for a single-mechanism account of lexical processing.

Existing accounts of single-word semantic priming phenomena incorporate multiple mechanisms, such as spreading activation, expectancy-based processes, and postlexical semantic matching. The authors provide empirical and computational support for a single-mechanism distributed network account. Previous studies have found greater semantic priming for low- than for high-frequency target words as well as inhibition following unrelated primes only at long stimulus-onset asynchronies (SOAs). A series of experiments examined the modulation of these effects by individual differences in age or perceptual ability. Third-grade, 6th-grade, and college students performed a lexical-decision task on high- and low-frequency target words preceded by related, unrelated, and nonword primes. Greater priming for low-frequency targets was exhibited only by participants with high perceptual ability. Moreover, unlike the college students, the children showed no inhibition even at the long SOA. The authors provide an account of these results in terms of the properties of distributed network models and support this account with an explicit computational simulation.

Are Non-Semantic Morphological Effects Incompatible with a Distributed Connectionist Approach to Lexical Processing?.

On a distributed connectionist approach, morphology reflects a learned sensitivity to the systematic relationships among the surface forms of words and their meanings. Performance on lexical tasks should thus exhibit graded effects of both semantic and formal similarity. Although there is evidence for such effects, there are also demonstrations of morphological effects in the absence of semantic similarity (when formal similarity is controlled) in morphologically rich languages like Hebrew. Such findings are typically interpreted as being problematic for the connectionist account. To evaluate whether this interpretation is valid, we carried out simulations in which a set of morphologically related words varying in semantic transparency were embedded in either a morphologically rich or impoverished artificial language. We found that morphological priming increased with degree of semantic transparency in both languages. Critically, priming extended to semantically opaque items in the morphologically rich language (consistent with findings in Hebrew) but not in the impoverished language (consistent with findings in English). Such priming arises because the processing of all items, including opaque forms, is influenced by the degree of morphological organisation of the entire system. These findings suggest that, rather than being challenged by the occurrence of non-semantic morphological effects in morphologically rich languages, the connectionist approach may provide an explanation for the cross-linguistic differences in the occurrence of these effects.

Structure and Function in the Lexical System: Insights from Distributed Models of Word Reading and Lexical Decision

The traditional view of the lexical system stipulates word-specific representations and separate pathways for regular and exception words. An alternative approach views lexical knowledge as developing from general learning principles applied to mappings among distributed representations of written and spoken words and their meanings. On this distributed account, distinctions among words, and between words and nonwords, are not reified in the structure of the system but reflect the sensitivity of learning to the relative systematicity in the various mappings. Two computational simulations address findings that have seemed problematic for the distributed approach. Both involve a consideration of the role of semantics in normal and impaired lexical processing. The first simulation accounts for patients with impaired comprehension but intact reading in terms of individual differences in the division of labour between the semantic and phonological pathways. The second simulation demonstrates that a distributed...

Double dissociation without modularity: Evidence from connectionist neuropsychology

Many theorists assume that the cognitive system is composed of a collection of encapsulated processing components or modules, each dedicated to performing a particular cognitive function. On this view, selective impairments of cognitive tasks following brain damage, as evidenced by double dissociations, are naturally interpreted in terms of the loss of particular processing components. By contrast, the current investigation examines in detail a double dissociation between concrete and abstract work reading after damage to a connectionist network that pronounces words via meaning and yet has no separable components (Plaut & Shallice, 1993). The functional specialization in the network that gives rise to the double dissociation is not transparently related to the networks structure, as modular theories assume. Furthermore, a consideration of the distribution of effects across quantitatively equivalent individual lesions in the network raises specific concerns about the interpretation of single-case studies. The findings underscore the necessity of relating neuropsychological data to cognitive theories in the context of specific computational assumptions about how the cognitive system operates normally and after damage.

Short-Term Memory for Serial Order: A Recurrent Neural Network Model

Despite a century of research, the mechanisms underlying short-term or working memory for serial order remain uncertain. Recent theoretical models have converged on a particular account, based on transient associations between independent item and context representations. In the present article, the authors present an alternative model, according to which sequence information is encoded through sustained patterns of activation within a recurrent neural network architecture. As demonstrated through a series of computer simulations, the model provides a parsimonious account for numerous benchmark characteristics of immediate serial recall, including data that have been considered to preclude the application of recurrent neural networks in this domain. Unlike most competing accounts, the model deals naturally with findings concerning the role of background knowledge in serial recall and makes contact with relevant neuroscientific data. Furthermore, the model gives rise to numerous testable predictions that differentiate it from competing theories. Taken together, the results presented indicate that recurrent neural networks may offer a useful framework for understanding short-term memory for serial order.

Doing without schema hierarchies: a recurrent connectionist approach to normal and impaired routine sequential action.

In everyday tasks, selecting actions in the proper sequence requires a continuously updated representation of temporal context. Previous models have addressed this problem by positing a hierarchy of processing units, mirroring the roughly hierarchical structure of naturalistic tasks themselves. The present study considers an alternative framework, in which the representation of context depends on recurrent connections within a network mapping from environmental inputs to actions. The ability of this approach to account for human performance was evaluated by applying it, through simulation, to a specific everyday task. The resulting model learned to deal flexibly with a complex set of sequencing constraints, encoding contextual information at multiple time scales within a single, distributed internal representation. Degrading this representation led to errors resembling those observed both in everyday behavior and in apraxia. Analysis of the models function yielded numerous predictions relevant to both normal and apraxic performance.

SD-Squared: On the Association Between Semantic Dementia and Surface Dyslexia

Within the connectionist triangle model of reading aloud, interaction between semantic and phonological representations occurs for all words but is particularly important for correct pronunciation of lower frequency exception words. This framework therefore predicts that (a) semantic dementia, which compromises semantic knowledge, should be accompanied by surface dyslexia, a frequency-modulated deficit in exception word reading, and (b) there should be a significant relationship between the severity of semantic degradation and the severity of surface dyslexia. The authors evaluated these claims with reference to 100 observations of reading data from 51 cases of semantic dementia. Surface dyslexia was rampant, and a simple composite semantic measure accounted for half of the variance in low-frequency exception word reading. Although in 3 cases initial testing revealed a moderate semantic impairment but normal exception word reading, all of these became surface dyslexic as their semantic knowledge deteriorated further. The connectionist account attributes such cases to premorbid individual variation in semantic reliance for accurate exception word reading. These results provide a striking demonstration of the association between semantic dementia and surface dyslexia, a phenomenon that the authors have dubbed SD-squared.

Letting structure emerge: connectionist and dynamical systems approaches to cognition

Connectionist and dynamical systems approaches explain human thought, language and behavior in terms of the emergent consequences of a large number of simple noncognitive processes. We view the entities that serve as the basis for structured probabilistic approaches as abstractions that are occasionally useful but often misleading: they have no real basis in the actual processes that give rise to linguistic and cognitive abilities or to the development of these abilities. Although structured probabilistic approaches can be useful in determining what would be optimal under certain assumptions, we propose that connectionist, dynamical systems, and related approaches, which focus on explaining the mechanisms that give rise to cognition, will be essential in achieving a full understanding of cognition and development.