James A. Reggia

Empirically derived probabilities for grapheme-to-phoneme correspondences in english

Prior probabilities of graphemes and conditional probabilities for their pronunciation as specific phonemes are given based on a corpus of 17,310 English words. Phonemes are as given in recent editions ofWebster’s New Collegiate Dictionary, with minor revisions; graphemes are defined as letters or letter clusters corresponding to single phonemes. Grapheme-phoneme probabilities were derived from a revised table of frequency of occurrence of phoneme-to-grapheme correspondences generated in a study of spelling regularities (P. R. Hanna, J. S. Hanna, Hodges, & Rudorf, 1966). This quantitative descriptive information provides an index of the strength of particular grapheme-phoneme associations in English. Suggestions are made for the utilization of these probabilities as estimates of spelling/sound predictability in reading research.

Automatic discovery of self-replicating structures in cellular automata

Previous computational models of self-replication using cellular automata (CA) have been manually designed, a difficult and time-consuming process. We show here how genetic algorithms can be applied to automatically discover rules governing self-replicating structures. The main difficulty in this problem lies in the choice of the fitness evaluation technique. The solution we present is based on a multiobjective fitness function consisting of three independent measures: growth in number of components, relative positioning of components, and the multiplicity of replicants. We introduce a new paradigm for CA models with weak rotational symmetry, called orientation-insensitive input, and hypothesize that it facilitates discovery of self-replicating structures by reducing search-space sizes. Experimental yields of self-replicating structures discovered using our technique are shown to be statistically significant. The discovered self-replicating structures compare favorably in terms of simplicity with those generated manually in the past, but differ in unexpected ways. These results suggest that further exploration in the space of possible self-replicating structures will yield additional new structures. Furthermore, this research sheds light on the process of creating self-replicating structures, opening the door to future studies on the discovery of novel self-replicating molecules and self-replicating assemblers in nanotechnology.

A computational model of visual hallucinations in migraine

Cortical spreading depression is a wave of electrical and biochemical changes that spreads across the cerebral cortex. It has been hypothesized to be an important underlying cause of the visual disturbances occurring during the migraine aura, but this is difficult to test in animals or humans. We created a computational model of cortical spreading depression and found that during the wave of biochemical changes the spatial pattern of neural activity broke up into irregular patterns of lines and small patches of highly activated elements. The corresponding visual disturbances that would be produced by these patterns of neural activity resemble the hallucinations reported during the migraine aura, providing strong support for the cortical spreading depression hypothesis of migraine. The model also makes the testable prediction that these hallucinations move at an exponentially increasing speed across the visual field.

Spreading depression in focal ischemia: a computational study.

When a cerebral infarction occurs, surrounding the core of dying tissue there usually is an ischemic penumbra of nonfunctional but still viable tissue. One current but controversial hypothesis is that this penumbra tissue often eventually dies because of the metabolic stress imposed by multiple cortical spreading depression (CSD) waves, that is, by ischemic depolarizations. We describe here a computational model of CSD developed to study the implications of this hypothesis. After simulated infarction, the model displays the linear relation between final infarct size and the number of CSD waves traversing the penumbra that has been reported experimentally, although damage with each individual wave progresses nonlinearly with time. It successfully reproduces the experimental dependency of final infarct size on midpenumbra cerebral blood flow and potassium reuptake rates, and predicts a critical penumbra blood flow rate beyond which damage does not occur. The model reproduces the dependency of CSD wave propagation on N-methyl-D-aspartate activation. It also makes testable predictions about the number, velocity, and duration of ischemic CSD waves and predicts a positive correlation between the duration of elevated potassium in the infarct core and the number of CSD waves. These findings support the hypothesis that CSD waves play an important causal role in the death of ischemic penumbra tissue.

Problem solving during artificial selection of self-replicating loops

Abstract Past cellular automata models of self-replication have generally done only one thing: replicate themselves. However, it has recently been demonstrated that such self-replicating structures can be programmed to also carry out a task during the replication process. Past models of this sort have been limited in that the “program” involved is copied unchanged from parent to child, so that each generation of replicants is executing exactly the same program on exactly the same data. Here we take a different approach in which each replicant receives a distinct partial solution that is modified during replication. Under artificial selection, replicants with promising solutions proliferate while those with failed solutions are lost. We show that this approach can be applied successfully to solve an NP-complete problem, the satisfiability problem. Bounds are given on the cellular space size and time needed to solve a given problem, and simulations demonstrate that this approach works effectively. These and other recent results raise the possibility of evolving self-replicating structures that have a simulated metabolism or that carry out useful tasks.

Computational studies of lateralization of phoneme sequence generation

The mechanisms underlying cerebral lateralization of language are poorly understood. Asymmetries in the size of hemispheric regions and other factors have been suggested as possible underlying causal factors, and the corpus callosum (interhemispheric connections) has also been postulated to play a role. To examine these issues, we created a neural model consisting of paired cerebral hemispheric regions interacting via the corpus callosum. The model was trained to generate the correct sequence of phonemes for 50 monosyllabic words (simulated reading aloud) under a variety of assumptions about hemispheric asymmetries and callosal effects. After training, the ability of the full model and each hemisphere acting alone to perform this task was measured. Lateralization occurred readily toward the side having larger size, higher excitability, or higher learning-rate parameter. Lateralization appeared most readily and intensely with strongly inhibitory callosal connections, supporting past arguments that the effective functionality of the corpus callosum is inhibitory. Many of the results are interpretable as the outcome of a race to learn between the models two hemispheric regions, leading to the concept that asymmetric hemispheric plasticity is a critical common causative factor in lateralization. To our knowledge, this is the first computational model to demonstrate spontaneous lateralization of function, and it suggests that such models can be useful for understanding the mechanisms of cerebral lateralization.

Generating plausible diagnostic hypotheses with self-processing causal networks

Abstract A recently proposed connectionist methodology for diagnostic problem-solving is critically examined for its ability to construct problem solutions. A sizeable causal network (56 manifestation nodes, 26 disorder nodes, 384 causal links) served as the basis of experimental simulations. Initial results were discouraging, with less than two-thirds of simulations leading to stable solution states (equilibria). Examination of these simulation results identified a critical period during simulations, and analysis of the connectionist models activation rule during this period led to an understanding of the models non-stable oscillatory behavior. Slower decrease in the models control parameter during the critical period resulted in all simulations reaching a stable equilibrium with plausible problem solutions. As a consequence of this work, it is possible to determine more rationally a schedule for control parameter variation during problem solving, and the way is now open for real-world experimental as...

A Computational Model of Acute Focal Cortical Lesions

BACKGROUND AND PURPOSE Determining how cerebral cortex adapts to sudden focal damage is important for gaining a better understanding of stroke. In this study we used a computational model to examine the hypothesis that cortical map reorganization after a simulated infarct is critically dependent on perilesion excitability and to identify factors that influence the extent of poststroke reorganization. METHODS A previously reported artificial neural network model of primary sensorimotor cortex, controlling a simulated arm, was subjected to acute focal damage. The perilesion excitability and cortical map reorganization were measured over time and compared. RESULTS Simulated lesions to cortical regions with increased perilesion excitability were associated with a remapping of the lesioned area into the immediate perilesion cortex, where responsiveness increased with time. In contrast, when lesions caused a perilesion zone of decreased activity to appear, this zone enlarged and intensified with time, with loss of the perilesion map. Increasing the assumed extent of intracortical connections produced a wider perilesion zone of inactivity. These effects were independent of lesion size. CONCLUSIONS These simulation results suggest that functional cortical reorganization after an ischemic stroke is a two-phase process in which perilesion excitability plays a critical role.

Discovery of self-replicating structures using a genetic algorithm

Previous computational models of self-replication in cellular spaces have been manually designed, a very difficult and time-consuming process. This paper introduces the use of genetic algorithms to discover automata rules that govern emergent self-replicating processes. Given dynamically evolving automata, identification of effective fitness functions for self-replicating structures is a difficult task, and we give one solution to this problem. A model consisting of movable automata embedded in a cellular space is introduced and discussed in this context. A genetic algorithm using two fitness criteria was applied to automate rule discovery. After parameter tuning, 6 self-replicating structures consisting of 2, 3 and 4 automata were discovered over a course of 75 genetic algorithm runs. These results indicate that the fitness functions employed are effective and that genetic algorithms can be used to successfully discover rules for self-replicating structures.

A generalized LSTM-like training algorithm for second-order recurrent neural networks

The long short term memory (LSTM) is a second-order recurrent neural network architecture that excels at storing sequential short-term memories and retrieving them many time-steps later. LSTMs original training algorithm provides the important properties of spatial and temporal locality, which are missing from other training approaches, at the cost of limiting its applicability to a small set of network architectures. Here we introduce the generalized long short-term memory(LSTM-g) training algorithm, which provides LSTM-like locality while being applicable without modification to a much wider range of second-order network architectures. With LSTM-g, all units have an identical set of operating instructions for both activation and learning, subject only to the configuration of their local environment in the network; this is in contrast to the original LSTM training algorithm, where each type of unit has its own activation and training instructions. When applied to LSTM architectures with peephole connections, LSTM-g takes advantage of an additional source of back-propagated error which can enable better performance than the original algorithm. Enabled by the broad architectural applicability of LSTM-g, we demonstrate that training recurrent networks engineered for specific tasks can produce better results than single-layer networks. We conclude that LSTM-g has the potential to both improve the performance and broaden the applicability of spatially and temporally local gradient-based training algorithms for recurrent neural networks.