Gabriel Recchia

More data trumps smarter algorithms: Comparing pointwise mutual information with latent semantic analysis

Computational models of lexical semantics, such as latent semantic analysis, can automatically generate semantic similarity measures between words from statistical redundancies in text. These measures are useful for experimental stimulus selection and for evaluating a model’s cognitive plausibility as a mechanism that people might use to organize meaning in memory. Although humans are exposed to enormous quantities of speech, practical constraints limit the amount of data that many current computational models can learn from. We follow up on previous work evaluating a simple metric of pointwise mutual information. Controlling for confounds in previous work, we demonstrate that this metric benefits from training on extremely large amounts of data and correlates more closely with human semantic similarity ratings than do publicly available implementations of several more complex models. We also present a simple tool for building simple and scalable models from large corpora quickly and efficiently.

The Shape of Action

How do people understand the everyday, yet intricate, behaviors that unfold around them? In the present research, we explored this by presenting viewers with self-paced slideshows of everyday activities and recording looking times, subjective segmentation (breakpoints) into action units, and slide-to-slide physical change. A detailed comparison of the joint time courses of these variables showed that looking time and physical change were locally maximal at breakpoints and greater for higher level action units than for lower level units. Even when slideshows were scrambled, breakpoints were regarded longer and were more physically different from ordinary moments, showing that breakpoints are distinct even out of context. Breakpoints are bridges: from one action to another, from one level to another, and from perception to conception.

The Role of Semantic Diversity in Lexical Organization

Recent research has challenged the notion that word frequency is the organizing principle underlying lexical access, pointing instead to the number of contexts that a word occurs in (Adelman, Brown, & Quesada, 2006). Counting contexts gives a better quantitative fit to human lexical decision and naming data than counting raw occurrences of words. However, this approach ignores the information redundancy of the contexts in which the word occurs, a factor we refer to as semantic diversity. Using both a corpus-based study and a controlled artificial language experiment, we demonstrate the importance of contextual redundancy in lexical access, suggesting that contextual repetitions in language only increase a words memory strength if the repetitions are accompanied by a modulation in semantic context. We introduce a cognitive process mechanism to explain the pattern of behaviour by encoding the words context relative to the information redundancy between the current context and the words current memory representation. The model gives a better account of identification latency data than models based on either raw frequency or document count, and also produces a better-organized space to simulate semantic similarity.

The semantic richness of abstract concepts.

We contrasted the predictive power of three measures of semantic richness—number of features (NFs), contextual dispersion (CD), and a novel measure of number of semantic neighbors (NSN)—for a large set of concrete and abstract concepts on lexical decision and naming tasks. NSN (but not NF) facilitated processing for abstract concepts, while NF (but not NSN) facilitated processing for the most concrete concepts, consistent with claims that linguistic information is more relevant for abstract concepts in early processing. Additionally, converging evidence from two datasets suggests that when NSN and CD are controlled for, the features that most facilitate processing are those associated with a concepts physical characteristics and real-world contexts. These results suggest that rich linguistic contexts (many semantic neighbors) facilitate early activation of abstract concepts, whereas concrete concepts benefit more from rich physical contexts (many associated objects and locations).

Toward a scalable holographic word-form representation

Phenomena in a variety of verbal tasks—for example, masked priming, lexical decision, and word naming—are typically explained in terms of similarity between word-forms. Despite the apparent commonalities between these sets of phenomena, the representations and similarity measures used to account for them are not often related. To show how this gap might be bridged, we build on the work of Hannagan, Dupoux, and Christophe, Cognitive Science 35:79-118, (2011) to explore several methods of representing visual word-forms using holographic reduced representations and to evaluate them on their ability to account for a wide range of effects in masked form priming, as well as data from lexical decision and word naming. A representation that assumes that word-internal letter groups are encoded relative to word-terminal letter groups is found to predict qualitative patterns in masked priming, as well as lexical decision and naming latencies. We then show how this representation can be integrated with the BEAGLE model of lexical semantics (Jones & Mewhort, Psychological Review 114:1–37, 2007) to enable the model to encompass a wider range of verbal tasks.

Projected Impact of Dengue Vaccination in Yucatán, Mexico.

Dengue vaccines will soon provide a new tool for reducing dengue disease, but the effectiveness of widespread vaccination campaigns has not yet been determined. We developed an agent-based dengue model representing movement of and transmission dynamics among people and mosquitoes in Yucatán, Mexico, and simulated various vaccine scenarios to evaluate effectiveness under those conditions. This model includes detailed spatial representation of the Yucatán population, including the location and movement of 1.8 million people between 375,000 households and 100,000 workplaces and schools. Where possible, we designed the model to use data sources with international coverage, to simplify re-parameterization for other regions. The simulation and analysis integrate 35 years of mild and severe case data (including dengue serotype when available), results of a seroprevalence survey, satellite imagery, and climatological, census, and economic data. To fit model parameters that are not directly informed by available data, such as disease reporting rates and dengue transmission parameters, we developed a parameter estimation toolkit called AbcSmc, which we have made publicly available. After fitting the simulation model to dengue case data, we forecasted transmission and assessed the relative effectiveness of several vaccination strategies over a 20 year period. Vaccine efficacy is based on phase III trial results for the Sanofi-Pasteur vaccine, Dengvaxia. We consider routine vaccination of 2, 9, or 16 year-olds, with and without a one-time catch-up campaign to age 30. Because the durability of Dengvaxia is not yet established, we consider hypothetical vaccines that confer either durable or waning immunity, and we evaluate the use of booster doses to counter waning. We find that plausible vaccination scenarios with a durable vaccine reduce annual dengue incidence by as much as 80% within five years. However, if vaccine efficacy wanes after administration, we find that there can be years with larger epidemics than would occur without any vaccination, and that vaccine booster doses are necessary to prevent this outcome.

Reproducing affective norms with lexical co-occurrence statistics: Predicting valence, arousal, and dominance.

Human ratings of valence, arousal, and dominance are frequently used to study the cognitive mechanisms of emotional attention, word recognition, and numerous other phenomena in which emotions are hypothesized to play an important role. Collecting such norms from human raters is expensive and time consuming. As a result, affective norms are available for only a small number of English words, are not available for proper nouns in English, and are sparse in other languages. This paper investigated whether affective ratings can be predicted from length, contextual diversity, co-occurrences with words of known valence, and orthographic similarity to words of known valence, providing an algorithm for estimating affective ratings for larger and different datasets. Our bootstrapped ratings achieved correlations with human ratings on valence, arousal, and dominance that are on par with previously reported correlations across gender, age, education and language boundaries. We release these bootstrapped norms for 23,495 English words.

Effect size matters: the role of language statistics and perceptual simulation in conceptual processing

The cognitive science literature increasingly demonstrates that perceptual representations are activated during conceptual processing. Such findings suggest that the debate on whether conceptual processing is predominantly symbolic or perceptual has been resolved. However, studies too frequently provide evidence for perceptual simulations without addressing whether other factors explain dependent variables as well, and if so, to what extent. The current paper examines effect sizes computed from 126 experiments in 51 published embodied cognition studies to clarify the conditions under which perceptual simulations are most important. Results showed that effects of language statistics tend to be as large or larger than those of perceptual stimulation. Moreover, factors that can be associated with immediate processing (button press, word processing) tend to reduce the effect size of perceptual simulation. These findings are considered in respect to the Symbol Interdependency Hypothesis, which argues that language encodes perceptual information, with language statistics explaining quick, good-enough representations and perceptual simulation explaining more effortful, detailed representations.

Encoding sequential information in semantic space models: comparing holographic reduced representation and random permutation

Circular convolution and random permutation have each been proposed as neurally plausible binding operators capable of encoding sequential information in semantic memory. We perform several controlled comparisons of circular convolution and random permutation as means of encoding paired associates as well as encoding sequential information. Random permutations outperformed convolution with respect to the number of paired associates that can be reliably stored in a single memory trace. Performance was equal on semantic tasks when using a small corpus, but random permutations were ultimately capable of achieving superior performance due to their higher scalability to large corpora. Finally, “noisy” permutations in which units are mapped to other units arbitrarily (no one-to-one mapping) perform nearly as well as true permutations. These findings increase the neurological plausibility of random permutations and highlight their utility in vector space models of semantics.

Graph‐Theoretic Properties of Networks Based on Word Association Norms: Implications for Models of Lexical Semantic Memory

We compared the ability of three different contextual models of lexical semantic memory (BEAGLE, Latent Semantic Analysis, and the Topic model) and of a simple associative model (POC) to predict the properties of semantic networks derived from word association norms. None of the semantic models were able to accurately predict all of the network properties. All three contextual models over-predicted clustering in the norms, whereas the associative model under-predicted clustering. Only a hybrid model that assumed that some of the responses were based on a contextual model and others on an associative network (POC) successfully predicted all of the network properties and predicted a words top five associates as well as or better than the better of the two constituent models. The results suggest that participants switch between a contextual representation and an associative network when generating free associations. We discuss the role that each of these representations may play in lexical semantic memory. Concordant with recent multicomponent theories of semantic memory, the associative network may encode coordinate relations between concepts (e.g., the relation between pea and bean, or between sparrow and robin), and contextual representations may be used to process information about more abstract concepts.