Mark A. Pitt

Applying Occam’s razor in modeling cognition: A Bayesian approach

In mathematical modeling of cognition, it is important to have well-justified criteria for choosing among differing explanations (i.e., models) of observed data. This paper introduces a Bayesian model selection approach that formalizes Occam’s razor, choosing the simplest model that describes the data well. The choice of a model is carried out by taking into account not only the traditional model selection criteria (i.e., a model’s fit to the data and the number of parameters) but also the extension of the parameter space, and, most importantly, the functional form of the model (i.e., the way in which the parameters are combined in the model’s equation). An advantage of the approach is that it can be applied to the comparison of non-nested models as well as nested ones. Application examples are presented and implications of the results for evaluating models of cognition are discussed.

When a good fit can be bad

How should we select among computational models of cognition? Although it is commonplace to measure how well each model fits the data, this is insufficient. Good fits can be misleading because they can result from properties of the model that have nothing to do with it being a close approximation to the cognitive process of interest (e.g. overfitting). Selection methods are introduced that factor in these properties when measuring fit. Their success in outperforming standard goodness-of-fit measures stems from a focus on measuring the generalizability of a models data-fitting abilities, which should be the goal of model selection.

The Buckeye corpus of conversational speech: labeling conventions and a test of transcriber reliability

This paper describes the Buckeye corpus of spontaneous American English speech, a 307,000-word corpus containing the speech of 40 talkers from central Ohio, USA. The method used to elicit and record the speech is described, followed by a description of the protocol that was developed to phonemically label what talkers said. The results of a test of labeling consistency are then presented. The corpus will be made available to the scientific community when labeling is completed.

Phonological processes and the perception of phonotactically illegal consonant clusters

The perception of consonant clusters that are phonotactically illegal word initially in English (e.g., /tl/, /sr/) was investigated to determine whether listeners’ phonological knowledge of the language influences speech processing. Experiment 1 examined whether the phonotactic context effect (Massaro & Cohen, 1983), a bias toward hearing illegal sequences (e.g., /tl/) as legal (e.g., /tr/), is more likely due to knowledge of the legal phoneme combinations in English or to a frequency effect. In Experiment 2, Experiment 1 was repeated with the clusters occurring word medially to assess whether phonotactic rules of syllabification modulate the phonotactic effect. Experiment 3 examined whether vowel epenthesis, another phonological process, might also affect listeners’ perception of illegal sequences as legal by biasing them to hear a vowel between the consonants of the cluster (e.g., /talee/). Results suggest that knowledge of the phonotactically permissible sequences in English can affect phoneme processing in multiple ways.

The use of rhythm in attending to speech

Three experiments examined attentional allocation during speech processing to determine whether listeners capitalize on the rhythmic nature of speech and attend more closely to stressed than to unstressed syllables. Ss performed a phoneme monitoring task in which the target phoneme occurred on a syllable that was either predicted to be stressed or unstressed by the context preceding the target word. Stimuli were digitally edited to eliminate the local acoustic correlates of stress. A sentential context and a context composed of word lists, in which all the words had the same stress pattern, were used. In both cases, the results suggest that attention may be preferentially allocated to stressed syllables during speech processing. However, a normal sentence context may not provide strong predictive cues to lexical stress, limiting the use of the attentional focus.

Global model analysis by parameter space partitioning

To model behavior, scientists need to know how models behave. This means learning what other behaviors a model can produce besides the one generated by participants in an experiment. This is a difficult problem because of the complexity of psychological models (e.g., their many parameters) and because the behavioral precision of models (e.g., interval-scale performance) often mismatches their testable precision in experiments, where qualitative, ordinal predictions are the norm. Parameter space partitioning is a solution that evaluates model performance at a qualitative level. There exists a partition on the models parameter space that divides it into regions that correspond to each data pattern. Three application examples demonstrate its potential and versatility for studying the global behavior of psychological models.

Toward an explanation of the power law artifact: Insights from response surface analysis

The power law (y =ax−b) has been shown to provide a good description of data collected in a wide range of fields in psychology. R. B. Anderson and Tweney (1997) suggested that the model’s data-fitting success may in part be artifactual, caused by a number of factors, one of which is the use of improper data averaging methods. The present paper follows up on their work and explains causes of the power law artifact. A method for studying the geometric relations among responses generated by mathematical models is introduced that shows the artifact is a result of the combined contributions of three factors: arithmetic averaging of data that are generated from a nonlinear model in the presence of individual differences.

Altering Context Speech Rate Can Cause Words to Appear or Disappear

Speech is produced over time, and this makes sensitivity to timing between speech events crucial for understanding language. Two experiments investigated whether perception of function words (e.g., or, are) is rate dependent in casual speech, which often contains phonetic segments that are spectrally quite reduced. In Experiment 1, talkers spoke sentences containing a target function word; slowing talkers’ speech rate around this word caused listeners to perceive sentences as lacking the word (e.g., leisure or time was perceived as leisure time). In Experiment 2, talkers spoke matched sentences lacking a function word; speeding talkers’ speech rate around the region in which the function word had been embedded in Experiment 1 caused listeners to perceive a function word that was never spoken (e.g., leisure time was perceived as leisure or time). The results suggest that listeners formed expectancies based on speech rate, and these expectancies influenced the number of words and word boundaries perceived. These findings may help explain the robustness of speech recognition when speech signals are distorted (e.g., because of a casual speaking style).

An empirical and meta-analytic evaluation of the phoneme identification task.

Recent studies that used Ganongs (1980) identification task have produced discrepant results. The present study sought to resolve these discrepancies by examining the influence of methodological factors on phoneme identification and differences in data analysis techniques. Three factors were examined across 2 experiments: position of target phoneme, phonetic contrast, and 2 task conditions in which stimulus quality (S/N ratio) or cognitive load varied. A meta-analysis was then performed on the results from all identification studies, including the present one, in an effort to obtain additional insight on factors that influence the task. The experiments and meta-analysis identified the importance of several methodological factors in affecting identification, most notably position of the target phoneme.

Adaptive design optimization: A mutual information-based approach to model discrimination in cognitive science

Discriminating among competing statistical models is a pressing issue for many experimentalists in the field of cognitive science. Resolving this issue begins with designing maximally informative experiments. To this end, the problem to be solved in adaptive design optimization is identifying experimental designs under which one can infer the underlying model in the fewest possible steps. When the models under consideration are nonlinear, as is often the case in cognitive science, this problem can be impossible to solve analytically without simplifying assumptions. However, as we show in this letter, a full solution can be found numerically with the help of a Bayesian computational trick derived from the statistics literature, which recasts the problem as a probability density simulation in which the optimal design is the mode of the density. We use a utility function based on mutual information and give three intuitive interpretations of the utility function in terms of Bayesian posterior estimates. As a proof of concept, we offer a simple example application to an experiment on memory retention.