Andrew L. Cohen

Model evaluation using grouped or individual data

Analyzing the data of individuals has several advantages over analyzing the data combined across the individuals (the latter we term group analysis): Grouping can distort the form of data, and different individuals might perform the task using different processes and parameters. These factors notwithstanding, we demonstrate conditions in which group analysis outperforms individual analysis. Such conditions include those in which there are relatively few trials per subject per condition, a situation that sometimes introduces distortions and biases when models are fit and parameters are estimated. We employed a simulation technique in which data were generated from each of two known models, each with parameter variation across simulated individuals. We examined how well the generating model and its competitor each fared in fitting (both sets of) the data, using both individual and group analysis. We examined the accuracy of model selection (the probability that the correct model would be selected by the analysis method). Trials per condition and individuals per experiment were varied systematically. Three pairs of cognitive models were compared: exponential versus power models of forgetting, generalized context versus prototype models of categorization, and the fuzzy logical model of perception versus the linear integration model of information integration. We show that there are situations in which small numbers of trials per condition cause group analysis to outperform individual analysis. Additional tables and figures may be downloaded from the Psychonomic Society Archive of Norms, Stimuli, and Data, www.psychonomic.org/archive.

Prototype and exemplar accounts of category learning and attentional allocation: a reassessment.

In a recent article. J. P. Minda and J. D. Smith (2002; see record 2002-00620-002) argued that an exemplar model provided worse quantitative fits than an alternative prototype model to individual subject data from the classic D. L. Medin and M. M. Schaffer (1978) 5/4 categorization paradigm. In addition, they argued that the exemplar model achieved its fits by making untenable assumptions regarding how observers distribute their attention. In this article, we demonstrate that when the models are equated in terms of their response-rule flexibility, the exemplar model provides a substantially better account of the categorization data than does a prototype or mixed model. In addition, we point to shortcomings in the attention-allocation analyses conducted by J. P. Minda and J. D. Smith (2002). When these shortcomings are corrected, we find no evidence that challenges the attention-allocation assumptions of the exemplar model.

Evaluating models of remember-know judgments: Complexity, mimicry, and discriminability

Remember-know judgments provide additional information in recognition memory tests, but the nature of this information and the attendant decision process are in dispute. Competing models have proposed that remember judgments reflect a sum of familiarity and recollective information (the one-dimensional model), are based on a difference between these strengths (STREAK), or are purely recollective (the dual-process model). A choice among these accounts is sometimes made by comparing the precision of their fits to data, but this strategy may be muddied by differences in model complexity: Some models that appear to provide good fits may simply be better able to mimic the data produced by other models. To evaluate this possibility, we simulated data with each of the models in each of three popular remember-know paradigms, then fit those data to each of the models. We found that the one-dimensional model is generally less complex than the others, but despite this handicap, it dominates the others as the best-fitting model. For both reasons, the one-dimensional model should be preferred. In addition, we found that some empirical paradigms are ill-suited for distinguishing among models. For example, data collected by soliciting remember/know/new judgments—that is, the trinary task—provide a particularly weak ground for distinguishing models. Additional tables and figures may be downloaded from the Psychonomic Society’s Archive of Norms, Stimuli, and Data, at www.psychonomic.org/archive.

An extension of the exemplar-based random-walk model to separable-dimension stimuli

An extension of Nosofsky and Palmeris (Psychol. Rev. 104 (1997a) 266) exemplar-based random-walk (EBRW) model of categorization is presented as a model of the time course of categorization of separable-dimension stimuli. Nosofsky and Palmeri (1997a) assumed that the perceptual encoding of all stimuli was identical. However, in the current model, we assume as in Lamberts (J. Exp. Psychol: General 124 (1995) 161) that the inclusion of individual stimulus dimension into the similarity calculations is a stochastic process with the probability of inclusion based or the perceptual salience of the dimensions. Thus, the exemplars that enter into the random-walk changes dynamically during the time course of processing. This model is implemented as a Markov chain. Its predictions are compared with alternative models in a speeded categorization experiment with separable-dimension stimuli.

Category variability, exemplar similarity, and perceptual classification.

Experiments were conducted in which observers learned to classify simple perceptual stimuli into low-variability and high-variability categories. Similarities between objects were measured in independent psychological-scaling tasks. The results showed that observers classified transfer stimuli into the high-variability categories with greater probability than was predicted by a baseline version of an exemplar-similarity model. Qualitative evidence for the role of category variability on perceptual classification, which could not be explained in terms of the baseline exemplar-similarity model, was obtained as well. Possible accounts of the effects of category variability are considered in the General Discussion section.

Contributions of invariants, heuristics, and exemplars to the visual perception of relative mass

Some potential contributions of invariants, heuristics, and exemplars to the perception of dynamic properties in the colliding balls task were explored. On each trial, an observer is asked to determine the heavier of 2 colliding balls. The invariant approach assumes that people can learn to detect complex visual patterns that reliably specify which ball is heavier. The heuristic approach assumes that observers only have access to simple motion cues. The exemplar-based approach assumes that people store particular exemplars of collisions in memory, which are later retrieved to perform the task. Mathematical models of these theories are contrasted in 2 experiments. Observers may use more than 1 strategy to determine relative mass. Although observers can learn to detect and use invariants, they may rely on either heuristics before the invariant has been learned or exemplars when memory demands and similarity relations allow.

Using graphical models to infer multiple visual classification features.

This paper describes a new model for human visual classification that enables the recovery of image features that explain performance on different visual classification tasks. Unlike some common methods, this algorithm does not explain performance with a single linear classifier operating on raw image pixels. Instead, it models classification as the result of combining the output of multiple feature detectors. This approach extracts more information about human visual classification than has been previously possible with other methods and provides a foundation for further exploration.

Bayesian statistical analysis of circadian oscillations in fibroblasts

Precise determination of a noisy biological oscillators period from limited experimental data can be challenging. The common practice is to calculate a single number (a point estimate) for the period of a particular time course. Uncertainty is inherent in any statistical estimator applied to noisy data, so our confidence in such point estimates depends on the quality and quantity of the data. Ideally, a period estimation method should both produce an accurate point estimate of the period and measure the uncertainty in that point estimate. A variety of period estimation methods are known, but few assess the uncertainty of the estimates, and a measure of uncertainty is rarely reported in the experimental literature. We compare the accuracy of point estimates using six common methods, only one of which can also produce uncertainty measures. We then illustrate the advantages of a new Bayesian method for estimating period, which outperforms the other six methods in accuracy of point estimates for simulated data and also provides a measure of uncertainty. We apply this method to analyze circadian oscillations of gene expression in individual mouse fibroblast cells and compute the number of cells and sampling duration required to reduce the uncertainty in period estimates to a desired level. This analysis indicates that, due to the stochastic variability of noisy intracellular oscillators, achieving a narrow margin of error can require an impractically large number of cells. In addition, we use a hierarchical model to determine the distribution of intrinsic cell periods, thereby separating the variability due to stochastic gene expression within each cell from the variability in period across the population of cells.

Software for the automatic correction of recorded eye fixation locations in reading experiments

Because the recorded location of an eyetracking fixation is not a perfect measure of the actual fixated location, the recorded fixation locations must be adjusted before analysis. Fixations are typically corrected manually. Making such changes, however, is time-consuming and necessarily involves a subjective component. The goal of this article is to introduce software to automate parts of the correction process. The initial focus is on the correction of vertical locations and the removal of outliers and ambiguous fixations in reading experiments. The basic idea behind the algorithm is to use linear regression to assign each fixation to a text line and to identify outliers. The freely available software is implemented as a function, , written in R.

Visual noise reveals category representations

How are categories represented in human memory? Exemplar models assume that a category is represented by individual instances from that category that have been experienced. More generally, a category might be represented by multiple templates stored in memory. A new item is classified according to its similarity to these templates.Prototype models represent a category with a single summary abstraction (i.e., a single template), often the central tendency of the experienced items. A new item is classified according to its similarity to these category prototypes. Here, we show how a technique for correlating observers’ responses with external noise can be used not only to distinguish single- from multiple-template representations, but also to induce the form of these templates. The technique is applied to two tasks requiring categorization of simple visual patterns; the results demonstrate that observers used multiple traces to represent their categories, and thus highlight the procedure’s potential for use in more complex settings.