Justin A. MacDonald

Prevalence-based decisions undermine visual search

In visual search, observers make decisions about the presence or absence of a target based on their perception of a target during search. The present study investigated whether decisions can be based on observers’ expectation rather than perception of a target. In Experiment 1, participants were allowed to make target-present responses by clicking on the target or, if the target was not perceived, a target-present button. Participants used the target-present button option more frequently in difficult search trials and when target prevalence was high. Experiment 2 and 3 employed a difficult search task that encouraged the use of prevalence-based decisions. Target presence was reported faster when target prevalence was high, indicating that decisions were, in part, cognitive, and not strictly perceptual. A similar pattern of responses were made even when no targets appeared in the search (Experiment 3). The implication of these prevalence-based decisions for visual search models is discussed.

Using PPT to account for randomness in perception

According to many theories of decision making, of which signal detection theory is the most prominent, randomness is the main factor responsible for imperfect performance. These theories imply that correcting for attenuation due to randomness should result in perfect scores as long as the participants use nonextreme decision criteria. On the basis of a recent advance termed potential performance theory (Trafimow & Rice, Psychological Review 115:447-462, 2008), we performed auditory and visual detection experiments and corrected the scores for attenuation. Most participants in both experiments tended to perform at a less-than-perfect level, even after their scores were corrected. The findings demonstrate that at least one systematic factor influences detection that is not included in signal detection theory.

Performing Inferential Statistics Prior to Data Collection.

Typically, in education and psychology research, the investigator collects data and subsequently performs descriptive and inferential statistics. For example, a researcher might compute group means and use the null hypothesis significance testing procedure to draw conclusions about the populations from which the groups were drawn. We propose an alternative inferential statistical procedure that is performed prior to data collection rather than afterwards. To use this procedure, the researcher specifies how close she or he desires the group means to be to their corresponding population means and how confident she or he wishes to be that this actually is so. We derive an equation that provides researchers with a way to determine the sample size needed to meet the specifications concerning closeness and confidence, regardless of the number of groups.

Decision criteria do not shift: Commentary on Mueller and Weidemann (2008)

The effects of base rates and payoffs on the shapes of rating receiver operating characteristic curves are inconsistent with the basic assumptions of signal detection theory (SDT), in particular the notion of a shifting decision criterion. Mueller and Weidemann (2008) propose that these unexpected phenomena are not due to problems with the decision- criterion construct but are instead due to two compounded effects: instability of the decision criterion across trials, and even greater instability in the flanking criteria that determine which confidence rating will be reported. There are several problems with the authors’ decision-noise hypothesis. First, even if their hypothesis about decision noise were taken for granted, the key feature of the ratings data that rejects the SDT model would remain a mystery. Second, the same violations of SDT that are exhibited in the ratings paradigm are also exhibited in the yes-no detection task when response time is substituted for confidence as a basis for analysis. Finally, the decision-noise hypothesis predicts that sensitivity will increase when one source of this variation-the response on a previous trial-is controlled for. This prediction was consistently violated in both the yes-no and ratings conditions of Mueller and Weidemann’s experiment. In an Addendum, we respond to Weidemann and Mueller’s (2008) reply to this Comment.

How often is prep close to the true replication probability

Largely due to dissatisfaction with the standard null hypothesis significance testing procedure, researchers have begun to consider alternatives. For example, Killeen (2005a) has argued that researchers should calculate prep that is purported to indicate the probability that, if the experiment in question were replicated, the obtained finding would be in the same direction as the original finding. However, Killeen also seems to indicate that rather than being the probability of replication, prep is actually the probability of obtaining a finding whereby the experimental group mean exceeds the control group mean. Our goal was to determine the relative frequency with which obtained prep statistics are close to true replication probabilities. Regardless of which way prep is defined, our simulations show that it is unlikely to be close to the true value unless both the population effect magnitude and the sample size are uncommonly large. The definitional problem in combination with the inaccuracy under either interpretation, constitutes an important challenge for those who espouse the routine computation of prep statistics.

Flexibility within working memory and the focus of attention for sequential verbal information does not depend on active maintenance

The focus of attention seems to be a static element within working memory when verbal information is serially presented, unless additional time is available for processing or active maintenance. Experiment 1 manipulated the reward associated with early and medial list positions in a probe recognition paradigm and found evidence that these nonterminal list positions could be retrieved faster and more accurately if participants were appropriately motivated—without additional time for processing or active maintenance. Experiment 2 used articulatory suppression and demonstrated that the underlying maintenance mechanism cannot be attributed to rehearsal, leaving attentional refreshing as the more likely mechanism. These findings suggest that the focus of attention within working memory can flexibly maintain nonterminal early and medial list representations at the expense of other list representations even when there is not additional time for processing or active maintenance. Maintenance seems to be accomplished through an attentional refreshing mechanism.

Self-construal priming affects speed of retrieval from short-term memory.

We investigated the effects of collective or individual self-construal priming on recall in a short-term memory (STM) task. We primed participants to either their individual or their collective self-construals or a neutral control condition. Participants then completed a STM retrieval task using either random or patterned digit strings. Findings revealed that priming an individual self-construal resulted in faster retrieval of information from STM for both stimulus types. These results indicate that individual self-accessibility improves retrieval speed of digits from STM, regardless of set configuration. More broadly, the present findings extend prior research by adding further evidence of the effects of self-construal priming on cognitive information processing.

An investigation of the accuracy of standardized path coefficients.

ABSTRACT Given the popular and ever-increasing use of path analytic research paradigms in the social sciences, it is desirable to conduct an investigation into the accuracy of the standardized path coefficients that are often the end-product of these paradigms. In pursuit of this goal, population parameters were preset concerning the correlations between all of the variables and their reliability coefficients. Based on these parameters, thousands of experiments were generated with varying numbers of cases (n). For each experiment, at each level of n, standard path analyses were conducted, and standardized path coefficients were obtained. These standardized path coefficients were then compared against the population path coefficients on which the simulations were based to determine their accuracy. The findings indicate mixed evidence for the accuracy of path analysis research paradigms.

Using the ideal observer to predict performance in perceptual tasks: An example from the auditory temporal masking domain

This article provides a demonstration of an analytical technique that can be used to investigate the causes of perceptual phenomena. The technique is based on the concept of the ideal observer, an optimal signal classifier that makes decisions that maximize the probability of a correct response. To demonstrate the technique, an analysis was conducted to investigate the role of the auditory periphery in the production of temporal masking effects. The ideal observer classified output from four models of the periphery. Since the ideal observer is the best of all possible observers, if it demonstrates masking effects, then all other observers must as well. If it does not demonstrate masking effects, then nothing about the periphery requires masking to occur, and therefore masking would occur somewhere else. The ideal observer exhibited several forward masking effects but did not exhibit backward masking, implying that the periphery has a causal role in forward but not backward masking. A general discussion of the strengths of the technique and supplementary equations are also included.

Simulating the Fidelity of Data for Large Stimulus Set Sizes and Variable Dimension Estimation in Multidimensional Scaling

Multidimensional scaling (MDS) is a statistical technique commonly used to model the psychological similarity among sets of stimulus items. Typically, MDS has been used with relatively small stimulus sets (30 items or fewer), in part due to the laborious nature of computational analysis and data collection. Modern computing power and newly advanced techniques for speeding data collection have made it possible to conduct MDS with many more stimuli. However, it is as yet unclear if MDS is as well-equipped to model the similarity of large stimulus sets as it is for more modest ones. Here, we conducted 337,500 simulation experiments, wherein hypothetical “true” MDS spaces were created, along with error-perturbed data from simulated “participants.” We examined the fidelity with which the spaces resulting from our “participants” captured the organization of the “true” spaces, as a function of item set size, amount of error in the data (i.e., noise), and dimensionality estimation. We found that although higher set sizes decrease model fit (i.e., they produce increased “stress”), they largely tended to increase determinacy of MDS spaces. These results are predicated, however, on the appropriate estimation of dimensionality of the MDS space. We argue that it is not only reasonable to adopt large stimulus set sizes but tends to be advantageous to do so. Applying MDS to larger sets is appealing, as it affords researchers greater flexibility in stimulus selection, more opportunity for exploration of their stimuli, and a higher likelihood that observed relationships are not due to stimulus-specific idiosyncrasies.