Frederik Aust

Seriousness checks are useful to improve data validity in online research

Nonserious answering behavior increases noise and reduces experimental power; it is therefore one of the most important threats to the validity of online research. A simple way to address the problem is to ask respondents about the seriousness of their participation and to exclude self-declared nonserious participants from analysis. To validate this approach, a survey was conducted in the week prior to the German 2009 federal election to the Bundestag. Serious participants answered a number of attitudinal and behavioral questions in a more consistent and predictively valid manner than did nonserious participants. We therefore recommend routinely employing seriousness checks in online surveys to improve data validity.

Many analysts, one dataset: Making transparent how variations in analytical choices affect results

Twenty-nine teams involving 61 analysts used the same data set to address the same research question: whether soccer referees are more likely to give red cards to dark-skin-toned players than to light-skin-toned players. Analytic approaches varied widely across the teams, and the estimated effect sizes ranged from 0.89 to 2.93 (Mdn = 1.31) in odds-ratio units. Twenty teams (69%) found a statistically significant positive effect, and 9 teams (31%) did not observe a significant relationship. Overall, the 29 different analyses used 21 unique combinations of covariates. Neither analysts’ prior beliefs about the effect of interest nor their level of expertise readily explained the variation in the outcomes of the analyses. Peer ratings of the quality of the analyses also did not account for the variability. These findings suggest that significant variation in the results of analyses of complex data may be difficult to avoid, even by experts with honest intentions. Crowdsourcing data analysis, a strategy in which numerous research teams are recruited to simultaneously investigate the same research question, makes transparent how defensible, yet subjective, analytic choices influence research results.

Subliminal influence on preferences? A test of evaluative conditioning for brief visual conditioned stimuli using auditory unconditioned stimuli

In the field of evaluative conditioning (EC), two opposing theories—propositional single-process theory versus dual-process theory—are currently being discussed in the literature. The present set of experiments test a crucial prediction to adjudicate between these two theories: Dual-process theory postulates that evaluative conditioning can occur without awareness of the contingency between conditioned stimulus (CS) and unconditioned stimulus (US); in contrast, single-process propositional theory postulates that EC requires CS-US contingency awareness. In a set of three studies, we experimentally manipulate contingency awareness by presenting the CSs very briefly, thereby rendering it unlikely to be processed consciously. We address potential issues with previous studies on EC with subliminal or near-threshold CSs that limited their interpretation. Across two experiments, we consistently found an EC effect for CSs presented for 1000 ms and consistently failed to find an EC effect for briefly presented CSs. In a third pre-registered experiment, we again found evidence for an EC effect with CSs presented for 1000 ms, and we found some indication for an EC effect for CSs presented for 20 ms.

From theories to models to predictions: A Bayesian model comparison approach

ABSTRACT A key goal in research is to use data to assess competing hypotheses or theories. An alternative to the conventional significance testing is Bayesian model comparison. The main idea is that competing theories are represented by statistical models. In the Bayesian framework, these models then yield predictions about data even before the data are seen. How well the data match the predictions under competing models may be calculated, and the ratio of these matches – the Bayes factor – is used to assess the evidence for one model compared to another. We illustrate the process of going from theories to models and to predictions in the context of two hypothetical examples about how exposure to media affects attitudes toward refugees.

Incremental validity of Useful Field of View subtests for the prediction of Instrumental Activities of Daily Living

ABSTRACT Introduction: The Useful Field of View Test (UFOV®) is a cognitive measure that predicts older adults’ ability to perform a range of everyday activities. However, little is known about the individual contribution of each subtest to these predictions, and the underlying constructs of UFOV performance remain a topic of debate. Method: We investigated the incremental validity of UFOV subtests for the prediction of Instrumental Activities of Daily Living (IADL) performance in two independent datasets, the SKILL (n = 828) and ACTIVE (n = 2426) studies. We then explored the cognitive and visual abilities assessed by UFOV using a range of neuropsychological and vision tests administered in the SKILL study. Results: In the four subtest variant of UFOV, only Subtests 2 and 3 consistently made independent contributions to the prediction of IADL performance across three different behavioral measures. In all cases, the incremental validity of UFOV Subtests 1 and 4 was negligible. Furthermore, we found that UFOV was related to processing speed, general nonspeeded cognition, and visual function; the omission of Subtests 1 and 4 from the test score did not affect these associations. Conclusions: UFOV Subtests 1 and 4 appear to be of limited use to predict IADL and possibly other everyday activities. Future research should investigate whether shortening UFOV by omitting these subtests is a reliable and valid assessment approach.

A memory-based judgment account of expectancy-liking dissociations in evaluative conditioning

Evaluative conditioning (EC) is a change in liking of neutral conditioned stimuli (CS) following pairings with positive or negative stimuli (unconditioned stimulus, US). A dissociation has been reported between US expectancy and CS evaluation in extinction learning: When CSs are presented alone subsequent to CS-US pairings, participants cease to expect USs but continue to exhibit EC effects. This dissociation is typically interpreted as demonstration that EC is resistant to extinction, and consequently, that EC is driven by a distinct learning process. We tested whether expectancy-liking dissociations are instead caused by different judgment strategies afforded by the dependent measures: CS evaluations are by default integrative judgments—summaries of large portions of the learning history—whereas US expectancy reflects momentary judgments that focus on recent events. In a counterconditioning and two extinction experiments, we eliminated the expectancy-liking dissociation by inducing nondefault momentary evaluative judgments, and demonstrated a reversed dissociation when we additionally induced nondefault integrative expectancy judgments. Our findings corroborated a priori predictions derived from the formal memory model MINERVA 2. Hence, dissociations between US expectancy and CS evaluation are consistent with a single-process learning model; they reflect different summaries of the learning history.

Operationalization of Learned Carelessness An Experimental Approach

The theory of learned carelessness offers an explanation why humans take unnecessary risks by omitting safety precautions against better judgment, but empirical research on learned carelessness is scarce. To test the theory 16 commercial aircraft pilots inspected flight plans on a multi-function display and the occurrence of flight plan errors was manipulated. Pilots rated effort of check performance, risk resulting from check omission and we measured the rate of falsely accepted erroneous flight plans. Participants who repeatedly encountered erroneous flight plans detected more errors during the test phase than participants who previously received only error-free flight plans (p < .01). The results provide evidence that subjective risk resulting from check omission affected the development of learned carelessness (p = .053), while effort of check performance displayed no effect (p = .80) due to invariance in ratings.