Miguel A. Vadillo

Justify your alpha

In response to recommendations to redefine statistical significance to P ≤ 0.005, we propose that researchers should transparently report and justify all choices they make when designing a study, including the alpha level.

Searching for the bottom of the ego well: failure to uncover ego depletion in Many Labs 3

According to a popular model of self-control, willpower depends on a limited resource that can be depleted when we perform a task demanding self-control. This theory has been put to the test in hundreds of experiments showing that completing a task that demands high self-control usually hinders performance in any secondary task that subsequently taxes self-control. Over the last 5 years, the reliability of the empirical evidence supporting this model has been questioned. In the present study, we reanalysed data from a large-scale study—Many Labs 3—to test whether performing a depleting task has any effect on a secondary task that also relies on self-control. Although we used a large sample of more than 2000 participants for our analyses, we did not find any significant evidence of ego depletion: persistence on an anagram-solving task (a typical measure of self-control) was not affected by previous completion of a Stroop task (a typical depleting task in this literature). Our results suggest that either ego depletion is not a real effect or, alternatively, persistence in anagram solving may not be an optimal measure to test it.

Learned predictiveness acquired through experience prevails over the influence of conflicting verbal instructions in rapid selective attention

Previous studies have provided evidence that selective attention tends to prioritize the processing of stimuli that are good predictors of upcoming events over nonpredictive stimuli. Moreover, studies using eye-tracking to measure attention demonstrate that this attentional bias towards predictive stimuli is at least partially under voluntary control and can be flexibly adapted via instruction. Our experiment took a similar approach to these prior studies, manipulating participants’ experience of the predictiveness of different stimuli over the course of trial-by-trial training; we then provided explicit verbal instructions regarding stimulus predictiveness that were designed to be either consistent or inconsistent with the previously established learned predictiveness. Critically, we measured the effects of training and instruction on attention to stimuli using a dot probe task, which allowed us to assess rapid shifts of attention (unlike the eye-gaze measures used in previous studies). Results revealed a rapid attentional bias towards stimuli experienced as predictive (versus those experienced as nonpredictive), that was completely unaffected by verbal instructions. This was not due to participants’ failure to recall or use instructions appropriately, as revealed by analyses of their learning about stimuli, and their memory for instructions. Overall, these findings suggest that rapid attentional biases such as those measured by the dot probe task are more strongly influenced by our prior experience during training than by our current explicit knowledge acquired via instruction.

A comparator-hypothesis account of biased contingency detection

Our ability to detect statistical dependencies between different events in the environment is strongly biased by the number of coincidences between them. Even when there is no true covariation between a cue and an outcome, if the marginal probability of either of them is high, people tend to perceive some degree of statistical contingency between both events. The present paper explores the ability of the Comparator Hypothesis to explain the general pattern of results observed in this literature. Our simulations show that this model can account for the biasing effects of the marginal probabilities of cues and outcomes. Furthermore, the overall fit of the Comparator Hypothesis to a sample of experimental conditions from previous studies is comparable to that of the popular Rescorla-Wagner model. These results should encourage researchers to further explore and put to the test the predictions of the Comparator Hypothesis in the domain of biased contingency detection.

Commentary: Can Ordinary People Detect Deception after All?

No one likes to call someone a liar. But the authors of the tipping point account (ten Brinke, Vohs, & Carney, 2016) claim that it is evolutionary prudent to spot lies that can harm us in order to determine who to trust. As such, they propose the reputational costs of confronting a liar might be overcome by detecting lies unconsciously. When confronted with information that creates a threat response, the unconscious can use the threat response to detect deceptive cues and to unconsciously infer deception, all the while keeping this information out of the conscious mind. The account suggests this is beneficial because conscious awareness of the deception “could impel the perceiver to confront the liar” (p. 580). nThe account is controversial insofar as it claims that people can detect deception, in contrast to past work showing otherwise (47% detection rate of lies, and 61% of truths, resulting from bias to judge statements as true: Bond & DePaulo, 2006), and also makes novel claims about an unconscious ability. Although it is welcoming to see new theoretical approaches to lie detection, the account (a) makes claims that do not match the data and conclusions presented in the studies cited to build its case, (b) offers no testable definition of unconscious processes, and (c) contains internal contradictions.

Metronome LKM: An open source virtual keyboard driver to measure experiment software latencies

Experiment software is often used to measure reaction times gathered with keyboards or other input devices. In previous studies, the accuracy and precision of time stamps has been assessed through several means: (a) generating accurate square wave signals from an external device connected to the parallel port of the computer running the experiment software, (b) triggering the typematic repeat feature of some keyboards to get an evenly separated series of keypress events, or (c) using a solenoid handled by a microcontroller to press the input device (keyboard, mouse button, touch screen) that will be used in the experimental setup. Despite the advantages of these approaches in some contexts, none of them can isolate the measurement error caused by the experiment software itself. Metronome LKM provides a virtual keyboard to assess an experiment’s software. Using this open source driver, researchers can generate keypress events using high-resolution timers and compare the time stamps collected by the experiment software with those gathered by Metronome LKM (with nanosecond resolution). Our software is highly configurable (in terms of keys pressed, intervals, SysRq activation) and runs on 2.6–4.8 Linux kernels.