Coping with the black swan in psychophysics

Abstract

Aberrant responses that are associated with lapses in attention and microsleeps during a behavioral experiment violate basic assumptions underlying signal detection theory. These responses (i.e., the black swan events) occur infrequently but could lead to severe biases in the estimated perceptual sensitivity (d’) and misinterpretations of the data. When adaptive procedures are used to estimate behavioral thresholds, performance lapses occurring early in an adaptive track may cause convergence failures. Moreover, microsleeps may become more frequent as the task difficulty decreases, leading to potential instabilities in common adaptive procedures. Through a series of Monte-Carlo simulations, the potential effects of performance lapses on typical psychophysical tasks (e.g., Yes/No, 2-alternative forced choice) were demonstrated. Several possible experimental and computational techniques to cope with performance lapses were investigated. These included (1) introducing additional parameters representing performance lapses into the psychometric model, (2) allowing a “Not Sure” response category, and (3) screening and removing the likely aberrant responses. These simulation studies provide useful guidelines to design adaptive psychophysical procedures for subject populations from which performance lapses are expected (e.g., young children and behaving animals).Aberrant responses that are associated with lapses in attention and microsleeps during a behavioral experiment violate basic assumptions underlying signal detection theory. These responses (i.e., the black swan events) occur infrequently but could lead to severe biases in the estimated perceptual sensitivity (d’) and misinterpretations of the data. When adaptive procedures are used to estimate behavioral thresholds, performance lapses occurring early in an adaptive track may cause convergence failures. Moreover, microsleeps may become more frequent as the task difficulty decreases, leading to potential instabilities in common adaptive procedures. Through a series of Monte-Carlo simulations, the potential effects of performance lapses on typical psychophysical tasks (e.g., Yes/No, 2-alternative forced choice) were demonstrated. Several possible experimental and computational techniques to cope with performance lapses were investigated. These included (1) introducing additional parameters representing perfo...