Heiko H. Schütt

Painfree and accurate Bayesian estimation of psychometric functions for (potentially) overdispersed data

The psychometric function describes how an experimental variable, such as stimulus strength, influences the behaviour of an observer. Estimation of psychometric functions from experimental data plays a central role in fields such as psychophysics, experimental psychology and in the behavioural neurosciences. Experimental data may exhibit substantial overdispersion, which may result from non-stationarity in the behaviour of observers. Here we extend the standard binomial model which is typically used for psychometric function estimation to a beta-binomial model. We show that the use of the beta-binomial model makes it possible to determine accurate credible intervals even in data which exhibit substantial overdispersion. This goes beyond classical measures for overdispersion-goodness-of-fit-which can detect overdispersion but provide no method to do correct inference for overdispersed data. We use Bayesian inference methods for estimating the posterior distribution of the parameters of the psychometric function. Unlike previous Bayesian psychometric inference methods our software implementation-psignifit 4-performs numerical integration of the posterior within automatically determined bounds. This avoids the use of Markov chain Monte Carlo (MCMC) methods typically requiring expert knowledge. Extensive numerical tests show the validity of the approach and we discuss implications of overdispersion for experimental design. A comprehensive MATLAB toolbox implementing the method is freely available; a python implementation providing the basic capabilities is also available.

Likelihood-based parameter estimation and comparison of dynamical cognitive models.

Dynamical models of cognition play an increasingly important role in driving theoretical and experimental research in psychology. Therefore, parameter estimation, model analysis and comparison of dynamical models are of essential importance. In this article, we propose a maximum likelihood approach for model analysis in a fully dynamical framework that includes time-ordered experimental data. Our methods can be applied to dynamical models for the prediction of discrete behavior (e.g., movement onsets); in particular, we use a dynamical model of saccade generation in scene viewing as a case study for our approach. For this model, the likelihood function can be computed directly by numerical simulation, which enables more efficient parameter estimation including Bayesian inference to obtain reliable estimates and corresponding credible intervals. Using hierarchical models inference is even possible for individual observers. Furthermore, our likelihood approach can be used to compare different models. In our example, the dynamical framework is shown to outperform nondynamical statistical models. Additionally, the likelihood based evaluation differentiates model variants, which produced indistinguishable predictions on hitherto used statistics. Our results indicate that the likelihood approach is a promising framework for dynamical cognitive models.

Psignifit 4: Pain-free Bayesian Inference for Psychometric Functions

Psychometric functions are frequently used in vision science to model task performance. These sigmoid functions can be fit to data using likelihood maximization, but this ignores the reliability or variance of the point estimates. In contrast Bayesian methods automatically calculate this reliability. However, using Bayesian methods in practice usually requires expert knowledge, user interaction and computation time. Also most methods---including Bayesian ones---are vulnerable to non-stationary observers (whose performance is not constant). For such observers all methods, which assume a stationary binomial observer are overconfident in the estimates. We present Psignifit 4, a new method for fitting psychometric functions, which provides an efficient Bayesian analysis based on numerical integration, which requires little user-interaction and runs in seconds on a common office computer. Additionally it fits a beta-binomial model increasing the stability against non-stationarity and contains standard settings including a heuristic to set the prior based on the interval of stimulus levels in the experimental data. Obviously all properties of the analysis can be adjusted. To test our method it was run on extensive simulated datasets. First we tested the numerical accuracy of our method with different settings and found settings which calculate a good estimate fast and reliably. Testing the statistical properties, we find that our method calculates correct or slightly conservative confidence intervals in all tested conditions, including different sampling schemes, beta-binomial observers, other non-stationary observers and adaptive methods. When enough data was collected to overcome the small sample bias caused by the prior, the point estimates are also essentially unbiased. In summary we present a user-friendly, fast, correct and comprehensively tested Bayesian method to fit psychometric functions, which handles non-stationary observers well and is freely available as an MATLAB implementation online. Meeting abstract presented at VSS 2015.

An image-computable psychophysical spatial vision model

A large part of classical visual psychophysics was concerned with the fundamental question of how pattern information is initially encoded in the human visual system. From these studies a relatively standard model of early spatial vision emerged, based on spatial frequency and orientation-specific channels followed by an accelerating nonlinearity and divisive normalization: contrast gain-control. Here we implement such a model in an image-computable way, allowing it to take arbitrary luminance images as input. Testing our implementation on classical psychophysical data, we find that it explains contrast detection data including the ModelFest data, contrast discrimination data, and oblique masking data, using a single set of parameters. Leveraging the advantage of an image-computable model, we test our model against a recent dataset using natural images as masks. We find that the model explains these data reasonably well, too. To explain data obtained at different presentation durations, our model requires different parameters to achieve an acceptable fit. In addition, we show that contrast gain-control with the fitted parameters results in a very sparse encoding of luminance information, in line with notions from efficient coding. Translating the standard early spatial vision model to be image-computable resulted in two further insights: First, the nonlinear processing requires a denser sampling of spatial frequency and orientation than optimal coding suggests. Second, the normalization needs to be fairly local in space to fit the data obtained with natural image masks. Finally, our image-computable model can serve as tool in future quantitative analyses: It allows optimized stimuli to be used to test the model and variants of it, with potential applications as an image-quality metric. In addition, it may serve as a building block for models of higher level processing.

Pain-free bayesian inference for psychometric functions

Bayesian Inference on psychometric functions. The method is fast and convenient to use. Furthermore, we provide suitable defaults for all common settings. By fitting a beta-binomial model our method exhibits increased robustness against nonstationary behaviour caused, e.g., by fluctuations in attention or learning. We performed extensive simulations to validate our method and software implementation. Finally we provide generic methods to compare several psychometric functions statistically. All methods are freely available:
 https://github.com/wichmann-lab/psignifitPsychophysical methodology is used to evaluate the strength of the visual signals contained in facial expressions. Stimuli were black and white images of faces expressing a neutral, positive (happy) or negative affect (e.g. fear or anger). A range of signal strengths (0-100%) of expressions were created by morphing the neutral and expressive images. Stimuli were presented for 200ms in a temporal two-interval forced-choice paradigm. One interval contained the neutral face (0%) and the other the expressive face (varied from 0 – 100%). Observers indicated the interval that contained the image that was more expressive. This emotion detection task showed that observers were more sensitive to happy compared with fearful expressions. This indicates that the emotion signals conveyed by a happy face are more salient than those conveyed by a fearful face. Future research on emotion recognition should consider using psychophysical methodology to perceptually equate stimuli with different expressions. This will remove signal strength as a possible confound.