Katarzyna Jaworska

How Predictable are “Spontaneous Decisions” and “Hidden Intentions”? Comparing Classification Results Based on Previous Responses with Multivariate Pattern Analysis of fMRI BOLD Signals

In two replication studies we examined response bias and dependencies in voluntary decisions. We trained a linear classifier to predict “spontaneous decisions” and in the second study “hidden intentions” from responses in preceding trials and achieved comparable prediction accuracies as reported for multivariate pattern classification based on voxel activities in frontopolar cortex. We discuss implications of our findings and suggest ways to improve classification analyses of fMRI BOLD signals that may help to reduce effects of response dependencies between trials.

Flipping a coin in your head without monitoring outcomes? Comments on predicting free choices and a demo program

In a recent study Soon et al. (2013) predicted abstract intentions from fMRI BOLD activities in localized areas of the brain. Activities in a spherical cluster of voxels served as input to a multivariate pattern classifier (linear SVM). The accuracy for predicting the intention to add or subtract two numbers was determined for clusters centered on different voxels. A prediction accuracy of 60% averaged across participants and based on 10-fold cross-validation was achieved for patterns of voxel activities in the medial frontopolar cortex and precuneus up to 4 s before participants reported being consciously aware of their decision. The prediction accuracy in this study was similar to studies on predicting spontaneous left or right motor decisions (Soon et al., 2008; Bode et al., 2011). Since the task demands placed on the participants create similar methodological issues as in previous studies (Lages and Jaworska, 2012), it seems possible that the multivariate classifier picked up sequential information processing between trials (Bode et al., 2012). Although the average prediction accuracy of 60% returned to chance level for patterns of voxel activity in the two brain areas shortly after the onset of a new trial and remained at 50% between trials, this observation is neither necessary nor sufficient for the absence of sequential information processing. In order to investigate sequential dependencies the outcome of at least one preceding trial and the current trial needs to be taken into account. Depending on task and response, sequential information processing between trials may emerge in distributed form within the default mode network (DMN) at variable time points (Guggisberg and Mottaz, 2013). In the following we illustrate the issue and suggest how the data may be analyzed. To illustrate sequential effects consider the following inconspicuous sequence of ten responses (S, A, A, A, S, A, S, A, S, S) where A and S stand for freely choosing addition and subtraction, respectively. There are five As and five Ss suggesting a binomial process with rate parameter p = 0.5. However, if we consider the nine subsequent pairs of responses {(S,A), (A,A), …, (S,S)} then we obtain unequal transition probabilities. A trained classifier that predicts the next response from the preceding response would be 3 out of 5 times or 60% correct if the preceding response is A and 3 out of 4 times or 75% correct if the response is S. Starting with a random guess in the first trial, this gives an average prediction accuracy of 65%. In two behavioral studies replicating two different choice tasks (Haynes et al., 2007; Soon et al., 2008) we used subsequent response pairs to train a linear classifier (SVM) and obtained an average prediction accuracy of 61.6 and 64.1%, respectively (Lages and Jaworska, 2012). When asked to generate a random sequence, people typically alternate between binary responses with a probability of about 0.6 (Lopes, 1982). This response pattern appears to be relevant in Soon et al.s study (2013) since the only behavioral evidence for memory-less choice in the 17 (out of 34) selected participants is a histogram plotting average frequencies for different lengths of response sequences fitted by an exponential distribution (Figure S1). The authors take the excellent fit as evidence for random performance. Recently Allefeld et al. (2013) released a detailed account of the behavioral data but there are no further details how the data were compiled and fit. Nevertheless, it is discernable from their Figure S1 that the fit represents an exponential function with two parameters rather than an exponential distribution with a single parameter and that observed frequencies do not add up to probability 1.0. In addition, an exponential distribution would only provide a meaningful approximation of the geometrically distributed phase lengths if choosing addition and subtraction were equally probable [p = (1−p) = 0.5]. However, even with a best-fitting rate parameter of 1-exp(−0.826) ≈0.56 the exponential distribution underestimates the relative frequency of alternations (A,S) and (S,A) with phase length 1 as well as repetitions (A,A,S) and (S,S,A) with phase length 2 (see Figure ​Figure1).1). Increased frequencies for short phase lengths are a hallmark of non-random human choice behavior (Wagenaar, 1972; Lopes, 1982; Treisman and Faulkner, 1987; Falk and Konold, 1997) and these characteristics are not only present in the behavioral data of Soon et al. (2013) but also in Soon et al. (2008); Bode et al. (2011), and Haynes et al. (2007) suggesting that free or spontaneous choice tasks result in non-random behavior. Figure 1 Histogram for length of response sequences (phase length or runs) re-plotted as relative frequencies (adapted from Figure S1 in Soon et al., 2013). The data points are fitted by an exponential function with two parameters (red curve) and an exponential ... A related concern arises from the searchlight analyses. If patterns of voxel activities are analyzed within a moving spherical cluster to predict behavioral responses then pre-processing of the data and definition of the searchlight are important (Etzel et al., 2013; Todd et al., 2013). The implementation of regions of interest, temporal constraints (hemodynamic delay), pre-processing (covariates), and data selection can invalidate the results of a searchlight analysis (Kriegeskorte et al., 2009; Lindquist et al., 2009). In Soon et al. (2013) trials were selected (undersampled) in order to balance the mean response rate. It is therefore possible that the searchlight found a cluster of voxels that was predictive of the next response in the context of the preceding response, simply because transitions between successive responses remained unbalanced. A repeated choice task with self-monitoring of the decision process invites sequential dependencies because the observer has to remember goals, constraints, and execution of the task. If, for example, participants shift a decision criterion following each response (Lages and Treisman, 1998, 2010; Lages and Paul, 2006; Treisman and Lages, 2010) or engage in metacognition by recalling the last response before making the next then neural correlates of these response-dependent processes introduce a confound that would be picked up by a searchlight analysis as soon as transitions between response categories are unbalanced. We recommend that rather than postulating a 50% chance level, prediction accuracy should be tested with a permutation test (Stelzer et al., 2013) and/or separate multivariate classification analyses conditional on the previous response. Only if individual prediction accuracies reliably exceed observable benchmarks such as response bias and transition probabilities can we rest assured that results are not confounded. The interested reader is invited to test predictability of their own free choice behavior by downloading the demo program in the Appendix.

With Age Comes Representational Wisdom in Social Signals

Summary In an increasingly aging society, age has become a foundational dimension of social grouping broadly targeted by advertising and governmental policies. However, perception of old age induces mainly strong negative social biases [1–3]. To characterize their cognitive and perceptual foundations, we modeled the mental representations of faces associated with three age groups (young age, middle age, and old age), in younger and older participants. We then validated the accuracy of each mental representation of age with independent validators. Using statistical image processing, we identified the features of mental representations that predict perceived age. Here, we show that whereas younger people mentally dichotomize aging into two groups, themselves (younger) and others (older), older participants faithfully represent the features of young age, middle age, and old age, with richer representations of all considered ages. Our results demonstrate that, contrary to popular public belief, older minds depict socially relevant information more accurately than their younger counterparts. Video Abstract

Fluctuations of visual awareness: combining motion-induced blindness with binocular rivalry

Binocular rivalry (BR) and motion-induced blindness (MIB) are two phenomena of visual awareness where perception alternates between multiple states despite constant retinal input. Both phenomena have been extensively studied, but the underlying processing remains unclear. It has been suggested that BR and MIB involve the same neural mechanism, but how the two phenomena compete for visual awareness in the same stimulus has not been systematically investigated. Here we introduce BR in a dichoptic stimulus display that can also elicit MIB and examine fluctuations of visual awareness over the course of each trial. Exploiting this paradigm we manipulated stimulus characteristics that are known to influence MIB and BR. In two experiments we found that effects on multistable percepts were incompatible with the idea of a common oscillator. The results suggest instead that local and global stimulus attributes can affect the dynamics of each percept differently. We conclude that the two phenomena of visual awareness share basic temporal characteristics but are most likely influenced by processing at different stages within the visual system.

The Deceptively Simple N170 Reflects Network Information Processing Mechanisms Involving Visual Feature Coding and Transfer Across Hemispheres

A key to understanding visual cognition is to determine “where”, “when”, and “how” brain responses reflect the processing of the specific visual features that modulate categorization behavior—the “what”. The N170 is the earliest Event-Related Potential (ERP) that preferentially responds to faces. Here, we demonstrate that a paradigmatic shift is necessary to interpret the N170 as the product of an information processing network that dynamically codes and transfers face features across hemispheres, rather than as a local stimulus-driven event. Reverse-correlation methods coupled with information-theoretic analyses revealed that visibility of the eyes influences face detection behavior. The N170 initially reflects coding of the behaviorally relevant eye contralateral to the sensor, followed by a causal communication of the other eye from the other hemisphere. These findings demonstrate that the deceptively simple N170 ERP hides a complex network information processing mechanism involving initial coding and subsequent cross-hemispheric transfer of visual features.

Fluctuations of visual awareness: Motion induced blindness and binocular rivalry

Motion-induced blindness (MIB) and binocular rivalry (BR) are popular paradigms to study visual awareness. It has been suggested that both phenomena are related and share a common oscillator (Carter & Pettigrew, 2003). In two experiments we tried to determine whether BR affects MIB by creating an experimental paradigm that can elicit both. In the first experiment eighteen observers fixated the center of a display with a moving mask and a superimposed stationary target in a split-screen Wheatstone configuration for 30 sec. Each observer reported disappearance and reappearance of a salient target dot in the upper visual field by pressing and releasing a labeled key. The mask was a rotating grid of crosses or a drifting sine-wave grating. In a within-subjects design the mask was presented in rivalry or not; with opposite rotation and orthogonal drift in the left and right eye or with the same rotation and drift in both eyes. In addition, the target was presented to both eyes (binocular target) or to one eye only (dichoptic target). Results show that MIB as measured by normalized disappearance was significantly increased for dichoptic targets but remained unaffected by binocular rivalry in the different masks. Independence of MIB from BR was confirmed in a second experiment in which isoluminant red and green target dots were presented to the left or right eye and observers reported perceived color as a measure of binocular rivalry in addition to target disappearance. In conclusion, our preliminary results suggest that MIB is independent of BR. Further analyses on the dynamics of target perception will inform whether or not the two phenomena fluctuate independently of each other.

Neural Processing of the Same, Behaviourally Relevant Face Features is Delayed by 40 ms in Healthy Ageing

Fast and accurate face perception is critical for successful human social interactions. Face perception declines with age both in behavioural and neural responses, although we do not yet understand why. Here, we tested the hypothesis that early brain mechanisms involved with face information processing are delayed in older participants. Using face detection - the most basic task for social interaction – we sampled visual information from faces (vs. noise) and reconstructed the features (mainly, the left eye) associated with detection behaviour in young (20-36 years) and older (60-86 years) adults. We then compared behavioural results to neural representations of face features revealed with simultaneously recorded EEG on the N170, an event-related potential associated with visual categorization. Whereas the right hemisphere N170 latency and amplitude represented the left eye in young participants, it was mostly amplitude that represented the eye with a 40 ms delay in older adults. Our results demonstrate that face processing speed declines in ageing with a delay in the early stages that process the visual information important for behaviour.Fast and accurate face processing is critical for everyday social interactions, but it declines and becomes delayed with age, as measured by both neural and behavioural responses. Here, we addressed the critical challenge of understanding how ageing changes neural information processing mechanisms to delay behaviour. Young (20-36 years) and older (60-86 years) adults performed the basic social interaction task detecting a face vs. noise while we recorded their electroencephalogram (EEG). In each participant, using a new information theoretic framework we reconstructed the features supporting face detection behaviour, and also where, when and how EEG activity represents them. We found that occipital-temporal pathway activity dynamically represents the eyes of the face images for behaviour ~170 ms post-stimulus, with a 40 ms delay in older adults that underlies their 200 ms behavioural deficit of slower reaction times. Our results therefore demonstrate how ageing can change neural information processing mechanisms that underlie behavioural slow down.

Face inversion does not affect the information content coded during the N170

Face inversion dramatically disrupts our ability to process faces, which is known as the face inversion effect. Previous electrophysiological studies have indicated that the N170 ERP component is delayed and sometimes larger in response to inverted compared to upright faces. However, the nature of these effects remains elusive because we do not yet understand the information coding function of the N170 in upright and inverted faces. Here, we assessed what facial information the N170 codes and when it does so for upright and inverted faces. To this aim, we used one of the simplest socially relevant tasks: face detection. In this task, 10 healthy adults (5 females, median age=23, 20-29) saw pictures of faces and noise textures revealed through ten small Gaussian apertures (bubbles). Participants performed two sessions of trials with upright faces, and two sessions with inverted faces, for a total of 4400 trials. We applied reverse-correlation methods coupled with information theory to reveal the image pixels statistically associated with behaviour and neural responses. In both upright and inverted faces, we found that presence of the left eye modulated the reaction times (RTs) of all participants. In upright faces, the eye contralateral to the left and right posterior lateral electrodes strongly modulated early face ERPs. In particular, the N170 latency and amplitude coded the presence of the contralateral eye (Rousselet et al. Journal of Vision 2014, 14(13): 7, 1-24). This association was about 35% weaker in inverted faces, and delayed by about 30 ms, compared to upright face. In conclusion, our results suggest that, in a face detection task, the N170 mostly code the presence of a single feature: the contralateral eye. Inversion leads to an inefficient coding of the same feature, which is reflected in weaker and delayed feature sensitivity. Meeting abstract presented at VSS 2015.

The deceptively simple N170 hides a complex diagnostic coding mechanism involving visual feature transfer across hemispheres.

A key to understanding visual cognition is to determine when and how brain responses are sensitive to the specific visual information underlying categorization behavior. We know that the N170 is the first brain response coding such diagnostic information (Schyns et al., Current Biology 2007), as recently shown in Rousselet et al.s experiment (Journal of Vision 2014). Using Bubbles (Gosselin & Schyns, Vision Research 2001), we found that the eyes are diagnostic for face detection and are coded in latency and amplitude modulations of the N170. Here, with new analyses we show that diagnostic coding of the eyes in the N170 involves cross-hemispheric transfer. We proceed in three steps. First, we show that the eye contralateral to the recording electrode (i.e. Occipito-Temporal Left, or Right, OTL, OTR) modulates the N170 latencies, whereas the ipsilateral eye modulates the later N170 amplitudes. Second, we show that single-trial N170 latencies and amplitudes fully account for all the coding of the eye that can be extracted from the EEG signal, at any time point. Finally, we show an important effect of the temporal ordering of the left and right N170s (on OTL and OTR) on coding of the eyes. Specifically, an earlier N170 on OTL (initially coding the right eye) has a strong effect on the later coding of the right eye on OTR (initially coding the left eye), suggesting causal transfer of the right eye across hemispheres. To summarize, in a face detection task where both eyes are diagnostic, the N170 ERP initially codes the contralateral eye, closely followed by the ipsilateral eye that is transferred from the earlier N170 in the opposite hemisphere. Our results suggest that the deceptively simple N170 ERP hides a complex mechanism of diagnostic visual feature coding involving cross-hemispheric feature transfer. Meeting abstract presented at VSS 2015.