Elise Payzan-LeNestour

Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings

Recently, evidence has emerged that humans approach learning using Bayesian updating rather than (model-free) reinforcement algorithms in a six-arm restless bandit problem. Here, we investigate what this implies for human appreciation of uncertainty. In our task, a Bayesian learner distinguishes three equally salient levels of uncertainty. First, the Bayesian perceives irreducible uncertainty or risk: even knowing the payoff probabilities of a given arm, the outcome remains uncertain. Second, there is (parameter) estimation uncertainty or ambiguity: payoff probabilities are unknown and need to be estimated. Third, the outcome probabilities of the arms change: the sudden jumps are referred to as unexpected uncertainty. We document how the three levels of uncertainty evolved during the course of our experiment and how it affected the learning rate. We then zoom in on estimation uncertainty, which has been suggested to be a driving force in exploration, in spite of evidence of widespread aversion to ambiguity. Our data corroborate the latter. We discuss neural evidence that foreshadowed the ability of humans to distinguish between the three levels of uncertainty. Finally, we investigate the boundaries of human capacity to implement Bayesian learning. We repeat the experiment with different instructions, reflecting varying levels of structural uncertainty. Under this fourth notion of uncertainty, choices were no better explained by Bayesian updating than by (model-free) reinforcement learning. Exit questionnaires revealed that participants remained unaware of the presence of unexpected uncertainty and failed to acquire the right model with which to implement Bayesian updating.

The Neural Representation of Unexpected Uncertainty during Value-Based Decision Making

Uncertainty is an inherent property of the environment and a central feature of models of decision-making and learning. Theoretical propositions suggest that one form, unexpected uncertainty, may be used to rapidly adapt to changes in the environment, while being influenced by two other forms: risk and estimation uncertainty. While previous studies have reported neural representations of estimation uncertainty and risk, relatively little is known about unexpected uncertainty. Here, participants performed a decision-making task while undergoing functional magnetic resonance imaging (fMRI), which, in combination with a Bayesian model-based analysis, enabled us to separately examine each form of uncertainty examined. We found representations of unexpected uncertainty in multiple cortical areas, as well as the noradrenergic brainstem nucleus locus coeruleus. Other unique cortical regions were found to encode risk, estimation uncertainty, and learning rate. Collectively, these findings support theoretical models in which several formally separable uncertainty computations determine the speed of learning.

Do not bet on the unknown versus try to find out more: estimation uncertainty and “unexpected uncertainty” both modulate exploration

Little is known about how humans solve the exploitation/exploration trade-off. In particular, the evidence for uncertainty-driven exploration is mixed. The current study proposes a novel hypothesis of exploration that helps reconcile prior findings that may seem contradictory at first. According to this hypothesis, uncertainty-driven exploration involves a dilemma between two motives: (i) to speed up learning about the unknown, which may beget novel reward opportunities; (ii) to avoid the unknown because it is potentially dangerous. We provide evidence for our hypothesis using both behavioral and simulated data, and briefly point to recent evidence that the brain differentiates between these two motives.

Learning to Choose the Right Investment in an Unstable World: Experimental Evidence Based on the Bandit Problem

When asset returns are unstable, investment performance directly depends on learning about their patterns optimally. Without optimal learning, strong investment performance is not possible. Yet, optimal learning is often considered too complex for investors to achieve. In order to test this experimentally, we simulate the return profiles of unstable assets with a multi-armed bandit in which the expected returns of the arms jump over the experiment. We find substantial evidence of optimal learning, despite the high difficulty of the task. This finding suggests that investors can in fact learn optimally in, at least, a subset of unstable financial contexts.

Volatility After-Effects: Evidence from the Field

Prolonged exposure to extremes of a variable inversely biases subsequent perceptions of that variable; an effect known as the “waterfall illusion” or “after-effect”. We present the first evidence that after-effects bias investors’ perceptions of volatility, resulting in significant distortions of prices in a sophisticated and liquid market. We find that after prolonged exposure to high (low) volatility, the marginal trader under- (over-) estimates volatility. The longer the exposure and the stronger the volatility level, the larger the subsequent bias. Additional tests rule out the most plausible competing explanations including changes in the variance risk premium, investor reactions to changes in jump risk, errors in expectations, and learning about kurtosis or other higher-order moments.

Can People Learn About ‘Black Swans’? Experimental Evidence

On the aftermath of the global financial crisis, model uncertainty about tail risk stood out as a major problem to overcome. I capture this problem in a decision-making task in which the Bayesian agent learns whether reward prospects are fat-tailed. In simulations of the task, following adaptive principles instead markedly impairs economic performance. When asked to perform the task for real money, lab participants learned in a Bayesian way. These findings suggest that an important requirement to cope with tail risk is to learn about it optimally, and that such learning is within the reach of the average investor’s intelligence.On the aftermath of the global financial crisis, model uncertainty about tail risk stood out as a major problem to overcome. I capture this problem in a decision-making task in which the Bayesian agent learns whether reward prospects are fat-tailed. In simulations of the task, following adaptive learning instead markedly impairs economic performance. When asked to perform the task for real money, lab participants learned in a Bayesian way. These findings suggest that an important requirement to cope with tail risk is to learn about it optimally, and that such learning is within the reach of the average investor’s intelligence.

The 'Waterfall Illusion' in the Financial Markets: Risk Perception Is Distorted After Prior Exposure to Extreme Risk

Prolonged exposure to extremes of a variable inversely biases subsequent perceptions of that variable; an effect known as the “waterfall illusion” or “after-effect”. We present the first evidence that after-effects bias investors’ perceptions of volatility, resulting in significant distortions of prices in a sophisticated and liquid market. We find that after prolonged exposure to high (low) volatility, the marginal trader under- (over-) estimates volatility. The longer the exposure and the stronger the volatility level, the larger the subsequent bias. Additional tests rule out the most plausible competing explanations including changes in the variance risk premium, investor reactions to changes in jump risk, errors in expectations, and learning about kurtosis or other higher-order moments.

The Waterfall Illusion, and How Prior Exposure to Extreme Risk Distorts Risk Perception

Prolonged exposure to extremes of a variable inversely biases subsequent perceptions of that variable; an effect known as the “waterfall illusion” or “after-effect”. We present the first evidence that after-effects bias investors’ perceptions of volatility, resulting in significant distortions of prices in a sophisticated and liquid market. We find that after prolonged exposure to high (low) volatility, the marginal trader under- (over-) estimates volatility. The longer the exposure and the stronger the volatility level, the larger the subsequent bias. Additional tests rule out the most plausible competing explanations including changes in the variance risk premium, investor reactions to changes in jump risk, errors in expectations, and learning about kurtosis or other higher-order moments.

The Waterfall Illusion, and How Exposure to Extreme Volatility Distorts Risk Perceptions in Markets

Prolonged exposure to extremes of a variable inversely biases subsequent perceptions of that variable; an effect known as the “waterfall illusion” or “after-effect”. We present the first evidence that after-effects bias investors’ perceptions of volatility, resulting in significant distortions of prices in a sophisticated and liquid market. We find that after prolonged exposure to high (low) volatility, the marginal trader under- (over-) estimates volatility. The longer the exposure and the stronger the volatility level, the larger the subsequent bias. Additional tests rule out the most plausible competing explanations including changes in the variance risk premium, investor reactions to changes in jump risk, errors in expectations, and learning about kurtosis or other higher-order moments.