Risk Sensitivity under Partially Observable Markov Decision Processes

Abstract

Many real-life decisions must be made in the face of risk that is due to uncertain information about the environment. Even facing the same environment, different people might behave differently due to their individual risk preferences. For instance, a risk-seeking gambler may overestimate the chance of favorable outcomes or the amount of money going to win in those cases and therefore prefers to gamble. In cognitive neuroscience, Bayesian inference is usually applied to model the objective perception of the unobservable state, under which riskneutral decisions are made by solving a partially observable Markov decision process (POMDP). However, the subjective evaluation of such inferred state information, which leads to different individual risk preferences, and the underlying neurobiological process are still poorly understood. Hence, we derived a risk-sensitive POMDP method that models human choice behavior and response time in a simulated investment task. Our risksensitive POMDP model fits the experimental data considerably better than the risk-neutral model. The model’s risk-sensitivity parameters explained subjects’ individual risk preference under state uncertainty at the decision time. Our results may pave the way for understanding human risk-sensitive choice under perceptual uncertainty using a unified quantitative POMDP framework.