Katsuhiko Miyazaki

Activation of Dorsal Raphe Serotonin Neurons Underlies Waiting for Delayed Rewards

The serotonergic system plays a key role in the control of impulsive behaviors. Forebrain serotonin depletion leads to premature actions and steepens discounting of delayed rewards. However, there has been no direct evidence for serotonin neuron activity in relation to actions for delayed rewards. Here we show that serotonin neurons increase their tonic firing while rats wait for food and water rewards and conditioned reinforcement tones. The rate of tonic firing during the delay period was significantly higher for rewards than for tones, for which rats could not wait as long. When the delay was extended, tonic firing persisted until reward or tone delivery. When rats gave up waiting because of extended delay or reward omission, serotonin neuron firing dropped preceding the exit from reward sites. Serotonin neurons did not show significant response when an expected reward was omitted, which was predicted by the theory that serotonin signals negative reward prediction errors. These results suggest that increased serotonin neuron firing facilitates a rats waiting behavior in prospect of forthcoming rewards and that higher serotonin activation enables longer waiting.

The Role of Serotonin in the Regulation of Patience and Impulsivity

Classic theories suggest that central serotonergic neurons are involved in the behavioral inhibition that is associated with the prediction of negative rewards or punishment. Failed behavioral inhibition can cause impulsive behaviors. However, the behavioral inhibition that results from predicting punishment is not sufficient to explain some forms of impulsive behavior. In this article, we propose that the forebrain serotonergic system is involved in “waiting to avoid punishment” for future punishments and “waiting to obtain reward” for future rewards. Recently, we have found that serotonergic neurons increase their tonic firing rate when rats await food and water rewards and conditioned reinforcer tones. The rate of tonic firing during the delay period was significantly higher when rats were waiting for rewards than for tones, and rats were unable to wait as long for tones as for rewards. These results suggest that increased serotonergic neuronal firing facilitates waiting behavior when there is the prospect of a forthcoming reward and that serotonergic activation contributes to the patience that allows rats to wait longer. We propose a working hypothesis to explain how the serotonergic system regulates patience while waiting for future rewards.

Activation of the central serotonergic system in response to delayed but not omitted rewards.

The forebrain serotonergic system is a crucial component in the control of impulsive behaviours. However, there is no direct evidence for natural serotonin activity during behaviours for delayed rewards as opposed to immediate rewards. Herein we show that serotonin efflux is enhanced while rats perform a task that requires waiting for a delayed reward. We simultaneously measured the levels of serotonin and dopamine in the dorsal raphe nucleus using in vivo microdialysis. Rats performed a sequential food–water navigation task under three reward conditions: immediate, delayed and intermittent. During the delayed reward condition, in which the rat had to wait for up to 4 s at the reward sites, the level of serotonin was significantly higher than that during the immediate reward condition, whereas the level of dopamine did not change significantly. By contrast, during the intermittent reward condition, in which food was given on only about one‐third of the site visits, the level of dopamine was lower than that during the immediate reward condition, whereas the level of serotonin did not change significantly. Dopamine efflux, but not serotonin efflux, was positively correlated with reward consumption during the task. There was no reciprocal relationship between serotonin and dopamine. This is the first direct evidence that activation of the serotonergic system occurs specifically in relation to waiting for a delayed reward.

Different representation of forthcoming reward in nucleus accumbens and medial prefrontal cortex.

We recorded single units in the rat nucleus accumbens (NAcc) and medial prefrontal cortex (mPFC) to investigate activity related to reward mediation and anticipation during execution of an alternating reward task. NAcc and mPFC neurons showed increased activity differently during an interposed delay preceding reward delivery. Some NAcc neurons increased their activity specifically during the delay period before reward presentation, discriminating forthcoming food or water presentation. A subset of neurons in the mPFC similarly discriminated between food and water reward during the delay; however, about half did not discriminate reward qualities. These results show that the NAcc and the mPFC contribute differently to the organization and execution of goal-directed behavior.

Reward probability and timing uncertainty alter the effect of dorsal raphe serotonin neurons on patience

Recent experiments have shown that optogenetic activation of serotonin neurons in the dorsal raphe nucleus (DRN) in mice enhances patience in waiting for future rewards. Here, we show that serotonin effect in promoting waiting is maximized by both high probability and high timing uncertainty of reward. Optogenetic activation of serotonergic neurons prolongs waiting time in no-reward trials in a task with 75% food reward probability, but not with 50 or 25% reward probabilities. Serotonin effect in promoting waiting increases when the timing of reward presentation becomes unpredictable. To coherently explain the experimental data, we propose a Bayesian decision model of waiting that assumes that serotonin neuron activation increases the prior probability or subjective confidence of reward delivery. The present data and modeling point to the possibility of a generalized role of serotonin in resolving trade-offs, not only between immediate and delayed rewards, but also between sensory evidence and subjective confidence.Activation of serotonin neurons in the dorsal raphe nucleus promotes patience in waiting for future rewards. Here the authors show that this effect is maximal for high probability reward or high temporal reward uncertainty suggesting that it boosts the prior probability of reward.

Brain computation mechanism of prediction and decision making by dorsal raphe serotonin neurons

予 測 ・ 意 思 決 定 ・ 情 動 の 脳 内 計 算 機 構 ― セ ロ ト ニ ン 研 究 の 新 展 開 ― 2 要約:光遺伝学技術によって神経活動と生体機能の因 果関係を調べることが可能となったが,神経修飾物質 であるセロトニン神経系に対しては,セロトニンが行 動,情動,認知などの多様な機能に関与しているもの の,セロトニン神経活動がどのような機能に直接関与 しているのかについてはいまだ不明な点が多い.筆者 らはこれまでに行ってきた複数の動物実験の結果に基 づき,セロトニンは将来の報酬のための待機行動を調 節するという新たな提案をしている.この検証にあた り近年光遺伝学の研究手法を導入し,背側縫線核セロ トニン神経の活性化が将来報酬を辛抱強く待つ行動を 促進することを明らかにした.筆者らの結果は,セロ トニンの神経活動は動物が将来獲得できる報酬を辛抱 強く待つかそれとも諦めるかの意思決定過程に重要な 役割を果たしていることを示唆している.