Ruud van den Bos

A concept of welfare based on reward evaluating mechanisms in the brain: anticipatory behaviour as an indicator for the state of reward systems

In this review we attempt to link the efficiency by which animals behave (economy of animal behaviour) to a neuronal substrate and subjective states to arrive at a definition of animal welfare which broadens the scope of its study. Welfare is defined as the balance between positive (reward, satisfaction) and negative (stress) experiences or affective states. The state of this balance may range from positive (good welfare) to negative (poor welfare). These affective states are momentary or transient states which occur against the background of and are integrated with the state of this balancing system. As will be argued the efficiency in behaviour requires that, for instance, satisfaction is like a moving target: reward provides the necessary feedback to guide behaviour; it is a not steady-state which can be maintained for long. This balancing system is reflected in the brain by the concerted action of opioid and mesolimbic dopaminergic systems. The state of this system reflects the coping capacity of the animal and is determined by previous events. In other words, this integrative approach of behavioural biology and neurobiology aims at understanding how the coping capacity of animals may be affected and measured. We argue that this balancing system underlies the economy of behaviour. Furthermore we argue that among other techniques anticipation in Pavlovian conditioning is an easy and useful tool to assess the state of this balancing system: for estimating the state of an animal in terms of welfare we focus on the conditions when an animal is facing a challenge.

Stress and decision-making in humans: performance is related to cortisol reactivity, albeit differently in men and women

Acutely elevated levels of cortisol are associated with euphoria and reward-like properties related to sensation-seeking behaviour. Thus, acute stress and elevated levels of cortisol may promote risk-taking behaviour. High cortisol responders are more sensitive to immediate rewards than low cortisol responders. In this study we therefore tested whether acute stress in male and female subjects, induced by the Trier Social Stress Test (TSST), affects decision-making as measured by the Iowa Gambling Task (IGT) and to what extent this is related to cortisol reactivity. Control subjects did not receive the stress manipulation. We specifically predict that high responders show risk-taking behaviour in the IGT compared to low responders and controls. The data show that the more (salivary) cortisol levels are elevated after the TSST the poorer the subsequent performance in the IGT in male subjects. In female subjects an inverse relationship between cortisol levels and IGT performance is observed: slightly elevated levels of cortisol after the TSST improve IGT performance, while highly elevated levels decrease IGT performance. Thus, acute stress as induced by the TSST affects decision-making behaviour of men and women differently and cortisol reactivity is associated with decision-making performance.

A critical review of sex differences in decision-making tasks: Focus on the Iowa Gambling Task

It has been observed that men and women show performance differences in the Iowa Gambling Task (IGT), a task of decision-making in which subjects through exploration learn to differentiate long-term advantageous from long-term disadvantageous decks of cards: men choose more cards from the long-term advantageous decks than women within the standard number of 100 trials. Here, we aim at discussing psychological mechanisms and neurobiological substrates underlying sex differences in IGT-like decision-making. Our review suggests that women focus on both win-loss frequencies and long-term pay-off of decks, while men focus on long-term pay-off. Furthermore, women may be more sensitive to occasional losses in the long-term advantageous decks than men. As a consequence hereof, women need 40-60 trials in addition before they reach the same level of performance as men. These performance differences are related to differences in activity in the orbitofrontal cortex and dorsolateral prefrontal cortex as well as in serotonergic activity and left-right hemispheric activity. Sex differences in orbitofrontal cortex activity may be due to organisational effects of gonadal hormones early in life. The behavioural and neurobiological differences in the IGT between men and women are an expression of more general sex differences in the regulation of emotions. We discuss these findings in the context of sex differences in information processing related to evolutionary processes. Furthermore we discuss the relationship between these findings and real world decision-making.

Serotonin transporter dosage modulates long-term decision-making in rat and human

Decision-making plays an important role in everyday life and is often disturbed in psychiatric conditions affected by the common human serotonin transporter promoter length polymorphism (5-HTTLPR). This raises the hypothesis that decision-making is modulated by the serotonergic system, but currently it is unclear how the 5-HTTLPR affects central serotonergic functioning. We tested healthy human volunteers genotyped for the 5-HTTLPR in the Iowa Gambling Task (IGT), which is one of the most frequently used neuropsychological tasks to assess decision-making. Furthermore, we tested female homozygous (SERT(-/-)) and heterozygous (SERT(+/-)) serotonin transporter knockout rats in a rodent version of the IGT. Women homozygous for the short (s) allele of the 5-HTTLPR were found to choose more disadvantageously than women homozygous for the long allele of the 5-HTTLPR as the IGT progressed. In the rat, SERT(-/-) and SERT(+/-) were associated with advantageous decision-making compared to SERT(+/+) as the IGT progressed. Combining the human and rat observations, we show that SERT dosage affects the maintenance of a once established choice option, irrespective of the choice (advantageous or disadvantageous) that has been made. We postulate that the SERT-mediated effects relate to deficits in the processing of choice outcome to guide subsequent choices in this gamble-based test, and that SERT(-/-) and SERT(+/-) rodent models in combination with studies in humans can be used to provide insight in the modulatory effects of 5-HTTLPR.

Rodent versions of the iowa gambling task: opportunities and challenges for the understanding of decision-making.

Impaired decision-making is a core problem in several psychiatric disorders including attention-deficit/hyperactivity disorder, schizophrenia, obsessive–compulsive disorder, mania, drug addiction, eating disorders, and substance abuse as well as in chronic pain. To ensure progress in the understanding of the neuropathophysiology of these disorders, animal models with good construct and predictive validity are indispensable. Many human studies aimed at measuring decision-making capacities use the Iowa gambling task (IGT), a task designed to model everyday life choices through a conflict between immediate gratification and long-term outcomes. Recently, new rodent models based on the same principle have been developed to investigate the neurobiological mechanisms underlying IGT-like decision-making on behavioral, neural, and pharmacological levels. The comparative strengths, as well as the similarities and differences between these paradigms are discussed. The contribution of these models to elucidate the neurobehavioral factors that lead to poor decision-making and to the development of better treatments for psychiatric illness is considered, along with important future directions and potential limitations.

Toward a rodent model of the Iowa gambling task

The Iowa gambling task in humans is, in principle, suited for the study of the long-term efficiency of behavior in a biologically relevant context. Key features of this task are uncertainty of outcomes and a conflict between the immediate and the long-term payoff options. Animal models allow us to study the underlying neurobiology of decision-making processes and the long-term efficiency of behavior in more detail and at a greater depth than is possible in humans. Therefore, we set out to develop a model of this task in rodents, using the task’s key features. In this article, we describe the results of the first series of experiments with rats and mice. The data thus far suggest that mice and rats behave in a way similar to humans; that is, they tend to choose the option with the best long-term payoff more often as the test progresses.

The effect of reward magnitude differences on choosing disadvantageous decks in the Iowa Gambling Task

In the Iowa Gambling Task, participants have to develop a long-term profitable monetary scenario in a situation of uncertainty and a conflict between the chance of encountering an immediate large reward (100 US dollars) in two long-term loosing decks (A and B; -250 US dollars per 10 cards) and the chance of encountering an immediate small reward (50 US dollars) in two long-term winning decks (C and D; +250 US dollars per 10 cards). The ratio of the immediate rewards in decks A/B and C/D is thus 2:1. Here, we manipulated these differences in reward magnitude between the advantageous (C/D) and disadvantageous (A/B) decks, while keeping the net gains and losses per 10 cards the same, to assess the impact of the conflict between immediate and distant pay-off on choice behaviour. Participants selected less cards from disadvantageous decks and won more money when the reward magnitude difference was decreased, A/B:C/D=1:1, while they selected more cards from disadvantageous decks and lost more money when reward magnitude differences were increased, A/B:C/D=4:1 and 6:1. This study shows that the outcome in the Iowa Gambling Task is sensitive to differences between the magnitude of immediate rewards in the advantageous and disadvantageous decks.

Evolved to satisfy our immediate needs: Self-control and the rewarding properties of food

Evolutionary explanations of overeating in modern society emphasize that humans have evolved to eat to their physiological limits when food is available. The present paper challenges the idea that eating is driven by the availability of food only and proposes that it is regulated by strategic anticipatory behaviour in service of the most profitable long-term scenario as well. Our alternative explanation emphasizes the interaction between the reward system that regulates the liking and wanting of food and the role of self-control, which is involved in maintaining the best outcome in the long run. r 2006 Elsevier Ltd. All rights reserved.

Mu-opioid receptor knockout mice show diminished food-anticipatory activity

We have previously suggested that during or prior to activation of anticipatory behaviour to a coming reward, µ‐opioid receptors are activated. To test this hypothesis schedule induced food‐anticipatory activity in µ‐opioid receptor knockout mice was measured using running wheels. We hypothesized that µ‐knockout mice show little food‐anticipatory activity. In wildtype mice we observed that food‐anticipatory activity increased proportional to reduced food intake levels during daily scheduled food access, and thus reflects the animals physiological need for food. µ‐Knockout mice do not adjust their schedule induced running wheel behaviour prior to and during feeding time in the same way as wildtype mice; rather than showing more running wheel activity before than during feeding, they showed an equal amount of activity before and during feeding. As food‐anticipatory activity is dependent on the mesolimbic dopamine system and µ‐opioid receptors regulate dopaminergic activity, these data suggest a change in the dopamine systems activity in µ‐knockout mice. As we observed that µ‐knockout mice tended to show a stronger locomotor activity response than wildtype mice to the indirect dopamine agonist d‐amphetamine, it appears that the dopaminergic system per se is intact and sensitive to activation. We found no differences in the expression of pro‐opiomelanocortin, a precursor of endogenous endorphin, in the arcuate nucleus between µ‐knockout mice and wildtype mice during restricted feeding, showing that the µ‐opioid receptor does not regulate endogenous endorphin levels. These data overall suggest a role for µ‐opioid receptors in adapting reward related behaviour to the requirements of the environment.

Transient Inactivation of the Medial Prefrontal Cortex Affects Both Anxiety and Decision-Making in Male Wistar Rats

In both humans and rats high levels of anxiety impair decision-making in the Iowa gambling task (IGT) in male subjects. Expression of the immediate early gene c-fos as marker of neural activity in rat studies indicated a role of the medial prefrontal cortex (prelimbic and infralimbic region; mPFC) in mediating the relationship between anxiety and decision-making. To delineate this relationship further and assess the underlying neurobiology in more detail, we inactivated in the present study the mPFC in male rats using a mixture of the GABA-receptor agonists muscimol and baclofen. Rats were exposed to the elevated plus maze (EPM) to measure effects on anxiety and to the rodent version of the IGT (r-IGT). Inactivation led to increased levels of anxiety on the EPM, while not affecting general activity. The effect in the r-IGT (trials 61–120) was dependent on levels of performance prior to inactivation (trial 41–60): inactivation of the mPFC hampered task performance in rats, which already showed a preference for the advantageous option, but not in rats which were still choosing in a random manner. These data suggest that the mPFC becomes more strongly involved as rats have learned task-contingencies, i.e., choose for the best long-term option. Furthermore they suggest, along with the data of our earlier study, that both anxiety and decision-making in rats are mediated through a neural circuitry including at least the mPFC. The data are discussed in relation to recent data of rodent studies on the neural circuitry underlying decision-making.