Matthew L. Locey

Effects of acute and chronic nicotine on impulsive choice in rats

Impulsive choice, or preference for small immediate reinforcers over large delayed reinforcers, has been associated with cigarette smoking. The direct effects of nicotine on impulsive choice in laboratory animals are unknown. We examined the effects of acute and chronic nicotine injections, and the termination of injections, on impulsive choice in rats. Five rats made choices between a one- and a three-pellet reinforcer in a discrete trials procedure. The delay to the smaller reinforcer was always 1u2009s. A computer adjusted the delay to the larger reinforcer until the pattern of choices reflected indifference between the two alternatives. We assessed the effects of acute and chronic nicotine (vehicle, 0.03, 0.1, 0.3 and 1.0u2009mg/kg nicotine). The latency to make the first response of the session increased under the acute 1.0u2009mg/kg dose. There were no consistent differences in the effects of acute and chronic nicotine on response latency and lever pressing during the delays between choices. Acute injections of nicotine dose-dependently increased impulsive responding. After chronic injections, impulsive responding was increased equivalently regardless of dose, and it was increased even in the absence of nicotine. After drug injections were terminated, behavior remained impulsive for about 30 drug-free sessions, and then responding gradually returned to baseline levels. The results suggest that increases in impulsive choice were not due to anorectic effects, response biases or changes in conditioned reinforcement. Nicotine may have decreased the value of delayed reinforcers. Chronic nicotine exposure produced long-lasting but reversible increases in impulsive choice.

ISOLATING BEHAVIORAL MECHANISMS OF INTERTEMPORAL CHOICE : NICOTINE EFFECTS ON DELAY DISCOUNTING AND AMOUNT SENSITIVITY

Many drugs of abuse produce changes in impulsive choice, that is, choice for a smaller-sooner reinforcer over a larger-later reinforcer. Because the alternatives differ in both delay and amount, it is not clear whether these drug effects are due to the differences in reinforcer delay or amount. To isolate the effects of delay, we used a titrating delay procedure. In phase 1, 9 rats made discrete choices between variable delays (1 or 19 s, equal probability of each) and a delay to a single food pellet. The computer titrated the delay to a single food pellet until the rats were indifferent between the two options. This indifference delay was used as the starting value for the titrating delay for all future sessions. We next evaluated the acute effects of nicotine (subcutaneous 1.0, 0.3, 0.1, and 0.03 mg/kg) on choice. If nicotine increases delay discounting, it should have increased preference for the variable delay. Instead, nicotine had very little effect on choice. In a second phase, the titrated delay alternative produced three food pellets instead of one, which was again produced by the variable delay (1 s or 19 s) alternative. Under this procedure, nicotine increased preference for the one pellet alternative. Nicotine-induced changes in impulsive choice are therefore likely due to differences in reinforcer amount rather than differences in reinforcer delay. In addition, it may be necessary to include an amount sensitivity parameter in any mathematical model of choice when the alternatives differ in reinforcer amount.

A behavioral analysis of altruism.

Altruistic acts have been defined, in economic terms, as …costly acts that confer economic benefits on other individuals (Fehr and Fischbacher, 2003). In multi-player, one-shot prisoners dilemma games, a significant number of players behave altruistically; their behavior benefits each of the other players but is costly to them. We consider three potential explanations for such altruism. The first explanation, following a suggestion by the philosopher Derek Parfit, assumes that players devise a strategy to avoid being free-loaders-and that in the present case this strategy dictates cooperation. The second explanation says that cooperators reject the one-shot aspect of the game and behave so as to maximize reward over a series of choices extending beyond the present situation (even though reward is not maximized in the present case). This explanation assumes that people may learn to extend the boundaries of their selves socially (beyond their own skin) as well as temporally (beyond the present moment). We propose a learning mechanism for such behavior analogous to the biological, evolutionary mechanism of group selection. The third explanation assumes that peoples altruism is based on a straightforward balancing of undiscounted costs to themselves against discounted benefits to others (social discounting). The three proposed explanations of altruism complement each other.

Human risky choice: delay sensitivity depends on reinforcer type.

The present study was designed to help bridge the methodological gap between human and nonhuman animal research in delay-based risky choice. In Part 1, 4 adult human subjects made repeated choices between variable-time and fixed-time schedules of 30-s video clips. Both alternatives had equal mean delays of 15 s, 30 s, or 60 s. Three of 4 subjects strongly preferred the variable-delay alternative across all conditions. In Part 2, these 3 subjects were then provided pairwise choices between 2 variable-time schedules with different delay distributions. Subjects generally preferred the variable-delay distributions with a higher probability of short-reinforcer delays, consistent with accounts based on nonlinear discounting of delayed reinforcement. There was only weak correspondence between experimental results and verbal reports. The overall pattern of results is inconsistent with prior risky choice research with human subjects but is consistent with prior results with nonhuman subjects, suggesting that procedural differences may be a critical factor determining risk-sensitivity across species.

Social discounting and the prisoner's dilemma game.

Altruistic behavior has been defined in economic terms as …costly acts that confer economic benefits on other individuals (Fehr & Fischbacher, 2003). In a prisoners dilemma game, cooperation benefits the group but is costly to the individual (relative to defection), yet a significant number of players choose to cooperate. We propose that people do value rewards to others, albeit at a discounted rate (social discounting), in a manner similar to discounting of delayed rewards (delay discounting). Two experiments opposed the personal benefit from defection to the socially discounted benefit to others from cooperation. The benefit to others was determined from a social discount function relating the individuals subjective value of a reward to another person and the social distance between that individual and the other person. In Experiment 1, the cost of cooperating was held constant while its social benefit was varied in terms of the number of other players, each gaining a fixed, hypothetical amount of money. In Experiment 2, the cost of cooperating was again held constant while the social benefit of cooperating was varied by the hypothetical amount of money earned by a single other player. In both experiments, significantly more participants cooperated when the social benefit was higher.

Valuing rewards to others in a prisoner’s dilemma game

People value rewards to others but discount those rewards based on social distance; rewards to a socially closer person are valued more than identical rewards to a socially more distant person (Jones and Rachlin, 2006). The concept of social discounting can explain cooperation and defection in two-player prisoners dilemma (PD) games (Axelrod, 1980). The contingencies of a PD game are such that in any single game cooperation is costly to each player herself but beneficial to the other player. From the viewpoint of each player, the costs of cooperation are fully realized, but the benefits of cooperation are discounted by the social distance to the other player. The present experiment measured cooperation and defection in two PD-game conditions with differing reward magnitudes. In one (the 1-2-3-4 condition), the cost of cooperation exceeded its socially discounted benefit, and players were predicted to defect; in the other (the 1-2-9-10 condition), the discounted benefit of cooperation exceeded its cost, and players were predicted to cooperate. Over the course of repeated trials defection increased with the 1-2-3-4 condition but not with the 1-2-9-10 condition. Moreover, participants who rated their partners as closer, relative to random classmates, cooperated at higher rates--consistent with social discounting.

SHAPING BEHAVIORAL PATTERNS

Pigeons were rewarded for distributing eight pecks across two keys (L and R) in various patterns. The simplest pattern was at least one switch between the two keys (LR or RL) anywhere during the sequence; the next simplest was at least one instance of LLRR or RRLL anywhere during the sequence; the next was LLLRRR or RRRLLL; the most complex was LLLLRRRR or RRRRLLLL. Note that each more complex pattern contains the simpler ones within it. Initially, all patterns were reinforced but amount of reinforcement varied directly with complexity of pattern. The pigeons typically began the eight-peck sequence by pecking on their dispreferred key and then switched to their preferred key during the sequence. In subsequent conditions, simpler patterns were progressively unreinforced until finally only the most complex pattern (exactly four pecks on one key followed by exactly four pecks on the other) was reinforced. Three of the 4 pigeons tested maintained responding under this contingency; responding of the 4th pigeon extinguished. A second group of 4 pigeons was exposed immediately after training to extinction of all patterns except the most complex one. Three of the pigeons failed to maintain responding and the 4th maintained responding at a very low level. These results are evidence that response patterns can be shaped directly without building them up from a sequence of individually reinforced responses. The results may serve as a model of how self-controlled and altruistic behavior can arise through reinforcement.

COMMITMENT AND SELF-CONTROL IN A PRISONER'S DILEMMA GAME

Humans often make seemingly irrational choices in situations of conflict between a particular smaller-sooner reinforcer and a more abstract, temporally extended, but larger reinforcer. In two experiments, the extent to which the availability of commitment responses-self-imposed restrictions on future choices-might improve self-control in such situations was investigated. Participants played a prisoners dilemma game against a computer that played a tit-for-tat strategy-cooperating after a participant cooperated, defecting after a participant defected. Defecting produced a small-immediate reinforcer (consisting of points convertible to gift cards) whereas cooperating increased the amount of subsequent reinforcers, yielding a greater overall reinforcer rate. Participants were normally free to cooperate or defect on each trial. Additionally, they could choose to make a commitment response that forced their choice for the ensuing five trials. For some participants, the commitment response forced cooperation; for others, it forced defection. Most participants, with either commitment response available, chose to commit repeatedly despite a minor point loss for doing so. After extended exposure to these contingencies, the commit-to-cooperate group cooperated significantly more than a control group (with no commitment available). The commit-to-defect group cooperated significantly less than the control group. When both commitment alternatives were simultaneously available-one for cooperation and one for defection-cooperation commitment was strongly preferred. In Experiment 2, the commitment alternative was removed at the end of the session; gains in cooperation, relative to the control group, were not sustained in the absence of the self-imposed behavioral scaffold.

Rate, Probability and Matching: Comments on “The Identities Hidden in the Matching Laws, and Their Uses” by David Thorne

David Thornes (2010) article, The identities hidden in the matching laws, and their uses performs a valuable service in pointing out alternative expressions of matching. However, some identities tend to obscure rather than illuminate empirical relationships. Three such problematic instances are discussed: interresponse time as a function of interval and ratio schedule parameters; probability equality as implying rate matching; the apparent simplicity of probabilistic functions, as opposed to response rate functions, of reinforcement rate.

Biological evolution and behavioral evolution: Two approaches to altruism

Altruism may be learned (behavioral evolution) in a way similar to that proposed in the target article for its biological evolution. Altruism (over social space) corresponds to self-control (over time). In both cases, one must learn to ignore the rewards to a particular (person or moment) and behave to maximize the rewards to a group (of people or moments).