Toshikazu Kuroda

Resistance to change and resurgence in humans engaging in a computer task.

The relation between persistence, as measured by resistance to change, and resurgence has been examined with nonhuman animals but not systematically with humans. The present study examined persistence and resurgence with undergraduate students engaging in a computer task for points exchangeable for money. In Phase 1, a target response was maintained on a multiple variable-interval (VI) 15-s (Rich) VI 60-s (Lean) schedule of reinforcement. In Phase 2, the target response was extinguished while an alternative response was reinforced at equal rates in both schedule components. In Phase 3, the target and the alternative responses were extinguished. In an additional test of persistence (Phase 4), target responding was reestablished as in Phase 1 and then disrupted by access to videos in both schedule components. In Phases 2 and 4, target responding was more persistent in the Rich than in the Lean component. Also, resurgence generally was greater in the Rich than in the Lean component in Phase 3. The present findings with humans extend the generality of those obtained with nonhuman animals showing that higher reinforcement rates produce both greater persistence and resurgence, and suggest that common processes underlie response persistence and relapse.

Reversal learning and resurgence of operant behavior in zebrafish (Danio rerio)

Zebrafish are used extensively as vertebrate animal models in biomedical research for having such features as a fully sequenced genome and transparent embryo. Yet, operant-conditioning studies with this species are scarce. The present study investigated reversal learning and resurgence of operant behavior in zebrafish. A target response (approaching a sensor) was reinforced in Phase 1. In Phase 2, the target response was extinguished while reinforcing an alternative response (approaching a different sensor). In Phase 3, extinction was in effect for the target and alternative responses. Reversal learning was demonstrated when responding tracked contingency changes between Phases 1 and 2. Moreover, resurgence occurred in 10 of 13 fish in Phase 3: Target response rates increased transiently and exceeded rates of an unreinforced control response. The present study provides the first evidence with zebrafish supporting reversal learning between discrete operant responses and a laboratory model of relapse. These findings open the possibility to assessing genetic influences of operant behavior generally and in models of relapse (e.g., resurgence, renewal, reinstatement).

Operant models of relapse in zebrafish (Danio rerio): Resurgence, renewal, and reinstatement

Zebrafish are a widely used animal model in biomedical research, as an alternative to mammals, for having features such as a fully sequenced genome, high fecundity, and low-cost maintenance, but behavioral research with these fish remains scarce. The present study investigated whether zebrafish could be a new animal model for studies on the relapse of behavior (e.g., addiction and overeating) after the behavior has been extinguished. Specifically, we examined whether zebrafish would show three different types of relapse commonly studied with other species: resurgence, renewal, and reinstatement. For resurgence, a target response (i.e., approaching a sensor) was established by presenting a reinforcer (i.e., shrimp eggs) contingent upon the response in Phase 1; the target response was extinguished while introducing reinforcement for an alternative response in Phase 2; neither response produced the reinforcer in Phase 3. For renewal, a target response was established under Context A in Phase 1 and was extinguished under Context B in Phase 2; the fish were placed back in Context A in Phase 3, where extinction remained in effect. For reinstatement, a target response was established in Phase 1 and was extinguished in Phase 2; the reinforcer was presented independently of responding in Phase 3. Each type of relapse occurred in Phase 3. These results replicate and extend previous findings on relapse to a new species and suggest that zebrafish can be a useful animal model for studying the interactions of biological and environmental factors that lead to relapse.

COMBINATIONS OF RESPONSE-REINFORCER RELATIONS IN PERIODIC AND APERIODIC SCHEDULES†‡

Key pecking of 4 pigeons was studied under a two-component multiple schedule in which food deliveries were arranged according to a fixed and a variable interfood interval. The percentage of response-dependent food in each component was varied, first in ascending (0, 10, 30, 70 and 100%) and then in descending orders, in successive conditions. The change in response rates was positively related to the percentage of response-dependent food in each schedule component. Across conditions, positively accelerated and linear patterns of responding occurred consistently in the fixed and variable components, respectively. These results suggest that the response-food dependency determines response rates in periodic and aperiodic schedules, and that the temporal distribution of food determines response patterns independently of the response-food dependency. Running rates, but not postfood pauses, also were positively related to the percentage of dependent food in each condition, in both fixed and variable components. Thus, the relation between overall response rate and the percentage of dependent food was mediated by responding that occurred after postfood pausing. The findings together extend previous studies wherein the dependency was either always present or absent, and increase the generality of the effects of variations in the response-food dependency from aperiodic to periodic schedules.

Signal functions in delayed discriminative stimulus control by reinforcement sources.

The discriminative functions of the response-reinforcer relation may contribute to the changes in response rates that occur when reinforcement is delayed. These properties were investigated in three experiments with pigeons using a discrete-trials conditional discrimination procedure. A concurrent variable-interval schedule was arranged on two side keys during a sample component. The key peck that ended the schedule (the sample response) initiated a delay with either a stimulus present throughout the delay interval (full signal), a stimulus present only during the first second of the interval (partial signal), or no stimulus present (unsignaled delay). The delay was followed by a choice component where one alternative was reinforced if the left sample response produced the choice component and the other if the right sample response produced the choice component. Accuracy was high with a full signal; slightly lower with a partial signal; and lowest without a signal. The results parallel the effects of similar delays programmed in conventional reinforcement schedules. This in turn suggests a possible discriminative effect of the response-reinforcer relation in the control of behavior by (delayed) reinforcement.

Response acquisition by zebrafish (Danio rerio) with delayed reinforcement: RESPONSE ACQUISITION BY ZEBRAFISH

Zebrafish (Danio rerio) is a common vertebrate animal model in biomedical research and is a promising species for studying how genes interact with environmental factors in determining behavior. The present study investigated how reinforcement parameters affect zebrafish behavior by assessing response acquisition with delayed reinforcement, which has been studied with other species (e.g., rats, pigeons, humans, etc.) but not with zebrafish. Twenty-four experimentally naïve subjects were exposed to a tandem fixed-ratio 1 differential-reinforcement-of-other-behavior x-s schedule of reinforcement, where x varied across subjects. There were six different delay-to-reinforcement durations and sets of four fish were assigned to each delay duration. All of the fish assigned to a 0-, 0.5-, or 1-s delay acquired responding. Two fish acquired responding with a 3-s delay and one fish appeared to have acquired it with a 6-s delay although the latter result was less clear. None acquired responding with a 12-s delay. These results suggest that zebrafish behavior is sensitive to delays to reinforcement and the time frame over which reinforcement is effective may be limited approximately to 6 s. This time frame is shorter than that found with other species. Practical and theoretical implications of the present finding are discussed.

Relative effects of reinforcement and punishment on human choice

ABSTRACT An experiment with humans was conducted to assess the relative effects of reinforcement and punishment on choice. Point gains and losses were programmed as reinforcers and punishers, respectively. Participants used a mouse to click two buttons on a computer screen while the ratio of point gains was varied according to a concurrent variable-interval schedule. Responding was sensitive to relative point gains in baselines. When a schedule of point loss was superimposed on one button, relative to baseline, bias away from the point-loss button was observed, indicating the punishment effects of losses. Finally, a schedule arranging equal rates of both point gains and losses was superimposed on one button. One participant showed no change in bias and two showed shifts in bias away from the alternative with the superimposed schedule relative to baseline. These results suggest that punishment effects are at least equal and sometimes greater than reinforcement effects..

A combination of Raspberry Pi and SoftEther VPN for controlling research devices via the Internet

Remote control over devices for experiments may increase the efficiency of operant research and expand the area where behavior can be studied. This article introduces a combination of Raspberry Pi® (Pi) and SoftEther VPN® that allows for such remote control via the Internet. The Pi is a small Linux computer with a great degree of flexibility for customization. Test results indicate that a Pi-based interface meets the requirement for conducting operant research. SoftEther VPN® allows for establishing an extensive private network on the Internet using a single private Wi-Fi router. Step-by-step instructions are provided in the present article for setting up the Pi along with SoftEther VPN® . Their potential for improving the way of conducting research is discussed.

An Analysis of an Autoclitic Analogue in Pigeons.

Using a conditional discrimination procedure, pigeons were exposed to a nonverbal analogue of qualifying autoclitics such as definitely and maybe. It has been suggested that these autoclitics are similar to tacts except that they are under the control of private discriminative stimuli. Instead of the conventional assumption of privacy, which precludes direct manipulation of the controlling variable, the autoclitic was here identified as a response that is jointly determined by its function as a modifier for the consequence of the tact and by some variable that modifies stimulus control of the tact. Following this modified conceptualization, a novel conditional discrimination procedure was developed as an analogue for establishing autoclitic-like behavior in pigeons. Under some conditions, autoclitic-like behavior was established. Methodological challenges in developing an autoclitic analogue in nonhumans are discussed, followed by a consideration of the implications of such analogues for the understanding of verbal and, more broadly, human behavior.

Behavioral control by the response-reinforcer correlation: RESPONSE-REINFORCER CORRELATION

Using a discrete-trials procedure, two experiments examined the effects of response-reinforcer correlations on responding while controlling molecular variables that operated at the moment of reinforcer delivery (e.g., response-reinforcer temporal contiguity, interresponse times preceding reinforcement). Each trial consisted of three successive components: Response, Timeout, and Reinforcement, with the duration of each component held constant. The correlation between the number of responses in the Response component and reinforcer deliveries in the Reinforcement component was varied. In the Positive-correlation condition, a larger number of responses in the Response component programmed a higher reinforcement rate (Experiment 1) or a shorter time to reinforcement (Experiment 2) in the Reinforcement component. Although programmed in this way, the actual reinforcer delivery was dependent on, and occurred immediately after, a response in the Reinforcement component. In the Zero-correlation condition, the programmed rates of reinforcement (Experiment 1) or the times to reinforcement (Experiment 2) in the Reinforcement component of each trial were yoked to those in the preceding Positive-correlation condition. Responding in the Response component was higher in the Positive- than in the Zero-correlation condition, without systematic changes in molecular variables. The results suggest that the response-reinforcer correlation can be a controlling variable of behavior.