Hackjin Kim

Is Avoiding an Aversive Outcome Rewarding? Neural Substrates of Avoidance Learning in the Human Brain

Avoidance learning poses a challenge for reinforcement-based theories of instrumental conditioning, because once an aversive outcome is successfully avoided an individual may no longer experience extrinsic reinforcement for their behavior. One possible account for this is to propose that avoiding an aversive outcome is in itself a reward, and thus avoidance behavior is positively reinforced on each trial when the aversive outcome is successfully avoided. In the present study we aimed to test this possibility by determining whether avoidance of an aversive outcome recruits the same neural circuitry as that elicited by a reward itself. We scanned 16 human participants with functional MRI while they performed an instrumental choice task, in which on each trial they chose from one of two actions in order to either win money or else avoid losing money. Neural activity in a region previously implicated in encoding stimulus reward value, the medial orbitofrontal cortex, was found to increase, not only following receipt of reward, but also following successful avoidance of an aversive outcome. This neural signal may itself act as an intrinsic reward, thereby serving to reinforce actions during instrumental avoidance.

Model-Based fMRI and Its Application to Reward Learning and Decision Making

Abstract:u2002 In model‐based functional magnetic resonance imaging (fMRI), signals derived from a computational model for a specific cognitive process are correlated against fMRI data from subjects performing a relevant task to determine brain regions showing a response profile consistent with that model. A key advantage of this technique over more conventional neuroimaging approaches is that model‐based fMRI can provide insights into how a particular cognitive process is implemented in a specific brain area as opposed to merely identifying where a particular process is located. This review will briefly summarize the approach of model‐based fMRI, with reference to the field of reward learning and decision making, where computational models have been used to probe the neural mechanisms underlying learning of reward associations, modifying action choice to obtain reward, as well as in encoding expected value signals that reflect the abstract structure of a decision problem. Finally, some of the limitations of this approach will be discussed.

Overlapping Responses for the Expectation of Juice and Money Rewards in Human Ventromedial Prefrontal Cortex

Although much is known about the neural substrates of reward, the question of whether expectation of different types of reinforcers engage distinct or overlapping brain circuitry has not been addressed definitively. In the present study, human subjects, while being scanned with functional magnetic resonance imaging, performed a simple reward-based action selection task to obtain different magnitudes of either monetary outcomes (winning or losing money) or juice outcomes (pleasant apple juice or an unpleasant salt flavor). At the group level, we found partially overlapping value-related activity within ventromedial prefrontal cortex (vmPFC) during anticipation of juice and money reward outcomes. Analogous results were found in the right anterior insula, except that this region showed negative correlations as a function of increasing expected reward. These results indicate that vmPFC and anterior insula contain overlapping representations of anticipatory value, consistent with the existence of a common currency for the value of expected outcomes in these regions.

Temporal isolation of neural processes underlying face preference decisions

Decisions about whether we like someone are often made so rapidly from first impressions that it is difficult to examine the engagement of neural structures at specific points in time. Here, we used a temporally extended decision-making paradigm to examine brain activation with functional MRI (fMRI) at sequential stages of the decision-making process. Activity in reward-related brain structures—the nucleus accumbens (NAC) and orbitofrontal cortex (OFC)—was found to occur at temporally dissociable phases while subjects decided which of two unfamiliar faces they preferred. Increases in activation in the OFC occurred late in the trial, consistent with a role for this area in computing the decision of which face to choose. Signal increases in the NAC occurred early in the trial, consistent with a role for this area in initial preference formation. Moreover, early signal increases in the NAC also occurred while subjects performed a control task (judging face roundness) when these data were analyzed on the basis of which of those faces were subsequently chosen as preferred in a later task. The findings support a model in which rapid, automatic engagement of the NAC conveys a preference signal to the OFC, which in turn is used to guide choice.

A neural basis for the effect of candidate appearance on election outcomes

Election outcomes correlate with judgments based on a candidates visual appearance, suggesting that the attributions viewers make based on appearance, so-called thin-slice judgments, influence voting. Yet, it is not known whether the effect of appearance on voting is more strongly influenced by positive or negative attributions, nor which neural mechanisms subserve this effect. We conducted two independent brain imaging studies to address this question. In Study 1, images of losing candidates elicited greater activation in the insula and ventral anterior cingulate than images of winning candidates. Winning candidates elicited no differential activation at all. This suggests that negative attributions from appearance exert greater influence on voting than do positive. We further tested this hypothesis in Study 2 by asking a separate group of participants to judge which unfamiliar candidate in a pair looked more attractive, competent, deceitful and threatening. When negative attribution processing was enhanced (specifically, under judgment of threat), images of losing candidates again elicited greater activation in the insula and ventral anterior cingulate. Together, these findings support the view that negative attributions play a critical role in mediating the effects of appearance on voter decisions, an effect that may be of special importance when other information is absent.

Parsing brain activity associated with acupuncture treatment in Parkinson's diseases

Acupuncture, a common treatment modality within complementary and alternative medicine, has been widely used for Parkinsons disease (PD). Using functional magnetic resonance imaging (fMRI), we explored the neural mechanisms underlying the effect of specific and genuine acupuncture treatment on the motor function in patients with PD. Three fMRI scans were performed in random order in a block design, one for verum acupuncture (VA) treatment, another one for a covert placebo (CP), and the third one for an overt placebo (OP) at the motor function implicated acupoint GB34 on the left foot of 10 patients with PD. We calculated the contrast that subtracts the blood‐oxygen‐level dependent (BOLD) response for the acupuncture effect (VA vs. CP) and the placebo effect (CP vs. OP). We found a significant improvement in the motor function of the affected hand after acupuncture treatment. The putamen and the primary motor cortex were activated when patients with PD received the acupuncture treatment (VA vs. CP) and these activations correlated with individual enhanced motor function. Expectation towards acupuncture modality (CP vs. OP) elicited activation over the anterior cingulate gyrus, the superior frontal gyrus, and the superior temporal gyrus. These findings suggest that acupuncture treatment might facilitate improvement in the motor functioning of patients with PD via the basal ganglia‐thalamocortical circuit.

The neural substrates of verum acupuncture compared to non-penetrating placebo needle: An fMRI study

Acupuncture, an ancient East Asian therapeutic technique, is currently emerging as an important modality in complementary and alternative medicine around the world. Several studies have provided useful information regarding neurophysiological mechanisms of acupuncture in human brain activation. We explored brain activation using functional magnetic resonance imaging (fMRI) and compared verum acupuncture to placebo needles. Two fMRI scans were taken in random order in a block design, one for verum acupuncture and one for non-penetrating placebo needles at the motor function-implicated acupoint LR2, on the left foot, in 10 healthy volunteers. We calculated the contrast that subtracted the blood oxygen level-dependent (BOLD) responses between the verum and sham acupuncture. Verum acupuncture stimulation elicited significant activation in both motor function-related brain areas, including the caudate, claustrum, and cerebellum, and limbic-related structures, such as the medial frontal gyrus, the cingulate gyrus, and the fusiform gyrus. These findings suggest that acupuncture not only elicited acupoint-implicated brain activation, but also modulated the affective components of the pain matrix. The current investigation of the specific pattern of the brain activation related to genuine acupuncture provides new information regarding the neurobiological basis of acupuncture.

Effect of pulsed radiofrequency for postherpetic neuralgia

Background: Postherpetic neuralgia (PHN) is one of the most intractable pain disorders, particularly among elderly patients. Lesioning of dorsal root ganglion (DRG) using pulsed radiofrequency (PRF) has shown pain reduction for PHN. We assessed the efficacy of PRF lesioning of DRG for PHN via an open, nonrandomized study.

Spatial gradient in value representation along the medial prefrontal cortex reflects individual differences in prosociality

Significance How do selfish and prosocial brains function differently with regard to valuing the welfare of others? The present study addresses this question by combining neuroimaging, computational modeling, and an instrumental conditioning paradigm. Contrary to the conventional notion of the dorsal medial prefrontal cortex (MPFC) implicated in mentalization, we found that it was selfish individuals who showed greater spatial segregation between ventral and dorsal MPFC, which encoded self- and other-regarding values, respectively. Prosocial individuals, on the other hand, were characterized by overlapping self–other representation in the ventral MPFC and by stronger functional coupling between MPFC and striatum while representing and updating the value of other-regarding choices. These findings provide rigorous scientific evidence of neural markers reflecting individual differences in human prosociality. Despite the importance of valuing another person’s welfare for prosocial behavior, currently we have only a limited understanding of how these values are represented in the brain and, more importantly, how they give rise to individual variability in prosociality. In the present study, participants underwent functional magnetic resonance imaging while performing a prosocial learning task in which they could choose to benefit themselves and/or another person. Choice behavior indicated that participants valued the welfare of another person, although less so than they valued their own welfare. Neural data revealed a spatial gradient in activity within the medial prefrontal cortex (MPFC), such that ventral parts predominantly represented self-regarding values and dorsal parts predominantly represented other-regarding values. Importantly, compared with selfish individuals, prosocial individuals showed a more gradual transition from self-regarding to other-regarding value signals in the MPFC and stronger MPFC–striatum coupling when they made choices for another person rather than for themselves. The present study provides evidence of neural markers reflecting individual differences in human prosociality.

Individual differences in smoking-related cue reactivity in smokers: An eye-tracking and fMRI study

Measures of cue reactivity provide a means of studying and understanding addictive behavior. We wanted to examine the relationship between different cue reactivity measures, such as attentional bias and subjective craving, and functional brain responses toward smoking-related cues in smokers. We used eye-tracking measurements, a questionnaire for smoking urges-brief and functional magnetic resonance imaging to assess the responses to smoking-related and neutral visual cues from 25 male smokers after 36 h of smoking abstinence. Regression analyses were conducted to determine the correlation between cue-evoked brain responses and the attentional bias to smoking-related cues. The eye gaze dwell time percentage was longer in response to smoking-related cues than neutral cues, indicating significant differences in attentional bias towards smoking-related cues. The attentional bias to smoking-related cues correlated with subjective craving ratings (r=0.660, p<0.001). The dorsolateral prefrontal cortex, the putamen, the posterior cingulate cortex and the primary motor cortex were associated with the attentional bias to smoking-related cues, whereas the orbitofrontal cortex, the insula and the superior temporal gyrus were associated with smoking-related cue-induced craving and smoking urges. These results suggest that attentional mechanisms in combination with motivational and reward-related mechanisms play a role in smoking-related cue reactivity. We confirmed a positive correlation between different smoking-related cue reactivities, such as attentional bias and subjective craving, and functional brain responses in various individuals. Further studies in this field might contribute to a better individualized understanding of addictive behavior.