William J. Brady

Contextual sensitivity in scientific reproducibility

Significance Scientific progress requires that findings can be reproduced by other scientists. However, there is widespread debate in psychology (and other fields) about how to interpret failed replications. Many have argued that contextual factors might account for several of these failed replications. We analyzed 100 replication attempts in psychology and found that the extent to which the research topic was likely to be contextually sensitive (varying in time, culture, or location) was associated with replication success. This relationship remained a significant predictor of replication success even after adjusting for characteristics of the original and replication studies that previously had been associated with replication success (e.g., effect size, statistical power). We offer recommendations for psychologists and other scientists interested in reproducibility. In recent years, scientists have paid increasing attention to reproducibility. For example, the Reproducibility Project, a large-scale replication attempt of 100 studies published in top psychology journals found that only 39% could be unambiguously reproduced. There is a growing consensus among scientists that the lack of reproducibility in psychology and other fields stems from various methodological factors, including low statistical power, researcher’s degrees of freedom, and an emphasis on publishing surprising positive results. However, there is a contentious debate about the extent to which failures to reproduce certain results might also reflect contextual differences (often termed “hidden moderators”) between the original research and the replication attempt. Although psychologists have found extensive evidence that contextual factors alter behavior, some have argued that context is unlikely to influence the results of direct replications precisely because these studies use the same methods as those used in the original research. To help resolve this debate, we recoded the 100 original studies from the Reproducibility Project on the extent to which the research topic of each study was contextually sensitive. Results suggested that the contextual sensitivity of the research topic was associated with replication success, even after statistically adjusting for several methodological characteristics (e.g., statistical power, effect size). The association between contextual sensitivity and replication success did not differ across psychological subdisciplines. These results suggest that researchers, replicators, and consumers should be mindful of contextual factors that might influence a psychological process. We offer several guidelines for dealing with contextual sensitivity in reproducibility.

Emotion shapes the diffusion of moralized content in social networks

Significance Twitter and other social media platforms are believed to have altered the course of numerous historical events, from the Arab Spring to the US presidential election. Online social networks have become a ubiquitous medium for discussing moral and political ideas. Nevertheless, the field of moral psychology has yet to investigate why some moral and political ideas spread more widely than others. Using a large sample of social media communications concerning polarizing issues in public policy debates (gun control, same-sex marriage, climate change), we found that the presence of moral-emotional language in political messages substantially increases their diffusion within (and less so between) ideological group boundaries. These findings offer insights into how moral ideas spread within networks during real political discussion. Political debate concerning moralized issues is increasingly common in online social networks. However, moral psychology has yet to incorporate the study of social networks to investigate processes by which some moral ideas spread more rapidly or broadly than others. Here, we show that the expression of moral emotion is key for the spread of moral and political ideas in online social networks, a process we call “moral contagion.” Using a large sample of social media communications about three polarizing moral/political issues (n = 563,312), we observed that the presence of moral-emotional words in messages increased their diffusion by a factor of 20% for each additional word. Furthermore, we found that moral contagion was bounded by group membership; moral-emotional language increased diffusion more strongly within liberal and conservative networks, and less between them. Our results highlight the importance of emotion in the social transmission of moral ideas and also demonstrate the utility of social network methods for studying morality. These findings offer insights into how people are exposed to moral and political ideas through social networks, thus expanding models of social influence and group polarization as people become increasingly immersed in social media networks.

A Study of Intimate Partner Violence, Substance Abuse, and Sexual Risk Behaviors Among Gay, Bisexual, and Other Men Who Have Sex With Men in a Sample of Geosocial-Networking Smartphone Application Users

Geosocial-networking smartphone applications (“apps”) are widely used by gay, bisexual, and other men who have sex with men (MSM) and facilitate connections between users based on proximity and attraction. MSM have sexual encounters and relationships of varying degrees of emotional and physical intimacy with app-met individuals, potentially placing them at risk for intimate partner violence (IPV). The purpose of the current study was to utilize a geosocial-networking application to investigate relationships between experiences of IPV victimization as it relates to substance use and sexual risk behaviors in a sample of MSM. Participants (n = 175) were recruited by means of broadcast advertisements on an application widely used by MSM (Grindr) to seek sexual partners. Multivariable regression models were fit to examine associations between IPV, substance abuse, and sexual risk behaviors. Lifetime experiences of IPV victimization were common, where 37.7% of respondents reported having experienced at least one form of IPV. While a marginally significant positive association between IPV and substance abuse was detected in multivariable models (p = .095), individual forms of IPV were strongly associated with substance abuse. For example, sexual IPV victimization was associated with an increase in substance abuse in the preceding month (p = .004). Experiences of IPV victimization were associated with higher numbers of partners for both condomless receptive and insertive anal intercourse (p < .05). Given the relatively high prevalence of IPV victimization and its associations with substance abuse and sexual risk behaviors, these findings suggest that IPV screening and prevention programs may reduce substance abuse and sexual risk behaviors in this population.

In (visual) search for a new distraction: the efficiency of a novel attentional deployment versus semantic meaning regulation strategies

Cognitive emotion regulation strategies are considered the king’s highway to control affective reactions. Two broad categories of cognitive regulation are attentional deployment and semantic meaning. The basic distinctive feature between these categories is the type of conflict between regulatory and emotional processes for dominance, with an early attentional selection conflict in attentional deployment and a late appraisal selection conflict in semantic meaning. However, prior studies that tested the relative efficacy of these two regulatory categories varied the type and the degree of conflict. Our major goal was to test the relative efficacy of a novel attentional deployment strategy (visual search distraction) and a classic semantic meaning strategy (reappraisal) that have a different type of conflict but a matched degree of conflict. Specifically, visual search distraction involves a strong degree of attentional selection conflict manifested in attending subtle non-emotional features that are camouflaged within potent negative emotional stimuli. Reappraisal involves a strong degree of appraisal selection conflict manifested in construing neutral reappraisals that rely on potent negative emotional appraisals. Based on our theoretical model we hypothesized and found that visual search distraction was as effective as cognitive reappraisal in down-regulating the experience of low intensity of negative emotion (Study 1), but more effective, less effortful, and more strongly blocking emotional information processing than cognitive reappraisal when regulating high intensity (Study 2). A final study ruled out a demand characteristics explanation by showing that participants’ expectations about how they should feel diverged from how they actually reported feeling following regulation (Study 3). Our findings suggest that the basic difference in the type rather than degree of conflict between attentional deployment and semantic meaning determines strategies’ outcome.

Reply to Inbar: Contextual sensitivity helps explain the reproducibility gap between social and cognitive psychology

We find that contextual sensitivity correlates with the reproducibility of 100 psychology studies from the Reproducibility Project (1). This relationship remains after adjusting for several methodological factors believed to account for reproducibility (e.g., sample size, effect size) and does not differ across subdisciplines (social vs. cognitive psychology). We also report a strong relationship between subdiscipline and context (1): social psychology studies are more likely to examine contextually sensitive topics than cognitive psychology studies. We conclude that context might play a role in reproducibility across (rather than within) subdisciplines. Inbar (2) notes that the reproducibility rate of social psychology (28%) is lower than cognitive psychology (53%). He also … [↵][1]1To whom correspondence should be addressed. Email: jay.vanbavel{at}nyu.edu. [1]: #xref-corresp-1-1

Acceptability of ecological momentary assessment among young men who have sex with men

ABSTRACT The study evaluated the acceptability of text message– and voice-based ecological momentary assessment (EMA) methods among a sample (N = 74) of young men who have sex with men (MSM). We assessed the acceptability of text message– and voice-based EMA methods. Almost all participants (96%) reported that they would be willing to accept texts on their smartphone to answer questions about their current mood, surroundings, or feelings. A large majority (89%) also reported being willing to accept phone calls to answer these questions. This work suggests that different EMA methods are acceptable for use among young MSM.