Anthony N. Washburn

The state of social and personality science: Rotten to the core, not so bad, getting better, or getting worse?

The scientific quality of social and personality psychology has been debated at great length in recent years. Despite research on the prevalence of Questionable Research Practices (QRPs) and the replicability of particular findings, the impact of the current discussion on research practices is unknown. The current studies examine whether and how practices have changed, if at all, over the last 10 years. In Study 1, we surveyed 1,166 social and personality psychologists about how the current debate has affected their perceptions of their own and the field’s research practices. In Study 2, we coded the research practices and critical test statistics from social and personality psychology articles published in 2003–2004 and 2013–2014. Together, these studies suggest that (a) perceptions of the current state of the field are more pessimistic than optimistic; (b) the discussion has increased researchers’ intentions to avoid QRPs and adopt proposed best practices, (c) the estimated replicability of research published in 2003–2004 may not be as bad as many feared, and (d) research published in 2013–2014 shows some improvement over research published in 2003–2004, a result that suggests the field is evolving in a positive direction.

Science Denial Across the Political Divide

We tested whether conservatives and liberals are similarly or differentially likely to deny scientific claims that conflict with their preferred conclusions. Participants were randomly assigned to read about a study with correct results that were either consistent or inconsistent with their attitude about one of several issues (e.g., carbon emissions). Participants were asked to interpret numerical results and decide what the study concluded. After being informed of the correct interpretation, participants rated how much they agreed with, found knowledgeable, and trusted the researchers’ correct interpretation. Both liberals and conservatives engaged in motivated interpretation of study results and denied the correct interpretation of those results when that interpretation conflicted with their attitudes. Our study suggests that the same motivational processes underlie differences in the political priorities of those on the left and the right.

Data from a pre-publication independent replication initiative examining ten moral judgement effects

We present the data from a crowdsourced project seeking to replicate findings in independent laboratories before (rather than after) they are published. In this Pre-Publication Independent Replication (PPIR) initiative, 25 research groups attempted to replicate 10 moral judgment effects from a single laboratory’s research pipeline of unpublished findings. The 10 effects were investigated using online/lab surveys containing psychological manipulations (vignettes) followed by questionnaires. Results revealed a mix of reliable, unreliable, and culturally moderated findings. Unlike any previous replication project, this dataset includes the data from not only the replications but also from the original studies, creating a unique corpus that researchers can use to better understand reproducibility and irreproducibility in science.

Why Do Some Psychology Researchers Resist Adopting Proposed Reforms to Research Practices? A Description of Researchers’ Rationales

In response to the replication crisis, many psychologists recommended that the field adopt several proposed reforms to research practices, such as preregistration, to make research more replicable. However, how researchers have received these proposals is not well known because, to our knowledge, no systematic investigation into use of these reforms has been conducted. We wanted to learn about the rationales researchers have for not adopting the proposed reforms. We analyzed survey data of 1,035 researchers in social and personality psychology who were asked to indicate whether they thought it was acceptable to not follow four specific proposed reforms and to explain their reasoning when they thought it was acceptable to not adopt these reforms. The four reforms were preregistering hypotheses and methods, making data publicly available online, conducting formal power analyses, and reporting effect sizes. Our results suggest that (a) researchers have adopted some of the proposed reforms (e.g., reporting effect sizes) more than others (e.g., preregistering studies) and (b) rationales for not adopting them reflect a need for more discussion and education about their utility and feasibility.

A checklist to facilitate objective hypothesis testing in social psychology research.

Social psychology is not a very politically diverse area of inquiry, something that could negatively affect the objectivity of social psychological theory and research, as Duarte et al. argue in the target article. This commentary offers a number of checks to help researchers uncover possible biases and identify when they are engaging in hypothesis confirmation and advocacy instead of hypothesis testing.

Moral and religious convictions: Are they the same or different things?

People often assume that moral and religious convictions are functionally the same thing. But are they? We report on 19 studies (N = 12,284) that tested whether people’s perceptions that their attitudes are reflections of their moral and religious convictions across 30 different issues were functionally the same (the equivalence hypothesis) or different constructs (the distinct constructs hypothesis), and whether the relationship between these constructs was conditional on political orientation (the political asymmetry hypothesis). Seven of these studies (N = 5,561, and 22 issues) also had data that allowed us to test whether moral and religious conviction are only closely related for those who are more rather than less religious (the secularization hypothesis), and a narrower form of the political asymmetry and secularization hypotheses, that is, that people’s moral and religious convictions may be tightly connected constructs only for religious conservatives. Meta-analytic tests of each of these hypotheses yielded weak support for the secularization hypothesis, no support for the equivalence or political asymmetry hypotheses, and the strongest support for the distinct constructs hypothesis.