Science Needs Systematic Replicability Audits

Abstract

The credibility revolution in social science has highlighted the importance of conducting replication studies. Despite this growing awareness, the value of direct replications is still hotly debated. In this article, we identify three main functions served by replication. We argue that replications are valuable when they target important or influential studies, when they provide a general estimate of the replicability rate of a population of published articles, and when they create incentives favoring replicable research. We therefore argue that the scientific community should organize systematic large-scale replication audits of two subsets of journals’ published articles: a subset of the most-cited articles, and a subset of randomly selected articles that would provide an estimate of the replicability of the journals articles. These replicability audits should pave the way for more general quality audits of scientific journals.