David Mellor

The preregistration revolution

Progress in science relies in part on generating hypotheses with existing observations and testing hypotheses with new observations. This distinction between postdiction and prediction is appreciated conceptually but is not respected in practice. Mistaking generation of postdictions with testing of predictions reduces the credibility of research findings. However, ordinary biases in human reasoning, such as hindsight bias, make it hard to avoid this mistake. An effective solution is to define the research questions and analysis plan before observing the research outcomes—a process called preregistration. Preregistration distinguishes analyses and outcomes that result from predictions from those that result from postdictions. A variety of practical strategies are available to make the best possible use of preregistration in circumstances that fall short of the ideal application, such as when the data are preexisting. Services are now available for preregistration across all disciplines, facilitating a rapid increase in the practice. Widespread adoption of preregistration will increase distinctiveness between hypothesis generation and hypothesis testing and will improve the credibility of research findings.

Studying citizen science through adaptive management and learning feedbacks as mechanisms for improving conservation

Citizen science has generated a growing interest among scientists and community groups, and citizen science programs have been created specifically for conservation. We examined collaborative science, a highly interactive form of citizen science, which we developed within a theoretically informed framework. In this essay, we focused on 2 aspects of our framework: social learning and adaptive management. Social learning, in contrast to individual-based learning, stresses collaborative and generative insight making and is well-suited for adaptive management. Adaptive-management integrates feedback loops that are informed by what is learned and is guided by iterative decision making. Participants engaged in citizen science are able to add to what they are learning through primary data collection, which can result in the real-time information that is often necessary for conservation. Our work is particularly timely because research publications consistently report a lack of established frameworks and evaluation plans to address the extent of conservation outcomes in citizen science. To illustrate how our framework supports conservation through citizen science, we examined how 2 programs enacted our collaborative science framework. Further, we inspected preliminary conservation outcomes of our case-study programs. These programs, despite their recent implementation, are demonstrating promise with regard to positive conservation outcomes. To date, they are independently earning funds to support research, earning buy-in from local partners to engage in experimentation, and, in the absence of leading scientists, are collecting data to test ideas. We argue that this success is due to citizen scientists being organized around local issues and engaging in iterative, collaborative, and adaptive learning.

Four simple recommendations to encourage best practices in research software

Scientific research relies on computer software, yet software is not always developed following practices that ensure its quality and sustainability. This manuscript does not aim to propose new software development best practices, but rather to provide simple recommendations that encourage the adoption of existing best practices. Software development best practices promote better quality software, and better quality software improves the reproducibility and reusability of research. These recommendations are designed around Open Source values, and provide practical suggestions that contribute to making research software and its source code more discoverable, reusable and transparent. This manuscript is aimed at developers, but also at organisations, projects, journals and funders that can increase the quality and sustainability of research software by encouraging the adoption of these recommendations.

Female preference in the context of male–male interactions in Maylandia zebra of Lake Malawi

The mbuna cichlids of Lake Malawi are a diverse, monophyletic, and recently derived clade. Sexual selection is thought to have accelerated their rapid diversification, though the mechanism by which this has occurred remains unknown. In this study, we examine the effect that male–male interactions have on female preference. We first used a short interaction experiment as a proxy for male dominance. We then measured female preference for (1) a single, isolated male; (2) a group of three, highly ranked males, relative to a group of lower ranked males; and (3) a group of three, mixed-ranked males against a similar group of mixed-rank males. We found that male dominance was highly correlated with male standard length. Female preference for male standard length and for dominance rank was significant in both isolated and group interactions. However, females only showed preference for groups of interacting males when males were segregated by dominance rank, not when groups were composed of both dominant and subordinate males. The results suggest that male–male contests influence female mating decisions. By investigating the rules that dictate the outcomes of such interactions we can elucidate the role that behavior plays in the diversification of this species-rich lineage.

Female Maylandia zebra prefer victorious males

Females of a widespread species of the rock-dwelling haplochromine cichlids of Lake Malawi, Maylandia zebra, show preference for males that successfully evict intruding males from their territory. This behaviour, experimentally induced by the investigators in a laboratory setting, was also preferred over males that were not permitted to interact with any other individual.

Male interactions in a group of Malaŵi cichlids

The haplochromine cichlids of the East African rift lakes comprise one of the most diverse radiations of vertebrates. The goal of the study was to investigate sexual selection in the Mbuna, a group of Lake Malaŵi haplochromine cichlids. In particular, the effects of male morphology, dominance behaviour, and territory were investigated in the context of female territory visits and male territory preference analyses. Female visits of males in the field were associated with aspects of male behaviour, body depth, and territory. Data from the laboratory suggest that males hold similar preferences for types of territory which to defend. It is suggested that male–male interaction should be given more attention in the study of the reproductive isolation and differential male mating success in the Mbuna cichlids of Lake Malaŵi.

Troubled Transitions into College and the Effects of a Small Intervention Course.

Student attrition from colleges in the United States is a widespread phenomenon, posing real stresses to students, their families, and to universities. We examined the causes of poor academic performance in students first semester through interviews and questionnaires and administered a small intervention course to freshmen on academic probation the following semester. This intervention had a modest positive effect on retention and academic self-efficacy but not on locus of control or grade point average. Finally, the 10% lower attrition rate among the students in our Portals to Academic Student Success courses suggests that small-scale interventions can mitigate attrition among students who later prove an ability to continue matriculating.

Effect of carotenoids on cichlid phenotype and mating behavior

Carotenoids are suspected to be honest indicators of health and adaptation for some cichlid species. In this study, male and female Metriaclima zebra cichlid fishes were fed a specialized diet of carotenoids, and their phenotypic expression and behavioral changes were measured. The effect of carotenoids on female–male mate choice behavioral interactions was also measured. When males were fed either a high- or low-carotenoid diet there was no statistically significant difference in coloration. There was, however, a statistically significant difference in coloration when males were reared in social versus non-social conditions. In Metriaclima zebra cichlids, carotenoids do not appear to be an important factor for mate choice, but other rearing conditions may be.

Reply to Ledgerwood: Predictions without analysis plans are inert

Ledgerwood (1) argues that there are two independent uses of preregistration that are conflated in Nosek et al. (2) and elsewhere: “Preregistering theoretical predictions enables theory falsifiability. Preregistering analysis plans enables type I error control.” We appreciate that the comment elevates the complementary roles of prediction and analysis plans in preregistration. We disagree that they are conflated in the sense of being “two types of preregistration.”nnTo enable theory falsification, we agree that a preregistration should offer a prediction derived from theory and provide the theoretical context. However, a prediction without an analysis plan is inert for falsification. An analysis plan is necessary to specify how the prediction will be tested with … nn[↵][1]1To whom correspondence should be addressed. Email: nosek{at}virginia.edu.nn [1]: #xref-corresp-1-1

Protocol transparency is vital for registered reports

To the Editor — We welcome Hardwicke and Ioannidis’ timely evaluation of the registered reports article type1, now offered at over 120 journals across the life and social sciences (https://cos.io/rr/). Hardwicke and Ioannidis identify two main shortcomings of registered reports: lack of protocol transparency and lack of standardized protocol registration. Both are important issues. Protocol transparency is essential for enabling readers to compare time-stamped, accepted stage 1 protocols with the Introduction and Methods of published stage 2 articles. Standardization of registration helps ensure that published protocols are comprehensible and verifiable. For example, work such as the COMPare campaign is only possible because of the transparency afforded by consistently registered clinical trials2. Here we report the steps we are taking to address these concerns. Since August 2017, the recommended ‘template’ editorial policy for registered reports at the Center for Open Science has stated that authors must register their stage 1 protocols on a recognized repository at the point of in-principle acceptance, either publicly or under temporary embargo until submission (or acceptance) of the stage 2 manuscript3. Since then, most new adopters have implemented this policy as a matter of course. At the time of Hardwicke and Ioannidis’ analysis, however, of the 70 journals that had adopted registered reports permanently as an available article type, only 50% required protocol registration or routinely published the accepted protocols. Beginning in April 2018, we have contacted the editors of these journals to recommend updating their policies. To date, 76% of the now 88 permanent adopters either require protocol transparency or will do so imminently (Fig. 1). While no editors have yet declined the update, 24% of journals have policies that are either unclear on protocol transparency or do not require it. These editors are either considering our request or have not responded. We will continue to pursue this matter and are confident of achieving near-total compliance, though journals are not obliged to follow our recommendations. To facilitate standardized registration, we have created a registry for stage 1 protocols that have been granted in-principle acceptance, accompanied by a simple interface4. The journals Cortex and Animal Behavior and Cognition now use this tool to register protocols on behalf of authors, further streamlining the process. An alternative strategy for protocol transparency is for the journal itself to publish protocols, as recommended by Wiley5. We will populate the Center for Open Science registry with as many unpublished protocols as possible, seeking them from the authors of the completed stage 2 articles and their respective journal editors. For protocols that remain unavailable, a simple entry will state so. Beyond issues of protocol transparency and standardization, there is the broader question of how registered reports differ from regular empirical articles. Ongoing studies are exploring indicators such as the citation impact and prevalence of positive results in registered reports6,7, and it will be important to also monitor the effectiveness of, and compliance with, registered report journal policies. As this meta-scientific endeavour continues, registered reports are transitioning into new fields, aligning with post-publication peer review8 and being integrated into funding streams9–12. Registered reports are a rapidly evolving initiative. Once considered impossibly Colla bra: P sych ology