Alicia Hofelich Mohr

Data availability, reusability, and analytic reproducibility: Evaluating the impact of a mandatory open data policy at the journal Cognition

Access to data is a critical feature of an efficient, progressive and ultimately self-correcting scientific ecosystem. But the extent to which in-principle benefits of data sharing are realized in practice is unclear. Crucially, it is largely unknown whether published findings can be reproduced by repeating reported analyses upon shared data (‘analytic reproducibility’). To investigate this, we conducted an observational evaluation of a mandatory open data policy introduced at the journal Cognition. Interrupted time-series analyses indicated a substantial post-policy increase in data available statements (104/417, 25% pre-policy to 136/174, 78% post-policy), although not all data appeared reusable (23/104, 22% pre-policy to 85/136, 62%, post-policy). For 35 of the articles determined to have reusable data, we attempted to reproduce 1324 target values. Ultimately, 64 values could not be reproduced within a 10% margin of error. For 22 articles all target values were reproduced, but 11 of these required author assistance. For 13 articles at least one value could not be reproduced despite author assistance. Importantly, there were no clear indications that original conclusions were seriously impacted. Mandatory open data policies can increase the frequency and quality of data sharing. However, suboptimal data curation, unclear analysis specification and reporting errors can impede analytic reproducibility, undermining the utility of data sharing and the credibility of scientific findings.

Thinking Inside the Box

While the visual design of a question has been shown to influence responses in survey research, it is less understood how these effects extend to assessment-based questions that attempt to measure how, rather than just what, a respondent thinks. For example, in a divergent thinking task, the number and elaboration of responses, not just how original they are, contribute to the assessment of creativity. Using the Alternative Uses Task in an online survey, we demonstrated that scores on fluency, elaboration, and originality, core constructs of participants’ assessed creative ability, were systematically influenced by the visual design of the response boxes. The extent to which participants were susceptible to these effects varied with individual differences in trait conscientiousness, as several of these effects were seen in participants with high, but not low, conscientiousness. Overall, our results are consistent with previous survey methodology findings, extend them to the domain of creativity research, and call for increased awareness and transparency of visual design decisions across research fields.

Risk prioritization of pork supply movements during an FMD outbreak in the US - Data and Materials

The Data.csv file contains the raw survey responses (location information collected by Qualtrics has been removed). Information about the variables and value labels can be found in the DataDictionary.txt file. The data can be read into the Analysis_Code.R file to perform analysis described in the paper and to create a static version of the Movements.html graph. Survey.pdf contains the survey questions with relevant skip and display logic.

Prioritization of Managed Pork Supply Movements during a FMD Outbreak in the US

In the event of a foot-and-mouth disease (FMD) outbreak in the United States, local, state, and federal authorities will implement a foreign animal disease emergency response plan restricting the pork supply chain movements and likely disrupting the continuity of the swine industry business. To minimize disruptions of the food supply while providing an effective response in an outbreak, it is necessary to have proactive measures in place to ensure minimal disease spread and maximum continuation of business. Therefore, it is critical to identify candidate movements for proactive risk assessments: those that are both most likely to contribute to disease spread and most necessary for business continuity. To do this, experts from production, harvest, retail, and allied pork industries assessed 30 common pork supply movements for risk of disease spread and industry criticality. The highest priority movements for conducting a risk assessment included the movement of weaned pigs originating from multiple sow farm sources to an off-site nursery or wean to finish facility, the movement of employees or commercial crews, the movement of vaccination crews, the movement of dedicated livestock hauling trucks, and the movement of commercial crews such as manure haulers and feed trucks onto, off, or between sites. These critical movements, along with several others identified in this study, will provide an initial guide for prioritization of risk management efforts and resources to be better prepared in the event of a FMD outbreak in the United States. By specifically and proactively targeting movements that experts agree are likely to spread the disease and are critical to the continuity of business operations, potentially catastrophic consequences in the event of an outbreak can be limited.