Maya Mathur

Data availability, reusability, and analytic reproducibility: Evaluating the impact of a mandatory open data policy at the journal Cognition

Access to data is a critical feature of an efficient, progressive and ultimately self-correcting scientific ecosystem. But the extent to which in-principle benefits of data sharing are realized in practice is unclear. Crucially, it is largely unknown whether published findings can be reproduced by repeating reported analyses upon shared data (‘analytic reproducibility’). To investigate this, we conducted an observational evaluation of a mandatory open data policy introduced at the journal Cognition. Interrupted time-series analyses indicated a substantial post-policy increase in data available statements (104/417, 25% pre-policy to 136/174, 78% post-policy), although not all data appeared reusable (23/104, 22% pre-policy to 85/136, 62%, post-policy). For 35 of the articles determined to have reusable data, we attempted to reproduce 1324 target values. Ultimately, 64 values could not be reproduced within a 10% margin of error. For 22 articles all target values were reproduced, but 11 of these required author assistance. For 13 articles at least one value could not be reproduced despite author assistance. Importantly, there were no clear indications that original conclusions were seriously impacted. Mandatory open data policies can increase the frequency and quality of data sharing. However, suboptimal data curation, unclear analysis specification and reporting errors can impede analytic reproducibility, undermining the utility of data sharing and the credibility of scientific findings.

Direct-to-Consumer Drug Advertisements Can Paradoxically Increase Intentions to Adopt Lifestyle Changes

Background: Direct-to-consumer (DTC) prescription drug advertisements are thought to induce “boomerang effects,” meaning they reduce the perceived effectiveness of a potential alternative option: non-pharmaceutical treatment via lifestyle change. Past research has observed such effects using artificially created, text-only advertisements that may not adequate capture the complex, conflicting portrayal of lifestyle change in real television advertisements. In other risk domains, individual “problem status” often moderates boomerang effects, such that subjects who currently engage in the risky behavior exhibit the strongest boomerang effects. Objectives: We aimed to assess whether priming with real DTC television advertisements elicited boomerang effects on perceptions of lifestyle change and whether these effects, if present, were moderated by individual problem status. Methods: We assembled a sample of real, previously aired DTC television advertisements in order to naturalistically capture the portrayal of lifestyle change in real advertisements. We randomized 819 adults in the United States recruited via Amazon Mechanical Turk to view or not view an advertisement for a prescription drug. We further randomized subjects to judge either lifestyle change or drugs on three measures: general effectiveness, disease severity for a hypothetical patient, and personal intention to use the intervention if diagnosed with the target health condition. Results: Advertisement exposure induced a statistically significant, but weak, boomerang effect on general effectiveness (p = 0.01, partial R2 = 0.007) and did not affect disease severity score (p = 0.32, partial R2 = 0.0009). Advertisement exposure elicited a reverse boomerang effect of similar effect size on personal intentions, such that advertisement-exposed subjects reported comparatively higher intentions to use lifestyle change relative to drugs (p = 0.006, partial R2 = 0.008). Individual problem status did not significantly moderate these effects. Conclusion: In contrast to previous literature finding large boomerang effects using artificial advertisement stimuli, real television advertisements elicited only a weak boomerang effect on perceived effectiveness and elicited an unexpected reverse boomerang effect on personal intentions to use lifestyle change versus drugs. These findings may reflect real advertisements’ induction of descriptive norms and self-efficacy; future research could address such possibilities by systematically manipulating advertisement content.