If You Ignore Manipulation Errors in Your Experiments You Might Be P-Hacking Without Knowing.

Abstract

This research brief is a cautionary note on the dangers of using p-values as empirical evidence for crossover effects in consumer research. If you rely on experimental manipulation to produce between-subjects variation on a moderator construct, and then test for interaction effects on the manipulation (rather than the moderator construct), your evidence of a crossover effect might be based on spurious statistical significance. The main purpose here is to alert experimenters about ignoring manipulation errors, rather than proposing specific solutions (even though I briefly review the literature proposing methods to ameliorate the situation).