Call for Papers “Advancing the Reproducibility of Psychological Assessment Across Borders and Populations”

Abstract

The role of psychological assessment across a variety of educational, clinical, and organisational settings is expanding. This rapid uptake is closely related to the growing demand for evidence-based practice and policymaking, along with the evidence of the economic benefits of effective measurement. This practice-oriented development of the discipline demands close attention to be paid to ensure the psychometric quality and translatability of methods and tools across borders, domains, and populations. Indeed, researchers have widely commented on such issues, including the acknowledgement that the majority of the findings and tools in the behavioural science are derived from WEIRD (Western, educated, industrialized, rich, and democratic) populations (Henrich, Heine, & Norenzayan, 2010). While this is a challenge for researchers, it provides fruitful grounds to advance both science and its role in society through better methodological standards. In particular, it demands addressing obstacles such as the selection of appropriate methods for assessment conducted across borders, populations, and languages. Doing so effectively improves the efficiency of data collection, achieving cross-cultural validation, and improving accessibility of assessments. Investment in such questions further creates potential to train a new generation of psychometricians able to work in dynamic contexts and to deliver complex concepts to various audiences outside the immediate assessment field. For such reasons, this Call for Papers is specifically aimed to highlight work conducted by junior researchers, particularly for studies either adapting to a new setting or language, or in attempting methods that combine multiple settings and languages as a feature of their research. Along with a growing need to adapt assessments and their delivery, there is a concern that in a scientific climate where novel and positive results are considered more publishable than replication studies and negative findings, researchers may have little incentive to conduct important validation work and publish failed replications. Such issues are gaining considerable attention, particularly in light of the replication crisis reported in psychological science (Open Science Collaboration, 2015). Registered Reports (RR) are a form of manuscript submission that has been suggested as a potential tool to improve the reproducibility and combat the misaligned incentive structures of the field (Munafò et al., 2017). In the field of assessment research, RRs should outline assessment details and analysis plans and are submitted for review before data collection begins. This format is favoured as it holds the potential to help eliminate questionable research practices and to ensure that the peer review process is not biased by the reported results.