A primer on cognitive diagnostic modelling for the healthcare professions education

Abstract

Criticism about the hegemonic psychometric paradigms currently used in the healthcare professions education includes claims of reductionism, objectification, and poor compliance to assumptions. Perhaps the most crucial criticism comes from learners difficulty in interpreting and making meaningful use of summative scores, besides the potentially detrimental impact of scores on learners. Therefore, the term post-psychometric era became popular, despite persisting calls for sensible use of modern psychometrics. In recent years cognitive diagnostic modelling (CDM) has emerged as a new psychometric paradigm capable of providing meaningful diagnostic feedback. CDM allows the measurement of multiple dimensions operationalised as categorical latent variables. These features enable more qualitative feedback over a much larger number of specific cognitive attributes, while concurrently preventing the necessity of providing numerical scores as feedback to test takers. This paper aims to provide an overview of CDM to teachers involved in developing and evaluating assessment tools used in the healthcare professions education. It introduces the foundations of CDM and illustrates its applications on a hypothetical medical knowledge test using simulated data as a proof-of-concept. CDMs may represent a revolutionary new psychometric paradigm, overcoming the known limitations found in the currently hegemonic psychometric approaches, offering the possibility of more qualitative feedback and better alignment with competency-based curricula and modern programmatic assessment frameworks.