Inferring signs from purposeful samples: The role of context in competency assessment.

Abstract

CONTEXT\nMedical students clinical competencies are customarily assessed using convenience samples of performance from real practice. The question is how these convenience samples can be turned into purposeful samples to extrapolate students overall competency profile at the end of medical school, particularly given the context-specificity of clinical performance. In this paper we will address this issue of inferring signs from samples using insights from the discipline of Psychology.\n\n\nTHEORETICAL PERSPECTIVE\nWe adapted Smith s theory of predictor validity of universals, occupationals and relationals to the context of clinical competency assessment. Universals are characteristics required by all working individuals and therefore not context dependent. Occupationals refer to characteristics required by certain jobs but not others and therefore are dependent on task-related features of an occupation. Relationals are required in a specific organizational context with habitual ways of working together.\n\n\nAPPLICATION\nThrough 7 propositions, we assert that generalizing from samples of assessed clinical competencies during clerkships to generic competencies (i.e., signs) is dependent on whether characteristics are universals, occupationals and relationals, with universals most and relationals least generalizable.\n\n\nCONCLUSION\nWhen determining what types of ratings to use to evaluate medical student competence, medical education has shown too little nuance in considering the degree to which particular characteristics are likely to be generalizable, approaching the issue in an all-or-none manner. Smith s distinction between Universals, Occupationals, and Relationals offers a promising way forward that has implications for assessment, student selection, and career choice.