Kevin W. Eva

What every teacher needs to know about clinical reasoning

Context  One of the core tasks assigned to clinical teachers is to enable students to sort through a cluster of features presented by a patient and accurately assign a diagnostic label, with the development of an appropriate treatment strategy being the end goal. Over the last 30 years there has been considerable debate within the health sciences education literature regarding the model that best describes how expert clinicians generate diagnostic decisions.

An admissions OSCE: the multiple mini‐interview

Context  Although health sciences programmes continue to value non‐cognitive variables such as interpersonal skills and professionalism, it is not clear that current admissions tools like the personal interview are capable of assessing ability in these domains. Hypothesising that many of the problems with the personal interview might be explained, at least in part, by it being yet another measurement tool that is plagued by context specificity, we have attempted to develop a multiple sample approach to the personal interview.

Diagnostic error and clinical reasoning

Context  There is a growing literature on diagnostic errors. The consensus of this literature is that most errors are cognitive and result from the application of one or more cognitive biases. Such biased reasoning is usually associated with ‘System 1’ (non‐analytic, pattern recognition) thinking.

Predictive validity of the multiple mini-interview for selecting medical trainees.

Introduction  In this paper we report on further tests of the validity of the multiple mini‐interview (MMI) selection process, comparing MMI scores with those achieved on a national high‐stakes clinical skills examination. We also continue to explore the stability of candidate performance and the extent to which so‐called ‘cognitive’ and ‘non‐cognitive’ qualities should be deemed independent of one another.

Factors influencing responsiveness to feedback: on the interplay between fear, confidence, and reasoning processes

Self-appraisal has repeatedly been shown to be inadequate as a mechanism for performance improvement. This has placed greater emphasis on understanding the processes through which self-perception and external feedback interact to influence professional development. As feedback is inevitably interpreted through the lens of one’s self-perceptions it is important to understand how learners interpret, accept, and use feedback (or not) and the factors that influence those interpretations. 134 participants from 8 health professional training/continuing competence programs were recruited to participate in focus groups. Analyses were designed to (a) elicit understandings of the processes used by learners and physicians to interpret, accept and use (or not) data to inform their perceptions of their clinical performance, and (b) further understand the factors (internal and external) believed to influence interpretation of feedback. Multiple influences appear to impact upon the interpretation and uptake of feedback. These include confidence, experience, and fear of not appearing knowledgeable. Importantly, however, each could have a paradoxical effect of both increasing and decreasing receptivity. Less prevalent but nonetheless important themes suggested mechanisms through which cognitive reasoning processes might impede growth from formative feedback. Many studies have examined the effectiveness of feedback through variable interventions focused on feedback delivery. This study suggests that it is equally important to consider feedback from the perspective of how it is received. The interplay observed between fear, confidence, and reasoning processes reinforces the notion that there is no simple recipe for the delivery of effective feedback. These factors should be taken into account when trying to understand (a) why self-appraisal can be flawed, (b) why appropriate external feedback is vital (yet can be ineffective), and (c) why we may need to disentangle the goals of performance improvement from the goals of improving self-assessment.

The processes and dimensions of informed self-assessment: a conceptual model.

Purpose To determine how learners and physicians engaged in various structured interventions to inform self-assessment, how they perceived and used self-assessment in clinical learning and practice, and the components and processes comprising informed self-assessment and factors that influence these. Method This was a qualitative study guided by principles of grounded theory. Using purposive sampling, eight programs were selected in Canada, the United States, the United Kingdom, the Netherlands, and Belgium, representing low, medium, and high degrees of structure/rigor in self-assessment activities. In 2008, 17 focus groups were conducted with 134 participants (53 undergraduate learners, 32 postgraduate learners, 49 physicians). Focus-group transcripts were analyzed interactively and iteratively by the research team to identify themes and compare and confirm findings. Results Informed self-assessment appeared as a flexible, dynamic process of accessing, interpreting, and responding to varied external and internal data. It was characterized by multiple tensions arising from complex interactions among competing internal and external data and multiple influencing conditions. The complex process was evident across the continuum of medical education and practice. A conceptual model of informed self-assessment emerged. Conclusions Central challenges to informing self-assessment are the dynamic interrelationships and underlying tensions among the components comprising self-assessment. Realizing this increases understanding of why self-assessment accuracy seems frequently unreliable. Findings suggest the need for attention to the varied influencing conditions and inherent tensions to progress in understanding self-assessment, how it is informed, and its role in self-directed learning and professional self-regulation. Informed self-assessment is a multidimensional, complex construct requiring further research.

The ability of the multiple mini-interview to predict preclerkship performance in medical school.

Problem Statement and Background. One of the greatest challenges continuing to face medical educators is the development of an admissions protocol that provides valid information pertaining to the noncognitive qualities candidates possess. An innovative protocol, the Multiple Mini-Interview, has recently been shown to be feasible, acceptable, and reliable. This article presents a first assessment of the techniques validity. Method. Forty five candidates to the Undergraduate MD program at McMaster University participated in an MMI in Spring 2002 and enrolled in the program the following autumn. Performance on this tool and on the traditional protocol was compared to performance on preclerkship evaluation exercises. Results. The MMI was the best predictor of objective structured clinical examination performance and grade point average was the most consistent predictor of performance on multiple-choice question examinations of medical knowledge. Conclusions. While further validity testing is required, the MMI appears better able to predict preclerkship performance relative to traditional tools designed to assess the noncognitive qualities of applicants.

Multiple mini‐interviews predict clerkship and licensing examination performance

Objective  The Multiple Mini‐Interview (MMI) has previously been shown to have a positive correlation with early medical school performance. Data have matured to allow comparison with clerkship evaluations and national licensing examinations.

The Aging Physician: Changes in Cognitive Processing and Their Impact on Medical Practice

According to Health Canada, the proportion of Canadians aged 65 and over rose from 4.8% in 1921 to 12.3% in 1998. This cohort is projected to grow further before reaching a plateau at more than 20% of the population by the year 2026. This trend is not unique; similar predictions have been made for both industrialized and nonindustrialized countries worldwide. Not surprisingly, a great deal of effort has been expended attempting to predict the impact of this demographic drift on the provision of health care. Less examined, however, has been the effect of this trend on health care providers themselves. Already in Ontario, Canada, however, one in four family practitioners and one in three specialists are over 55 years of age. While the rate of physicians’ retirements has been increasing, these older physicians are going to become an ever-more-valuable resource as the ratio of physicians to population declines. Most of the medical education research that has used age as an independent variable has been performed in the context of physician-review programs. This literature suggests that aging induces cognitive changes in the way that diagnosticians approach clinical cases. There are discrepant findings, however, in terms of whether clinical performance improves or declines with aging. In this paper we systematically examine the small amount of evidence available in medical education that highlights the issue of aging and attempt to reconcile contradictory findings by drawing on the much larger psychological literature on pre-senile aging. Finally, we identify some of the specific implications for continuing education.

On the generality of specificity

Personality, clinical expertise, student evaluation, personal interviews, problem solving, professionalism, critical thinking and human factors engineering: central to all of these seemingly diverse areas of basic research and application is a widespread influence of context specificity. Context specificity is commonly defined by the observation that an individual’s performance on a particular problem or in a particular situation is only weakly predictive of the same individual’s performance on a different problem or in a different situation. For example, century-old studies have shown that training in formal logic, Latin and mathematics have no measurable influence on other cognitive domains. More recently, Norman et al. asked 30 clinicians of varying levels of expertise to work through a series of 10 clinical problems in cardiology and rheumatology. Within these 10 problems, 2 from each specialty area were exactly the same, with the exception of the actor portraying the patient in the simulated encounter. Despite the ‘content’ being exactly the same in these 2 problems, the change in context induced by a different actor was enough to result in an average correlation between performance scores of only 0Æ28.