Nancy McNaughton

Osce checklists do not capture increasing levels of expertise

PURPOSE To evaluate the effectiveness of binary content checklists in measuring increasing levels of clinical competence. METHOD Fourteen clinical clerks, 14 family practice residents, and 14 family physicians participated in two 15-minute standardized patient interviews. An examiner rated each participants performance using a binary content checklist and a global process rating. The participants provided a diagnosis two minutes into and at the end of the interview. RESULTS On global scales, the experienced clinicians scored significantly better than did the residents and clerks, but on checklists, the experienced clinicians scored significantly worse than did the residents and clerks. Diagnostic accuracy increased for all groups between the two-minute and 15-minute marks without significant differences between the groups. CONCLUSION These findings are consistent with the hypothesis that binary checklists may not be valid measures of increasing clinical competence.

Using focus groups in medical education research: AMEE Guide No. 91

Abstract Qualitative research methodology has become an established part of the medical education research field. A very popular data-collection technique used in qualitative research is the “focus group”. Focus groups in this Guide are defined as “… group discussions organized to explore a specific set of issues … The group is focused in the sense that it involves some kind of collective activity … crucially, focus groups are distinguished from the broader category of group interview by the explicit use of the group interaction as research data” (Kitzinger 1994, p. 103). This Guide has been designed to provide people who are interested in using focus groups with the information and tools to organize, conduct, analyze and publish sound focus group research within a broader understanding of the background and theoretical grounding of the focus group method. The Guide is organized as follows: Firstly, to describe the evolution of the focus group in the social sciences research domain. Secondly, to describe the paradigmatic fit of focus groups within qualitative research approaches in the field of medical education. After defining, the nature of focus groups and when, and when not, to use them, the Guide takes on a more practical approach, taking the reader through the various steps that need to be taken in conducting effective focus group research. Finally, the Guide finishes with practical hints towards writing up a focus group study for publication.

Quality of standardised patient research reports in the medical education literature: review and recommendations

Context  In order to assess or replicate the research findings of published reports, authors must provide adequate and transparent descriptions of their methods. We conducted 2 consecutive studies, the first to define reporting standards relating to the use of standardised patients (SPs) in research, and the second to evaluate the current literature according to these standards.

Effects of Portraying Psychologically and Emotionally Complex Standardized Patient Roles

Background: The use of standardized patients (SPs) to portray emotionally intense roles has stimulated inquiry into the effects such roles might have on the actors. Purpose: Our study endeavored to obtain a rich description of the consequences of highly affective psychiatric roles on SPs. We wanted to find out what conditions made these consequences worse or better and what countermeasures, if any, SPs had evolved to address the effects of case simulation. Methods: In a pilot phase, 16 SPs completed a survey exploring the extent to which they were affected by playing emotionally intense roles. Based on these surveys, questions were developed for subsequent focus groups examining these effects. Four focus groups of 9 SPs each (N = 36) were taped, transcribed, and coded by 2 independent raters. Results: In the pilot survey, 11 of 16 SPs (69%) described residual psychophysiological effects. In the focus groups, all SPs reported some effect of portraying emotional roles, sometimes lasting several days. Severa...

Discourse(s) of emotion within medical education: the ever-present absence

Context  Emotion in medical education rests between the idealised and the invisible, sitting uneasily at the intersection between objective fact and subjective values. Examining the different ways in which emotion is theorised within medical education is important for a number of reasons. Most significant is the possibility that ideas about emotion can inform a broader understanding of issues related to competency and professionalism.

The challenge of creating new OSCE measures to capture the characteristics of expertise.

Purpose  Although expert clinicians approach interviewing in a different manner than novices, OSCE measures have not traditionally been designed to take into account levels of expertise. Creating better OSCE measures requires an understanding of how the interviewing style of experts differs objectively from novices.

Medical education… meet Michel Foucault

There have been repeated calls for the greater use of conceptual frameworks and of theory in medical education. Although it is familiar to few medical educators, Michel Foucaults work is a helpful theoretical and methodological source.

Adolescent Standardized Patients: Method of Selection and Assessment of Benefits and Risks

Background: Our psychiatric Objective Structured Clinical Examination (OSCE) group wishes to develop adolescent psychiatry OSCE stations. The literature regarding adolescent standardized patient (SP) selection methods and simulation effects, however, offered limited assurance that such adolescents would not experience adverse simulation effects. Purpose: Evaluation of adolescent SP selection methods and simulation effects for low- and high-stress roles. Method: A two-component (employment-psychological) SP selection method was used. Carefully selected SPs were assigned across three conditions: low-stress medical role, high-stress psychosocial role, and wait list control. Qualitative and quantitative measures were used to assess simulation effects. Results: Our selection method excluded 21% (7% employment and 14% psychological) of SP applicants. For SP participants, beneficial effects included acquisition of job skills and satisfaction in making an important contribution to society. SP reactions of discomfort to roles were reported. Long-term adverse effects were not identified. Conclusions: A two-component adolescent SP selection method is recommended. Adolescent SP benefits outweigh risks.

Key challenges in simulated patient programs: An international comparative case study

BackgroundThe literature on simulated or standardized patient (SP) methodology is expanding. However, at the level of the program, there are several gaps in the literature. We seek to fill this gap through documenting experiences from four programs in Australia, Canada, Switzerland and the United Kingdom. We focused on challenges in SP methodology, faculty, organisational structure and quality assurance.MethodsWe used a multiple case study method with cross-case synthesis. Over eighteen months during a series of informal and formal interactions (focused meetings and conference presentations) we documented key characteristics of programs and drew on secondary document sources.ResultsAlthough programs shared challenges in SP methodology they also experienced differences. Key challenges common to programs included systematic quality assurance and the opportunity for research. There were differences in the terminology used to describe SPs, in their recruitment and training. Other differences reflected local conditions and demands in organisational structure, funding relationships with the host institution and national trends, especially in assessments.ConclusionThis international case study reveals similarities and differences in SP methodology. Programs were highly contextualised and have emerged in response to local, institutional, profession/discipline and national conditions. Broader trends in healthcare education have also influenced development. Each of the programs experienced challenges in the same themes but the nature of the challenges often varied widely.

What do Psychiatry Residents Think of an Objective Structured Clinical Examination

Although interest is increasing in the use of Objective Structured Clinical Examinations (OSCEs) for the assessment of psychiatry residents, there are very few reports of the experiences and attitudes of residents to this form of evaluation. Eighteen residents in psychiatry participated in an OSCE consisting of eight 12-minute standardized patient interviews. Of the 15 who completed a postexamination survey, 93% rated the scenarios as very realistic and 80% considered the scenarios reflective of clinical situations they had experienced. The residents were very confident that such an examination could discriminate between incompetent and competent medical students, but the residents were less certain of the OSCE’s power to do so between competent and incompetent residents. While residents rejected the idea of such an OSCE for board certification, many felt they would like the experience to be a formative part of their training.