Renée E. Stalmeijer

Using focus groups in medical education research: AMEE Guide No. 91

Abstract Qualitative research methodology has become an established part of the medical education research field. A very popular data-collection technique used in qualitative research is the “focus group”. Focus groups in this Guide are defined as “… group discussions organized to explore a specific set of issues … The group is focused in the sense that it involves some kind of collective activity … crucially, focus groups are distinguished from the broader category of group interview by the explicit use of the group interaction as research data” (Kitzinger 1994, p. 103). This Guide has been designed to provide people who are interested in using focus groups with the information and tools to organize, conduct, analyze and publish sound focus group research within a broader understanding of the background and theoretical grounding of the focus group method. The Guide is organized as follows: Firstly, to describe the evolution of the focus group in the social sciences research domain. Secondly, to describe the paradigmatic fit of focus groups within qualitative research approaches in the field of medical education. After defining, the nature of focus groups and when, and when not, to use them, the Guide takes on a more practical approach, taking the reader through the various steps that need to be taken in conducting effective focus group research. Finally, the Guide finishes with practical hints towards writing up a focus group study for publication.

The development of an instrument for evaluating clinical teachers: involving stakeholders to determine content validity

Background: Research indicates that the quality of supervision strongly influences the learning of medical students in clinical practice. Clinical teachers need feedback to improve their supervisory skills. The available instruments either lack a clear theoretical framework or are not suitable for providing feedback to individual teachers. We developed an evaluation instrument based on the ‘cognitive apprenticeship model’. Aim: The aim was to estimate the content validity of the developed instrument. Method: Item relevance was rated on a five-point scale (1 = highly irrelevant, 5 = highly relevant) by three groups of stakeholders in undergraduate clinical teaching: educationalists (N = 12), doctors (N = 16) and students (N = 12). Additionally, stakeholders commented on content, wording and omission of items. Results: The items were generally rated as very relevant (Mean = 4.3, SD = 0.38, response = 95%) and any differences between the stakeholder groups were small. The results led to elimination of 4 items, rewording of 13 items and addition of 1 item. Discussion: The cognitive apprenticeship model appears to offer a useful framework for the development of an evaluation instrument aimed at providing feedback to individual clinical teachers on the quality of student supervision. Further studies in larger populations will have to establish the instruments statistical validity and generalizability.

Training Physicians to Provide High-Value, Cost-Conscious Care: A Systematic Review

IMPORTANCE Increasing health care expenditures are taxing the sustainability of the health care system. Physicians should be prepared to deliver high-value, cost-conscious care. OBJECTIVE To understand the circumstances in which the delivery of high-value, cost-conscious care is learned, with a goal of informing development of effective educational interventions. DATA SOURCES PubMed, EMBASE, ERIC, and Cochrane databases were searched from inception until September 5, 2015, to identify learners and cost-related topics. STUDY SELECTION Studies were included on the basis of topic relevance, implementation of intervention, evaluation of intervention, educational components in intervention, and appropriate target group. There was no restriction on study design. DATA EXTRACTION AND SYNTHESIS Data extraction was guided by a merged and modified version of a Best Evidence in Medical Education abstraction form and a Cochrane data coding sheet. Articles were analyzed using the realist review method, a narrative review technique that focuses on understanding the underlying mechanisms in interventions. Recurrent patterns were identified in the data through thematic analyses. Resulting themes were discussed within the research team until consensus was reached. MAIN OUTCOMES AND MEASURES Main outcomes were factors that promote education in delivering high-value, cost-conscious care. FINDINGS The initial search identified 2650 articles; 79 met the inclusion criteria, of which 14 were randomized clinical trials. The majority of the studies were conducted in North America (78.5%) using a pre-post interventional design (58.2%; at least 1619 participants); they focused on practicing physicians (36.7%; at least 3448 participants), resident physicians (6.3%; n = 516), and medical students (15.2%; n = 275). Among the 14 randomized clinical trials, 12 addressed knowledge transmission, 7 reflective practice, and 1 supportive environment; 10 (71%) concluded that the intervention was effective. The data analysis suggested that 3 factors aid successful learning: (1) effective transmission of knowledge, related, for example, to general health economics and prices of health services, to scientific evidence regarding guidelines and the benefits and harms of health care, and to patient preferences and personal values (67 articles); (2) facilitation of reflective practice, such as providing feedback or asking reflective questions regarding decisions related to laboratory ordering or prescribing to give trainees insight into their past and current behavior (56 articles); and (3) creation of a supportive environment in which the organization of the health care system, the presence of role models of delivering high-value, cost-conscious care, and a culture of high-value, cost-conscious care reinforce the desired training goals (27 articles). CONCLUSIONS AND RELEVANCE Research on educating physicians to deliver high-value, cost-conscious care suggests that learning by practicing physicians, resident physicians, and medical students is promoted by combining specific knowledge transmission, reflective practice, and a supportive environment. These factors should be considered when educational interventions are being developed.

Expertise in performance assessment: assessors’ perspectives

The recent rise of interest among the medical education community in individual faculty making subjective judgments about medical trainee performance appears to be directly related to the introduction of notions of integrated competency-based education and assessment for learning. Although it is known that assessor expertise plays an important role in performance assessment, the roles played by different factors remain to be unraveled. We therefore conducted an exploratory study with the aim of building a preliminary model to gain a better understanding of assessor expertise. Using a grounded theory approach, we conducted seventeen semi-structured interviews with individual faculty members who differed in professional background and assessment experience. The interviews focused on participants’ perceptions of how they arrived at judgments about student performance. The analysis resulted in three categories and three recurring themes within these categories: the categories assessor characteristics, assessors’ perceptions of the assessment tasks, and the assessment context, and the themes perceived challenges, coping strategies, and personal development. Central to understanding the key processes in performance assessment appear to be the dynamic interrelatedness of the different factors and the developmental nature of the processes. The results are supported by literature from the field of expertise development and in line with findings from social cognition research. The conceptual framework has implications for faculty development and the design of programs of assessment.

Combined student ratings and self-assessment provide useful feedback for clinical teachers

Many evaluation instruments have been developed to provide feedback to physicians on their clinical teaching but written feedback alone is not always effective. We explored whether feedback effectiveness improved when teachers’ self-assessment was added to written feedback based on student ratings. 37 physicians (10 residents, 27 attending physicians) from different specialties (Internal Medicine, Surgery, Obstetrics/Gynecology, Pediatrics, Neurology, Dermatology, Ophthalmology, ENT, and Psychiatry) were invited to fill out a self-assessment questionnaire on their teaching skills. Students completed an almost identical questionnaire to evaluate the same teachers based on their experiences during clerkships. After receiving written feedback incorporating their self-assessment and the student ratings, the teachers indicated their perceptions of the self-assessment exercise and the written feedback in a questionnaire (five-point Likert scale items) and next, in more detail, in semi-structured interviews with a purposive sample of 12 of the participating teachers. 25 physicians participated (67%). The results showed that self-assessment and student feedback were both perceived as useful (3.7, SD 1.0) but the latter was considered more effective. The physicians we interviewed considered the combination of self-assessment with student ratings more effective than either self-assessment or written feedback alone. Notably, discrepancies between student ratings and self-assessment were deemed a strong incentive for change. We conclude that self-assessment can be a useful tool to stimulate improvement of clinical teaching when it is combined with written feedback based on student ratings. Future research among larger groups is needed to confirm our findings and examine whether these combined tools actually lead to improved teaching.

Clinical teaching based on principles of cognitive apprenticeship: views of experienced clinical teachers.

Purpose To explore (1) whether an instructional model based on principles of cognitive apprenticeship fits with the practice of experienced clinical teachers and (2) which factors influence clinical teaching during clerkships from an environmental, teacher, and student level as perceived by the clinical teachers themselves. The model was designed to apply directly to teaching behaviors of clinical teachers and consists of three phases, advocating teaching behaviors such as modeling, creating a safe learning environment, coaching, knowledge articulation, and exploration. Method A purposive sample of 17 experienced clinical teachers from five different disciplines and four different teaching hospitals took part in semistructured individual interviews. Two researchers independently performed a thematic analysis of the interview transcripts. Coding was discussed within the research team until consensus was reached. Results All participants recognized the theoretical model as a structured picture of the practice of teaching activities during both regular and senior clerkships. According to participants, modeling and creating a safe learning environment were fundamental to the learning process of both regular and senior clerkship students. Division of teaching responsibilities, longer rotations, and proactive behavior of teachers and students ensured that teachers were able to apply all steps in the model. Conclusions The theoretical model can offer valuable guidance in structuring clinical teaching activities and offers suggestions for the design of effective clerkships.

How interdisciplinary teams can create multi-disciplinary education: the interplay between team processes and educational quality

Context  Many undergraduate medical education programmes offer integrated multi‐disciplinary courses, which are generally developed by a team of teachers from different disciplines. Research has shown that multi‐disciplinary teams may encounter problems, which can be detrimental to productive co‐operation, which in turn may diminish educational quality. Because we expected that charting these problems might yield suggestions for addressing them, we examined the relationships between team diversity, team processes and course quality.

Conceptions of how a learning or teaching curriculum, workplace culture and agency of individuals shape medical student learning and supervisory practices in the clinical workplace

Abstract The role of workplace supervisors in the clinical education of medical students is currently under debate. However, few studies have addressed how supervisors conceptualize workplace learning and how conceptions relate to current sociocultural workplace learning theory. We explored physician conceptions of: (a) medical student learning in the clinical workplace and (b) how they contribute to student learning. The methodology included a combination of a qualitative, inductive (conventional) and deductive (directed) content analysis approach. The study triangulated two types of interview data from 4 focus group interviews and 34 individual interviews. A total of 55 physicians participated. Three overarching themes emerged from the data: learning as membership, learning as partnership and learning as ownership. The themes described how physician conceptions of learning and supervision were guided by the notions of learning-as-participation and learning-as-acquisition. The clinical workplace was either conceptualized as a context in which student learning is based on a learning curriculum, continuity of participation and partnerships with supervisors, or as a temporary source of knowledge within a teaching curriculum. The process of learning was shaped through the reciprocity between different factors in the workplace context and the agency of students and supervising physicians. A systems-thinking approach merged with the “co-participation” conceptual framework advocated by Billet proved to be useful for analyzing variations in conceptions. The findings suggest that mapping workplace supervisor conceptions of learning can be a valuable starting point for medical schools and educational developers working with changes in clinical educational and faculty development practices.

You've got to know the rules to play the game: how medical students negotiate the hidden curriculum of surgical careers

The hidden curriculum may be framed as the culture, beliefs and behaviours of a community that are passed to students outside formal course offerings. Medical careers involve diverse specialties, each with a different culture, yet how medical students negotiate these cultures has not been fully explored. Using surgery as a case study, we aimed to establish, first, whether a specialty‐specific hidden curriculum existed for students, and second, how students encountered and negotiated surgical career options.

AM Last Page: A Guide to Research Paradigms Relevant to Medical Education

Suggestions for further reading: • Bunniss S, Kelly DR. Research paradigms in medical education research. Med Educ. 2010;44:358-366. • Carter SM, Little M. Justifying knowledge, justifying method, taking action: Epistemologies, methodologies and methods in qualitative research. Qual Health Res. 2007;17:1316-1328. • Illing J. Thinking About Research: Frameworks, Ethics and Scholarship. Edinburgh, UK: Association for the Study of Medical Education; 2007. Research Design Examples include experimental design, ethnography, case study, action research, grounded theory, phenomenology, and discourse analysis. Methods Techniques for data collection include randomized controlled trial, questionnaires, interviews, focus groups, observation, and document analysis. Paradigm