Dorene F. Balmer

Innovation in pediatric residency education: the role of evaluation.

Evaluation is central to the development and implementation of successful educational activities in all aspects of physician training. Evaluation is also central to scholarship in the area of educational leadership and administration. The purpose of this short report is to explain the “what,” “why,” and “how” of evaluation for prospective participants in the Innovation in Pediatric Education (IIPE) program and for all readers who need to build evaluation strategies into their education research. Evaluation is the systematic appraisal of the merit or worth of an activity or an object or, in the case of IIPE, the merit or worth of a project.1 Although merit and worth are often used interchangeably, there are important differences.2 Merit assesses inherent, context-free value; it answers the question, “Does the project do what it is intended to do?” In contrast, worth assesses practical, real-life value and answers the question, “Is the project achieving its intended benefits?” IIPE aims to facilitate cutting-edge projects in pediatric education. Similar to other supporting agencies, it asks for, and scrutinizes, evaluation plans to identify projects with the greatest likelihood of success and greatest potential for dissemination to other residency programs. Evaluation is critical for assessing the quality of the project, both in terms of process (how things are done) and outcome (what actually happens). A needs assessment is often the first step in an evaluation plan. It determines who would most benefit from the intervention or program, the extent of the need, and what might work to meet the need. Formative evaluation assesses process. It informs the project as it … Address correspondence to Dorene F. Balmer, PhD, Center for Education Research and Evaluation, 701 W 168th St, Hammer Health Science Center, Lobby 10-A, New York, NY 10032. E-mail: db2595{at}columbia.edu

Sounding narrative medicine: studying students' professional identity development at Columbia University College of Physicians and Surgeons.

Purpose To learn what medical students derive from training in humanities, social sciences, and the arts in a narrative medicine curriculum and to explore narrative medicine’s framework as it relates to students’ professional development. Method On completion of required intensive, half-semester narrative medicine seminars in 2010, 130 second-year medical students at Columbia University College of Physicians and Surgeons participated in focus group discussions of their experiences. Focus group transcriptions were submitted to close iterative reading by a team who performed a grounded-theory-guided content analysis, generating a list of codes into which statements were sorted to develop overarching themes. Provisional interpretations emerged from the close and repeated readings, suggesting a fresh conceptual understanding of how and through what avenues such education achieves its goals in clinical training. Results Students’ comments articulated the known features of narrative medicine—attention, representation, and affiliation—and endorsed all three as being valuable to professional identity development. They spoke of the salience of their work in narrative medicine to medicine and medical education and its dividends of critical thinking, reflection, and pleasure. Critiques constituted a small percentage of the statements in each category. Conclusions Students report that narrative medicine seminars support complex interior, interpersonal, perceptual, and expressive capacities. Students’ lived experiences confirm some expectations of narrative medicine curricular planners while exposing fresh effects of such work to view.

Understanding paediatric resident−continuity preceptor relationships through the lens of apprenticeship learning

Context  Apprenticeship learning is common in medical education, but is often situated in theoretical frameworks which highlight its cognitive but not its social dimension.

Clerkship-based reflective writing: a rubric for feedback.

Context and setting The next generation of doctors must be educated in certain areas that previously may not have been considered relevant. The institution of a new curriculum at our university aims to keep pace with ever-evolving societal and scientific developments that have and will continue to change the way that medicine is practised. Why the idea was necessary It had become apparent that the use of outdated educational methodologies at our university was no longer appropriate. The goal of the new curriculum is to cultivate a wellrounded doctor who is well versed in ethics, culturally competent, skilled in critical thinking and knowledgeable of the human facets of medicine. Curriculum changes included smaller classes, the introduction of problem-based learning, earlier integration of clinical experience, and a wide range of elective courses, including those focusing on the social sciences and the humanities. What was done One such course, available to students in Years 1 and 2, is entitled ‘Psychiatry and Western Literature’. The elective course aims to outline major current issues in psychiatry, as well as some of the more common psychiatric disorders. It also aspires to show the human features of these disorders and to serve as an introduction to Western culture. One or more psychiatric topic(s) is paired with a work of literature and explored through observation, personal reflection and group discussion. For example, Don Quixote’s examination of realism versus idealism lends itself to the discussion of hallucinatory and paranoid disorders. The Metamorphosis is used to teach students about the burden on the family and the loss of social identity that come with illness. Anna Karenina allows for in-depth discussion of depression and suicide, and the cognitive mechanics behind them. Other works include Jane Eyre, Ward No. 6, The Plague, The Sorrows of Young Werther, The Catcher in the Rye, Madame Bovary, David Copperfield, Oliver Twist, Crime and Punishment, The Idiot and Scarlet and Black. The course’s main lecturer is from the psychiatry department. Guest lecturers have included artists, novelists, psychiatrists and doctors, who teach psychiatry and art as it applies to their field. For example, a professional musician teaches how a mentally ill opera character uses the melody in his or her aria to portray certain emotions. A novelist discusses how his own depression has become a part of his work. Evaluation of results and impact Psychiatry and Western Literature has become one of the most popular courses at our university. Each year since its inception, the number of students enrolled has continued to climb. The course has won awards from both the Ministry of Education and the Faculty of Medicine. Students have appreciated the help that literature has given them in probing the minds of people who are mentally ill. In feedback given to faculty staff, students suggested that it was the unique ability to perceive a sick person’s inner thoughts and motivations that made the course and its content so memorable. Students also reported increased interest in psychiatry as a possible career choice.

Sustaining an advisory dean program through continuous improvement and evaluation.

Purpose In 2003, the advisory dean program at Columbia University College of Physicians and Surgeons was created to better connect students and faculty by supporting student academic progress and improving career advising. With the program in its eighth year, the authors were interested in identifying key factors in maintaining ongoing vitality and effectiveness. Method In 2011, the authors conducted a reflective analysis to study the program, using available information from dean interviews, student surveys, meeting agendas, and program leader reflections, aided by the Bolman and Deal four-part framework for organizational functioning (structural, human resource, political, and symbolic). Results Structural factors included reframing program goals to match program activities, situating the program within broader academic advising and counseling resources, and increasing face time between entering students and their deans. Human resource factors included managing higher-than-expected turnover of deans with dean selection and orientation strategies that balance diversity and consistency, and providing ongoing training to promote continual professional growth. Political factors included balancing resources (e.g., money, administrative support) from the school and departments to help the deans protect and manage their time. Symbolic factors were related to leveraging the deans as symbols of institutional values and commitment to education (e.g., participation in the white coat ceremony), and being aware of “hidden meanings” associated with decisions within other frames (e.g., student-to-dean ratio). Conclusions A variety of strategies across frames were used to maintain the program. This report can serve as a guide to program maintenance for other institutions.

A Qualitative Study Exploring Moral Distress Among Pediatric Resuscitation Team Clinicians: Challenges to Professional Integrity*

Objective: Our study objectives were to explore moral distress among pediatric team clinicians within the context of resuscitation experiences, and determine whether there were any distinctively ethical perspectives on moral distress that could be conceptualized as challenges to professional integrity, rather than to previously described psychological responses of clinicians. Design: Descriptive, exploratory qualitative study. Setting: A large tertiary pediatric academic hospital in Houston, TX. Subjects: Twenty-five PICU resuscitation team clinicians were interviewed from December 2012 to April 2013. Interventions: None. Measurements and Main Results: All clinicians reported experiencing moral distress during certain resuscitations. Twenty-one of 25 clinicians reflected and acknowledged that their sense of professional integrity had been challenged during those resuscitation events. Four main components of resuscitation experience that induced moral distress were identified: 1) experiences where there was lack of understanding of the big picture; 2) experiences where there was suboptimal team leadership; 3) experiences where there was variable meanings to the word “resuscitation”; and 4) experiences were there was uncertainty of role responsibility. Conclusions: The perception of moral distress exists among pediatric clinicians during resuscitations and could be conceptualized as challenges to professional integrity. This ethical framework offers an alternative approach to understanding and investigating the complex layers of moral distress.

How do medical students navigate the interplay of explicit curricula, implicit curricula, and extracurricula to learn curricular objectives?

Purpose Current focus in medical education on competencies and curricular objectives draws attention to boundaries rather than the openness inherent in the learning process. This qualitative study explored the tension between boundedness (mandated curricular objectives) and openness (variability in learning experience as students traverse the explicit, implicit, and extracurriculum) in the curriculum. Method Following the revision and implementation of 10 curricular objectives for Columbia University College of Physicians and Surgeons, the authors interviewed 18 fourth-year medical students in spring 2011. For each objective, students indicated the relative influence of the explicit curriculum, implicit curriculum, and extracurriculum on their learning. Students were asked to think aloud and assign points as they made these judgments. Quantitative and qualitative data were analyzed to understand students’ perceptions of learning across curricula and for each curricular objective. Results There was marked variability in students’ learning experience. For two objectives, students perceived that learning occurred mainly in the explicit curriculum and consumed a disproportionate amount of study time. For two other objectives, students perceived that learning occurred mainly in the extracurriculum because opportunities to learn these objectives in the implicit and explicit curricula were sparse. For six objectives, students perceived that learning occurred mostly in the implicit curriculum, often through “watching” or interacting with peers. Conclusions The findings can inform discussions about how to balance the boundedness of curricular mandates with the inherent openness of students’ learning experiences.

The Initiative for Innovation in Pediatric Education: A Snapshot of a Program Evaluation

* Abbreviations: GME — : graduate medical education IIPE — : Initiative for Innovation in Pediatric Education LOI — : letter of intent As the Residency Review and Redesign Project came to a close in 2009, the Initiative for Innovation in Pediatric Education (IIPE) was ushered in as the next phase of ongoing evaluation and transformation of pediatric residency education. The mission of the IIPE is to initiate, facilitate, and oversee innovative change in pediatric residency education through carefully monitored, outcome-directed experimentation.1 The IIPE pursues this mission by supporting residency programs in their pursuit of innovative educational projects via a structured process for project review, consultation, monitoring, and dissemination. At the start of the IIPE, leaders of the Residency Review and Redesign Project wrote, “We need to recognize that conducting the business of graduate medical education (GME) is a project that is never complete and requires acknowledgment of a certain degree of ambiguity and uncertainty.”2 Recognizing that GME is a complex system, the IIPE leadership adopted developmental evaluation as its approach to program evaluation because it embraces ambiguity and uncertainty and because it could inform program development. Specifically, developmental evaluation aims to facilitate systematic, data-informed reflection and decision-making as the program evolves, whereas traditional evaluation approaches generally aim to improve a relatively stable program (process evaluation) or judge the merit and worth of an established program (outcome … Address correspondence to Dorene Balmer, PhD, RD, 701 West 168th St, Hammer Health Science Center, Lobby 10-A, New York, NY 10032. E-mail: db2595{at}columbia.edu

Longitudinal qualitative research in medical education

A Qualitative Space highlights research approaches that push readers and scholars deeper into qualitative methods and methodologies. Contributors to A Qualitative Spacemay: advance new ideas about qualitative methodologies, methods, and/or techniques; debate current and historical trends in qualitative research; craft and share nuanced reflections on how data collection methods should be revised or modified; reflect on the epistemological bases of qualitative research; or argue that some qualitative practices should end. Share your thoughts on Twitter using the hashtag: #aqualspace.

Learning across the explicit, implicit, and extra-curricula: an exploratory study of the relative proportions of residents' perceived learning in clinical areas at three pediatric residency programs.

Purpose This exploratory multisite study investigated relative proportions of residents’ perceived learning across the explicit, implicit (typically called hidden or informal), and extra-curricula for six Clinical Learning Environment Review (CLER) focus areas—patient safety, health care quality, care transitions, supervision, fatigue management, and professionalism—using qualitative and numeric data. Method In April through June 2013, the authors recruited and interviewed third-year categorical pediatric residents from three sites. For each CLER focus area, the authors asked residents to think aloud while they assigned a total of 60 points to the explicit, implicit, and extra-curricula, according to where they perceived their learning occurred. All interviews were audio taped and transcribed verbatim. The authors coded qualitative data from interviews using the constant comparative method, scrutinized qualitative data for themes, and reviewed qualitative and numeric data. Results A total of 28/79 (35%) residents participated. Residents perceived learning to occur most often in the implicit curriculum for five of the six CLER focus areas; the one exception being health care quality, which predominantly took place in the explicit curriculum. In the implicit curriculum, role modeling and “learning by doing” were frequently reported modes of learning. The explicit curriculum was perceived as an important baseline for understanding clinical areas. Relatively less learning was perceived to occur in the extra-curriculum. Conclusions The authors believe that recognizing learning in “other-than-explicit” curricula could broaden the medical education community’s understanding of the purview of the medical education curriculum and help educators tap into underused educational opportunities for important clinical topics.