Kimberly A. Gifford

Doctor coach: a deliberate practice approach to teaching and learning clinical skills.

PROBLEM The rapidly evolving medical education landscape requires restructuring the approach to teaching and learning across the continuum of medical education. The deliberate practice strategies used to coach learners in disciplines beyond medicine can also be used to train medical learners. However, these deliberate practice strategies are not explicitly taught in most medical schools or residencies. APPROACH The authors designed the Doctor Coach framework and competencies in 2007-2008 to serve as the foundation for new faculty development and resident-as-teacher programs. In addition to teaching deliberate practice strategies, the programs model a deliberate practice approach that promotes the continuous integration of newly developed coaching competencies by participants into their daily teaching practice. OUTCOMES Early evaluation demonstrated the feasibility and efficacy of implementing the Doctor Coach framework across the continuum of medical education. Additionally, the Doctor Coach framework has been disseminated through national workshops, which have resulted in additional institutions applying the framework and competencies to develop their own coaching programs. NEXT STEPS Design of a multisource evaluation tool based on the coaching competencies will enable more rigorous study of the Doctor Coach framework and training programs and provide a richer feedback mechanism for participants. The framework will also facilitate the faculty development needed to implement the milestones and entrustable professional activities in medical education.

In pursuit of meaningful use of learning goals in residency: A qualitative study of pediatric residents

Purpose Medical education aims to equip physicians for lifelong learning, an objective supported by the conceptual framework of self-regulated learning (SRL). Learning goals have been used to develop SRL skills in learners across the medical education continuum. This study’s purpose was to elicit residents’ perspectives on learning goal use and to develop explanations suggesting how aspects of the learning environment may facilitate or hinder the meaningful use of learning goals in residency. Method Resident focus groups and program director interviews were conducted in 2012–2013, audio-recorded, and transcribed. Programs were selected to maximize diversity of size, geographic location, type of program, and current use of learning goals. Data were analyzed using the constant comparative method associated with grounded theory. Further analysis compared themes frequently occurring together to strengthen the understanding of relationships between the themes. Through iterative discussions, investigators built a grounded theory. Results Ninety-five third-year residents and 12 program directors at 12 pediatric residency programs participated. The analysis identified 21 subthemes grouped into 5 themes: program support, faculty roles, goal characteristics and purposes, resident attributes, and accountability and goal follow-through. Review of relationships between the themes revealed a pyramid of support with program support as the foundation that facilitates the layers above it, leading to goal follow-through. Conclusions Program support facilitates each step of the SRL process that leads to meaningful use of learning goals in residency. A strong foundation of program support should include attention to aspects of the implicit curriculum as well as the explicit curriculum.

Practical Suggestions for the Creation and Use of Meaningful Learning Goals in Graduate Medical Education.

From the Department of Pediatrics, Nationwide Children’s Hospital, Ohio State University School of Medicine, Columbus, Ohio (Drs Reed and Mahan); Department of Pediatrics, University of Colorado School of Medicine, Aurora, Colo (Drs Lockspeiser, Hanson, and Rosenberg); Department of Pediatrics, Wright State University Boonshoft School of Medicine, Dayton Children’s Hospital, Dayton, Ohio (Dr Burke); Department of Pediatrics, Children’s Hospital at Dartmouth, Geisel School of Medicine at Dartmouth, Hanover, NH (Dr Gifford); Department of Pediatrics, Riley Hospital for Children and IU School of Medicine, Indianapolis, Ind (Dr McKenna); and Department of Pediatrics, University of California Davis School of Medicine, Sacramento, Calif (Dr Li) The authors declare that they have no conflict of interest. Address correspondence to Suzanne Reed, MD, Division of Hematology/Oncology/BMT, Nationwide Children’s Hospital, 700 Children’s Dr, Columbus, OH 43205 (e-mail: suzanne.reed@nationwidechildrens.org).

Competent for Unsupervised Practice: Use of Pediatric Residency Training Milestones to Assess Readiness.

Purpose To describe clinical skills progression during pediatric residency using the distribution of pediatric milestone assessments by subcompetency and year of training and to determine reasonable milestone expectations at time of graduation. Method Multi-institutional cohort study of the milestones reported to the Accreditation Council for Graduate Medical Education for all 21 pediatric subcompetencies. Most subcompetencies were measured using five milestone levels (1 = novice, 2 = advanced beginner, 3 = competent, 4 = proficient, 5 = master); 3 subcompetencies had only four levels defined. Results Milestone assessments for 2,030 pediatric residents in 47 programs during academic year 2013–2014 were obtained. There was significant variation in end-of-year milestone ratings for residents within each level of training, which decreased as training level increased. Most (78.9%; 434/550) graduating third-year pediatric residents received a milestone rating of ≥ 3 in all 21 subcompetencies; fewer (21.1%; 116/550) received a rating of ≥ 4 in all subcompetencies. Across all training levels, professionalism and interpersonal communication skills were rated highest; quality improvement was rated lowest. Conclusions Trainees entered residency with a wide range of skills. As they advanced, skill variability within a training level decreased. Most graduating pediatric residents were still advancing on the milestone continuum toward proficiency and mastery, and an expectation of milestone ratings ≥ 4 in all categories upon graduation is unrealistic; milestone ratings ≥ 3 upon graduation may be more realistic. Understanding current pediatric residents’ and graduates’ skills can help to identify key areas that should be specifically targeted during training.

A novel workplace-based assessment for competency-based decisions and learner feedback

Abstract Background: Increased recognition of the importance of competency-based education and assessment has led to the need for practical and reliable methods to assess relevant skills in the workplace. Methods: A novel milestone-based workplace assessment system was implemented in 15 pediatrics residency programs. The system provided: (1) web-based multisource feedback (MSF) and structured clinical observation (SCO) instruments that could be completed on any computer or mobile device; and (2) monthly feedback reports that included competency-level scores and recommendations for improvement. Results: For the final instruments, an average of five MSF and 3.7 SCO assessment instruments were completed for each of 292 interns; instruments required an average of 4–8 min to complete. Generalizability coefficients >0.80 were attainable with six MSF observations. Users indicated that the new system added value to their existing assessment program; the need to complete the local assessments in addition to the new assessments was identified as a burden of the overall process. Conclusions: Outcomes – including high participation rates and high reliability compared to what has traditionally been found with workplace-based assessment – provide evidence for the validity of scores resulting from this novel competency-based assessment system. The development of this assessment model is generalizable to other specialties.

Identifying Gaps in the Performance of Pediatric Trainees Who Receive Marginal/Unsatisfactory Ratings.

Purpose To perform a derivation study to determine in which subcompetencies marginal/unsatisfactory pediatric residents had the greatest deficits compared with their satisfactorily performing peers and which subcompetencies best discriminated between marginal/unsatisfactory and satisfactorily performing residents. Method Multi-institutional cohort study of all 21 milestones (rated on four or five levels) reported to the Accreditation Council for Graduate Medical Education, and global marginal/unsatisfactory versus satisfactory performance reported to the American Board of Pediatrics. Data were gathered in 2013–2014. For each level of training (postgraduate year [PGY] 1, 2, and 3), mean differences between milestone levels of residents with marginal/unsatisfactory and satisfactory performance adjusted for clustering by program and C-statistics (area under receiver operating characteristic curve) were calculated. A Bonferroni-corrected significance threshold of .0007963 was used to account for multiple comparisons. Results Milestone and overall performance evaluations for 1,704 pediatric residents in 41 programs were obtained. For PGY1s, two subcompetencies had almost a one-point difference in milestone levels between marginal/unsatisfactory and satisfactory trainees and outstanding discrimination (≥ 0.90): organize/prioritize (0.93; C-statistic: 0.91) and transfer of care (0.97; C-statistic: 0.90). The largest difference between marginal/unsatisfactory and satisfactory PGY2s was trustworthiness (0.78). The largest differences between marginal/unsatisfactory and satisfactory PGY3s were ethical behavior (1.17), incorporating feedback (1.03), and professionalization (0.96). For PGY2s and PGY3s, no subcompetencies had outstanding discrimination. Conclusions Marginal/unsatisfactory pediatric residents had different subcompetency gaps at different training levels. While PGY1s may have global deficits, senior residents may have different performance deficiencies requiring individualized counseling and targeted performance improvement plans.