James R. Martindale

Predicting pass rates on the american board of internal medicine certifying examination

Our objective was to determine the ability of the internal medicine In-Training Examination (ITE) to predict pass or fail outcomes on the American Board of Internal Medicine (ABIM) certifying examination and to develop and externally validated predictive model and a simple equation that can be used by residency directors to provide probability feedback for their residency programs. We collected a study sample of 155 internal medicine residents from the three Virginia internal medicine programs and a validation sample of 64 internal medicine residents from a residency program outside Virginia. Scores from both samples were collected across three class cohorts. The Kolmogorov-Smirnov z test indicated no statistically significant difference between the distribution of scores for the two samples (z=1.284, p=.074). Results of the logistic model yielded a statistically significant prediction of ABIM pass or fail performance from ITE scores (Wald=35.49, SE=0.036, df=1, p<.005) and overall correct classifications for the study sample and validation sample at 79% and 75%, respectively. The ITE is a useful tool in assessing the likelihood of a resident’s passing or failing the ABIM certifying examination but is less predictive for residents who received ITE scores between 49 and 66.

Why are medical students ‘checking out’ of active learning in a new curriculum?

The University of Virginia School of Medicine recently transformed its pre‐clerkship medical education programme to emphasise student engagement and active learning in the classroom. As in other medical schools, many students are opting out of attending class and others are inattentive while in class. We sought to understand why, especially with a new student‐centred curriculum, so many students were still opting to learn on their own outside of class or to disengage from educational activities while in class.

The Efficacy of Standardized Patient Feedback in Clinical Teaching: A Mixed Methods Analysis

Abstract: Introduction. The purpose of the current study was to investigate the effects of oral feedback from standardized patients on medical students’ overall perceptions of an educational exercise. We chose a mixed-methods approach to better understand the following research questions: Does satisfaction with the standardized patient exercise differ among those students who receive oral feedback and those who do not? What is the quality of oral feedback provided by standardized patients? Procedures. In order to address the first question, a basic randomized design comparing treatment (or those receiving SP feedback) to control (those not receiving SP feedback) was conducted. To address the second question, students in the treatment group were surveyed about their impressions of the quality of the feedback provided to them by their SP. One hundred and thirty six first year medical students were divided into treatment and control groups and interviewed one standardized patient during a single 20-minute encounter. Standardized patients were trained to simulate one of two outpatient cases and provide feedback using standard training materials. Both treatment and control groups completed a rating scale and questionnaire regarding their satisfaction with the encounter and students in the treatment group responded to additional questions regarding the quality of the SP feedback. Results. A one-way multivariate analysis of variance (MANOVA) revealed significant differences among control and treatment groups on the seven combined dependent variables, Wilks’ Students reported that the quality of SP feedback was very strong and additional qualitative analysis revealed further evidence to support the efficacy of providing oral SP feedback in a formative pre-clinical educational activity.

Development and evaluation of an instrument for assessing brief behavioral change interventions

OBJECTIVE To develop an observational coding instrument for evaluating the fidelity and quality of brief behavioral change interventions based on the behavioral theories of the 5 As, Stages of Change and Motivational Interviewing. METHODS Content and face validity were assessed prior to an intervention where psychometric properties were evaluated with a prospective cohort of 116 medical students. Properties assessed included the inter-rater reliability of the instrument, internal consistency of the full scale and sub-scales and descriptive statistics of the instrument. Construct validity was assessed based on students scores. RESULTS Inter-rater reliability for the instrument was 0.82 (intraclass correlation). Internal consistency for the full scale was 0.70 (KR20). Internal consistencies for the sub-scales were as follows: MI intervention component (KR20=.7); stage-appropriate MI-based intervention (KR20=.55); MI spirit (KR20=.5); appropriate assessment (KR20=.45) and appropriate assisting (KR20=.56). CONCLUSIONS The instrument demonstrated good inter-rater reliability and moderate overall internal consistency when used to assess performing brief behavioral change interventions by medical students. PRACTICE IMPLICATIONS This practical instrument can be used with minimal training and demonstrates promising psychometric properties when evaluated with medical students counseling standardized patients. Further testing is required to evaluate its usefulness in clinical settings.

Using a Critical Incident Scenario With Virtual Humans to Assess Educational Needs of Nurses in a Postanesthesia Care Unit

Introduction: During critical incidents, teamwork failures can compromise patient safety. This study provides evidence that virtual humans can be used in simulated critical incidents to assess the learning needs of health professionals, and provide important information that can inform the development of continuing education programs in patient safety. We explored the effectiveness of information transfer during a devolving medical situation between postanesthesia care unit (PACU) nurses and a virtual attending physician. Methods: We designed a three‐stage scenario: tutorial, patient transfer, and critical incident. We developed 2 checklists to assess information transfer: Critical Patient Information and Interprofessional Communication Skills. All participants were videotaped; 2 raters reviewed all videos and assessed performance using the checklists. Results: Participants (n = 43) who completed all 3 stages scored 62.3% correct on critical patient information transfer and 61.6% correct on interprofessional communication skills. Almost 87% missed a fatal drug error. The checklists measured each item on a 1/0 (done/not) calculation. Additionally, no relationship was found between years of nursing experience and performance on either checklist. Discussion: The PACU nurses in this study did not consistently share critical information with an attending (virtual) physician during a critical incident, and most missed a fatal dosage error. These findings strongly suggest a crucial need for additional structured team training among practicing health care teams, and they demonstrate the utility of using virtual humans to simulate team members.

Consistency in Drug Use Self-Reported by Incarcerated Adolescents to Correctional Staff vs. Research Interviewers

Abstract Incarcerated adolescents are typically evaluated for their substance abuse treatment needs, often based on self-report information if their offense is not related to abuse. This study tests consistency of self-reports of alcohol and drug use by comparing responses youth give to corrections staff and to university research interviewers only weeks apart. Significantly more incarcerated adolescents (n= 894) report lifetime drug and alcohol use to researchers than to correctional staff. Sex and ethnic differences were found. Overall, a correlation of .57 is found between prevalence of alcohol and drug use as measured by the correctional staff and that measured by the researchers.

Active learning, the accreditation process and 20-something students.

Editor – As the authors of ‘Why are medical students ‘checking out’ of active learning in a new curriculum?’, we thank Liz Mossop for her accompanying commentary ‘The curse of the teenage learner’. We appreciate her characterisation of our study as ‘frank and honest’ and agree that helping students to reflect on their learning styles and abilities is important for their professional development. This process is a formal component of our orientation activities for new students.