Win May

A ten-year review of the literature on the use of standardized patients in teaching and learning: 1996-2005

Background: Although there is a growing body of literature on the educational use of standardized patients (SP) in teaching and learning, there have been no reviews on their value. Objective: To determine whether the educational use of SPs has an effect on the knowledge, skills, and behaviour of learners in the health professions. Methods: English-language articles covering the period 1996–2005 were reviewed to address the issue of to what extent has the use of SPs affected the knowledge, skills and performance of learners. Out of 797 abstracts, 69 articles, which met the review criteria, were selected. An adaptation of Kirkpatricks model was used to classify and analyse the articles. Results: Most of the learners were students in medicine and nursing. SPs were used mostly to teach communication skills and clinical skills. The study designs were case-control (29%), pre-test/post-test (24.6%), post-test only (26.1%) and qualitative studies (20.3%). Methodological issues: Most of the studies had weak research designs. More rigorous designs with control or comparison groups should be used in future research. Conclusions: Most studies reported that the educational use of SPs was valuable. More rigorous studies would support the evidence-based use of SPs in teaching and learning.

Effectiveness of a clinically integrated e-learning course in evidence-based medicine for reproductive health training: a randomized trial.

CONTEXT For evidence-based practice to embed culturally in the workplace, teaching of evidence-based medicine (EBM) should be clinically integrated. In low-middle-income countries (LMICs) there is a scarcity of EBM-trained clinical tutors, lack of protected time for teaching EBM, and poor access to relevant databases in languages other than English. OBJECTIVE To evaluate the effects of a clinically integrated e-learning EBM course incorporating the World Health Organization (WHO) Reproductive Health Library (RHL) on knowledge, skills, and educational environment compared with traditional EBM teaching. DESIGN, SETTING, AND PARTICIPANTS International cluster randomized trial conducted between April 2009 and November 2010 among postgraduate trainees in obstetrics-gynecology in 7 LMICs (Argentina, Brazil, Democratic Republic of the Congo, India, Philippines, South Africa, Thailand). Each training unit was randomized to an experimental clinically integrated course consisting of e-modules using the RHL for learning activities and trainee assessments (31 clusters, 123 participants) or to a control self-directed EBM course incorporating the RHL (29 clusters, 81 participants). A facilitator with EBM teaching experience was available at all teaching units. Courses were administered for 8 weeks, with assessments at baseline and 4 weeks after course completion. The study was completed in 24 experimental clusters (98 participants) and 22 control clusters (68 participants). MAIN OUTCOME MEASURES Primary outcomes were change in EBM knowledge (score range, 0-62) and skills (score range, 0-14). Secondary outcome was educational environment (5-point Likert scale anchored between 1 [strongly agree] and 5 [strongly disagree]). RESULTS At baseline, the study groups were similar in age, year of training, and EBM-related attitudes and knowledge. After the trial, the experimental group had higher mean scores in knowledge (38.1 [95% CI, 36.7 to 39.4] in the control group vs 43.1 [95% CI, 42.0 to 44.1] in the experimental group; adjusted difference, 4.9 [95% CI, 2.9 to 6.8]; P < .001) and skills (8.3 [95% CI, 7.9 to 8.7] vs 9.1 [95% CI, 8.7 to 9.4]; adjusted difference, 0.7 [95% CI, 0.1 to 1.3]; P = .02). Although there was no difference in improvement for the overall score for educational environment (6.0 [95% CI, -0.1 to 12.0] vs 13.6 [95% CI, 8.0 to 19.2]; adjusted difference, 9.6 [95% CI, -6.8 to 26.1]; P = .25), there was an associated mean improvement in the domains of general relationships and support (-0.5 [95% CI, -1.5 to 0.4] vs 0.3 [95% CI, -0.6 to 1.1]; adjusted difference, 2.3 [95% CI, 0.2 to 4.3]; P = .03) and EBM application opportunities (0.5 [95% CI, -0.7 to 1.8] vs 2.9 [95%, CI, 1.8 to 4.1]; adjusted difference, 3.3 [95% CI, 0.1 to 6.5]; P = .04). CONCLUSION In a group of LMICs, a clinically integrated e-learning EBM curriculum in reproductive health compared with a self-directed EBM course resulted in higher knowledge and skill scores and improved educational environment. TRIAL REGISTRATION anzctr.org.au Identifier: ACTRN12609000198224.

Shaping professionalism in pre-clinical medical students: Professionalism and the practice of medicine

Background: Increasing emphasis is placed on teaching and assessment of professionalism in the continuum of medical education. Consistent and longitudinal instruction and assessment are crucial factors that learners need in order to internalize the tenets of professionalism. Aim: We aimed to develop a novel longitudinal course in professionalism spanning the first 2 years in a medical curriculum. Methods: This is a description of the process undertaken over the past 7 years to develop and implement a professionalism curriculum. We used the conceptual framework of constructivism, principles of adult learning, experiential learning and reflective practice to integrate learning with experience. We included student input in session development. Faculty mentors serve as role models to guide, assist and counsel students. Assessment of learners is accomplished using self, peer and mentor evaluation, and a student portfolio. Program evaluation is by course and faculty evaluation. Results: Students are given a final grade of pass or fail, together with a brief narrative. Course evaluations were positive. A survey questionnaire showed that more than 60% of the students reported gaining skills related to course goals. Conclusions: A longitudinal curriculum for the pre-clinical years was successfully launched. Plans are under way to expand this into the clinical years.

The relationship between medical students' learning approaches and performance on a summative high-stakes clinical performance examination

Background: A learning approach embeds the intention of the student when starting a task and the learning processes and strategies used to carry out a task. Student approaches to learning have been categorized as deep, strategic, and surface. Aim: To explore the relationships among medical students’ learning approaches, gender, and performance on a summative high-stakes clinical performance examination (CPX). Methods: We measured medical students’ learning approaches at the beginning of year four using the Approaches and Study Skills Inventory for Students and compared results with CPX scores. Results: Student scores in the top two quartiles of the CPX were significantly higher on the deep approach than student CPX scores in the bottom quartile, and student scores in the bottom quartile of the CPX were significantly higher on the surface approach than scores for the other three CPX quartiles. CPX patient–physician interaction scores showed a significant positive correlation with deep approach scores, and CPX overall patient satisfaction scores showed a significant positive correlation with deep and strategic approach scores. Surface approach scores correlated negatively with all CPX score categories. Conclusion: Approach to learning was associated with performance on a high-stakes CPX.

Assessing Patient-centered Care: One Approach to Health Disparities Education

BACKGROUNDPatient-centered care has been described as one approach to cultural competency education that could reduce racial and ethnic health disparities by preparing providers to deliver care that is respectful and responsive to the preferences of each patient. In order to evaluate the effectiveness of a curriculum in teaching patient-centered care (PCC) behaviors to medical students, we drew on the work of Kleinman, Eisenberg, and Good to develop a scale that could be embedded across cases in an objective structured clinical examination (OSCE).OBJECTIVETo compare the reliability, validity, and feasibility of an embedded patient-centered care scale with the use of a single culturally challenging case in measuring students′ use of PCC behaviors as part of a comprehensive OSCE.METHODSA total of 322 students from two California medical schools participated in the OSCE as beginning seniors. Cronbach’s alpha was used to assess the internal consistency of each approach. Construct validity was addressed by establishing convergent and divergent validity using the cultural challenge case total score and OSCE component scores. Feasibility assessment considered cost and training needs for the standardized patients (SPs).RESULTSMedical students demonstrated a moderate level of patient-centered skill (mean = 63%, SD = 11%). The PCC Scale demonstrated an acceptable level of internal consistency (alpha = 0.68) over the single case scale (alpha = 0.60). Both convergent and divergent validities were established through low to moderate correlation coefficients.DISCUSSIONThe insertion of PCC items across multiple cases in a comprehensive OSCE can provide a reliable estimate of students′ use of PCC behaviors without incurring extra costs associated with implementing a special cross-cultural OSCE. This approach is particularly feasible when an OSCE is already part of the standard assessment of clinical skills. Reliability may be increased with an additional investment in SP training.

A comparison of two standard-setting approaches in high-stakes clinical performance assessment using generalizability theory.

Purpose Scoring clinical assessments in a reliable and valid manner using criterion-referenced standards remains an important issue and directly affects decisions made regarding examinee proficiency. This generalizability study of students’ clinical performance examination (CPX) scores examines the reliability of those scores and of their interpretation, particularly according to a newly introduced, “critical actions” criterion-referenced standard and scoring approach. Method The authors applied a generalizability framework to the performance scores of 477 third-year students attending three different medical schools in 2008. The norm-referenced standard included all station checklist items. The criterion-referenced standard included only those items deemed critical to patient care by a faculty panel. The authors calculated and compared variance components and generalizability coefficients for each standard across six common stations. Results Norm-referenced scores had moderate generalizability (&rgr;2 = 0.51), whereas criterion-referenced scores showed low dependability (&phgr; = 0.20). The estimated 63% of measurement error associated with the person-by-station interaction suggests case specificity. Increasing the number of stations on the CPX from 6 to 24, an impractical solution both for cost and time, would still yield only moderate dependability (&phgr; = 0.50). Conclusions Though the performance assessment of complex skills, like clinical competence, seems intrinsically valid, careful consideration of the scoring standard and approach is needed to avoid misinterpretation of proficiency. Further study is needed to determine how best to improve the reliability of criterion-referenced scores, by implementing changes to the examination structure, the process of standard-setting, or both.

Effect of feedback from standardized patients on medical students' performance and perceptions of the neurological examination.

Background: Feedback can have a powerful influence on the performance of learners, and has traditionally been provided by faculty. Aim: This study set out to explore whether feedback from a standardized patient (SP) can improve students’ performance of the neurological examination. Methods: A randomized controlled design was used with final year medical students. The control group did not receive any feedback. The intervention groups received either written feedback or a combination of written and verbal feedback. A written test was given prior to the intervention to assess comparability of the three groups. Pretest and post-test scores on the neurological examination were compared. Attitudinal questionnaires were administered at the time of the posttest, and 6 months later. Results: Students receiving feedback from the SPs had significantly greater scores on the posttest than the control group. In the intervention groups, students receiving both verbal and written feedback scored significantly higher than those who received only written feedback. More positive perceptions of learning outcomes and the value of SP feedback were noted in the intervention groups. Conclusion: SP feedback was associated with a significant increase in student scores on the neurological examination, as well as more favorable perceptions of the experience.

Training Standardized Patients for a High-Stakes Clinical Performance Examination in the California Consortium for the Assessment of Clinical Competence

The use of standardized patients in teaching and assessment of clinical skills has become more ubiquitous in medical schools in the United States and Canada since Dr Howard Barrows introduced the first standardized patient at the University of Southern California in 1963. This increased usage is also due to the fact that the national licensing examination in the United States, includes a component to assess the clinical skills of the learners (United States Medical Licensure Examination Step 2 CS). The eight medical schools in California form a Consortium for the Assessment of Clinical Competence, which enables them to develop and implement a common clinical assessment tool, the Clinical Performance Examination (CPX), for final year medical students across the state. All medical schools in the Consortium share the same standardized patient cases and checklists. The standardization of training across the eight medical schools is presented. This paper describes the methods that have been used to train the SPs so that they can portray the gestalt of the patient, provide effective feedback, and reliably evaluate the students at the Keck School of Medicine of the University of Southern California. Quality assurance measures to ensure both performance and checklist accuracy are also described.

A comparison of medical students' learning approaches between the first and fourth years.

Objectives We hypothesized that medical students exposed to a case-based curriculum in years 1 and 2 and clinical cases in the year 3 clerkship would demonstrate a longitudinal increase in the deep approach to learning and a decrease in the surface apathetic approach. Methods A cohort of first-year medical students completed the Approaches and Study Skills Inventory for Students at the beginning of their first term and again at the beginning of their fourth year. Approaches and Study Skills Inventory for Students scores were aggregated into three main learning approach scales: deep, strategic, and surface apathetic. Results On average, deep and strategic scores did not significantly change between years 1 and 4, but the surface apathetic mean score decreased as a result of lower syllabus boundness and fear of failure subscale scores. Effect sizes were small (d = 0.30, 0.34, respectively). Conclusions The deep approach to learning is a complex process and did not change in our students after 3 years of medical school, even though a case-based curriculum was believed to foster deeper learning. By the end of year 3, our students were, on average, less bound to syllabi and feared failure less.

Assessing 3rd year medical students' interprofessional collaborative practice behaviors during a standardized patient encounter: A multi-institutional, cross-sectional study.

Abstract Background: To understand how third-year medical student interprofessional collaborative practice (IPCP) is affected by self-efficacy and interprofessional experiences (extracurricular experiences and formal curricula). Methods: The authors measured learner IPCP using an objective structured clinical examination (OSCE) with a standardized nurse (SN) and standardized patient (SP) during a statewide clinical performance examination. At four California medical schools from April to August 2012, SPs and SNs rated learner IPCP (10 items, range 0–100) and patient-centered communication (10 items, range 0–100). Post-OSCE, students reported their interprofessional self-efficacy (16 items, 2 factors, range 1–10) and prior extracurricular interprofessional experiences (3 items). School representatives shared their interprofessional curricula during guided interviews. Results: Four hundred sixty-four of 530 eligible medical students (88%) participated. Mean IPCP performance was 79.6 ± 14.1 and mean self-efficacy scores were 7.9 (interprofessional teamwork) and 7.1 (interprofessional feedback and evaluation). Seventy percent of students reported prior extracurricular interprofessional experiences; all schools offered formal interprofessional curricula. IPCP was associated with self-efficacy for interprofessional teamwork (β = 1.6, 95% CI [0.1, 3.1], p = 0.04) and patient-centered communication (β = 12.5, 95% CI [2.7, 22.3], p = 0.01). Conclusions: Medical student IPCP performance was associated with self-efficacy for interprofessional teamwork and patient-centered communication. Increasing interprofessional opportunities that influence medical students’ self-efficacy may increase engagement in IPCP.