James M. Shumway

AMEE Guide No. 25: The assessment of learning outcomes for the competent and reflective physician

Two important features of contemporary medical education are recognized. The first is an emphasis on assessment as a tool to ensure quality in training programmes, to motivate students and to direct what they learn. The second is a move to outcome-based education where the learning outcomes are defined and decisions about the curriculum are based on these. These two trends are closely related. If teachers are to do a better job of assessing their students, they need an understanding of the assessment process, an appreciation of the learning outcomes to be assessed and a recognition of the most appropriate tools to assess each outcome. Assessment tools selected should be valid, reliable, practical and have an appropriate impact on student learning. The preferred assessment tool will vary with the outcome to be assessed. It is likely to be some form of written test, a performance test such as an OSCE in which the student’s competence can be tested in a simulated situation, and a test of the student’s behaviour over time in clinical practice, based on tutors’ reports and students’ portfolios. An assessment profile can be produced for each student which highlights the learning outcomes the student has achieved at the required standard and other outcomes where this is not the case. For educational as well as economic reasons, there should be collaboration across the continuum of education in test development as it relates to the assessment of learning outcomes and in the implementation of a competence-based approach to assessment.

Planning with elderly outpatients for contingencies of severe illness - A survey and clinical trial

The authors examined whether elderly patients would report positive or adverse emotional effects after their doctor, during a routine clinic visit, asked them to begin planning for future serious illness. Seventy-four patients, 65 years old or older, who were followed at a university hospital medical clinic were randomly allocated to an intervention or a control group. The intervention was a detailed discussion with the patient’s physician of the patient’s wishes about decision making and life support therapy in the event of extreme or incapacitating illness. A blinded interviewer then asked all consenting patients how they felt about the physician, the clinic visit, and their medical care. Intervention-group patients were questioned about their reactions to the physician and the discussion. Four important findings emerged: 1) Some emotional uncertainty was created when doctors raised these questions unexpectedly: one patient became visibly upset during the discussion, and three who gave consent to be interviewed afterward said that the discussion had made them wonder about their health. Nonetheless, all patients who received the intervention and completed the study were pleased that their doctor had asked. 2) Only 44% of all consenting patients reported having discussed these issues previously; only one had done so with a doctor. 3) 97% of patients who responded wanted to be kept informed by the doctor about their medical situations in times of serious illness. 4) Patients’ replies to specific questions about life-sustaining therapy in the event of their own severe illnesses were quite variable. During routine clinic visits doctors can encourage most elderly patients to begin specific planning for potential severe illnesses.

Components of quality: competence, leadership, teamwork, continuing learning and service

The article appearing in this issue, ‘Using systems-based practice to integrate education and clinical services’ by Dickey et al. (2004) is a significant case study on how one institution has taken to heart the recommendations of the Accreditation Council for Graduate Medical Education (ACGME) and the Institute of Medicine (IOM). The ACGME identified general outcomes that all residents must attain to be competent. The IOM identified standards of quality healthcare to minimize errors. The IOM defines quality of care as ‘‘the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge’’ (Lohr, 1990). Marrying an institution’s responsibility to the public that its graduates are competent and that the education they receive helps minimize errors in patient care is of the highest importance in medical education today. The purpose of this commentary is to provide a little more explanation of the competences that are referred to in the aforementioned article and to introduce other, equally important components of quality.

Assessing Medical Student Professionalism: An Analysis of a Peer Assessment

Abstract Purpose: Students’ professional development is an essential aim of medical school. The purpose of this paper is to report how an assessment of first-year medical students’ professional behavior was designed and to investigate its measurement characteristics. Methods: The assessment was implemented as a peer assessment of professional skills, which were delineated according to a formal professional code developed by our curriculum committee. During the last week of the Fall 2005 semester, the professionalism assessment was administered online to students in a problem-based learning course. Results: The internal consistency of the assessment is adequate. The generalizability study found that raters nested within persons accounted for the majority of variance. While the inter-rater reliability is relatively low, using multiple raters may yield an acceptable estimate of the relative reliability. Conclusions: The results suggest that this peer assessment is a practical assessment, evidenced by the 91% compliance rate. However, future research and modifications will be needed to address the variance of responses, helping to discriminate between “poor” and “good” observations of professionalism. In addition, multiple raters are required to supply reliable estimates of students’ professional behavior. Coupling this evaluation with other professionalism evaluations may help reveal a more complete picture of students’ professional behavior.

Using information contained in the curriculum management information tool (CurrMIT) to capture opportunities for student learning and development.

The purpose of this paper is to examine how West Virginia University Medical School used the AAMC Curriculum Management Information Tool (CurrMIT) to map the undergraduate medical school curriculum. Information gleaned from this analysis identified what students are expected to learn, how they learn and how they are assessed. Information about the curriculum was entered into CurrMIT, creating a comprehensive picture of the curricular landscape. Learning outcomes were parceled out according to a competence-based framework. In addition, learning methods and assessment measures were identified. A total of 639 learning outcomes were identified across several competences. A total of 13 learning methods and 13 assessment measures were also identified in the undergraduate curriculum. The results suggest that students are expected to acquire varied knowledge, skills and attitudes. Further, students are presented with diverse learning methods and assessment measures. The curriculum map ascertains whether the programs components, such as learning outcomes, learning approaches and assessment methods, are designed and linked to further students’ learning. This analysis will lead to curricular improvements. The implications of this work can help faculty, students and other academic stakeholders shift tacit expectations of learning and development to a curricular reality and, in turn, help prepare future physicians for the changing field of medicine.

Survey of medicine/pediatrics residency training programs

The American Board of Internal Medicine and the American Board of Pediatrics agreed in 1967 to create combined medicine/pediatrics residency training programs. These programs span four years and provide 24 months of training in each discipline, leading to Board eligibility in both. Little is known about their curricula because there is no separate residency review committee to critique the current programs. The directors of the 65 current programs were surveyed by mail. Fifty-seven (88%) responded to questions about: lengths of time programs had been in operation, attitudes toward quality of residents, program structures and curricula, and performances of graduates taking the Boards. More than half of the programs were established after 1980. Forty of the programs’ graduates have passed the Medicine Boards, and 48 have passed the Pediatrics written Boards. Most programs were structured to have residents switch specialties every six months. Program Directors, both in Medicine and in Pediatrics, rated the quality of combined program residents the same as or slightly better than that of residents in non-combined programs. Medicine/pediatrics residencies have become a successful and important source of training for generalists’ careers in and outside of academe, and in both primary and specialty care.

Thinking outside the box: reflections on teaching.

(2001). Thinking outside the box: reflections on teaching. Medical Teacher: Vol. 23, No. 3, pp. 229-230.

The relationship between the educational process and students’ confidence

D ocumenting medical students’ clinical experiences is an ambitious process that is often complicated by diverse learning contexts. Medical students work with several teachers in different settings (for example, with in-patients; at outpatient clinics), and attend different campuses, such as regional clinical campuses. Other challenges include augmenting faculty members’ existing responsibilities for teaching and guiding students to further their knowledge and skills in dynamic environments. These have been characterised as a ‘black box’ that reveals little about the educational process in clinical settings. The confluence of these challenges makes it difficult to ascertain what students are learning in clerkships, how they are learning, and where they are gaining clinical experience.

A Teaching Oath: A Commitment to Medical Students’ Learning and Development

Background: There are several oaths and affirmations that are integrated into the academic customs of the health sciences, such as the Hippocratic Oath and the Ethical Affirmation for Scientists. What current oaths do not communicate is that teaching and learning are the foundation of the professions. Summary: We articulated an oath to punctuate the important role of teaching and to emphasize that educating students is not a marginal responsibility but an important duty. The goal of this oath is to include all educators who contribute to teaching medical students, including basic science and clinical faculty, residents, nurses, and healthcare providers. This oath is also designed to be concise, allowing for a public declaration during ceremonies that call attention to teaching and learning. Conclusions: Publically declaring the Teaching Oath is an opportunity to clarify the highest standards of teaching and to energize educators to fulfill the promise of a dynamic learning community.