Mark A. Albanese

Problem-based learning: a review of literature on its outcomes and implementation issues.

&NA; The effects of problem‐based learning (PBL) were examined by conducting a meta‐analysis‐type review of the English‐language international literature from 1972 to 1992. Compared with conventional instruction, PBL, as suggested by the findings, is more nurturing and enjoyable; PBL graduates perform as well, and sometimes better, on clinical examinations and faculty evaluations; and they are more likely to enter family medicine. Further, faculty tend to enjoy teaching using PBL. However, PBL students in a few instances scored lower on basic sciences examinations and viewed themselves as less well prepared in the basic sciences than were their conventionally trained counterparts. PBL graduates tended to engage in back‐ward reasoning rather than the forward reasoning experts engage in, and there appeared to be gaps in their cognitive knowl‐edge base that could affect practice outcomes. The costs of PBL may slow its implementation in schools with class sizes larger than 100. While weaknesses in the criteria used to assess the outcomes of PBL and general weaknesses in study design limit the confidence one can give conclusions drawn from the literature, the authors recommend that caution be exercised in making comprehensive, curriculum‐wide conversions to PBL until more is learned about (1) the extent to which faculty should direct students throughout medical training, (2) PBL methods that are less costly, (3) cognitive‐processing weaknesses shown by PBL students, and (4) the apparent high resource utilization by PBL graduates. Academic Medicine 68(1993):52–81.

Problem‐based learning: why curricula are likely to show little effect on knowledge and clinical skills

A recent review of problem‐based learning’s effect on knowledge and clinical skills updated findings reported in 1993. The author argues that effect sizes (ES) seen with PBL have not lived up to expectations (0.8–1.0) and the theoretical basis for PBL, contextual learning theory, is weak. The purposes of this study were to analyse what constitutes reasonable ES in terms of the impacts on individuals and published reports, and to elaborate upon various theories pertaining to PBL.

Assessing personal qualities in medical school admissions.

The authors analyze the challenges to using academic measures (MCAT scores and GPAs) as thresholds for admissions and, for applicants exceeding the threshold, using personal qualities for admission decisions; review the literature on using the medical school interview and other admission data to assess personal qualities of applicants; identify challenges of developing better methods of assessing personal qualities; and propose a unified system for assessment. The authors discuss three challenges to using the threshold approach: institutional self-interest, inertia, and philosophical and historical factors. Institutional self-interest arises from the potential for admitting students with lower academic credentials, which could negatively influence indicators used to rank medical schools. Inertia can make introducing a new system complex. Philosophical and historical factors are those that tend to value maximizing academic measures. The literature identifies up to 87 different personal qualities relevant to the practice of medicine, and selecting the most salient of these that can be practically measured is a challenging task. The challenges to developing better personal quality measures include selecting and operationally defining the most important qualities, measuring the qualities in a cost-effective manner, and overcoming “cunning” adversaries who, with the incentive and resourcefulness, can potentially invalidate such measures. The authors discuss potential methods of measuring personal qualities and propose a unified system of assessment that would pool resources from certification and recertification efforts to develop competencies across the continuum with a dynamic, integrated approach to assessment.

Defining characteristics of educational competencies

Context  Doctor competencies have become an increasing focus of medical education at all levels. However, confusion exists regarding what constitutes a competency versus a goal, objective or outcome.

Ensuring reliability of outcome measures in multicenter clinical trials of treatments for acute ischemic stroke. The program developed for the Trial of Org 10172 in Acute Stroke Treatment (TOAST).

Ensuring the reliability and validity of outcome measures used in clinical trials is essential to the success of the trial. The Trial of Org 10172 in Acute Stroke Treatment (TOAST) is a multicenter clinical trial that is recruiting patients with acute ischemic stroke seen at medical centers across the United States. Methods This paper describes an approach to train physicians to use three clinical measures: the National Institutes of Health (NIH) Stroke Scale, a supplemental motor examination, and the Glasgow Outcome Scale. The program included education, certification, remediation when needed, monitoring, and reliability assessment. The goal was to ensure that interrater assessments were as equivalent to one another as possible. Results Of the first 95 clinicians who began the certification process, 75 passed during the first evaluation. Eighteen of the other physicians were able to complete the process after remediation. The intraclass correlations of both the NIH Stroke Scale and supplemental motor examination exceeded 0.95. The K values for the Glasgow Outcome Scale were 0.61 and 0.62 for the first and second ratings of the videotape, respectively. Conclusions Our experience suggests that a program that includes educational and certification processes can be performed as part of the design of a multicenter clinical trial. The method of providing educational and testing videotapes to each site so that physicians can be trained and certified is an effective, inexpensive, and practical approach for enhancing and certifying the expertise of the large number of physicians involved in a multicenter study.

Evaluation of a computer-based program for teaching cardiac anatomy.

RATIONALE AND OBJECTIVESThe authors determined to what extent a computer-based program could enhance or substitute for cadaver dissection in teaching cardiac anatomy to first-year medical students. METHODSFirst-year medical students (n = 175) were randomized into four groups. Group 1 (control) received no instruction, group 2 participated in cardiac dissection, group 3 viewed the computer application, and group 4 performed cardiac dissection and then viewed the computer application. Each group was tested with 10 ultrafast computed tomographic static images and 8 cardiac cadaver specimens. RESULTSThe computer program plus dissection was superior to either the computer program alone or dissection alone; however, the results varied according to the subtest used to assess the outcomes. CONCLUSIONSCardiac computer instruction after dissection resulted in dramatically improved image testing performance. However, computer instruction should not replace dissection for teaching cardiac anatomy.

Effectiveness of teaching radiologic image interpretation in gross anatomy: A long-term follow-up

This prospective study was designed to gauge the effectiveness of teaching radiologic interpretation during a gross anatomy course for first-year medical students by measuring short- and long-term ability to identify normal anatomic structures on radiologic diagnostic images. The evaluation required students to identify normal anatomic structures on radiographs, computed tomographs, ultrasonograms, and magnetic resonance images (MRIs). The assessments were made before (pre-test) and during (post-test) the experimental radiology portion of the gross anatomy course. The students were then retested 14 to 17 months later (long term). The pre-test correct response rate of 17% improved to 88% on the post-tests. After 14 to 17 months, the students had a 74% correct response rate on the same images and anatomic structures. This high level of long-term retention documents the effectiveness of integrating diagnostic radiologic imaging into normal gross anatomy instruction.

Medical Communication Behavior System. An interactional analysis system for medical interactions.

The study assessed the psychometric properties of the Medical Communication Behavior System. This observation system records time spent by the physicians and patients on specific behaviors in the categories of informational, relational, and negative situation behaviors by using hand-held electronic devices. The study included observations of 101 genetic counseling sessions and also assessed the outcome measures of patient knowledge and satisfaction. In addition, 41 of the sessions were rated using the Roter Interactional Analysis System, and 20 additional control subjects completed the post-counseling information without being observed to examine the effects of recording the session. Results showed good interobserver reliability, and evidence of concurrent, construct, and predictive validity. No differences were found between the observed and unobserved groups of any of the outcome measures.

Building a Competency-Based Curriculum: The Agony and the Ecstasy.

Physician competencies have increasingly been a focus of medical education at all levels. Although competencies are not a new concept, when the Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Medical Specialties (ABMS) jointly agreed on six competencies for certification and maintenance of certification of physicians in 1999, it brought about renewed interest. This article gives a brief overview of how a competency-based curriculum differs from other approaches and then describes the issues that need to be considered in the design and implementation of such a curriculum. In order to achieve success, a competency-based curriculum requires careful planning, preparation and a long-term commitment from everyone involved in the educational process. Building a competency-based curriculum is really about maintaining quality control and relinquishing control to those who care the most about medical education, our students. In the face of the many challenges that are facing undergraduate medical education (UME), including declining availability of teaching patients and over-burdened faculty, instituting quality control and relinquishing control will be necessary to maintain high quality.

Toward Setting a Research Agenda for Systematic Reviews of Evidence of the Effects of Medical Education

Purpose: To provide an update on, and a preliminary research agenda for, best evidence medical education (BEME). Summary: Efforts related to evidence-based medical education are summarized briefly, including BEME, the newly formed Campbell Collaboration, and the Cochrane Collaborations Effective Practice and Organization of Care review group. A list of topics and priorities for which evidence of effectiveness in medical education should be systematically reviewed is provided based on the results of a session at the July 2000 annual meeting of the Society of Directors of Research in Medical Education. The highest ranked topics clustered around four major conceptual areas: (a) curricular design, (b) learning and instructional methods, (c) testing and assessment, and (d) outcomes. Conclusions: BEME is gaining momentum with growing numbers of people becoming involved as well as an increased number of pertinent workshops, publications, and Web sites. The work of creating pertinent systematic reviews of the medical education literature is at hand.