Jerry A. Colliver

Effectiveness of Problem-based Learning Curricula: Research and Theory.

Purpose This article provides a critical overview of problem-based learning (PBL), its effectiveness for knowledge acquisition and clinical performance, and the underlying educational theory. The focus of the paper is on (1) the credibility of claims (both empirical and theoretical) about the ties between PBL and educational outcomes and (2) the magnitude of the effects. Method The author reviewed the medical education literature, starting with three reviews published in 1993 and moving on to research published from 1992 through 1998 in the primary sources for research in medical education. For each study the author wrote a summary, which included study design, outcome measures, effect sizes, and any other information relevant to the research conclusion. Results and Conclusion The review of the literature revealed no convincing evidence that PBL improves knowledge base and clinical performance, at least not of the magnitude that would be expected given the resources required for a PBL curriculum. The results were considered in light of the educational theory that underlies PBL and its basic research. The author concludes that the ties between educational theory and research (both basic and applied) are loose at best.

The electrocardiogram in obesity: Statistical analysis of 1,029 patients

The electrocardiogram in 1,029 obese subjects was correlated with the severity of obesity and with age, sex and blood pressure. The heart rate, PR interval, QRS duration, QTc interval and voltage (R + S or Q wave in leads I, II and III) increased, and the QRS vector shifted to the left with increasing obesity. These changes were independent of age, sex and blood pressure. Bradycardia was present in 19% of the patients, but tachycardia in only 0.5%. ST and T wave abnormalities were present in 11%, correlating better with increasing age and blood pressure than with severity of obesity. Conduction abnormalities were infrequent. Low voltage was present in only 3.9% of the patients and QTc prolongation was present in 28.3%. The heart rate and QRS voltage increase with increasing obesity. Conduction is slowed, and the QRS vector shifts toward the left as percent overweight increases. These changes must be considered when evaluating both baseline electrocardiographic studies in obese patients and the changes seen during weight reduction.

Psychometric properties of a standardized-patient checklist and rating-scale form used to assess interpersonal and communication skills

The results show that the SP checklist scores and the SP ratings of interpersonal and communication skills have comparable psychometric properties. The reliabilities of the five-item rating form (.76) and the single global rating of patient satisfaction (.70) were slightly higher than the reliability of the 17-item checklist (.65); this finding is of particular significance, given the greater length of the checklist. Also, the checklist scores and ratings appear to be measuring the same underlying dimension, with correlations of the checklist with the five ratings and with the single global rating being .82 and .81, respectively. Van der Vleuten and associates, in two excellent articles, noted a recent shift away from the use of subjective measures of clinical competence, such as rating scales, toward the use of presumably more objective measures, such as SP checklists. Their concern was that these objective measures may focus on somewhat trivial and easily measured aspects of the clinical encounter, and that more subtle but critical factors in clinical performance may be overlooked or ignored. They referred to such measurement as “objectified” rather than objective. The shift is based on the presumption that objective or objectified measurement is superior to subjective measurement, such as ratings, with respect to psychometric properties such as reliability. On the basis of a survey of several studies, though, the authors concluded that “objectified methods do not inherently provide more reliable scores” and “may even provide unwanted outcomes, such as negative effects on study behavior and triviality of the content being measured.” The results of the present study support this conclusion, showing somewhat higher reliabilities for subjective ratings than for the objective (or perhaps objectified) checklist. Also, the high uncorrected correlations suggest that the more reliable ratings are measuring the same underlying dimension as are the checklist scores. The present study also illustrates the application of a recently proposed method for constructing a valid SP checklist, which would consist of items that best reflect global ratings of performance. In this study, the ratings were provided by the SPs themselves, but ratings could be obtained from faculty-physician experts who observe student performance on the SP case. Thus, performance on individual checklist items would be correlated with expert ratings, to identify the items that best predict the ratings. The checklist, then, would be constructed of just those items that best predict the ratings, and the checklist could be used for future testing without the need for further faculty ratings (yet the checklist scores would reflect the faculty ratings). With this approach, it would seem possible to construct checklists for history-taking and physical-examination skills, as well as for interpersonal and communication skills. Thus, the faculty ratings would provide a basis for case development and refinement, including scoring and standard setting, and scores on the checklist would serve as a proxy for the gold-standard faculty ratings. The study suggests that SP ratings may be more efficient and more reliable than SP checklists for assessing interpersonal and communication skills. The study also demonstrates that global ratings by SPs (or by expert physician observers) can provide a basis for SP-test construction.

Full-curriculum interventions and small-scale studies of transfer: implications for psychology-type theory

Full-curriculum interventions have recently come under attack, apparently in an attempt to defend the theoretical principles on which these interventions are said to be based. Norman, for example presents a very interesting discussion of full-curriculum interventions vs. small-scale laboratory-type studies and concludes: (1) that curriculum-wide studies are not worth the effort involved in doing them ; and (2) that small-scale laboratory studies have revealed insights that are of practical significance in terms of identifying more effective teaching strategies. My view is that both types of studies provide important, even complementary, information for educational practice and that both are needed. For instance, the results on transfer from the small-scale studies referred to by Norman are consistent with the findings of full-curriculum studies, and together the results raise questions about the value of thinking in terms of psychology-type theoretical mechanisms in medical education.

Double-blind, placebo-controlled study of metrifonate, an acetylcholinesterase inhibitor, for Alzheimer disease

Summary:Fifty patients with probable Alzheimer disease (AD) completed a 3-month double-blind study to compare metrifonate to placebo. We dosed metrifonate to achieve a 40–60% inhibition of red blood cell acetylcholinesterase activity. The Alzheimer Disease Assessment Scale cognitive subscale score (ADAS-C) served as the primary outcome measure. At the completion of 3 months of treatment, the metrifonate group ADAS-C score differed significantly from the placebo group score by 2.6 points (p<0.01). A 0.75-point trend toward improvement occurred during treatment in the ADAS cognitive performance of the metrifonate group (p=0.15), and a 1.10-point deterioration in cognitive performance was found in the placebo group (p<0.02). On the Global Improvement Scale (GIS), the two groups differed significantly on their changes from baseline to treatment phase (p<0.02). Significant deterioration occurred in GIS scores (p<0.01) and in Mini Mental State Examination (MMSE) scores (p<0.03) in the placebo-treated group. Adverse effects were uncommon and did not require adjustment of the dose of metrifonate or discontinuation of treatment. We achieved a mean of 52.3% decrease in red blood cell acetylcholinesterase activity. During up to 18 months of subsequent open metrifonate treatment of patients, we found a deterioration of 1.68 points per year in MMSE performance. These findings support further study of the effects of metrifonate on deterioration rate in AD.

Educational Theory and Medical Education Practice: A Cautionary Note for Medical School Faculty.

Educational theory is routinely cited as justification for practice in medical education, even though the justification for the theory itself is unclear. Problem-based learning (PBL), for example, is said to be based on powerful educational principles that should result in strong effects on learning and performance. But research over the past 20 years has produced little convincing evidence for the educational effectiveness of PBL, which naturally raises doubts about the underlying theory. This essay reflects on educational theory, in particular cognitive theory, and concludes that the theory is little more than metaphor, not rigorous, tested, confirmed scientific theory. This metaphor/theory may lead to ideas for basic and applied research, which in turn may facilitate the development of theory. In the meantime, however, the theory cannot be trusted to determine practice in medical education. Despite the intuitive appeal of educational theory, medical educators have a responsibility to set aside their enthusiasm and make it clear to medical school faculty and administrators that educational innovations and practice claims are, at best, founded on conjecture, not on evidence-based science.

Direct, standardized assessment of clinical competence

Summary. Doctor ratings of clerkship performance are often discounted as not accurately reflecting clinical competence. Such ratings are influenced by the following uncontrolled variables: case difficulty; differing rater focus and standards; lack of agreement on what constitutes acceptable performance; and collective patient care responsibility masks individual contributions. Standardized direct measures of clinical competence were developed to control these factors and allow direct comparisons of student performance. Students saw 18 patients representing frequently occurring and important patient problems. Student actions and decisions were recorded and subsequent responses to questions revealed knowledge of pathophysiology, basis for actions, use and interpretation of laboratory investigations, and management. Actions and responses were graded using a pre‐set key. The examination covered 73% of designated clinical competencies. Examination scores corresponded with independent measures of clinical competence. Reliability studies indicated that new cases can be substituted in subsequent years with confidence that scores will maintain similar meaning. Costs are £6.95 per student per case, which is modest considering the quality and quantity of information acquired. Methods described are practical for evaluation of clerks and residents and for licensing and specialty certification examinations.

Reliability of performance on standardized patient cases: A comparison of consistency measures based on generalizability theory

Standardized patient cases have assumed an important role in the assessment of clinical competence in recent years. The reliability (consistency) of performance across standardized patient cases has been determined with consistency measures derived from generalizability theory—namely, the generalizability coefficient, Ep2; the dependability index, ; and the dependability index with cutoff, ϕ(C). These three consistency measures can be computed for quantitatively scored cases and for dichotomously scored cases; hence, six consistency measures could be computed for a given examination. Our purpose was to draw attention to the sizable differences among the computed values of these consistency measures for a new set of clinical competence examination data and to provide a review of the interpretations of the different measures. The findings showed considerable differences among the consistency measures, the number of cases needed to achieve the 0.80 reliability level, and the time required to administer th...

Simultaneous auditory stimuli shorten saccade latencies.

Eye movement responses induced by vestibular, smooth pursuit, and optokinetic stimulation are largely dependent on the velocity of the stimulus. Saccade latencies, on the other hand, are a function of the anatomic and physiologic components of the reflex. By producing an audible click simultaneously with the movement of a visual target used to stimulate a saccadic eye movement, the saccade latency is significantly shortened. This does not appear to be due to alerting. It is probably due to an auditory input to the superior colliculus which decreased threshold for initiating a saccadic eye movement. Twenty normal participants were tested with or without the click stimulus. Latencies were significantly shorter when the click stimulus was presented simultaneously with the eye movement. The anatomic pathways in this reflex are reviewed in the presentation. This study provides further information on sensory interaction in saccade reflexes and emphasizes the need to control stimulus conditions during saccade testing.

B Cell Depletion Therapy for New-Onset Opsoclonus-Myoclonus

Twelve immunotherapy‐naïve children with opsoclonus‐myoclonus syndrome and CSF B cell expansion received rituximab, adrenocorticotropic hormone (ACTH), and IVIg. Motor severity lessened 73% by 6 mo and 81% at 1 yr (P < 0.0001). Opsoclonus and action myoclonus disappeared rapidly, whereas gait ataxia and some other motor components improved more slowly. ACTH dose was tapered by 87%. Reduction in total CSF B cells was profound at 6 mo (‐93%). By study end, peripheral B cells returned to 53% of baseline and serum IgM levels to 63%. Overall clinical response trailed peripheral B cell and IgM depletion, but improvement continued after their levels recovered. All but one non‐ambulatory subject became ambulatory without additional chemotherapy; two relapsed and remitted; four had rituximab‐related or possibly related adverse events; and two had low‐titer human anti‐chimeric antibody. Combination of rituximab with conventional agents as initial therapy was effective and safe. A controlled trial with long‐term safety monitoring is indicated.