Charles P. Friedman

Achieving a Nationwide Learning Health System

An infrastructure that enables secure reuse of patient data can speed the progression of new knowledge from bench to bedside. We outline the fundamental properties of a highly participatory rapid learning system that can be developed in part from meaningful use of electronic health records (EHRs). Future widespread adoption of EHRs will make increasing amounts of medical information available in computable form. Secured and trusted use of these data, beyond their original purpose of supporting the health care of individual patients, can speed the progression of knowledge from the laboratory bench to the patient’s bedside and provide a cornerstone for health care reform.

Do physicians know when their diagnoses are correct? Implications for decision support and error reduction.

AbstractOBJECTIVE: This study explores the alignment between physicians’ confidence in their diagnoses and the “correctness” of these diagnoses, as a function of clinical experience, and whether subjects were prone to over-or underconfidence. DESIGN: Prospective, counterbalanced experimental design. SETTING: Laboratory study conducted under controlled conditions at three academic medical centers. PARTICIPANTS: Seventy-two senior medical students, 72 senior medical residents, and 72 faculty internists. INTERVENTION: We created highly detailed, 2-to 4-page synopses of 36 diagnostically challenging medical cases, each with a definitive correct diagnosis. Subjects generated a differential diagnosis for each of 9 assigned cases, and indicated their level of confidence in each diagnosis. MEASUREMENTS AND MAIN RESULTS: A differential was considered “correct” if the clinically true diagnosis was listed in that subject’s hypothesis list. To assess confidence, subjects rated the likelihood that they would, at the time they generated the differential, seek assistance in reaching a diagnosis. Subjects’ confidence and correctness were “mildly” aligned (k=.314 for all subjects, .285 for faculty, .227 for residents, and .349 for students). Residents were overconfident in 41% of cases where their confidence and correctness were not aligned, whereas faculty were overconfident in 36% of such cases and students in 25%. CONCLUSIONS: Even experienced clinicians may be unaware of the correctness of their diagnoses at the time they make them. Medical decision support systems, and other interventions designed to reduce medical errors, cannot rely exclusively on clinicians’ perceptions of their needs for such support.

The research we should be doing.

No abstract available.

Charting the Winds of Change: Evaluating Innovative Medical Curricula.

Abstract The increased interest, in North America and around the world, in problem‐based and community‐oriented medical curricula has sparked interest in the evaluation of these innovative programs. In January 1989, the Josiah Macy Jr. Foundation sponsored a conference to consider designs for evaluation studies and the potential distinctive outcomes of the innovative curricula that might be foci of these studies. After defining an “innovative curriculum,” the participants identified seven characteristics of “important evaluation studies,” particularly endorsing studies that compare curricula as whole entities. The participants then identified 26 areas where differences between graduates of innovative and traditional curricula might be expected, and five equally important areas where differences are not expected. Distinctive outcomes of innovative curricula were anticipated in areas such as interpersonal skills, continuing learning, and professional satisfaction. Overall, these recommendations are offered to stimulate creative evaluations of the growing number of innovative programs in medical education. Acad. Med. 65(1990):8–14.

Toward a science of learning systems: a research agenda for the high-functioning Learning Health System

Objective The capability to share data, and harness its potential to generate knowledge rapidly and inform decisions, can have transformative effects that improve health. The infrastructure to achieve this goal at scale—marrying technology, process, and policy—is commonly referred to as the Learning Health System (LHS). Achieving an LHS raises numerous scientific challenges. Materials and methods The National Science Foundation convened an invitational workshop to identify the fundamental scientific and engineering research challenges to achieving a national-scale LHS. The workshop was planned by a 12-member committee and ultimately engaged 45 prominent researchers spanning multiple disciplines over 2 days in Washington, DC on 11–12 April 2013. Results The workshop participants collectively identified 106 research questions organized around four system-level requirements that a high-functioning LHS must satisfy. The workshop participants also identified a new cross-disciplinary integrative science of cyber-social ecosystems that will be required to address these challenges. Conclusions The intellectual merit and potential broad impacts of the innovations that will be driven by investments in an LHS are of great potential significance. The specific research questions that emerged from the workshop, alongside the potential for diverse communities to assemble to address them through a ‘new science of learning systems’, create an important agenda for informatics and related disciplines.

Training the Next Generation of Informaticians: The Impact of “BISTI” and Bioinformatics—A Report from the American College of Medical Informatics

In 2002-2003, the American College of Medical Informatics (ACMI) undertook a study of the future of informatics training. This project capitalized on the rapidly expanding interest in the role of computation in basic biological research, well characterized in the National Institutes of Health (NIH) Biomedical Information Science and Technology Initiative (BISTI) report. The defining activity of the project was the three-day 2002 Annual Symposium of the College. A committee, comprised of the authors of this report, subsequently carried out activities, including interviews with a broader informatics and biological sciences constituency, collation and categorization of observations, and generation of recommendations. The committee viewed biomedical informatics as an interdisciplinary field, combining basic informational and computational sciences with application domains, including health care, biological research, and education. Consequently, effective training in informatics, viewed from a national perspective, should encompass four key elements: (1). curricula that integrate experiences in the computational sciences and application domains rather than just concatenating them; (2). diversity among trainees, with individualized, interdisciplinary cross-training allowing each trainee to develop key competencies that he or she does not initially possess; (3). direct immersion in research and development activities; and (4). exposure across the wide range of basic informational and computational sciences. Informatics training programs that implement these features, irrespective of their funding sources, will meet and exceed the challenges raised by the BISTI report, and optimally prepare their trainees for careers in a field that continues to evolve.

Anatomy of the Clinical Simulation.

Computer-based clinical simulations have been used in medical education for the past 25 years. During this period, the technology has evolved from mainframe computers to microcomputers to multimedia. All designers of simulations must decide which elements of reality to include explicitly in a simulated case, which to leave to the users imagination, and when to intervene for educational purposes. Once these decisions are made, developers of simulations have many options for structuring the simulation itself. They can develop simulations with single or multiple patient encounters, with menu or natural-language requests for data, with varying levels of volunteered information about the simulated patient, with interpreted or uninterpreted clinical findings, with deterministic or probablistic evolution of the case, with various ways to give users feedback about their progress through the case, and with manual or automated creation of specific cases. Simulations derive their specific character from how these options are implemented.

Construct Validity of Medical Clinical Competence Measures: A Multitrait-Multimethod Matrix Study Using Confirmatory Factor Analysis

This study investigates the construct validity of three methods used to evaluate clinical competence in medicine: standardized test, supervisor performance ratings, and peer performance ratings. Three attributes of clinical competence are investigated: cognitive abilities, interpersonal skills, and professional qualities. Measures representing each attribute-method combination include: National Board of Medical Examiners (NBME) examination (standardized test of cognitive abilities); two scales derived from the California Psychological Inventory (CPI) (standardized test of interpersonal skills and professional qualities); and the three scales derived from the Resident Evaluation Form (REF) (peer and supervisor ratings of all three attributes). Scores for each attribute-method combination were obtained from a convenience sample of 166 resident physicians in three primary care specialties. These scores were cast into a multitrait-multimethod matrix design and analyzed using confirmatory factor analysis. Results suggest a lack of construct validity for the CPI and REF scales, moderate convergent validity for the NBME, and substantial method variance in the REF-derived ratings. Findings are discussed in terms of the implications for a theory of medical clinical competence, further research and development in clinical competence measurement, and current measurement practice in medical education.

The Need To Incorporate Health Information Technology Into Physicians’ Education And Professional Development

Nationwide, as physicians and health care systems adopt electronic health records, health information technology is becoming integral to the practice of medicine. But current medical education and professional development curricula do not systematically prepare physicians to use electronic health records and the data these systems collect. We detail how training in meaningful use of electronic health records could be incorporated into physician training, from medical school, through licensure and board certification, to continuing medical education and the maintenance of licensure and board certification. We identify six near-term opportunities for professional organizations to accelerate the integration of health information technology into their requirements.

The Marvelous Medical Education Machine or How Medical Education can be 'Unstuck' in Time

In this paper I will actually argue that medical education has become s̀tuck’ , not only in time but also in space and content. It has become stuck in time because events considered to be educational largely occur through interactions that require the learners and the faculty to be simultaneously participating in these interactions. It has become stuck in space because its mechanisms of delivery are largely bound to a speci® c physical location, the academic medical center with its classrooms and associated healthcare delivery venues. It has become stuck in content because the topics that are the focus of educational interactions are insufficiently under the control of the students, and the teachers. Increasingly, there is no reason for any of these requirements to be imposed on the educational process. Moreover, medical education remains stuck in an era when much of the rest of human enterprise is becoming unstuck, the result of a sweeping set of cultural changes made possible by information technology and primarily by the phenomenal proliferation of the global Internet (Drucker, 1999). I will further argue in this paper that medical education can gradually be `unstuck’ in space, time, and content through appropriate use of emerging technology, with emphasis on simulation methods that have become widespread in the use of training pilots and professionals in other disciplines. Modern ̄ ight simulators have become so sophisticated that experienced pilots being certi® ed to ̄ y a new aircraft might have a load of passengers in the back the ® rst time they actually ̄ y the plane (Dawson & Kaufman, 1998). While there will always be a pilot experienced in ̄ ying this aircraft alongside the neophyte in the cockpit, this practice clearly testi® es to the educational power of simulations. Recently, the US Navy adopted the inexpensive Microsoft `Flight Simulator’y program as standard training for its new pilots, after a trainee who practiced extensively on this program recorded the best performance ever on an initial training ̄ ight (Brewin, 2000). The `marvelous medical education machine’ , as the concept will be developed in this paper, is the complete simulator for medical education, analogous to the best of contemporary ̄ ight simulators. But like Vonnegut’s novel, the marvelous machine is currently a work of ® ction. It does not exist, although bits and pieces of it do exist, and these suggest what might be possible in the not-too-distant future. In the sections that follow, I will describe the need for the marvelous machine in greater detail, discuss what it can potentially do when built, expose the internal anatomy of the complete machine, review some of the pieces that exist now and how we might build it from here, and ® nally discuss some of the key educational research questions that will have to be illuminated along the way. This paper, in its entirety, will argue that building the marvelous machine should be a top priority for medical education nationally and internationally.