Eta S. Berner

Overconfidence as a Cause of Diagnostic Error in Medicine

The great majority of medical diagnoses are made using automatic, efficient cognitive processes, and these diagnoses are correct most of the time. This analytic review concerns the exceptions: the times when these cognitive processes fail and the final diagnosis is missed or wrong. We argue that physicians in general underappreciate the likelihood that their diagnoses are wrong and that this tendency to overconfidence is related to both intrinsic and systemically reinforced factors. We present a comprehensive review of the available literature and current thinking related to these issues. The review covers the incidence and impact of diagnostic error, data on physician overconfidence as a contributing cause of errors, strategies to improve the accuracy of diagnostic decision making, and recommendations for future research.

Overview of Clinical Decision Support Systems

Clinical decision support systems (CDSS) are computer systems designed to impact clinician decision making about individual patients at the point in time that these decisions are made.With the increased focus on the prevention of medical errors that has occurred since the publication of the landmark Institute of Medicine report, To Err Is Human, computer-based physician order entry (CPOE) systems, coupled with CDSS, have been proposed as a key element of systems’ approaches to improving patient safety.1, 2, 3, 4 If used properly, CDSS have the potential to change the way medicine has been taught and practiced. This chapter will provide an overview of clinical decision support systems, summarize current data on the use and impact of clinical decision support systems in practice, and will provide guidelines for users to consider as these systems begin to be incorporated in commercial systems, and implemented outside the research and development settings. The other chapters in this book will explore these issues in more depth.

Do local opinion leaders augment hospital quality improvement efforts? A randomized trial to promote adherence to unstable angina guidelines

Background. The influence of an opinion leader intervention on adherence to Unstable Angina (UA) guidelines compared with a traditional quality improvement model was investigated. Research Design. A group-randomized controlled trial with 2210 patients from 21 hospitals was designed. There were three intervention arms: (1) no intervention (NI); (2) a traditional Health Care Quality Improvement Program (HCQIP); and (3) a physician opinion leader in addition to the HCQIP model (OL). Quality indicators included: electrocardiogram within 20 minutes, antiplatelet therapy within 24 hours and at discharge, and heparin and &bgr;-blockers during hospitalization. Hospitals could determine the specific indicators they wished to target. Potential cases of UA were identified from Medicare claims data. UA confirmation was determined by a clinical algorithm based on data abstracted from medical records. Data analyses included both hospital level analysis (analysis of variance) and patient level analysis (generalized linear models). Results. The only statistically significant postintervention difference in percentage compliant was greater improvement for the OL group in the use of antiplatelet therapy at 24 hours in both hospital level (P = 0.01) and patient level analyses (P <0.05) compared with the HCQIP and NI groups. When analyses were confined to hospitals that targeted specific indicators, compared with the HCQIP hospitals, the OL hospitals showed significantly greater change in percentage compliant postintervention in both antiplatelet therapy during the first 24 hours (20.2% vs. −3.9%, P = 0.02) and heparin (31.0% vs.9.1%, P = 0.05). Conclusions. The influence of physician opinion leaders was unequivocally positive for only one of five quality indicators. To maximize adherence to best practices through physician opinion leaders, more research on how these physicians influence health care delivery in their organizations will be required.

Full Implementation of Computerized Physician Order Entry and Medication-Related Quality Outcomes: A Study of 3364 Hospitals

This study compares quality of care measures for hospitals with fully implemented computerized physician order entry (CPOE) systems with hospitals that have not fully implemented such a system. Using a cross-sectional design, this study linked hospital quality data from the Centers for Medicare and Medicaid Services to the Health Information Management Systems Society Analytics database, which contains hospital CPOE adoption information. Performance on quality measures was assessed using univariate and multivariate methods. In all, 8% of hospitals have fully implemented CPOE systems; CPOE hospitals were more frequently larger, not-for-profit, and teaching hospitals. After controlling for confounders, CPOE hospitals outperformed comparison hospitals on 5 of 11 measures related to ordering medications and on 1 of 9 nonmedication-related quality measures. Using a large sample of hospitals, our study found significant positive associations between specific objective quality indicators and CPOE implementation. (Am J Med Qual 2009;24:278-286)

Paradigms and Problem-Solving: A Literature Review.

Thomas Kuhns conceptions of the influence of paradigms on the progress of science from the framework for analyzing how medical educators have approached research on medical problem-solving. Kuhns ideas help to explain the often conflicting research results that have appeared over the last 20 years. Specifically, underlying assumptions about the problem-solving process are explored in light of data that demonstrate (a) low correlations between different measures of problem-solving, (b) inconsistent relationships between the amount of clinical experience and problem-solving ability, and (c) case specificity of performance. A new paradigm emphasizing multiple types of problems with varied solution strategies is proposed.

Applying Strategies to Overcome User Resistance in a Group of Clinical Managers to a Business Software Application: A Case Study

User resistance is a common occurrence when new information systems are implemented within health care organizations. Individuals responsible for overseeing implementation of these systems in the health care environment may encounter more resistance than trainers in other environments. It is important to be aware of methods to reduce resistance in end users. Proper training of end users is an important strategy for minimizing resistance. This article reviews the literature on the reasons for user resistance to health care information systems and the implications of this literature for designing training programs. The other principles for reducing resistance (communication, user involvement, strategic use of consultants) are illustrated with a case study involving training clinical managers on business applications. Individuals responsible for health care information system implementations should recognize that end user resistance can lead to system failure and should employ these best practices when embarking on new implementations.

Added value of video compared to audio lectures for distance learning

Asynchronous distance learning programs in health informatics are becoming more prevalent, but there is little research on the best practices of asynchronous learning technologies. There are virtually no data on the value of video of the instructor compared to an audio slide presentation. We conducted a randomized controlled trial in which we showed the same slide presentation to two groups of students. One group saw a video of the instructor giving the lecture and the other group saw the same presentation with audio only. Our results show that adding video to an audio presentation does not lead to greater satisfaction or greater learning. Students may think they want the video, but those who have it are not uniformly pleased with it. These results demonstrate that despite the intuitive appeal of streaming video, the addition of a video to an audio presentation may not be worth the extra expense and effort.

Healthcare information technology and economics

At the 2011 American College of Medical Informatics (ACMI) Winter Symposium we studied the overlap between health IT and economics and what leading healthcare delivery organizations are achieving today using IT that might offer paths for the nation to follow for using health IT in healthcare reform. We recognized that health IT by itself can improve health value, but its main contribution to health value may be that it can make possible new care delivery models to achieve much larger value. Health IT is a critically important enabler to fundamental healthcare system changes that may be a way out of our current, severe problem of rising costs and national deficit. We review the current state of healthcare costs, federal health IT stimulus programs, and experiences of several leading organizations, and offer a model for how health IT fits into our health economic future.

Diagnostic Decision Support Systems: How to Determine the Gold Standard?

In 1996 in an editorial on evaluation of decision support systems, Miller proposed that the bottom line in evaluating clinical decision support systems (CDSSs) should be “whether the user plus the system is better than the unaided user with respect to a specified task….”1 Since 1996, several studies have examined that issue, and, yet, there is still disagreement on the way to operationalize Millers proposition. In this issue of the Journal , Ramnarayan et al.2 describe a variety of metrics to evaluate the performance of a new pediatric diagnostic program, ISABEL. In a previous issue, Fraser et al.3 also described metrics to evaluate a heart disease program, the HDP. Both Ramnarayan et al. and Fraser et al. discussed how their measures compared with the earlier measures used by Berner et al.4 and Friedman et al.5 to evaluate other diagnostic programs. Why should it be so difficult to agree on a reasonable metric for evaluating these systems? Those of us who have struggled with this issue in our research have come to appreciate some of the difficulties that may not be immediately obvious in the published literature, but are important to articulate. Many of these issues are not unique to the diagnostic programs, but are a challenge in evaluating any CDSS. However, diagnostic programs are particularly challenging because, as Ramnarayan et al. indicate, diagnostic programs should influence both the diagnosis and the management plans. With that in mind, and with Millers injunction to focus on evaluating how the system and clinician work together, I would like to discuss the problems that arise with the different “gold standards” that researchers have used and also would like to offer suggestions for researchers and developers of diagnostic CDSS. Most researchers have included in their metrics the production of the …

A model for assessing information retrieval and application skills of medical students.

Purpose To develop and evaluate a model for assessing information retrieval and application skills, and to compare the performances on the assessment exercises of students who were and were not instructed in these skills. Method The authors developed a set of four examination stations, each with multiple subtasks, and administered the exams to students at two medical schools. Students at one school had intensive instruction in literature searching and filtering skills for information quality (instructed group), and those at the other school had minimal instruction in these areas (uninstructed group). The stations addressed pediatrics content and the skills of searching Medline and the World Wide Web, evaluating research articles, evaluating the accuracy of information from the Web, and using the information to make recommendations to patients. The authors determined the psychometric characteristics of the stations and compared the performances of the two groups of students. Results Students in the instructed group performed significantly better and with less variability than the uninstructed group on four tasks and no differently on seven tasks. There was no task on which the uninstructed group performed significantly better than the instructed group. Conclusion The prototype stations showed predictable differences across curricula, indicating that they have promise as assessment tools for the essential skills of information retrieval and application.