Randolph A. Miller

INTERNIST-I, An Experimental Computer-Based Diagnostic Consultant for General Internal Medicine

INTERNIST-I is an experimental computer program capable of making multiple and complex diagnoses in internal medicine. It differs from most other programs for computer-assisted diagnosis in the generality of its approach and the size and diversity of its knowledge base. To document the strengths and weaknesses of the program we performed a systematic evaluation of the capabilities of INTERNIST-I. Its performance on a series of 19 clinicopathological exercises (Case Records of the Massachusetts General Hospital) published in the Journal appeared qualitatively similar to that of the hospital clinicians but inferior to that of the case discussants. The evaluation demonstrated that the present form of the program is not sufficiently reliable for clinical applications. Specific deficiencies that must be overcome include the program’s inability to reason anatomically or temporally, its inability to construct differential diagnoses spanning multiple problem areas, its occasional attribution of findings to improper causes, and its inability to explain its “thinking.” (N Engl J Med. 1982; 307:468–76.

Commonly used surrogates for baseline renal function affect the classification and prognosis of acute kidney injury

Studies of acute kidney injury usually lack data on pre-admission kidney function and often substitute an inpatient or imputed serum creatinine as an estimate for baseline renal function. In this study, we compared the potential error introduced by using surrogates such as (1) an estimated glomerular filtration rate of 75 ml/min per 1.73 m(2) (suggested by the Acute Dialysis Quality Initiative), (2) a minimum inpatient serum creatinine value, and (3) the first admission serum creatinine value, with values computed using pre-admission renal function. The study covered a 12-month period and included a cohort of 4863 adults admitted to the Vanderbilt University Hospital. Use of both imputed and minimum baseline serum creatinine values significantly inflated the incidence of acute kidney injury by about half, producing low specificities of 77-80%. In contrast, use of the admission serum creatinine value as baseline significantly underestimated the incidence by about a third, yielding a low sensitivity of 39%. Application of any surrogate marker led to frequent misclassification of patient deaths after acute kidney injury and differences in both in-hospital and 60-day mortality rates. Our study found that commonly used surrogates for baseline serum creatinine result in bi-directional misclassification of the incidence and prognosis of acute kidney injury in a hospital setting.

The impact of peer management on test-ordering behavior

Context Can simple electronic aids help physicians reduce unnecessary, costly test ordering? Contribution In this interrupted time-series study from a large academic hospital, a committee of peer leaders selected ways to use their care provider order entry (CPOE) system to reduce unnecessary test ordering. Computer prompts questioning repetitive orders for routine tests and unbundling of tests within metabolic panel tests both reduced test orders. Patient readmission rates, length of stay, transfer to intensive care units, and mortality rates remained stable. Implications Peer-designed interventions using CPOE systems can improve provider test-ordering behavior. The Editors Providers of clinical care order excessive tests for hospitalized patients for defensive reasons (1) or ease of access (2) or because they cannot manage the fear of uncertainty (3, 4). Excessive ordering increases the use of technology and adds unnecessary costs to the delivery of health care. Motivated by studies demonstrating substantial variation in testing behaviors among providers (2, 5-14), inappropriate or unnecessary testing (15-23), and test addiction (24-26), investigators over the past decade have tried to impose sustainable limits on diagnostic evaluations. However, many recommended approaches are too time-consuming (27), difficult to scale across an institution (28), counterproductive to training (29), detrimental to clinical decision making (26), or inappropriately intrusive (26). One study suggested that short-term reductions in the amount of testing were not sustainable (30). In a review of various approaches to limit testing, Solomon and colleagues (24) noted that multifaceted interventions are most likely to succeed. The Institute of Medicine (31, 32) and industry leaders (33, 34) recently advocated the use of information systems to improve health care delivery, especially in the area of care provider order entry (CPOE) (35). Several studies document that computer-based reminders (25, 36-38) and just-in-time decision support (39) improve test-ordering practices. Care provider order entry systems also are an effective way to manage and implement change (38, 40) and can be used to reduce variability in provider behavior (41). Citing an alarming increase in the use of expensive or duplicate testing, the Vanderbilt University Medical Center, Nashville, Tennessee, chartered a resource utilization committee (RUC) to reduce variability in laboratory testing, imaging, and formulary use without restricting access to necessary or reasoned inquiry. Members of the committee included many clinical leaders in the institution (Appendix). The committee first identified specific patterns of excessive resource utilization in the hospital and subsequently devised several interventions using CPOE to reduce repetitive testing. The institutional review board approved the study, and the need for informed consent was waived. Methods Study Sample Vanderbilt University Hospital is a 658-bed tertiary care facility that houses 2 floors of the Vanderbilt Childrens Hospital. During the study period (1999 to 2001), more than 10000 orders were placed daily through the use of CPOE systems from 35 of the 37 patient care units; these 35 units cover approximately 600 beds of the hospital. The pediatric and neonatal intensive care units (ICUs) were not using CPOE systems during this interval. The study sample consisted of attending physicians, housestaff, medical students, nurses, advance practice nurses, and other clinical staff at Vanderbilt University Hospital who used CPOE systems. Physicians directly entered 70% of orders, and other members of the patient care team entered the remainder of orders. Care Provider Order Entry Like many systems, our CPOE system processes test orders as follows. First, a provider enters an order with a specified duration of recurrences. Second, the system generates up to 1 week of orders for individual tests. Third, each test is performed as scheduled unless a provider cancels subsequent occurrences. Finally, for recurring orders still active after each week, the software queues up a subsequent week of individual occurrences. Resource Utilization Committee Interventions To determine how and where to intervene, the RUC analyzed past CPOE log files for testing patterns and used bibliographic resources and its own expertise to determine optimal strategies for ordering individual tests. From December 1999 through the study period, during weekly to monthly committee meetings with all RUC members invited, the committee reviewed CPOE summary data that indicated the volume of laboratory, radiology, and cardiology tests that were ordered per month on each hospital ward. This was done prospectively to identify opportunities for intervention and was also done after the intervention to determine effectiveness. (No study intervention described in this paper was changed on the basis of this feedback, although the transition from the first intervention method to the second intervention method was catalyzed by such analysis.) Physician behaviors were not analyzed individually. Simple RUC member consensus after committee discussions determined which interventions to implementinformed by the data, the expertise of the chiefs of the clinical services serving on the RUC (who at times also consulted faculty experts within their departments and the literature), and the informatics faculty members of the RUC (who could speak to feasibility of various proposed CPOE-based interventions). In designing the educational components of the interventions, various RUC members (or their expert faculty designees within their departments) often provided literature-based synopses of evidence that were converted to hypertext markup language (HTML) documents and made available through the CPOE system at ordering time. Individuals creating such documents were responsible for regularly reviewing them to keep their content current. The first RUC intervention was implemented on 5 December 1999 as a broad attempt to reduce open-ended test ordering beyond 72 hours in the future. Each morning, the CPOE system would display a pop-up message that listed orders for scheduled laboratory tests, radiography, and electrocardiography extending beyond 72 hours. The pop-up prompted the provider to choose whether to continue the order, discontinue the order, or defer a decision until later in the day. If the provider chose to continue or discontinue the order, no other provider would receive pop-up reminders about that order until possibly the next day. The second RUC intervention involved several specific ordering constraints. The RUC reasoned that most repetitive orders for routine blood tests, radiology, and electrocardiography could not be justified without an intervening bedside visit. They then developed several specific ordering constraints. First, individual orders were limited to 1 occurrence in a fixed period of time. Second, the metabolic panel was unbundled and could be ordered only as individual components. Third, a graphical display of results from the previous week was placed on the ordering page for frequently ordered serum chemistry tests. This display made it difficult to claim that previous results were unknown at the time when additional tests were ordered. On 20 January 2000, the RUC initiated the second intervention by making all portable chest radiography orders one-time only. Starting on 1 February 2000, electrocardiograms could be ordered only once or twice in 8 hours per individual order. Providers still could order more electrocardiograms or portable chest radiographs by entering additional one-time orders with different start dates and times. On 21 March 2000, the RUC also implemented specific ordering constraints for unbundled components of the serum metabolic panel: Sodium, potassium, chloride, bicarbonate, and glucose tests could be ordered once or at recurring intervals up to hourly but not beyond 24 hours; blood urea nitrogen (BUN) or serum creatinine tests could be ordered only once in 24 hours. Orders for a complete blood count were not constrained during this second intervention period so that the complete blood count test could be used as a control for ordering behavior. Statistical Analysis The RUC examined 2 methods of counting test orders: on the basis of the day tests were first ordered or on the basis of the day tests were intended to occur. Because providers frequently enter orders to discontinue tests, the RUC defined net orders as the number of tests not discontinued before their time of occurrence. Some tests could be ordered as panels, so that a metabolic panel contributed 7 tests (sodium, potassium, chloride, bicarbonate, glucose, BUN, and creatinine tests) to the overall count of ordered component tests, whereas a portable chest radiograph or electrocardiogram counted as 1 test each. The data were evaluated by using interrupted time-series analyses. Patient name, individual ordering provider, and attending physician were not identified as part of the analysis. Each order was assessed in 3 ways to account for all possible outcomes. First, we noted the date that the order was written to determine whether constraining the duration of the order resulted in increased daily ordering. Second, we analyzed the daily number of net orders to approximate the number of ordered tests performed each day. Third, we counted the number of tests resulted in our institutional data repository to determine the actual number of tests performed. We ultimately used orders rather than test results as our primary measure because log file review revealed that net orders for a test closely reflected the actual number of tests performed and because tests ordered during system downtime were not subject to the intervention. The primary outcome was the daily number of new tests ordered and discontinued. Every CPOE order for each targeted test was considered. We ev

Clinical Decision Support and Electronic Prescribing Systems: A Time for Responsible Thought and Action

Electronic prescribing (e-prescribing) systems can provide computer-based support for the creation, transmission, dispensing, and monitoring of pharmacological therapies. In the United States and other countries, such systems have been documented, under certain conditions, to increase the safety and quality of patient care.1–5 The authors applaud the initial efforts of Teich and colleagues in the Joint Clinical Decision Support Workgroup (Joint CDS WG) to outline e-prescribing desiderata, as reported in this issue of JAMIA by Teich et al.6 Their article is published as an endorsed policy of the American Medical Informatics Association (AMIA). Previously, Bell et al. published an excellent list of desiderata for outpatient e-prescribing and sorted the desiderata into functional categories.7 Subsequently, Wang et al. surveyed e-prescribing vendor systems to determine that existing systems on average met only half the desiderata, with none exceeding 64% fulfillment.8 The recommendations outlined in the tables of the Joint CDS WG provide a useful point of departure for future discussions. Of note, the Joint CDS WG guidelines were developed as a “commissioned work” with externally determined foci, time limitations, and priorities, so that those guidelines do not fully cover all relevant areas. The Joint CDS WG document therefore represents an important first step in an evolving approach to a complex set of problems. The Joint CDS WG recommendations present a scenario of how e-prescribing features might be rolled out. The authors of this commentary would like to supplement, from what we believe is a broader perspective, the focused set of Joint CDS WG recommendations. The Joint CDS WG proposal has several strengths, including the recommendations that the United States should develop and promote shareable standards for e-prescribing and related decision support systems, a consensus should be developed on how to implement and evaluate decision support systems, and …

Recommendations for Responsible Monitoring and Regulation of Clinical Software Systems

In mid-1996, the FDA called for discussions on regulation of clinical software programs as medical devices. In response, a consortium of organizations dedicated to improving health care through information technology has developed recommendations for the responsible regulation and monitoring of clinical software systems by users, vendors, and regulatory agencies. Organizations assisting in development of recommendations, or endorsing the consortium position include the American Medical Informatics Association, the Computer-based Patient Record Institute, the Medical Library Association, the Association of Academic Health Sciences Libraries, the American Health Information Management Association, the American Nurses Association, the Center for Healthcare Information Management, and the American College of Physicians. The consortium proposes four categories of clinical system risks and four classes of measured monitoring and regulatory actions that can be applied strategically based on the level of risk in a given setting. The consortium recommends local oversight of clinical software systems, and adoption by healthcare information system developers of a code of good business practices. Budgetary and other constraints limit the type and number of systems that the FDA can regulate effectively. FDA regulation should exempt most clinical software systems and focus on those systems posing highest clinical risk, with limited opportunities for competent human intervention.

A computer-assisted medical diagnostic consultation service. Implementation and prospective evaluation of a prototype.

STUDY OBJECTIVE To evaluate the accuracy of a computer-aided consultation service using academic general internists and the Quick Medical Reference (QMR) diagnostic program: and to study the impact of the consultation on the diagnostic behavior of physicians caring for patients. DESIGN Prospective study of the diagnostic accuracy of computer-aided consultation in 31 cases, as well as a prospective study of ward team diagnoses and opinions before and after consultation. SETTING General medicine wards of two tertiary care centers. PARTICIPANTS Thirty-one patients identified as posing a diagnostic challenge and meeting eligibility criteria, as well as the housestaff caring for these patients. MEASUREMENTS AND MAIN RESULTS After 6 months follow-up, diagnoses were established in 20 of 31 cases. The diagnostic sensitivity of the computer-assisted diagnoses, 85% (95% CI, 56% to 97%), was similar to that of the consult service physicians, 80% (95% CI, 55% to 94%), but better than that of the ward teams, 60% (95% CI, 33% to 81%; P = 0.03 using the binomial test). The consultation influenced the postconsultation differential diagnoses of the ward teams in 26 of the 31 cases (95% CI, 92% to 95%). House officers rated the consultation service as being educationally helpful in 25 of the 31 cases (95% CI, 62% to 94%). CONCLUSIONS Computer-aided diagnostic consultation, when provided by physicians familiar with the limitations of the system and capable of overriding inappropriate suggestions, was both accurate and educationally helpful in most cases. The system provided reasonable diagnostic suggestions not previously considered by the ward teams and these suggestions were valued sufficiently to cause alteration of the original differential diagnoses.

Computer-based Insulin Infusion Protocol Improves Glycemia Control over Manual Protocol

OBJECTIVE Hyperglycemia worsens clinical outcomes in critically ill patients. Precise glycemia control using intravenous insulin improves outcomes. To determine if we could improve glycemia control over a previous paper-based, manual protocol, authors implemented, in a surgical intensive care unit (SICU), an intravenous insulin protocol integrated into a care provider order entry (CPOE) system. DESIGN Retrospective before-after study of consecutive adult patients admitted to a SICU during pre (manual protocol, 32 days) and post (computer-based protocol, 49 days) periods. MEASUREMENTS Percentage of glucose readings in ideal range of 70-109 mg/dl, and minutes spent in ideal range of control during the first 5 days of SICU stay. RESULTS The computer-based protocol reduced time from first glucose measurement to initiation of insulin protocol, improved the percentage of all SICU glucose readings in the ideal range, and improved control in patients on IV insulin for > or =24 hours. Hypoglycemia (<40 mg/dl) was rare in both groups. CONCLUSION The CPOE-based intravenous insulin protocol improved glycemia control in SICU patients compared to a previous manual protocol, and reduced time to insulin therapy initiation. Integrating a computer-based insulin protocol into a CPOE system achieved efficient, safe, and effective glycemia control in SICU patients.

Ethical and Legal Issues Related to the Use of Computer Programs in Clinical Medicine

As computer programs are used with increasing frequency in the clinical setting, ethicists, lawyers, computer scientists, clinicians, and patients must confront a group of problems: In what situations is it appropriate to use a medical computer program? Who should use these programs and how should they be used? What is the legal status of a computer program that provides medical advice? Can a proper balance be achieved between confidentiality of patient information and shared access to records by health care personnel? How can regulatory agencies, physicians, and patients determine if a program is safe for human use? Will programs be able to communicate with users well enough to prevent clinically harmful misunderstandings? Because few if any definitive answers are yet available, these questions remain the subject of much discussion.

The Basis for Using the Internet to Support the Information Needs of Primary Care

Synthesizing the state of the art from the published literature, this review assesses the basis for employing the Internet to support the information needs of primary care. The authors survey what has been published about the information needs of clinical practice, including primary care, and discuss currently available information resources potentially relevant to primary care. Potential methods of linking information needs with appropriate information resources are described in the context of previous classifications of clinical information needs. Also described is the role that existing terminology mapping systems, such as the National Library of Medicines Unified Medical Language System, may play in representing and linking information needs to answers.

A Computerized Insulin Infusion Titration Protocol Improves Glucose Control With Less Hypoglycemia Compared to a Manual Titration Protocol in a Trauma Intensive Care Unit

BACKGROUND Previous studies reflect reduced morbidity and mortality with intensive blood glucose control in critically ill patients. Unfortunately, implementation of such protocols has proved challenging. This study evaluated the degree of glucose control using manual paper-based vs computer-based insulin protocols in a trauma intensive care unit. METHODS Of 1455 trauma admissions from May 31 to December 31, 2005, a cohort of 552 critically ill patients met study entry criteria. The patients received intensive blood glucose management with IV insulin infusions. Using Fishers exact test, the authors compared patients managed with a computerized protocol vs a paper-based insulin protocol with respect to the portion of glucose values in a target range of 80-110 mg/dL, the incidence of hyperglycemia (> or =150 mg/dL), and the incidence of hypoglycemia (< or =40 mg/dL). RESULTS Three hundred nine patients were managed with a manual paper-based protocol and 243 were managed with a computerized protocol. The total number of blood glucose values across both groups was 21,178. Mean admission glucose was higher in the computer-based protocol group (170 vs 152 mg/dL; p < .001, t-test). Despite this finding by Fishers exact test, glucose control was superior in the computerized group; a higher portion of glucose values was in range 80-110 mg/dL (41.8% vs 34.0%; p < .001), less hyperglycemia occurred (12.8% vs 15.1%; p < .001), and less hypoglycemia occurred (0.2% vs 0.5%; p < .001). CONCLUSIONS A computerized insulin titration protocol improves glucose control by (1) increasing the percentage of glucose values in range, (2) reducing hyperglycemia, and (3) reducing severe hypoglycemia.