Peter Glassman

Assessing the accuracy of administrative data in health information systems.

Background:Administrative data play a central role in health care. Inaccuracies in such data are costly to health systems, they obscure health research, and they affect the quality of patient care. Objectives:We sought to prospectively determine the accuracy of the primary and secondary diagnoses recorded in administrative data sets. Research Design:Between March and July 2002, standardized patients (SPs) completed unannounced visits at 3 sites. We abstracted the 348 medical records from these visits to obtain the written diagnoses made by physicians. We also examined the patient files to identify the diagnoses recorded on the administrative encounter forms and extracted data from the computerized administrative databases. Because the correct diagnosis was defined by the SP visit, we could determine whether the final diagnosis in the administrative data set was correct and, if not, whether it was caused by physician diagnostic error, missing encounter forms, or incorrectly filled out forms. Subjects:General internal medicine outpatient clinics at 2 Veterans Administration facilities and a large, private medical center participated in this study. Measures:A total of 45 trained SPs presented to physicians with 4 common outpatient conditions. Results:The correct primary diagnosis was recorded for 57% of visits. Thirteen percent of errors were caused by physician diagnostic error, 8% to missing encounter forms, and 22% to incorrectly entered data. Findings varied by condition and site but not by level of training. Accuracy of secondary diagnosis data (27%) was even poorer. Conclusions:Although more research is needed to evaluate the cause of inaccuracies and the relative contributions of patient, provider, and system level effects, it appears that significant inaccuracies in administrative data are common. Interventions aimed at correcting these errors appear feasible.

Systematic Review: Comparison of the Quality of Medical Care in Veterans Affairs and Non-veterans Affairs Settings

Background:The Veterans Health Administration, the nations largest integrated delivery system, launched an organizational transformation in the mid 1990s to improve the quality of its care. Purpose:To synthesize the evidence comparing the quality of medical and other nonsurgical care in Veterans Affairs (VA) and non-VA settings. Data Sources:MEDLINE database and bibliographies of retrieved studies. Study Selection:Studies comparing the technical quality of nonsurgical care in VA and US non-VA settings published between 1990 and August 2009. Data Extraction:Two physicians independently reviewed 175 unique studies identified using the search strategy and abstracted data related to 6 domains of study quality. Data Synthesis:Thirty-six studies met the inclusion criteria. All 9 general comparative studies showed greater adherence to accepted processes of care or better health outcomes in the VA compared with care delivered outside the VA. Five studies of mortality following an acute coronary event found no clear survival differences between VA and non-VA settings. Three studies of care processes after an acute myocardial infarction found greater rates of evidence-based drug therapy in VA, and 1 found lower use of clinically-appropriate angiography in the VA. Three studies of diabetes care processes demonstrated a performance advantage for the VA. Studies of hospital mortality found similar risk-adjusted mortality rates in VA and non-VA hospitals. Limitations:Most studies used decade-old data, assessed self-reported service use, or included only a few VA or non-VA sites. Conclusions:Studies that assessed recommended processes of care almost always demonstrated that the VA performed better than non-VA comparison groups. Studies that assessed risk-adjusted mortality generally found similar rates for patients in VA and non-VA settings.

Recommendations to Improve the Usability of Drug-Drug Interaction Clinical Decision Support Alerts

OBJECTIVE To establish preferred strategies for presenting drug-drug interaction (DDI) clinical decision support alerts. MATERIALS AND METHODS A DDI Clinical Decision Support Conference Series included a workgroup consisting of 24 clinical, usability, and informatics experts representing academia, health information technology (IT) vendors, healthcare organizations, and the Office of the National Coordinator for Health IT. Workgroup members met via web-based meetings 12 times from January 2013 to February 2014, and two in-person meetings to reach consensus on recommendations to improve decision support for DDIs. We addressed three key questions: (1) what, how, where, and when do we display DDI decision support? (2) should presentation of DDI decision support vary by clinicians? and (3) how should effectiveness of DDI decision support be measured? RESULTS Our recommendations include the consistent use of terminology, visual cues, minimal text, formatting, content, and reporting standards to facilitate usability. All clinicians involved in the medication use process should be able to view DDI alerts and actions by other clinicians. Override rates are common but may not be a good measure of effectiveness. DISCUSSION Seven core elements should be included with DDI decision support. DDI information should be presented to all clinicians. Finally, in their current form, override rates have limited capability to evaluate alert effectiveness. CONCLUSION DDI clinical decision support alerts need major improvements. We provide recommendations for healthcare organizations and IT vendors to improve the clinician interface of DDI alerts, with the aim of reducing alert fatigue and improving patient safety.

Clinical Relevance of Automated Drug Alerts From the Perspective of Medical Providers

The authors used a real-time survey instrument and subsequent focus group among primary care clinicians at a large healthcare system to assess usefulness of automated drug alerts. Of 108 alerts encountered, 0.9% (n = 1) represented critical alerts, and 16% (n = 17) were significant drug interaction alerts. Sixty-one percent (n = 66) involved duplication of a medication or medication class. The rest (n = 24) involved topical medications, inhalers, or vaccines. Of the 84 potentially relevant alerts, providers classified 11% (9/84), or about 1 in 9, as useful. Drug interaction alerts were more often deemed useful than drug duplication alerts (44.4% versus 1.5%, P < .001). Focus group participants generally echoed these results when ranking the relevance of 15 selected alerts, although there was wide variance in ratings for individual alerts. Hence, a “smarter” system that utilizes a set of mandatory alerts while allowing providers to tailor use of other automated warnings may improve clinical relevance of drug alert systems.

Perceptions of Standards-based Electronic Prescribing Systems as Implemented in Outpatient Primary Care: A Physician Survey

OBJECTIVE To compare the experiences of e-prescribing users and nonusers regarding prescription safety and workload and to assess the use of information from two e-prescribing standards (for medication history and formulary and benefit information), as they are implemented. DESIGN Cross-sectional survey of physicians who either had installed or were awaiting installation of one of two commercial e-prescribing systems. MEASUREMENTS Perceptions about medication history and formulary and benefit information among all respondents, and among e-prescribing users, experiences with system usability, job performance impact, and amount of e-prescribing. RESULTS Of 395 eligible physicians, 228 (58%) completed the survey. E-prescribers (n = 139) were more likely than non-e-prescribers (n = 89) to perceive that they could identify clinically important drug-drug interactions (83 versus 67%, p = 0.004) but not that they could identify prescriptions from other providers (65 versus 60%, p = 0.49). They also perceived no significant difference in calls about drug coverage problems (76 versus 71% reported getting 10 or fewer such calls per week; p = 0.43). Most e-prescribers reported high satisfaction with their systems, but 17% had stopped using the system and another 46% said they sometimes reverted to handwriting for prescriptions that they could write electronically. The volume of e-prescribing was correlated with perceptions that it enhanced job performance, whereas quitting was associated with perceptions of poor usability. CONCLUSIONS E-prescribing users reported patient safety benefits but they did not perceive the enhanced benefits expected from using standardized medication history or formulary and benefit information. Additional work is needed for these standards to have the desired effects.

Exposure to automated drug alerts over time: effects on clinicians' knowledge and perceptions.

Objective:We tested whether interval exposure to an automated drug alert system that included approximately 2000 drug–drug interaction alerts increased recognition of selected interacting drug pairs. We also examined other perceptions about computerized order entry. Research Design:We administered cross-sectional surveys in 2000 and 2002 that included more than 260 eligible clinicians in each time period. Subjects:We studied clinicians practicing in ambulatory settings within a Southern California Veterans Affairs Healthcare System and who responded to both surveys (97 respondents). Measures:We sought to measure (1) recognition of selected drug–drug and drug–condition interactions and (2) other benefits and barriers to using automated drug alerts. Results:Clinicians correctly categorized similar percentages of the 7 interacting drug–drug pairs at baseline and follow-up (53% vs. 54%, P = 0.51) but improved their overall recognition of the 3 contraindicated drug–drug pairs (51% vs. 60%, P = 0.01). No significant changes from baseline to follow-up were found for the 8 interacting drug–condition pairs (60% vs. 62%, P = 0.43) or the 4 contraindicated drug–condition pairs (52% vs. 56%, P = 0.24). More providers preferred using order entry at follow-up than baseline (63% vs. 45%, P < 0.001). Signal-to-noise ratio remained the biggest reported problem at follow-up and baseline (54 vs. 57%, P = 0.75). In 2002, clinicians reported seeing a median of 5 drug alerts per week (representing approximately 12.5% of prescriptions entered), with a median 5% reportedly leading to an action. Conclusions:Interval exposure to automated drug alerts had little to no effect on recognition of selected drug–drug interactions. The primary perceived barrier to effective utilization of drug alerts remained the same over time.

The effect of pharmacy benefit design on patient-physician communication about costs.

AbstractBACKGROUND: Incentive-based formularies have been widely instituted to control the rising costs of prescription drugs. To work properly, such formularies depend on patients to be aware of financial incentives and communicate their cost preferences with prescribing physicians. The impact of financial incentives on patient awareness of and communication about those costs is unknown. OBJECTIVE: To evaluate the relationship between enrollment in incentive-based pharmacy benefit plans and awareness of out-of-pocket costs and rates of communication about out-of-pocket costs. DESIGN: A matched telephone survey of patients and their primary care physicians. SETTING: Los Angeles County. PARTICIPANTS: One thousand nine hundred and seventeen patients aged 53 to 82 (73% response rate). MEASUREMENTS: Patient-reported pharmacy benefit design, knowledge of out-of-pocket costs, and discussion of out-of-pocket costs with physicians. RESULTS: Sixty-two percent of patients who had prescription drug coverage and were aware of their pharmacy benefit design reported being enrolled in incentive-based plans. The majority of these (54%) were “never” or only “sometimes” aware of their out-of-pocket cost requirements at the time of the physician visit. After controlling for numerous physician and patient level variables, we found that patients enrolled in pharmacy benefit designs requiring no copayments were more likely to report they “never” discuss out-of-pocket costs with physicians compared with patients enrolled in incentive-based pharmacy benefit designs (81% vs 67%, P=.001) and patients with no prescription drug insurance (57%, P<.001). CONCLUSIONS: Incentive-based pharmacy benefit plans and lack of insurance are associated with increased rates of discussions about out-of-pocket costs. Nonetheless, most incentive-based enrollees are unaware of out-of-pocket costs when prescriptions are written and never discuss out-of-pocket costs with their physicians, likely mitigating the effectiveness of financial incentives to guide decision making. Considering that out-of-pocket costs are associated with adherence to medical therapy, interventions to improve patient access to out-of-pocket cost information and the frequency of patient-physician discussions about costs are needed.

Physicians’ Perceived Knowledge of and Responsibility for Managing Patients’ Out-of-Pocket Costs for Prescription Drugs

Background: Most insurers in the US have implemented incentive-based formularies that rely on out-of-pocket costs to influence prescription drug utilization. Medicare Part D plans have broadly adopted such benefit designs. Objective: To evaluate physicians’ perceptions of their knowledge of formularies and out-of-pocket costs, factors that influence knowledge of costs, physicians’ perceived responsibility for helping patients manage their out-of-pocket costs for prescription drugs, and physicians’ perceptions of the role of pharmacists in managing these costs. Methods: A multiple-choice survey was mailed to a random sample of 1200 physician members of the California Medical Association; a phone survey of nonresponders was then conducted. Results: Of 1027 surveys delivered to correct addresses, 509 (49.6%) responses were received. Thirty-three percent of physicians reported that they were usually or always aware of patients’ formularies and 20% were usually or always aware of patients’ out-of-pocket costs for medications. Surgeons, emergency department physicians, and physicians that prescribe from more formularies than other physicians are less likely to be aware of patients’ out-of-pocket costs, while physicians in large practices and those who use computers to prescribe are more aware. While 91% of physicians agreed that it is important that patients’ out-of-pocket costs be managed, 40% somewhat or strongly agreed that it is their responsibility to help. Sixty-five percent of physicians believed that it is the responsibility of the pharmacist to be familiar with patients’ out-of-pocket costs. Conclusions: Physicians often lack the knowledge to assist patients in the management of their out-of-pocket costs for prescription drugs and they depend on pharmacists to help patients manage those costs. Computer order entry and resources available in large physician organizations improve physicians’ awareness of out-of-pocket costs when prescribing.

Improving Pneumococcal Vaccine Rates: Nurse Protocols Versus Clinical Reminders

AbstractOBJECTIVE: To compare the effectiveness of three interventions designed to improve the pneumococcal vaccination rate. DESIGN: A prospective controlled trial. SETTING: Department of Veterans Affairs ambulatory care clinic. PATIENTS/PARTICIPANTS: There were 3,502 outpatients with scheduled visits divided into three clinic teams (A, B, or C). INTERVENTIONS: During a 12-week period, each clinic team received one intervention: (A) nurse standing orders with comparative feedback as well as patient and clinician reminders; (B) nurse standing orders with compliance reminders as well as patient and clinician reminders; and (C) patient and clinician reminders alone. Team A nurses (comparative feedback group) received information on their vaccine rates relative to those of team B nurses. Team B nurses (compliance reminders group) received reminders to vaccinate but no information on vaccine rates. MEASUREMENTS AND MAIN RESULTS: Team A nurses assessed more patients than team B nurses (39% vs 34%, p=.009). However, vaccination rates per total patient population were similar (22% vs 25%, p=.09). The vaccination rates for both team A and team B were significantly higher than the 5% vaccination rate for team C (p<.001). CONCLUSIONS: Nurse-initiated vaccine protocols raised vaccination rates substantially more than a physician and patient reminder system. The nurse-initiated protocol with comparative feedback modestly improved the assessment rate compared with the protocol with compliance reminders, but overall vaccination rates were similar.

Comparative Effectiveness of Vancomycin and Metronidazole for the Prevention of Recurrence and Death in Patients With Clostridium difficile Infection

Importance Metronidazole hydrochloride has historically been considered first-line therapy for patients with mild to moderate Clostridium difficile infection (CDI) but is inferior to vancomycin hydrochloride for clinical cure. The choice of therapy may likewise have substantial consequences on other downstream outcomes, such as recurrence and mortality, although these secondary outcomes have been less studied. Objective To evaluate the risk of recurrence and all-cause 30-day mortality among patients receiving metronidazole or vancomycin for the treatment of mild to moderate and severe CDI. Design, Setting, and Participants This retrospective, propensity-matched cohort study evaluated patients treated for CDI, defined as a positive laboratory test result for the presence of C difficile toxins or toxin genes in a stool sample, in the US Department of Veterans Affairs health care system from January 1, 2005, through December 31, 2012. Data analysis was performed from February 7, 2015, through November 22, 2016. Exposures Treatment with vancomycin or metronidazole. Main Outcomes and Measures The outcomes of interest in this study were CDI recurrence and all-cause 30-day mortality. Recurrence was defined as a second positive laboratory test result within 8 weeks of the initial CDI diagnosis. All-cause 30-day mortality was defined as death from any cause within 30 days of the initial CDI diagnosis. Results A total of 47 471 patients (mean [SD] age, 68.8 [13.3] years; 1947 women [4.1%] and 45 524 men [95.9%]) developed CDI, were treated with vancomycin or metronidazole, and met criteria for entry into the study. Of 47 147 eligible first treatment episodes, 2068 (4.4%) were with vancomycin. Those 2068 patients were matched to 8069 patients in the metronidazole group for a total of 10 137 included patients. Subcohorts were constructed that comprised 5452 patients with mild to moderate disease and 3130 patients with severe disease. There were no differences in the risk of recurrence between patients treated with vancomycin vs those treated with metronidazole in any of the disease severity cohorts. Among patients in the any severity cohort, those who were treated with vancomycin were less likely to die (adjusted relative risk, 0.86; 95% CI, 0.74 to 0.98; adjusted risk difference, –0.02; 95% CI, –0.03 to –0.01). No significant difference was found in the risk of mortality between treatment groups among patients with mild to moderate CDI, but vancomycin significantly reduced the risk of all-cause 30-day mortality among patients with severe CDI (adjusted relative risk, 0.79; 95% CI, 0.65 to 0.97; adjusted risk difference, –0.04; 95% CI, –0.07 to –0.01). Conclusions and Relevance Recurrence rates were similar among patients treated with vancomycin and metronidazole. However, the risk of 30-day mortality was significantly reduced among patients who received vancomycin. Our findings may further justify the use of vancomycin as initial therapy for severe CDI.