Lynn A. Volk

Ten Commandments for Effective Clinical Decision Support: Making the Practice of Evidence-based Medicine a Reality

While evidence-based medicine has increasingly broad-based support in health care, it remains difficult to get physicians to actually practice it. Across most domains in medicine, practice has lagged behind knowledge by at least several years. The authors believe that the key tools for closing this gap will be information systems that provide decision support to users at the time they make decisions, which should result in improved quality of care. Furthermore, providers make many errors, and clinical decision support can be useful for finding and preventing such errors. Over the last eight years the authors have implemented and studied the impact of decision support across a broad array of domains and have found a number of common elements important to success. The goal of this report is to discuss these lessons learned in the interest of informing the efforts of others working to make the practice of evidence-based medicine a reality.

Relationship between use of electronic health record features and health care quality: results of a statewide survey.

Background:Electronic health records (EHRs) are widely viewed as useful tools for supporting the provision of high quality healthcare. However, evidence regarding their effectiveness for this purpose is mixed, and existing studies have generally considered EHR usage a binary factor and have not considered the availability and use of specific EHR features. Objective:To assess the relationship between the use of an EHR and the use of specific EHR features with quality of care. Research Design:A statewide mail survey of physicians in Massachusetts conducted in 2005. The results of the survey were linked with Healthcare Effectiveness Data and Information Set (HEDIS) quality measures, and generalized linear regression models were estimated to examine the associations between the use of EHRs and specific EHR features with quality measures, adjusting for physician practice characteristics. Subjects:A stratified random sample of 1884 licensed physicians in Massachusetts, 1345 of whom responded. Of these, 507 had HEDIS measures available and were included in the analysis (measures are only available for primary care providers). Measure:Performance on HEDIS quality measures. Results:The survey had a response rate of 71%. There was no statistically significant association between use of an EHR as a binary factor and performance on any of the HEDIS measure groups. However, there were statistically significant associations between the use of many, but not all, specific EHR features and HEDIS measure group scores. The associations were strongest for the problem list, visit note and radiology test result EHR features and for quality measures relating to womens health, colon cancer screening, and cancer prevention. For example, users of problem list functionality performed better on womens health, depression, colon cancer screening, and cancer prevention measures, with problem list users outperforming nonusers by 3.3% to 9.6% points on HEDIS measure group scores (all significant at the P < 0.05 level). However, these associations were not universal. Conclusions:Consistent with past studies, there was no significant relationship between use of EHR as a binary factor and performance on quality measures. However, availability and use of specific EHR features by primary care physicians was associated with higher performance on certain quality measures. These results suggest that, to maximize health care quality, developers, implementers and certifiers of EHRs should focus on increasing the adoption of robust EHR systems and increasing the use of specific features rather than simply aiming to deploy an EHR regardless of functionality.

Overrides of medication-related clinical decision support alerts in outpatients

BACKGROUND Electronic prescribing is increasingly used, in part because of government incentives for its use. Many of its benefits come from clinical decision support (CDS), but often too many alerts are displayed, resulting in alert fatigue. OBJECTIVE To characterize the override rates for medication-related CDS alerts in the outpatient setting, the reasons cited for overrides at the time of prescribing, and the appropriateness of overrides. METHODS We measured CDS alert override rates and the coded reasons for overrides cited by providers at the time of prescribing. Our primary outcome was the rate of CDS alert overrides; our secondary outcomes were the rate of overrides by alert type, reasons cited for overrides at the time of prescribing, and override appropriateness for a subset of 600 alert overrides. Through detailed chart reviews of alert override cases, and selective literature review, we developed appropriateness criteria for each alert type, which were modified iteratively as necessary until consensus was reached on all criteria. RESULTS We reviewed 157,483 CDS alerts (7.9% alert rate) on 2,004,069 medication orders during the study period. 82,889 (52.6%) of alerts were overridden. The most common alerts were duplicate drug (33.1%), patient allergy (16.8%), and drug-drug interactions (15.8%). The most likely alerts to be overridden were formulary substitutions (85.0%), age-based recommendations (79.0%), renal recommendations (78.0%), and patient allergies (77.4%). An average of 53% of overrides were classified as appropriate, and rates of appropriateness varied by alert type (p<0.0001) from 12% for renal recommendations to 92% for patient allergies. DISCUSSION About half of CDS alerts were overridden by providers and about half of the overrides were classified as appropriate, but the likelihood of overriding an alert varied widely by alert type. Refinement of these alerts has the potential to improve the relevance of alerts and reduce alert fatigue.

Electronic health records: which practices have them, and how are clinicians using them?

BACKGROUND Limited data exist to estimate the use of electronic health records (EHRs) in ambulatory care practices in the United States. METHODS We surveyed a stratified random sample of 1829 office practices in Massachusetts in 2005. The one-page survey measured use of health information technology, plans for EHR adoption and perceived barriers to adoption. RESULTS A total of 847 surveys were returned, for a response rate of 46%. Overall, 18% of office practices reported having an EHR. Primary-care-only and mixed practices reported similar adoption rates (23% and 25%, respectively, P = 0.70). The adoption rate in specialty practices (14%) was lower compared with both primary-care-only (P < 0.01) and mixed (P < 0.05) practices. The number of clinicians in the practice strongly correlated with EHR adoption (P < 0.001), with fewer small practices adopting EHRs. Among practices that have EHRs with laboratory and radiology result retrieval capabilities, at least 87% of practices report that a majority of their clinicians actively use these functionalities, while 74% of practices with electronic decision support report that the majority of clinicians actively use it. Among the practices without an EHR, 13% plan to implement one within the next 12 months, 24% within the next 1-2 years, 11% within the next 3-5 years, and 52% reported having no plans to implement an EHR in the foreseeable future. The most frequently reported barrier to implementation was lack of adequate funding (42%). CONCLUSIONS Overall, fewer than 1 in 5 medical practices in Massachusetts have an EHR. Even among adopters, though, doctor usage of EHR functions varied considerably by functionality and across practices. Many clinicians are not actively using functionalities that are necessary to improve health care quality and patient safety. Furthermore, among practices that do not have EHRs, more than half have no plan for adoption. Inadequate funding remains an important barrier to EHR adoption in ambulatory care practices in the United States.

Opportunities to enhance patient and physician e-mail contact.

The purpose of our study was to evaluate how e-mail is currently used between physicians and patients in an integrated delivery system, and to identify developments that might promote increased use of this form of communication. A paper-based survey questionnaire was administered to 94 primary care physicians. We evaluated the role e-mail currently plays in a physicians typical work day, physician views on the impact of e-mail on phone use and the barriers to increasing the use of e-mail with patients. 76% of physicians surveyed responded. All respondents currently use e-mail. Close to 75% of physicians use e-mail with their patients, but the vast majority do so with only 1-5% of those patients. 50% of physicians believe that up to 25% of their patients would send e-mail to them if given the option, with an additional 37% believing that between 25% and 50% of patients would value this option. The main reported barriers to physician-patient e-mail related to workload, security and payment. Our survey findings indicate that with adequate pre-screening, triage, and reimbursement mechanisms physicians would be open to substantially increasing e-mail communication with patients.

Physicians' Use of Key Functions in Electronic Health Records from 2005 to 2007: A Statewide Survey

OBJECTIVE Electronic health records (EHRs) have potential to improve quality and safety, but many physicians do not use these systems to full capacity. The objective of this study was to determine whether this usage gap is narrowing over time. DESIGN Follow-up mail survey of 1,144 physicians in Massachusetts who completed a 2005 survey. MEASUREMENTS Adoption of EHRs and availability and use of 10 EHR functions. RESULTS The response rate was 79.4%. In 2007, 35% of practices had EHRs, up from 23% in 2005. Among practices with EHRs, there was little change between 2005 and 2007 in the availability of nine of ten EHR features; the notable exception was electronic prescribing, reported as available in 44.7% of practices with EHRs in 2005 and 70.8% in 2007. Use of EHR functions changed inconsequentially, with more than one out of five physicians not using each available function regularly in both 2005 and 2007. Only electronic prescribing increased substantially: in 2005, 19.9% of physicians with this function available used it most or all the time, compared with 42.6% in 2007 (p < 0.001). CONCLUSIONS By 2007, more than one third of practices in Massachusetts reported having EHRs; the availability and use of electronic prescribing within these systems has increased. In contrast, physicians reported little change in the availability and use of other EHR functions. System refinements, certification efforts, and health policies, including standards development, should address the gaps in both EHR adoption and the use of key functions.

Randomized Controlled Trial of Health Maintenance Reminders Provided Directly to Patients Through an Electronic PHR

BACKGROUNDProvider and patient reminders can be effective in increasing rates of preventive screenings and vaccinations. However, the effect of patient-directed electronic reminders is understudied.OBJECTIVETo determine whether providing reminders directly to patients via an electronic Personal Health Record (PHR) improved adherence to care recommendations.DESIGNWe conducted a cluster randomized trial without blinding from 2005 to 2007 at 11 primary care practices in the Partners HealthCare system.PARTICIPANTSA total of 21,533 patients with access to a PHR were invited to the study, and 3,979 (18.5%) consented to enroll.INTERVENTIONSPatients in the intervention arm received health maintenance (HM) reminders via a secure PHR “eJournal,” which allowed them to review and update HM and family history information. Patients in the active control arm received access to an eJournal that allowed them to input and review information related to medications, allergies and diabetes management.MAIN MEASURESThe primary outcome measure was adherence to guideline-based care recommendations.KEY RESULTSIntention-to-treat analysis showed that patients in the intervention arm were significantly more likely to receive mammography (48.6% vs 29.5%, p = 0.006) and influenza vaccinations (22.0% vs 14.0%, p = 0.018). No significant improvement was observed in rates of other screenings. Although Pap smear completion rates were higher in the intervention arm (41.0% vs 10.4%, p < 0.001), this finding was no longer significant after excluding women’s health clinics. Additional on-treatment analysis showed significant increases in mammography (p = 0.019) and influenza vaccination (p = 0.015) for intervention arm patients who opened an eJournal compared to control arm patients, but no differences for any measure among patients who did not open an eJournal.CONCLUSIONSProviding patients with HM reminders via a PHR may be effective in improving some elements of preventive care.

Implementing practice-linked pre-visit electronic journals in primary care: patient and physician use and satisfaction

Electronic health records (EHRs) and EHR-connected patient portals offer patient-provider collaboration tools for visit-based care. During a randomized controlled trial, primary care patients completed pre-visit electronic journals (eJournals) containing EHR-based medication, allergies, and diabetes (study arm 1) or health maintenance, personal history, and family history (study arm 2) topics to share with their provider. Assessment with surveys and usage data showed that among 2027 patients invited to complete an eJournal, 70.3% submitted one and 71.1% of submitters had one opened by their provider. Surveyed patients reported they felt more prepared for the visit (55.9%) and their provider had more accurate information about them (58.0%). More arm 1 versus arm 2 providers reported that eJournals were visit-time neutral (100% vs 53%; p<0.013), helpful to patients in visit preparation (66% vs 20%; p=0.082), and would recommend them to colleagues (78% vs 22%; p=0.0143). eJournal integration into practice warrants further study.

Usability of a novel clinician interface for genetic results

The complexity and rapid growth of genetic data demand investment in information technology to support effective use of this information. Creating infrastructure to communicate genetic information to healthcare providers and enable them to manage that data can positively affect a patients care in many ways. However, genetic data are complex and present many challenges. We report on the usability of a novel application designed to assist providers in receiving and managing a patients genetic profile, including ongoing updated interpretations of the genetic variants in those patients. Because these interpretations are constantly evolving, managing them represents a challenge. We conducted usability tests with potential users of this application and reported findings to the application development team, many of which were addressed in subsequent versions. Clinicians were excited about the value this tool provides in pushing out variant updates to providers and overall gave the application high usability ratings, but had some difficulty interpreting elements of the interface. Many issues identified required relatively little development effort to fix suggesting that consistently incorporating this type of analysis in the development process can be highly beneficial. For genetic decision support applications, our findings suggest the importance of designing a system that can deliver the most current knowledge and highlight the significance of new genetic information for clinical care. Our results demonstrate that using a development and design process that is user focused helped optimize the value of this application for personalized medicine.

Evaluation of medication alerts in electronic health records for compliance with human factors principles

INTRODUCTION Increasing the adoption of electronic health records (EHRs) with integrated clinical decision support (CDS) is a key initiative of the current US healthcare administration. High over-ride rates of CDS alerts strongly limit these potential benefits. As a result, EHR designers aspire to improve alert design to achieve better acceptance rates. In this study, we evaluated drug-drug interaction (DDI) alerts generated in EHRs and compared them for compliance with human factors principles. METHODS We utilized a previously validated questionnaire, the I-MeDeSA, to assess compliance with nine human factors principles of DDI alerts generated in 14 EHRs. Two reviewers independently assigned scores evaluating the human factors characteristics of each EHR. Rankings were assigned based on these scores and recommendations for appropriate alert design were derived. RESULTS The 14 EHRs evaluated in this study received scores ranging from 8 to 18.33, with a maximum possible score of 26. Cohens κ (κ=0.86) reflected excellent agreement among reviewers. The six vendor products tied for second and third place rankings, while the top system and bottom five systems were home-grown products. The most common weaknesses included the absence of characteristics such as alert prioritization, clear and concise alert messages indicating interacting drugs, actions for clinical management, and a statement indicating the consequences of over-riding the alert. CONCLUSIONS We provided detailed analyses of the human factors principles which were assessed and described our recommendations for effective alert design. Future studies should assess whether adherence to these recommendations can improve alert acceptance.