Harley Z. Ramelson

A method and knowledge base for automated inference of patient problems from structured data in an electronic medical record

BACKGROUND Accurate knowledge of a patients medical problems is critical for clinical decision making, quality measurement, research, billing and clinical decision support. Common structured sources of problem information include the patient problem list and billing data; however, these sources are often inaccurate or incomplete. OBJECTIVE To develop and validate methods of automatically inferring patient problems from clinical and billing data, and to provide a knowledge base for inferring problems. STUDY DESIGN AND METHODS We identified 17 target conditions and designed and validated a set of rules for identifying patient problems based on medications, laboratory results, billing codes, and vital signs. A panel of physicians provided input on a preliminary set of rules. Based on this input, we tested candidate rules on a sample of 100,000 patient records to assess their performance compared to gold standard manual chart review. The physician panel selected a final rule for each condition, which was validated on an independent sample of 100,000 records to assess its accuracy. RESULTS Seventeen rules were developed for inferring patient problems. Analysis using a validation set of 100,000 randomly selected patients showed high sensitivity (range: 62.8-100.0%) and positive predictive value (range: 79.8-99.6%) for most rules. Overall, the inference rules performed better than using either the problem list or billing data alone. CONCLUSION We developed and validated a set of rules for inferring patient problems. These rules have a variety of applications, including clinical decision support, care improvement, augmentation of the problem list, and identification of patients for research cohorts.

An unintended consequence of electronic prescriptions: prevalence and impact of internal discrepancies

Many e-prescribing systems allow for both structured and free-text fields in prescriptions, making possible internal discrepancies. This study reviewed 2914 electronic prescriptions that contained free-text fields. Internal discrepancies were found in 16.1% of the prescriptions. Most (83.8%) of the discrepancies could potentially lead to adverse events and many (16.8%) to severe adverse events, involving a hospital admission or death. Discrepancies in doses, routes or complex regimens were most likely to have a potential for a severe event (p=0.0001). Discrepancies between structured and free-text fields in electronic prescriptions are common and can cause patient harm. Improvements in electronic medical record design are necessary to minimize the risk of discrepancies and resulting adverse events.

Improving completeness of electronic problem lists through clinical decision support: a randomized, controlled trial

Background Accurate clinical problem lists are critical for patient care, clinical decision support, population reporting, quality improvement, and research. However, problem lists are often incomplete or out of date. Objective To determine whether a clinical alerting system, which uses inference rules to notify providers of undocumented problems, improves problem list documentation. Study Design and Methods Inference rules for 17 conditions were constructed and an electronic health record-based intervention was evaluated to improve problem documentation. A cluster randomized trial was conducted of 11 participating clinics affiliated with a large academic medical center, totaling 28 primary care clinical areas, with 14 receiving the intervention and 14 as controls. The intervention was a clinical alert directed to the provider that suggested adding a problem to the electronic problem list based on inference rules. The primary outcome measure was acceptance of the alert. The number of study problems added in each arm as a pre-specified secondary outcome was also assessed. Data were collected during 6-month pre-intervention (11/2009–5/2010) and intervention (5/2010–11/2010) periods. Results 17 043 alerts were presented, of which 41.1% were accepted. In the intervention arm, providers documented significantly more study problems (adjusted OR=3.4, p<0.001), with an absolute difference of 6277 additional problems. In the intervention group, 70.4% of all study problems were added via the problem list alerts. Significant increases in problem notation were observed for 13 of 17 conditions. Conclusion Problem inference alerts significantly increase notation of important patient problems in primary care, which in turn has the potential to facilitate quality improvement. Trial Registration ClinicalTrials.gov: NCT01105923.

Evaluating nicotine replacement therapy and stage-based therapies in a population-based effectiveness trial.

Pharmacological interventions for smoking cessation are typically evaluated using volunteer samples (efficacy trials) but should also be evaluated in population-based trials (effectiveness trials). Nicotine replacement therapy (NRT) alone and in combination with behavioral interventions was evaluated on a population of smokers from a New England Veterans Affairs Medical Center. Telephone interviews were completed with 3,239 smokers, and 2,054 agreed to participate (64%). Participants were randomly assigned to one of four conditions: stage-matched manuals (MAN); NRT plus manuals (NRT + MAN); expert system plus NRT and manuals (EXP + NRT + MAN); and automated counseling plus NRT, manuals, and expert system (TEL + EXP + NRT + MAN). Assessments were completed at baseline, 10, 20, and 30 months. The point prevalence cessation rates at final follow-up (30 months) were MAN, 20.3%; NRT + MAN, 19.3%; EXP + NRT + MAN, 17.6%; and TEL + EXP + NRT + MAN, 19.9%. Stage-matched manuals provided cessation rates comparable with previous studies. The addition of NRT, expert system interventions, and automated telephone counseling failed to produce a further increase in intervention effectiveness.

How physicians document outpatient visit notes in an electronic health record

BACKGROUND Clinical documentation, an essential process within electronic health records (EHRs), takes a significant amount of clinician time. How best to optimize documentation methods to deliver effective care remains unclear. OBJECTIVE We evaluated whether EHR visit note documentation method was influenced by physician or practice characteristics, and the association of physician satisfaction with an EHR notes module. MEASUREMENTS We surveyed primary care physicians (PCPs) and specialists, and used EHR and provider data to perform a multinomial logistic regression of visit notes from 2008. We measured physician documentation method use and satisfaction with an EHR notes module and determined the relationship between method and physician and practice characteristics. RESULTS Of 1088 physicians, 85% used a single method to document the majority of their visits. PCPs predominantly documented using templates (60%) compared to 34% of specialists, while 38% of specialists predominantly dictated. Physicians affiliated with academic medical centers (OR 1.96, CI (1.23, 3.12)), based at a hospital (OR 1.57, 95% CI (1.04, 2.36)) and using the EHR for longer (OR 1.13, 95% CI (1.03, 1.25)) were more likely to dictate than use templates. Most physicians of 383 survey responders were satisfied with the EHR notes module, regardless of their preferred documentation method. CONCLUSIONS Physicians predominantly utilized a single method of visit note documentation and were satisfied with their approach, but the approaches they chose varied. Demographic characteristics were associated with preferred documentation method. Further research should focus on why variation exists, and the quality of the documentation resulting from different methods used.

Increasing Patient Engagement: Patients’ Responses to Viewing Problem Lists Online

OBJECTIVE To characterize the opinions, emotions, and actions taken by patients who viewed their electronic problem list via an online personal health record (PHR). MATERIALS AND METHODS An online survey of patients who viewed their problem lists, as maintained by their healthcare provider, in a web-based PHR linked to an electronic health record for the first time. RESULTS A total 3,649 patients completed the survey, yielding a response rate of 42.1%. Patient attitudes towards the problem list function were positive overall, with 90.4% rating it at least somewhat useful and 86.7% reporting they would probably or definitely use it again. Nearly half (45.6%) of patients identified at least one major or minor problem missing from their list. After viewing the list, 56.1% of patients reported taking at least one action in response, with 32.4% of patients reporting that they researched a condition on the Internet, 18.3% reported that they contacted their healthcare provider and 16.7% reported changing or planning to change a health behavior (patients could report multiple actions). 64.7% of patients reported feeling at least somewhat happy while viewing their problem list, though others reported feeling sad (30.4%), worried (35.7%) or scared (23.8%) (patients could report multiple emotions). A smaller number of patients reported feeling angry (16.6%) or ashamed (14.3%). Patients who experienced an emotional response were more likely to take action. CONCLUSION Overall, patients found the ability to view their problem lists very useful and took action in response to the information. However, some had negative emotions. More research is needed into optimal strategies for supporting patients receiving this information.

Awareness and Use of the After-Visit Summary Through a Patient Portal: Evaluation of Patient Characteristics and an Application of the Theory of Planned Behavior

Background Patient portals are being used to provide a clinical summary of the office visit or the after-visit summary (AVS) to patients. There has been relatively little research on the characteristics of patients who access the AVS through a patient portal and their beliefs about the AVS. Objective The aim was to (1) assess the characteristics of patients who are aware of and access the AVS through a patient portal and (2) apply the Theory of Planned Behavior (TPB) to predict behavioral intention of patients toward accessing the AVS provided through a patient portal. Methods We developed a survey capturing the components of TPB (beliefs, attitude, perceived norm, and perceived behavioral control). Over a 6-month period, patients with a patient portal account with an office visit in the previous week were identified using our organization’s scheduling system. These patients were sent an email about the study and a link to the survey via their portal account. We applied univariate statistical analysis (Pearson chi-square and 1-way ANOVA) to assess differences among groups (aware/unaware of AVS and accessed/did not access AVS). We reported means and standard deviations to depict belief strengths and presented correlations between beliefs and attitude, perceived norm, and perceived behavioral control. We used hierarchical regression analysis to predict behavioral intention toward accessing the AVS through the patient portal. Results Of the 23,336 patients who were sent the survey, 5370 responded for a response rate of 23.01%. Overall, 76.52% (4109/5370) were aware that the AVS was available through the patient portal and 54.71% of those (2248/4109) accessed the AVS within 5 days of the office visit. Patients who accessed the AVS had a greater number of sessions with the portal (mean 119, SD 221.5) than those who did not access the AVS (mean 79.1, SD 123.3, P<.001); the difference was not significant for awareness of the AVS. The strongest behavioral beliefs with accessing the AVS were being able to track visits and tests (mean 2.53, SD 1.00) followed by having medical information more readily accessible (mean 2.48, SD 1.07). In all, 56.7% of the variance in intention to access the AVS through the portal was accounted for by attitude, perceived norm, and perceived behavioral control. Conclusions Most users of a patient portal were aware that the AVS was accessible through the portal. Patients had stronger beliefs about accessing the AVS with the goal of timely and efficient access of information than with engaging in their health care. Interventions to improve patient access of the AVS can focus on providers promoting patient beliefs about the value of the AVS for tracking tests and visits, and timely and efficient access of information.

Physician Perceptions and Beliefs about Generating and Providing a Clinical Summary of the Office Visit

BACKGROUND A core measure of the meaningful use of EHR incentive program is the generation and provision of the clinical summary of the office visit, or the after visit summary (AVS), to patients. However, little research has been conducted on physician perceptions and beliefs about the AVS. OBJECTIVES Evaluate physician perceptions and beliefs about the AVS and the effect of the AVS on workload, patient outcomes, and the care the physician delivers. METHODS A cross-sectional online survey of physicians at two academic medical centers (AMCs) in the northeast who are participating in the meaningful use EHR incentive program. RESULTS Of the 1 795 physicians at both AMCs participating in the incentive program, 853 completed the survey for a response rate of 47.5%. Eighty percent of the respondents reported that the AVS was easy (very easy or quite easy or somewhat easy) to generate and provide to patients. Nonetheless, more than three-fourths of the respondents reported a negative effect of generating and providing the AVS on workload of office staff (78%) and workload of physicians (76%). Primary care physicians had more positive beliefs about the effect of the AVS on patient outcomes than specialists (p<0.001) and also had more positive beliefs about the effect of the AVS on the care they delivered than specialists (p<0.001). CONCLUSIONS Achieving the core meaningful use measure of generating and providing the AVS was easy for physicians but it did not necessarily translate into positive beliefs about the effect of the AVS on patient outcomes or the care the physician delivered. Physicians also had negative beliefs about the effect of the AVS on workload. To promote positive beliefs among physicians around the AVS, organizations should obtain physician input into the design and implementation of the AVS and develop strategies to mitigate its negative impacts on workload.

Relationship between documentation method and quality of chronic disease visit notes

OBJECTIVE To assses the relationship between methods of documenting visit notes and note quality for primary care providers (PCPs) and specialists, and to determine the factors that contribute to higher quality notes for two chronic diseases. METHODS Retrospective chart review of visit notes at two academic medical centers. Two physicians rated the subjective quality of content areas of the note (vital signs, medications, lifestyle, labs, symptoms, assessment & plan), overall quality, and completed the 9 item Physician Documentation Quality Instrument (PDQI-9). We evaluated quality ratings in relation to the primary method of documentation (templates, free-form or dictation) for both PCPs and specialists. A one factor analysis of variance test was used to examine differences in mean quality scores among the methods. RESULTS A total of 112 physicians, 71 primary care physicians (PCP) and 41 specialists, wrote 240 notes. For specialists, templated notes had the highest overall quality scores (p≤0.001) while for PCPs, there was no statistically significant difference in overall quality score. For PCPs, free form received higher quality ratings on vital signs (p = 0.01), labs (p = 0.002), and lifestyle (p = 0.002) than other methods; templated notes had a higher rating on medications (p≤0.001). For specialists, templated notes received higher ratings on vital signs, labs, lifestyle and medications (p = 0.001). DISCUSSION There was no significant difference in subjective quality of visit notes written using free-form documentation, dictation or templates for PCPs. The subjective quality rating of templated notes was higher than that of dictated notes for specialists. CONCLUSION As there is wide variation in physician documentation methods, and no significant difference in note quality between methods, recommending one approach for all physicians may not deliver optimal results.

Development of a cognitive framework of patient record summary review in the formative phase of user-centered design

Excellent usability characteristics allow electronic health record (EHR) systems to more effectively support clinicians providing care and contribute to better quality and safety. The Office of the National Coordinator for Health IT (ONC) therefore requires all vendors to follow a User-Centered Design (UCD) process to increase the usability of their products in order to meet certification criteria for the Safety-Enhanced Design part of the Meaningful Use (stage 2) EHR incentive program. This report describes the initial stage of a UCD process in which foundational design concepts were formulated. We designed a functional prototype of an EHR module intended to help clinicians to efficiently complete a summary review of an electronic patient record before an ambulatory visit. Cognitively-based studies were performed and the results used to develop a cognitive framework that subsequently guided design of a prototype. Results showed that clinicians categorized and reasoned with patient data in distinct patterns; they preferred to review relevant history in the assessment and plan section of the most recent note, to search for changes in health and for new episodes of care since the last visit and to look up current-day data such as vital signs. These basic concepts were represented in the design, for instance, by screen division into vertical thirds that had historical content to the left and most recent data to the right. Other characteristics such as visual association of contextual information or direct, one-click access to the assessment and plan section of visit notes were directly informed by our findings and refined in a series of UCD-specific iterative testing. Understanding of tasks and cognitive demands early in the UCD process was critically important for developing a tool optimized for reasoning and workflow preferences of clinicians.