Gretchen M. Hultman

Characterizing Patient-Generated Clinical Data and Associated Implications for Electronic Health Records.

Patient-facing technologies are increasingly utilized for direct patient data entry for potential incorporation into the electronic health record. We analyzed patient-entered data during implementation of a patient-facing data entry technology using an online patient portal and clinic-based tablet computers at a University-based tertiary medical center clinic, including entries for past medical history, past surgical history, and social history. Entries were assessed for granularity, clinical accuracy, and the addition of novel information into the record. We found that over half of patient-generated diagnoses were duplicates of lesser or equal granularity compared to previous provider-entered diagnoses. Approximately one fifth of patient-generated diagnoses were found to meet the criteria for new, meaningful additions to the medical record. Our findings demonstrate that while patient-generated data provides important additional information, it may also present challenges including generating inaccurate or less granular information.

Single Health System Adherence to 2012 Cervical Cancer Screening Guidelines at Extremes of Age and Posthysterectomy

OBJECTIVE To estimate the proportion of guideline nonadherent Pap tests in women aged younger than 21 years and older than 65 years and posthysterectomy in a single large health system. Secondary objectives were to describe temporal trends and patient and health care provider characteristics associated with screening in these groups. METHODS A retrospective cross-sectional chart review was performed at Fairview Health Services and University of Minnesota Physicians. Reasons for testing and patient and health care provider information were collected. Tests were designated as indicated or nonindicated per the 2012 cervical cancer screening guidelines. Point estimates and descriptive statistics were calculated. Patient and health care provider characteristics were compared between indicated and nonindicated groups using χ and Wilcoxon rank-sum tests. RESULTS A total of 3,920 Pap tests were performed between September 9, 2012, and August 31, 2014. A total of 257 (51%; 95% confidence interval [CI] 46.1-54.9%) of tests in the younger than 21 years group, 536 (40%; 95% CI 37.7-43.1%) in the older than 65 years group, and 605 (29%; 95% CI 27.1-31.0%) in the posthysterectomy group were not indicated. White race in the older than 65 years group was the only patient characteristic associated with receipt of a nonindicated Pap test (P=.007). Health care provider characteristics associated with nonindicated Pap tests varied by screening group. Temporal trends showed a decrease in the proportion of nonindicated tests in the younger than 21 years group but an increase in the posthysterectomy group. CONCLUSION For women aged younger than 21 years and older than 65 years and posthysterectomy, 35% of Pap tests performed in our health system were not guideline-adherent. There were no patient or health care provider characteristics associated with guideline nonadherent screening across all groups.

Usability Evaluation of Electronic Health Record System around Clinical Notes Usage–An Ethnographic Study

Background A substantial gap exists between current Electronic Health Record (EHR) usability and potential optimal usability. One of the fundamental reasons for this discrepancy is poor incorporation of a User-Centered Design (UCD) approach during the Graphical User Interface (GUI) development process. Objective To evaluate usability strengths and weaknesses of two widely implemented EHR GUIs for critical clinical notes usage tasks. Methods Twelve Internal Medicine resident physicians interacting with one of the two EHR systems (System-1 at Location-A and System-2 at Location-B) were observed by two usability evaluators employing an ethnographic approach. User comments and observer findings were analyzed for two critical tasks: (1) clinical notes entry and (2) related information-seeking tasks. Data were analyzed from two standpoints: (1) usability references categorized by usability evaluators as positive, negative, or equivocal and (2) usability impact of each feature measured through a 7-point severity rating scale. Findings were also validated by user responses to a post observation questionnaire. Results For clinical notes entry, System-1 surpassed System-2 with more positive (26% vs. 12%) than negative (12% vs. 34%) usability references. Greatest impact features on EHR usability (severity score pertaining to each feature) for clinical notes entry were: autopopulation (6), screen options (5.5), communication (5), copy pasting (4.5), error prevention (4.5), edit ability (4), and dictation and transcription (3.5). Both systems performed equally well on information-seeking tasks and features with greatest impacts on EHR usability were navigation for notes (7) and others (e.g., looking for ancillary data; 5.5). Ethnographic observations were supported by follow-up questionnaire responses. Conclusion This study provides usability-specific insights to inform future, improved, EHR interface that is better aligned with UCD approach.

Usability evaluation of an EHR's clinical notes interface from the perspective of attending and resident Physicians: An exploratory study

Usability gaps between current and future improved Electronic Health Record (EHR) system designs exist due to insufficient incorporation of User-Centered Design (UCD) principles during System Development Life Cycle (SDLC). Usability of a commercial, inpatient EHR clinical notes documentation interface was analyzed from standpoints of two provider groups employing two standardized patient cases. Both objective and subjective data were collected from attending (n = 6) and resident physicians (n = 8) through usability testing employing a mixed method approach. The study results suggested that (i) EHR usability and desirability is influenced by user characteristics, (ii) workloads associated with H&P and progress notes writing are perceived differently between two groups, (iii) repeated task performance improves user efficiency and (iv) user performance is correlated to their subjective system assessments. Understanding usability of clinical documentation interface from perspectives of two different user groups, provides interface designers with an opportunity to develop an EHR system centered on UCD principles.

Usability Evaluation of NLP-PIER: A Clinical Document Search Engine for Researchers.

NLP-PIER (Natural Language Processing – Patient Information Extraction for Research) is a self-service platform with a search engine for clinical researchers to perform natural language processing (NLP) queries using clinical notes. We conducted user-centered testing of NLP-PIER’s usability to inform future design decisions. Quantitative and qualitative data were analyzed. Our findings will be used to improve the usability of NLP-PIER.

Usability testing of two ambulatory EHR navigators

BACKGROUND Despite widespread electronic health record (EHR) adoption, poor EHR system usability continues to be a significant barrier to effective system use for end users. One key to addressing usability problems is to employ user testing and user-centered design. OBJECTIVES To understand if redesigning an EHR-based navigation tool with clinician input improved user performance and satisfaction. METHODS A usability evaluation was conducted to compare two versions of a redesigned ambulatory navigator. Participants completed tasks for five patient cases using the navigators, while employing a think-aloud protocol. The tasks were based on Meaningful Use (MU) requirements. RESULTS The version of navigator did not affect perceived workload, and time to complete tasks was longer in the redesigned navigator. A relatively small portion of navigator content was used to complete the MU-related tasks, though navigation patterns were highly variable across participants for both navigators. Preferences for EHR navigation structures appeared to be individualized. CONCLUSIONS This study demonstrates the importance of EHR usability assessments to evaluate group and individual performance of different interfaces and preferences for each design.