Oluwabunmi Tokede

Detection and characterization of usability problems in structured data entry interfaces in dentistry

BACKGROUND Poor usability is one of the major barriers for optimally using electronic health records (EHRs). Dentists are increasingly adopting EHRs, and are using structured data entry interfaces to enter data such that the data can be easily retrieved and exchanged. Until recently, dentists have lacked a standardized terminology to consistently represent oral health diagnoses. OBJECTIVES In this study we evaluated the usability of a widely used EHR interface that allow the entry of diagnostic terms, using multi-faceted methods to identify problems and work with the vendor to correct them using an iterative design method. METHODS Fieldwork was undertaken at two clinical sites, and dental providers as subjects participated in user testing (n=32), interviews (n=36) and observations (n=24). RESULTS User testing revealed that only 22-41% of users were able to successfully complete a simple task of entering one diagnosis, while no user was able to complete a more complex task. We identified and characterized 24 high-level usability problems reducing efficiency and causing user errors. Interface-related problems included unexpected approaches for displaying diagnosis, lack of visibility, and inconsistent use of UI widgets. Terminology related issues included missing and mis-categorized concepts. Work domain issues involved both absent and superfluous functions. In collaboration with the vendor, each usability problem was prioritized and a timeline set to resolve the concerns. DISCUSSION Mixed methods evaluations identified a number of critical usability issues relating to the user interface, underlying terminology of the work domain. The usability challenges were found to prevent most users from successfully completing the tasks. Our further work we will determine if changes to the interface, terminology and work domain do result in improved usability.

Are three methods better than one? A comparative assessment of usability evaluation methods in an EHR

OBJECTIVE To comparatively evaluate the effectiveness of three different methods involving end-users for detecting usability problems in an EHR: user testing, semi-structured interviews and surveys. MATERIALS AND METHODS Data were collected at two major urban dental schools from faculty, residents and dental students to assess the usability of a dental EHR for developing a treatment plan. These included user testing (N=32), semi-structured interviews (N=36), and surveys (N=35). RESULTS The three methods together identified a total of 187 usability violations: 54% via user testing, 28% via the semi-structured interview and 18% from the survey method, with modest overlap. These usability problems were classified into 24 problem themes in 3 broad categories. User testing covered the broadest range of themes (83%), followed by the interview (63%) and survey (29%) methods. DISCUSSION Multiple evaluation methods provide a comprehensive approach to identifying EHR usability challenges and specific problems. The three methods were found to be complementary, and thus each can provide unique insights for software enhancement. Interview and survey methods were found not to be sufficient by themselves, but when used in conjunction with the user testing method, they provided a comprehensive evaluation of the EHR. CONCLUSION We recommend using a multi-method approach when testing the usability of health information technology because it provides a more comprehensive picture of usability challenges.

Treatment planning in dentistry using an electronic health record: implications for undergraduate education

OBJECTIVE Treatment planning, an essential component of clinical practice, has received little attention in the dental literature and there appears to be no consistent format being followed in the teaching and development of treatment plans within dental school curricula. No investigation, to our knowledge, has been carried out to explore the subject of treatment planning since the advent of electronic health record (EHR) use in dentistry. It is therefore important to examine the topic of treatment planning in the context of EHRs. METHODS This paper reports on how 25 predoctoral dental students from two U.S. schools performed when asked to complete diagnosis and treatment planning exercises for two clinical scenarios in an EHR. Three calibrated clinical teaching faculty scored diagnosis entry, diagnosis-treatment (procedure) pairing, and sequencing of treatment according to criteria taught in their curriculum. Scores were then converted to percent correct and reported as means (with standard deviations). RESULTS Overall, the participants earned 48.2% of the possible points. Participants at School 2 earned a mean of 54.3% compared with participants at School 1, who earned 41.9%. Students fared better selecting the appropriate treatment (59.8%) compared with choosing the correct diagnoses (41.9%) but performed least favorably when organizing the sequence of their treatment plans (41.7%). CONCLUSION Our results highlight the need to improve the current process by which treatment planning is taught and also to consider the impact of technology on the fundamental skills of diagnosis and treatment planning within the modern educational setting.

Dental Providers’ Perspectives on Diagnosis-Driven Dentistry: Strategies to Enhance Adoption of Dental Diagnostic Terminology

The routine use of standardized diagnostic terminologies (DxTMs) in dentistry has long been the subject of academic debate. This paper discusses the strategies suggested by a group of dental stakeholders to enhance the uptake of DxTMs. Through unstructured interviewing at the ‘Toward a Diagnosis-Driven Profession’ National Conference held on 19 March 2016 in Los Angeles, CA, USA participants were asked how enthusiastic they were about implementing and consistently using DxTMs at their work. They also brainstormed on strategies to improve the widespread use of DxTMs. Their responses are summarized by recursive abstraction and presented in themes. Conference participants were very enthusiastic about using a DxTM in their place of work. Participants enumerated several strategies to make DxTMs more appealing including: the use of mandates, a value proposition for providers, communication and education, and integration with EHRs and existing systems. All groups across the dental healthcare delivery spectrum will need to work together for the success of the widespread and consistent use of DxTMs. Understanding the provider perspective is however the most critical step in achieving this goal, as they are the group who will ultimately be saddled with the critical task of ensuring DxTM use at the point of care.

Feasibility of Electronic Health Record–Based Triggers in Detecting Dental Adverse Events

BACKGROUND We can now quantify and characterize the harm patients suffer in the dental chair by mining data from electronic health records (EHRs). Most dental institutions currently deploy a random audit of charts using locally developed definitions to identify such patient safety incidents. Instead, selection of patient charts using triggers and assessment through calibrated reviewers may more efficiently identify dental adverse events (AEs). OBJECTIVE Our goal was to develop and test EHR-based triggers at four academic institutions and find dental AEs, defined as moderate or severe physical harm due to dental treatment. METHODS We used an iterative and consensus-based process to develop 11 EHR-based triggers to identify dental AEs. Two dental experts at each institution independently reviewed a sample of triggered charts using a common AE definition and classification system. An expert panel provided a second level of review to confirm AEs identified by sites reviewers. We calculated the performance of each trigger and identified strategies for improvement. RESULTS A total of 100 AEs were identified by 10 of the 11 triggers. In 57% of the cases, pain was the most common AE identified, followed by infection and hard tissue damage. Positive predictive value (PPV) for the triggers ranged from 0 to 0.29. The best performing triggers were those developed to identify infections (PPV = 0.29), allergies (PPV = 0.23), failed implants (PPV = 0.21), and nerve injuries (PPV = 0.19). Most AEs (90%) were categorized as temporary moderate-to-severe harm (E2) and the remainder as permanent moderate-to-severe harm (G2). CONCLUSION EHR-based triggers are a promising approach to unearth AEs among dental patients compared with a manual audit of random charts. Data in dental EHRs appear to be sufficiently structured to allow the use of triggers. Pain was the most common AE type followed by infection and hard tissue damage.