Elsbeth Kalenderian

Detection and characterization of usability problems in structured data entry interfaces in dentistry

BACKGROUND Poor usability is one of the major barriers for optimally using electronic health records (EHRs). Dentists are increasingly adopting EHRs, and are using structured data entry interfaces to enter data such that the data can be easily retrieved and exchanged. Until recently, dentists have lacked a standardized terminology to consistently represent oral health diagnoses. OBJECTIVES In this study we evaluated the usability of a widely used EHR interface that allow the entry of diagnostic terms, using multi-faceted methods to identify problems and work with the vendor to correct them using an iterative design method. METHODS Fieldwork was undertaken at two clinical sites, and dental providers as subjects participated in user testing (n=32), interviews (n=36) and observations (n=24). RESULTS User testing revealed that only 22-41% of users were able to successfully complete a simple task of entering one diagnosis, while no user was able to complete a more complex task. We identified and characterized 24 high-level usability problems reducing efficiency and causing user errors. Interface-related problems included unexpected approaches for displaying diagnosis, lack of visibility, and inconsistent use of UI widgets. Terminology related issues included missing and mis-categorized concepts. Work domain issues involved both absent and superfluous functions. In collaboration with the vendor, each usability problem was prioritized and a timeline set to resolve the concerns. DISCUSSION Mixed methods evaluations identified a number of critical usability issues relating to the user interface, underlying terminology of the work domain. The usability challenges were found to prevent most users from successfully completing the tasks. Our further work we will determine if changes to the interface, terminology and work domain do result in improved usability.

Facial nerve graft repair: suture versus laser-assisted anastomosis

An experimental study was conducted to evaluate the effectiveness of laser-assisted versus conventional suture repair of facial nerve grafts in the rabbit. Electrophysiologic assessment of function revealed no significant difference between the two techniques. Histological differences were apparent, however, early postoperatively from the effects of heat energy on the perimeter of fascicular morphology. At a later stage, this effect had disappeared. The laser technique was also associated with an apparent decreased escape and entrapment of axons.

Are three methods better than one? A comparative assessment of usability evaluation methods in an EHR

OBJECTIVE To comparatively evaluate the effectiveness of three different methods involving end-users for detecting usability problems in an EHR: user testing, semi-structured interviews and surveys. MATERIALS AND METHODS Data were collected at two major urban dental schools from faculty, residents and dental students to assess the usability of a dental EHR for developing a treatment plan. These included user testing (N=32), semi-structured interviews (N=36), and surveys (N=35). RESULTS The three methods together identified a total of 187 usability violations: 54% via user testing, 28% via the semi-structured interview and 18% from the survey method, with modest overlap. These usability problems were classified into 24 problem themes in 3 broad categories. User testing covered the broadest range of themes (83%), followed by the interview (63%) and survey (29%) methods. DISCUSSION Multiple evaluation methods provide a comprehensive approach to identifying EHR usability challenges and specific problems. The three methods were found to be complementary, and thus each can provide unique insights for software enhancement. Interview and survey methods were found not to be sufficient by themselves, but when used in conjunction with the user testing method, they provided a comprehensive evaluation of the EHR. CONCLUSION We recommend using a multi-method approach when testing the usability of health information technology because it provides a more comprehensive picture of usability challenges.

BigMouth: a multi-institutional dental data repository.

Few oral health databases are available for research and the advancement of evidence-based dentistry. In this work we developed a centralized data repository derived from electronic health records (EHRs) at four dental schools participating in the Consortium of Oral Health Research and Informatics. A multi-stakeholder committee developed a data governance framework that encouraged data sharing while allowing control of contributed data. We adopted the i2b2 data warehousing platform and mapped data from each institution to a common reference terminology. We realized that dental EHRs urgently need to adopt common terminologies. While all used the same treatment code set, only three of the four sites used a common diagnostic terminology, and there were wide discrepancies in how medical and dental histories were documented. BigMouth was successfully launched in August 2012 with data on 1.1 million patients, and made available to users at the contributing institutions.

Treatment planning in dentistry using an electronic health record: implications for undergraduate education

OBJECTIVE Treatment planning, an essential component of clinical practice, has received little attention in the dental literature and there appears to be no consistent format being followed in the teaching and development of treatment plans within dental school curricula. No investigation, to our knowledge, has been carried out to explore the subject of treatment planning since the advent of electronic health record (EHR) use in dentistry. It is therefore important to examine the topic of treatment planning in the context of EHRs. METHODS This paper reports on how 25 predoctoral dental students from two U.S. schools performed when asked to complete diagnosis and treatment planning exercises for two clinical scenarios in an EHR. Three calibrated clinical teaching faculty scored diagnosis entry, diagnosis-treatment (procedure) pairing, and sequencing of treatment according to criteria taught in their curriculum. Scores were then converted to percent correct and reported as means (with standard deviations). RESULTS Overall, the participants earned 48.2% of the possible points. Participants at School 2 earned a mean of 54.3% compared with participants at School 1, who earned 41.9%. Students fared better selecting the appropriate treatment (59.8%) compared with choosing the correct diagnoses (41.9%) but performed least favorably when organizing the sequence of their treatment plans (41.7%). CONCLUSION Our results highlight the need to improve the current process by which treatment planning is taught and also to consider the impact of technology on the fundamental skills of diagnosis and treatment planning within the modern educational setting.

The dangers of dental devices as reported in the Food and Drug Administration Manufacturer and User Facility Device Experience Database.

BACKGROUND The authors conducted a study to determine the frequency and type of adverse events (AEs) associated with dental devices reported to the Food and Drug Administration Manufacturer and User Facility Device Experience (MAUDE) database. METHODS The authors downloaded and reviewed the dental device-related AEs reported to MAUDE from January 1, 1996, through December 31,2011. RESULTS MAUDE received a total of 1,978,056 reports between January 1, 1996, and December 31, 2011. Among these reports, 28,046 (1.4%) AE reports were associated with dental devices. Within the dental AE reports that had event type information, 17,261 reported injuries, 7,777 reported device malfunctions, and 66 reported deaths. Among the 66 entries classified as death reports, 52 reported a death in the description; the remaining were either misclassified or lacked sufficient information in the report to determine whether a death had occurred. Of the dental device-associated AEs, 53.5% pertained to endosseous implants. CONCLUSIONS A plethora of devices are used in dental care. To achieve Element 1 of Agency for Healthcare Research and Qualitys Patient Safety Initiative, clinicians and researchers must be able to monitor the safety of dental devices. Although MAUDE was identified by the authors as essentially the sole source of this valuable information on adverse events, their investigations led them to conclude that MAUDE had substantial limitations that prevent it from being the broad-based patient safety sentinel the profession requires. PRACTICAL IMPLICATIONS As potential contributors to MAUDE, dental care teams play a key role in improving the professions access to information about the safety of dental devices.

Surgical repair of defects in the rabbit temporomandibular joint disc: A comparison of various techniques

A study was designed to evaluate the effects of different types of repair (suture vs. graft), donor graft tissues (dermis vs. fascia), and methods of graft fixation (sutured vs. laser-assisted) on excisional defects of the rabbit TMJ disc made anterior to the retrodiscal tissue. Groups created to compare the different variables were assessed at 30 and 90 days postoperatively by macroscopic and histologic methods, microangiography, and mechanical strength testing. Without repair, all excisional defects of the disc failed to heal. Suturing of the reapproximated margins did not appear to promote healing or provide much strength beyond the properties of the suture material. Dermal grafts, fixed by suturing, induced healing by becoming incorporated into the disc, restoring discal continuity, and approximating normal disc strength. The use of laser-assisted tissue welding for dermal graft fixation was found to be inadequate and was subsequently abandoned. Fascial grafts could be fixed into position by both suture and laser-assistance, but consistently failed to exhibit successful transplantation and promote healing at the disc site.

Dental clinical research: an illustration of the value of standardized diagnostic terms

Abstract Objective Secondary data are a significant resource for in‐depth epidemiologic and public health research. It also allows for effective quality control and clinical outcomes measurement. To illustrate the value of structured diagnostic entry, a use case was developed to quantify adherence to current practice guidelines for managing chronic moderate periodontitis (CMP). Methods Six dental schools using the same electronic health record (EHR) contribute data to a dental data repository (BigMouth) based on the i2b2 data‐warehousing platform. Participating institutions are able to query across the full repository without being able to back trace specific data to its originating institution. At each of the three sites whose data are included in this analysis, the Dental Diagnostic System (DDS) terminology was used to document diagnoses in the clinics. We ran multiple queries against this multi‐institutional database, and the output was validated by manually reviewing a subset of patient charts. Results Over the period under study, 1,866 patients were diagnosed with CMP. Of these, 15 percent received only periodontal prophylaxis treatment, 20 percent received only periodontal maintenance treatment, and only 41 percent received periodontal maintenance treatment in combination with other AAP guideline treatments. Conclusions Our results showed that most patients with CMP were not treated according to the AAP guidelines. On the basis of this use case, we conclude that the availability and habitual use of a structured diagnosis in an EHR allow for the aggregation and secondary analyses of clinical data to support downstream analyses for quality improvement and epidemiological assessments.

When Advocacy Obscures Accuracy Online: Digital Pandemics of Public Health Misinformation Through an Antifluoride Case Study

OBJECTIVES In an antifluoridation case study, we explored digital pandemics and the social spread of scientifically inaccurate health information across the Web, and we considered the potential health effects. METHODS Using the social networking site Facebook and the open source applications Netvizz and Gephi, we analyzed the connectedness of antifluoride networks as a measure of social influence, the social diffusion of information based on conversations about a sample scientific publication as a measure of spread, and the engagement and sentiment about the publication as a measure of attitudes and behaviors. RESULTS Our study sample was significantly more connected than was the social networking site overall (P<.001). Social diffusion was evident; users were forced to navigate multiple pages or never reached the sample publication being discussed 60% and 12% of the time, respectively. Users had a 1 in 2 chance of encountering negative and nonempirical content about fluoride unrelated to the sample publication. CONCLUSIONS Network sociology may be as influential as the information content and scientific validity of a particular health topic discussed using social media. Public health must employ social strategies for improved communication management.

The importance of using diagnostic codes

To the Editor: We are a group of academic dentists who have developed a comprehensive dental diagnostic terminology and codes and implemented them. Our work has been catalyzed by the Consortium for Oral Health– Related Informatics (COHRI), a collaboration of 20 dental academic centers. The primary objective of COHRI is to share data for research and quality improvement purposes. One of the work products of this group was to create dental diagnostic terminology and codes. These terms and codes incorporated all of the existing dental diagnostic concepts, including the oral health concepts encompassed by the International Classification of Diseases (ICD,) 9th and 10th editions. In addition, the diagnostic terms and codes were further developed to allow for finer granularity of diagnoses, a limitation of the ICD system. These diagnostic terms and codes were developed through a consensus process within COHRI and now include 13 major classifications of disease and 1,321 terms and codes. These diagnostic terms and codes will continue to evolve with the profession, as we have instituted a biannual update process. We are currently funded by the National Institute of Dental and Craniofacial Research to enhance and implement diagnostic terminology and codes in an electronic health record. One objective of our grant is to increase the utilization of diagnostic terms and codes by lowering the barrier to documenting diagnoses in a standardized fashion. As part of our effort, we are in the process of creating a validated mapping of diagnostic terminology to the currently accepted dental procedure codes, thereby reinforcing the link between diagnosis and treatment. We agree with Dr Miller’s description that as part of dental education, the student has to learn to make a diagnosis before making the treatment plan, just as the dental practitioner needs to determine a diagnosis and then make a treatment plan. These are core values of why we implemented the diagnostic terminology and codes within a treatment plan module in the electronic health record (Axium; Exan Corp., Vancouver, Canada). The treatment plan module guides the student and practitioner through the steps of collecting data, identification of problems, determination of one or more diagnoses and then the construction of different options for a treatment plan. Dr Miller mentioned several benefits to the standardized documentation of dental diagnoses, i.e., the use of a common language and standard criteria, the requirement of evidence for use of diagnostic codes, the ability to track relationships between diagnoses and treatment, and qualitative and quantitative outcome measures that contribute to improvements in public health. We agree wholeheartedly with Dr Miller’s observations and would like to add that standardized diagnostic terms also enhance patient communication and the sharing of data across practices. Attempts at standardizing diagnostic codes in the past have not gained traction owing in part to fragmentation of efforts to create coding systems as well as the absence of meaningful incentives, as pointed out by Dr Miller. Through COHRI, it has been possible to ensure widespread acceptance of a common diagnostic terminology. The Diagnostic Terminology and Codes (2011) is protected under copyright to ensure their integrity and is available through a license.