Brent Hill

Evaluation of a pictograph enhancement system for patient instruction: a recall study

OBJECTIVE We developed a novel computer application called Glyph that automatically converts text to sets of illustrations using natural language processing and computer graphics techniques to provide high quality pictographs for health communication. In this study, we evaluated the ability of the Glyph system to illustrate a set of actual patient instructions, and tested patient recall of the original and Glyph illustrated instructions. METHODS We used Glyph to illustrate 49 patient instructions representing 10 different discharge templates from the University of Utah Cardiology Service. 84 participants were recruited through convenience sampling. To test the recall of illustrated versus non-illustrated instructions, participants were asked to review and then recall a set questionnaires that contained five pictograph-enhanced and five non-pictograph-enhanced items. RESULTS The mean score without pictographs was 0.47 (SD 0.23), or 47% recall. With pictographs, this mean score increased to 0.52 (SD 0.22), or 52% recall. In a multivariable mixed effects linear regression model, this 0.05 mean increase was statistically significant (95% CI 0.03 to 0.06, p<0.001). DISCUSSION In our study, the presence of Glyph pictographs improved discharge instruction recall (p<0.001). Education, age, and English as first language were associated with better instruction recall and transcription. CONCLUSIONS Automated illustration is a novel approach to improve the comprehension and recall of discharge instructions. Our results showed a statistically significant in recall with automated illustrations. Subjects with no-colleague education and younger subjects appeared to benefit more from the illustrations than others.

A picture's meaning: The design and evaluation of pictographs illustrating patient discharge instructions

Introduction: Several studies have reported that illustrations may improve patient comprehension of health communications. However, developing effective illustrations remains a challenge. To improve discharge instructions with illustrations, we developed a computer application (Glyph) that automatically converts text to pictures using natural language processing and computer graphics techniques. Methods: We evaluated a set of pictographs created for cardiovascular discharge instructions through recognition testing, and examined illustration approaches work for specific types of medical concepts and subpopulations. We tested a large set of illustrations (n = 488) on a diverse population of subjects (n = 150) for overall recognition rates by demographic groups, representation strategies, and semantic types, as well as the effectiveness of representation strategies in relationship to recognition. Results: A majority (63%) was determined to be recognizable based on recognition scores. Results were confirmed through both descriptive statistics and a multivariable regression model analyses. Predictors of successful recognition were participant factors such as: white race, male gender, college education, and native English language. Pictograph features that predicted success included: direct representation strategy, proper level of image detail, and use in familiar contexts. Conclusion: While post-discharge care and coordination involves many complexities, the design and use of patient-centered discharge instructions may substantially impact.

An Evolving Ecosystem for Natural Language Processing in Department of Veterans Affairs

In an ideal clinical Natural Language Processing (NLP) ecosystem, researchers and developers would be able to collaborate with others, undertake validation of NLP systems, components, and related resources, and disseminate them. We captured requirements and formative evaluation data from the Veterans Affairs (VA) Clinical NLP Ecosystem stakeholders using semi-structured interviews and meeting discussions. We developed a coding rubric to code interviews. We assessed inter-coder reliability using percent agreement and the kappa statistic. We undertook 15 interviews and held two workshop discussions. The main areas of requirements related to; design and functionality, resources, and information. Stakeholders also confirmed the vision of the second generation of the Ecosystem and recommendations included; adding mechanisms to better understand terms, measuring collaboration to demonstrate value, and datasets/tools to navigate spelling errors with consumer language, among others. Stakeholders also recommended capability to: communicate with developers working on the next version of the VA electronic health record (VistA Evolution), provide a mechanism to automatically monitor download of tools and to automatically provide a summary of the downloads to Ecosystem contributors and funders. After three rounds of coding and discussion, we determined the percent agreement of two coders to be 97.2% and the kappa to be 0.7851. The vision of the VA Clinical NLP Ecosystem met stakeholder needs. Interviews and discussion provided key requirements that inform the design of the VA Clinical NLP Ecosystem.