Po-Yin Yen

Review of health information technology usability study methodologies

Usability factors are a major obstacle to health information technology (IT) adoption. The purpose of this paper is to review and categorize health IT usability study methods and to provide practical guidance on health IT usability evaluation. 2025 references were initially retrieved from the Medline database from 2003 to 2009 that evaluated health IT used by clinicians. Titles and abstracts were first reviewed for inclusion. Full-text articles were then examined to identify final eligibility studies. 629 studies were categorized into the five stages of an integrated usability specification and evaluation framework that was based on a usability model and the system development life cycle (SDLC)-associated stages of evaluation. Theoretical and methodological aspects of 319 studies were extracted in greater detail and studies that focused on system validation (SDLC stage 2) were not assessed further. The number of studies by stage was: stage 1, task-based or user–task interaction, n=42; stage 2, system–task interaction, n=310; stage 3, user–task–system interaction, n=69; stage 4, user–task–system–environment interaction, n=54; and stage 5, user–task–system–environment interaction in routine use, n=199. The studies applied a variety of quantitative and qualitative approaches. Methodological issues included lack of theoretical framework/model, lack of details regarding qualitative study approaches, single evaluation focus, environmental factors not evaluated in the early stages, and guideline adherence as the primary outcome for decision support system evaluations. Based on the findings, a three-level stratified view of health IT usability evaluation is proposed and methodological guidance is offered based upon the type of interaction that is of primary interest in the evaluation.

Computational challenges and human factors influencing the design and use of clinical research participant eligibility pre-screening tools

BackgroundClinical trials are the primary mechanism for advancing clinical care and evidenced-based practice, yet challenges with the recruitment of participants for such trials are widely recognized as a major barrier to these types of studies. Data warehouses (DW) store large amounts of heterogenous clinical data that can be used to enhance recruitment practices, but multiple challenges exist when using a data warehouse for such activities, due to the manner of collection, management, integration, analysis, and dissemination of the data. A critical step in leveraging the DW for recruitment purposes is being able to match trial eligibility criteria to discrete and semi-structured data types in the data warehouse, though trial eligibility criteria tend to be written without concern for their computability. We present the multi-modal evaluation of a web-based tool that can be used for pre-screening patients for clinical trial eligibility and assess the ability of this tool to be practically used for clinical research pre-screening and recruitment.MethodsThe study used a validation study, usability testing, and a heuristic evaluation to evaluate and characterize the operational characteristics of the software as well as human factors affecting its use.ResultsClinical trials from the Division of Cardiology and the Department of Family Medicine were used for this multi-modal evaluation, which included a validation study, usability study, and a heuristic evaluation. From the results of the validation study, the software demonstrated a positive predictive value (PPV) of 54.12% and 0.7%, respectively, and a negative predictive value (NPV) of 73.3% and 87.5%, respectively, for two types of clinical trials. Heuristic principles concerning error prevention and documentation were characterized as the major usability issues during the heuristic evaluation.ConclusionsThis software is intended to provide an initial list of eligible patients to a clinical study coordinators, which provides a starting point for further eligibility screening by the coordinator. Because this software has a high “rule in” ability, meaning that it is able to remove patients who are not eligible for the study, the use of an automated tool built to leverage an existing enterprise DW can be beneficial to determining eligibility and facilitating clinical trial recruitment through pre-screening. While the results of this study are promising, further refinement and study of this and related approaches to automated eligibility screening, including comparison to other approaches and stakeholder perceptions, are needed and future studies are planned to address these needs.

Identifying Consumer’s Needs of Health Information Technology through an Innovative Participatory Design Approach among English- and Spanish-speaking Urban Older Adults

OBJECTIVES We describe an innovative community-centered participatory design approach, Consumer-centered Participatory Design (C2PD), and the results of applying C2PD to design and develop a web-based fall prevention system. METHODS We conducted focus groups and design sessions with English- and Spanish-speaking community-dwelling older adults. Focus group data were summarized and used to inform the context of the design sessions. Descriptive content analysis methods were used to develop categorical descriptions of design session informants needs related to information technology. RESULTS The C2PD approach enabled the assessment and identification of informants needs of health information technology (HIT) that informed the development of a falls prevention system. We learned that our informants needed a system that provides variation in functions/content; differentiates between actionable/non-actionable information/structures; and contains sensory cues that support wide-ranging and complex tasks in a varied, simple, and clear interface to facilitate self-management. CONCLUSIONS The C2PD approach provides community-based organizations, academic researchers, and commercial entities with a systematic theoretically informed approach to develop HIT innovations. Our community-centered participatory design approach focuses on consumers technology needs while taking into account core public health functions.

Using ontology-based semantic similarity to facilitate the article screening process for systematic reviews

OBJECTIVE Systematic Reviews (SRs) are utilized to summarize evidence from high quality studies and are considered the preferred source of evidence-based practice (EBP). However, conducting SRs can be time and labor intensive due to the high cost of article screening. In previous studies, we demonstrated utilizing established (lexical) article relationships to facilitate the identification of relevant articles in an efficient and effective manner. Here we propose to enhance article relationships with background semantic knowledge derived from Unified Medical Language System (UMLS) concepts and ontologies. METHODS We developed a pipelined semantic concepts representation process to represent articles from an SR into an optimized and enriched semantic space of UMLS concepts. Throughout the process, we leveraged concepts and concept relations encoded in biomedical ontologies (SNOMED-CT and MeSH) within the UMLS framework to prompt concept features of each article. Article relationships (similarities) were established and represented as a semantic article network, which was readily applied to assist with the article screening process. We incorporated the concept of active learning to simulate an interactive article recommendation process, and evaluated the performance on 15 completed SRs. We used work saved over sampling at 95% recall (WSS95) as the performance measure. RESULTS We compared the WSS95 performance of our ontology-based semantic approach to existing lexical feature approaches and corpus-based semantic approaches, and found that we had better WSS95 in most SRs. We also had the highest average WSS95 of 43.81% and the highest total WSS95 of 657.18%. CONCLUSION We demonstrated using ontology-based semantics to facilitate the identification of relevant articles for SRs. Effective concepts and concept relations derived from UMLS ontologies can be utilized to establish article semantic relationships. Our approach provided a promising performance and can easily apply to any SR topics in the biomedical domain with generalizability.

Nursing Informatics Competency Assessment for the Nurse Leader: The Delphi Study

OBJECTIVE The aim of this study was to identify nursing informatics competencies perceived as relevant and required by nurse leaders. BACKGROUND To participate as a full partner in healthcare leadership among rapidly advancing health information technologies (HITs), nurse leaders must attain knowledge of informatics competencies related to their clinical leadership roles and responsibilities. Despite this increased need to engage in HIT-related decision making, a gap remains in validated informatics competencies specific to the needs of nurse leaders. METHODS An environmental scan and 3-round survey using Delphi methods used with nurse leaders for competency identification were used. RESULTS Between 26 and 41 participants responded to each Delphi round. Most nurse leaders acquired HIT knowledge through on-the-job training. We identified 74 competencies from an initial list of 108 competencies. CONCLUSION This work can advance nursing practice to move beyond “on-the-job informatics training” to a more competency-based model of nursing informatics education and practice.

Health Information Technology (HIT) Adaptation: Refocusing on the Journey to Successful HIT Implementation

In past years, policies and regulations required hospitals to implement advanced capabilities of certified electronic health records (EHRs) in order to receive financial incentives. This has led to accelerated implementation of health information technologies (HIT) in health care settings. However, measures commonly used to evaluate the success of HIT implementation, such as HIT adoption, technology acceptance, and clinical quality, fail to account for complex sociotechnical variability across contexts and the different trajectories within organizations because of different implementation plans and timelines. We propose a new focus, HIT adaptation, to illuminate factors that facilitate or hinder the connection between use of the EHR and improved quality of care as well as to explore the trajectory of changes in the HIT implementation journey as it is impacted by frequent system upgrades and optimizations. Future research should develop instruments to evaluate the progress of HIT adaptation in both its longitudinal design and its focus on adaptation progress rather than on one cross-sectional outcome, allowing for more generalizability and knowledge transfer.

Nurses’ Experience With Health Information Technology: Longitudinal Qualitative Study

Background Nurses are the largest group of health information technology (HIT) users. As such, nurses’ adaptations are critical for HIT implementation success. However, longitudinal approaches to understanding nurses’ perceptions of HIT remain underexplored. Previous studies of nurses’ perceptions demonstrate that the progress and timing for acceptance of and adaptation to HIT varies. Objective This study aimed to explore nurses’ experience regarding implementation of HIT over time. Methods A phenomenological approach was used for this longitudinal qualitative study to explore nurses’ perceptions of HIT implementation over time, focusing on three time points (rounds) at 3, 9, and 18 months after implementation of electronic health records and bar code medication administration. The purposive sample was comprised of clinical nurses who worked on a medical-surgical unit in an academic center. Results Major findings were categorized into 7 main themes with 54 subthemes. Nurses reported personal-level and organizational-level factors that facilitated HIT adaptation. We also generated network graphs to illustrate the occurrence of themes. Thematic interconnectivity differed due to nurses’ concerns and satisfaction at different time points. Equipment and workflow were the most frequent themes across all three rounds. Nurses were the most dissatisfied approximately 9 months after HIT implementation. Eighteen months after HIT implementation, nurses’ perceptions appeared more balanced. Conclusions It is recommended that organizations invest in equipment (ie, wireless barcode scanners), refine policies to reflect nursing practice, and improve systems to focus on patient safety. Future research is necessary to confirm patterns of nurses’ adaptation to HIT in other samples.

Evaluation of the timing and coordination of prandial insulin administration in the hospital

AIMS The objective of this study was to examine the relationship between measures of coordinated insulin delivery and capillary blood glucose (CBG) levels among hospitalized patients and to assess nurse perceptions of insulin administration. METHODS Hospitalized patients (n=451) receiving rapid acting insulin analog (RAIA) using carbohydrate counting were retrospectively analyzed. Nurses (n=35) were asked to complete an 18-item anonymous survey assessing perception of RAIA dosing. RESULTS The median time from breakfast CBG to RAIA dose was 93 (IQR 57-138) min. There was no association between timeliness measures and mean CBG at lunch or dinner. Hypoglycemia was rare (N=2). More than half (54%) of nurses were confident all of the time in determining the correct dose of RAIA, though none were confident in administering it on time. The majority of nurses perceived an electronic dosing calculator and a patient reminder to notify the nurse at the end of the meal favorably. CONCLUSIONS The data demonstrate suboptimal coordination of CBG monitoring and insulin doses using a flexible meal insulin dosing strategy, though there was minimal impact on glycemic control. Nurses reported high confidence in the ability to calculate the correct insulin dose but not in the ability to administer it on time.