Frédéric Ehrler

Data-poor categorization and passage retrieval for Gene Ontology Annotation in Swiss-Prot

BackgroundIn the context of the BioCreative competition, where training data were very sparse, we investigated two complementary tasks: 1) given a Swiss-Prot triplet, containing a protein, a GO (Gene Ontology) term and a relevant article, extraction of a short passage that justifies the GO category assignement; 2) given a Swiss-Prot pair, containing a protein and a relevant article, automatic assignement of a set of categories.MethodsSentence is the basic retrieval unit. Our classifier computes a distance between each sentence and the GO category provided with the Swiss-Prot entry. The Text Categorizer computes a distance between each GO term and the text of the article. Evaluations are reported both based on annotator judgements as established by the competition and based on mean average precision measures computed using a curated sample of Swiss-Prot.ResultsOur system achieved the best recall and precision combination both for passage retrieval and text categorization as evaluated by official evaluators. However, text categorization results were far below those in other data-poor text categorization experiments The top proposed term is relevant in less that 20% of cases, while categorization with other biomedical controlled vocabulary, such as the Medical Subject Headings, we achieved more than 90% precision. We also observe that the scoring methods used in our experiments, based on the retrieval status value of our engines, exhibits effective confidence estimation capabilities.ConclusionFrom a comparative perspective, the combination of retrieval and natural language processing methods we designed, achieved very competitive performances. Largely data-independent, our systems were no less effective that data-intensive approaches. These results suggests that the overall strategy could benefit a large class of information extraction tasks, especially when training data are missing. However, from a user perspective, results were disappointing. Further investigations are needed to design applicable end-user text mining tools for biologists.

Gene Ontology density estimation and discourse analysis for automatic GeneRiF extraction

BackgroundThis paper describes and evaluates a sentence selection engine that extracts a GeneRiF (Gene Reference into Functions) as defined in ENTREZ-Gene based on a MEDLINE record. Inputs for this task include both a gene and a pointer to a MEDLINE reference. In the suggested approach we merge two independent sentence extraction strategies. The first proposed strategy (LASt) uses argumentative features, inspired by discourse-analysis models. The second extraction scheme (GOEx) uses an automatic text categorizer to estimate the density of Gene Ontology categories in every sentence; thus providing a full ranking of all possible candidate GeneRiFs. A combination of the two approaches is proposed, which also aims at reducing the size of the selected segment by filtering out non-content bearing rhetorical phrases.ResultsBased on the TREC-2003 Genomics collection for GeneRiF identification, the LASt extraction strategy is already competitive (52.78%). When used in a combined approach, the extraction task clearly shows improvement, achieving a Dice score of over 57% (+10%).ConclusionsArgumentative representation levels and conceptual density estimation using Gene Ontology contents appear complementary for functional annotation in proteomics.

Assessing the Usability of Six Data Entry Mobile Interfaces for Caregivers: A Randomized Trial

Background There is an increased demand in hospitals for tools, such as dedicated mobile device apps, that enable the recording of clinical information in an electronic format at the patient’s bedside. Although the human-machine interface design on mobile devices strongly influences the accuracy and effectiveness of data recording, there is still a lack of evidence as to which interface design offers the best guarantee for ease of use and quality of recording. Therefore, interfaces need to be assessed both for usability and reliability because recording errors can seriously impact the overall level of quality of the data and affect the care provided. Objective In this randomized crossover trial, we formally compared 6 handheld device interfaces for both speed of data entry and accuracy of recorded information. Three types of numerical data commonly recorded at the patient’s bedside were used to evaluate the interfaces. Methods In total, 150 health care professionals from the University Hospitals of Geneva volunteered to record a series of randomly generated data on each of the 6 interfaces provided on a smartphone. The interfaces were presented in a randomized order as part of fully automated data entry scenarios. During the data entry process, accuracy and effectiveness were automatically recorded by the software. Results Various types of errors occurred, which ranged from 0.7% for the most reliable design to 18.5% for the least reliable one. The length of time needed for data recording ranged from 2.81 sec to 14.68 sec, depending on the interface. The numeric keyboard interface delivered the best performance for pulse data entry with a mean time of 3.08 sec (SD 0.06) and an accuracy of 99.3%. Conclusions Our study highlights the critical impact the choice of an interface can have on the quality of recorded data. Selecting an interface should be driven less by the needs of specific end-user groups or the necessity to facilitate the developer’s task (eg, by opting for default solutions provided by commercial platforms) than by the level of speed and accuracy an interface can provide for recording information. An important effort must be made to properly validate mobile device interfaces intended for use in the clinical setting. In this regard, our study identified the numeric keyboard, among the proposed designs, as the most accurate interface for entering specific numerical values. This is an important step toward providing clearer guidelines on which interface to choose for the appropriate use of handheld device interfaces in the health care setting.

Challenges in the Implementation of a Mobile Application in Clinical Practice: Case Study in the Context of an Application that Manages the Daily Interventions of Nurses.

Background Working in a clinical environment requires unfettered mobility. This is especially true for nurses who are always on the move providing patients’ care in different locations. Since the introduction of clinical information systems in hospitals, this mobility has often been considered hampered by interactions with computers. The popularity of personal mobile assistants such as smartphones makes it possible to gain easy access to clinical data anywhere. Objective To identify the challenges involved in the deployment of clinical applications on handheld devices and to share our solutions to these problems. Methods A team of experts underwent an iterative development process of a mobile application prototype that aimed to improve the mobility of nurses during their daily clinical activities. Through the process, challenges inherent to mobile platforms have emerged. These issues have been classified, focusing on factors related to ensuring information safety and quality, as well as pleasant and efficient user experiences. Results The team identified five main challenges related to the deployment of clinical mobile applications and presents solutions to overcome each of them: (1) Financial: Equipping every care giver with a new mobile device requires substantial investment that can be lowered if users use their personal device instead, (2) Hardware: The constraints inherent to the clinical environment made us choose the mobile device with the best tradeoff between size and portability, (3) Communication: the connection of the mobile application with any existing clinical information systems (CIS) is insured by a bridge formatting the information appropriately, (4) Security: In order to guarantee the confidentiality and safety of the data, the amount of data stored on the device is minimized, and (5) User interface: The design of our user interface relied on homogeneity, hierarchy, and indexicality principles to prevent an increase in data acquisition errors. Conclusions The introduction of nomadic computing often raises enthusiastic reactions from users, but several challenges due to specific constraints of mobile platforms must be overcome. The ease of development of mobile applications and their rapid spread should not overshadow the real challenges of clinical applications and the potential threats for patient safety and the liability of people and organizations using them. For example, careful attention must be given to the overall architecture of the system and to user interfaces. If these precautions are not taken, it can easily lead to unexpected failures such as an increased number of input errors, loss of data, or decreased efficiency.

Adherence to AHA Guidelines When Adapted for Augmented Reality Glasses for Assisted Pediatric Cardiopulmonary Resuscitation: A Randomized Controlled Trial

Background The American Heart Association (AHA) guidelines for cardiopulmonary resuscitation (CPR) are nowadays recognized as the world’s most authoritative resuscitation guidelines. Adherence to these guidelines optimizes the management of critically ill patients and increases their chances of survival after cardiac arrest. Despite their availability, suboptimal quality of CPR is still common. Currently, the median hospital survival rate after pediatric in-hospital cardiac arrest is 36%, whereas it falls below 10% for out-of-hospital cardiac arrest. Among emerging information technologies and devices able to support caregivers during resuscitation and increase adherence to AHA guidelines, augmented reality (AR) glasses have not yet been assessed. In order to assess their potential, we adapted AHA Pediatric Advanced Life Support (PALS) guidelines for AR glasses. Objective The study aimed to determine whether adapting AHA guidelines for AR glasses increased adherence by reducing deviation and time to initiation of critical life-saving maneuvers during pediatric CPR when compared with the use of PALS pocket reference cards. Methods We conducted a randomized controlled trial with two parallel groups of voluntary pediatric residents, comparing AR glasses to PALS pocket reference cards during a simulation-based pediatric cardiac arrest scenario—pulseless ventricular tachycardia (pVT). The primary outcome was the elapsed time in seconds in each allocation group, from onset of pVT to the first defibrillation attempt. Secondary outcomes were time elapsed to (1) initiation of chest compression, (2) subsequent defibrillation attempts, and (3) administration of drugs, as well as the time intervals between defibrillation attempts and drug doses, shock doses, and number of shocks. All these outcomes were assessed for deviation from AHA guidelines. Results Twenty residents were randomized into 2 groups. Time to first defibrillation attempt (mean: 146 s) and adherence to AHA guidelines in terms of time to other critical resuscitation endpoints and drug dose delivery were not improved using AR glasses. However, errors and deviations were significantly reduced in terms of defibrillation doses when compared with the use of the PALS pocket reference cards. In a total of 40 defibrillation attempts, residents not wearing AR glasses used wrong doses in 65% (26/40) of cases, including 21 shock overdoses >100 J, for a cumulative defibrillation dose of 18.7 Joules per kg. These errors were reduced by 53% (21/40, P<.001) and cumulative defibrillation dose by 37% (5.14/14, P=.001) with AR glasses. Conclusions AR glasses did not decrease time to first defibrillation attempt and other critical resuscitation endpoints when compared with PALS pocket cards. However, they improved adherence and performance among residents in terms of administering the defibrillation doses set by AHA.

A Mobile App (BEDSide Mobility) to Support Nurses’ Tasks at the Patient's Bedside: Usability Study

Background The introduction of clinical information systems has increased the amount of clinical documentation. Although this documentation generally improves patient safety, it has become a time-consuming task for nurses, which limits their time with the patient. On the basis of a user-centered methodology, we have developed a mobile app named BEDSide Mobility to support nurses in their daily workflow and to facilitate documentation at the bedside. Objective The aim of the study was to assess the usability of the BEDSide Mobility app in terms of the navigation and interaction design through usability testing. Methods Nurses were asked to complete a scenario reflecting their daily work with patients. Their interactions with the app were captured with eye-tracking glasses and by using the think aloud protocol. After completing the tasks, participants filled out the system usability scale questionnaire. Descriptive statistics were used to summarize task completion rates and the users’ performance. Results A total of 10 nurses (aged 21-50) participated in the study. Overall, they were satisfied with the navigation, layout, and interaction design of the app, with the exception of one user who was unfamiliar with smartphones. The problems identified were related to the ambiguity of some icons, the navigation logic, and design inconsistency. Conclusions Besides the usability issues identified in the app, the participants’ results do indicate good usability, high acceptance, and high satisfaction with the developed app. However, the results must be taken with caution because of the poor ecological validity of the experimental setting.

A Mobile Device App to Reduce Medication Errors and Time to Drug Delivery During Pediatric Cardiopulmonary Resuscitation: Study Protocol of a Multicenter Randomized Controlled Crossover Trial

Background During pediatric cardiopulmonary resuscitation (CPR), vasoactive drug preparation for continuous infusions is complex and time-consuming. The need for individual specific weight-based drug dose calculation and preparation places children at higher risk than adults for medication errors. Following an evidence-based and ergonomic driven approach, we developed a mobile device app called Pediatric Accurate Medication in Emergency Situations (PedAMINES), intended to guide caregivers step-by-step from preparation to delivery of drugs requiring continuous infusion. In a prior single center randomized controlled trial, medication errors were reduced from 70% to 0% by using PedAMINES when compared with conventional preparation methods. Objective The purpose of this study is to determine whether the use of PedAMINES in both university and smaller hospitals reduces medication dosage errors (primary outcome), time to drug preparation (TDP), and time to drug delivery (TDD) (secondary outcomes) during pediatric CPR when compared with conventional preparation methods. Methods This is a multicenter, prospective, randomized controlled crossover trial with 2 parallel groups comparing PedAMINES with a conventional and internationally used drug infusion rate table in the preparation of continuous drug infusion. The evaluation setting uses a simulation-based pediatric CPR cardiac arrest scenario with a high-fidelity manikin. The study involving 120 certified nurses (sample size) will take place in the resuscitation rooms of 3 tertiary pediatric emergency departments and 3 smaller hospitals. After epinephrine-induced return of spontaneous circulation, nurses will be asked to prepare a continuous infusion of dopamine using either PedAMINES (intervention group) or the infusion table (control group) and then prepare a continuous infusion of norepinephrine by crossing the procedure. The primary outcome is the medication dosage error rate. The secondary outcome is the time in seconds elapsed since the oral prescription by the physician to drug delivery by the nurse in each allocation group. TDD includes TDP. Stress level during the resuscitation scenario will be assessed for each participant by questionnaire and recorded by the heart rate monitor of a fitness watch. The study is formatted according to the Consolidated Standards of Reporting Trials Statement for Randomized Controlled Trials of Electronic and Mobile Health Applications and Online TeleHealth (CONSORT-EHEALTH) and the Reporting Guidelines for Health Care Simulation Research. Results Enrollment and data analysis started in March 2017. We anticipate the intervention will be completed in late 2017, and study results will be submitted in early 2018 for publication expected in mid-2018. Results will be reported in line with recommendations from CONSORT-EHEALTH and the Reporting Guidelines for Health Care Simulation Research . Conclusions This paper describes the protocol used for a clinical trial assessing the impact of a mobile device app to reduce the rate of medication errors, time to drug preparation, and time to drug delivery during pediatric resuscitation. As research in this area is scarce, results generated from this study will be of great importance and might be sufficient to change and improve the pediatric emergency care practice. Trial Registration ClinicalTrials.gov NCT03021122; https://clinicaltrials.gov/ct2/show/NCT03021122 (Archived by WebCite at http://www.webcitation.org/6nfVJ5b4R)

How Regrouping Alerts in Computerized Physician Order Entry Layout Influences Physicians’ Prescription Behavior: Results of a Crossover Randomized Trial

Background As demonstrated in several publications, low positive predictive value alerts in computerized physician order entry (CPOE) induce fatigue and may interrupt physicians unnecessarily during prescription of medication. Although it is difficult to increase the consideration of medical alerts by physician through an improvement of their predictive value, another approach consists to act on the way they are presented. The interruption management model inspired us to propose an alternative alert display strategy of regrouping the alerts in the screen layout, as a possible solution for reducing the interruption in physicians’ workflow. Objective In this study, we compared 2 CPOE designs based on a particular alert presentation strategy: one design involved regrouping the alerts in a single place on the screen, and in the other, the alerts were located next to the triggering information. Our objective was to evaluate experimentally whether the new design led to fewer interruptions in workflow and if it affected alert handling. Methods The 2 CPOE designs were compared in a controlled crossover randomized trial. All interactions with the system and eye movements were stored for quantitative analysis. Results The study involved a group of 22 users consisting of physicians and medical students who solved medical scenarios containing prescription tasks. Scenario completion time was shorter when the alerts were regrouped (mean 117.29 seconds, SD 36.68) than when disseminated on the screen (mean 145.58 seconds, SD 75.07; P=.045). Eye tracking revealed that physicians fixated longer on alerts in the classic design (mean 119.71 seconds, SD 76.77) than in the centralized alert design (mean 70.58 seconds, SD 33.53; P=.001). Visual switches between prescription and alert areas, indicating interruption, were reduced with centralized alerts (mean 41.29, SD 21.26) compared with the classic design (mean 57.81, SD 35.97; P=.04). Prescription behavior (ie, prescription changes after alerting), however, did not change significantly between the 2 strategies of display. The After-Scenario Questionnaire (ASQ) that was filled out after each scenario showed that overall satisfaction was significantly rated lower when alerts were regrouped (mean 4.37, SD 1.23) than when displayed next to the triggering information (mean 5.32, SD 0.94; P=.02). Conclusions Centralization of alerts in a table might be a way to motivate physicians to manage alerts more actively, in a meaningful way, rather than just being interrupted by them. Our study could not provide clear recommendations yet, but provides objective data through a cognitive psychological approach. Future tests should work on standardized scenarios that would enable to not only measure physicians’ behavior (visual fixations and handling of alerts) but also validate those actions using clinical criteria.

Unsupervised Documents Categorization Using New Threshold-Sensitive Weighting Technique

As the number of published documents increase quickly, there is a crucial need for fast and sensitive categorization methods to manage the produced information. In this paper, we focused on the categorization of biomedical documents with concepts of the Gene Ontology, an ontology dedicated to gene description. Our approach discovers associations between the predefined concepts and the documents using string matching techniques. The assignations are ranked according to a score computed given several strategies. The effects of these different scoring strategies on the categorization effectiveness are evaluated. More especially a new weighting technique based on term frequency is presented. This new weighting technique improves the categorization effectiveness on most of the experiment performed. This paper shows that a cleaver use of the frequency can bring substantial benefits when performing automatic categorization on large collection of documents.