Stéfan Jacques Darmoni

Online clinical reasoning assessment with the Script Concordance test: a feasibility study

BackgroundThe script concordance (SC) test is an assessment tool that measures capacity to solve ill-defined problems, that is, reasoning in context of uncertainty. This tool has been used up to now mainly in medicine. The purpose of this pilot study is to assess the feasibility of the test delivered on the Web to French urologists.MethodsThe principle of SC test construction and the development of the Web site are described. A secure Web site was created with two sequential modules: (a) The first one for the reference panel (n = 26) with two sub-tasks: to validate the content of the test and to elaborate the scoring system; (b) The second for candidates with different levels of experience in Urology: Board certified urologists, residents, medical students (5 or 6th year). Minimum expected number of participants is 150 for urologists, 100 for residents and 50 for medical students. Each candidate is provided with an individual access code to this Web site. He/she may complete the Script Concordance test several times during his/her curriculum.ResultsThe Web site has been operational since April 2004. The reference panel validated the test in June of the same year during the annual seminar of the French Society of Urology. The Web site is available for the candidates since September 2004. In six months, 80% of the target figure for the urologists, 68% of the target figure for the residents and 20% of the target figure for the student passed the test online. During these six months, no technical problem was encountered.ConclusionThe feasibility of the web-based SC test is successful as two-thirds of the expected number of participants was included within six months. Psychometric properties (validity, reliability) of the test will be evaluated on a large scale (N = 300). If positive, educational impact of this assessment tool will be useful to help urologists during their curriculum for the acquisition of clinical reasoning skills, which is crucial for professional competence.

Indexing biomedical documents with a possibilistic network

In this article, we propose a new approach for indexing biomedical documents based on a possibilistic network that carries out partial matching between documents and biomedical vocabulary. The main contribution of our approach is to deal with the imprecision and uncertainty of the indexing task using possibility theory. We enhance estimation of the similarity between a document and a given concept using the two measures of possibility and necessity. Possibility estimates the extent to which a document is not similar to the concept. The second measure can provide confirmation that the document is similar to the concept. Our contribution also reduces the limitation of partial matching. Although the latter allows extracting from the document other variants of terms than those in dictionaries, it also generates irrelevant information. Our objective is to filter the index using the knowledge provided by the Unified Medical Language System®. Experiments were carried out on different corpora, showing encouraging results (the improvement rate is +26.37% in terms of main average precision when compared with the baseline).

Biomedical Concepts Extraction Based on Possibilistic Network and Vector Space Model

This paper proposes a new approach for indexing biomedical documents based on the combination of a Possibilistic Network and a Vector Space Model. This later carries out partial matching between documents and biomedical vocabularies. The main contribution of the proposed approach is to combine the cosine similarity and the two measures of possibility and necessity to enhance the estimation of the similarity between a document and a given concept. The possibility estimates the extent to which a document is not similar to the concept. The necessity allows the confirmation that the document is similar to the concept. Experiments were carried out on the OSHUMED corpora and showed encouraging results.

Évaluation d’un outil d’aide á l’anonymisation des documents médicaux basé sur le traitement automatique du langage naturel

Anonymization of personal data is a legal requirement for their use as part of a research project. In the context of developing a tool for detecting hospital-acquired infections, 2000 medical documents were needed for the research project ALADIN. To help annotators to anonymize this corpus of documents, a tool for the anonymization has been developed, relying on Natural Language Processing techniques. The recall, precision and F-score of the automatic phase of the anonymizer were respectively 79.7, 85.2 and 82.4%. The gold- standard used for the evaluation was the manual anonymization of the documents. The performance of the automatic anonymization can still be improved but the tool is already a considerable help in this process in terms of saving time and in terms of quality of anonymization (including the accuracy of labeling anonymized terms and computation of time duration).

F-MTI: outil d’indexation muititerminologique: application à l’indexation automatique de la SNOMED International

Background SNOMED is becoming a major health terminology to index electronic medical records. Most of developed countries have chosen SNOMED CT. France has chosen SNOMED International which is already translated in French. We developed the F-MTI tool, a generic automatic indexing tool able to index documentation in several health terminologies written in French (CCAM, TUV) or translated in French (MeSH, ICD-lO, SNOMED International).

Knowledge-Based Query Expansion over a Medical Terminology Oriented Ontology on the Web

This paper deals with the problem of information retrieval on the Web and present the CISMeF project (acronym of Catalogue and Index of French-speaking Medical Sites). Information retrieval in the CISMeF catalogue is done with a terminology that is similar to ontology of medical domain and a set of metadata. This allows us to place the project at an overlap between the present Web, which is informal, and the forthcoming Semantic Web. We also describe an ongoing work, which consists of applying thr ee knowledge-based methods in order to enhance information retrieval.

Rewriting Natural Language Queries Using Patterns

In this paper, a method based on pre-defined patterns, which rewrites natural language queries into a multi-layer, flexible, scalable and object-oriented query language, is presented. The method has been conceived to assist physicians in their search for clinical information in an Electronic Health Records system. Indeed, the query language of the system being difficult to handle for physicians, this method allows querying using natural language vs. using dedicated object-oriented query language. The information extraction method that has been developed can be seen as a named entity recognition system based on regular expressions that tags pieces of the query. The patterns are constructed recursively from the initial natural language query and from atomic patterns that correspond to the entities, the relationships and the constraints of the underlying model representing Electronic Health Records. Further evaluation is needed, but the preliminary results obtained by testing a set of natural language queries are very encouraging.

A web-based teaching resource to prepare for final undergraduate examination: a French pilot study

Access to accurate and quality-controlled health information on the Internet for medical students is not an easy task. CISMeF is the search tool of a MeSH-indexed directory of medical Internet resources in French. Since 2004, a new French Pre-Residency Examination (PRE) is compulsory for all medical students in the 6th year of the curriculum. The goal of this study is to evaluate CISMeF as a tool to provide teaching resources available on the Internet covering PRE material. The CISMeF terminology and the PRE CISMeF module are described. To assess the CISMeF performance in covering PRE program, its precision (number of relevant resources/number of overall resources extracted by CISMeF) and coverage (number of PRE questions covered by at least one resource in the CISMeF gateway) were computed. The CISMeF module for the new French Pre-Residency Examination is efficient as it already covers 95.7% of the program with a precision of 82.2%. Our data demonstrates that CISMeF is acceptable to guide students’ learning and should be a useful teaching resource for the preparation of the French Pre-Residency Examination.

Accuracy of using natural language processing methods for identifying healthcare-associated infections

OBJECTIVE There is a growing interest in using natural language processing (NLP) for healthcare-associated infections (HAIs) monitoring. A French project consortium, SYNODOS, developed a NLP solution for detecting medical events in electronic medical records for epidemiological purposes. The objective of this study was to evaluate the performance of the SYNODOS data processing chain for detecting HAIs in clinical documents. MATERIALS AND METHODS The collection of textual records in these hospitals was carried out between October 2009 and December 2010 in three French University hospitals (Lyon, Rouen and Nice). The following medical specialties were included in the study: digestive surgery, neurosurgery, orthopedic surgery, adult intensive-care units. Reference Standard surveillance was compared with the results of automatic detection using NLP. Sensitivity on 56 HAI cases and specificity on 57 non-HAI cases were calculated. RESULTS The accuracy rate was 84% (n = 95/113). The overall sensitivity of automatic detection of HAIs was 83.9% (CI 95%: 71.7-92.4) and the specificity was 84.2% (CI 95%: 72.1-92.5). The sensitivity varies from one specialty to the other, from 69.2% (CI 95%: 38.6-90.9) for intensive care to 93.3% (CI 95%: 68.1-99.8) for orthopedic surgery. The manual review of classification errors showed that the most frequent cause was an inaccurate temporal labeling of medical events, which is an important factor for HAI detection. CONCLUSION This study confirmed the feasibility of using NLP for the HAI detection in hospital facilities. Automatic HAI detection algorithms could offer better surveillance standardization for hospital comparisons.

Ontological Representation of Laboratory Test Observables: Challenges and Perspectives in the SNOMED CT Observable Entity Model Adoption

The emergence of electronic health records has highlighted the need for semantic standards for representation of observations in laboratory medicine. Two such standards are LOINC, with a focus on detailed encoding of lab tests, and SNOMED CT, which is more general, including the representation of qualitative and ordinal test results. In this paper we will discuss how lab observation entries can be represented using SNOMED CT. We use resources provided by the Regenstrief Institute and SNOMED International collaboration, which formalize LOINC terms as SNOMED CT post-coordinated expressions. We demonstrate the benefits brought by SNOMED CT to classify lab tests. We then propose a SNOMED CT based model for lab observation entries aligned with the BioTopLite2 (BTL2) upper level ontology. We provide examples showing how a model designed with no ontological foundation can produce misleading interpretations of inferred observation results. Our solution based on a BTL2 conformant formal interpretation of SNOMED CT concepts allows representing lab test without creating unintended models. We argue in favour of an ontologically explicit bridge between compositional clinical terminologies, in order to safely use their formal representations in intelligent systems.