Ronald Cornet

Debugging Incoherent Terminologies

In this paper we study the diagnosis and repair of incoherent terminologies. We define a number of new nonstandard reasoning services to explain incoherence through pinpointing, and we present algorithms for all of these services. For one of the core tasks of debugging, the calculation of minimal unsatisfiability preserving subterminologies, we developed two different algorithms, one implementing a bottom-up approach using support of an external description logic reasoner, the other implementing a specialized tableau-based calculus. Both algorithms have been prototypically implemented. We study the effectiveness of our algorithms in two ways: we present a realistic case study where we diagnose a terminology used in a practical application, and we perform controlled benchmark experiments to get a better understanding of the computational properties of our algorithms in particular and the debugging problem in general.

Comparison of reasoners for large ontologies in the OWL 2 EL profile

This paper provides a survey to and a comparison of state-of-the-art Semantic Web reasoners that succeed in classifying large ontologies expressed in the tractable OWL 2 EL profile. Reasoners are characterized along several dimensions: The first dimension comprises underlying reasoning characteristics, such as the employed reasoning method and its correctness as well as the expressivity and worst-case computational complexity of its supported language and whether the reasoner supports incremental classification, rules, justifications for inconsistent concepts and ABox reasoning tasks. The second dimension is practical usability: whether the reasoner implements the OWL API and can be used via OWLlink, whether it is available as Protege plugin, on which platforms it runs, whether its source is open or closed and which license it comes with. The last dimension contains performance indicators that can be evaluated empirically, such as classification, concept satisfiability, subsumption checking and consistency checking performance as well as required heap space and practical correctness, which is determined by comparing the computed concept hierarchies with each other. For the very large ontology SNOMED CT, which is released both in stated and inferred form, we test whether the computed concept hierarchies are correct by comparing them to the inferred form of the official distribution. The reasoners are categorized along the defined characteristics and benchmarked against well-known biomedical ontologies. The main conclusion from this study is that reasoners vary significantly with regard to all included characteristics, and therefore a critical assessment and evaluation of requirements is needed before selecting a reasoner for a real-life application.

A survey of SNOMED CT implementations

Abstract The Systematised Nomenclature of Medicine Clinical Terms (SNOMED CT) has been designated as the recommended clinical reference terminology for use in clinical information systems around the world and is reported to be used in over 50 countries. However, there are still few implementation details. This study examined the implementation of SNOMED CT in terms of design, use and maintenance issues involved in 13 healthcare organisations across eight countries through a series of interviews with 14 individuals. While a great deal of effort has been spent on developing and refining SNOMED CT, there is still much work ahead to bring SNOMED CT into routine clinical use.

Guest Editorial: Special Issue on Auditing of Terminologies

This special issue is the first in any journal to deal exclusively with the auditing of medical terminologies and ontologies. In the early stages of many emerging technical fields, the emphasis is primarily on creating systems that work. The emphasis eventually shifts to guaranteeing a high level of quality. In fact, two sure signs that a technical field is maturing are extensive activities regarding standards—both national and international—and quality assurance initiatives. In the field of medical terminologies, we have seen a great deal of standards activity in recent years, such as the work of the ISO/ TC215 Working Group 3 (Health Informatics – Semantic Content) and the American National Standards Institute guidelines for designing controlled terminologies (ANSI/NISO Z39.19-2005). There is also a heightened awareness of the need for terminology quality assurance, which has manifested itself in different ways. For example, at the 2007 AMIA Annual Symposium (Chicago), a session was devoted exclusively to terminology assessments (S10). One can also discern a constant stream of research papers in the medical informatics literature dealing with auditing. The breadth of this work, as measured by the number of different terminologies being considered, is growing. The reference list and Table 6 of the paper by Zhu et al. [1] in this special issue give evidence to these observations. Furthermore, many health care information systems rely on terminologies to provide semantically uniform access to knowledge expressed in different ways and following various paradigms. As a result of this reliance, an error in a terminology may propagate to errors in systems ranging from decision-support systems to pharmacy information systems that are monitoring, for example, drug allergies and drug-drug interactions. Hence, an error in a terminology may further propagate, resulting in an error in the treatment of patients. Realizing these facts, the guest editors came to the conclusion that the field of medical terminologies is maturing and that there is enough interest within the research community to warrant producing this unprecedented special issue on ‘‘Auditing of Terminologies.” The intention is to capture the current state of the art in the field of terminology auditing. In addition to announcing the Call for Papers for this special issue on the JBI Web-site and in other forums, the guest editors directly solicited prominent researchers in this area to submit papers. Fortunately, most of them agreed. The current issue features many authors with outstanding track records in the areas of medical terminologies and their quality assurance. To ensure high-quality published papers, many of the authors were asked to serve in the role of reviewer as well. Some researchers who were too busy to prepare papers themselves graciously agreed

Auditing description-logic-based medical terminological systems by detecting equivalent concept definitions

OBJECTIVE To specify and evaluate a method for auditing medical terminological systems (TSs) based on detecting concepts with equivalent definitions. This method addresses two important problems: redundancy, where the same concept is represented more than once (described by different terms), and underspecification, where different concepts have the same representation and hence appear indistinguishable from each other. DESIGN The auditing method is applicable for TSs that are or can be represented in a description logic (DL). The method relies on the assumption that concept definitions are non-primitive (i.e. they are regarded as providing necessary and sufficient conditions). Whereas this assumption may not hold for many definitions, it does serve the purpose of detecting sets of logically equivalent concepts by a DL reasoner. Such a set may include the same concept which is defined more than once and/or different concepts that are underspecified as they appear indistinguishable from each other by their represented properties. Analysis of these sets provides insight into the representation quality of concepts and provides hints at improving the TS. MEASUREMENTS In our case study the method is applied to the DICE TS, a comprehensive TS in intensive care. It comprises about 2500 concepts and 40 properties and relations. RESULTS In DICE we found four concepts that were defined twice. Furthermore, 100 sets were found containing more than 300 underspecified concepts. The sizes of these sets ranged from 2 to 13. Analysis revealed that many concepts can be more completely defined, either by adding existing relations, or by the introduction of new relations into the terminological system. CONCLUSION The method proved both usable and valuable for auditing TSs. DL reasoning is fully automated and all equivalent concept definitions are systematically found. The resulting sets of equivalent concepts clearly point out which concept definitions are to be reviewed, as they contain duplicate definitions of a concept, and (inherently or unnecessarily) underspecified concepts.

A usability evaluation of a SNOMED CT based compositional interface terminology for intensive care

OBJECTIVE To evaluate the usability of a large compositional interface terminology based on SNOMED CT and the terminology application for registration of the reasons for intensive care admission in a Patient Data Management System. DESIGN Observational study with user-based usability evaluations before and 3 months after the system was implemented and routinely used. MEASUREMENTS Usability was defined by five aspects: effectiveness, efficiency, learnability, overall user satisfaction, and experienced usability problems. Qualitative (the Think-Aloud user testing method) and quantitative (the System Usability Scale questionnaire and Time-on-Task analyses) methods were used to examine these usability aspects. RESULTS The results of the evaluation study revealed that the usability of the interface terminology fell short (SUS scores before and after implementation of 47.2 out of 100 and 37.5 respectively out of 100). The qualitative measurements revealed a high number (n=35) of distinct usability problems, leading to ineffective and inefficient registration of reasons for admission. The effectiveness and efficiency of the system did not change over time. About 14% (n=5) of the revealed usability problems were related to the terminology content based on SNOMED CT, while the remaining 86% (n=30) was related to the terminology application. The problems related to the terminology content were more severe than the problems related to the terminology application. CONCLUSIONS This study provides a detailed insight into how clinicians interact with a controlled compositional terminology through a terminology application. The extensiveness, complexity of the hierarchy, and the language usage of an interface terminology are defining for its usability. Carefully crafted domain-specific subsets and a well-designed terminology application are needed to facilitate the use of a complex compositional interface terminology based on SNOMED CT.

PROTÉGÉ as a vehicle for developing medical terminological systems

A medical terminological system (TS) is essentially an ontology consisting of concepts, attributes and relationships pertaining to medical terms. There are many TSs around today, most of which are essentially frame-based. Various efforts have been made to get a better understanding of the requirements and the conceptual and formal structures of TSs. However, the actual implementation of a TS consisted so far of ad hoc approaches starting from scratch and, due to ad hoc semantics of the representation, the interoperability with external applications of the knowledge represented is diminished. In recent years, PROTEGE has been gaining in popularity as a software environment for the development of knowledge-based systems. It provides an architecture for integrating frame-based ontologies with knowledge acquisition and other applications operating on these ontologies. In its recent version, PROTEGE provides the ability to specify meta-classes and -slots. This contributes to an explicit separability of knowledge levels and allows for an increased modeling flexibility. These properties, and the fact that it complies with a standard knowledge model, enable PROTEGE to be an attractive candidate for the implementation of frame-based TSs. This paper investigates how to specify a TS in PROTEGE and demonstrates this in a specific application in the domain of intensive care. Our approach is characterized by the utilization of a conceptual framework for understanding TSs and mapping its components onto PROTEGE constructs. This results in specifications of knowledge components for the implementation of terminological systems. The significance of our work stems from the generality of these specifications. This facilitates their reuse, leading to a principled process for the development of terminological systems for a broad spectrum of medical domains.

Using Description Logics for Managing Medical Terminologies

Medical terminological knowledge bases play an increasingly important role in medicine. As their size and complexity are growing, the need arises for a means to verify and maintain the consistency and correctness of their contents. This is important for their management as well as for providing their users with confidence about the validity of their contents. In this paper we describe a method for the detection of modeling errors in a terminological knowledge base. The method uses a Description Logic (DL) for the representation of the medical knowledge and is based on the migration from a frame-based representation to a DL-based one. It is characterized by initially using strong assumptions in concept definitions thereby forcing the detection of concepts and relationships that might comprise a source of inconsistency. We demonstrate the utility of the approach in a real world case study of a terminological knowledge base in the Intensive Care domain and we discuss decisions pertaining to building DL-based representations

Description logic-based methods for auditing frame-based medical terminological systems

OBJECTIVE Medical terminological systems (TSs) play an increasingly important role in health care by supporting recording, retrieval and analysis of patient information. As the size and complexity of TSs are growing, the need arises for means to audit them, i.e. verify and maintain (logical) consistency and (semantic) correctness of their contents. This is not only important for the management of TSs but also for providing their users with confidence about the reliability of their contents. Formal methods have the potential to play an important role in the audit of TSs, although there are few empirical studies to assess the benefits of using these methods. METHODS AND MATERIAL In this paper we propose a method based on description logics (DLs) for the audit of TSs. This method is based on the migration of the medical TS from a frame-based representation to a DL-based one. Our method is characterized by a process in which initially stringent assumptions are made about concept definitions. The assumptions allow the detection of concepts and relations that might comprise a source of logical inconsistency. If the assumptions hold then definitions are to be altered to eliminate the inconsistency, otherwise the assumptions are revised. RESULTS In order to demonstrate the utility of the approach in a real-world case study we audit a TS in the intensive care domain and discuss decisions pertaining to building DL-based representations. This case study demonstrates that certain types of inconsistencies can indeed be detected by applying the method to a medical terminological system. CONCLUSION The added value of the method described in this paper is that it provides a means to evaluate the compliance to a number of common modeling principles in a formal manner. The proposed method reveals potential modeling inconsistencies, helping to audit and (if possible) improve the medical TS. In this way, it contributes to providing confidence in the contents of the terminological system.

Definitions and Qualifiers in SNOMED CT

OBJECTIVES An important feature of SNOMED CT is post-coordination, which is enabled by the SNOMED CT representation specifying whether a relationship is a defining or a qualifier relationship. In this paper the use of qualifier relationships in SNOMED CT is analyzed, as well as the extent to which qualifiers interact with defining relationships, so that pre-coordinated concepts can also be post-coordinated. METHODS The July 2007 release of SNOMED CT was imported into a database. Analyses were performed by querying this database. RESULTS Qualifier relationships occur in 10 out of 61 types of attribute relationships, and it is shown that generally pre-coordinated concepts cannot be constructed by applying post-coordination using qualifier relationships. Most of the qualifier relationships have generic target concepts, making it possible to construct concepts which are not clinically sensible. A logic-based representation is proposed to overcome the drawbacks of the current model. CONCLUSIONS Defining and qualifier relationships both enable post-coordination in SNOMED CT. Introducing qualifiers for more types of relationships, and using qualifier relationships with more specific target concepts will further improve post-coordination in SNOMED CT.