Robert A. Greenes

Assessment of Diagnostic Tests When Disease Verification is Subject to Selection Bias

In the assessment of the statistical properties of a diagnostic test, for example the sensitivity and specificity of the test, it is common to derive estimates from a sample limited to those cases for whom subsequent definitive disease verification is obtained. Omission of nonverified cases can seriously bias the estimates. In order to adjust the estimates it is necessary to make assumptions about the mechanism for selecting cases for verification. Methods for making the necessary adjustments can then be derived.

Clinical decision support systems for the practice of evidence-based medicine.

Background: The use of clinical decision support systems to facilitate the practice of evidence-based medicine promises to substantially improve health care quality. Objective: To describe, on the basis of the proceedings of the Evidence and Decision Support track at the 2000 AMIA Spring Symposium, the research and policy challenges for capturing research and practice-based evidence in machine-interpretable repositories, and to present recommendations for accelerating the development and adoption of clinical decision support systems for evidence-based medicine.

The GuideLine Interchange Format: A Model for Representing Guidelines

OBJECTIVE To allow exchange of clinical practice guidelines among institutions and computer-based applications. DESIGN The GuideLine Interchange Format (GLIF) specification consists of GLIF model and the GLIF syntax. The GLIF model is an object-oriented representation that consists of a set of classes for guideline entities, attributes for those classes, and data types for the attribute values. The GLIF syntax specifies the format of the test file that contains the encoding. METHODS Researchers from the InterMed Collaboratory at Columbia University, Harvard University (Brigham and Womens Hospital and Massachusetts General Hospital), and Stanford University analyzed four existing guideline systems to derive a set of requirements for guideline representation. The GLIF specification is a consensus representation developed through a brainstorming process. Four clinical guidelines were encoded in GLIF to assess its expressivity and to study the variability that occurs when two people from different sites encode the same guideline. RESULTS The encoders reported that GLIF was adequately expressive. A comparison of the encodings revealed substantial variability. CONCLUSION GLIF was sufficient to model the guidelines for the four conditions that were examined. GLIF needs improvement in standard representation of medical concepts, criterion logic, temporal information, and uncertainty.

GLIF3: a representation format for sharable computer-interpretable clinical practice guidelines

The Guideline Interchange Format (GLIF) is a model for representation of sharable computer-interpretable guidelines. The current version of GLIF (GLIF3) is a substantial update and enhancement of the model since the previous version (GLIF2). GLIF3 enables encoding of a guideline at three levels: a conceptual flowchart, a computable specification that can be verified for logical consistency and completeness, and an implementable specification that is intended to be incorporated into particular institutional information systems. The representation has been tested on a wide variety of guidelines that are typical of the range of guidelines in clinical use. It builds upon GLIF2 by adding several constructs that enable interpretation of encoded guidelines in computer-based decision-support systems. GLIF3 leverages standards being developed in Health Level 7 in order to allow integration of guidelines with clinical information systems. The GLIF3 specification consists of an extensible object-oriented model and a structured syntax based on the resource description framework (RDF). Empirical validation of the ability to generate appropriate recommendations using GLIF3 has been tested by executing encoded guidelines against actual patient data. GLIF3 is accordingly ready for broader experimentation and prototype use by organizations that wish to evaluate its ability to capture the logic of clinical guidelines, to implement them in clinical systems, and thereby to provide integrated decision support to assist clinicians.

Testing the technology acceptance model for evaluating healthcare professionals' intention to use an adverse event reporting system

BACKGROUND Many healthcare organizations have implemented adverse event reporting systems in the hope of learning from experience to prevent adverse events and medical errors. However, a number of these applications have failed or not been implemented as predicted. OBJECTIVE This study presents an extended technology acceptance model that integrates variables connoting trust and management support into the model to investigate what determines acceptance of adverse event reporting systems by healthcare professionals. METHOD The proposed model was empirically tested using data collected from a survey in the hospital environment. A confirmatory factor analysis was performed to examine the reliability and validity of the measurement model, and a structural equation modeling technique was used to evaluate the causal model. RESULTS The results indicated that perceived usefulness, perceived ease of use, subjective norm, and trust had a significant effect on a professionals intention to use an adverse event reporting system. Among them, subjective norm had the most contribution (total effect). Perceived ease of use and subjective norm also had a direct effect on perceived usefulness and trust, respectively. Management support had a direct effect on perceived usefulness, perceived ease of use, and subjective norm. CONCLUSION The proposed model provides a means to understand what factors determine the behavioral intention of healthcare professionals to use an adverse event reporting system and how this may affect future use. In addition, understanding the factors contributing to behavioral intent may potentially be used in advance of system development to predict reporting systems acceptance.

Representation primitives, process models and patient data in computer-interpretable clinical practice guidelines: a literature review of guideline representation models.

Representation of clinical practice guidelines in a computer-interpretable format is a critical issue for guideline development, implementation, and evaluation. We studied 11 types of guideline representation models that can be used to encode guidelines in computer-interpretable formats. We have consistently found in all reviewed models that primitives for representation of actions and decisions are necessary components of a guideline representation model. Patient states and execution states are important concepts that closely relate to each other. Scheduling constraints on representation primitives can be modeled as sequences, concurrences, alternatives, and loops in a guidelines application process. Nesting of guidelines provides multiple views to a guideline with different granularities. Integration of guidelines with electronic medical records can be facilitated by the introduction of a formal model for patient data. Data collection, decision, patient state, and intervention constitute four basic types of primitives in a guidelines logic flow. Decisions clarify our understanding on a patients clinical state, while interventions lead to the change from one patient state to another.

Design and implementation of a clinical data management system

Abstract Increasing activity in the use of computers for acquisition, storage, and retrieval of medical information has been stimulated by the growing complexity of medical care, and the needs for standardization, quality control, and retrievability of clinical data. Criteria for the design of a clinical data management system include flexibility in its interface with its environment, the capability of handling variable length text string data, and of organizing it in tree-structured files, the availability of this data to a multi-user environment, and the existence of a high-level language facility for programming and development of the system. The scale and cost of the computer configuration required to meet these demands must nevertheless permit gradual expansion, modularity, and usually duplication of hardware. The MGH Utility Multi-Programming System (MUMPS) is a compact time-sharing system on a medium-scale computer dedicated to clinical data management applications. A novel system design based on a reentrant high-level language interpreter has permitted the implementation of a highly responsive, flexible system, both for research and development and for economical, reliable service operation.

Construction of Receiver Operating Characteristic Curves when Disease Verification Is Subject to Selection Bias

The estimates of ROC curves, which are frequently used in the assessment of diagnostic tests, may be biased if the sample is restricted to subjects whose disease status has been definitively verified. A method to provide an unbiased estimate of the ROC curve under these sample conditions is proposed. The new method requires information on the probability distribution of test results in the population from which the verified sample is drawn. It is illustrated using data from a study of computed tomography for fever of uncertain origin.

SMART—An Integrated Wireless System for Monitoring Unattended Patients

Monitoring vital signs and locations of certain classes of ambulatory patients can be useful in overcrowded emergency departments and at disaster scenes, both on-site and during transportation. To be useful, such monitoring needs to be portable and low cost, and have minimal adverse impact on emergency personnel, e.g., by not raising an excessive number of alarms. The SMART (Scalable Medical Alert Response Technology) system integrates wireless patient monitoring (ECG, SpO(2)), geo-positioning, signal processing, targeted alerting, and a wireless interface for caregivers. A prototype implementation of SMART was piloted in the waiting area of an emergency department and evaluated with 145 post-triage patients. System deployment aspects were also evaluated during a small-scale disaster-drill exercise.

Assessment of diagnostic technologies. Methodology for unbiased estimation from samples of selectively verified patients

Concern with the efficacy of diagnostic technologies has stimulated numerous studies aimed at quantifying the discriminatory properties of various tests and procedures. These have focused principally on estimations of the result conditional probabilities, given disease status, eg, the sensitivity and specificity or the ROC curve. A source of bias in estimating these probabilities that is often unavoidable is created by the existence of a nonrandom selection mechanism for determining which patients initially tested will receive definitive verification of disease status. Correction for verification bias requires frequency data on the test results and any symptoms or other factors that influence selection for verification, both in the verified sample and in the source sample of patients tested.