Nicolette F. de Keizer

Defining and Improving Data Quality in Medical Registries: A Literature Review, Case Study, and Generic Framework

Over the past years the number of medical registries has increased sharply. Their value strongly depends on the quality of the data contained in the registry. To optimize data quality, special procedures have to be followed. A literature review and a case study of data quality formed the basis for the development of a framework of procedures for data quality assurance in medical registries. Procedures in the framework have been divided into procedures for the co-ordinating center of the registry (central) and procedures for the centers where the data are collected (local). These central and local procedures are further subdivided into (a) the prevention of insufficient data quality, (b) the detection of imperfect data and their causes, and (c) actions to be taken / corrections. The framework can be used to set up a new registry or to identify procedures in existing registries that need adjustment to improve data quality.

Variation in critical care services across North America and Western Europe

Objective:Critical care represents a large percentage of healthcare spending in developed countries. Yet, little is known regarding international variation in critical care services. We sought to understand differences in critical care delivery by comparing data on the distribution of services in eight countries. Design:Retrospective review of existing national administrative data. We identified sources of data in each country to provide information on acute care hospitals and beds, intensive care units and beds, intensive care admissions, and definitions of intensive care beds. Data were all referenced and from as close to 2005 as possible. Setting:United States, France, United Kingdom, Canada, Belgium, Germany, The Netherlands, and Spain. Patients:Not available. Interventions:None. Measurements and Main Results:No standard definition existed for acute care hospital or intensive care unit beds across countries. Hospital beds varied three-fold from 221/100,000 population in the United States to 593/100,000 in Germany. Adult intensive care unit beds also ranged seven-fold from 3.3/100,000 population in the United Kingdom to 24.0/100,000 in Germany. Volume of intensive care unit admissions per year varied ten-fold from 216/100,000 population in the United Kingdom to 2353/100,000 in Germany. The ratio of intensive care unit beds to hospital beds was highly correlated across all countries except the United States (r = .90). There was minimal correlation between the number of intensive care unit beds per capita and health care spending per capita (r = .45), but high inverse correlation between intensive care unit beds and hospital mortality for intensive care unit patients across countries (r = −.82). Conclusions:Absolute critical care services vary dramatically between countries with wide differences in both numbers of beds and volume of admissions. The number of intensive care unit beds per capita is not strongly correlated with overall health expenditure, but does correlate strongly with mortality. These findings demonstrate the need for critical care data from all countries, as they are essential for interpretation of studies, and policy decisions regarding critical care services.

The impact of computerized physician medication order entry in hospitalized patients--a systematic review.

OBJECTIVE To identify all published studies evaluating computerized physician order entry (CPOE) in the inpatient setting and uniformly classify these studies on outcome measure and study design. DATA SOURCES All studies that evaluated the effect of CPOE on outcomes pertaining to the medication process in inpatients were electronically searched in MEDLINE (1966 to August 2006), EMBASE (1980 to August 2006) and the Cochrane library. In addition, the bibliographies of retrieved articles were manually searched. Articles were selected if one of their main objectives was CPOE evaluation in an inpatient setting. REVIEW METHOD Identified titles and abstracts were independently screened by three reviewers to determine eligibility for further review. RESULTS We found 67 articles, which included articles on CPOE evaluation on some outcome at the time of ordering. Most papers evaluated multiple outcome measures. The outcome measures were clustered in the following categories: adherence (n=22); alerts and appropriateness of alerts (n=7); safety (n=21); time (n=7); costs and (organizational) efficiency (n=23); and satisfaction, usage and usability (n=10). Most studies used a before-after design (n=35) followed by observational studies (n=24) and randomized controlled trials (n=8). CONCLUSION The impact of CPOE systems was especially positive in the category adherence to guidelines, but also to some extent in alerts and appropriateness of alerts; costs and organizational efficiency; and satisfaction and usability. Although on average, there seems to be a positive effect of CPOE on safety, studies tended to be non-randomized and were focused on medication error rates, not powered to detect a difference in adverse drug event rates. Some recent studies suggested that errors, adverse drug events (ADEs) and even mortality increased after CPOE implementation. Only in the category time the impact has been shown to be negative, but this only refers to the physicians time, not the net time. Except for safety, on the whole spectrum of outcomes, results of RCT studies were in line with non-RCT study results.

Evaluation of Outpatient Computerized Physician Medication Order Entry Systems: A Systematic Review

This paper provides a systematic literature review of CPOE evaluation studies in the outpatient setting on: safety; cost and efficiency; adherence to guideline; alerts; time; and satisfaction, usage, and usability. Thirty articles with original data (randomized clinical trial, non-randomized clinical trial, or observational study designs) met the inclusion criteria. Only four studies assessed the effect of CPOE on safety. The effect was not significant on the number of adverse drug events. Only one study showed a significant reduction of the number of medication errors. Three studies showed significant reductions in medication costs; five other studies could not support this. Most studies on adherence to guidelines showed a significant positive effect. The relatively small number of evaluation studies published to date do not provide adequate evidence that CPOE systems enhance safety and reduce cost in the outpatient settings. There is however evidence for (a) increasing adherence to guidelines, (b) increasing total prescribing time, and (c) high frequency of ignored alerts.

Association between administered oxygen, arterial partial oxygen pressure and mortality in mechanically ventilated intensive care unit patients

IntroductionThe aim of this study was to investigate whether in-hospital mortality was associated with the administered fraction of oxygen in inspired air (FiO2) and achieved arterial partial pressure of oxygen (PaO2).MethodsThis was a retrospective, observational study on data from the first 24 h after admission from 36,307 consecutive patients admitted to 50 Dutch intensive care units (ICUs) and treated with mechanical ventilation. Oxygenation data from all admission days were analysed in a subset of 3,322 patients in 5 ICUs.ResultsMean PaO2 and FiO2 in the first 24 h after ICU admission were 13.2 kPa (standard deviation (SD) 6.5) and 50% (SD 20%) respectively. Mean PaO2 and FiO2 from all admission days were 12.4 kPa (SD 5.5) and 53% (SD 18). Focusing on oxygenation in the first 24 h of admission, in-hospital mortality was shown to be linearly related to FiO2 value and had a U-shaped relationship with PaO2 (both lower and higher PaO2 values were associated with a higher mortality), independent of each other and of Simplified Acute Physiology Score (SAPS) II, age, admission type, reduced Glasgow Coma Scale (GCS) score, and individual ICU. Focusing on the entire ICU stay, in-hospital mortality was independently associated with mean FiO2 during ICU stay and with the lower two quintiles of mean PaO2 value during ICU stay.ConclusionsActually achieved PaO2 values in ICU patients in The Netherlands are higher than generally recommended in the literature. High FiO2, and both low PaO2 and high PaO2 in the first 24 h after admission are independently associated with in-hospital mortality in ICU patients. Future research should study whether this association is causal or merely a reflection of differences in severity of illness insufficiently corrected for in the multivariate analysis.

STARE-HI - Statement on reporting of evaluation studies in Health Informatics

OBJECTIVE Development of guidelines for publication of evaluation studies of Health Informatics applications. METHODS An initial list of issues to be addressed in reports on evaluation studies was drafted based on experiences as editors and reviewers of journals in Health Informatics and as authors of systematic reviews of Health Informatics studies, taking into account guidelines for reporting of medical research. This list has been discussed in several rounds by an increasing number of experts in Health Informatics evaluation during conferences and by using e-mail and has been put up for comments on the web. RESULTS A set of STARE-HI principles to be addressed in papers describing evaluations of Health Informatics interventions is presented. These principles include formulation of title and abstract, of introduction (e.g. scientific background, study objectives), study context (e.g. organizational setting, system details), methods (e.g. study design, outcome measures), results (e.g. study findings, unexpected observations) and discussion and conclusion of an IT evaluation paper. CONCLUSION A comprehensive list of principles relevant for properly describing Health Informatics evaluations has been developed. When manuscripts submitted to Health Informatics journals and general medical journals adhere to these aspects, readers will be better positioned to place the studies in a proper context and judge their validity and generalisability. It will also be possible to judge better whether papers will fit in the scope of meta-analyses of Health Informatics interventions. STARE-HI may also be used for study planning and hence positively influence the quality of evaluation studies in Health Informatics. We believe that better publication of both quantitative and qualitative evaluation studies is an important step toward the vision of evidence-based Health Informatics. LIMITATIONS This study is based on experiences from editors, reviewers, authors of systematic reviews and readers of the scientific literature. The applicability of the principles has not been evaluated in real practice. Only when authors start to use these principles for reporting, shortcomings in the principles will emerge.

Comparison of reasoners for large ontologies in the OWL 2 EL profile

This paper provides a survey to and a comparison of state-of-the-art Semantic Web reasoners that succeed in classifying large ontologies expressed in the tractable OWL 2 EL profile. Reasoners are characterized along several dimensions: The first dimension comprises underlying reasoning characteristics, such as the employed reasoning method and its correctness as well as the expressivity and worst-case computational complexity of its supported language and whether the reasoner supports incremental classification, rules, justifications for inconsistent concepts and ABox reasoning tasks. The second dimension is practical usability: whether the reasoner implements the OWL API and can be used via OWLlink, whether it is available as Protege plugin, on which platforms it runs, whether its source is open or closed and which license it comes with. The last dimension contains performance indicators that can be evaluated empirically, such as classification, concept satisfiability, subsumption checking and consistency checking performance as well as required heap space and practical correctness, which is determined by comparing the computed concept hierarchies with each other. For the very large ontology SNOMED CT, which is released both in stated and inferred form, we test whether the computed concept hierarchies are correct by comparing them to the inferred form of the official distribution. The reasoners are categorized along the defined characteristics and benchmarked against well-known biomedical ontologies. The main conclusion from this study is that reasoners vary significantly with regard to all included characteristics, and therefore a critical assessment and evaluation of requirements is needed before selecting a reasoner for a real-life application.

Body mass index is associated with hospital mortality in critically ill patients: an observational cohort study.

Objective:Obesity is associated with a variety of diseases, which results in a decreased overall life expectancy. Nevertheless, some studies suggest that being overweight may reduce hospital mortality of certain patient groups, referred to as obesity paradox. Conflicting results for critically ill patients are reported. Therefore, we wished to investigate the association of body mass index and hospital mortality in critically ill patients. Design:Observational cohort study in Dutch critically ill patients. Setting:A dataset from the Dutch National Intensive Care Evaluation registry that includes patients admitted to Dutch ICUs was used. Patients:One hundred fifty-four thousand three hundred and eight ICU patients of teaching and nonteaching units in urban and nonurban hospitals. Interventions:None. Measurements and Main Results:We used logistic regression analysis, correcting for case mix (Simplified Acute Physiology Score II, age, gender, admission type, neoplasm, AIDS, hematologic malignancy, immunologic insufficiency, mechanical ventilation, and calendar year), to determine the relationship between body mass index and hospital mortality. Body mass index was included in the model as a continuous nonlinear covariate in a restricted regression spline transformation. To facilitate interpretation, adjusted odds ratios were calculated for the World Health Organization-based body mass index classes. Body mass index was found to be significantly associated with hospital mortality, with risks quickly increasing for underweight patients (body mass index < 18.5 kg/m2). Obese and seriously obese patients, with a body mass index of 30–39.9 kg/m2, had the lowest risk of death with an adjusted odds ratio of 0.86 (0.83–0.90). Conclusions:This large observational database shows an inverse association between obesity and hospital mortality in critically ill patients that could not be explained by a variety of known confounders.

Improving quality of care. A systematic review on how medical registries provide information feedback to health care providers.

OBJECTIVE To determine (1) how medical registries provide information feedback to health care professionals, (2) whether this feedback has any effect on the quality of care and (3) what the barriers and success factors are to the effectiveness of feedback. DATA SOURCES Original articles in English found in MEDLINE Pubmed covering the period January 1990 to August 2007. REVIEW METHOD Titles and abstracts of 6223 original articles were independently screened by two reviewers to determine relevance for further review. DATA EXTRACTION AND ANALYSIS We used a standardized data abstraction form to collect information on the feedback initiatives and their effectiveness. The effect of the feedback was only described for analytic papers, i.e. papers that attempted to objectively quantify the effect on the quality of care and to relate this effect to feedback as an intervention. For analysis of the effectiveness, we categorized the initiatives based on the number of elements added to the feedback. RESULTS We included 53 papers, describing 50 feedback initiatives, of which 39 were part of a multifaceted approach. Our results confirm previous research findings that adding elements to a feedback strategy positively influences its effectiveness. We found 22 analytic studies, four of which found a positive effect on all outcome measures, eight found a mix of positive- and no effects and ten did not find any effects (neither positive nor negative). Of the 43 process of care measures evaluated in the analytic studies, 26 were positively affected by the feedback initiative. Of the 36 evaluated outcome of care measures, five were positively affected. The most frequently mentioned factors influencing the effectiveness of the feedback were: (trust in) quality of the data, motivation of the recipients, organizational factors and outcome expectancy of the feedback recipients. CONCLUSIONS The literature on methods and effects of information feedback by medical registries is heterogeneous, making it difficult to draw definite conclusions on its effectiveness. However, the positive effects cannot be discarded. Although our review confirms findings from previous studies that process of care measures are more positively influenced by feedback than outcome of care measures, further research should attempt to identify outcome of care measures that are sensitive to behaviour change as a result of feedback strategies. Furthermore, future studies evaluating the effectiveness of feedback should include a more extensive description of their intervention in order to increase the reproducibility of feedback initiatives and the generalizability of the results.

Teledermatologic consultation and reduction in referrals to dermatologists: a cluster randomized controlled trial.

OBJECTIVE To determine whether teledermatologic consultations can reduce referrals to a dermatologist by general practitioners (GPs). DESIGN Multicenter cluster randomized controlled trial. SETTING AND PARTICIPANTS We recruited 85 GPs from 35 general practices in 2 regions in the Netherlands (Almere and Zeist); 5 dermatologists from 2 nonacademic hospitals were also included in the study. Interventions The GPs randomized to the intervention used a teledermatologic consultation system to confer with a dermatologist, whereas those in the control group referred their patients according to usual practice. All patients, regardless of their condition, were seen in the office by a dermatologist after approximately 1 month. OUTCOME MEASURES The main outcome measure was the proportion of office visits prevented by teledermatologic consultation, as determined by dermatologists at approximately the 1-month office visit. The secondary outcome measure was patient satisfaction, measured using the Patient Satisfaction Questionnaire III developed by Ware et al. RESULTS The 85 study GPs enrolled 631 patients (46 intervention GPs, 327 patients; 39 control GPs, 304 patients). The 5 dermatologists considered a consultation preventable for 39.0% of patients who received teledermatologic consultation and 18.3% of 169 control patients, a difference of 20.7% (95% confidence interval, 8.5%-32.9%). At the 1-month dermatologist visit, 20.0% of patients who received teledermatologic consultation had recovered compared with 4.1% of control patients. No significant differences in patient satisfaction were found between groups. CONCLUSIONS Teledermatologic consultation offers the promise of reducing referrals to a dermatologist by 20.7%. Providing teledermatologic consultation by GPs with more extended knowledge of dermatology may further reduce the need for dermatologist referrals. Trial Registration Current Controlled Trials No. ISRCTN57478950.