Jim Briggs

A Pragmatist Argument for Mixed Methodology in Medical Informatics

The aim of this article is to advance the case of pragmatism as a research philosophy and to illustrate its applicability as a mixed methodology perspective in medical informatics. Epistemology is empirical not foundational. Pragmatism offers a practical starting point for a pluralist methodology. Medical practice is pragmatist, empirical, and situated. Medical informatics is a hybrid sociotechnical field that requires multimethod research.

Decision-tree early warning score (DTEWS) validates the design of the National Early Warning Score (NEWS).

AIM OF STUDY To compare the performance of a human-generated, trial and error-optimised early warning score (EWS), i.e., National Early Warning Score (NEWS), with one generated entirely algorithmically using Decision Tree (DT) analysis. MATERIALS AND METHODS We used DT analysis to construct a decision-tree EWS (DTEWS) from a database of 198,755 vital signs observation sets collected from 35,585 consecutive, completed acute medical admissions. We evaluated the ability of DTEWS to discriminate patients at risk of cardiac arrest, unanticipated intensive care unit admission or death, each within 24h of a given vital signs observation. We compared the performance of DTEWS and NEWS using the area under the receiver-operating characteristic (AUROC) curve. RESULTS The structures of DTEWS and NEWS were very similar. The AUROC (95% CI) for DTEWS for cardiac arrest, unanticipated ICU admission, death, and any of the outcomes, all within 24h, were 0.708 (0.669-0.747), 0.862 (0.852-0.872), 0.899 (0.892-0.907), and 0.877 (0.870-0.883), respectively. Values for NEWS were 0.722 (0.685-0.759) [cardiac arrest], 0.857 (0.847-0.868) [unanticipated ICU admission}, 0.894 (0.887-0.902) [death], and 0.873 (0.866-0.879) [any outcome]. CONCLUSIONS The decision-tree technique independently validates the composition and weightings of NEWS. The DT approach quickly provided an almost identical EWS to NEWS, although one that admittedly would benefit from fine-tuning using clinical knowledge. We believe that DT analysis could be used to quickly develop candidate models for disease-specific EWSs, which may be required in future.

Development and validation of a decision tree early warning score based on routine laboratory test results for the discrimination of hospital mortality in emergency medical admissions

AIM OF STUDY To build an early warning score (EWS) based exclusively on routinely undertaken laboratory tests that might provide early discrimination of in-hospital death and could be easily implemented on paper. MATERIALS AND METHODS Using a database of combined haematology and biochemistry results for 86,472 discharged adult patients for whom the admission specialty was Medicine, we used decision tree (DT) analysis to generate a laboratory decision tree early warning score (LDT-EWS) for each gender. LDT-EWS was developed for a single set (n=3496) (Q1) and validated in 22 other discrete sets each of three months long (Q2, Q3…Q23) (total n=82,976; range of n=3428 to 4093) by testing its ability to discriminate in-hospital death using the area under the receiver-operating characteristic (AUROC) curve. RESULTS The data generated slightly different models for male and female patients. The ranges of AUROC values (95% CI) for LDT-EWS with in-hospital death as the outcome for the validation sets Q2-Q23 were: 0.755 (0.727-0.783) (Q16) to 0.801 (0.776-0.826) [all patients combined, n=82,976]; 0.744 (0.704-0.784, Q16) to 0.824 (0.792-0.856, Q2) [39,591 males]; and 0.742 (0.707-0.777, Q10) to 0.826 (0.796-0.856, Q12) [43,385 females]. CONCLUSIONS This study provides evidence that the results of commonly measured laboratory tests collected soon after hospital admission can be represented in a simple, paper-based EWS (LDT-EWS) to discriminate in-hospital mortality. We hypothesise that, with appropriate modification, it might be possible to extend the use of LDT-EWS throughout the patients hospital stay.

Which is more useful in predicting hospital mortality - dichotomised blood test results or actual test values? a retrospective study in two hospitals

Background Routine blood tests are an integral part of clinical medicine and in interpreting blood test results clinicians have two broad options. (1) Dichotomise the blood tests into normal/abnormal or (2) use the actual values and overlook the reference values. We refer to these as the “binary” and the “non-binary” strategy respectively. We investigate which strategy is better at predicting the risk of death in hospital based on seven routinely undertaken blood tests (albumin, creatinine, haemoglobin, potassium, sodium, urea, and white blood cell count) using tree models to implement the two strategies. Methodology A retrospective database study of emergency admissions to an acute hospital during April 2009 to March 2010, involving 10,050 emergency admissions with routine blood tests undertaken within 24 hours of admission. We compared the area under the Receiver Operating Characteristics (ROC) curve for predicting in-hospital mortality using the binary and non-binary strategy. Results The mortality rate was 6.98% (701/10050). The mean predicted risk of death in those who died was significantly (p-value <0.0001) lower using the binary strategy (risk = 0.181 95%CI: 0.193 to 0.210) versus the non-binary strategy (risk = 0.222 95%CI: 0.194 to 0.251), representing a risk difference of 28.74 deaths in the deceased patients (n = 701). The binary strategy had a significantly (p-value <0.0001) lower area under the ROC curve of 0.832 (95% CI: 0.819 to 0.845) versus the non-binary strategy (0.853 95% CI: 0.840 to 0.867). Similar results were obtained using data from another hospital. Conclusions Dichotomising routine blood test results is less accurate in predicting in-hospital mortality than using actual test values because it underestimates the risk of death in patients who died. Further research into the use of actual blood test values in clinical decision making is required especially as the infrastructure to implement this potentially promising strategy already exists in most hospitals.

A Comparison of the Ability of the Physiologic Components of Medical Emergency Team Criteria and the U.k. National Early Warning Score to Discriminate Patients at Risk of a Range of Adverse Clinical Outcomes.

Objective:To compare the ability of medical emergency team criteria and the National Early Warning Score to discriminate cardiac arrest, unanticipated ICU admission and death within 24 hours of a vital signs measurement, and to quantify the associated workload. Design:Retrospective cohort study. Setting:A large U.K. National Health Service District General Hospital. Patients:Adults hospitalized from May 25, 2011, to December 31, 2013. Interventions:None. Measurements and Main Results:We applied the National Early Warning Score and 44 sets of medical emergency team criteria to a database of 2,245,778 vital signs sets (103,998 admissions). The National Early Warning Score’s performance was assessed using the area under the receiver-operating characteristic curve and compared with sensitivity/specificity for different medical emergency team criteria. Area under the receiver-operating characteristic curve (95% CI) for the National Early Warning Score for the combined outcome (i.e., death, cardiac arrest, or unanticipated ICU admission) was 0.88 (0.88–0.88). A National Early Warning Score value of 7 had sensitivity/specificity values of 44.5% and 97.4%, respectively. For the 44 sets of medical emergency team criteria studied, sensitivity ranged from 19.6% to 71.2% and specificity from 71.5% to 98.5%. For all outcomes, the position of the National Early Warning Score receiver-operating characteristic curve was above and to the left of all medical emergency team criteria points, indicating better discrimination. Similarly, the positions of all medical emergency team criteria points were above and to the left of the National Early Warning Score efficiency curve, indicating higher workloads (trigger rates). Conclusions:When medical emergency team systems are compared to a National Early Warning Score value of greater than or equal to 7, some medical emergency team systems have a higher sensitivity than National Early Warning Score values of greater than or equal to 7. However, all of these medical emergency team systems have a lower specificity and would generate greater workloads.

Measuring clinical performance using routinely collected clinical data

Following the well-publicized problems with paediatric cardiac surgery at the Bristol Royal Infirmary, there is wide public interest in measures of hospital performance. The Kennedy report on the BRI events suggested that such measures should be meaningful to the public, case-mix-adjusted, and based on data collected as part of routine clinical care. We have found that it is possible to predict in-hospital mortality (a measure readily understood by the public) using simple routine data—age, mode of admission, sex, and routine blood test results. The clinical data items can be obtained at a single venesection, are commonly collected in the routine care of patients, are already stored on hospital core IT systems, and so place no extra burden on the clinical staff providing care. Such risk models could provide a metric for use in evidence-based clinical performance management. National application is logistically feasible.

STAT-HI: A Socio-Technical Assessment Tool for Health Informatics Implementations

This paper proposes a socio-technical assessment tool (STAT-HI) for health informatics implementations. We explore why even projects allegedly using sound methodologies repeatedly fail to give adequate attention to socio-technical issues, and we present an initial draft of a structured assessment tool for health informatics implementation that encapsulates socio-technical good practice. Further work is proposed to enrich and validate the proposed instrument. This proposal was presented for discussion at a meeting of the UK Faculty of Health Informatics in December 2009.

The ISABEL user survey

ISABEL is a web-based clinical decision-support system for use by healthcare professionals. The Web site has been developed by the ISABEL Medical Charity. The system has come to the attention of the Department of Health, which is examining its potential effectiveness in the wider clinical context and exploring options for promoting its wider use in the NHS. The objectives of the work reported here were to review the existing use of ISABEL and to identify impediments to its development. A questionnaire was sent by e-mail to selected users of the system. Based on an analysis of the results (n = 518), we found ISABEL to be a useful tool with many users. We believe that there is evidence of its success sufficient to support its continued availability and development. However, the largest hurdles to its increased use are systemic ones within the NHS and the way services are delivered.

The association between nurse staffing and omissions in nursing care: a systematic review

Abstract Aims To identify nursing care most frequently missed in acute adult inpatient wards and to determine evidence for the association of missed care with nurse staffing. Background Research has established associations between nurse staffing levels and adverse patient outcomes including in‐hospital mortality. However, the causal nature of this relationship is uncertain and omissions of nursing care (referred as missed care, care left undone or rationed care) have been proposed as a factor which may provide a more direct indicator of nurse staffing adequacy. Design Systematic review. Data Sources We searched the Cochrane Library, CINAHL, Embase and Medline for quantitative studies of associations between staffing and missed care. We searched key journals, personal libraries and reference lists of articles. Review Methods Two reviewers independently selected studies. Quality appraisal was based on the National Institute for Health and Care Excellence quality appraisal checklist for studies reporting correlations and associations. Data were abstracted on study design, missed care prevalence and measures of association. Synthesis was narrative. Results Eighteen studies gave subjective reports of missed care. Seventy‐five per cent or more nurses reported omitting some care. Fourteen studies found low nurse staffing levels were significantly associated with higher reports of missed care. There was little evidence that adding support workers to the team reduced missed care. Conclusions Low Registered Nurse staffing is associated with reports of missed nursing care in hospitals. Missed care is a promising indicator of nurse staffing adequacy. The extent to which the relationships observed represent actual failures, is yet to be investigated.

Comparison of the National Early Warning Score in non-elective medical and surgical patients

The National Early Warning Score (NEWS) is used to identify deteriorating patients in hospital. NEWS is a better discriminator of outcomes than other early warning scores in acute medical admissions, but it has not been evaluated in a surgical population. The study aims were to evaluate the ability of NEWS to discriminate cardiac arrest, death and unanticipated ICU admission in patients admitted to surgical specialties, and to compare the performance of NEWS in admissions to medical and surgical specialties.