Brian H. Willis

Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies: The PRISMA-DTA Statement

Importance Systematic reviews of diagnostic test accuracy synthesize data from primary diagnostic studies that have evaluated the accuracy of 1 or more index tests against a reference standard, provide estimates of test performance, allow comparisons of the accuracy of different tests, and facilitate the identification of sources of variability in test accuracy. Objective To develop the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagnostic test accuracy guideline as a stand-alone extension of the PRISMA statement. Modifications to the PRISMA statement reflect the specific requirements for reporting of systematic reviews and meta-analyses of diagnostic test accuracy studies and the abstracts for these reviews. Design Established standards from the Enhancing the Quality and Transparency of Health Research (EQUATOR) Network were followed for the development of the guideline. The original PRISMA statement was used as a framework on which to modify and add items. A group of 24 multidisciplinary experts used a systematic review of articles on existing reporting guidelines and methods, a 3-round Delphi process, a consensus meeting, pilot testing, and iterative refinement to develop the PRISMA diagnostic test accuracy guideline. The final version of the PRISMA diagnostic test accuracy guideline checklist was approved by the group. Findings The systematic review (produced 64 items) and the Delphi process (provided feedback on 7 proposed items; 1 item was later split into 2 items) identified 71 potentially relevant items for consideration. The Delphi process reduced these to 60 items that were discussed at the consensus meeting. Following the meeting, pilot testing and iterative feedback were used to generate the 27-item PRISMA diagnostic test accuracy checklist. To reflect specific or optimal contemporary systematic review methods for diagnostic test accuracy, 8 of the 27 original PRISMA items were left unchanged, 17 were modified, 2 were added, and 2 were omitted. Conclusions and Relevance The 27-item PRISMA diagnostic test accuracy checklist provides specific guidance for reporting of systematic reviews. The PRISMA diagnostic test accuracy guideline can facilitate the transparent reporting of reviews, and may assist in the evaluation of validity and applicability, enhance replicability of reviews, and make the results from systematic reviews of diagnostic test accuracy studies more useful.

Summarising and validating test accuracy results across multiple studies for use in clinical practice

Following a meta‐analysis of test accuracy studies, the translation of summary results into clinical practice is potentially problematic. The sensitivity, specificity and positive (PPV) and negative (NPV) predictive values of a test may differ substantially from the average meta‐analysis findings, because of heterogeneity. Clinicians thus need more guidance: given the meta‐analysis, is a test likely to be useful in new populations, and if so, how should test results inform the probability of existing disease (for a diagnostic test) or future adverse outcome (for a prognostic test)? We propose ways to address this. Firstly, following a meta‐analysis, we suggest deriving prediction intervals and probability statements about the potential accuracy of a test in a new population. Secondly, we suggest strategies on how clinicians should derive post‐test probabilities (PPV and NPV) in a new population based on existing meta‐analysis results and propose a cross‐validation approach for examining and comparing their calibration performance. Application is made to two clinical examples. In the first example, the joint probability that both sensitivity and specificity will be >80% in a new population is just 0.19, because of a low sensitivity. However, the summary PPV of 0.97 is high and calibrates well in new populations, with a probability of 0.78 that the true PPV will be at least 0.95. In the second example, post‐test probabilities calibrate better when tailored to the prevalence in the new population, with cross‐validation revealing a probability of 0.97 that the observed NPV will be within 10% of the predicted NPV.

Empirical evidence that disease prevalence may affect the performance of diagnostic tests with an implicit threshold: a cross-sectional study

Objective To investigate the effects that prevalence has on the diagnostic performance of junior doctors in interpreting x-rays. Design Two-armed cross-sectional design using systematic sampling. Setting Emergency department in the UK. Participants From a sample of 2593 patients (1434 men and 1159 women) taken from an unselected attending cohort between January and April 2002, 967 x-rays were analysed. The sex distribution was 558 men and 409 women, and the mean age of those receiving an x-ray was 34.6. Interventions The interpretation of x-rays by junior doctors after their triage into high- and low-prevalence populations by radiographers. Main outcome measures Sensitivity, specificity, likelihood ratios, diagnostic odds ratios and receiver operator characteristic curve. Results There were statistically significant differences in the performance characteristics of junior doctors when interpreting high-probability and low-probability x-rays. For the high- and low-probability populations, respectively, the sensitivities were 95.8% (95% CI 91.1% to 98.1%) and 78.3% (95% CI 65.7% to 87.2%) and the specificities were 56.0% (95% CI 41.9% to 69.2%) and 92.3% (95% CI 90.0% to 94.2%). Hierarchical logistic regression showed that the sensitivity did depend on the type of x-ray being interpreted but the diagnostic odds ratios did not vary significantly with prevalence, suggesting that doctors were changing their implicit threshold between the two populations along a common receiver operator characteristic curve. Conclusions This study provides evidence on how the prevalence may affect the performance of diagnostic tests with an implicit threshold and potentially includes the clinical history and examination. This has implications both for clinicians applying research findings to their practice and the design of future studies.

All-Cause Mortality in Patients With Diabetes Under Treatment With Dapagliflozin: A Population-Based, Open-Cohort Study in The Health Improvement Network Database

Context Empagliflozin was found to decrease mortality in patients with type 2 diabetes mellitus (T2DM) and a prior cardiovascular disease (CVD) event. Objectives To establish whether these benefits can be replicated in a real-world setting, should be expected with the use of dapagliflozin, and apply to T2DM patients at low risk of CVD. Design General practice, population-based, retrospective cohort study (January 2013 to September 2015). Setting The Health Improvement Network database. Participants A total of 22,124 T2DM patients (4444 exposed to dapagliflozin; 17,680 unexposed T2DM patients) matched for age, sex, body mass index, T2DM duration, and smoking. Main Outcome Measures The primary outcome was all-cause mortality (high and low risk for CVD) in the total study population, expressed as the adjusted incidence rate ratio (aIRR) with 95% confidence intervals (CIs). As a secondary analysis in the low-risk population, all-cause mortality and incident CVD were considered. Results Patients with T2DM exposed to dapagliflozin were significantly less likely to die of any cause (aIRR: 0.50; 95% CI: 0.33 to 0.75; P = 0.001). Similarly, in low-risk patients, death from any cause was significantly lower in the cohort exposed to dapagliflozin (aIRR: 0.44; 95% CI: 0.25 to 0.78; P = 0.002). The difference in the risk of incident CVD did not reach statistical significance between groups in low-risk patients (aIRR: 0.89; 95% CI: 0.61 to 1.31; P = 0.546). Conclusions Patients with T2DM who were exposed to dapagliflozin had a lower risk of death from any cause irrespective of baseline CVD status.

Opt-out organ donation: on evidence and public policy.

Deceased organ donation in the United Kingdom (UK) is governed by the Human Tissue Act 2004 and the Human Tissue Act (Scotland) 2006 and operates under an opt-in system of donation. Despite its rejection by the Organ Donation Taskforce in 2008, there have been continued calls to move to an opt-out system (sometimes, perhaps incorrectly, termed presumed consent). For example, the British Medical Association has long supported such a move and there have been sustained attempts from some academic commentators who argue for legislative change. Recently, the National Assembly for Wales voted in favour of a draft Human Transplantation (Wales) Bill, which will enable the creation of an opt-out system. This follows a referendum held in early 2011 which resulted in the Assembly gaining further law-making powers, including in relation to health and health services. The new law is slated to be on the statute books by 2015. In addition, earlier this year, the Northern Irish Executive announced plans to consult on organ donation including a possible move to an opt-out system. It has been widely assumed that the evidence available makes credible the suggestion that opt-out systems yield significantly higher rates of organ donation, something that a recent review commissioned by the Welsh Assembly supposedly reaffirms. In this essay piece, we take issue with the way in which the evidence has been used to make inferences and reach conclusions not necessarily (strongly) supported by the evidence. The available evidence is weaker than sometimes assumed, yet it is being used to support ideological, ethical and political commitments. In the absence of strong evidence, time and effort spent on legislative change misses the opportunity to focus on non-legislative action, which could have greater impact. Measuring the potential impact of legislative change

Philosophy of science and the diagnostic process

This is an overview of the principles that underpin philosophy of science and how they may provide a framework for the diagnostic process. Although philosophy dates back to antiquity, it is only more recently that philosophers have begun to enunciate the scientific method. Since Aristotle formulated deduction, other modes of reasoning including induction, inference to best explanation, falsificationism, theory-laden observations and Bayesian inference have emerged. Thus, rather than representing a single overriding dogma, the scientific method is a toolkit of ideas and principles of reasoning. Here we demonstrate that the diagnostic process is an example of science in action and is therefore subject to the principles encompassed by the scientific method. Although a number of the different forms of reasoning are used readily by clinicians in practice, without a clear understanding of their pitfalls and the assumptions on which they are based, it leaves doctors open to diagnostic error. We conclude by providing a case example from the medico-legal literature in which diagnostic errors were made, to illustrate how applying the scientific method may mitigate the chance for diagnostic error.

All-cause mortality in patients with diabetes under glucagon-like peptide-1 agonists: A population-based, open cohort study

AIM The glucagon-like peptide-1 receptor agonist (GLP1a) liraglutide has been described to benefit patients with type 2 diabetes mellitus (T2DM) at high cardiovascular risk. However, there are still uncertainties relating to these cardiovascular benefits: whether they also apply to an unselected diabetic population that includes low-risk patients, represent a class-effect, and could be observed in a real-world setting. METHODS We conducted a population-based, retrospective open cohort study using data derived from The Health Improvement Network database between Jan 2008 to Sept 2015. Patients with T2DM exposed to GLP1a (n=8345) were compared to age, gender, body mass index, duration of T2DM and smoking status-matched patients with T2DM unexposed to GLP1a (n=16,541). RESULTS Patients with diabetes receiving GLP1a were significantly less likely to die from any cause compared to matched control patients with diabetes (adjusted incidence rate ratio [aIRR]: 0.64, 95% CI: 0.56-0.74, P-value<0.0001). Similar findings were observed in low-risk patients (aIRR: 0.64, 95% CI: 0.53-0.76, P -value=0.0001). No significant difference in the risk of incident CVD was detected in the low-risk patients (aIRR: 0.93, 95% CI: 0.83-1.12). Subgroup analyses suggested that effect is persistent in the elderly or across glycated haemoglobin categories. CONCLUSIONS GLP1a treatment in a real-world setting may confer additional mortality benefit in patients with T2DM irrespective of their baseline CVD risk, age or baseline glycated haemoglobin and was sustained over the observation period.

What is the test's accuracy in my practice population? Tailored meta-analysis provides a plausible estimate

OBJECTIVES Diagnostic test accuracy studies and meta-analyses may, in some cases, provide estimates that are highly improbable in practice; tailored meta-analysis provides a potential solution. To investigate the utility of tailored meta-analysis in synthesizing estimates of a tests accuracy compared with conventional meta-analysis for three case examples. STUDY DESIGN AND SETTING MEDLINE, Embase, and CINAHL were searched for relevant studies, and routine data were collected on the test positive rate and disease prevalence from the case settings to define an applicable region for each setting. Three cases were evaluated: mammography in the NHS Breast Screening Programme, Patient Health Questionnaire-9 to screen for depression in general practice, and Centors criteria used to diagnose group A β-hemolytic streptococcus in general practice. For conventional meta-analysis, studies were selected using standard systematic review methods; for tailored meta-analysis, this selection was refined to those with results compatible with the applicable region for the setting. RESULTS In each example, studies were excluded as a result of incorporating an applicable region for the setting. Comparing tailored with conventional meta-analysis, the positive likelihood ratios (with 95% confidence intervals in brackets) were 36.5 (23.0, 57.9) and 19.8 (12.8, 30.9), respectively, for mammography and 4.89 (2.02, 11.8) and 2.35 (1.51, 3.67), respectively, for Centors criteria. This had the effect of increasing the positive predictive value from 17% to 27% for mammography and 23% to 38% for Centors criteria. CONCLUSION Tailored meta-analysis has the potential to provide a plausible estimate for a tests accuracy, which is specific to the practice setting. When compared with conventional meta-analysis, the difference may, in some cases, be sufficient to lead to different decisions on patient management.

Measuring the statistical validity of summary meta‐analysis and meta‐regression results for use in clinical practice

An important question for clinicians appraising a meta‐analysis is: are the findings likely to be valid in their own practice—does the reported effect accurately represent the effect that would occur in their own clinical population? To this end we advance the concept of statistical validity—where the parameter being estimated equals the corresponding parameter for a new independent study. Using a simple (‘leave‐one‐out’) cross‐validation technique, we demonstrate how we may test meta‐analysis estimates for statistical validity using a new validation statistic, Vn, and derive its distribution. We compare this with the usual approach of investigating heterogeneity in meta‐analyses and demonstrate the link between statistical validity and homogeneity. Using a simulation study, the properties of Vn and the Q statistic are compared for univariate random effects meta‐analysis and a tailored meta‐regression model, where information from the setting (included as model covariates) is used to calibrate the summary estimate to the setting of application. Their properties are found to be similar when there are 50 studies or more, but for fewer studies Vn has greater power but a higher type 1 error rate than Q. The power and type 1 error rate of Vn are also shown to depend on the within‐study variance, between‐study variance, study sample size, and the number of studies in the meta‐analysis. Finally, we apply Vn to two published meta‐analyses and conclude that it usefully augments standard methods when deciding upon the likely validity of summary meta‐analysis estimates in clinical practice.

The association between idiopathic thrombocytopenic purpura and cardiovascular disease : a retrospective cohort study

Essentials We estimated the cardiovascular risk of patients with idiopathic thrombocytopenic purpura (ITP). The risk of cardiovascular disease was 38% higher in ITP patients compared with controls. Among the ITP patients, splenectomy was associated with higher cardiovascular disease. Clinicians should consider cardiovascular risk when managing ITP patients.