Patrick M. Bossuyt

The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews

BackgroundIn the era of evidence based medicine, with systematic reviews as its cornerstone, adequate quality assessment tools should be available. There is currently a lack of a systematically developed and evaluated tool for the assessment of diagnostic accuracy studies. The aim of this project was to combine empirical evidence and expert opinion in a formal consensus method to develop a tool to be used in systematic reviews to assess the quality of primary studies of diagnostic accuracy.MethodsWe conducted a Delphi procedure to develop the quality assessment tool by refining an initial list of items. Members of the Delphi panel were experts in the area of diagnostic research. The results of three previously conducted reviews of the diagnostic literature were used to generate a list of potential items for inclusion in the tool and to provide an evidence base upon which to develop the tool.ResultsA total of nine experts in the field of diagnostics took part in the Delphi procedure. The Delphi procedure consisted of four rounds, after which agreement was reached on the items to be included in the tool which we have called QUADAS. The initial list of 28 items was reduced to fourteen items in the final tool. Items included covered patient spectrum, reference standard, disease progression bias, verification bias, review bias, clinical review bias, incorporation bias, test execution, study withdrawals, and indeterminate results. The QUADAS tool is presented together with guidelines for scoring each of the items included in the tool.ConclusionsThis project has produced an evidence based quality assessment tool to be used in systematic reviews of diagnostic accuracy studies. Further work to determine the usability and validity of the tool continues.

The diagnostic odds ratio: a single indicator of test performance

Diagnostic testing can be used to discriminate subjects with a target disorder from subjects without it. Several indicators of diagnostic performance have been proposed, such as sensitivity and specificity. Using paired indicators can be a disadvantage in comparing the performance of competing tests, especially if one test does not outperform the other on both indicators. Here we propose the use of the odds ratio as a single indicator of diagnostic performance. The diagnostic odds ratio is closely linked to existing indicators, it facilitates formal meta-analysis of studies on diagnostic test performance, and it is derived from logistic models, which allow for the inclusion of additional variables to correct for heterogeneity. A disadvantage is the impossibility of weighing the true positive and false positive rate separately. In this article the application of the diagnostic odds ratio in test evaluation is illustrated.

Polyp miss rate determined by tandem colonoscopy: a systematic review

BACKGROUND AND AIMS:Colonoscopy is the best available method to detect and remove colonic polyps and therefore serves as the gold standard for less invasive tests such as virtual colonoscopy. Although gastroenterologists agree that colonoscopy is not infallible, there is no clarity on the numbers and rates of missed polyps. The purpose of this systematic review was to obtain summary estimates of the polyp miss rate as determined by tandem colonoscopy.METHODS:An extensive search was performed within PUBMED, EMBASE, and the Cochrane Library databases to identify studies in which patients had undergone two same-day colonoscopies with polypectomy. Random effects models based on the binomial distribution were used to calculate pooled estimates of miss rates.RESULTS:Six studies with a total of 465 patients could be included. The pooled miss rate for polyps of any size was 22% (95% CI: 19–26%; 370/1,650 polyps). Adenoma miss rate by size was, respectively, 2.1% (95% CI: 0.3–7.3%; 2/96 adenomas ≥10 mm), 13% (95% CI: 8.0–18%; 16/124 adenomas 5–10 mm), and 26% (95% CI: 27–35%; 151/587 adenomas 1–5 mm). Three studies reported data on nonadenomatous polyps: zero of eight nonadenomatous polyps ≥10 mm were missed (0%; 95% CI: 0–36.9%) and 83 of 384 nonadenomatous polyps <10 mm were missed (22%; 95% CI: 18–26%).CONCLUSIONS:Colonoscopy rarely misses polyps ≥10 mm, but the miss rate increases significantly in smaller sized polyps. The available evidence is based on a small number of studies with heterogeneous study designs and inclusion criteria.

Preoperative biliary drainage for cancer of the head of the pancreas

BACKGROUND The benefits of preoperative biliary drainage, which was introduced to improve the postoperative outcome in patients with obstructive jaundice caused by a tumor of the pancreatic head, are unclear. METHODS In this multicenter, randomized trial, we compared preoperative biliary drainage with surgery alone for patients with cancer of the pancreatic head. Patients with obstructive jaundice and a bilirubin level of 40 to 250 micromol per liter (2.3 to 14.6 mg per deciliter) were randomly assigned to undergo either preoperative biliary drainage for 4 to 6 weeks, followed by surgery, or surgery alone within 1 week after diagnosis. Preoperative biliary drainage was attempted primarily with the placement of an endoprosthesis by means of endoscopic retrograde cholangiopancreatography. The primary outcome was the rate of serious complications within 120 days after randomization. RESULTS We enrolled 202 patients; 96 were assigned to undergo early surgery and 106 to undergo preoperative biliary drainage; 6 patients were excluded from the analysis. The rates of serious complications were 39% (37 patients) in the early-surgery group and 74% (75 patients) in the biliary-drainage group (relative risk in the early-surgery group, 0.54; 95% confidence interval [CI], 0.41 to 0.71; P<0.001). Preoperative biliary drainage was successful in 96 patients (94%) after one or more attempts, with complications in 47 patients (46%). Surgery-related complications occurred in 35 patients (37%) in the early-surgery group and in 48 patients (47%) in the biliary-drainage group (relative risk, 0.79; 95% CI, 0.57 to 1.11; P=0.14). Mortality and the length of hospital stay did not differ significantly between the two groups. CONCLUSIONS Routine preoperative biliary drainage in patients undergoing surgery for cancer of the pancreatic head increases the rate of complications. (Current Controlled Trials number, ISRCTN31939699.)

Effects of selective decontamination of digestive tract on mortality and acquisition of resistant bacteria in intensive care: a randomised controlled trial

BACKGROUND Selective decontamination of the digestive tract (SDD) is an infection-prevention regimen used in critically ill patients. We assessed the effects of SDD on intensive-care-unit (ICU) and hospital mortality, and on the acquisition of resistant bacteria in adult patients admitted to intensive care. METHODS We did a prospective, controlled, randomised, unblinded clinical trial. 934 patients admitted to a surgical and medical ICU were randomly assigned oral and enteral polymyxin E, tobramycin, and amphotericin B combined with an initial 4-day course of intravenous cefotaxime (SDD group n=466), or standard treatment (controls n=468). Primary endpoints were ICU and hospital mortality and the acquisition of resistant bacteria. FINDINGS In the SDD group 69 (15%) patients died in the ICU compared with 107 (23%) in the control group (p=0.002). Hospital mortality was lower in the SDD groups than in the control group (113 [24%] vs 146 [31%], p=0.02). During their stay in intensive care, colonisation with gram-negative bacteria resistant to ceftazidime, ciprofloxacin, imipenem, polymyxin E, or tobramycin occurred in 61 (16%) of 378 SDD patients and in 104 (26%) of 395 patients in the control group (p=0.001). Colonisation with vancomycin-resistant enterococcus occurred in five (1%) SDD patients and in four (1%) controls (p=1.0). No patient in either group was colonised with meticillin-resistant Staphylococcus aureus. INTERPRETATION In a setting with low prevalence of vancomycin-resistant enterococcus and meticillin-resistant S aureus, SDD can decrease ICU and hospital mortality and colonisation with resistant gram-negative aerobic bacteria.

Reducing waste from incomplete or unusable reports of biomedical research

Research publication can both communicate and miscommunicate. Unless research is adequately reported, the time and resources invested in the conduct of research is wasted. Reporting guidelines such as CONSORT, STARD, PRISMA, and ARRIVE aim to improve the quality of research reports, but all are much less adopted and adhered to than they should be. Adequate reports of research should clearly describe which questions were addressed and why, what was done, what was shown, and what the findings mean. However, substantial failures occur in each of these elements. For example, studies of published trial reports showed that the poor description of interventions meant that 40-89% were non-replicable; comparisons of protocols with publications showed that most studies had at least one primary outcome changed, introduced, or omitted; and investigators of new trials rarely set their findings in the context of a systematic review, and cited a very small and biased selection of previous relevant trials. Although best documented in reports of controlled trials, inadequate reporting occurs in all types of studies-animal and other preclinical studies, diagnostic studies, epidemiological studies, clinical prediction research, surveys, and qualitative studies. In this report, and in the Series more generally, we point to a waste at all stages in medical research. Although a more nuanced understanding of the complex systems involved in the conduct, writing, and publication of research is desirable, some immediate action can be taken to improve the reporting of research. Evidence for some recommendations is clear: change the current system of research rewards and regulations to encourage better and more complete reporting, and fund the development and maintenance of infrastructure to support better reporting, linkage, and archiving of all elements of research. However, the high amount of waste also warrants future investment in the monitoring of and research into reporting of research, and active implementation of the findings to ensure that research reports better address the needs of the range of research users.

Pivotal evaluation of the accuracy of a biomarker used for classification or prediction: standards for study design.

Research methods for biomarker evaluation lag behind those for evaluating therapeutic treatments. Although a phased approach to development of biomarkers exists and guidelines are available for reporting study results, a coherent and comprehensive set of guidelines for study design has not been delineated. We describe a nested case–control study design that involves prospective collection of specimens before outcome ascertainment from a study cohort that is relevant to the clinical application. The biomarker is assayed in a blinded fashion on specimens from randomly selected case patients and control subjects in the study cohort. We separately describe aspects of the design that relate to the clinical context, biomarker performance criteria, the biomarker test, and study size. The design can be applied to studies of biomarkers intended for use in disease diagnosis, screening, or prognosis. Common biases that pervade the biomarker research literature would be eliminated if these rigorous standards were followed.

Evidence of bias and variation in diagnostic accuracy studies

Background: Studies with methodologic shortcomings can overestimate the accuracy of a medical test. We sought to determine and compare the direction and magnitude of the effects of a number of potential sources of bias and variation in studies on estimates of diagnostic accuracy. Methods: We identified meta-analyses of the diagnostic accuracy of tests through an electronic search of the databases MEDLINE, EMBASE, DARE and MEDION (1999–2002). We included meta-analyses with at least 10 primary studies without preselection based on design features. Pairs of reviewers independently extracted study characteristics and original data from the primary studies. We used a multivariable meta-epidemiologic regression model to investigate the direction and strength of the association between 15 study features on estimates of diagnostic accuracy. Results: We selected 31 meta-analyses with 487 primary studies of test evaluations. Only 1 study had no design deficiencies. The quality of reporting was poor in most of the studies. We found significantly higher estimates of diagnostic accuracy in studies with nonconsecutive inclusion of patients (relative diagnostic odds ratio [RDOR] 1.5, 95% confidence interval [CI] 1.0–2.1) and retrospective data collection (RDOR 1.6, 95% CI 1.1–2.2). The estimates were highest in studies that had severe cases and healthy controls (RDOR 4.9, 95% CI 0.6–37.3). Studies that selected patients based on whether they had been referred for the index test, rather than on clinical symptoms, produced significantly lower estimates of diagnostic accuracy (RDOR 0.5, 95% CI 0.3–0.9). The variance between meta-analyses of the effect of design features was large to moderate for type of design (cohort v. case–control), the use of composite reference standards and the use of differential verification; the variance was close to zero for the other design features. Interpretation: Shortcomings in study design can affect estimates of diagnostic accuracy, but the magnitude of the effect may vary from one situation to another. Design features and clinical characteristics of patient groups should be carefully considered by researchers when designing new studies and by readers when appraising the results of such studies. Unfortunately, incomplete reporting hampers the evaluation of potential sources of bias in diagnostic accuracy studies.

Randomised trial of endoscopic balloon dilation versus endoscopic sphincterotomy for removal of bileduct stones

BACKGROUND Endoscopic sphincterotomy (EST) for the removal of bileduct stones is associated with acute complications and a permanent loss of biliary-sphincter function. Endoscopic balloon dilation (EBD) causes less trauma to the biliary sphincter, but may be less effective in allowing stone removal. METHODS 218 consecutive patients with bileduct stones on endoscopic retrograde cholangiopancreatography (ERCP) were enrolled. 202 who met all eligibility criteria were randomly assigned EST or EBD. The patients were observed in hospital for at least 24 h and followed up at 1 month and 6 months. Complications were classified by an expert panel unaware of treatment allocation and outcome. Analysis was done by intention to treat. FINDINGS After a single ERCP, all stones were removed from 92 (91%) of 101 patients assigned EST and 90 (89%) of 101 assigned EBD (p = 0.81); in nine of the latter, successful removal required additional EST. Mechanical lithotripsy was used to fragment stones in 31 EBD procedures and 13 EST procedures (p < 0.005). Early complications (before 15 days) occurred in 24 EST patients and 17 EBD patients (p = 0.29). One patient died of retroperitoneal perforation after EBD. Four patients had bleeding after EST. Seven patients in each group had pancreatitis. Complications during follow-up occurred in 23 EST patients and 18 EBD patients (p = 0.48). Acute cholecystitis was observed in seven EST patients and one EBD patient (p < 0.05). INTERPRETATION The success rate of EBD was similar to that of EST. We found there is no evidence of the previously suggested higher risk of pancreatitis with EBD and suggest that EBD is preferred in patients at risk of bleeding after EST. Preservation of biliary-sphincter function after EBD may prevent long-term complications and reduce the risk of acute cholecystitis during follow-up. This procedure is a valuable alternative to EST in patients with bileduct stones.

Diagnostic accuracy of D-dimer test for exclusion of venous thromboembolism: a systematic review

Summary.  Background: The reported diagnostic accuracy of the D‐dimer test for exclusion of deep vein thrombosis (DVT) and pulmonary embolism (PE) varies. It is unknown to what extent this is due to differences in study design or patient groups, or to genuine differences between D‐dimer assays. Methods: Studies evaluating the diagnostic accuracy of the D‐dimer test in the diagnosis of venous thromboembolism were systematically searched for in the MEDLINE and EMBASE databases up to March 2005. Reference lists of all included studies and of reviews related to the topic of the present meta‐analysis were manually searched for other additional potentially eligible studies. Two reviewers independently extracted study characteristics using standardized forms. Results: In total, 217 D‐dimer test evaluations for DVT and 111 for PE were analyzed. Several study design characteristics were associated with systematic differences in diagnostic accuracy. After adjustment for these features, the sensitivities of the D‐dimer enzyme‐linked immunofluorescence assay (ELFA) (DVT 96%; PE 97%), microplate enzyme‐linked immunosorbent assay (ELISA) (DVT 94%; PE 95%), and latex quantitative assay (DVT 93%; PE 95%) were superior to those of the whole‐blood D‐dimer assay (DVT 83%; PE 87%), latex semiquantitative assay (DVT 85%; PE 88%) and latex qualitative assay (DVT 69%; PE 75%). The latex qualitative and whole‐blood D‐dimer assays had the highest specificities (DVT 99%, 71%; PE 99%, 69%). Conclusions: Compared to other D‐dimer assays, the ELFA, microplate ELISA and latex quantitative assays have higher sensitivity but lower specificity, resulting in a more confident exclusion of the disease at the expense of more additional imaging testing. These conclusions are based on the most up‐to‐date and extensive systematic review of the topic area, including 184 articles, with 328 D‐dimer test evaluations.