Les Irwig

The STARD Statement for Reporting Studies of Diagnostic Accuracy: Explanation and Elaboration

The quality of reporting of studies of diagnostic accuracy is less than optimal. Complete and accurate reporting is necessary to enable readers to assess the potential for bias in the study and to evaluate the generalizability of the results. A group of scientists and editors has developed the STARD (Standards for Reporting of Diagnostic Accuracy) statement to improve the reporting the quality of reporting of studies of diagnostic accuracy. The statement consists of a checklist of 25 items and flow diagram that authors can use to ensure that all relevant information is present. This explanatory document aims to facilitate the use, understanding, and dissemination of the checklist. The document contains a clarification of the meaning, rationale, and optimal use of each item on the checklist, as well as a short summary of the available evidence on bias and applicability. The STARD statement, checklist, flowchart, and this explanation and elaboration document should be useful resources to improve reporting of diagnostic accuracy studies. Complete and informative reporting can only lead to better decisions in health care.

The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration.

Introduction In studies of diagnostic accuracy, results from one or more tests are compared with the results obtained with the reference standard on the same subjects. Such accuracy studies are a vital step in the evaluation of new and existing diagnostic technologies (1, 2). Several factors threaten the internal and external validity of a study of diagnostic accuracy (3-8). Some of these factors have to do with the design of such studies, others with the selection of patients, the execution of the tests, or the analysis of the data. In a study involving several meta-analyses a number of design deficiencies were shown to be related to overly optimistic estimates of diagnostic accuracy (9). Exaggerated results from poorly designed studies can trigger premature adoption of diagnostic tests and can mislead physicians to incorrect decisions about the care for individual patients. Reviewers and other readers of diagnostic studies must therefore be aware of the potential for bias and a possible lack of applicability. A survey of studies of diagnostic accuracy published in four major medical journals between 1978 and 1993 revealed that the methodological quality was mediocre at best (8). Furthermore, this review showed that information on key elements of design, conduct, and analysis of diagnostic studies was often not reported (8). To improve the quality of reporting of studies of diagnostic accuracy the Standards for Reporting of Diagnostic Accuracy (STARD) initiative was started. The objective of the STARD initiative is to improve the quality of reporting of studies of diagnostic accuracy. Complete and accurate reporting allows the reader to detect the potential for bias in the study and to judge the generalizability and applicability of the results. For this purpose, the STARD project group has developed a single-page checklist. Where possible, the decision to include items in the checklist was based on evidence linking these items to bias, variability in results, or limitations of the applicability of results to other settings. The checklist can be used to verify that all essential elements are included in the report of a study. This explanatory document aims to facilitate the use, understanding, and dissemination of the checklist. The document contains a clarification of the meaning, rationale, and optimal use of each item on the checklist, as well as a short summary of the available evidence on bias and applicability. The first part of this document contains a summary of the design and terminology of diagnostic accuracy studies. The second part contains an item-by-item discussion with examples. Studies of Diagnostic Accuracy Studies of diagnostic accuracy have a common basic structure (10). One or more tests are evaluated, with the purpose of detecting or predicting a target condition. The target condition can refer to a particular disease, a disease stage, a health status, or any other identifiable condition within a patient, such as staging a disease already known to be present, or a health condition that should prompt clinical action, such as the initiation, modification, or termination of treatment. Here test refers to any method for obtaining additional information on a patients health status. This includes laboratory tests, imaging tests, function tests, pathology, history, and physical examination. In a diagnostic accuracy study, the test under evaluationreferred to here as the index testis applied to a series of subjects. The results obtained with the index test are compared with the results of the reference standard, obtained in the same subjects. In this framework, the reference standard is the best available method for establishing the presence or absence of the target condition. The reference standard can be a single test, or a combination of methods and techniques, including clinical follow-up of tested subjects. The term accuracy refers to the amount of agreement between the results from the index test and those from the reference standard. Diagnostic accuracy can be expressed in a number of ways, including sensitivityspecificity pairs, likelihood ratios, diagnostic odds ratios, and areas under ROC [receiver-operating characteristic] curves (11, 12). Study Question, Design, and Potential for Bias Early in the evaluation of a test, the author may simply want to know if the test is able to discriminate. The appropriate early question may be Do the test results in patients with the target condition differ from the results in healthy people? If preliminary studies answer this question affirmatively, the next study question is, Are patients with specific test results more likely to have the target disorder than similar patients with other test results? The usual study design to answer this is to apply the index test and the reference standard to a number of patients who are suspected of the target condition. Some study designs are more prone to bias and have a more limited applicability than others. In this article, the term bias refers to difference between the observed measures of test performance and the true measures. No single design is guaranteed to be both feasible and able to provide valid, informative, and relevant answers with optimal precision to all study questions. For each study, the reader must judge the relevance, the potential for bias, and the limitations to applicability, making full and transparent reporting critical. For this reason, checklist items refer to the research question that prompted the study of diagnostic accuracy and ask for an explicit and complete description of the study design and results. Variability Measures of test accuracy may vary from study to study. Variability may reflect differences in patient groups, differences in setting, differences in definition of the target condition, and differences in test protocols or in criteria for test positivity (13). For example, bias may occur if a test is evaluated under circumstances that do not correspond to those of the research question. Examples are evaluating a screening test for early disease in patients with advanced stages of the disease and evaluating a physicians office test device in the specialty department of a university hospital. The checklist contains a number of items to make sure that a study report contains a clear description of the inclusion criteria for patients, the testing protocols and the criteria for positivity, as well as an adequate account of subjects included in the study and their results. These items will enable readers to judge if the study results apply to their circumstances. Items in the Checklist The next section contains a point-by-point discussion of the items on the checklist. The order of the items corresponds to the sequence used in many publications of diagnostic accuracy studies. Specific requirements made by journals could lead to a different order. Item 1. Identify the Article as a Study of Diagnostic Accuracy (Recommend MeSH Heading Sensitivity and Specificity) Example (an Excerpt from a Structured Abstract) Purpose: To determine the sensitivity and specificity of computed tomographic colonography for colorectal polyp and cancer detection by using colonoscopy as the reference standard (14). Electronic databases have become indispensable tools to identify studies. To facilitate retrieval of their study, authors should explicitly identify it as a report of a study of diagnostic accuracy. We recommend the use of the term diagnostic accuracy in the title or abstract of a report that compares the results of one or more index tests with the results of a reference standard. In 1991 the National Library of Medicines MEDLINE database introduced a specific keyword (MeSH heading) for diagnostic studies: Sensitivity and Specificity. Using this keyword to search for studies of diagnostic accuracy remains problematic (15-19). In a selected set of MEDLINE journals covering publications between 1992 through 1995, the use of the MeSH heading Sensitivity and Specificity identified only 51% of all studies of diagnostic accuracy and incorrectly identified many articles that were not reports of studies on diagnostic accuracy (18). In the example, the authors used the more general term Performance Characteristics of CT Colonography in the title. The purpose section of the structured abstract explicitly mentions sensitivity and specificity. The MEDLINE record for this paper contains the MeSH Sensitivity and Specificity. Item 2. State the Research Questions or Study Aims, Such as Estimating Diagnostic Accuracy or Comparing Accuracy between Tests or across Participant Groups Example Invasive x-ray coronary angiography remains the gold standard for the identification of clinically significant coronary artery disease . A noninvasive test would be desirable. Coronary magnetic resonance angiography performed while the patient is breathing freely has reached sufficient technical maturity to allow more widespread application with a standardized protocol. Therefore, we conducted a study to determine the [accuracy] of coronary magnetic resonance angiography in the diagnosis of native-vessel coronary artery disease (20). The Helsinki Declaration states that biomedical research involving people should be based on a thorough knowledge of the scientific literature (21). In the introduction of scientific reports authors describe the scientific background, previous work on the subject, the remaining uncertainty, and, hence, the rationale for their study. Clearly specified research questions help the readers to judge the appropriateness of the study design and data analysis. A single general description, such as diagnostic value or clinical usefulness, is usually not very helpful to the readers. In the example, the authors use the introduction section of their paper to describe the potential of coronary magnetic resonance angiography as a non-invasive alternative to conventional x-ray angiography in the diagn

Cochrane systematic review of colorectal cancer screening using the fecal occult blood test (hemoccult): an update.

BACKGROUND AND AIMS:Reducing mortality from colorectal cancer (CRC) may be achieved by the introduction of population-based screening programs. The aim of the systematic review was to update previous research to determine whether screening for CRC using the fecal occult blood test (FOBT) reduces CRC mortality and to consider the benefits, harms, and potential consequences of screening.METHODS:We searched eight electronic databases (Cochrane Library, MEDLINE, EMBASE, CINAHL, PsychINFO, AMED, SIGLE, and HMIC). We identified nine articles describing four randomized controlled trials (RCTs) involving over 320,000 participants with follow-up ranging from 8 to 18 yr. The primary analyses used intention to screen and a secondary analysis adjusted for nonattendance. We calculated the relative risks and risk differences for each trial, and then overall, using fixed and random effects models.RESULTS:Combined results from the four eligible RCTs indicated that screening had a 16% reduction in the relative risk (RR) of CRC mortality (RR 0.84, 95% confidence interval [CI] 0.78–0.90). There was a 15% RR reduction (RR 0.85, 95% CI 0.78–0.92) in CRC mortality for studies that used biennial screening. When adjusted for screening attendance in the individual studies, there was a 25% RR reduction (RR 0.75, 95% CI 0.66–0.84) for those attending at least one round of screening using the FOBT. There was no difference in all-cause mortality (RR 1.00, 95% CI 0.99–1.02) or all-cause mortality excluding CRC (RR 1.01, 95% CI 1.00–1.03).CONCLUSIONS:The present review includes seven new publications and unpublished data concerning CRC screening using FOBT. This review confirms previous research demonstrating that FOBT screening reduces the risk of CRC mortality. The results also indicate that there is no difference in all-cause mortality between the screened and nonscreened populations.

Accuracy and Surgical Impact of Magnetic Resonance Imaging in Breast Cancer Staging: Systematic Review and Meta-Analysis in Detection of Multifocal and Multicentric Cancer

PURPOSE We review the evidence on magnetic resonance imaging (MRI) in staging the affected breast to determine its accuracy and impact on treatment. METHODS Systematic review and meta-analysis of the accuracy of MRI in detection of multifocal (MF) and/or multicentric (MC) cancer not identified on conventional imaging. We estimated summary receiver operating characteristic curves, positive predictive value (PPV), true-positive (TP) to false positive (FP) ratio, and examined their variability according to quality criteria. Pooled estimates of the proportion of women whose surgery was altered were calculated. Results Data from 19 studies showed MRI detects additional disease in 16% of women with breast cancer (N = 2,610). MRI incremental accuracy differed according to the reference standard (RS; P = .016) decreasing from 99% to 86% as the quality of the RS increased. Summary PPV was 66% (95% CI, 52% to 77%) and TP:FP ratio was 1.91 (95% CI, 1.09 to 3.34). Conversion from wide local excision (WLE) to mastectomy was 8.1% (95% CI, 5.9 to 11.3), from WLE to more extensive surgery was 11.3% in MF/MC disease (95% CI, 6.8 to 18.3). Due to MRI-detected lesions (in women who did not have additional malignancy on histology) conversion from WLE to mastectomy was 1.1% (95% CI, 0.3 to 3.6) and from WLE to more extensive surgery was 5.5% (95% CI, 3.1 to 9.5). CONCLUSION MRI staging causes more extensive breast surgery in an important proportion of women by identifying additional cancer, however there is a need to reduce FP MRI detection. Randomized trials are needed to determine the clinical value of detecting additional disease which changes surgical treatment in women with apparently localized breast cancer.

STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies

Incomplete reporting has been identified as a major source of avoidable waste in biomedical research. Essential information is often not provided in study reports, impeding the identification, critical appraisal, and replication of studies. To improve the quality of reporting of diagnostic accuracy studies, the Standards for Reporting Diagnostic Accuracy (STARD) statement was developed. Here we present STARD 2015, an updated list of 30 essential items that should be included in every report of a diagnostic accuracy study. This update incorporates recent evidence about sources of bias and variability in diagnostic accuracy and is intended to facilitate the use of STARD. As such, STARD 2015 may help to improve completeness and transparency in reporting of diagnostic accuracy studies.

Meta-analytic methods for diagnostic test accuracy.

Meta-analyses of diagnostic test accuracy are uncommon and often based on separate pooling of sensitivity and specificity, which can lead to biased estimates. Recently, several appropriate methods have been developed for meta-analysing diagnostic test data from primary studies. Primary studies usually only provide binary test data, for which Moses et al. have developed a method to estimate Summary Receiver Operating Characteristic Curves, thereby taking account of possible test threshold differences between studies. Several methods are also available for analysing multicategory and continuous test data. The usefulness of applying these methods is constrained by publication bias and the generally poor quality of primary studies of diagnostic test accuracy. Meta-analysts need to highlight important defects in quality and how they affect summary estimates to ensure that better primary studies are available for meta-analysis in the future.

Meta-analysis of the impact of surgical margins on local recurrence in women with early-stage invasive breast cancer treated with breast-conserving therapy

PURPOSE There is no consensus on what constitutes adequate negative margins in breast-conserving therapy (BCT). We review the evidence on surgical margins in BCT for early-stage invasive breast cancer. METHODS Meta-analysis of studies reporting local recurrence (LR) relative to quantified final microscopic margin status and the threshold distance for negative margins. The proportion of LR was modelled using random effects logistic meta-regression. RESULTS Based on 21 studies (LR in 1,026 of 14,571 subjects) the odds of LR were associated with margin status [model 1: odds ratio (OR) = 2.02 for positive/close versus negative; model 2: OR = 1.80 for close versus negative, 2.42 for positive versus negative (P<0.001 both models)] but not with margin distance [1mm versus 2mm versus 5mm (P > 0.10 both models)], adjusting for median follow-up time. However, there was weak evidence in both models that the odds of LR decreased as the threshold distance for declaring negative margins increased. This bordered significance in model 2 [OR for 1mm, 2mm, 5mm: 1.0, 0.75, 0.51 (P = 0.097 for trend)], and was not significant in model 1 [OR for 1mm, 2mm, 5mm: 1.0, 0.85, 0.58 (P = 0.11 for trend)] but was evident when one study (of women ≤ 40 years) was excluded from this model [OR for 1mm, 2mm, 5mm: 1.0, 0.72, 0.52 (P = 0.058 for trend)]: this trend was rendered insignificant by adjustment for the proportion of subjects receiving a radiation boost or the proportion of subjects receiving endocrine therapy. CONCLUSIONS Margin status has a prognostic effect in all women treated for invasive breast cancer; increasing the threshold distance for declaring negative margins is weakly associated with reduced odds of LR, however adjustment for covariates (adjuvant therapy) removes the significance of this effect. Adoption of wider margins, relative to narrower widths, for declaring negative margins is unlikely to a have substantial additional benefit for long-term local control in BCT.

Antibiotic Prophylaxis and Recurrent Urinary Tract Infection in Children

BACKGROUND Antibiotics are widely administered to children with the intention of preventing urinary tract infection, but adequately powered, placebo-controlled trials regarding efficacy are lacking. This study from four Australian centers examined whether low-dose, continuous oral antibiotic therapy prevents urinary tract infection in predisposed children. METHODS We randomly assigned children under the age of 18 years who had had one or more microbiologically proven urinary tract infections to receive either daily trimethoprim-sulfamethoxazole suspension (as 2 mg of trimethoprim plus 10 mg of sulfamethoxazole per kilogram of body weight) or placebo for 12 months. The primary outcome was microbiologically confirmed symptomatic urinary tract infection. Intention-to-treat analyses were performed with the use of time-to-event data. RESULTS From December 1998 to March 2007, a total of 576 children (of 780 planned) underwent randomization. The median age at entry was 14 months; 64% of the patients were girls, 42% had known vesicoureteral reflux (at least grade III in 53% of these patients), and 71% were enrolled after the first diagnosis of urinary tract infection. During the study, urinary tract infection developed in 36 of 288 patients (13%) in the group receiving trimethoprim-sulfamethoxazole (antibiotic group) and in 55 of 288 patients (19%) in the placebo group (hazard ratio in the antibiotic group, 0.61; 95% confidence interval, 0.40 to 0.93; P = 0.02 by the log-rank test). In the antibiotic group, the reduction in the absolute risk of urinary tract infection (6 percentage points) appeared to be consistent across all subgroups of patients (P > or = 0.20 for all interactions). CONCLUSIONS Long-term, low-dose trimethoprim-sulfamethoxazole was associated with a decreased number of urinary tract infections in predisposed children. The treatment effect appeared to be consistent but modest across subgroups. (Australian New Zealand Clinical Trials Registry number, ACTRN12608000470392.)

An evidence based approach to individualising treatment

To which groups of patients can the results of clinical trials be applied? This question is often inappropriately answered by reference to the trial entry criteria. Instead, the benefit and harm (adverse events, discomfort of treatment, etc) of treatment could be assessed separately for individual patients. Patients at greatest risk of a disease will have the greatest net benefit as benefit to patients usually increases with risk while harm remains comparatively fixed. To assess net benefit, the relative risks should come from (a meta-analysis of) randomised trials; the risk in individual patients should come from multivariate risk equations derived from cohort studies. However, before making firm conclusions, the assumptions of fixed adverse effects and constant reduction in relative risk need to be checked.

Comparative accuracy: assessing new tests against existing diagnostic pathways

Most studies of diagnostic accuracy only compare a test with the reference standard. Is this helpful?