Jeroen G. Lijmer

The STARD Statement for Reporting Studies of Diagnostic Accuracy: Explanation and Elaboration

The quality of reporting of studies of diagnostic accuracy is less than optimal. Complete and accurate reporting is necessary to enable readers to assess the potential for bias in the study and to evaluate the generalizability of the results. A group of scientists and editors has developed the STARD (Standards for Reporting of Diagnostic Accuracy) statement to improve the reporting the quality of reporting of studies of diagnostic accuracy. The statement consists of a checklist of 25 items and flow diagram that authors can use to ensure that all relevant information is present. This explanatory document aims to facilitate the use, understanding, and dissemination of the checklist. The document contains a clarification of the meaning, rationale, and optimal use of each item on the checklist, as well as a short summary of the available evidence on bias and applicability. The STARD statement, checklist, flowchart, and this explanation and elaboration document should be useful resources to improve reporting of diagnostic accuracy studies. Complete and informative reporting can only lead to better decisions in health care.

The diagnostic odds ratio: a single indicator of test performance

Diagnostic testing can be used to discriminate subjects with a target disorder from subjects without it. Several indicators of diagnostic performance have been proposed, such as sensitivity and specificity. Using paired indicators can be a disadvantage in comparing the performance of competing tests, especially if one test does not outperform the other on both indicators. Here we propose the use of the odds ratio as a single indicator of diagnostic performance. The diagnostic odds ratio is closely linked to existing indicators, it facilitates formal meta-analysis of studies on diagnostic test performance, and it is derived from logistic models, which allow for the inclusion of additional variables to correct for heterogeneity. A disadvantage is the impossibility of weighing the true positive and false positive rate separately. In this article the application of the diagnostic odds ratio in test evaluation is illustrated.

The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration.

Introduction In studies of diagnostic accuracy, results from one or more tests are compared with the results obtained with the reference standard on the same subjects. Such accuracy studies are a vital step in the evaluation of new and existing diagnostic technologies (1, 2). Several factors threaten the internal and external validity of a study of diagnostic accuracy (3-8). Some of these factors have to do with the design of such studies, others with the selection of patients, the execution of the tests, or the analysis of the data. In a study involving several meta-analyses a number of design deficiencies were shown to be related to overly optimistic estimates of diagnostic accuracy (9). Exaggerated results from poorly designed studies can trigger premature adoption of diagnostic tests and can mislead physicians to incorrect decisions about the care for individual patients. Reviewers and other readers of diagnostic studies must therefore be aware of the potential for bias and a possible lack of applicability. A survey of studies of diagnostic accuracy published in four major medical journals between 1978 and 1993 revealed that the methodological quality was mediocre at best (8). Furthermore, this review showed that information on key elements of design, conduct, and analysis of diagnostic studies was often not reported (8). To improve the quality of reporting of studies of diagnostic accuracy the Standards for Reporting of Diagnostic Accuracy (STARD) initiative was started. The objective of the STARD initiative is to improve the quality of reporting of studies of diagnostic accuracy. Complete and accurate reporting allows the reader to detect the potential for bias in the study and to judge the generalizability and applicability of the results. For this purpose, the STARD project group has developed a single-page checklist. Where possible, the decision to include items in the checklist was based on evidence linking these items to bias, variability in results, or limitations of the applicability of results to other settings. The checklist can be used to verify that all essential elements are included in the report of a study. This explanatory document aims to facilitate the use, understanding, and dissemination of the checklist. The document contains a clarification of the meaning, rationale, and optimal use of each item on the checklist, as well as a short summary of the available evidence on bias and applicability. The first part of this document contains a summary of the design and terminology of diagnostic accuracy studies. The second part contains an item-by-item discussion with examples. Studies of Diagnostic Accuracy Studies of diagnostic accuracy have a common basic structure (10). One or more tests are evaluated, with the purpose of detecting or predicting a target condition. The target condition can refer to a particular disease, a disease stage, a health status, or any other identifiable condition within a patient, such as staging a disease already known to be present, or a health condition that should prompt clinical action, such as the initiation, modification, or termination of treatment. Here test refers to any method for obtaining additional information on a patients health status. This includes laboratory tests, imaging tests, function tests, pathology, history, and physical examination. In a diagnostic accuracy study, the test under evaluationreferred to here as the index testis applied to a series of subjects. The results obtained with the index test are compared with the results of the reference standard, obtained in the same subjects. In this framework, the reference standard is the best available method for establishing the presence or absence of the target condition. The reference standard can be a single test, or a combination of methods and techniques, including clinical follow-up of tested subjects. The term accuracy refers to the amount of agreement between the results from the index test and those from the reference standard. Diagnostic accuracy can be expressed in a number of ways, including sensitivityspecificity pairs, likelihood ratios, diagnostic odds ratios, and areas under ROC [receiver-operating characteristic] curves (11, 12). Study Question, Design, and Potential for Bias Early in the evaluation of a test, the author may simply want to know if the test is able to discriminate. The appropriate early question may be Do the test results in patients with the target condition differ from the results in healthy people? If preliminary studies answer this question affirmatively, the next study question is, Are patients with specific test results more likely to have the target disorder than similar patients with other test results? The usual study design to answer this is to apply the index test and the reference standard to a number of patients who are suspected of the target condition. Some study designs are more prone to bias and have a more limited applicability than others. In this article, the term bias refers to difference between the observed measures of test performance and the true measures. No single design is guaranteed to be both feasible and able to provide valid, informative, and relevant answers with optimal precision to all study questions. For each study, the reader must judge the relevance, the potential for bias, and the limitations to applicability, making full and transparent reporting critical. For this reason, checklist items refer to the research question that prompted the study of diagnostic accuracy and ask for an explicit and complete description of the study design and results. Variability Measures of test accuracy may vary from study to study. Variability may reflect differences in patient groups, differences in setting, differences in definition of the target condition, and differences in test protocols or in criteria for test positivity (13). For example, bias may occur if a test is evaluated under circumstances that do not correspond to those of the research question. Examples are evaluating a screening test for early disease in patients with advanced stages of the disease and evaluating a physicians office test device in the specialty department of a university hospital. The checklist contains a number of items to make sure that a study report contains a clear description of the inclusion criteria for patients, the testing protocols and the criteria for positivity, as well as an adequate account of subjects included in the study and their results. These items will enable readers to judge if the study results apply to their circumstances. Items in the Checklist The next section contains a point-by-point discussion of the items on the checklist. The order of the items corresponds to the sequence used in many publications of diagnostic accuracy studies. Specific requirements made by journals could lead to a different order. Item 1. Identify the Article as a Study of Diagnostic Accuracy (Recommend MeSH Heading Sensitivity and Specificity) Example (an Excerpt from a Structured Abstract) Purpose: To determine the sensitivity and specificity of computed tomographic colonography for colorectal polyp and cancer detection by using colonoscopy as the reference standard (14). Electronic databases have become indispensable tools to identify studies. To facilitate retrieval of their study, authors should explicitly identify it as a report of a study of diagnostic accuracy. We recommend the use of the term diagnostic accuracy in the title or abstract of a report that compares the results of one or more index tests with the results of a reference standard. In 1991 the National Library of Medicines MEDLINE database introduced a specific keyword (MeSH heading) for diagnostic studies: Sensitivity and Specificity. Using this keyword to search for studies of diagnostic accuracy remains problematic (15-19). In a selected set of MEDLINE journals covering publications between 1992 through 1995, the use of the MeSH heading Sensitivity and Specificity identified only 51% of all studies of diagnostic accuracy and incorrectly identified many articles that were not reports of studies on diagnostic accuracy (18). In the example, the authors used the more general term Performance Characteristics of CT Colonography in the title. The purpose section of the structured abstract explicitly mentions sensitivity and specificity. The MEDLINE record for this paper contains the MeSH Sensitivity and Specificity. Item 2. State the Research Questions or Study Aims, Such as Estimating Diagnostic Accuracy or Comparing Accuracy between Tests or across Participant Groups Example Invasive x-ray coronary angiography remains the gold standard for the identification of clinically significant coronary artery disease . A noninvasive test would be desirable. Coronary magnetic resonance angiography performed while the patient is breathing freely has reached sufficient technical maturity to allow more widespread application with a standardized protocol. Therefore, we conducted a study to determine the [accuracy] of coronary magnetic resonance angiography in the diagnosis of native-vessel coronary artery disease (20). The Helsinki Declaration states that biomedical research involving people should be based on a thorough knowledge of the scientific literature (21). In the introduction of scientific reports authors describe the scientific background, previous work on the subject, the remaining uncertainty, and, hence, the rationale for their study. Clearly specified research questions help the readers to judge the appropriateness of the study design and data analysis. A single general description, such as diagnostic value or clinical usefulness, is usually not very helpful to the readers. In the example, the authors use the introduction section of their paper to describe the potential of coronary magnetic resonance angiography as a non-invasive alternative to conventional x-ray angiography in the diagn

STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies

Incomplete reporting has been identified as a major source of avoidable waste in biomedical research. Essential information is often not provided in study reports, impeding the identification, critical appraisal, and replication of studies. To improve the quality of reporting of diagnostic accuracy studies, the Standards for Reporting Diagnostic Accuracy (STARD) statement was developed. Here we present STARD 2015, an updated list of 30 essential items that should be included in every report of a diagnostic accuracy study. This update incorporates recent evidence about sources of bias and variability in diagnostic accuracy and is intended to facilitate the use of STARD. As such, STARD 2015 may help to improve completeness and transparency in reporting of diagnostic accuracy studies.

The performance of CA-125 measurement in the detection of endometriosis: a meta-analysis

OBJECTIVE To assess the diagnostic performance of serum CA-125 measurement in the detection of endometriosis. DESIGN Meta-analysis. SETTING AND PATIENT(S) Twenty-three studies comparing serum CA-125 levels and laparoscopically confirmed endometriosis. INTERVENTION(S) Serum CA-125 measurement and laparoscopy. MAIN OUTCOME MEASURE(S) Sensitivity and specificity of serum CA-125 measurement in the diagnosis of endometriosis with laparoscopy as the reference standard. RESULT(S) The estimated summary receiver operating characteristic curves showed that the performance of serum CA-125 measurement in the diagnosis of endometriosis grade I/IV is limited, whereas its performance in the diagnosis of endometriosis grade III/IV is better. CONCLUSION(S) Despite its limited diagnostic performance, we believe that the routine use of serum CA-125 measurement in patients with infertility might be justified. In contrast to laparoscopy, serum CA-125 measurement is an inexpensive test that is not a burden for the patient. It could identify a subgroup of patients who are more likely to benefit from early laparoscopy. Studies reporting on the mutual dependence between serum CA-125 measurement and data from the history and physical examination are needed.

Designing studies to ensure that estimates of test accuracy are transferable

There is merit in studies with heterogeneous study populations. They allow exploration of the extent to which the performance of a diagnostic test depends on prespecified predictors, and how much residual variation exists. The more variation there is in study populations, the greater the potential to know how the test will perform in various settings.

ROC analysis of noninvasive tests for peripheral arterial disease

The purpose of this study is to evaluate the diagnostic accuracy of selected noninvasive tests for assessing peripheral arterial disease. The ankle/brachial index (ABI) and the femoral and popliteal pulsatility indices (PI), and combinations of these tests, were evaluated using receiver operating characteristic (ROC) analysis to determine their diagnostic accuracy depending on the localization of the disease. Verification bias, introduced by the preferential selection of patients for angiography based on the noninvasive test results, was examined. This study suggests that, in assessing whether a patient has significant peripheral arterial disease (lesions > or = 50%), determining an ABI is justified (ROC area 0.95 +/- 0.02). For disease localized to the aortoiliac segment, performing a single test, the femoral PI, is sufficient (ROC area 0.80 +/- 0.04). For disease including the femoropopliteal and infrapopliteal segments, a combination of tests is necessary. Utilized threshold values need to be adjusted for verification bias.

Randomised comparisons of medical tests: sometimes invalid, not always efficient

and the number of randomised trials with diagnostic tests is rising. Randomised comparisons Randomised comparisons have several advantages over other methods of comparing medical interventions. Random assignment of patients to the strategies under study should prevent any bias in the selection of patients: differences at baseline between groups of patients have to be attributed to chance. 4,5 This basic principle opens up the application of experimental statistical design, such as testing for significance and calculating confidence intervals. Randomised controlled trials are also attractive from a pragmatic point of view: if randomisation coincides with a choice between two management strategies, trial design closely mimics existing clinical dilemmas. The precision of the estimates will mainly depend on the number of individuals included. One trial design is said to be more statistically efficient than another if it yields more precise estimates when applied in similarly sized groups. 6

The accuracy of serum chlamydial antibodies in the diagnosis of tubal pathology: a meta-analysis

OBJECTIVE To assess the discriminative capacity of Chlamydia antibody titers in the diagnosis of tubed pathology in subfertile patients. DESIGN Meta-analysis of studies comparing Chlamydia antibody titers and laparoscopy for tubal patency and peritubal adhesions. PATIENTS A total of 2,729 patients with subfertility in 23 studies. INTERVENTION(S) Chlamydia antibody titer and laparoscopy as part of subfertility work-up. MAIN OUTCOME MEASURE Sensitivity and specificity of Chlamydia antibody titers in the diagnosis of tubal pathology using laparoscopy with chromopertubation as the reference standard. RESULT(S) The discriminative capacity of Chlamydia antibody titers depended on the type of assay that was used. Summary receiver operating characteristic (ROC) curves of studies using ELISA or (micro)immunofluorescence revealed a better discrimination than the summary ROC-curve of studies using immunoperoxidase assay. CONCLUSION(S) The discriminative capacity of Chlamydia antibody titers by means of ELISA, microimmunofluorescence, or immunofluorescence in the diagnosis of any tubal pathology is comparable to that of hysterosalpingography (HSG) in the diagnosis of tubal occlusion. Chlamydia antibody testing involves little burden but provides no details on the anatomy of uterus and tubes. Whether or not Chlamydia antibody testing can replace HSG depends on the perspective taken in the diagnostic work-up of subfertility.

STARD 2015: An Updated List of Essential Items for Reporting Diagnostic Accuracy Studies

Incomplete reporting has been identified as a major source of avoidable waste in biomedical research. Essential information is often not provided in study reports, impeding the identification, critical appraisal, and replication of studies. To improve the quality of reporting of diagnostic accuracy studies, the Standards for Reporting of Diagnostic Accuracy Studies (STARD) statement was developed. Here we present STARD 2015, an updated list of 30 essential items that should be included in every report of a diagnostic accuracy study. This update incorporates recent evidence about sources of bias and variability in diagnostic accuracy and is intended to facilitate the use of STARD. As such, STARD 2015 may help to improve completeness and transparency in reporting of diagnostic accuracy studies.