Piet N. Post

GRADE guidelines: 8. Rating the quality of evidence-Indirectness

Direct evidence comes from research that directly compares the interventions in which we are interested when applied to the populations in which we are interested and measures outcomes important to patients. Evidence can be indirect in one of four ways. First, patients may differ from those of interest (the term applicability is often used for this form of indirectness). Secondly, the intervention tested may differ from the intervention of interest. Decisions regarding indirectness of patients and interventions depend on an understanding of whether biological or social factors are sufficiently different that one might expect substantial differences in the magnitude of effect. Thirdly, outcomes may differ from those of primary interest-for instance, surrogate outcomes that are not themselves important, but measured in the presumption that changes in the surrogate reflect changes in an outcome important to patients. A fourth type of indirectness, conceptually different from the first three, occurs when clinicians must choose between interventions that have not been tested in head-to-head comparisons. Making comparisons between treatments under these circumstances requires specific statistical methods and will be rated down in quality one or two levels depending on the extent of differences between the patient populations, co-interventions, measurements of the outcome, and the methods of the trials of the candidate interventions.

GRADE guidelines: 15. Going from evidence to recommendation-determinants of a recommendation's direction and strength.

In the GRADE approach, the strength of a recommendation reflects the extent to which we can be confident that the composite desirable effects of a management strategy outweigh the composite undesirable effects. This article addresses GRADEs approach to determining the direction and strength of a recommendation. The GRADE describes the balance of desirable and undesirable outcomes of interest among alternative management strategies depending on four domains, namely estimates of effect for desirable and undesirable outcomes of interest, confidence in the estimates of effect, estimates of values and preferences, and resource use. Ultimately, guideline panels must use judgment in integrating these factors to make a strong or weak recommendation for or against an intervention.

GRADE guidelines: 12. Preparing summary of findings tables-binary outcomes.

Summary of Findings (SoF) tables present, for each of the seven (or fewer) most important outcomes, the following: the number of studies and number of participants; the confidence in effect estimates (quality of evidence); and the best estimates of relative and absolute effects. Potentially challenging choices in preparing SoF table include using direct evidence (which may have very few events) or indirect evidence (from a surrogate) as the best evidence for a treatment effect. If a surrogate is chosen, it must be labeled as substituting for the corresponding patient-important outcome. Another such choice is presenting evidence from low-quality randomized trials or high-quality observational studies. When in doubt, a reasonable approach is to present both sets of evidence; if the two bodies of evidence have similar quality but discrepant results, one would rate down further for inconsistency. For binary outcomes, relative risks (RRs) are the preferred measure of relative effect and, in most instances, are applied to the baseline or control group risks to generate absolute risks. Ideally, the baseline risks come from observational studies including representative patients and identifying easily measured prognostic factors that define groups at differing risk. In the absence of such studies, relevant randomized trials provide estimates of baseline risk. When confidence intervals (CIs) around the relative effect include no difference, one may simply state in the absolute risk column that results fail to show a difference, omit the point estimate and report only the CIs, or add a comment emphasizing the uncertainty associated with the point estimate.

Systematic review and meta‐analysis of the volume–outcome relationship in pancreatic surgery

Many studies have shown lower mortality and higher survival rates after pancreatic surgery with high‐volume providers, suggesting that centralization of pancreatic surgery can improve outcomes. The methodological quality of these studies is open to question. This study involves a systematic review of the volume–outcome relationship for pancreatic surgery with a meta‐analysis of studies considered to be of good quality.

Serious Co-morbidity Among Unselected Cancer Patients Newly Diagnosed in the Southeastern Part of The Netherlands in 1993-1996

The purpose of this study was to determine the prevalence of serious concomitant conditions at diagnosis among unselected patients with cancer, increasingly older in industrialized countries. About 34,000 newly diagnosed cancer patients were recorded in the Eindhoven Cancer Registry between 1993 and 1996; subsequently data on serious co-morbidity, classified according to the Charlson scheme (J Chron Dis 1987; 40: 373-383), were collected from the clinical records by registry personnel. Co-morbid conditions were present in 12% of adult patients below 45 years of age, 28% of those 45-59 years, 53% of those 60-74 years, and 63% of patients over 75 years of age, the prevalence being highest for patients with lung (58%), kidney (54%), stomach (53%), bladder (53%), and prostate cancer (51%). Males exhibited a 10% higher prevalence than females with similar tumors. Among patients over 60 years the most frequent conditions were heart and vascular diseases (ranging across the various tumors from 10% to 30%), hypertension (11-25%), another cancer (10-20%), COPD (chronic obstructive pulmonary disease) (3-25%), and diabetes mellitus (5-25%). Inclusion of frequent co-morbid conditions in prognostic research as well as the development of specific guidelines for patient care seems warranted.

The GRADE approach is reproducible in assessing the quality of evidence of quantitative evidence syntheses

OBJECTIVE We evaluated the inter-rater reliability (IRR) of assessing the quality of evidence (QoE) using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach. STUDY DESIGN AND SETTING On completing two training exercises, participants worked independently as individual raters to assess the QoE of 16 outcomes. After recording their initial impression using a global rating, raters graded the QoE following the GRADE approach. Subsequently, randomly paired raters submitted a consensus rating. RESULTS The IRR without using the GRADE approach for two individual raters was 0.31 (95% confidence interval [95% CI] = 0.21-0.42) among Health Research Methodology students (n = 10) and 0.27 (95% CI = 0.19-0.37) among the GRADE working group members (n = 15). The corresponding IRR of the GRADE approach in assessing the QoE was significantly higher, that is, 0.66 (95% CI = 0.56-0.75) and 0.72 (95% CI = 0.61-0.79), respectively. The IRR further increased for three (0.80 [95% CI = 0.73-0.86] and 0.74 [95% CI = 0.65-0.81]) or four raters (0.84 [95% CI = 0.78-0.89] and 0.79 [95% CI = 0.71-0.85]). The IRR did not improve when QoE was assessed through a consensus rating. CONCLUSION Our findings suggest that trained individuals using the GRADE approach improves reliability in comparison to intuitive judgments about the QoE and that two individual raters can reliably assess the QoE using the GRADE system.

A systematic review and meta-analysis of the relationship between hospital/surgeon volume and outcome for radical cystectomy: an update for the ongoing debate.

CONTEXT There is an ongoing debate about centralisation of radical cystectomy (RC) procedures. OBJECTIVE To conduct a systematic review of the literature on the volume-outcome relationship for RC for bladder cancer (BCa) with consideration for the methodologic quality of the available evidence and to perform a meta-analysis on the studies meeting predefined quality criteria. EVIDENCE ACQUISITION A systematic search was performed to identify all articles examining the effects of procedure volume on clinical outcome for cystectomy. Reviews, opinion articles, and surveys were excluded. All articles were critically appraised for methodologic quality and risk of bias. Meta-analysis was performed to calculate the overall effect of higher surgeon or hospital volume on patient outcome. EVIDENCE SYNTHESIS Ten studies of good methodologic quality were included for meta-analysis. Eight studies were based on administrative data, two studies on clinical data. The results showed a significant association between high-volume hospitals and low mortality. A meta-analysis of the seven studies on hospital mortality showed a pooled estimated effect of odds ratio (OR) 0.55 (range: 0.44-0.69). The result was moderate heterogeneity (I(2)=50). A large variation in cut-off points used was observed. Sensitivity analyses did not show different effects in any of the subgroup analyses. Also, no significant differences in effect sizes were observed for different cut-off points. The data were not suggestive for publication bias. One study showed a positive effect of hospital volume on survival (hazard ratio [HR]: 0.89; p=0.06). Two studies showed a beneficial effect of surgeon volume on mortality (OR: 0.55; OR: 0.64). Only one study on the impact of surgeon volume on survival was found; it showed no significant positive effect for higher volume (HR: 0.83; p=0.26). CONCLUSIONS Postoperative mortality after cystectomy is significantly inversely associated with high-volume providers. However, additional quality criteria, such as infrastructure and level of specialisation, should be formulated to direct centralisation initiatives. The Dutch Association of Urology in 2010 implemented a national quality of care (QoC) registration programme for all patients treated by surgery for muscle-invasive BCa, including multiple parameters defining QoC.

The independent prognostic value of comorbidity among men aged < 75 years with localized prostate cancer: a population-based study

Objective To investigate which prognostic factors apply in patients with localized prostate cancer diagnosed after the introduction of prostate‐specific antigen (PSA) testing, as comorbidity has significant prognostic value for patients who were diagnosed with localized prostate cancer in the 1970s.

A systematic review and meta-analysis of the volume-outcome relationship in the surgical treatment of breast cancer. Are breast cancer patients better of with a high volume provider?

AIMS To conduct a systematic review of the literature on the volume-outcome relationship for the surgical treatment of breast cancer with consideration of the methodological quality of the available evidence and to perform a meta-analysis on the studies of considered good quality. METHODS A systematic search was done to identify all articles examining the effects of hospital or surgeon volume on clinical outcome of the surgical treatment of breast cancer. Reviews, opinion articles and surveys were excluded. All articles were critically appraised on methodological quality and risk of bias. After strict inclusion, meta-analysis assuming a random effects model was done to estimate the effect of higher hospital or surgeon volume on patient outcome. RESULTS We found 12 studies of good methodological quality which could be included for meta-analysis. The results showed a significant association between high volume providers and an improved survival. The association is the most robust for surgeon volume (HR 0.80 (0.71-0.90) and RR 0.85 (0.80-0.90). In addition there is an effect of hospital volume on the in-hospital mortality, although the mortality was very low (0.1-0.2%). Results of meta-analysis were heterogeneous. Sensitivity analysis showed a larger effect size for studies also adjusting for comorbidity for both studies on hospital and surgeon volume. The data were not suggestive for publication bias. CONCLUSIONS The results show that survival after breast cancer surgery is significantly associated with high volume providers.

The Relationship Between Volume or Surgeon Specialty and Outcome in the Surgical Treatment of Lung Cancer A Systematic Review and Meta-Analysis

Background: Whether improvement of quality of surgical cancer care can be achieved by centralizing care in high-volume specialized centers is a subject of ongoing debate. We have conducted a meta-analysis of the literature on the effect of procedural volume or surgeon specialty on outcome of lung resections for cancer. Methods: A systematic search of articles published between January 1, 1990 and January 20, 2011 on the effects of surgeon specialty and hospital or surgeon volume of lung resections on mortality and survival was conducted. After strict inclusion, meta-analysis assuming a random-effects model was performed. Meta-regression was used to identify volume cutoff values. Heterogeneity and the risk of publication bias were evaluated. Results: Nineteen relevant studies were found. Studies were heterogeneous, especially in defining volume categories. The pooled estimated effect size was significant in favor of high-volume hospitals regarding postoperative mortality (odds ratio [OR] 0.71; confidence interval 0.62–0.81), but not for survival (OR 0.93; confidence interval 0.84–1.03). Surgeon volume showed no significant effect on outcome. General surgeons had significantly higher mortality risks than general thoracic (OR 0.78; 0.70–0.88) or cardiothoracic surgeons (OR 0.82; 0.69–0.96). A minimal annual volume of resections for lung cancer could not be identified. Conclusions: Hospital volume and surgeon specialty are important determinants of outcome in lung cancer resections, but evidence-based minimal-volume standards are lacking. Evaluation of individual institutions in a national audit program might help elucidate the influence of individual quality-of-care parameters, including hospital volume, on outcome.