Phil Alderson

Adapting clinical guidelines to take account of multimorbidity

Care of patients with multimorbidity could be improved if new technology is used to bring together guidelines on individual conditions and tailor advice to each patient’s circumstances, say Bruce Guthrie and colleagues

Absence of evidence for the effectiveness of five interventions routinely used in the intensive care management of severe head injury: a systematic review

OBJECTIVES To assess the effectiveness of interventions routinely used in the intensive care management of severe head injury, specifically, the effectiveness of hyperventilation, mannitol, CSF drainage, barbiturates, and corticosteroids. METHODS Systematic review of all unconfounded randomised trials, published or unpublished, that were available by August 1996. RESULTS None of the interventions has been reliably shown to reduce death or disability after severe head injury. One trial of hyperventilation was identified of 77 participants. The relative risk for death was 0.73 (95% confidence interval (95% CI) 0.36–1.49), and for death or disability it was 1.14 (95% CI 0.82–1.58). One trial of mannitol was identified of 41 participants. The relative risk for death was 1.75 (95% CI 0.48–6.38), no data were available for disability. No randomised trials of CSF drainage were identified. Two randomised trials of barbiturate therapy were identified, including 126 participants. The pooled relative risk for death was 1.12 (95% CI 0.81–1.54). Disability data were available for one trial. The relative risk for death or disability was 0.96 (95% CI 0.62–1.49). Thirteen randomised trials of corticosteroids were identified, comprising 2073 participants. The pooled relative risk for death was 0.95 (0.84 to 1.07) and for death or disability it was 1.01 (95% CI 0.91 to 1.11). On the basis of the currently available randomised evidence, for every intervention studied it is impossible to refute either a moderate increase or a moderate decrease in the risk of death or disability. CONCLUSION Existing trials have been too small to support or refute the existence of a real benefit from using hyperventilation, mannitol, CSF drainage, barbiturates, or corticosteroids. Further large scale randomised trials of these interventions are required.

Absence of evidence is not evidence of absence

We need to report uncertain results and do it clearly The title of this editorial is not new. For example, it was used nearly a decade ago for an article in the BMJ s Statistics Notes series.1 Altman and Bland considered the dangers of misinterpreting differences that do not reach significance, criticising use of the term “negative” to describe studies that had not found statistically significant differences. Such studies may not have been large enough to exclude important differences. To leave the impression that they have proved that no effect or no difference exists is misleading. As an example, a randomised trial of behavioural and specific sexually transmitted infection interventions for reducing transmission of HIV-1 was published in the Lancet. 2 The incidence rate ratios for the outcome of HIV-1 infection were 0.94 (95% confidence interval 0.60 to 1.45) and 1.00 (0.63 to 1.58) for two intervention groups compared with control. In …

Drug-disease and drug-drug interactions: systematic examination of recommendations in 12 UK national clinical guidelines

Objective To identify the number of drug-disease and drug-drug interactions for exemplar index conditions within National Institute of Health and Care Excellence (NICE) clinical guidelines. Design Systematic identification, quantification, and classification of potentially serious drug-disease and drug-drug interactions for drugs recommended by NICE clinical guidelines for type 2 diabetes, heart failure, and depression in relation to 11 other common conditions and drugs recommended by NICE guidelines for those conditions. Setting NICE clinical guidelines for type 2 diabetes, heart failure, and depression Main outcome measures Potentially serious drug-disease and drug-drug interactions. Results Following recommendations for prescription in 12 national clinical guidelines would result in several potentially serious drug interactions. There were 32 potentially serious drug-disease interactions between drugs recommended in the guideline for type 2 diabetes and the 11 other conditions compared with six for drugs recommended in the guideline for depression and 10 for drugs recommended in the guideline for heart failure. Of these drug-disease interactions, 27 (84%) in the type 2 diabetes guideline and all of those in the two other guidelines were between the recommended drug and chronic kidney disease. More potentially serious drug-drug interactions were identified between drugs recommended by guidelines for each of the three index conditions and drugs recommended by the guidelines for the 11 other conditions: 133 drug-drug interactions for drugs recommended in the type 2 diabetes guideline, 89 for depression, and 111 for heart failure. Few of these drug-disease or drug-drug interactions were highlighted in the guidelines for the three index conditions. Conclusions Drug-disease interactions were relatively uncommon with the exception of interactions when a patient also has chronic kidney disease. Guideline developers could consider a more systematic approach regarding the potential for drug-disease interactions, based on epidemiological knowledge of the comorbidities of people with the disease the guideline is focused on, and should particularly consider whether chronic kidney disease is common in the target population. In contrast, potentially serious drug-drug interactions between recommended drugs for different conditions were common. The extensive number of potentially serious interactions requires innovative interactive approaches to the production and dissemination of guidelines to allow clinicians and patients with multimorbidity to make informed decisions about drug selection.

Should journals publish systematic reviews that find no evidence to guide practice? Examples from injury research

Many systematic reviews are inconclusive and reinforce the message that there is clinical uncertainty. Phil Alderson and Ian Roberts argue that journals should make a point of publishing such reviews rather than waiting for reviews that show marked benefit or harm. Some experts disagree, however, but we failed to persuade them to commit their views to print. Studies with dramatic findings make interesting reading. Journal editors understandably want to publish articles that their readers will enjoy. This is one cause of publication bias, where research with less dramatic results tends to be published in journals with a smaller circulation, if indeed it is published at all. Systematic reviews are no less vulnerable to this bias than other types of research. Should journals resist this pressure and make a point of publishing systematic reviews even if all they show is continuing clinical uncertainty? The answer will depend on the importance we attach to demonstrating uncertainty in medical practice. #### Summary points Denying uncertainty does not benefit patients and may increase health service costs More large scale randomised trials need to be conducted based on the “uncertainty principle” Systematic reviews with more dramatic results tend …

Survey of claims of no effect in abstracts of Cochrane reviews

It is never correct to claim that treatments have no effect or that there is no difference in the effects of treatments. It is impossible to prove a negative or that two treatments have the same effect. There will always be some uncertainty surrounding estimates of treatment effects, and a small difference can never be excluded.1 Claims of no effect or no difference may mean that patients continue to be denied or exposed to interventions with important effects, either beneficial or harmful. They may also suggest that further research is unnecessary, so delaying satisfactory estimates of treatment effects. The impossibility of proving no effect or no difference should be distinguished from the concept used for equivalence trials, where bounds are set on the differences that are deemed practically important. An analysis of 45 reports of trials purporting to test equivalence …

Median life span of a cohort of National Institute for Health and Care Excellence clinical guidelines was about 60 months

OBJECTIVES To describe the length of time National Institute for Health and Care Excellence (NICE) clinical guidelines have remained valid. STUDY DESIGN AND SETTING The present study is a survival analysis of a cohort of published NICE clinical guidelines. The National Health Service in England and Wales uses NICE clinical practice guidelines as a reference for treatment and care of individuals. They need to be updated as new evidence arises, to remain credible and relevant, and are currently assessed 3 years after publication. RESULTS Survival analysis suggested that about 86% of guidelines are still up-to-date 3 years after their publication. The median life span was 60 months (95% confidence interval: 51, 69). CONCLUSION These findings are similar to those in other studies of the life span of guidelines. Efficient mechanisms must be in place to detect the minority of guidelines that become outdated quickly.

The impact of Cochrane Systematic Reviews: a mixed method evaluation of outputs from Cochrane Review Groups supported by the UK National Institute for Health Research

BackgroundThere has been a growing emphasis on evidence-informed decision-making in health care. Systematic reviews, such as those produced by the Cochrane Collaboration, have been a key component of this movement. The UK National Institute for Health Research (NIHR) Systematic Review Programme currently supports 20 Cochrane Review Groups (CRGs). The aim of this study was to identify the impacts of Cochrane reviews published by NIHR-funded CRGs during the years 2007–2011.MethodsWe sent questionnaires to CRGs and review authors, interviewed guideline developers and used bibliometrics and documentary review to get an overview of CRG impact and to evaluate the impact of a sample of 60 Cochrane reviews. We used a framework with four categories (knowledge production, research targeting, informing policy development and impact on practice/services).ResultsA total of 1,502 new and updated reviews were produced by the 20 NIHR-funded CRGs between 2007 and 2011. The clearest impacts were on policy with a total of 483 systematic reviews cited in 247 sets of guidance: 62 were international, 175 national (87 from the UK) and 10 local. Review authors and CRGs provided some examples of impact on practice or services, for example, safer use of medication, the identification of new effective drugs or treatments and potential economic benefits through the reduction in the use of unproven or unnecessary procedures. However, such impacts are difficult to objectively document, and the majority of reviewers were unsure if their review had produced specific impacts. Qualitative data suggested that Cochrane reviews often play an instrumental role in informing guidance, although a poor fit with guideline scope or methods, reviews being out of date and a lack of communication between CRGs and guideline developers were barriers to their use.ConclusionsHealth and economic impacts of research are generally difficult to measure. We found that to be the case with this evaluation. Impacts on knowledge production and clinical guidance were easier to identify and substantiate than those on clinical practice. Questions remain about how we define and measure impact, and more work is needed to develop suitable methods for impact analysis.

Managing conflicts of interest in the UK National Institute for Health and Care Excellence (NICE) clinical guidelines programme: qualitative study.

Background There is international concern that conflicts of interest (COI) may bias clinical guideline development and render it untrustworthy. Guideline COI policies exist with the aim of reducing this bias but it is not known how such policies are interpreted and used by guideline producing organisations. This study sought to determine how conflicts of interest (COIs) are disclosed and managed by a national clinical guideline developer (NICE: the UK National Institute for Health and Care Excellence). Methods Qualitative study using semi-structured telephone interviews with 14 key informants: 8 senior staff of NICE’s guideline development centres and 6 chairs of guideline development groups (GDGs). We conducted a thematic analysis. Results Participants regard the NICE COI policy as comprehensive leading to transparent and independent guidance. The application of the NICE COI policy is, however, not straightforward and clarity could be improved. Disclosure of COI relies on self reporting and guideline developers have to take “on trust” the information they receive, certain types of COI (non-financial) are difficult to categorise and manage and disclosed COI can impact on the ability to recruit clinical experts to GDGs. Participants considered it both disruptive and stressful to exclude members from GDG meetings when required by the COI policy. Nonetheless the impact of this disruption can be minimised with good group chairing skills. Conclusions We consider that the successful implementation of a COI policy in clinical guideline development requires clear policies and procedures, appropriate training of GDG chairs and an evaluation of how the policy is used in practice.

How individual participant data meta-analyses have influenced trial design, conduct, and analysis

Objectives To demonstrate how individual participant data (IPD) meta-analyses have impacted directly on the design and conduct of trials and highlight other advantages IPD might offer. Study Design and Setting Potential examples of the impact of IPD meta-analyses on trials were identified at an international workshop, attended by individuals with experience in the conduct of IPD meta-analyses and knowledge of trials in their respective clinical areas. Experts in the field who did not attend were asked to provide any further examples. We then examined relevant trial protocols, publications, and Web sites to verify the impacts of the IPD meta-analyses. A subgroup of workshop attendees sought further examples and identified other aspects of trial design and conduct that may inform IPD meta-analyses. Results We identified 52 examples of IPD meta-analyses thought to have had a direct impact on the design or conduct of trials. After screening relevant trial protocols and publications, we identified 28 instances where IPD meta-analyses had clearly impacted on trials. They have influenced the selection of comparators and participants, sample size calculations, analysis and interpretation of subsequent trials, and the conduct and analysis of ongoing trials, sometimes in ways that would not possible with systematic reviews of aggregate data. We identified additional potential ways that IPD meta-analyses could be used to influence trials. Conclusions IPD meta-analysis could be better used to inform the design, conduct, analysis, and interpretation of trials.