Alison Eastwood

Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews

Objective To investigate basic assumptions and other methodological problems in the application of indirect comparison in systematic reviews of competing healthcare interventions. Design Survey of published systematic reviews. Inclusion criteria Systematic reviews published between 2000 and 2007 in which an indirect approach had been explicitly used. Data extraction Identified reviews were assessed for comprehensiveness of the literature search, method for indirect comparison, and whether assumptions about similarity and consistency were explicitly mentioned. Results The survey included 88 review reports. In 13 reviews, indirect comparison was informal. Results from different trials were naively compared without using a common control in six reviews. Adjusted indirect comparison was usually done using classic frequentist methods (n=49) or more complex methods (n=18). The key assumption of trial similarity was explicitly mentioned in only 40 of the 88 reviews. The consistency assumption was not explicit in most cases where direct and indirect evidence were compared or combined (18/30). Evidence from head to head comparison trials was not systematically searched for or not included in nine cases. Conclusions Identified methodological problems were an unclear understanding of underlying assumptions, inappropriate search and selection of relevant trials, use of inappropriate or flawed methods, lack of objective and validated methods to assess or improve trial similarity, and inadequate comparison or inappropriate combination of direct and indirect evidence. Adequate understanding of basic assumptions underlying indirect and mixed treatment comparison is crucial to resolve these methodological problems. Appendix 1 PubMed search strategy Appendix 2 Characteristics of identified reports Appendix 3 Identified studies References of included studies

Inconsistency between direct and indirect comparisons of competing interventions: meta-epidemiological study

Objective To investigate the agreement between direct and indirect comparisons of competing healthcare interventions. Design Meta-epidemiological study based on sample of meta-analyses of randomised controlled trials. Data sources Cochrane Database of Systematic Reviews and PubMed. Inclusion criteria Systematic reviews that provided sufficient data for both direct comparison and independent indirect comparisons of two interventions on the basis of a common comparator and in which the odds ratio could be used as the outcome statistic. Main outcome measure Inconsistency measured by the difference in the log odds ratio between the direct and indirect methods. Results The study included 112 independent trial networks (including 1552 trials with 478 775 patients in total) that allowed both direct and indirect comparison of two interventions. Indirect comparison had already been explicitly done in only 13 of the 85 Cochrane reviews included. The inconsistency between the direct and indirect comparison was statistically significant in 16 cases (14%, 95% confidence interval 9% to 22%). The statistically significant inconsistency was associated with fewer trials, subjectively assessed outcomes, and statistically significant effects of treatment in either direct or indirect comparisons. Owing to considerable inconsistency, many (14/39) of the statistically significant effects by direct comparison became non-significant when the direct and indirect estimates were combined. Conclusions Significant inconsistency between direct and indirect comparisons may be more prevalent than previously observed. Direct and indirect estimates should be combined in mixed treatment comparisons only after adequate assessment of the consistency of the evidence.

A systematic review of photodynamic therapy in the treatment of pre-cancerous skin conditions, Barrett's oesophagus and cancers of the biliary tract, brain, head and neck, lung, oesophagus and skin

BACKGROUND Photodynamic therapy (PDT) is the use of a light-sensitive drug, in combination with light of a visible wavelength, to destroy target cells. PDT is used either as a primary treatment or as an adjunctive treatment. It is fairly well accepted in clinical practice for some types of skin cancer but has yet to be fully explored as a treatment for other forms of cancer. OBJECTIVE To systematically review the clinical effectiveness and safety of PDT in the treatment of Barretts oesophagus, pre-cancerous skin conditions and the following cancers: biliary tract, brain, head and neck, lung, oesophageal and skin. DATA SOURCES The search strategy included searching electronic databases (between August and October 2008), followed by update searches in May 2009, along with relevant bibliographies, existing reviews, conference abstracts and contact with experts in the field. STUDY DESIGNS Randomised controlled trials (RCTs) in skin conditions and Barretts oesophagus, non-randomised trials for all other sites. PARTICIPANTS People with Barretts oesophagus, pre-cancerous skin conditions or primary cancer in the following sites: biliary tract, brain, head and neck, lung, oesophageal and skin. INTERVENTION Any type of PDT for either curative or palliative treatment. COMPARATORS Any comparator including differing applications of PDT treatments (relevant comparators varied according to the condition). MAIN OUTCOMES The outcomes measured were mortality, morbidity, quality of life, adverse events and resource use. REVIEW METHODS A standardised data extraction form was used. The quality of RCTs and non-randomised controlled studies was assessed using standard checklists. Data extracted from the studies were tabulated and discussed in a narrative synthesis, and the influence of study quality on results was discussed. Meta-analysis was used to estimate a summary measure of effect on relevant outcomes, with assessment of both clinical and statistical heterogeneity. Two reviewers independently screened all titles and abstracts, and data extracted and quality assessed the trials, with discrepancies resolved by discussion or referral to a third reviewer. A scoping review was also undertaken. RESULTS Overall, 88 trials reported in 141 publications were included, with some trials covering more than one condition. For actinic keratosis (AK), the only clear evidence of effectiveness was that PDT appeared to be superior to placebo. For Bowens disease, better outcomes with PDT were suggested when compared with cryotherapy or fluorouracil. For basal cell carcinoma (BCC), PDT may result in similar lesion response rates to surgery or cryotherapy but with better cosmetic outcomes. For nodular lesions, PDT appeared to be superior to placebo and less effective than surgery but suggestive of better cosmetic outcome. For Barretts oesophagus, PDT in addition to omeprazole appeared to be more effective than omeprazole alone at long-term ablation of high-grade dysplasia and slowing/preventing progression to cancer. No firm conclusions could be drawn for PDT in oesophageal cancer. Further research into the role of PDT in lung cancer is needed. For cholangiocarcinoma, PDT may improve survival when compared with stenting alone. There was limited evidence on PDT for brain cancer and cancers of the head and neck. A wide variety of photosensitisers were used and, overall, no serious adverse effects were linked to PDT. LIMITATIONS There were few well-conducted, adequately powered RCTs, and quality of life (QoL) and resource outcomes were under-reported. Problems were identified with reporting of key study features and quality parameters, making the reliability of some studies uncertain. Methodological limitations and gaps in the evidence base made it difficult to draw firm conclusions. CONCLUSIONS Evidence of effectiveness was found for PDT in the treatment of AK and nodular BCC in relation to placebo, and possibly for treating Barretts oesophagus. However, the effectiveness of PDT in relation to other treatments is not yet apparent. High-quality trials are needed to compare PDT with relevant comparators for all meaningful outcomes, including QoL and adverse effects. Further research is also needed on patient experience of PDT, as well as on the cost-effectiveness of PDT.

The causes, consequences and detection of publication bias in psychiatry.

Objective: Publication bias threatens the validity of published research, although this topic has received little attention in psychiatry. The purpose of this article is to produce a systematic overview of the causes and consequences of publication bias and to summarize the available methods with which it is detected and corrected.

Increasing participation of cancer patients in randomised controlled trials: a systematic review

BackgroundThere are many barriers to patient participation in randomised controlled trials of cancer treatments. To increase participation in trials, strategies need to be identified to overcome these barriers. Our aim was to assess the effectiveness of interventions to overcome barriers to patient participation in randomised controlled trials (RCTs) of cancer treatments.MethodsA systematic review was conducted. Published and unpublished studies in any language were searched for in fifteen electronic databases, including MEDLINE, EMBASE, CINAHL and PsycINFO, from inception to the end of 2004.Studies of any interventions to improve cancer patient participation in RCTs, which reported the change in recruitment rates, were eligible for inclusion. RCTs and non-randomised controlled trials as well as before and after studies reporting baseline rates specific to the population being investigated were included. Data were extracted by one reviewer into structured summary tables and checked for accuracy by a second reviewer. Each included study was assessed against a checklist for methodological quality by one reviewer and checked by a second reviewer. A narrative synthesis was conducted.ResultsEight studies were identified that met the inclusion criteria: three RCTs, two non-randomised controlled trials and three observational studies. Six of the studies had an intervention that had some relevance to the UK. There was no robust evidence that any of the interventions investigated led to an increase in cancer patient participation in RCTs, though one good quality RCT found that urologists and nurses were equally effective at recruiting participants to a treatment trial for prostate cancer. Although there was no evidence of an effect in any of the studies, the evidence was not of sufficient quality to be able to conclude that these interventions therefore do not work.ConclusionThere is not a strong evidence-base for interventions that increase cancer patient participation in randomised trials. Further research is required to evaluate the effectiveness of strategies to increase participation in cancer treatment trials.

Effectiveness and implementation of enhanced recovery after surgery programmes: a rapid evidence synthesis

Objectives To assess the evidence on the impact of enhanced recovery programmes for patients undergoing elective surgery in acute hospital settings in the UK. Design Rapid evidence synthesis. Eight databases were searched from 1990 to March 2013 without language restrictions. Relevant reports and guidelines, websites and reference lists of retrieved articles were scanned to identify additional studies. Systematic reviews, RCTs not included in the systematic reviews, economic evaluations and UK NHS cost analysis, implementation case studies and surveys of patient experience in a UK setting were eligible for inclusion. Primary and secondary outcome measures We assessed the impact of enhanced recovery programmes on health or cost-related outcomes, and assessed implementation case studies and patient experience in UK settings. Studies were quality assessed where appropriate using the Centre for Reviews and Dissemination Database of Abstracts of Reviews of Effects critical appraisal process. Results 17 systematic reviews and 12 additional RCTs were included. Ten relevant economic evaluations were included. No cost analysis studies were identified. Most of the evidence focused on colorectal surgery. 14 innovation case studies and 15 implementation case studies undertaken in National Health Service settings described factors critical to the success of an enhanced recovery programme. Evidence for colorectal surgery suggests that enhanced recovery programmes may reduce hospital stays by 0.5–3.5 days compared with conventional care. There were no significant differences in reported readmission rates. Other surgical specialties showed greater variation in reductions in length of stay reflecting the limited evidence identified. Findings relating to other outcomes were hampered by a lack of robust evidence and poor reporting. Conclusions There is consistent, albeit limited, evidence that enhanced recovery programmes can reduce length of patient hospital stay without increasing readmission rates. The extent to which managers and clinicians considering implementing enhanced recovery programmes in UK settings can realise savings will depend on length of stay achieved under their existing care pathway.

Palliative thoracic radiotherapy for non-small-cell lung cancer: a systematic review.

&NA; Non‐small‐cell lung cancer is one of the most common malignant tumors worldwide. The majority of patients are not treatable with curative intent because of the extent of disease or patient comorbidity. Radiotherapy to the primary intrathoracic tumor is used with the aim of palliating troublesome local symptoms in approximately 25% of patients. The dose/fractionation regimens used evolved empirically, and surveys have shown widespread variation. It has not yet been clearly established which regimens give the most benefit and least toxicity. This systematic review identified 12 randomized controlled trials comparing palliative external beam radiotherapy regimens. Narrative synthesis has been performed. Palliative radiotherapy is effective in controlling symptoms. There is no strong evidence that better palliation is obtained with higher radiation doses but good evidence that toxicity is greater. There is evidence of a modest survival benefit with higher dose schedules in patients with good performance status. The majority of patients should receive short courses (one or two fractions) of hypofractionated radiotherapy, Selected patients with good performance status should be considered for higher dose regimens if the chance of modest improvement in survival and palliation is considered worth the additional inconvenience and toxicity.

Volume of clinical activity in hospitals and healthcare outcomes, costs, and patient access.

Background Concentration of the provision of hospital services is sometimes seen as a way to reduce costs and improve the quality and efficiency of care. This paper, based on an issue of Effective Health Care Vol 2, No 8 summarises the results of systematic reviews carried out at the University of York to assess research into the possible relation between volume of clinical activity in hospitals and the outcomes of quality of health care, hospital costs (economies of scale), and patient access. Full reports of the methods used and the results of these reviews, including details of the studies included, are available from the National Health Service (NHS) Centre for Reviews and Dissemination and cannot be presented here due to lack of space. d

Trastuzumab for the treatment of HER2-positive metastatic gastric cancer : a NICE single technology appraisal.

The National Institute for Health and Clinical Excellence (NICE) invited the manufacturer of trastuzumab, Roche Pharmaceuticals, to submit evidence for the clinical and cost effectiveness of this drug for the treatment of advanced gastric cancer (aGC), as part of the Institute’s single technology appraisal (STA) process. The Centre for Reviews and Dissemination (CRD) and the Centre for Health Economics (CHE) Technology Appraisal Group at the University of York was commissioned to act as the evidence review group (ERG). This article provides a description of the company submission, the ERG report and NICE’s subsequent decisions. In the initial appraisal by NICE, trastuzumab was rejected for use in the licensed population. Given this result, the manufacturer submitted additional evidence. In the final appraisal decision, trastuzumab was approved, in accordance with supplementary guidance issued by NICE on appraising life-extending, end-of-life treatments, for patients whose human epidermal growth factor receptor 2 (HER2) status was defined by an immunohistochemistry 3 positive (IHC3+) result. This appraisal highlights the need to fully assess the impact of different approaches to diagnostic testing on the cost effectiveness of targeted treatments. In this appraisal, it was found that the diagnostic strategy influenced the effectiveness and cost of trastuzumab. In the future, different diagnostic strategies should be compared in the incremental cost-effectiveness analysis.

Organisation of asthma care: what difference does it make? A systematic review of the literature.

OBJECTIVES: To evaluate the effectiveness of different forms of organisation (delivery) of asthma care. METHODS: A systematic review of the published evidence of effectiveness organisational methods of asthma management. Searches on computerised databases including Medline, CINAHL, and HELMIS, and relevant citations and letters to experts were used to identify relevant studies. RESULTS: 27 studies were identified that evaluated different organisational methods of delivery across both primary and secondary sectors, such as shared care, general practice asthma clinics, outpatient programmes, inpatient admissions policies, and the use of specialists. Only one third of the studies used a randomised controlled trial and many had small sample sizes. No conclusive evidence was found to favour any particular organisational form, although limited evidence would suggest that specialist care is better than general care and that shared care can be as effective as hospital led care. CONCLUSIONS: There is little good published research evaluating different ways of organising the delivery of asthma care. There is need for quality research on organisational methods of delivery of asthma care that could be used to inform policy makers, in particular examining whether patients treated by healthcare professional with expertise and interest in asthma will experience better outcomes.