Gill Norman

A checklist designed to aid consistency and reproducibility of GRADE assessments: development and pilot validation

BackgroundThe grading of recommendation, assessment, development and evaluation (GRADE) approach is widely implemented in health technology assessment and guideline development organisations throughout the world. GRADE provides a transparent approach to reaching judgements about the quality of evidence on the effects of a health care intervention, but is complex and therefore challenging to apply in a consistent manner.MethodsWe developed a checklist to guide the researcher to extract the data required to make a GRADE assessment. We applied the checklist to 29 meta-analyses of randomised controlled trials on the effectiveness of health care interventions. Two reviewers used the checklist for each paper and used these data to rate the quality of evidence for a particular outcome.ResultsFor most (70%) checklist items, there was good agreement between reviewers. The main problems were for items relating to indirectness where considerable judgement is required.ConclusionsThere was consistent agreement between reviewers on most items in the checklist. The use of this checklist may be an aid to improving the consistency and reproducibility of GRADE assessments, particularly for inexperienced users or in rapid reviews without the resources to conduct assessments by two researchers independently.

Omalizumab for the treatment of severe persistent allergic asthma: a systematic review and economic evaluation

BACKGROUND Allergic asthma is a long-term disorder of the airways resulting from overexpression of immunoglobulin E (IgE) in response to environmental allergens. Patients with poorly controlled asthma are at high risk of exacerbations requiring additional treatment, including hospitalisations. Severe exacerbations are potentially life threatening. Guidelines identify five treatment steps for both adults and children. Omalizumab (Xolair(®)) is a recombinant DNA-derived humanised monoclonal antibody indicated as an add-on therapy in patients aged ≥ 6 years with severe persistent allergic asthma uncontrolled at treatment step 4 or 5. OBJECTIVE To determine the clinical effectiveness, safety and cost-effectiveness of omalizumab, as an add-on therapy to standard care, within its licensed indication, compared with standard therapy alone for the treatment of severe persistent allergic asthma in adults and adolescents aged ≥ 12 years and children aged 6-11 years. DATA SOURCES Eleven electronic databases (including MEDLINE, EMBASE and the Cochrane Central Register of Controlled Trials) and additional sources including regulatory agency reports were searched from inception to October 2011. Additional data sources include: the manufacturers submission (MS); two previous National Institute for Health and Care Excellence (NICE) single technology appraisal (STA) submissions; and existing reviews on the safety of omalizumab and oral corticosteroids (OCSs). REVIEW METHODS Systematic reviews of the clinical effectiveness and cost-effectiveness evidence for omalizumab were performed. The primary outcome was number of clinically significant (CS) exacerbations. Other outcomes included asthma symptoms, unscheduled health-care use, asthma-related mortality, OCS use and health-related quality of life (HRQoL). Because of methodological and clinical heterogeneity between trials, a narrative synthesis was applied. Pragmatic reviews with best evidence syntheses were used to assess adverse events of omalizumab and OCSs. The cost-effectiveness of omalizumab was assessed from the perspective of the UK NHS in the two separate populations: adults and adolescents, and children, using a cohort Markov model. Costs and outcomes were discounted at 3.5% per annum. Results are presented for additional subgroup populations: (1) hospitalised for asthma in the previous year, (2) adults and adolescents on maintenance OCSs and (3) three or more exacerbations in the previous year. RESULTS Eleven randomised controlled trials (RCTs) and 13 observational studies were identified, including four RCTs/subgroups in the adult licensed population and one subgroup in children. A minority of patients were on maintenance OCSs. No evidence comparing omalizumab with OCSs was identified. Omalizumab significantly reduced the incidence of CS exacerbations in both adults and children [adults: INvestigatioN of Omalizumab in seVere Asthma Trial (INNOVATE): rate ratio 0.74; 95% CI 0.55 to 1.00; children IA-05 EUP (the a priori subgroup of patients who met the European Medicines Agency license criteria) 0.66; 95% CI 0.44 to 1.00]. Significant benefits were observed for a range of other outcomes in adults. Subgroup evidence showed benefits in adults on maintenance OCSs. Evidence for an OCS-sparing effect of omalizumab was limited but consistent. Omalizumab is available as 75 mg and 150 mg prefilled syringes at prices of £128.07 and £256.15 respectively. The incremental cost-effectiveness ratio (ICER) for adults and adolescents is £83,822 per quality-adjusted life-year (QALY) gained, whereas the ICER for children is £78,009 per QALY gained. The results are similar for the subgroup population of ≥ 3 exacerbations in the previous year, whereas the ICER for the other subgroup populations are lower; £46,431 for the hospitalisation subgroup in adults and adolescents, £44,142 for the hospitalisation subgroup in children and £50,181 for the maintenance OCS subgroup. CONCLUSION Omalizumab reduces the incidence of CS exacerbations in adults and children, with benefits on other outcomes in adults. Limited, underpowered subgroup evidence exists that omalizumab reduces exacerbations and OCS requirements in adults on OCSs. Evidence in children is weaker and more uncertain. The ICERs are above conventional NHS thresholds of cost-effectiveness. The key drivers of cost-effectiveness are asthma-related mortality risk and, to a lesser extent, HRQoL improvement and OCS-related adverse effects. An adequately powered double-blind RCT in both adults and children on maintenance OCSs and an individual patient data meta-analysis of existing trials should be considered. A registry of all patients on omalizumab should be established. STUDY REGISTRATION The study was registered as PROSPERO CRD42011001625. FUNDING This report was commissioned by the National Institute for Health Research Health Technology Assessment programme on behalf of NICE as project number HTA 10/128/01.

A systematic review of presumed consent systems for deceased organ donation

OBJECTIVES To examine the impact of presumed consent legislation on organ donation and to review data on attitudes to presumed consent among the public, professionals and any other stakeholders. DATA SOURCES Eight electronic databases (MEDLINE, MEDLINE In-Process, EMBASE, CINAHL, PsycINFO, HMIC, PAIS International and OpenSIGLE) were searched from inception to January 2008. Supplementary internet searches were also performed. REVIEW METHODS A systematic review of studies comparing donation rates in a single country before and after the introduction of a presumed consent law or in countries with and without presumed consent systems. The methodological quality of these studies was assessed and a narrative synthesis of results undertaken. Surveys of attitudes towards presumed consent legislation were also included. RESULTS Over 2000 potentially relevant citations were identified, of which 13 studies met the inclusion criteria for the primary objective and 13 for the secondary objective. For the primary objective, eight studies were between-country comparisons and five were before-and-after studies. Four of the between-country comparisons were of sufficient methodological quality to provide reliable results. In all four studies presumed consent law or practice was associated with increased rates of organ donation, ranging from an increase of 2.7 donors per million population (pmp) in one study to 6.14 donors per million in another, and an increase of between 20% and 30% in two other studies. Factors other than presumed consent that had an impact on organ donation rates were mortality from road traffic accidents and cerebrovascular accident, the transplant capacity of a country, gross domestic product per capita and health expenditure per capita, religion, education, public access to information and a common law legal system. The five before-and-after studies represented three countries, all of which reported an increase in donation rates following the introduction of a presumed consent system (Austria, from 4.6 to 27.2 donors pmp over a 5-year period; Belgium, increase in kidney donation from 10.9 to 41.3 pmp during a 3-year period; Singapore, increase in kidney procurement from 4.7 to 31.3 per year in the 3 years after the change in legislation). There was very limited investigation of any other changes taking place concurrently with the changes in legislation across this set of studies. Of the 13 studies addressing the secondary objective, eight were surveys of the UK public, four were from other countries and one was an international survey of health professionals. There was variation among the UK surveys in the level of support for presumed consent, with surveys conducted before 2000 reporting the lowest levels of support (28-57%). The most recent survey by YouGov in 2007 reported that 64% of respondents supported a change to presumed consent. CONCLUSIONS Presumed consent alone is unlikely to explain the variation in organ donation rates between different countries. A combination of legislation, availability of donors, transplantation system organisation and infrastructure, wealth and investment in health care, as well as underlying public attitudes to and awareness of organ donation and transplantation, may all play a role, although the relative importance of each is not clear. Further reviews could investigate the factors likely to modify donor rates, such as procedures for family involvement. The way in which families of any potential donor are approached is likely to be an important factor and a review of qualitative research examining the experience of relatives in this context would be useful.

Eustachian tube dysfunction: consensus statement on definition, types, clinical presentation and diagnosis.

A recent systematic review of treatments of Eustachian tube dysfunction commissioned by the UK NIHR Health Technology Assessment (HTA) Programme revealed that an important limitation with the available evidence is a lack of consensus on the definition and diagnosis of this disorder.1 The HTA report recommended that key to advancing research in this field is achieving consensus on diagnostic criteria for Eustachian tube dysfunction (to identify eligible patients for future trials) and on important clinical outcomes. To address this need, an international forum of scientists and physicians with expertise in the field of Eustachian tube disorders met at a workshop in Amsterdam on 21 June 2014 and was tasked to come to an agreement on the definition, clinical presentation and diagnosis of Eustachian tube dysfunction, and areas for future research. This study summarises the outcomes of that meeting.

Interventions for adult Eustachian tube dysfunction: a systematic review.

BACKGROUND Eustachian tube dysfunction (ETD) is the inability of the Eustachian tube (ET) to adequately perform at least one of its functions: to protect the middle ear from sources of disease, to ventilate the middle ear, and to help drain secretions away from the middle ear. There are a number of treatment options for ETD, but there is little consensus about management. OBJECTIVES To determine the clinical effectiveness of interventions for adult ETD and to identify gaps in the evidence to inform future research. DATA SOURCES Twelve databases were searched up to October 2012 for published and unpublished studies in English (e.g. MEDLINE from 1946, EMBASE from 1974, Biosis Previews from 1969 and Cumulative Index to Nursing and Allied Health Literature from inception). References of included studies, relevant systematic reviews and regulatory agency websites were checked. REVIEW METHODS A systematic review was undertaken. Controlled studies evaluating prespecified treatments for adult patients diagnosed with ETD were eligible. Uncontrolled studies with at least 10 participants were included for interventions where no controlled studies were found. Outcomes included change in symptoms severity/frequency (primary outcome), quality of life, middle ear function, hearing, clearance of middle ear effusion, early ventilation tube extrusion, additional treatment, adverse events and complications. All aspects of the review process were performed using methods to reduce reviewer error and bias. Owing to heterogeneous data, a quantitative synthesis could not be performed, and results were reported in a narrative synthesis. RESULTS Nineteen studies were included: three randomised controlled trials (RCTs) and two non-RCTs evaluating pharmacological interventions or mechanical devices for middle ear pressure equalisation; and 13 case series and one retrospective controlled before-and-after study evaluating surgical interventions. None was conducted in the UK. All studies were small (11 to 108 participants). Most non-surgical studies reported including mixed populations of adults and children. All except two studies were at high risk of bias, and subject to multiple limitations. Based on a single RCT, nasal steroids showed no improvement in symptoms or middle ear function for patients with otitis media with effusion and/or negative middle ear pressure. Very short-term improvements in middle ear function were observed in patients receiving directly applied topical decongestants or a combination of antihistamine and ephedrine. Single trials found two pressure equalisation devices were each associated with significant short-term improvements in symptoms, middle ear function and/or hearing. Eustachian tuboplasty (seven case series) and balloon dilatation (three case series) were associated with improved outcomes. Positive results were also reported for myringotomy (two case series), directly applied topical steroids (one case series) and laser point coagulation (one controlled before-and-after study). High rates of co-interventions were documented. Minor complications of surgery and pharmacological treatments but no serious adverse effects were reported. LIMITATIONS The evidence was limited in quantity and overall was of poor quality. No data were identified on several interventions despite extensive searches. CONCLUSIONS It is not possible to draw conclusions regarding the effectiveness of any of the interventions for the treatment of adults with an ETD diagnosis, and there is insufficient evidence to recommend a trial of any particular intervention. Further research is needed to address lack of consensus on several issues, including the definition of ETD in adults, its relation to broader middle ear ventilation problems and clear diagnostic criteria. STUDY REGISTRATION This study is registered as PROSPERO CRD42012003035. FUNDING The National Institute for Health Research Health Technology Assessment programme.

Trastuzumab for the treatment of HER2-positive metastatic gastric cancer : a NICE single technology appraisal.

The National Institute for Health and Clinical Excellence (NICE) invited the manufacturer of trastuzumab, Roche Pharmaceuticals, to submit evidence for the clinical and cost effectiveness of this drug for the treatment of advanced gastric cancer (aGC), as part of the Institute’s single technology appraisal (STA) process. The Centre for Reviews and Dissemination (CRD) and the Centre for Health Economics (CHE) Technology Appraisal Group at the University of York was commissioned to act as the evidence review group (ERG). This article provides a description of the company submission, the ERG report and NICE’s subsequent decisions. In the initial appraisal by NICE, trastuzumab was rejected for use in the licensed population. Given this result, the manufacturer submitted additional evidence. In the final appraisal decision, trastuzumab was approved, in accordance with supplementary guidance issued by NICE on appraising life-extending, end-of-life treatments, for patients whose human epidermal growth factor receptor 2 (HER2) status was defined by an immunohistochemistry 3 positive (IHC3+) result. This appraisal highlights the need to fully assess the impact of different approaches to diagnostic testing on the cost effectiveness of targeted treatments. In this appraisal, it was found that the diagnostic strategy influenced the effectiveness and cost of trastuzumab. In the future, different diagnostic strategies should be compared in the incremental cost-effectiveness analysis.

Systematic review of the limited evidence base for treatments of Eustachian tube dysfunction: a health technology assessment

The Health Technology Assessment programme commissioned a wide‐ranging review of treatments for adult Eustachian tube dysfunction. Treatments range from advice and observation and pharmacological treatments to surgical options.

Rimonabant for the treatment of overweight and obese people.

This paper presents a summary of the evidence review group (ERG) report into the clinical and cost-effectiveness of rimonabant for the treatment of obese or overweight patients based upon a review of the manufacturers submission to the National Centre for Health and Clinical Excellence (NICE) as part of the single technology appraisal (STA) process. The submissions main evidence came from four randomised controlled trials. Rimonabant resulted in a significantly greater benefit than placebo for all primary weight loss outcomes. At 1 year, rimonabant had a statistically significant beneficial effect on systolic blood pressure, high-density lipoprotein cholesterol, triglycerides and fasting plasma glucose in diabetics and non-diabetics, and glycosylated haemoglobin in diabetics. Improvements were maintained over 2 years with rimonabant; withdrawal of rimonabant at 1 year resulted in a reduction in weight loss until there was no difference from placebo at 2 years. Psychiatric adverse events were experienced by 26% and 14% of rimonabant and placebo patients respectively; figures for symptoms of depression were 9% and 5% respectively. Pairwise comparisons of orlistat, sibutramine and rimonabant showed beneficial effects of rimonabant over orlistat and sibutramine for weight loss outcomes; however, response hurdles imposed on orlistat or sibutramine in clinical practice may not have been applied in the orlistat and sibutramine trials. The manufacturers Markov cohort model evaluated rimonabant versus orlistat, sibutramine and diet and exercise alone for three base-case populations. The incremental cost-effectiveness ratio (ICER) of rimonabant varied from 10,534 pounds to 13,236 pounds per quality-adjusted life-year (QALY) versus diet and exercise, to 8977 pounds to 12,138 pounds per QALY versus orlistat, to 1463 pounds to 3908 pounds per QALY versus sibutramine. In subgroup analysis there was a wider variation in the ICER estimates although none exceeded 20,000 pounds per QALY. The ICER of rimonabant remained under 20,000 pounds per QALY in reanalyses by the manufacturer and the ERG, with the results sensitive to the source of health-related quality of life (HRQoL) benefits in the model. Four treatment strategies were modelled in comparisons of rimonabant versus diet and exercise alone and orlistat and sibutramine in which rimonabant was continued only in patients achieving 5% weight loss at 3, 6, 9 or 12 months. In pairwise comparisons rimonabant remained below a threshold of 30,000 pounds per QALY in 70% of the comparisons reported. The results were most sensitive to the decrement applied to depression and the costs of screening for depression. In conclusion, areas of uncertainty remain in relation to the clinical effectiveness and cost-effectiveness of rimonabant, for example lack of evidence on long-term outcomes and the effect of rimonabant on cardiovascular events, developing diabetes and mortality, and lack of data on the HRQoL benefits associated with rimonabant. The lack of response hurdles applied to sibutramine and orlistat means that the comparator strategies were not considered by the ERG to reflect their respective product licenses or current NHS use. The NICE guidance issued as a result of the STA states that rimonabant is recommended as an adjunct to diet and exercise for adults who are obese or overweight and who have had an inadequate response to, are intolerant of or are contraindicated to orlistat and sibutramine.

An emerging evidence base for PET-CT in the management of childhood rhabdomyosarcoma: systematic review

Introduction Rhabdomyosarcoma (RMS) management depends on risk stratification at diagnosis and treatment response. Assessment methods include CT, MRI, bone scintigraphy, histological analysis and bone marrow biopsy. Advanced functional imaging (FI) has potential to improve staging accuracy and management strategies. Methods and analysis We conducted a systematic review (PROSPERO 2013:CRD42013006128) of diagnostic accuracy and clinical effectiveness of FI in histologically proven paediatric RMS. PRISMA guidance was followed. We searched 10 databases to November 2013. Studies with ≥10 patients with RMS which compared positron emission tomography (PET), PET-CT or diffusion-weighted imaging (DWI) MRI to conventional imaging at any treatment stage were included. Study quality was assessed. Limited, heterogeneous effectiveness data required narrative synthesis, illustrated by plotting sensitivity and specificity in receiver operating curve (ROC) space. Results Eight studies (six PET-CT, two PET) with 272 RMS patients in total were included. No DWI-MRI studies met inclusion criteria. Pooled estimates were not calculated due to sparseness of data. Limited evidence indicated initial PET-CT results were predictive of survival. PET-CT changed management of 7/40 patients. Nodal involvement PET-CT: sensitivity ranged from 80% to 100%; specificity from 89% to 100%. Distant metastatic involvement: PET-CT sensitivity ranged from 95% to 100%; specificity from 80% to100%. Data on metastases in different sites were sparse. Limited data were found on outcome prediction by PET-CT response. Dissemination and ethics PET/PET-CT may increase initial staging accuracy in paediatric RMS, specifically in the detection of nodal involvement and distant metastatic spread. There is a need to further assess PET-CT for this population, ideally in a representative, unbiased and transparently selected cohort of patients.

Parenteral oestrogen in the treatment of prostate cancer: a systematic review

The objectives of this study were to assess the effectiveness and safety of parenteral oestrogen in the treatment of prostate cancer, and to examine any dose relationship. A systematic review was undertaken. Electronic databases, published paper and internet resources were searched to locate published and unpublished studies with no restriction by language or publication date. Studies included were randomised controlled trials of parenteral oestrogen in patients with prostate cancer; other study designs were also included to examine dose–response. Study selection, appraisal, data extraction and quality assessment were performed by one reviewer and independently checked by another. Twenty trials were included in the review. The trials differed with regard to the included patients, formulation and dose of parenteral oestrogen, comparator used, outcome measures reported and the duration of follow-up. The results provide no evidence to suggest that parenteral oestrogen, in doses sufficient to produce castrate levels of testosterone, is less effective than luteinising hormone-releasing hormone (LHRH) or orchidectomy in controlling prostate cancer, or that it is consistently associated with an increase in cardiovascular mortality. Further well-conducted trials of parenteral oestrogen are required. A pilot randomised controlled trial comparing transdermal oestrogen to LHRH analogues in men with locally advanced or metastatic prostate cancer is underway in the United Kingdom.