Anne-Marie Glenny

Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses

Abstract Objective: To determine the validity of adjusted indirect comparisons by using data from published meta-analyses of randomised trials. Design: Direct comparison of different interventions in randomised trials and adjusted indirect comparison in which two interventions were compared through their relative effect versus a common comparator. The discrepancy between the direct and adjusted indirect comparison was measured by the difference between the two estimates. Data sources: Database of abstracts of reviews of effectiveness (1994-8), the Cochrane database of systematic reviews, Medline, and references of retrieved articles. Results: 44 published meta-analyses (from 28 systematic reviews) provided sufficient data. In most cases, results of adjusted indirect comparisons were not significantly different from those of direct comparisons. A significant discrepancy (P<0.05) was observed in three of the 44 comparisons between the direct and the adjusted indirect estimates. There was a moderate agreement between the statistical conclusions from the direct and adjusted indirect comparisons (κ 0.51). The direction of discrepancy between the two estimates was inconsistent. Conclusions: Adjusted indirect comparisons usually but not always agree with the results of head to head randomised trials. When there is no or insufficient direct evidence from randomised trials, the adjusted indirect comparison may provide useful or supplementary information on the relative efficacy of competing interventions. The validity of the adjusted indirect comparisons depends on the internal validity and similarity of the included trials. What is already known on this topic Many competing interventions have not been compared in randomised trials Indirect comparison of competing interventions has been carried out in systematic reviews, often implicitly Indirect comparison adjusted by a common control can partially take account of prognostic characteristics of patients in different trials What this study adds Results of adjusted indirect comparison usually, but not always, agree with those of head to head randomised trials The validity of adjusted indirect comparisons depends on the internal validity and similarity of the trials involved

The treatment and prevention of obesity: a systematic review of the literature.

OBJECTIVE: To determine the effectiveness of interventions designed to prevent and treat obesity, and maintain weight loss. DESIGN: A systematic review of randomised controlled trials. SUBJECTS: Overweight and obese adults and children. MEASUREMENTS: Post-intervention changes in weight, fat content and fat distribution, measured relative to baseline. RESULTS: For obese children, family therapy and lifestyle modification appear to be effective in prevention and treatment, respectively. The effectiveness of interventions to prevent and treat obesity in adults remains unclear, although behavioural therapy and multicomponent strategies may be useful. Continued therapist contact appears to be useful for maintaining weight loss. Pharmacological interventions appear to be effective for up to 9 months, after which regain occurs. Surgery appears to be effective for the morbidly obese and gastric bypass is more effective than gastroplasty. In general, the methodological quality of studies was poor. CONCLUSION: Due to problems with methodological quality, it is recommended that research findings indicative of promising interventions are replicated. EDITORIAL NOTE: The full version of this review contained tabular and illustrative information that was too extensive and detailed to publish in the journal. This information is available on the CRD website (http://www.york.ac.uk/inst/crd/obesity.htm), or in the full CRD publication (contact first author).

Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews

Objective To investigate basic assumptions and other methodological problems in the application of indirect comparison in systematic reviews of competing healthcare interventions. Design Survey of published systematic reviews. Inclusion criteria Systematic reviews published between 2000 and 2007 in which an indirect approach had been explicitly used. Data extraction Identified reviews were assessed for comprehensiveness of the literature search, method for indirect comparison, and whether assumptions about similarity and consistency were explicitly mentioned. Results The survey included 88 review reports. In 13 reviews, indirect comparison was informal. Results from different trials were naively compared without using a common control in six reviews. Adjusted indirect comparison was usually done using classic frequentist methods (n=49) or more complex methods (n=18). The key assumption of trial similarity was explicitly mentioned in only 40 of the 88 reviews. The consistency assumption was not explicit in most cases where direct and indirect evidence were compared or combined (18/30). Evidence from head to head comparison trials was not systematically searched for or not included in nine cases. Conclusions Identified methodological problems were an unclear understanding of underlying assumptions, inappropriate search and selection of relevant trials, use of inappropriate or flawed methods, lack of objective and validated methods to assess or improve trial similarity, and inadequate comparison or inappropriate combination of direct and indirect evidence. Adequate understanding of basic assumptions underlying indirect and mixed treatment comparison is crucial to resolve these methodological problems. Appendix 1 PubMed search strategy Appendix 2 Characteristics of identified reports Appendix 3 Identified studies References of included studies

Inconsistency between direct and indirect comparisons of competing interventions: meta-epidemiological study

Objective To investigate the agreement between direct and indirect comparisons of competing healthcare interventions. Design Meta-epidemiological study based on sample of meta-analyses of randomised controlled trials. Data sources Cochrane Database of Systematic Reviews and PubMed. Inclusion criteria Systematic reviews that provided sufficient data for both direct comparison and independent indirect comparisons of two interventions on the basis of a common comparator and in which the odds ratio could be used as the outcome statistic. Main outcome measure Inconsistency measured by the difference in the log odds ratio between the direct and indirect methods. Results The study included 112 independent trial networks (including 1552 trials with 478 775 patients in total) that allowed both direct and indirect comparison of two interventions. Indirect comparison had already been explicitly done in only 13 of the 85 Cochrane reviews included. The inconsistency between the direct and indirect comparison was statistically significant in 16 cases (14%, 95% confidence interval 9% to 22%). The statistically significant inconsistency was associated with fewer trials, subjectively assessed outcomes, and statistically significant effects of treatment in either direct or indirect comparisons. Owing to considerable inconsistency, many (14/39) of the statistically significant effects by direct comparison became non-significant when the direct and indirect estimates were combined. Conclusions Significant inconsistency between direct and indirect comparisons may be more prevalent than previously observed. Direct and indirect estimates should be combined in mixed treatment comparisons only after adequate assessment of the consistency of the evidence.

Cochrane Reviews on the Benefits/Risks of Fluoride Toothpastes:

This concise review presents two Cochrane Reviews undertaken to determine: (1) the relative effectiveness of fluoride toothpastes of different concentrations in preventing dental caries in children and adolescents; and (2) the relationship between the use of topical fluorides in young children and their risk of developing dental fluorosis. To determine the relative effectiveness of fluoride toothpastes of different concentrations, we undertook a network meta-analysis utilizing both direct and indirect comparisons from randomized controlled trials (RCTs). The review examining fluorosis included evidence from experimental and observational studies. The findings of the reviews confirm the benefits of using fluoride toothpaste, when compared with placebo, in preventing caries in children and adolescents, but only significantly for fluoride concentrations of 1000 ppm and above. The relative caries-preventive effects of fluoride toothpastes of different concentrations increase with higher fluoride concentration. However, there is weak, unreliable evidence that starting the use of fluoride toothpaste in children under 12 months of age may be associated with an increased risk of fluorosis. The decision of what fluoride levels to use for children under 6 years should be balanced between the risk of developing dental caries and that of mild fluorosis.

Indirect comparison in evaluating relative efficacy illustrated by antimicrobial prophylaxis in colorectal surgery.

This paper aims to explore the potential usefulness and limitations of indirect comparisons in evaluating the relative efficacy of interventions. From a systematic review of antimicrobial prophylaxis in colorectal surgery, we identified 11 sets of randomized trials that can be used to compare antibiotics both directly and indirectly. The discrepancy between the direct and the indirect comparison is defined as the absolute value of difference in log odds ratio. The adjusted indirect comparison has the advantages that the prognostic factors of participants in different trials can be partially taken into account and more uncertainty be incorporated into its result by providing a wider confidence interval. However, considerable discrepancies exist between the direct and the adjusted indirect comparisons. When there is no direct comparison, the adjusted indirect method may be used to obtain some evidence about the relative efficacy of competing interventions, although such indirect results should be interpreted with great caution. Further empirical and methodologic research is needed to explore the validity and generalizability of the adjusted indirect comparison for evaluating different interventions.

An updated systematic review of interventions to improve health professionals’ management of obesity

The objective of this article was twofold (1) to determine the existence and effectiveness of interventions to improve health professionals’ management of obesity or the organization of care for overweight and obese people; and (2) to update a previous systematic review on this topic with new or additional studies. The study design was a systematic review of intervention studies, undertaken according to standard methods developed by the Cochrane Effective Practice and Organization of Care (EPOC) Group. Participants were trained health care professionals and overweight and obese patients. The measurements were objective measures of health professionals’ practice and behaviours, and patient outcomes including satisfaction, behaviour, psychological factors, disease status, risk factors and measures of body weight, fat, or body mass index (BMI). Twelve studies were included in the original review. A further six were included in this update. Six of the 18 studies were randomized controlled trials of health professional‐orientated interventions (such as the use of reminders and training) and one was a controlled before‐and‐after study to improve collaboration between a hospital clinic and general practitioners (GPs). Ten randomized controlled trials and two controlled clinical trials of interventions comparing either the deliverer of weight‐loss interventions or the setting of the delivery of the intervention, were identified. The heterogeneity and generally limited quality of identified studies make it difficult to provide recommendations for improving health professionals’ obesity management. To conclude, at present, there are few solid leads about improving obesity management, although reminder systems, brief training interventions, shared care, inpatient care and dietitian‐led treatments may all be worth further investigation. Therefore, decisions for the improvement of provision of services must be based on the existing evidence on interventions with patients and good clinical judgement. Further research is needed to identify cost‐effective strategies for improving the management of obesity. A full version of this review (including detailed descriptions of the included studies and their methodological quality, and results and excluded studies tables) is available in the Cochrane Library. The Cochrane Library is a database of systematic review and other evidence on the effects of health care, continuously updated as new information emerges. It is available on CD ROM from Update Software. For further information see: http://www.update‐software.com/cochrane

Parents’ Emotional and Social Experiences of Caring for a Child Through Cleft Treatment

Little is known about the experiences of parents caring for a child through long-term treatment for cleft lip and/or cleft palate. We conducted in-depth interviews with 35 parents with children between the ages of 20 weeks and 21 years to explore experiences across the treatment program. We analyzed the data using a constructivist grounded theory approach and present in detail in this article one subcategory from the analysis: managing emotions. Throughout childhood and adolescence, parents experienced conflicting emotions about their child’s impairment, uncertainty about cleft treatment, and stigmatizing attitudes. Although parents attempted to manage emotional tensions by pursuing cleft treatments, the interventions could themselves be a source of conflict for them. We suggest that routine assessment of parents’ emotional and social well-being should be included in cleft treatment programs, and access to psychosocial support made available.

Interventions for replacing missing teeth: denture chewing surface designs in edentulous people

Reason for withdrawal from publication The Cochrane Oral Health Group withdrew this review as of Issue 12, 2015. The review is out of date and does not meet current Cochrane methodological standards.

A systematic review of interventions to improve health professionals' management of obesity

OBJECTIVE: To determine the existence and effectiveness of interventions to improve health professionals’ management of obesity or the organisation of care for overweight and obese people.DESIGN: A systematic review of intervention studies, undertaken according to standard methods developed by the Cochrane Effective Practice and Organisation of Care (EPOC) Group.PARTICIPANTS: Trained health care professionals and overweight and obese patients.MEASUREMENTS: Objective measures of health professionals’ practice and behaviours, and patient outcomes including satisfaction, behaviour, psychological factors, disease status, risk factors, and measures of body weight, fat, or BMI.RESULTS: Twelve studies that met all the review inclusion criteria were identified. Three were randomised controlled trials of health professional-oriented interventions (such as the use of reminders and training) and one was a controlled before and after study to improve collaboration between a hospital clinic and GPs. A further eight randomised controlled trials were identified of interventions comparing either the deliverer of weight loss interventions or the setting of the delivery of the intervention. The heterogeneity and generally limited quality of identified studies make it difficult to provide recommendations for improving health professionals’ obesity management.CONCLUSIONS: At present, decisions about improving the provision of services for overweight and obese people must be based on the evidence from patient interventions and good clinical judgement. Future research is required to identify cost-effective strategies for improving health professionals’ management and the organisation of care for overweight and obese people.A full version of this review, including detailed descriptions of the included studies and data tables, is available in the Cochrane Library. The Cochrane Library is a database of systematic review and other evidence on the effects of health care, available on CD ROM from Update Software. For further information see: http://www.cochrane.co.uk.