Sibylle Sturtz

Unsolved issues of mixed treatment comparison meta-analysis: network size and inconsistency.

Indirect comparisons and mixed treatment comparison (MTC) meta-analyses are increasingly used in medical research. These methods allow a simultaneous analysis of all relevant interventions in a connected network even if direct evidence regarding two interventions is missing. The framework of MTC meta-analysis provides a flexible approach for complex networks. However, this method has yet some unsolved problems, in particular the choice of the network size and the assessment of inconsistency. In this paper, we describe the practical application of MTC meta-analysis by using a data set on antidepressants. We focus on the impact of the size of the chosen network and the assumption of consistency. A larger network is based on more evidence but may show inconsistencies, whereas a smaller network contains less evidence but may show no clear inconsistencies. A choice is required which network should be used in practice. In summary, MTC meta-analysis represents a promising approach; however, clear application standards are still lacking. Especially, standards for the identification of inconsistency and the way to deal with potential inconsistency are required. Copyright

Indirect Comparisons and Network Meta-Analyses.

BACKGROUND Systematic reviews provide a structured summary of the results of trials that have been carried out on any particular subject. If the data from multiple trials are sufficiently homogenous, a meta-analysis can be performed to calculate pooled effect estimates. Traditional meta-analysis involves groups of trials that compare the same two interventions directly (head to head). Lately, however, indirect comparisons and network metaanalyses have become increasingly common. METHODS Various methods of indirect comparison and network meta-analysis are presented and discussed on the basis of a selective review of the literature. The main assumptions and requirements of these methods are described, and a checklist is provided as an aid to the evaluation of published indirect comparisons and network meta-analyses. RESULTS When no head-to-head trials of two interventions are available, indirect comparisons and network metaanalyses enable the estimation of effects as well as the simultaneous analysis of networks involving more than two interventions. Network meta-analyses and indirect comparisons can only be useful if the trial or patient characteristics are similar and the observed effects are sufficiently homogeneous. Moreover, there should be no major discrepancy between the direct and indirect evidence. If trials are available that compare each of two treatments against a third one, but not against each other, then the third intervention can be used as a common comparator to enable a comparison of the other two. CONCLUSION Indirect comparisons and network metaanalyses are an important further development of traditional meta-analysis. Clear and detailed documentation is needed so that findings obtained by these new methods can be reliably judged.

Absolute risks rather than incidence rates should be used to estimate the number needed to treat from time-to-event data.

BACKGROUND When estimating the number needed to treat (NNT) from randomized controlled trials (RCTs) with time-to-event outcomes, varying follow-up times have to be considered. Two methods have been proposed, namely (1) inverting risk differences estimated by survival time methods (RD approach) and (2) inverting incidence differences (ID approach). STUDY DESIGN AND SETTING A simulation study was conducted to compare the RD and the ID approaches regarding bias and coverage probability (CP) considering various distributions, baseline risks, effect sizes, and sample sizes. Additionally, the two approaches were compared by using two real data examples. RESULTS The RD approach showed good estimation and coverage properties with only a few exceptions in the case of small sample sizes and small effect sizes. The ID approach showed considerable bias and low CPs in most of the considered data situations. CONCLUSIONS Absolute risks estimated by means of survival time methods rather than incidence rates should be used to estimate NNTs in RCTs with time-to-event outcomes.

Assumptions of Mixed Treatment Comparisons in Health Technology Assessments - Challenges and Possible Steps for Practical Application

The validity of mixed treatment comparisons (MTCs), also called network meta-analysis, relies on whether it is reasonable to accept the underlying assumptions on similarity, homogeneity, and consistency. The aim of this paper is to propose a practicable approach to addressing the underlying assumptions of MTCs. Using data from clinical studies of antidepressants included in a health technology assessment (HTA), we present a stepwise approach to dealing with challenges related to checking the above assumptions and to judging the robustness of the results of an MTC. At each step, studies that were dissimilar or contributed to substantial heterogeneity or inconsistency were excluded from the primary analysis. In a comparison of the MTC estimates from the consistent network with the MTC estimates from the homogeneous network including inconsistencies, few were affected by notable changes; that is, a change in effect size (factor 2), direction of effect or statistical significance. Considering the small proportion of studies excluded from the network due to inconsistency, as well as the number of notable changes, the MTC results were deemed sufficiently robust. In the absence of standard methods, our approach to checking assumptions in MTCs may inform other researchers in need of practical options, particularly in HTA.

Trial registry searches for RCTs of new drugs required registry-specific adaptation to achieve adequate sensitivity

OBJECTIVES To analyze the availability of randomized controlled trials (RCTs) of new drugs in trial registries and to develop and test different search strategies in ClinicalTrials.gov (CT.gov), the EU Clinical Trials Register (EU-CTR), and the International Clinical Trials Registry Platform (ICTRP). STUDY DESIGN AND SETTING Information from dossiers submitted by pharmaceutical companies was analyzed regarding the registration of the included RCTs in CT.gov, EU-CTR and ICTRP; different search strategies were developed and tested to determine performance. RESULTS A total of 192 (95%) of 203 RCTs on newly approved drugs were registered in CT.gov; the 11 nonregistered trials were completed before 2005 or represented non-RCTs. Simple searches for RCTs on 18 new drugs using the generic drug name yielded a sensitivity of 94% in CT.gov (EU-CTR: 71%; ICTRP: 60%). The main reason for study nondetection was the sole use of the drug code in the registry entries. Simple searches for RCTs on 13 conditions using reasonably inferred search terms yielded a sensitivity of 100% in CT.gov. CONCLUSION Almost all relevant RCTs on newly approved drugs will probably be identified in CT.gov alone. A sensitive search in CT.gov can be conducted using single search terms. The searches in ICTRP and EU-CTR should include several search terms (e.g., derived via text analysis).

Comparison of Bayesian methods for flexible modeling of spatial risk surfaces in disease mapping.

Bayesian hierarchical models usually model the risk surface on the same arbitrary geographical units for all data sources. Poisson/gamma random field models overcome this restriction as the underlying risk surface can be specified independently to the resolution of the data. Moreover, covariates may be considered as either excess or relative risk factors. We compare the performance of the Poisson/gamma random field model to the Markov random field (MRF)-based ecologic regression model and the Bayesian Detection of Clusters and Discontinuities (BDCD) model, in both a simulation study and a real data example. We find the BDCD model to have advantages in situations dominated by abruptly changing risk while the Poisson/gamma random field model convinces by its flexibility in the estimation of random field structures and by its flexibility incorporating covariates. The MRF-based ecologic regression model is inferior. WinBUGS code for Poisson/gamma random field models is provided.