Sylwia Bujkiewicz

How valuable are multiple treatment comparison methods in evidence-based health-care evaluation?

OBJECTIVES To compare the use of pair-wise meta-analysis methods to multiple treatment comparison (MTC) methods for evidence-based health-care evaluation to estimate the effectiveness and cost-effectiveness of alternative health-care interventions based on the available evidence. METHODS Pair-wise meta-analysis and more complex evidence syntheses, incorporating an MTC component, are applied to three examples: 1) clinical effectiveness of interventions for preventing strokes in people with atrial fibrillation; 2) clinical and cost-effectiveness of using drug-eluting stents in percutaneous coronary intervention in patients with coronary artery disease; and 3) clinical and cost-effectiveness of using neuraminidase inhibitors in the treatment of influenza. We compare the two synthesis approaches with respect to the assumptions made, empirical estimates produced, and conclusions drawn. RESULTS The difference between point estimates of effectiveness produced by the pair-wise and MTC approaches was generally unpredictable-sometimes agreeing closely whereas in other instances differing considerably. In all three examples, the MTC approach allowed the inclusion of randomized controlled trial evidence ignored in the pair-wise meta-analysis approach. This generally increased the precision of the effectiveness estimates from the MTC model. CONCLUSIONS The MTC approach to synthesis allows the evidence base on clinical effectiveness to be treated as a coherent whole, include more data, and sometimes relax the assumptions made in the pair-wise approaches. However, MTC models are necessarily more complex than those developed for pair-wise meta-analysis and thus could be seen as less transparent. Therefore, it is important that model details and the assumptions made are carefully reported alongside the results.

The effectiveness of anti-TNF-alpha therapies when used sequentially in rheumatoid arthritis patients: a systematic review and meta-analysis.

Objectives. To systematically review and meta-analyse evidence on the effectiveness of the TNF-α inhibitors when used sequentially. Methods. Systematic review of comparative and single-arm observational studies. Data were synthesized using random-effects meta-analysis. Treatment effects were estimated using four outcome measures from the included studies: European League Against Rheumatism (EULAR) and ACR20 response rates and mean improvement in disease activity score-28 (DAS-20) and HAQ. The effect of other factors was explored via meta-regression and sub-group analyses. Results. Twenty studies comprising 2705 patients were included in the analysis. All studies were observational and most had no control group. Therefore, our primary analysis considered patient changes from baseline. The mean percentage of ACR20 responders was 60.8% (95% CI 53.8, 67.4), EULAR responders 70.5% (95% CI 63.7, 76.6), mean overall improvement in DAS-28 scores was 1.53 (95% CI 1.25, 1.80) and in HAQ scores was 0.25 (95% CI 0.11, 0.40). Four studies made comparisons with patients who received TNF-α inhibitors for the first time. Response rates associated with sequential TNF-α inhibitor treatment were lower than for first-time use. Conclusions. Sequential TNF-α inhibitor use is likely to lead to treatment benefit in terms of the signs and symptoms of disease and physical function. There is also some evidence to suggest that the probability of achieving a response is lower, and the average magnitude of response is lower than the first use. Further evidence from randomized controlled trials is required to confirm and further quantify the role specific anti-TNF-α agents have when used sequentially.

Multivariate meta-analysis of mixed outcomes: a Bayesian approach

Multivariate random effects meta-analysis (MRMA) is an appropriate way for synthesizing data from studies reporting multiple correlated outcomes. In a Bayesian framework, it has great potential for integrating evidence from a variety of sources. In this paper, we propose a Bayesian model for MRMA of mixed outcomes, which extends previously developed bivariate models to the trivariate case and also allows for combination of multiple outcomes that are both continuous and binary. We have constructed informative prior distributions for the correlations by using external evidence. Prior distributions for the within-study correlations were constructed by employing external individual patent data and using a double bootstrap method to obtain the correlations between mixed outcomes. The between-study model of MRMA was parameterized in the form of a product of a series of univariate conditional normal distributions. This allowed us to place explicit prior distributions on the between-study correlations, which were constructed using external summary data. Traditionally, independent ‘vague’ prior distributions are placed on all parameters of the model. In contrast to this approach, we constructed prior distributions for the between-study model parameters in a way that takes into account the inter-relationship between them. This is a flexible method that can be extended to incorporate mixed outcomes other than continuous and binary and beyond the trivariate case. We have applied this model to a motivating example in rheumatoid arthritis with the aim of incorporating all available evidence in the synthesis and potentially reducing uncertainty around the estimate of interest.

Network meta-analysis of multiple outcome measures accounting for borrowing of information across outcomes

BackgroundNetwork meta-analysis (NMA) enables simultaneous comparison of multiple treatments while preserving randomisation. When summarising evidence to inform an economic evaluation, it is important that the analysis accurately reflects the dependency structure within the data, as correlations between outcomes may have implication for estimating the net benefit associated with treatment. A multivariate NMA offers a framework for evaluating multiple treatments across multiple outcome measures while accounting for the correlation structure between outcomes.MethodsThe standard NMA model is extended to multiple outcome settings in two stages. In the first stage, information is borrowed across outcomes as well across studies through modelling the within-study and between-study correlation structure. In the second stage, we make use of the additional assumption that intervention effects are exchangeable between outcomes to predict effect estimates for all outcomes, including effect estimates on outcomes where evidence is either sparse or the treatment had not been considered by any one of the studies included in the analysis. We apply the methods to binary outcome data from a systematic review evaluating the effectiveness of nine home safety interventions on uptake of three poisoning prevention practices (safe storage of medicines, safe storage of other household products, and possession of poison centre control telephone number) in households with children. Analyses are conducted in WinBUGS using Markov Chain Monte Carlo (MCMC) simulations.ResultsUnivariate and the first stage multivariate models produced broadly similar point estimates of intervention effects but the uncertainty around the multivariate estimates varied depending on the prior distribution specified for the between-study covariance structure. The second stage multivariate analyses produced more precise effect estimates while enabling intervention effects to be predicted for all outcomes, including intervention effects on outcomes not directly considered by the studies included in the analysis.ConclusionsAccounting for the dependency between outcomes in a multivariate meta-analysis may or may not improve the precision of effect estimates from a network meta-analysis compared to analysing each outcome separately.

Development of a Transparent Interactive Decision Interrogator to Facilitate the Decision-Making Process in Health Care

Background Decisions about the use of new technologies in health care are often based on complex economic models. Decision makers frequently make informal judgments about evidence, uncertainty, and the assumptions that underpin these models. Objectives Transparent interactive decision interrogator (TIDI) facilitates more formal critique of decision models by decision makers such as members of appraisal committees of the National Institute for Health and Clinical Excellence in the UK. By allowing them to run advanced statistical models under different scenarios in real time, TIDI can make the decision process more efficient and transparent, while avoiding limitations on pre-prepared analysis. Methods TIDI, programmed in Visual Basic for applications within Excel, provides an interface for controlling all components of a decision model developed in the appropriate software (e.g., meta-analysis in WinBUGS and the decision model in R) by linking software packages using RExcel and R2WinBUGS. TIDIs graphical controls allow the user to modify assumptions and to run the decision model, and results are returned to an Excel spreadsheet. A tool displaying tornado plots helps to evaluate the influence of individual parameters on the model outcomes, and an interactive meta-analysis module allows the user to select any combination of available studies, explore the impact of bias adjustment, and view results using forest plots. We demonstrate TIDI using an example of a decision model in antenatal care. Conclusion Use of TIDI during the NICE appraisal of tumor necrosis factor-alpha inhibitors (in psoriatic arthritis) successfully demonstrated its ability to facilitate critiques of the decision models by decision makers.

Nutritional problems in children treated for medulloblastoma: Implications for enteral nutrition support

The aim of this study was to identify the nature and severity of nutritional problems associated with the current treatment of medulloblastoma and to identify any risk factors for nutritional morbidity during treatment.

Presentational approaches used in the UK for reporting evidence synthesis using indirect and mixed treatment comparisons

Objectives To establish current guidance and practice in UK on presentation of indirect comparison and mixed treatment comparison analyses; to provide recommendations to improve indirect comparison/mixed treatment comparison reporting and to identify research priorities for improved presentation. Methods Existing institutional guidance for conducting indirect comparison/mixed treatment comparison alongside current practice in health technology assessment was reviewed. Reports published in UK by the Health Technology Assessment programme since 1997, which utilized indirect comparison/mixed treatment comparison methods, were reviewed with respect to the presentation of study data, statistical models and results. Recommendations for presentation were developed. Results Guidance exists that provide the details necessary to conduct a successful indirect comparison/mixed treatment comparison analysis but recommendations on presentation are limited. Of 205 health technology assessment reports that contained evidence synthesis for effectiveness, 19 used indirect comparison/mixed treatment comparison methods. These reports utilized numerous presentational formats from which the following key components were identified: network table/diagram for presenting data; model description to allow reproducibility; and tables, forest plots, matrix tables and summary forest plots for presenting a range of results. Recommendations were developed to ensure that reporting is explicit, transparent and reproducible. Approaches most understandable by non-technical decision makers, and areas where future research is required, are outlined. Conclusions There is no standard presentational style used in UK for reporting indirect comparison/mixed treatment comparison, and the use of graphical tools is limited. Standardization of reporting and innovation in graphical representation of indirect comparison/mixed treatment comparison results is required.

Use of bayesian multivariate meta-analysis to estimate the HAQ for mapping onto the EQ-5D questionnaire in rheumatoid arthritis

Background In health technology assessment, decisions about reimbursement for new health technologies are largely based on effectiveness estimates. Sometimes, however, the target effectiveness estimates are not readily available. This may be because many alternative instruments measuring these outcomes are being used (and not all always reported) or an extended follow-up time of clinical trials is needed to evaluate long-term end points, leading to the limited data on the target clinical outcome. In the areas of highest priority in health care, decisions are required to be made on a short time scale. Therefore, alternative clinical outcomes, including surrogate end points, are increasingly being considered for use in evidence synthesis as part of economic evaluation. Objective To illustrate the potential effect of reduced uncertainty around the clinical outcome on the utility when estimating it from a multivariate meta-analysis. Methods Bayesian multivariate meta-analysis has been used to synthesize data on correlated outcomes in rheumatoid arthritis and to incorporate external data in the model in the form of informative prior distributions. Estimates of Health Assessment Questionnaire were then mapped onto the health-related quality-of-life measure EuroQol five-dimensional questionnaire, and the effect was compared with mapping the Health Assessment Questionnaire obtained from the univariate approach. Results The use of multivariate meta-analysis can lead to reduced uncertainty around the effectiveness parameter and ultimately uncertainty around the utility. Conclusions By allowing all the relevant data to be incorporated in estimating clinical effectiveness outcomes, multivariate meta-analysis can improve the estimation of health utilities estimated through mapping methods. While reduced uncertainty may have an effect on decisions based on economic evaluation of new health technologies, the use of short-term surrogate end points can allow for early decisions. More research is needed to determine the circumstances under which uncertainty is reduced.

Bayesian meta‐analytical methods to incorporate multiple surrogate endpoints in drug development process

A number of meta‐analytical methods have been proposed that aim to evaluate surrogate endpoints. Bivariate meta‐analytical methods can be used to predict the treatment effect for the final outcome from the treatment effect estimate measured on the surrogate endpoint while taking into account the uncertainty around the effect estimate for the surrogate endpoint. In this paper, extensions to multivariate models are developed aiming to include multiple surrogate endpoints with the potential benefit of reducing the uncertainty when making predictions. In this Bayesian multivariate meta‐analytic framework, the between‐study variability is modelled in a formulation of a product of normal univariate distributions. This formulation is particularly convenient for including multiple surrogate endpoints and flexible for modelling the outcomes which can be surrogate endpoints to the final outcome and potentially to one another. Two models are proposed, first, using an unstructured between‐study covariance matrix by assuming the treatment effects on all outcomes are correlated and second, using a structured between‐study covariance matrix by assuming treatment effects on some of the outcomes are conditionally independent. While the two models are developed for the summary data on a study level, the individual‐level association is taken into account by the use of the Prentices criteria (obtained from individual patient data) to inform the within study correlations in the models. The modelling techniques are investigated using an example in relapsing remitting multiple sclerosis where the disability worsening is the final outcome, while relapse rate and MRI lesions are potential surrogates to the disability progression.

The influence of admission characteristics on outcome: evidence from a medium secure forensic cohort.

OBJECTIVE Outcomes for any mental health service will vary with the characteristics of those admitted as well as with the clinical provision of the service itself. This study aims to explore, for a medium secure forensic service in England, temporal changes in (1) characteristics of those admitted and (2) outcome after discharge and (3) to examine whether such changes are related. METHOD Baseline characteristics and reconviction outcomes were derived from multiple data sources for 550 first admissions to a medium secure forensic unit for a 20-year period. Time to reconviction was examined using Kaplan-Meier analysis and Cox regression. RESULTS Over time, severity of admissions increased, as did discharges to prison; discharges to non-secure hospitals reduced. Risk of reconviction increased by 3.9%-4.2% for each year of admission from 1983, which was explained by the increased admission of higher-risk patients. CONCLUSION This medium secure service admitted patients with increasing levels of risk; reoffending rates reflect admission characteristics. Service funding decisions should take account of the characteristics of those admitted. SIGNIFICANT OUTCOMES This study indicates that the profile of patients admitted over a 20-year period increased in severity. Over time, reconviction after discharge occurred earlier after release. This increase in reconviction was explained by the type of patient admitted. LIMITATIONS Examination of a cohort from a single medium secure unit limits the generalizability of the findings. The study focuses on a criminological outcome measure (i.e. reconviction); other domains may be equally relevant (e.g. the relief of psychological distress). Examining an entire series of admissions introduces heterogeneity by, for example, considering the outcome of men and women together.