Tommi Tervonen

Implementing stochastic multicriteria acceptability analysis

Abstract Stochastic multicriteria acceptability analysis (SMAA) is a family of methods for aiding multicriteria group decision making in problems with inaccurate, uncertain, or missing information. These methods are based on exploring the weight space in order to describe the preferences that make each alternative the most preferred one, or that would give a certain rank for a specific alternative. The main results of the analysis are rank acceptability indices, central weight vectors and confidence factors for different alternatives. The rank acceptability indices describe the variety of different preferences resulting in a certain rank for an alternative, the central weight vectors represent the typical preferences favouring each alternative, and the confidence factors measure whether the criteria measurements are sufficiently accurate for making an informed decision. The computations in SMAA require the evaluation of multidimensional integrals that must in practice be computed numerically. In this paper we present efficient methods for performing the computations through Monte Carlo simulation, analyze the complexity, and assess the accuracy of the presented algorithms. We also test the efficiency of these methods empirically. Based on the tests, the implementation is fast enough to analyze typical-sized discrete problems interactively within seconds. Due to almost linear time complexity, the method is also suitable for analysing very large decision problems, for example, discrete approximations of continuous decision problems.

ADDIS: A decision support system for evidence-based medicine

Clinical trials are the main source of information for the efficacy and safety evaluation of medical treatments. Although they are of pivotal importance in evidence-based medicine, there is a lack of usable information systems providing data-analysis and decision support capabilities for aggregate clinical trial results. This is partly caused by unavailability (i) of trial data in a structured format suitable for re-analysis, and (ii) of a complete data model for aggregate level results. In this paper, we develop a unifying data model that enables the development of evidence-based decision support in the absence of a complete data model. We describe the supported decision processes and show how these are implemented in the open source ADDIS software. ADDIS enables semi-automated construction of meta-analyses, network meta-analyses and benefit-risk decision models, and provides visualization of all results.

Hit-And-Run enables efficient weight generation for simulation-based multiple criteria decision analysis

In our previous work published in this journal, we showed how the Hit-And-Run (HAR) procedure enables efficient sampling of criteria weights from a space formed by restricting a simplex with arbitrary linear inequality constraints. In this short communication, we note that the method for generating a basis of the sampling space can be generalized to also handle arbitrary linear equality constraints. This enables the application of HAR to sampling spaces that do not coincide with the simplex, thereby allowing the combined use of imprecise and precise preference statements. In addition, it has come to our attention that one of the methods we proposed for generating a starting point for the Markov chain was flawed. To correct this, we provide an alternative method that is guaranteed to produce a starting point that lies within the interior of the sampling space.

Robust multi-criteria ranking with additive value models and holistic pair-wise preference statements

We consider a problem of ranking alternatives based on their deterministic performance evaluations on multiple criteria. We apply additive value theory and assume the Decision Maker’s (DM) preferences to be representable with general additive monotone value functions. The DM provides indirect preference information in form of pair-wise comparisons of reference alternatives, and we use this to derive the set of compatible value functions. Then, this set is analyzed to describe (1) the possible and necessary preference relations, (2) probabilities of the possible relations, (3) ranges of ranks the alternatives may obtain, and (4) the distributions of these ranks. Our work combines previous results from Robust Ordinal Regression, Extreme Ranking Analysis and Stochastic Multicriteria Acceptability Analysis under a unified decision support framework. We show how the four different results complement each other, discuss extensions of the main proposal, and demonstrate practical use of the approach by considering a problem of ranking 20 European countries in terms of 4 criteria reflecting the quality of their universities.

A stochastic multicriteria model for evidence‐based decision making in drug benefit‐risk analysis

Drug benefit-risk (BR) analysis is based on firm clinical evidence regarding various safety and efficacy outcomes. In this paper, we propose a new and more formal approach for constructing a supporting multi-criteria model that fully takes into account the evidence on efficacy and adverse drug reactions. Our approach is based on the stochastic multi-criteria acceptability analysis methodology, which allows us to compute the typical value judgments that support a decision, to quantify decision uncertainty, and to compute a comprehensive BR profile. We construct a multi-criteria model for the therapeutic group of second-generation antidepressants. We assess fluoxetine and venlafaxine together with placebo according to incidence of treatment response and three common adverse drug reactions by using data from a published study. Our model shows that there are clear trade-offs among the treatment alternatives.

Stochastic ordinal regression for multiple criteria sorting problems

We present a new approach for multiple criteria sorting problems. We consider sorting procedures applying general additive value functions compatible with the given assignment examples. For the decision alternatives, we provide four types of results: (1) necessary and possible assignments from Robust Ordinal Regression (ROR), (2) class acceptability indices from a suitably adapted Stochastic Multicriteria Acceptability Analysis (SMAA) model, (3) necessary and possible assignment-based preference relations, and (4) assignment-based pair-wise outranking indices. We show how the results provided by ROR and SMAA complement each other and combine them under a unified decision aiding framework. Application of the approach is demonstrated by classifying 27 countries in 4 democracy regimes.

Multicriteria benefit-risk assessment using network meta-analysis

OBJECTIVE To enable multicriteria benefit-risk (BR) assessment of any number of alternative treatments using all available evidence from a network of clinical trials. STUDY DESIGN AND SETTING We design a general method for multicriteria decision aiding with criteria measurements from Mixed Treatment Comparison (MTC) analyses. To evaluate the method, we apply it to BR assessment of four second-generation antidepressants and placebo in the setting of a published peer-reviewed systematic review. RESULTS The analysis without preference information shows that placebo is supported by a wide range of possible preferences. Preference information provided by a clinical expert showed that although treatment with antidepressants is warranted for severely depressed patients, for mildly depressed patients placebo is likely to be the best option. It is difficult to choose between the four antidepressants, and the results of the model indicate a high degree of uncertainty. CONCLUSIONS The designed method enables quantitative BR analysis of alternative treatments using all available evidence from a network of clinical trials. The preference-free analysis can be useful in presenting the results of an MTC considering multiple outcomes.

JSMAA: open source software for SMAA computations

Most software for multi-criteria decision analysis (MCDA) implement a small set of compatible methods as a closed monolithic program. With such software tools, the decision models have to be input by hand. In some applications, however, the model can be generated using external information sources, and thus it would be beneficial if the MCDA software could integrate in the comprehensive information infrastructure. This article motivates for the need of model generation in the methodological context of stochastic multicriteria acceptability analysis (SMAA), and describes the JSMAA software that implements SMAA-2, SMAA-O and SMAA-TRI methods. JSMAA is an open source and divided in separate graphical user interface and library components, enabling its use in systems with a model generation subsystem.

Algorithmic parameterization of mixed treatment comparisons

Mixed Treatment Comparisons (MTCs) enable the simultaneous meta-analysis (data pooling) of networks of clinical trials comparing ≥2 alternative treatments. Inconsistency models are critical in MTC to assess the overall consistency between evidence sources. Only in the absence of considerable inconsistency can the results of an MTC (consistency) model be trusted. However, inconsistency model specification is non-trivial when multi-arm trials are present in the evidence structure. In this paper, we define the parameterization problem for inconsistency models in mathematical terms and provide an algorithm for the generation of inconsistency models. We evaluate running-time of the algorithm by generating models for 15 published evidence structures.

Applying Multiple Criteria Decision Analysis to Comparative Benefit-Risk Assessment Choosing among Statins in Primary Prevention

Decision makers in different health care settings need to weigh the benefits and harms of alternative treatment strategies. Such health care decisions include marketing authorization by regulatory agencies, practice guideline formulation by clinical groups, and treatment selection by prescribers and patients in clinical practice. Multiple criteria decision analysis (MCDA) is a family of formal methods that help make explicit the tradeoffs that decision makers accept between the benefit and risk outcomes of different treatment options. Despite the recent interest in MCDA, certain methodological aspects are poorly understood. This paper presents 7 guidelines for applying MCDA in benefit-risk assessment and illustrates their use in the selection of a statin drug for the primary prevention of cardiovascular disease. We provide guidance on the key methodological issues of how to define the decision problem, how to select a set of nonoverlapping evaluation criteria, how to synthesize and summarize the evidence, how to translate relative measures to absolute ones that permit comparisons between the criteria, how to define suitable scale ranges, how to elicit partial preference information from the decision makers, and how to incorporate uncertainty in the analysis. Our example on statins indicates that fluvastatin is likely to be the most preferred drug by our decision maker and that this result is insensitive to the amount of preference information incorporated in the analysis.