I. Corro Ramos

AdViSHE: A Validation-Assessment Tool of Health-Economic Models for Decision Makers and Model Users.

BackgroundA trade-off exists between building confidence in health-economic (HE) decision models and the use of scarce resources. We aimed to create a practical tool providing model users with a structured view into the validation status of HE decision models, to address this trade-off.MethodsA Delphi panel was organized, and was completed by a workshop during an international conference. The proposed tool was constructed iteratively based on comments from, and the discussion amongst, panellists. During the Delphi process, comments were solicited on the importance and feasibility of possible validation techniques for modellers, their relevance for decision makers, and the overall structure and formulation in the tool.ResultsThe panel consisted of 47 experts in HE modelling and HE decision making from various professional and international backgrounds. In addition, 50 discussants actively engaged in the discussion at the conference workshop and returned 19 questionnaires with additional comments. The final version consists of 13 items covering all relevant aspects of HE decision models: the conceptual model, the input data, the implemented software program, and the model outcomes.ConclusionsAssessment of the Validation Status of Health-Economic decision models (AdViSHE) is a validation-assessment tool in which model developers report in a systematic way both on validation efforts performed and on their outcomes. Subsequently, model users can establish whether confidence in the model is justified or whether additional validation efforts should be undertaken. In this way, AdViSHE enhances transparency of the validation status of HE models and supports efficient model validation.

SeHCAT [tauroselcholic (selenium-75) acid] for the investigation of bile acid malabsorption and measurement of bile acid pool loss: a systematic review and cost-effectiveness analysis.

BACKGROUND The principal diagnosis/indication for this assessment is chronic diarrhoea due to bile acid malabsorption (BAM). Diarrhoea can be defined as the abnormal passage of loose or liquid stools more than three times daily and/or a daily stool weight > 200 g per day and is considered to be chronic if it persists for more than 4 weeks. The cause of chronic diarrhoea in adults is often difficult to ascertain and patients may undergo several investigations without a definitive cause being identified. BAM is one of several causes of chronic diarrhoea and results from failure to absorb bile acids (which are required for the absorption of dietary fats and sterols in the intestine) in the distal ileum. OBJECTIVE For people with chronic diarrhoea with unknown cause and in people with Crohns disease and chronic diarrhoea with unknown cause (i.e. before resection): (1) What are the effects of selenium-75-homocholic acid taurine (SeHCAT) compared with no SeHCAT in terms of chronic diarrhoea, other health outcomes and costs? (2) What are the effects of bile acid sequestrants (BASs) compared with no BASs in people with a positive or negative SeHCAT test? (3) Does a positive or negative SeHCAT test predict improvement in terms of chronic diarrhoea, other health outcomes and costs? DATA SOURCES A systematic review was conducted to summarise the evidence on the clinical effectiveness of SeHCAT for the assessment of BAM and the measurement of bile acid pool loss. Search strategies were based on target condition and intervention, as recommended in the Centre for Reviews and Dissemination (CRD) guidance for undertaking reviews in health care and the Cochrane Handbook for Diagnostic Test Accuracy Reviews. The following databases were searched up to April 2012: MEDLINE; MEDLINE In-Process & Other Non-Indexed Citations; EMBASE; the Cochrane Databases; Database of Abstracts of Reviews of Effects; Health Technology Assessment (HTA) Database; and Science Citation Index. Research registers and conference proceedings were also searched. REVIEW METHODS Systematic review methods followed the principles outlined in the CRD guidance for undertaking reviews in health care and the National Institute for Health and Care Excellence (NICE) Diagnostic Assessment Programme interim methods statement. In the health economic analysis, the cost-effectiveness of SeHCAT for the assessment of BAM, in patients with chronic diarrhoea, was estimated in two different populations. The first is the population of patients with chronic diarrhoea with unknown cause and symptoms suggestive of diarrhoea-predominant irritable bowel syndrome (IBS-D) and the second population concerns patients with Crohns disease without ileal resection with chronic diarrhoea. For each population, three models were combined: (1) a short-term decision tree that models the diagnostic pathway and initial response to treatment (first 6 months); (2) a long-term Markov model that estimates the lifetime costs and effects for patients initially receiving BAS; and (3) a long-term Markov model that estimates the lifetime costs and effects for patients initially receiving regular treatment (IBS-D treatment in the first population and Crohns treatment in the second population). Incremental cost-effectiveness ratios were estimated as additional cost per additional responder in the short term (first 6 months) and per additional quality-adjusted life-year (QALY) in the long term (lifetime). RESULTS We found three studies assessing the relationship between the SeHCAT test and response to treatment with cholestyramine. However, the studies had small numbers of patients with unknown cause chronic diarrhoea, and they used different cut-offs to define BAM. For the short term (first 6 months), when trial of treatment is not considered as a comparator, the optimal choice depends on the willingness to pay for an additional responder. For lower values (between £1500 and £4600) the choice will be no SeHCAT in all scenarios; for higher values either SeHCAT 10% or SeHCAT 15% becomes cost-effective. For the lifetime perspective, the various scenarios showed widely differing results: in the threshold range of £20,000-30,000 per QALY gained we found as optimal choice either no SeHCAT, SeHCAT 5% (only IBS-D) or SeHCAT 15%. When trial of treatment is considered a comparator, the analysis showed that for the short term, trial of treatment is the optimal choice across a range of scenarios. For the lifetime perspective with trial of treatment, again the various scenarios show widely differing results. Depending on the scenario, in the threshold range of £20,000-30,000 per QALY gained, we found as optimal choice either trial of treatment, no SeHCAT or SeHCAT 15%. CONCLUSIONS In conclusion, the various analyses show that for both populations considerable decision uncertainty exists and that no firm conclusions can be formulated about which strategy is optimal. Standardisation of the definition of a positive SeHCAT test should be the first step in assessing the usefulness of this test. As there is no reference standard for the diagnosis of BAM and SeHCAT testing provides a continuous measure of metabolic function, diagnostic test accuracy (DTA) studies are not the most appropriate study design. However, in studies where all patients are tested with SeHCAT and all patients are treated with BASs, response to treatment can provide a surrogate reference standard; further DTA studies of this type may provide information on the ability of SeHCAT to predict response to BASs. A potentially more informative option would be multivariate regression modelling of treatment response (dependent variable), with SeHCAT result and other candidate clinical predictors as covariates. Such a study design could also inform the definition of a positive SeHCAT result. STUDY REGISTRATION The study is registered as PROSPERO CRD42012001911. FUNDING The National Institute for Health Research Health Technology Assessment programme.

ADVISHE: A new tool to report validation of health-economic decision models

Background: Modelers and reimbursement decision makers could both profit from a more systematic reporting of the efforts to validate health-economic (HE) models. Objectives: Development of a tool to systematically report validation efforts of HE decision models and their outcomes. Methods: A gross list of model validation techniques was collected using a literature review, including sources outside the HE field. A panel then selected the most important items. Based on the Delphi method, the panel members could score items in three e-mail rounds. Participants were HE modelling experts, covering various nationalities and work environments. They could comment on relevance, feasibility and formulation of the items and received feedback on comments from others. This resulted in a draft tool of selected items, which was tested and improved in two further rounds. In addition, the Dutch National Health Care Institute commented on usefulness for decision makers, while a separate group of 50 HE experts could comment during a workshop at ISPOR Montreal 2014. Results: 35 Validation techniques were identified and grouped into four categories: conceptual model validation, computerized model validation, data validation and operational validation. Around 30 HE experts commented in each of the first three Delphi rounds, resulting in a 15 item draft tool. The Dutch health care advisory institute suggested to add one more item. Participants from the ISPOR workshop delivered 19 filled-in questionnaires. A fourth round resulted in 17 responses. This led to a refined version containing 16 items, which is currently sent out for a final, fifth round. Conclusions: When filled out by the modellers, AdVISHE (Assessment of the ValIdation Status of Health- Economic decision models) supports model users in assessing the validation status of a model It will be useful as part of reimbursement dossiers, by providing systematic and transparent insight into the validation efforts performed and their results.

Operational validation of health economic decision analytic models

Objectives: To validate health economic (HE) models by means of statistical comparison of model outcomes against empirical observations. Such a comparison is structured and the applicability of several existing validation techniques is discussed, with a special focus on statistical testing. When standard methods (95%-confidence intervals) are used several problems, both of a technical and philosophical nature, are encountered. These problems are discussed. A new statistical approach is consequently proposed. Methods: The proposed method can be applied to validate HE models when the uncertainty around the input parameters of the model is assessed via probabilistic sensitivity analysis (PSA). It is based on the idea of establishing a level of accuracy in advance for the empirical observations and model outcomes should meet. If the model result falls within the limits determined by the pre-required accuracy, then the model result is considered valid. The number of valid results obtained in a PSA defines a measure of the reliability of the model. Embodying the method in a Bayesian framework allows defining such a reliability measure with statistical properties. Results: Existing approaches suffer from technical and interpretational problems. In addition, these methods are lacking a measure of overall reliability. Our new method (1) departs from classical statistical techniques, circumventing the noted problems, (2) can be used for both cohort and patient-level models and (3) makes use of all PSA outcomes. The method is demonstrated with the help of a case study in a published diabetes model (MICADO). Conclusions: Standard statistical techniques have to be applied very carefully on the comparison of model outcomes to empirical observations. They suffer from several problems. A new promising Bayesian approach is proposed that solves some of these issues. Our new method allows stepwise validation of the model as new data becomes available, which may increase the models validation status.