TECH-VER verification checklist for economic models

Abstract

The newly developed TECHnical VERification (TECHVER) checklist for validating health economic decisionanalytical models can help identify causes of model implementation errors and be used as a training and quality control tool for the development of models, say authors of a report published in PharmacoEconomics. EMBASE and MEDLINE databases were searched up to May 2019 for studies on the credibility, validation and verification of models for health technology assessment (HTA) or cost-effectiveness analysis. A draft checklist was developed based on findings from studies identified in the literature review, and applied to health economic models which varied with respect to stakeholder developer, purpose and the maturity of clinical evidence. Iterative revision of the checklist was performed and checked after implementation. The checklist then underwent further revision after discussions with other health economists, until the final version of TECH-VER was created. The TECH-VER requires a model reviewer, a transparent model, input sources, and detailed documentation reporting the concept, implementation, model inputs and results. It includes five domains: input calculations, event-state calculations, result calculations, uncertainty analysis calculations, and overall checks. The reviewer should assess the justifications of methods used in calculations. Verification tests conducted to check the correctness of implementation of these calculations should include (in consecutive order): black-box tests to check that model calculations align with a priori expectations; white-box testing of program code line by line, or cell by cell for crucial calculations if black-box test results are unexpected; and model replication/parallel programming if issues related to unexpected results from black-box tests cannot be resolved through whitebox testing. The TECH-VER checklist is a comprehensive checklist for the technical verification of decision analytical models, aiming to help identify model implementation errors and their root causes while improving the transparency and efficiency of the verification efforts, said the authors. It is the authors’ aim that the TECHVER checklist transforms itself to an open-source living document, with possible future versions, or ’bolt-on’ extensions for specific applications with additional ’fitfor-purpose’ tests, as well as ’tips and tricks’ and some demonstrative error examples, they commented.