Koen Degeling

A systematic review and checklist presenting the main challenges for health economic modeling in personalized medicine: towards implementing patient-level models.

ABSTRACT Introduction: The ongoing development of genomic medicine and the use of molecular and imaging markers in personalized medicine (PM) has arguably challenged the field of health economic modeling (HEM). This study aims to provide detailed insights into the current status of HEM in PM, in order to identify if and how modeling methods are used to address the challenges described in literature. Areas covered: A review was performed on studies that simulate health economic outcomes for personalized clinical pathways. Decision tree modeling and Markov modeling were the most observed methods. Not all identified challenges were frequently found, challenges regarding companion diagnostics, diagnostic performance, and evidence gaps were most often found. However, the extent to which challenges were addressed varied considerably between studies. Expert commentary: Challenges for HEM in PM are not yet routinely addressed which may indicate that either (1) their impact is less severe than expected, (2) they are hard to address and therefore not managed appropriately, or (3) HEM in PM is still in an early stage. As evidence on the impact of these challenges is still lacking, we believe that more concrete examples are needed to illustrate the identified challenges and to demonstrate methods to handle them.

Comparison of Timed Automata with Discrete Event Simulation for Modeling of Biomarker-Based Treatment Decisions: An Illustration for Metastatic Castration-Resistant Prostate Cancer

BACKGROUND With the advent of personalized medicine, the field of health economic modeling is being challenged and the use of patient-level dynamic modeling techniques might be required. OBJECTIVES To illustrate the usability of two such techniques, timed automata (TA) and discrete event simulation (DES), for modeling personalized treatment decisions. METHODS An early health technology assessment on the use of circulating tumor cells, compared with prostate-specific antigen and bone scintigraphy, to inform treatment decisions in metastatic castration-resistant prostate cancer was performed. Both modeling techniques were assessed quantitatively, in terms of intermediate outcomes (e.g., overtreatment) and health economic outcomes (e.g., net monetary benefit). Qualitatively, among others, model structure, agent interactions, data management (i.e., importing and exporting data), and model transparency were assessed. RESULTS Both models yielded realistic and similar intermediate and health economic outcomes. Overtreatment was reduced by 6.99 and 7.02 weeks by applying circulating tumor cell as a response marker at a net monetary benefit of -€1033 and -€1104 for the TA model and the DES model, respectively. Software-specific differences were observed regarding data management features and the support for statistical distributions, which were considered better for the DES software. Regarding method-specific differences, interactions were modeled more straightforward using TA, benefiting from its compositional model structure. CONCLUSIONS Both techniques prove suitable for modeling personalized treatment decisions, although DES would be preferred given the current software-specific limitations of TA. When these limitations are resolved, TA would be an interesting modeling alternative if interactions are key or its compositional structure is useful to manage multi-agent complex problems.

Matching the model with the evidence: comparing discrete event simulation and state-transition modeling for time-to-event predictions in a cost-effectiveness analysis of treatment in metastatic colorectal cancer patients

BACKGROUND Individual patient data, e.g. from clinical trials, often need to be extrapolated or combined with additional evidence when assessing long-term impact in cost-effectiveness modeling studies. Different modeling methods can be used to represent the complex dynamics of clinical practice; the choice of which may impact cost-effectiveness outcomes. We compare the use of a previously designed cohort discrete-time state-transition model (DT-STM) with a discrete event simulation (DES) model. METHODS The original DT-STM was replicated and a DES model developed using AnyLogic software. Models were populated using individual patient data of a phase III study in metastatic colorectal cancer patients, and compared based on their evidence structure, internal validity, and cost-effectiveness outcomes. The DT-STM used time-dependent transition probabilities, whereas the DES model was populated using parametric distributions. RESULTS The estimated time-dependent transition probabilities for the DT-STM were irregular and more sensitive to single events due to the required small cycle length and limited number of event observations, whereas parametric distributions resulted in smooth time-to-event curves for the DES model. Although the DT-STM and DES model both yielded similar time-to-event curves, the DES model represented the trial data more accurately in terms of mean health-state durations. The incremental cost-effectiveness ratio (ICER) was €172,443 and €168,383 per Quality Adjusted Life Year gained for the DT-STM and DES model, respectively. CONCLUSION DES represents time-to-event data from clinical trials more naturally and accurately than DT-STM when few events are observed per time cycle. As a consequence, DES is expected to yield a more accurate ICER.

Accounting for parameter uncertainty in the definition of parametric distributions used to describe individual patient variation in health economic models

BackgroundParametric distributions based on individual patient data can be used to represent both stochastic and parameter uncertainty. Although general guidance is available on how parameter uncertainty should be accounted for in probabilistic sensitivity analysis, there is no comprehensive guidance on reflecting parameter uncertainty in the (correlated) parameters of distributions used to represent stochastic uncertainty in patient-level models. This study aims to provide this guidance by proposing appropriate methods and illustrating the impact of this uncertainty on modeling outcomes.MethodsTwo approaches, 1) using non-parametric bootstrapping and 2) using multivariate Normal distributions, were applied in a simulation and case study. The approaches were compared based on point-estimates and distributions of time-to-event and health economic outcomes. To assess sample size impact on the uncertainty in these outcomes, sample size was varied in the simulation study and subgroup analyses were performed for the case-study.ResultsAccounting for parameter uncertainty in distributions that reflect stochastic uncertainty substantially increased the uncertainty surrounding health economic outcomes, illustrated by larger confidence ellipses surrounding the cost-effectiveness point-estimates and different cost-effectiveness acceptability curves. Although both approaches performed similar for larger sample sizes (i.e. n = 500), the second approach was more sensitive to extreme values for small sample sizes (i.e. n = 25), yielding infeasible modeling outcomes.ConclusionsModelers should be aware that parameter uncertainty in distributions used to describe stochastic uncertainty needs to be reflected in probabilistic sensitivity analysis, as it could substantially impact the total amount of uncertainty surrounding health economic outcomes. If feasible, the bootstrap approach is recommended to account for this uncertainty.

PRM113 - Timed Automata Modeling of The Personalized Treatment Decisions In Metastatic Castration Resistant Prostate Cancer

Objectives: The Timed Automata modeling paradigm has emerged from Computer Science as a mature tool for the functional analysis and performance evaluation of timed distributed systems. This study is a first exploration of the suitability of Timed Automata for health economic modeling, using a case study on personalized treatment for metastatic Castration Resistant Prostate Cancer (mCRPC). Methods: The treatment process has been modeled by creating several independent timed automata, where an automaton represents a patient, a physician, a test, or a treatment/testing guideline schedule. These automata interact via message passing and are fully parameterized with quantitative information. Messages can be passed, asynchronously, from one automaton to one or more other automata, at any point in time, thereby triggering events and decisions in the treatment process. In the automata time is continuous, and both QALYs and costs can be incorporated using (assignable) local clocks. Uncertainty can be modeled using probabilities and timing intervals that can be uniformly or exponentially distributed. Software for building timed automata is freely available for academic use and includes procedures for statistical model checking (SMC) to validate the (internal) behavior and results of the model. Results: In several days a Timed Automata model has been produced that is compositional, easy to understand and easy to update. The behavior and results of the model have been assessed using the SMC tool. Actual results for the mCRPC case study obtained from the Timed Automata model are compared with results of a Discrete Event Simulation model in a separate study. Conclusions: The Timed Automata paradigm can be successfully applied to evaluate the potential benefits of a personalized treatment process of mCRPC. The compositional nature of the resulting model provides a good separation of all relevant components. This leads to models that are easy to formulate, validate, understand, maintain and update.

Comparison of Timed Automata with Discrete Event Simulation for Modeling Personalized Treatment Decisions: the Case of Metastatic Castration Resistant Prostate Cancer.

Objectives The aim of this study is to compare the usefulness of two promising alternative modeling techniques, Timed Automata (TA) originating from informatics, and Discrete Event Simulation (DES) known in operations research, for modeling todays complex and personalized treatment decisions over time, involving multiple interactions and decision gates. Methods The usefulness of both modeling techniques was assessed in a case study on the treatment of metastatic Castration Resistant Prostate Cancer (mCRPC) in which Circulating Tumor Cells (CTC) may be used as a response marker for switching first to second line treatment. Techniques were compared on user-friendliness, input requirements, input possibilities, model checking facilities, and results. Input parameters were similar for both models, consisting of costs, QoL, treatment effectiveness, diagnostic performance, physicians’ behavior and survival. Primary outcome measures were health outcomes, expressed in QALYs, and costs. Results Modelling was considered easier using TA, as this approach allows independent modeling of the actors and elements comprising the treatment process, such as patients, physicians, tests and treatments, and their mutual interaction and communication. Furthermore, the statistical model checking feature in the TA software was found to be a powerful tool for validation. Input requirements and possibilities were similar for both modelling approaches in this case study. Both modelling approaches yield comparable results. Using TA, CTC reduced first and second line treatment by, on average, 108.9 and 107.6 days, respectively. Using DES, treatment was reduced by 83.6 and 85.0 days. CTC therefore reduced healthcare costs by €28,998 and €21,992 according to TA and DES, respectively. Conclusions Both Timed Automata and Discrete Event Simulation seem to be suitable for modeling complex and personalized treatment processes like that of mCRPC. Timed Automata is a new and interesting alternative modeling technique, as it allows explicit separation of model components and supports statistical model checking to validate models.