Michael C. Wolfson

Validation of population-based disease simulation models: a review of concepts and methods

BackgroundComputer simulation models are used increasingly to support public health research and policy, but questions about their quality persist. The purpose of this article is to review the principles and methods for validation of population-based disease simulation models.MethodsWe developed a comprehensive framework for validating population-based chronic disease simulation models and used this framework in a review of published model validation guidelines. Based on the review, we formulated a set of recommendations for gathering evidence of model credibility.ResultsEvidence of model credibility derives from examining: 1) the process of model development, 2) the performance of a model, and 3) the quality of decisions based on the model. Many important issues in model validation are insufficiently addressed by current guidelines. These issues include a detailed evaluation of different data sources, graphical representation of models, computer programming, model calibration, between-model comparisons, sensitivity analysis, and predictive validity. The role of external data in model validation depends on the purpose of the model (e.g., decision analysis versus prediction). More research is needed on the methods of comparing the quality of decisions based on different models.ConclusionAs the role of simulation modeling in population health is increasing and models are becoming more complex, there is a need for further improvements in model validation methodology and common standards for evaluating model credibility.

Cost-effectiveness of Lung Cancer Screening in Canada

IMPORTANCE The US National Lung Screening Trial supports screening for lung cancer among smokers using low-dose computed tomographic (LDCT) scans. The cost-effectiveness of screening in a publically funded health care system remains a concern. OBJECTIVE To assess the cost-effectiveness of LDCT scan screening for lung cancer within the Canadian health care system. DESIGN, SETTING, AND PARTICIPANTS The Cancer Risk Management Model (CRMM) simulated individual lives within the Canadian population from 2014 to 2034, incorporating cancer risk, disease management, outcome, and cost data. Smokers and former smokers eligible for lung cancer screening (30 pack-year smoking history, ages 55-74 years, for the reference scenario) were modeled, and performance parameters were calibrated to the National Lung Screening Trial (NLST). The reference screening scenario assumes annual scans to age 75 years, 60% participation by 10 years, 70% adherence to screening, and unchanged smoking rates. The CRMM outputs are aggregated, and costs (2008 Canadian dollars) and life-years are discounted 3% annually. MAIN OUTCOMES AND MEASURES The incremental cost-effectiveness ratio. RESULTS Compared with no screening, the reference scenario saved 51,000 quality-adjusted life-years (QALY) and had an incremental cost-effectiveness ratio of CaD

Canadian cancer risk management model: Evaluation of cancer control

52,000/QALY. If smoking history is modeled for 20 or 40 pack-years, incremental cost-effectiveness ratios of CaD

Computed Tomography Screening for Lung Cancer without a Smoking Cessation Program—Not a Cost-Effective Idea

62,000 and CaD

Using the Cancer Risk Management Model to evaluate colorectal cancer screening options for Canada

43,000/QALY, respectively, were generated. Changes in participation rates altered life years saved but not the incremental cost-effectiveness ratio, while the incremental cost-effectiveness ratio is sensitive to changes in adherence. An adjunct smoking cessation program improving the quit rate by 22.5% improves the incremental cost-effectiveness ratio to CaD

XECON: An Experimental / Evolutionary Model of Economic Growth

24,000/QALY. CONCLUSIONS AND RELEVANCE Lung cancer screening with LDCT appears cost-effective in the publicly funded Canadian health care system. An adjunct smoking cessation program has the potential to improve outcomes.

The Population Health Model (POHEM): an overview of rationale, methods and applications

OBJECTIVES The aim of this study was to develop a decision support tool to assess the potential benefits and costs of new healthcare interventions. METHODS The Canadian Partnership Against Cancer (CPAC) commissioned the development of a Cancer Risk Management Model (CRMM)--a computer microsimulation model that simulates individual lives one at a time, from birth to death, taking account of Canadian demographic and labor force characteristics, risk factor exposures, and health histories. Information from all the simulated lives is combined to produce aggregate measures of health outcomes for the population or for particular subpopulations. RESULTS The CRMM can project the population health and economic impacts of cancer control programs in Canada and the impacts of major risk factors, cancer prevention, and screening programs and new cancer treatments on population health and costs to the healthcare system. It estimates both the direct costs of medical care, as well as lost earnings and impacts on tax revenues. The lung and colorectal modules are available through the CPAC Web site (www.cancerview.ca/cancerrriskmanagement) to registered users where structured scenarios can be explored for their projected impacts. Advanced users will be able to specify new scenarios or change existing modules by varying input parameters or by accessing open source code. Model development is now being extended to cervical and breast cancers.

Effects of Reductions in Body Mass Index on the Future Osteoarthritis Burden in Canada: A Population‐Based Microsimulation Study

It is hard not to be excited by the recently reported results of the National Lung Screening Trial (NLST). 1 Although advances have been made in the surgical, radiotherapeutic, and chemotherapeutic management of lung cancer over the past several decades, the long-term survival rate for lung cancer remains very low. 2 The 20% reduction in mortality observed in this large randomized multicenter lung cancer screening study with low-dose computed tomography (CT) (20% of the usual dose) compared with chest x-ray in high-risk individuals (30 or more pack-years smokers or former smokers who had quit smoking within 15 years of study entry and currently 55–74 years of age) is definitely something to celebrate. But is population-based lung cancer screening really ready for prime time? Despite the positive NLST result, we do not yet know how optimally to define the “at-risk” population, when to start screening, what screening interval to use, 3 and for how long. Moreover, there is the issue of the cost of a population-based screening program and its cost-effectiveness compared with other lung cancer control strategies. In an increasingly cost-constrained world, this issue has to be confronted. When breast cancer screening was first introduced, over two decades ago, there were no calls for cost-effectiveness analyses before its implementation on a population basis, but times are different now. National economies cannot continue to absorb the cost of new health technologies without a thoughtful examination of the value for money proposition. This was recognized by the International Association for the Study of Lung Cancer CT Screening Task Force which recommended that the cost-effectiveness of CT screening needed to be examined before it is implemented at the national level. 4 The article by McMahon et al. 5 addresses the issue of cost-effectiveness using an existing Lung Cancer Policy Model which simulates lung cancer development, disease progression, detection, treatment, and survival. The lung cancer natural history parameters in the model have been calibrated against US tumor registry data on age-specific cancer incidence rates; on the distribution by size, stage, and cell types of incident lung cancers; and by lung cancer-specific survival. The model simulates symptomatic, incidental, and screen-detected benign and malignant pulmonary nodules. The sensitivity of screening CT examinations was adjusted for nodule size and location of the nodule in the chest. In the base case, nodules less than 4 mm were not followed but those between 4 and 6 mm had serial high-resolution CT scans at 9 and 24 months and those 6 to 8 mm were scheduled for CT at 6, 12, and 24 months. In sensitivity analyses, fewer high-resolution CT examinations were simulated. Nodules greater than 8 mm were biopsied. Survival was modeled as a function of treatment and underlying disease characteristics. Treatment followed National Comprehensive Cancer Network consensus guidelines in place in 2000. The model simulated the life-time histories of individuals in each of six cohorts (500,000 histories per cohort) defined by age and gender (males and females; age 50, 60, or 70

SimPHO: an ontology for simulation modeling of population health

BACKGROUND Several screening methods for colorectal cancer (crc) are available, and some have been shown by randomized trials to be effective. In the present study, we used a well-developed population health simulation model to compare the risks and benefits of a variety of screening scenarios. Tests considered were the fecal occult blood test (fobt), the fecal immunochemical test (fit), flexible sigmoidoscopy, and colonoscopy. Outcomes considered included years of life gained, crc cases and deaths prevented, and direct health system costs. METHODS A natural history model of crc was implemented and calibrated to specified targets within the framework of the Cancer Risk Management Model (crmm) from the Canadian Partnership Against Cancer. The crmm-crc permits users to enter their own parameter values or to use program-specified base values. For each of 23 screening scenarios, we used the crmm-crc to run 10 million replicate simulations. RESULTS Using base parameter values and some user-specified values in the crmm-crc, and comparing our screening scenarios with no screening, all screening scenarios were found to reduce the incidence of and mortality from crc. The fobt was the least effective test; it was not associated with lower net cost. Colonoscopy screening was the most effective test; it had net costs comparable to those for several other strategies considered, but required more than 3 times the colonoscopy resources needed by other approaches. After colonoscopy, strategies based on the fit were predicted to be the most effective. In sensitivity analyses performed for the fobt and fit screening strategies, fobt parameter values associated with high-sensitivity formulations were associated with a substantial increase in test effectiveness. The fit was more cost-effective at the 50 ng/mL threshold than at the 100 ng/mL threshold. CONCLUSIONS The crmm-crc provides a sophisticated and flexible environment in which to evaluate crc control options. All screening scenarios considered in this study effectively reduced crc mortality, although sensitivity analyses demonstrated some uncertainty in the magnitude of the improvements. Where possible, local data should be used to reduce uncertainty in the parameters.

Biennial lung cancer screening in Canada with smoking cessation—outcomes and cost-effectiveness

The role of technical innovation in economic growth is both a current matter of keen public policy interest, and active exploration in economic theory. However, formal economic theorizing is often constrained by considerations of mathematical tractability. Evolutionary economic theories which are realized as computerized microsimulation models offer significant promise both for transcending mathematical constraints and addressing fundamental questions in a more realistic and flexible manner. This paper sketches XEcon, a microsimulation model of economic growth in the evolutionary tradition.