Oguzhan Alagoz

State-Transition Modeling A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force–3

State-transition modeling (STM) is an intuitive, flexible, and transparent approach of computer-based decision-analytic modeling, including both Markov model cohort simulation as well as individual-based (first-order Monte Carlo) microsimulation. Conceptualizing a decision problem in terms of a set of (health) states and transitions among these states, STM is one of the most widespread modeling techniques in clinical decision analysis, health technology assessment, and health-economic evaluation. STMs have been used in many different populations and diseases, and their applications range from personalized health care strategies to public health programs. Most frequently, state-transition models are used in the evaluation of risk factor interventions, screening, diagnostic procedures, treatment strategies, and disease management programs.

The Optimal Timing of Living-Donor Liver Transplantation

Living donors are a significant and increasing source of livers for transplantation, mainly because of the insufficient supply of cadaveric organs. We consider the problem of optimally timing a living-donor liver transplant to maximize the patients total reward, such as quality-adjusted life expectancy. We formulate a Markov decision process (MDP) model in which the state of the process is described by patient health. We derive structural properties of the MDP model, including a set of intuitive conditions that ensure the existence of a control-limit optimal policy. We use clinical data in our computational experiments, which show that the optimal policy is typically of control-limit type.

Modeling secrecy and deception in a multiple-period attacker-defender signaling game

In this paper, we apply game theory to model strategies of secrecy and deception in a multiple-period attacker-defender resource-allocation and signaling game with incomplete information. At each period, we allow one of the three possible types of defender signals--truthful disclosure, secrecy, and deception. We also allow two types of information updating--the attacker updates his knowledge about the defender type after observing the defenders signals, and also after observing the result of a contest (if one occurs in any given time period). Our multiple-period model provides insights into the balance between capital and expense for defensive investments (and the effects of defender private information, such as defense effectiveness, target valuations, and costs), and also shows that defenders can achieve more cost-effective security through secrecy and deception (possibly lasting more than one period), in a multiple-period game. This paper helps to fill a significant gap in the literature. In particular, to our knowledge, no past work has studied defender secrecy and deception in a multiple-period game. Moreover, we believe that the solution approach developed and applied in this paper would prove useful in other types of multiple-period games.

Tipping the balance of benefits and harms to favor screening mammography starting at age 40 years: a comparative modeling study of risk.

BACKGROUND Timing of initiation of screening for breast cancer is controversial in the United States. OBJECTIVE To determine the threshold relative risk (RR) at which the harm-benefit ratio of screening women aged 40 to 49 years equals that of biennial screening for women aged 50 to 74 years. DESIGN Comparative modeling study. DATA SOURCES Surveillance, Epidemiology, and End Results program, Breast Cancer Surveillance Consortium, and medical literature. TARGET POPULATION A contemporary cohort of women eligible for routine screening. TIME HORIZON Lifetime. PERSPECTIVE Societal. INTERVENTION Mammography screening starting at age 40 versus 50 years with different screening methods (film, digital) and screening intervals (annual, biennial). OUTCOME MEASURES BENEFITS life-years gained, breast cancer deaths averted; harms: false-positive mammography findings; harm-benefit ratios: false-positive findings/life-years gained, false-positive findings/deaths averted. RESULTS OF BASE-CASE ANALYSIS Screening average-risk women aged 50 to 74 years biennially yields the same false-positive findings/life-years gained as biennial screening with digital mammography starting at age 40 years for women with a 2-fold increased risk above average (median threshold RR, 1.9 [range across models, 1.5 to 4.4]). The threshold RRs are higher for annual screening with digital mammography (median, 4.3 [range, 3.3 to 10]) and when false-positive findings/deaths averted is used as an outcome measure instead of false-positive findings/life-years gained. The harm-benefit ratio for film mammography is more favorable than for digital mammography because film has a lower false-positive rate. RESULTS OF SENSITIVITY ANALYSIS The threshold RRs changed slightly when a more comprehensive measure of harm was used and were relatively insensitive to lower adherence assumptions. LIMITATION Risk was assumed to influence onset of disease without influencing screening performance. CONCLUSION Women aged 40 to 49 years with a 2-fold increased risk have similar harm-benefit ratios for biennial screening mammography as average-risk women aged 50 to 74 years. Threshold RRs required for favorable harm-benefit ratios vary by screening method, interval, and outcome measure. PRIMARY FUNDING SOURCE National Cancer Institute.

Markov Decision Processes: A Tool for Sequential Decision Making under Uncertainty

We provide a tutorial on the construction and evaluation of Markov decision processes (MDPs), which are powerful analytical tools used for sequential decision making under uncertainty that have been widely used in many industrial and manufacturing applications but are underutilized in medical decision making (MDM). We demonstrate the use of an MDP to solve a sequential clinical treatment problem under uncertainty. Markov decision processes generalize standard Markov models in that a decision process is embedded in the model and multiple decisions are made over time. Furthermore, they have significant advantages over standard decision analysis. We compare MDPs to standard Markov-based simulation models by solving the problem of the optimal timing of living-donor liver transplantation using both methods. Both models result in the same optimal transplantation policy and the same total life expectancies for the same patient and living donor. The computation time for solving the MDP model is significantly smaller than that for solving the Markov model. We briefly describe the growing literature of MDPs applied to medical decisions.

A Clinically Based Discrete-Event Simulation of End-Stage Liver Disease and the Organ Allocation Process

Background . The optimal allocation of scarce donor livers is a contentious health care issue requiring careful analysis. The objective of this article was to design a biologically based discrete-event simulation to test proposed changes in allocation policies. Methods . The authors used data from multiple sources to simulate end-stage liver disease and the complex allocation system. To validate the model, they compared simulation output with historical data. Results . Simulation outcomes were within 1% to 2% of actual results for measures such as new candidates, donated livers, and transplants by year. The model overestimated the yearly size of the waiting list by 5% in the last year of the simulation and the total number of pretransplant deaths by 10%. Conclusion . The authors created a discrete-event simulation model that represents the biology of end-stage liver disease and the health care organization of transplantation in the United States.

Determining the Acceptance of Cadaveric Livers Using an Implicit Model of the Waiting List

The only available therapy for patients with end-stage liver disease is organ transplantation. In the United States, patients with end-stage liver disease are placed on a waiting list and offered livers based on location and waiting time, as well as current and past health. Although there is a shortage of cadaveric livers, 45% of all cadaveric liver offers are declined by the first transplant surgeon and/or patient to whom they are offered. We consider the decision problem faced by these patients: Should an offered organ of a given quality be accepted or declined? We formulate a Markov decision process model in which the state of the process is described by patient state and organ quality. We use a detailed model of patient health to estimate the parameters of our decision model and implicitly consider the effects of the waiting list through our patient-state-dependent definition of the organ arrival probabilities. We derive structural properties of the model, including a set of intuitive conditions that ensure the existence of control-limit optimal policies. We use clinical data in our computational experiments, which confirm that the optimal policy is typically of control-limit type.

Collaborative Modeling of the Benefits and Harms Associated With Different U.S. Breast Cancer Screening Strategies.

Context Multiple alternative mammography screening strategies exist. Contribution This modeling study estimated outcomes of 8 strategies that differed by starting age and interval. Biennial screening from age 50 to 74 years avoided a median of 7 breast cancer deaths; in contrast, annual screening from age 40 to 74 years avoided an additional 3 deaths but yielded 1988 more false-positive results and 11 more overdiagnoses per 1000 women screened. Annual screening from age 40 years for high-risk women had similar outcomes as screening average-risk women biennially from 50 to 74 years of age. Caution Imaging technologies other than mammography and nonadherence were not modeled. Implication Biennial mammography screening for breast cancer is efficient for average-risk women. Despite decades of mammography screening for early detection of breast cancer, there is no consensus on optimal strategies, target populations, or the magnitude of harms and benefits (111). The 2009 US Preventive Services Task Force (USPSTF) recommended biennial film mammography from age 50 to 74 years and suggested shared decision making about screening for women in their 40s (12). Since that recommendation was formulated, new data on the benefits of screening have emerged (2, 6, 8, 9, 11, 13, 14), digital mammography has essentially replaced plain film (15), and increasingly effective systemic treatment regimens for breast cancer have become standard (16). There has also been growing interest in consumer preferences and personalized screening approaches (1720). These factors could each affect the outcomes of breast cancer screening programs or alter policy decisions about population screening strategies (17). Modeling can inform screening policy decisions because it uses the best available evidence to evaluate a wide range of strategies while holding selected conditions (such as treatment effects) constant, facilitating strategy comparisons (21, 22). Modeling also provides a quantitative summary of outcomes in different groups and assesses how preferences affect results. Collaboration of several models provides a range of plausible effects and illustrates the effects of differences in model assumptions on results (1, 7, 23). We used 6 well-established simulation models to synthesize current data and examine the outcomes of digital mammography screening at various starting ages and intervals among average-risk women. We also examined how breast density, risk, or comorbidity levels affect results and whether preferences for health states related to screening and its downstream consequences affected conclusions. Methods Strategies We evaluated 8 strategies that varied by starting age (40, 45, or 50 years) and interval (annual, biennial, and hybrid [annual for women in their 40s and biennial thereafter]); all strategies stop screening at age 74 years. We included no screening as a baseline. Model Descriptions The models used to evaluate the screening strategies were developed within the Cancer Intervention and Surveillance Modeling Network (CISNET) (2430), and the research was institutional review boardapproved. They were named model D (Dana-Farber Cancer Institute, Boston, Massachusetts), model E (Erasmus MC, University Medical Center Rotterdam, Rotterdam, the Netherlands), model GE (Georgetown University Medical Center, Washington, DC, and Albert Einstein College of Medicine, Bronx, New York), model M (MD Anderson Cancer Center, Houston, Texas), model S (Stanford University, Stanford, California), and model W (University of Wisconsin, Madison, Wisconsin, and Harvard Medical School, Boston, Massachusetts). The Appendix provides information on model validation. Since earlier analyses (1), the models have undergone substantial revision to reflect advances in breast cancer control, including portrayal of molecular subtypes based on estrogen receptor (ER) and human epidermal growth factor-2 receptor (HER2) status (23); current population incidence (31) and competing nonbreast cancer mortality; digital screening; and the most current therapies (32). All models except model S include ductal carcinoma in situ (DCIS). The general modeling approach is summarized in this article; full details, including approach, construction, data sources, assumptions, and implementation, are available at https://resources.cisnet.cancer.gov/registry and reference 33. Additional information is available on request, and the models are available for use via collaboration. The models begin with estimates of breast cancer incidence (31) and ER/HER2-specific survival trends without screening or adjuvant treatment, and then overlay data on screening and molecular subtypespecific adjuvant treatment to generate observed incidence and breast cancerspecific mortality trends in the U.S. population (1, 7, 17, 23, 33, 34). Breast cancer has a distribution of preclinical screen-detectable periods (sojourn time) and clinical detection points. Performance characteristics of digital mammography depend on age, first versus subsequent screen, time since last mammogram, and breast density. ER/HER2 status is assigned at diagnosis on the basis of stage and age. Molecular subtype and stage-specific treatment reduces the hazard of breast cancer death (models D, GE, M, and S) or results in a cure for some cases (models E and W). Women can die of breast cancer or other causes. Screen detection of cancer during the preclinical screen-detectable period can result in the identification and treatment of earlier-stage or smaller tumors than might occur via clinical detection, with a corresponding reduction in breast cancer mortality. We used a cohort of women born in 1970 with average-risk and average breast density and followed them from age 25 years (because breast cancer is rare before this age [0.08% of cases]) until death or age 100 years. Model Input Parameters The models used a common set of age-specific variables for breast cancer incidence, performance of digital mammography, treatment effects, and average and comorbidity-specific nonbreast cancer causes of death (20, 35). The parameter values are available at www.uspreventiveservicestaskforce.org/Page/Document/modeling-report-collaborative-modeling-of-us-breast-cancer-1/breast-cancer-screening1 (33). In addition, each group included model-specific inputs (or intermediate outputs) to represent preclinical detectable times, lead time, and age- and ER/HER2-specific stage distribution in screen- versus nonscreen-detected women on the basis of their specific model structure (1, 7, 2330). These model-specific parameters were based on assumptions about combinations of values that reproduced U.S. trends in incidence and breast cancerspecific mortality, including proportions of DCIS that were nonprogressive and would not be detected without screening. Models M and W also assumed some small nonprogressive invasive cancers. The models adopted an ageperiodcohort modeling approach to project incidence rates of breast cancer in the absence of screening (31, 36); model M used 19751979 rates from the Surveillance, Epidemiology, and End Results program. The models assumed 100% adherence to screening and receipt of the most effective treatment to isolate the effect of varying screening strategies. Four models used age-specific sensitivity values for digital mammography that were observed in the Breast Cancer Surveillance Consortium (BCSC) for detection of invasive and DCIS cancers combined (model S uses data for invasive cancers only). Separate values were used for initial and subsequent mammography by screening interval, using standard BCSC definitions: Annual includes data from screens occurring within 9 to 18 months of the prior screen, and biennial includes data on screens within 19 to 30 months (37, 38). Model D used these data as input variables (28), and models GE, S, and W used the data for calibration (24, 25, 27). Models E and M fit estimates from the BCSC and other data (26, 29). Women with ER-positive tumors received 5 years of hormone therapy and an anthracycline-based regimen accompanied by a taxane. Women with ER-negative invasive tumors received anthracycline-based regimens with a taxane. Those with HER2-positive tumors also received trastuzumab. Women with ER-positive DCIS received hormonal therapy (16). Treatment effectiveness was based on clinical trials and was modeled as a reduction in breast cancerspecific mortality risk or increase in the proportion cured compared with ER/HER2-specific survival in the absence of adjuvant treatment (32). Benefits Screening benefits (vs. no screening or incremental to other strategies) included percentage of reduction in breast cancer mortality, breast cancer deaths averted, and life-years and quality-adjusted life-years (QALYs) gained because of averted or delayed breast cancer death. Benefits (and harms) were accumulated from age 40 to 100 years to capture the lifetime effect of screening. We considered preferences, or utilities, to account for morbidity from screening and treatment. A disutility for age- and sex-specific general population health was first applied to quality-adjust the life-years (39). These were further adjusted to account for additional decrements in life-years related to undergoing screening (0.006 for 1 week, or 1 hour), evaluating a positive screen (0.105 for 5 weeks, or 3.7 days), undergoing initial treatment by stage (for the first 2 years after diagnosis), and having distant disease (for the last year of life for all women who die of breast cancer) (Appendix Table 1) (33, 40, 41). Appendix Table 1. Utility Input Parameter Values Use and Harms Use of services focused on the number of mammograms required for the screening strategy. Harms included false-positive mammograms, benign biopsies, and overdiagnosis. Rates of false-positive mammograms were calculated as mammograms read as abnormal or needing further work-up in women without cancer divided by the total number of screening

OR Forum---A POMDP Approach to Personalize Mammography Screening Decisions

Breast cancer is the most common nonskin cancer and the second leading cause of cancer death in U.S. women. Although mammography is the most effective modality for breast cancer screening, it has several potential risks, including high false-positive rates. Therefore, the balance of benefits and risks, which depend on personal characteristics, is critical in designing a mammography screening schedule. In contrast to prior research and existing guidelines that consider population-based screening recommendations, we propose a personalized mammography screening policy based on the prior screening history and personal risk characteristics of women. We formulate a finite-horizon, partially observable Markov decision process POMDP model for this problem. Our POMDP model incorporates two methods of detection self or screen, age-specific unobservable disease progression, and age-specific mammography test characteristics. We solve this POMDP optimally after setting transition probabilities to values estimated from a validated microsimulation model. Additional published data is used to specify other model inputs such as sensitivity and specificity of test results. Our results show that our proposed personalized screening schedules outperform the existing guidelines with respect to the total expected quality-adjusted life years, while significantly decreasing the number of mammograms and false-positives. We also report the lifetime risk of developing undetected invasive cancer associated with each screening scenario.

Benefits, harms, and cost-effectiveness of supplemental ultrasonography screening for women with dense breasts

Background At least nineteen states have laws that require telling women with dense breasts and a negative screening mammogram to consider supplemental screening. The most readily available supplemental screening modality is ultrasound, yet little is known about its effectiveness.