Aaron Wells

Exploring Robust Methods for Evaluating Treatment and Comparison Groups in Chronic Care Management Programs

Evaluation of chronic care management (CCM) programs is necessary to determine the behavioral, clinical, and financial value of the programs. Financial outcomes of members who are exposed to interventions (treatment group) typically are compared to those not exposed (comparison group) in a quasi-experimental study design. However, because member assignment is not randomized, outcomes reported from these designs may be biased or inefficient if study groups are not comparable or balanced prior to analysis. Two matching techniques used to achieve balanced groups are Propensity Score Matching (PSM) and Coarsened Exact Matching (CEM). Unlike PSM, CEM has been shown to yield estimates of causal (program) effects that are lowest in variance and bias for any given sample size. The objective of this case study was to provide a comprehensive comparison of these 2 matching methods within an evaluation of a CCM program administered to a large health plan during a 2-year time period. Descriptive and statistical methods were used to assess the level of balance between comparison and treatment members pre matching. Compared with PSM, CEM retained more members, achieved better balance between matched members, and resulted in a statistically insignificant Wald test statistic for group aggregation. In terms of program performance, the results showed an overall higher medical cost savings among treatment members matched using CEM compared with those matched using PSM (-

Comparing the contributions of well-being and disease status to employee productivity.

25.57 versus -

Impact of a Chronic Disease Management Program on Hospital Admissions and Readmissions in an Australian Population with Heart Disease or Diabetes

19.78, respectively). Collectively, the results suggest CEM is a viable alternative, if not the most appropriate matching method, to apply when evaluating CCM program performance.

The Impact of a Proactive Chronic Care Management Program on Hospital Admission Rates in a German Health Insurance Society

Objective: To compare employee overall well-being to chronic disease status, which has a long-established relationship to productivity, as relative contributors to on-the-job productivity. Methods: Data from two annual surveys of three companies were used in longitudinal analyses of well-being as a predictor of productivity level and productivity change among 2629 employees with diabetes or without any chronic conditions. Results: Well-being was the most significant predictor of productivity cross-sectionally in a model that included disease status and demographic characteristics. Longitudinally, changes in well-being contributed to changes in productivity above and beyond what could be explained by the presence of chronic disease or other fixed characteristics. Conclusions: These findings support the use of well-being as the broader framework for understanding, explaining, and improving employee productivity in both the healthy and those with disease.

Avoiding Randomization Failure in Program Evaluation, with Application to the Medicare Health Support Program

Chronic disease management programs (CDMPs) were introduced in Australia to reduce unnecessary health care utilization by the growing population with chronic conditions; however, evidence of effectiveness is needed. This study evaluated the impact of a comprehensive CDMP, My Health Guardian (MHG), on rate of hospital admissions, readmissions, and average length of hospital stay (ALOS) for insured individuals with heart disease or diabetes. Primary outcomes were assessed through retrospective comparison of members in MHG (treatment; n=5053) to similar nonparticipating members (comparison; n=23,077) using a difference-in-differences approach with the year before program commencement serving as baseline and the subsequent 12 or 18 months serving as the program periods. All outcomes were evaluated for the total study population and for disease-matched subgroups (heart disease and diabetes). Statistical tests were performed using multivariate regression controlling for age, sex, number of chronic diseases, and past hospitalization status. After both 12 and 18 months, treatment members displayed decreases in admissions (both, P≤0.001) and readmissions (both, P≤0.01), and ALOS after 18 months (P≤0.01) versus the comparison group; magnitude of impact increased over time for these 3 measures. All outcomes for both disease-matched subgroups directionally mirrored the total study group, but the diabetes subgroup did not achieve significance for readmissions or ALOS. Within the treatment group, admissions decreased with increasing care calls to members (12 and 18 months, P<0.0001). These results show that MHG successfully reduced the frequency and duration of hospital admissions and presents a promising approach to reduce the burden associated with hospitalizations in populations with chronic disease.

The Impact of Proactive Chronic Care Management on Hospital Admissions in a German Senior Population

Hospital admissions are the source of significant health care expenses, although a large proportion of these admissions can be avoided through proper management of chronic disease. In the present study, we evaluate the impact of a proactive chronic care management program for members of a German insurance society who suffer from chronic disease. Specifically, we tested the impact of nurse-delivered care calls on hospital admission rates. Study participants were insured individuals with coronary artery disease, heart failure, diabetes, or chronic obstructive pulmonary disease who consented to participate in the chronic care management program. Intervention (n  = 17,319) and Comparison (n  = 5668) groups were defined based on records of participating (or not participating) in telephonic interactions. Changes in admission rates were calculated from the year prior to (Base) and year after program commencement. Comparative analyses were adjusted for age, sex, region of residence, and disease severity (stratification of 3 [least severe] to 1 [most severe]). Overall, the admission rate in the Intervention group decreased by 6.2% compared with a 14.9% increase in the Comparison group (P  <  0.001). The overall decrease in admissions for the Intervention group was driven by risk stratification levels 2 and 1, for which admissions decreased by 8.2% and 14.2% compared to Comparison group increases of 12.1% and 7.9%, respectively. Additionally, Intervention group admissions decreased as the number of calls increased (P  =  0.004), indicating a dose-response relationship. These findings indicate that proactive chronic care management care calls can help reduce hospital admissions among German health insurance members with chronic disease.

Increased adherence to cardiac standards of care during participation in cardiac disease management programs.

We highlight common problems in the application of random treatment assignment in large-scale program evaluation. Random assignment is the defining feature of modern experimental design, yet errors in design, implementation, and analysis often result in real-world applications not benefiting from its advantages. The errors discussed here cover the control of variability, levels of randomization, size of treatment arms, and power to detect causal effects, as well as the many problems that commonly lead to post-treatment bias. We illustrate these issues by identifying numerous serious errors in the Medicare Health Support evaluation and offering recommendations to improve the design and analysis of this and other large-scale randomized experiments.

The Value of a Well-Being Improvement Strategy: Longitudinal Success across Subjective and Objective Measures Observed in a Firm Adopting a Consumer-Driven Health Plan

An increase in chronic disease prevalence is contributing to health care cost growth and decreased quality of life in industrialized nations worldwide. Inadequate management of chronic diseases is a leading cause of hospitalizations and, thus, avoidable expenditures. In this study, we evaluated the impact of nurse-delivered care calls, the primary intervention of a proactive chronic care management (CCM) program, in a population aged 65 and older in Germany. In this analysis, hospital admission rates were evaluated among program enrollees who were diagnosed with diabetes, heart failure, coronary heart disease, or chronic obstructive pulmonary disease. The Intervention group comprised those members who participated in care calls (n=13,486), whereas the Comparison group included enrollees who did not participate in these calls (n=4,582). Changes in admission rates were calculated between the year prior to and year after program commencement. Comparative analyses were adjusted for age, sex, region of residence, and disease severity (stratification of 3 [least severe] to 1 [most severe]). Overall, a 6.0% decrease in admissions was observed among Intervention group members compared with an 18.9% increase among Comparison group members (P ≤ 0.0001). This decrease in admissions was driven by participants with the highest levels of risk. In addition, a dose-response relationship was observed in which admissions decreased with an increased number of care calls (P=0.0001). These results indicate that proactive CCM interventions are effective in reducing hospital admission rates in a senior population with chronic disease.

Comparative Effectiveness of Matching Methods for Causal Inference

Adherence to cardiovascular disease standards of care is critically important for minimizing the risk of mortality and morbidity for individuals with coronary heart disease (CHD) and heart failure (HF). The purpose of this study was to assess the ability of cardiac disease management (DM) programs to assist members with their adherence to evidence-based medicine for cardiovascular diseases. A total of 20,202 members with CHD and/or HF were evaluated 12 months prior to the start of DM programs and during their first 12 months of participation in the programs. Members were assessed for their adherence to appropriate cardiac medications. In addition, low-density lipoprotein (LDL) testing rates and clinical control of LDL values (defined as <100 mg/dL) were measured. The association between LDL control and use of lipid-lowering statins also was assessed. During participation in the cardiac programs, members achieved significant improvement in their adherence to angiotensin-converting enzyme inhibitors/angiotensin receptor blockers, and beta-blockers (P < 0.0001). The cardiac population also achieved a significant increase in LDL testing rates and statin use (P < 0.0001). More members attained appropriate LDL control in year 1 compared to baseline (36% relative increase), and this improvement was associated with a 40% relative increase in statin use. In summary, participation in these cardiac DM programs assisted members to improve their adherence to cardiac medications and standards of care guidelines. Such improvements in cardiovascular disease care are likely associated with improved quality of life and reduced risk for mortality.

Extending coarsened exact matching to multiple cohorts: an application to longitudinal well-being program evaluation within an employer population

Objective: The objective of this study is to evaluate effectiveness of a firms 5-year strategy toward improving well-being while lowering health care costs amidst adoption of a Consumer-Driven Health Plan. Methods: Repeated measures statistical models were employed to test and quantify association between key demographic factors, employment type, year, individual well-being, and outcomes of health care costs, obesity, smoking, absence, and performance. Results: Average individual well-being trended upward by 13.5% over 5 years, monthly allowed amount health care costs declined 5.2% on average per person per year, and obesity and smoking rates declined by 4.8 and 9.7%, respectively, on average each year. The results show that individual well-being was significantly associated with each outcome and in the expected direction. Conclusions: The firms strategy was successful in driving statistically significant, longitudinal well-being, biometric and productivity improvements, and health care cost reduction.