Jeremy B. Sussman

Examining the Evidence: A Systematic Review of the Inclusion and Analysis of Older Adults in Randomized Controlled Trials

ABSTRACTBACKGROUNDDue to a shortage of studies focusing on older adults, clinicians and policy makers frequently rely on clinical trials of the general population to provide supportive evidence for treating complex, older patients.OBJECTIVESTo examine the inclusion and analysis of complex, older adults in randomized controlled trials.REVIEW METHODSA PubMed search identified phase III or IV randomized controlled trials published in 2007 in JAMA, NEJM, Lancet, Circulation, and BMJ. Therapeutic interventions that assessed major morbidity or mortality in adults were included. For each study, age eligibility, average age of study population, primary and secondary outcomes, exclusion criteria, and the frequency, characteristics, and methodology of age-specific subgroup analyses were reviewed.RESULTSOf the 109 clinical trials reviewed in full, 22 (20.2%) excluded patients above a specified age. Almost half (45.6%) of the remaining trials excluded individuals using criteria that could disproportionately impact older adults. Only one in four trials (26.6%) examined outcomes that are considered highly relevant to older adults, such as health status or quality of life. Of the 42 (38.5%) trials that performed an age-specific subgroup analysis, fewer than half examined potential confounders of differential treatment effects by age, such as comorbidities or risk of primary outcome. Trials with age-specific subgroup analyses were more likely than those without to be multicenter trials (97.6% vs. 79.1%, p < 0.01) and funded by industry (83.3% vs. 62.7%, p < 0.05). Differential benefit by age was found in seven trials (16.7%).CONCLUSIONClinical trial evidence guiding treatment of complex, older adults could be improved by eliminating upper age limits for study inclusion, by reducing the use of eligibility criteria that disproportionately affect multimorbid older patients, by evaluating outcomes that are highly relevant to older individuals, and by encouraging adherence to recommended analytic methods for evaluating differential treatment effects by age.

Evidence-Based Risk Communication: A Systematic Review

Shared decision making is a collaborative process that allows patients and medical professionals to consider the best scientific evidence available, along with patients values and preferences, to make health care decisions (1). A recent Institute of Medicine report concluded that although people desire a patient experience that includes deep engagement in shared decision making, there are gaps between what patients want and what they get (2). For patients to get the experience they want, providers must effectively communicate evidence about benefits and harms. To improve the decision-making process, the Institute of Medicine recommended development and dissemination of high-quality communication tools (2). New tools, however, must match patients numerical abilities, which are often limited. For example, in one study, as many as 40% of high school graduates could not perform basic numerical operations, such as converting 1% of 1000 to 10 of 1000. This collective statistical illiteracy is a major barrier to the interpretation of health statistics (3). Physicians may also find statistical information difficult to interpret and explain (4). Existing literature about methods of communicating benefits and harms is broad. One review, based on 19 studies, concluded that the choice of a specific graphic is not as important as whether the graphic frames the frequency of an event with a visual representation of the total population in which it occurs (5). Another review, involving a limited literature search, found that comprehension improved when using frequencies (such as 1 in 5) instead of event rates (such as 20%) and using absolute risk reductions (ARRs) instead of relative risk reductions (RRRs) (6). The review did not assess affective outcomes, such as patient satisfaction, and behavioral outcomes, such as changes in decision making. Yet another review identified strong evidence that patients misinterpret RRRs and supported the effectiveness of graphs in communicating harms (7). However, they did not examine the comparative effectiveness of such approaches. More narrowly focused Cochrane reviews examined the communication of risk specific to screening tests (8, 9); numerical presentations, such as ARRs, RRRs, and numbers needed to treat (NNTs) (10); and effects of decision aids (11). An expert commentary about effective risk communication recommended using plain language, icon arrays, and absolute risks and providing time intervals with risk information (12). A group of experts identified 11 key components of risk communication, including presenting numerical estimates in context with evaluative labels, conveying uncertainty, and tailoring estimates (13). The aim of this systematic review is to comprehensively examine the comparative effectiveness of all methods of communicating probabilistic information about benefits and harms to patients to maximize their understanding, satisfaction, and decision-making ability. Methods We developed and followed a plan for the review that included several searches and dual abstraction of study data using standardized abstraction forms. Data Sources and Study Selection We searched PubMed (1966 to March 2014), CINAHL, EMBASE, and the Cochrane Central Register of Controlled Trials (1966 to December 2011) using keywords and structured terms related to the concepts of patients; communication; riskbenefit; and outcomes, such as understanding or comprehension, preferences or satisfaction, and decision making. Supplement 1 shows the detailed search strategy. Supplement 1. Search Strategies We included cross-sectional or prospective, longitudinal trials that were published in English and had an active control group that recruited patients or healthy volunteers and compared any method of communicating probabilistic information with another method. We focused on different methods of communicating the same specific probabilities to eliminate any independent effects that could result from different probabilities being studied (for example, different magnitudes or directions of effect). Studies of personalized risks, which may vary from person to person, were included when participants were randomly assigned. When studies of personalized risks were not randomized, the risks were considered to differ between the groups and were excluded. No limits were placed on study size, location, or duration or on the nature of the communication method. When needed, we reviewed sources specified in the articles, such as Web sites, to directly review the interventions and determine whether probabilistic information was addressed. Studies of medical students, health professionals, and public health or mass media campaigns were excluded. One independent reviewer screened each title and abstract and excluded citations that were not original studies or were unrelated to probabilistic information. Two independent reviewers screened the full text of the remaining citations to identify eligible articles. Disagreements between the 2 reviewers were resolved by consensus, with a third reviewer arbitrating any unresolved disagreements. Data Extraction and Quality Assessment Two reviewers independently abstracted detailed information about the study population, interventions, primary outcomes, and risk of bias from each included study using a standardized abstraction form, which was developed a priori (Supplement 2). A third reviewer resolved any disagreements. We categorized outcomes in 1 of 3 domains: cognitive (or understanding, such as accuracy in answering questions related to probabilistic information, or general comprehension of the probabilistic information), affective (such as preferences for or satisfaction with the method of communicating probabilistic information), and behavioral (such as real or theoretical decision making). Supplement 2. Abstraction Form Risk of bias in randomized, controlled trials was assessed on the basis of adequacy of randomization, allocation concealment, similarity of study groups at baseline, blinding, equal treatment of groups throughout the study, completeness of follow-up, and intention to treat (participants analyzed in the groups to which they were randomly assigned) (14). Risk of bias in observational studies was assessed with a modified set of criteria adapted from the NewcastleOttawa Scale (15). Data Synthesis and Analysis Data were tabulated, and the frequency of all head-to-head comparisons in studies was assessed to identify clusters of comparisons. In many instances, several interventions were bundled in a single study group (such as event rate plus icon array, or event rate plus natural frequencies plus ARRs). Bundles were not separated or combined with similar interventions because it could not be determined which component of the bundle drove the intervention. Descriptive statistics were used. We decided a priori not to do meta-analysis because of study heterogeneity. We emphasized findings from randomized studies as well as nonrandomized studies when findings were supported by more than 1 study. Role of the Funding Source No funding supported this study. The authors participated within their role on the Evidence-Based Medicine Task Force of the Society of General Internal Medicine. Results The initial search through December 2011 retrieved 22103 citations (16661 from PubMed, 1194 from CINAHL, 2861 from the Cochrane Central Register of Controlled Trials, and 1387 from EMBASE), and 20076 remained after removing duplicates. We updated the PubMed search through 30 March 2014, yielding 6529 additional citations; 5970 remained after removing duplicates, for a total of 26046 citations for review. A total of 630 articles were selected for full-text review and 84 were included, representing 91 unique studies (1699). Reasons for exclusion are noted in Figure 1, and study details are provided in Supplement 3. Figure 1. Summary of evidence search and selection. Supplement 3. Details of All Included Studies Seventy-four (81.3%) of the 91 included studies were randomized trials, most with cross-sectional designs. The median number of participants in randomized trials was 268 (range, 31 to 4685), and the median in all studies was 268 (range, 24 to 16133). Thirty-three studies (36.3%) included patients at specific risk for the target condition of interest. Forty-eight studies (52.7%) presented probabilistic data about benefits of a therapy or intervention (with 7 [14.6%] also presenting harms), 21 (23.1%) presented data only on harms, and 9 (10%) involved screening tests. Forty-nine studies (54.4%) delivered interventions on paper and 39 (42.9%) on a computer, typically over the Internet. The characteristics of study participants are presented in Tables 1 and 2. Table 1. Characteristics of Study Participants Table 2. Proportion of Studies Including Participants at Risk Versus Not at Risk for Target Condition Risk of bias for the included randomized trials was moderate (Figure 2). Randomization was adequate in 32 trials (42.7%), inadequate in 3 (4.0%), and unclear in 40 (53.3%). Allocation concealment was not stated in 55 trials (73.3%). Similarity of groups at baseline was adequate in 37 trials (49.3%) and unclear in 32 (42.7%). Blinding, equal treatment, and intention-to-treat items were similarly difficult to assess from reported information. Figure 2. Risk of bias for randomized, controlled trials (n = 74). Adapted from reference 100. Study Interventions and Comparators A frequency table (heat map) of all study intervention comparisons was created to identify clusters of comparisons (Supplement 4). The heat map represents study group comparisons, so one study may contribute several comparisons. The most commonly studied numerical presentations of data were natural frequencies, defined as the numbers of persons with events juxtaposed with a baseline denominator of persons (for example, 4 out of 100 persons had the outcome); event rates, defined as the proportions of persons wi

An IV for the RCT: using instrumental variables to adjust for treatment contamination in randomised controlled trials

Although the randomised controlled trial is the “gold standard” for studying the efficacy and safety of medical treatments, it is not necessarily free from bias. When patients do not follow the protocol for their assigned treatment, the resultant “treatment contamination” can produce misleading findings. The methods used historically to deal with this problem, the “as treated” and “per protocol” analysis techniques, are flawed and inaccurate. Intention to treat analysis is the solution most often used to analyse randomised controlled trials, but this approach ignores this issue of treatment contamination. Intention to treat analysis estimates the effect of recommending a treatment to study participants, not the effect of the treatment on those study participants who actually received it. In this article, we describe a simple yet rarely used analytical technique, the “contamination adjusted intention to treat analysis,” which complements the intention to treat approach by producing a better estimate of the benefits and harms of receiving a treatment. This method uses the statistical technique of instrumental variable analysis to address contamination. We discuss the strengths and limitations of the current methods of addressing treatment contamination and the contamination adjusted intention to treat technique, provide examples of effective uses, and discuss how using estimates generated by contamination adjusted intention to treat analysis can improve clinical decision making and patient care.

Effect of Perioperative Statins on Death, Myocardial Infarction, Atrial Fibrillation, and Length of Stay A Systematic Review and Meta-analysis

OBJECTIVE To assess the influence of perioperative statin treatment on the risk of death, myocardial infarction, atrial fibrillation, and hospital and intensive care unit length of stay in statin-naive patients undergoing cardiac or noncardiac surgery. DATA SOURCES MEDLINE via PubMed, EMBASE, Biosis, and the Cochrane Central Register of Controlled Trials via Ovid. Additional studies were identified through hand searches of bibliographies, trial Web sites, and clinical experts. Randomized controlled trials reporting the effect of perioperative statins in statin-naive patients undergoing cardiac and noncardiac surgery were included. STUDY SELECTION Two investigators independently selected eligible studies from original research published in any language studying the effects of statin use on perioperative outcomes of interest. DATA EXTRACTION Two investigators performed independent article abstraction and quality assessment. DATA SYNTHESIS Fifteen randomized controlled studies involving 2292 patients met the eligibility criteria. Random-effects meta-analyses of unadjusted and adjusted data were performed according to the method described by DerSimonian and Laird. Perioperative statin treatment decreased the risk of atrial fibrillation in patients undergoing cardiac surgery (relative risk [RR], 0.56; 95% CI, 0.45 to 0.69; number needed to treat [NNT], 6). In cardiac and noncardiac surgery, perioperative statin treatment reduced the risk of myocardial infarction (RR, 0.53; 95% CI, 0.38 to 0.74; NNT, 23) but not the risk of death (RR, 0.62; 95% CI, 0.34 to 1.14). Statin treatment reduced mean length of hospital stay (standardized mean difference, -0.32; 95% CI, -0.53 to -0.11) but had no effect on length of intensive care unit stay (standardized mean difference, -0.08; 95% CI, -0.25 to 0.10). CONCLUSIONS Perioperative statin treatment in statin-naive patients reduces atrial fibrillation, myocardial infarction, and duration of hospital stay. Wider use of statins to improve cardiac outcomes in patients undergoing high-risk procedures seems warranted.

Evidence-based risk communication

Shared decision making is a collaborative process that allows patients and medical professionals to consider the best scientific evidence available, along with patients values and preferences, to make health care decisions (1). A recent Institute of Medicine report concluded that although people desire a patient experience that includes deep engagement in shared decision making, there are gaps between what patients want and what they get (2). For patients to get the experience they want, providers must effectively communicate evidence about benefits and harms. To improve the decision-making process, the Institute of Medicine recommended development and dissemination of high-quality communication tools (2). New tools, however, must match patients numerical abilities, which are often limited. For example, in one study, as many as 40% of high school graduates could not perform basic numerical operations, such as converting 1% of 1000 to 10 of 1000. This collective statistical illiteracy is a major barrier to the interpretation of health statistics (3). Physicians may also find statistical information difficult to interpret and explain (4). Existing literature about methods of communicating benefits and harms is broad. One review, based on 19 studies, concluded that the choice of a specific graphic is not as important as whether the graphic frames the frequency of an event with a visual representation of the total population in which it occurs (5). Another review, involving a limited literature search, found that comprehension improved when using frequencies (such as 1 in 5) instead of event rates (such as 20%) and using absolute risk reductions (ARRs) instead of relative risk reductions (RRRs) (6). The review did not assess affective outcomes, such as patient satisfaction, and behavioral outcomes, such as changes in decision making. Yet another review identified strong evidence that patients misinterpret RRRs and supported the effectiveness of graphs in communicating harms (7). However, they did not examine the comparative effectiveness of such approaches. More narrowly focused Cochrane reviews examined the communication of risk specific to screening tests (8, 9); numerical presentations, such as ARRs, RRRs, and numbers needed to treat (NNTs) (10); and effects of decision aids (11). An expert commentary about effective risk communication recommended using plain language, icon arrays, and absolute risks and providing time intervals with risk information (12). A group of experts identified 11 key components of risk communication, including presenting numerical estimates in context with evaluative labels, conveying uncertainty, and tailoring estimates (13). The aim of this systematic review is to comprehensively examine the comparative effectiveness of all methods of communicating probabilistic information about benefits and harms to patients to maximize their understanding, satisfaction, and decision-making ability. Methods We developed and followed a plan for the review that included several searches and dual abstraction of study data using standardized abstraction forms. Data Sources and Study Selection We searched PubMed (1966 to March 2014), CINAHL, EMBASE, and the Cochrane Central Register of Controlled Trials (1966 to December 2011) using keywords and structured terms related to the concepts of patients; communication; riskbenefit; and outcomes, such as understanding or comprehension, preferences or satisfaction, and decision making. Supplement 1 shows the detailed search strategy. Supplement 1. Search Strategies We included cross-sectional or prospective, longitudinal trials that were published in English and had an active control group that recruited patients or healthy volunteers and compared any method of communicating probabilistic information with another method. We focused on different methods of communicating the same specific probabilities to eliminate any independent effects that could result from different probabilities being studied (for example, different magnitudes or directions of effect). Studies of personalized risks, which may vary from person to person, were included when participants were randomly assigned. When studies of personalized risks were not randomized, the risks were considered to differ between the groups and were excluded. No limits were placed on study size, location, or duration or on the nature of the communication method. When needed, we reviewed sources specified in the articles, such as Web sites, to directly review the interventions and determine whether probabilistic information was addressed. Studies of medical students, health professionals, and public health or mass media campaigns were excluded. One independent reviewer screened each title and abstract and excluded citations that were not original studies or were unrelated to probabilistic information. Two independent reviewers screened the full text of the remaining citations to identify eligible articles. Disagreements between the 2 reviewers were resolved by consensus, with a third reviewer arbitrating any unresolved disagreements. Data Extraction and Quality Assessment Two reviewers independently abstracted detailed information about the study population, interventions, primary outcomes, and risk of bias from each included study using a standardized abstraction form, which was developed a priori (Supplement 2). A third reviewer resolved any disagreements. We categorized outcomes in 1 of 3 domains: cognitive (or understanding, such as accuracy in answering questions related to probabilistic information, or general comprehension of the probabilistic information), affective (such as preferences for or satisfaction with the method of communicating probabilistic information), and behavioral (such as real or theoretical decision making). Supplement 2. Abstraction Form Risk of bias in randomized, controlled trials was assessed on the basis of adequacy of randomization, allocation concealment, similarity of study groups at baseline, blinding, equal treatment of groups throughout the study, completeness of follow-up, and intention to treat (participants analyzed in the groups to which they were randomly assigned) (14). Risk of bias in observational studies was assessed with a modified set of criteria adapted from the NewcastleOttawa Scale (15). Data Synthesis and Analysis Data were tabulated, and the frequency of all head-to-head comparisons in studies was assessed to identify clusters of comparisons. In many instances, several interventions were bundled in a single study group (such as event rate plus icon array, or event rate plus natural frequencies plus ARRs). Bundles were not separated or combined with similar interventions because it could not be determined which component of the bundle drove the intervention. Descriptive statistics were used. We decided a priori not to do meta-analysis because of study heterogeneity. We emphasized findings from randomized studies as well as nonrandomized studies when findings were supported by more than 1 study. Role of the Funding Source No funding supported this study. The authors participated within their role on the Evidence-Based Medicine Task Force of the Society of General Internal Medicine. Results The initial search through December 2011 retrieved 22103 citations (16661 from PubMed, 1194 from CINAHL, 2861 from the Cochrane Central Register of Controlled Trials, and 1387 from EMBASE), and 20076 remained after removing duplicates. We updated the PubMed search through 30 March 2014, yielding 6529 additional citations; 5970 remained after removing duplicates, for a total of 26046 citations for review. A total of 630 articles were selected for full-text review and 84 were included, representing 91 unique studies (1699). Reasons for exclusion are noted in Figure 1, and study details are provided in Supplement 3. Figure 1. Summary of evidence search and selection. Supplement 3. Details of All Included Studies Seventy-four (81.3%) of the 91 included studies were randomized trials, most with cross-sectional designs. The median number of participants in randomized trials was 268 (range, 31 to 4685), and the median in all studies was 268 (range, 24 to 16133). Thirty-three studies (36.3%) included patients at specific risk for the target condition of interest. Forty-eight studies (52.7%) presented probabilistic data about benefits of a therapy or intervention (with 7 [14.6%] also presenting harms), 21 (23.1%) presented data only on harms, and 9 (10%) involved screening tests. Forty-nine studies (54.4%) delivered interventions on paper and 39 (42.9%) on a computer, typically over the Internet. The characteristics of study participants are presented in Tables 1 and 2. Table 1. Characteristics of Study Participants Table 2. Proportion of Studies Including Participants at Risk Versus Not at Risk for Target Condition Risk of bias for the included randomized trials was moderate (Figure 2). Randomization was adequate in 32 trials (42.7%), inadequate in 3 (4.0%), and unclear in 40 (53.3%). Allocation concealment was not stated in 55 trials (73.3%). Similarity of groups at baseline was adequate in 37 trials (49.3%) and unclear in 32 (42.7%). Blinding, equal treatment, and intention-to-treat items were similarly difficult to assess from reported information. Figure 2. Risk of bias for randomized, controlled trials (n = 74). Adapted from reference 100. Study Interventions and Comparators A frequency table (heat map) of all study intervention comparisons was created to identify clusters of comparisons (Supplement 4). The heat map represents study group comparisons, so one study may contribute several comparisons. The most commonly studied numerical presentations of data were natural frequencies, defined as the numbers of persons with events juxtaposed with a baseline denominator of persons (for example, 4 out of 100 persons had the outcome); event rates, defined as the proportions of persons wi

Rates of Deintensification of Blood Pressure and Glycemic Medication Treatment Based on Levels of Control and Life Expectancy in Older Patients With Diabetes Mellitus

IMPORTANCE Older patients with diabetes mellitus receiving medical treatment whose blood pressure (BP) or blood glucose level are potentially dangerously low are rarely deintensified. Given the established risks of low blood pressure and blood glucose, this is a major opportunity to decrease medication harm. OBJECTIVE To examine the rate of BP- and blood glucose-lowering medicine deintensification among older patients with type 1 or 2 diabetes mellitus who potentially receive overtreatment. DESIGN, SETTING, AND PARTICIPANTS Retrospective cohort study conducted using data from the US Veterans Health Administration. Participants included 211 667 patients older than 70 years with diabetes mellitus who were receiving active treatment (defined as BP-lowering medications other than angiotensin-converting enzyme inhibitors or angiotensin receptor blockers, or glucose-lowering medications other than metformin hydrochloride) from January 1 to December 31, 2012. Data analysis was performed December 10, 2013, to July 20, 2015. EXPOSURES Participants were eligible for deintensification of treatment if they had low BP or a low hemoglobin A1c (HbA1c) level in their last measurement in 2012. We defined very low BP as less than 120/65 mm Hg, moderately low as systolic BP of 120 to 129 mm Hg or diastolic BP (DBP) less than 65 mm Hg, very low HbA1c as less than 6.0%, and moderately low HbA1c as 6.0% to 6.4%. All other values were not considered low. MAIN OUTCOMES AND MEASURES Medication deintensification, defined as discontinuation or dosage decrease within 6 months after the index measurement. RESULTS The actively treated BP cohort included 211,667 participants, more than half of whom had moderately or very low BP levels. Of 104,486 patients with BP levels that were not low, treatment in 15.1% was deintensified. Of 25,955 patients with moderately low BP levels, treatment in 16.0% was deintensified. Among 81,226 patients with very low BP levels, 18.8% underwent BP medication deintensification. Of patients with very low BP levels whose treatment was not deintensified, only 0.2% had a follow-up BP measurement that was elevated (BP ≥140/90 mm Hg). The actively treated HbA1c cohort included 179,991 participants. Of 143,305 patients with HbA1c levels that were not low, treatment in 17.5% was deintensified. Of 23,769 patients with moderately low HbA1c levels, treatment in 20.9% was deintensified. Among 12,917 patients with very low HbA1c levels, 27.0% underwent medication deintensification. Of patients with very low HbA1c levels whose treatment was not deintensified, fewer than 0.8% had a follow-up HbA1c measurement that was elevated (≥7.5%). CONCLUSIONS AND RELEVANCE Among older patients whose treatment resulted in very low levels of HbA1c or BP, 27% or fewer underwent deintensification, representing a lost opportunity to reduce overtreatment. Low HbA1c or BP values or low life expectancy had little association with deintensification events. Practice guidelines and performance measures should place more focus on reducing overtreatment through deintensification.

Using Benefit-Based Tailored Treatment to Improve the Use of Antihypertensive Medications

Background— Current guidelines for prescribing antihypertensive medications focus on reaching specific blood pressure targets. We sought to determine whether antihypertensive medications could be used more effectively by a treatment strategy based on tailored estimates of cardiovascular disease events prevented. Methods and Results— We developed a nationally representative sample of American adults aged 30 to 85 years with no history of myocardial infarction, stroke, or severe congestive heart failure using the National Health and Nutrition Examination Survey III. We then created a simulation model to estimate the effects of 5 years of treatment with treat-to-target (treatment to specific blood pressure goals) and benefit-based tailored treatment (treatment based on estimated cardiovascular disease event reduction) approaches to antihypertensive medication management. All effect size estimates were derived directly from meta-analyses of randomized trials. We found that 55% of the overall population of 176 million Americans would be treated identically under the 2 treatment approaches. Benefit-based tailored treatment would prevent 900 000 more cardiovascular disease events and save 2.8 million more quality-adjusted life-years, despite using 6% fewer medications over 5 years. In the 45% of the population treated differently by the strategies, benefit-based tailored treatment would save 159 quality-adjusted life-years per 1000 treated versus 74 quality-adjusted life-years per 1000 treated by the treat-to-target approach. The findings were robust to sensitivity analyses. Conclusions— We found that benefit-based tailored treatment was both more effective and required less antihypertensive medication than current guidelines based on treating to specific blood pressure goals.

Improving diabetes prevention with benefit based tailored treatment: risk based reanalysis of Diabetes Prevention Program

Objective To determine whether some participants in the Diabetes Prevention Program were more or less likely to benefit from metformin or a structured lifestyle modification program. Design Post hoc analysis of the Diabetes Prevention Program, a randomized controlled trial. Setting Ambulatory care patients. Participants 3060 people without diabetes but with evidence of impaired glucose metabolism. Intervention Intervention groups received metformin or a lifestyle modification program with the goals of weight loss and physical activity. Main outcome measure Development of diabetes, stratified by the risk of developing diabetes according to a diabetes risk prediction model. Results Of the 3081 participants with impaired glucose metabolism at baseline, 655 (21%) progressed to diabetes over a median 2.8 years’ follow-up. The diabetes risk model had good discrimination (C statistic=0.73) and calibration. Although the lifestyle intervention provided a sixfold greater absolute risk reduction in the highest risk quarter than in the lowest risk quarter, patients in the lowest risk quarter still received substantial benefit (three year absolute risk reduction 4.9% v 28.3% in highest risk quarter; numbers needed to treat of 20.4 and 3.5, respectively). The benefit of metformin, however, was seen almost entirely in patients in the top quarter of risk of diabetes. No benefit was seen in the lowest risk quarter. Participants in the highest risk quarter averaged a 21.4% three year absolute risk reduction (number needed to treat 4.6). Conclusions Patients at high risk of diabetes have substantial variation in their likelihood of receiving benefit from diabetes prevention treatments. Using this knowledge could decrease overtreatment and make prevention of diabetes far more efficient, effective, and patient centered, provided that decision making is based on an accurate risk prediction tool.

Three simple rules to ensure reasonably credible subgroup analyses

The limitations of subgroup analyses are well established—false positives due to multiple comparisons, false negatives due to inadequate power, and limited ability to inform individual treatment decisions because patients have multiple characteristics that vary simultaneously. In this article, we apply Bayes’s rule to determine the probability that a positive subgroup analysis is a true positive. From this framework, we derive simple rules to determine when subgroup analyses can be performed as hypothesis testing analyses and thus inform when subgroup analyses should influence how we practice medicine.

Improved cardiovascular risk prediction using nonparametric regression and electronic health record data

Background:Use of the electronic health record (EHR) is expected to increase rapidly in the near future, yet little research exists on whether analyzing internal EHR data using flexible, adaptive statistical methods could improve clinical risk prediction. Extensive implementation of EHR in the Veterans Health Administration provides an opportunity for exploration. Objectives:To compare the performance of various approaches for predicting risk of cerebrovascular and cardiovascular (CCV) death, using traditional risk predictors versus more comprehensive EHR data. Research Design:Retrospective cohort study. We identified all Veterans Health Administration patients without recent CCV events treated at 12 facilities from 2003 to 2007, and predicted risk using the Framingham risk score, logistic regression, generalized additive modeling, and gradient tree boosting. Measures:The outcome was CCV-related death within 5 years. We assessed each method’s predictive performance with the area under the receiver operating characteristic curve (AUC), the Hosmer-Lemeshow goodness-of-fit test, plots of estimated risk, and reclassification tables, using cross-validation to penalize overfitting. Results:Regression methods outperformed the Framingham risk score, even with the same predictors (AUC increased from 71% to 73% and calibration also improved). Even better performance was attained in models using additional EHR-derived predictor variables (AUC increased to 78% and net reclassification improvement was as large as 0.29). Nonparametric regression further improved calibration and discrimination compared with logistic regression. Conclusions:Despite the EHR lacking some risk factors and its imperfect data quality, health care systems may be able to substantially improve risk prediction for their patients by using internally developed EHR-derived models and flexible statistical methodology.