Elizabeth Cooney

Outcomes and statistical power in adult critical care randomized trials.

RATIONALE Intensive care unit (ICU)-based randomized clinical trials (RCTs) among adult critically ill patients commonly fail to detect treatment benefits. OBJECTIVES Appraise the rates of success, outcomes used, statistical power, and design characteristics of published trials. METHODS One hundred forty-six ICU-based RCTs of diagnostic, therapeutic, or process/systems interventions published from January 2007 to May 2013 in 16 high-impact general or critical care journals were studied. MEASUREMENT AND MAIN RESULTS Of 146 RCTs, 54 (37%) were positive (i.e., the a priori hypothesis was found to be statistically significant). The most common primary outcomes were mortality (n = 40 trials), infection-related outcomes (n = 33), and ventilation-related outcomes (n = 30), with positive results found in 10, 58, and 43%, respectively. Statistical power was discussed in 135 RCTs (92%); 92 cited a rationale for their power parameters. Twenty trials failed to achieve at least 95% of their reported target sample size, including 11 that were stopped early due to insufficient accrual/logistical issues. Of 34 superiority RCTs comparing mortality between treatment arms, 13 (38%) accrued a sample size large enough to find an absolute mortality reduction of 10% or less. In 22 of these trials the observed control-arm mortality rate differed from the predicted rate by at least 7.5%. CONCLUSIONS ICU-based RCTs are commonly negative and powered to identify what appear to be unrealistic treatment effects, particularly when using mortality as the primary outcome. Additional concerns include a lack of standardized methods for assessing common outcomes, unclear justifications for statistical power calculations, insufficient patient accrual, and incorrect predictions of baseline event rates.

Discriminative Accuracy of Physician and Nurse Predictions for Survival and Functional Outcomes 6 Months After an ICU Admission

Importance Predictions of long-term survival and functional outcomes influence decision making for critically ill patients, yet little is known regarding their accuracy. Objective To determine the discriminative accuracy of intensive care unit (ICU) physicians and nurses in predicting 6-month patient mortality and morbidity, including ambulation, toileting, and cognition. Design, Setting, and Participants Prospective cohort study conducted in 5 ICUs in 3 hospitals in Philadelphia, Pennsylvania, and enrolling patients who spent at least 3 days in the ICU from October 2013 until May 2014 and required mechanical ventilation, vasopressors, or both. These patients’ attending physicians and bedside nurses were also enrolled. Follow-up was completed in December 2014. Main Outcomes and Measures ICU physicians’ and nurses’ binary predictions of in-hospital mortality and 6-month outcomes, including mortality, return to original residence, ability to toilet independently, ability to ambulate up 10 stairs independently, and ability to remember most things, think clearly, and solve day-to-day problems (ie, normal cognition). For each outcome, physicians and nurses provided a dichotomous prediction and rated their confidence in that prediction on a 5-point Likert scale. Outcomes were assessed via interviews with surviving patients or their surrogates at 6 months. Discriminative accuracy was measured using positive and negative likelihood ratios (LRs), C statistics, and other operating characteristics. Results Among 340 patients approached, 303 (89%) consented (median age, 62 years [interquartile range, 53-71]; 57% men; 32% African American); 6-month follow-up was completed for 299 (99%), of whom 169 (57%) were alive. Predictions were made by 47 physicians and 128 nurses. Physicians most accurately predicted 6-month mortality (positive LR, 5.91 [95% CI, 3.74-9.32]; negative LR, 0.41 [95% CI, 0.33-0.52]; C statistic, 0.76 [95% CI, 0.72-0.81]) and least accurately predicted cognition (positive LR, 2.36 [95% CI, 1.36-4.12]; negative LR, 0.75 [95% CI, 0.61-0.92]; C statistic, 0.61 [95% CI, 0.54-0.68]). Nurses most accurately predicted in-hospital mortality (positive LR, 4.71 [95% CI, 2.94-7.56]; negative LR, 0.61 [95% CI, 0.49-0.75]; C statistic, 0.68 [95% CI, 0.62-0.74]) and least accurately predicted cognition (positive LR, 1.50 [95% CI, 0.86-2.60]; negative LR, 0.88 [95% CI, 0.73-1.06]; C statistic, 0.55 [95% CI, 0.48-0.62]). Discriminative accuracy was higher when physicians and nurses were confident about their predictions (eg, for physicians’ confident predictions of 6-month mortality: positive LR, 33.00 [95% CI, 8.34-130.63]; negative LR, 0.18 [95% CI, 0.09-0.35]; C statistic, 0.90 [95% CI, 0.84-0.96]). Compared with a predictive model including objective clinical variables, a model that also included physician and nurse predictions had significantly higher discriminative accuracy for in-hospital mortality, 6-month mortality, and return to original residence (P < .01 for all). Conclusions and Relevance ICU physicians’ and nurses’ discriminative accuracy in predicting 6-month outcomes of critically ill patients varied depending on the outcome being predicted and confidence of the predictors. Further research is needed to better understand how clinicians derive prognostic estimates of long-term outcomes.

An Observational Study of Decision Making by Medical Intensivists.

Objectives:The ICU is a place of frequent, high-stakes decision making. However, the number and types of decisions made by intensivists have not been well characterized. We sought to describe intensivist decision making and determine how the number and types of decisions are affected by patient, provider, and systems factors. Design:Direct observation of intensivist decision making during patient rounds. Setting:Twenty-four-bed academic medical ICU. Subjects:Medical intensivists leading patient care rounds. Intervention:None. Measurements and Main Results:During 920 observed patient rounds on 374 unique patients, intensivists made 8,174 critical care decisions (mean, 8.9 decisions per patient daily, 102.2 total decisions daily) over a mean of 3.7 hours. Patient factors associated with increased numbers of decisions included a shorter time since ICU admission and an earlier slot in rounding order (both p < 0.05). Intensivist identity explained the greatest proportion of variance in number of decisions per patient even when controlling for all other factors significant in bivariable regression. A given intensivist made more decisions per patient during days later in the 14-day rotation (p < 0.05). Female intensivists made significantly more decisions than male intensivists (p < 0.05). Conclusions:Intensivists made over 100 daily critical care decisions during rounds. The number of decisions was influenced by a variety of patient- and system-related factors and was highly variable among intensivists. Future work is needed to explore effects of the decision-making burden on providers’ choices and on patient outcomes.

Do Windows or Natural Views Affect Outcomes or Costs Among Patients in Icus

Objective:To determine whether potential exposure to natural light via windows or to more pleasing views through windows affects outcomes or costs among critically ill patients. Design:Retrospective cohort study. Setting:An academic hospital in Philadelphia, PA. Patients:Six thousand one hundred thirty-eight patients admitted to a 24-bed medical ICU and 6,631 patients admitted to a 24-bed surgical ICU from July 1, 2006, to June 30, 2010. Interventions:Assignment to medical ICU rooms with vs. without windows and to surgical ICU rooms with natural vs. industrial views based on bed availability. Measurements and Main Results:In primary analyses adjusting for patient characteristics, medical ICU patients admitted to rooms with (n = 4,093) versus without (n = 2,243) windows did not differ in rates of ICU (p = 0.25) or in-hospital (p = 0.94) mortality, ICU readmissions (p = 0.37), or delirium (p = 0.56). Surgical ICU patients admitted to rooms with natural (n = 3,072) versus industrial (n = 3,588) views experienced slightly shorter ICU lengths of stay and slightly lower variable costs. Instrumental variable analyses based on initial bed assignment and exposure time did not show any differences in any outcomes in either the medical ICU or surgical ICU cohorts, and none of the differences noted in primary analyses remained statistically significant when adjusting for multiple comparisons. In a prespecified subgroup analysis among patients with ICU length of stay greater than 72 hours, MICU windows were associated with reduced ICU (p = 0.02) and hospital mortality (p = 0.04); these results did not meet criteria for significance after adjustment for multiple comparisons. Conclusions:ICU rooms with windows or natural views do not improve outcomes or reduce costs of in-hospital care for general populations of medical and surgical ICU patients. Future work is needed to determine whether targeting light from windows directly toward patients influences outcomes and to explore these effects in patients at high risk for adverse outcomes.

Rationale and Design of the Randomized Evaluation of Default Access to Palliative Services (REDAPS) Trial.

The substantial nationwide investment in inpatient palliative care services stems from their great promise to improve patient-centered outcomes and reduce costs. However, robust experimental evidence of these benefits is lacking. The Randomized Evaluation of Default Access to Palliative Services (REDAPS) study is a pragmatic, stepped-wedge, cluster randomized trial designed to test the efficacy and costs of specialized palliative care consultative services for hospitalized patients with advanced chronic obstructive pulmonary disease, dementia, or end-stage renal disease, as well as the overall effectiveness of ordering such services by default. Additional aims are to identify the types of services that are most beneficial and the types of patients most likely to benefit, including comparisons between ward and intensive care unit patients. We hypothesize that patient-centered outcomes can be improved without increasing costs by simply changing the default option for palliative care consultation from opt-in to opt-out for patients with life-limiting illnesses. Patients aged 65 years or older are enrolled at 11 hospitals using an integrated electronic health record. As a pragmatic trial designed to enroll between 12,000 and 15,000 patients, eligibility is determined using a validated, electronic health record-based algorithm, and all outcomes are captured via the electronic health record and billing systems data. The time at which each hospital transitions from control, opt-in palliative care consultation to intervention, opt-out consultation is randomly assigned. The primary outcome is a composite measure of in-hospital mortality and length of stay. Secondary outcomes include palliative care process measures and clinical and economic outcomes. Clinical trial registered with www.clinicaltrials.gov (NCT02505035).

Public Opinion Regarding Financial Incentives to Engage in Advance Care Planning and Complete Advance Directives

Background: The Centers for Medicare & Medicaid Services (CMS) recently instituted physician reimbursements for advance care planning (ACP) discussions with patients. Aim: To measure public support for similar programs. Design: Cross-sectional online and in-person surveys. Setting/Participants: English-speaking adults recruited at public parks in Philadelphia, Pennsylvania, from July to August 2013 and online through survey sampling international Web-based recruitment platform in July 2015. Participants indicated support for 6 programs designed to increase advance directive (AD) completion or ACP discussion using 5-point Likert scales. Participants also indicated how much money (US

No improvement in the reporting of clinical trial subgroup effects in high-impact general medical journals.

0-US

Default options in advance directives: study protocol for a randomised clinical trial

1000) was appropriate to incentivize such behaviors, compared to smoking cessation or colonoscopy screening. Results: We recruited 883 participants: 503 online and 380 in-person. The status quo of no systematic approach to motivate AD completion was supported by 67.0% of participants (63.9%-70.1%). The most popular programs were paying patients to complete ADs (58.0%; 54.5%-61.2%) and requiring patients to complete ADs or declination forms for health insurance (54.1%; 50.8%-57.4%). Participants more commonly supported paying patients to complete ADs than paying physicians whose patients complete ADs (22.6%; 19.8%-25.4%) or paying physicians who document ACP discussions (19.1%; 16.5%-21.7%; both P < .001). Participants supported smaller payments for AD completion and ACP than for obtaining screening colonoscopies or stopping smoking. Conclusions: Americans view payments for AD completion or ACP more skeptically than for other health behaviors and prefer that such payments go to patients rather than physicians. The current CMS policy of reimbursing physicians for ACP conversations with patients was the least preferred of the programs evaluated.

A Randomized Trial of Expanding Choice Sets to Motivate Advance Directive Completion

BackgroundWhen subgroup analyses are not correctly analyzed and reported, incorrect conclusions may be drawn, and inappropriate treatments provided. Despite the increased recognition of the importance of subgroup analysis, little information exists regarding the prevalence, appropriateness, and study characteristics that influence subgroup analysis. The objective of this study is to determine (1) if the use of subgroup analyses and multivariable risk indices has increased, (2) whether statistical methodology has improved over time, and (3) which study characteristics predict subgroup analysis.MethodsWe randomly selected randomized controlled trials (RCTs) from five high-impact general medical journals during three time periods. Data from these articles were abstracted in duplicate using standard forms and a standard protocol. Subgroup analysis was defined as reporting any subgroup effect. Appropriate methods for subgroup analysis included a formal test for heterogeneity or interaction across treatment-by-covariate groups. We used logistic regression to determine the variables significantly associated with any subgroup analysis or, among RCTs reporting subgroup analyses, using appropriate methodology.ResultsThe final sample of 416 articles reported 437 RCTs, of which 270 (62 %) reported subgroup analysis. Among these, 185 (69 %) used appropriate methods to conduct such analyses. Subgroup analysis was reported in 62, 55, and 67 % of the articles from 2007, 2010, and 2013, respectively. The percentage using appropriate methods decreased over the three time points from 77 % in 2007 to 63 % in 2013 (p < 0.05). Significant predictors of reporting subgroup analysis included industry funding (OR 1.94 (95 % CI 1.17, 3.21)), sample size (OR 1.98 per quintile (1.64, 2.40), and a significant primary outcome (OR 0.55 (0.33, 0.92)). The use of appropriate methods to conduct subgroup analysis decreased by year (OR 0.88 (0.76, 1.00)) and was less common with industry funding (OR 0.35 (0.18, 0.70)). Only 33 (18 %) of the RCTs examined subgroup effects using a multivariable risk index.ConclusionsWhile we found no significant increase in the reporting of subgroup analysis over time, our results show a significant decrease in the reporting of subgroup analyses using appropriate methods during recent years. Industry-sponsored trials may more commonly report subgroup analyses, but without utilizing appropriate methods. Suboptimal reporting of subgroup effects may impact optimal physician-patient decision-making.

Six-Month Morbidity and Mortality among Intensive Care Unit Patients Receiving Life-Sustaining Therapy. A Prospective Cohort Study

Introduction Although most seriously ill Americans wish to avoid burdensome and aggressive care at the end of life, such care is often provided unless patients or family members specifically request otherwise. Advance directives (ADs) were created to provide opportunities to set limits on aggressive care near lifes end. This study tests the hypothesis that redesigning ADs such that comfort-oriented care is provided as the default, rather than requiring patients to actively choose it, will promote better patient-centred outcomes. Methods and analysis This multicentre trial randomises seriously ill adults to receive 1 of 3 different ADs: (1) a traditional AD that requires patients to actively choose their goals of care or preferences for specific interventions (eg, feeding tube insertion) or otherwise have their care guided by their surrogates and the prevailing societal default toward aggressive care; (2) an AD that defaults to life-extending care and receipt of life-sustaining interventions, enabling patients to opt out from such care; or (3) an AD that defaults to comfort care, enabling patients to opt into life-extending care. We seek to enrol 270 patients who return complete, legally valid ADs so as to generate sufficient power to detect differences in the primary outcome of hospital-free days (days alive and not in an acute care facility). Secondary outcomes include hospital and intensive care unit admissions, costs of care, hospice usage, decision conflict and satisfaction, quality of life, concordance of preferences with care received and bereavement outcomes for surrogates of patients who die. Ethics and dissemination This study has been approved by the Institutional Review Boards at all trial centres, and is guided by a data safety and monitoring board and an ethics advisory board. Study results will be disseminated using methods that describe the results in ways that key stakeholders can best understand and implement. Trial registration number NCT02017548; Pre-results.