Jennifer L. Paruch

Development and Evaluation of the Universal ACS NSQIP Surgical Risk Calculator: A Decision Aid and Informed Consent Tool for Patients and Surgeons

BACKGROUND Accurately estimating surgical risks is critical for shared decision making and informed consent. The Centers for Medicare and Medicaid Services may soon put forth a measure requiring surgeons to provide patients with patient-specific, empirically derived estimates of postoperative complications. Our objectives were to develop a universal surgical risk estimation tool, to compare performance of the universal vs previous procedure-specific surgical risk calculators, and to allow surgeons to empirically adjust the estimates of risk. STUDY DESIGN Using standardized clinical data from 393 ACS NSQIP hospitals, a web-based tool was developed to allow surgeons to easily enter 21 preoperative factors (demographics, comorbidities, procedure). Regression models were developed to predict 8 outcomes based on the preoperative risk factors. The universal model was compared with procedure-specific models. To incorporate surgeon input, a subjective surgeon adjustment score, allowing risk estimates to vary within the estimates confidence interval, was introduced and tested with 80 surgeons using 10 case scenarios. RESULTS Based on 1,414,006 patients encompassing 1,557 unique CPT codes, a universal surgical risk calculator model was developed that had excellent performance for mortality (c-statistic = 0.944; Brier score = 0.011 [where scores approaching 0 are better]), morbidity (c-statistic = 0.816, Brier score = 0.069), and 6 additional complications (c-statistics > 0.8). Predictions were similarly robust for the universal calculator vs procedure-specific calculators (eg, colorectal). Surgeons demonstrated considerable agreement on the case scenario scoring (80% to 100% agreement), suggesting reliable score assignment between surgeons. CONCLUSIONS The ACS NSQIP surgical risk calculator is a decision-support tool based on reliable multi-institutional clinical data, which can be used to estimate the risks of most operations. The ACS NSQIP surgical risk calculator will allow clinicians and patients to make decisions using empirically derived, patient-specific postoperative risks.

Postoperative complications reduce adjuvant chemotherapy use in resectable pancreatic cancer

Objective:To assess the impact of postoperative complications on the receipt of adjuvant chemotherapy. Background:Randomized trials have demonstrated that adjuvant chemotherapy is associated with improved long-term survival. However, pancreatic surgery is associated with significant morbidity and the degree to which complications limit subsequent treatment options is unknown. Methods:Patients from the American College of Surgeons National Surgical Quality Improvement Program and the National Cancer Data Base who underwent pancreatic resection for cancer were linked (2006–2009). The associations between complications and adjuvant chemotherapy use or treatment delay (≥70 days from surgery) were assessed using multivariable regression methods. Results:From 149 hospitals, 2047 patients underwent resection for stage I-III pancreatic adenocarcinoma of which 23.2% had at least 1 serious complication. Overall adjuvant chemotherapy receipt was 57.7%: 61.8% among patients not experiencing any complication and 43.6% among those who had a serious complication. Serious complications increased the likelihood of not receiving adjuvant therapy over twofold [odds ratio (OR) = 2.20, 95% confidence interval (CI): 1.73–2.80]. Specific complications associated with adjuvant chemotherapy omission were reintubation (OR = 7.79, 95% CI: 3.59–16.87), prolonged ventilation (OR = 5.92, 95% CI: 3.23–10.86), pneumonia (OR = 2.83, 95% CI: 1.63–4.90), sepsis/shock (OR = 2.76, 95% CI: 2.02–3.76), organ space/deep surgical site infection (OR = 2.19, 95% CI: 1.53–3.13), venous thromboembolism (OR = 1.92, 95% CI: 1.08–3.43), and urinary tract infection (OR = 1.61, 95% CI: 1.02–2.54). Serious complications also doubled the likelihood of delaying adjuvant treatment administration (OR = 2.08, 95% CI: 1.42–3.05). Sensitivity analysis in a younger, healthier patient cohort demonstrated similar associations. Conclusions:Postoperative complications are common following pancreatic surgery and are associated with adjuvant chemotherapy omission and treatment delays.

An Opportunity to Improve Informed Consent and Shared Decision Making: The Role of the ACS NSQIP Surgical Risk Calculator in Oncology

Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, IL; Department of Surgery, Pritzker School of Medicine, University of Chicago, Chicago, IL; Department of Surgery, University of California, Los Angeles (UCLA) and VA Greater Los Angeles Healthcare System, Los Angeles, CA; Northwestern Institute for Comparative Effectiveness Research in Oncology, Robert H. Lurie Comprehensive Cancer Center, Feinberg School of Medicine, Northwestern University, Chicago, IL; Surgical Outcomes and Quality Improvement Center, Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, IL

Effect of Cancer Surgery Complexity on Short-Term Outcomes, Risk Predictions, and Hospital Comparisons

BACKGROUND Concern exists that oncologic surgical complexity is not adequately captured by the primary procedure code alone. Our objectives were to characterize the association between secondary procedures and 30-day outcomes, evaluate the effect of surgical complexity on risk predictions, and assess the influence of surgical complexity on hospital-quality comparisons. STUDY DESIGN Patients who underwent colon, rectal, or pancreatic resection for cancer (2007-2011) were identified from the American College of Surgeons NSQIP. Complexity was assessed by creating categorical complexity variables using secondary procedure codes and using total work relative value units. Regression methods were used to evaluate surgical complexity and hospital-quality comparisons. RESULTS Patients had at least one secondary procedure documented in 48.0% of colon, 55.5% of rectal, and 63.1% of pancreatic cases. Surgical complexity variables were associated with worse outcomes across nearly all complications assessed. For example, serious morbidity was increased after an index colon resection with a synchronous liver resection (odds ratio = 1.39; 95% CI, 1.10-1.76) and a pancreatic resection with vascular reconstruction (odds ratio = 1.21; 95% CI, 1.01-1.45). Based on discrimination improvement indices and the likelihood ratio test, model-based predictions were enhanced with the addition of secondary surgical complexity variables, as well as total work relative value units, for nearly all procedures and outcomes assessed. Models that included total work relative value units had similar or marginally better discrimination compared with models with secondary procedure categories. Hospital performance did not change substantially after complexity adjustment. CONCLUSIONS Surgical complexity adjustment is feasible and improves risk estimation of 30-day postoperative outcomes for colon, rectal, and pancreatic resections for cancer. Oncology-specific risk-adjustment models should include complexity adjustment using secondary procedure codes.

Relationship between cancer center accreditation and performance on publicly reported quality measures.

Objective:To evaluate differences in hospital structural quality characteristics and assess the association between national publicly reported quality indicators and cancer center accreditation status. Background:Cancer center accreditation and public reporting are 2 approaches available to help guide patients with cancer to high-quality hospitals. It is unknown whether hospital performance on these measures differs by cancer accreditation. Methods:Data from Medicares Hospital Compare and the American Hospital Association were merged. Hospitals were categorized into 3 mutually exclusive groups: National Cancer Institute–Designated Cancer Centers (NCI-CCs), Commission on Cancer (CoC) centers, and “nonaccredited” hospitals. Performance was assessed on the basis of structural, processes-of-care, patient-reported experiences, costs, and outcomes. Results:A total of 3563 hospitals (56 NCI-CCs, 1112 CoC centers, and 2395 nonaccredited hospitals) were eligible for analysis. Cancer centers (NCI-CCs and CoC centers) were more likely larger, higher volume teaching hospitals with additional services and specialists than nonaccredited hospitals (P < 0.001). Cancer centers performed better on 3 of 4 process measures, 8 of 10 patient-reported experience measures, and Medicare spending per beneficiary than nonaccredited hospitals. NCI-CCs performed worse than both CoC centers and nonaccredited hospitals on 8 of 10 outcome measures. Similarly, CoC centers performed worse than nonaccredited hospitals on 5 measures. For example, 35% of NCI-CCs, 13.5% of CoC centers, and 3.5% of nonaccredited hospitals were poor performers for serious complications. Conclusions:Accredited cancer centers performed better on most process and patient experience measures but showed worse performance on most outcome measures. These discordant findings emphasize the need to focus on oncology-specific measurement strategies.

Adherence with postdischarge venous thromboembolism chemoprophylaxis recommendations after colorectal cancer surgery among elderly Medicare beneficiaries.

Objectives:To assess national adherence with extended venous thromboembolism (VTE) chemoprophylaxis guideline recommendations after colorectal cancer surgery. Background:Postoperative VTE remains a major cause of morbidity and mortality after abdominal cancer surgery. On the basis of the results from randomized controlled trials, since 2007, national guidelines have suggested that these patients be discharged on VTE chemoprophylaxis. Methods:Medicare beneficiaries undergoing open colorectal cancer resections in 2008–2009 were identified using the Medicare Provider Analysis and Review data and limited to those who were enrolled and used Part D for their postoperative prescriptions. Postdischarge use of low-molecular-weight-heparin and other anticoagulants was assessed. Results:A total of 5078 patients underwent open colorectal cancer surgery and met the inclusion criteria. Of these, 77% underwent colectomy and 23% underwent proctectomy. A prescription for an anticoagulant was filled immediately after discharge for 77 (1.5%) patients, and a low-molecular-weight-heparin for 60 (1.2%) patients. On multivariable analysis, patients were more likely to receive postdischarge VTE chemoprophylaxis if undergoing rectal cancer surgery [incidence rate ratio (IRR), 1.83; 95% confidence interval, 1.07–3.12; vs colon], if higher educational status (IRR, 2.20; 95% confidence interval, 1.23–3.95; vs low education), or if they had a higher Elixhauser comorbidity index (IRR, 1.13; 95% confidence interval, 1.01–1.25; vs lower index). Conclusions:Although VTE remains a major issue after abdominal cancer surgery, only 1.5% of Medicare beneficiaries undergoing colorectal cancer surgery received care consistent with established guidelines for postdischarge VTE chemoprophylaxis. Barriers to adherence must be elucidated to improve the quality of care for abdominal and pelvic cancer surgery patients.

Risk of Discharge to Postacute Care A Patient-Centered Outcome for the American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator

IMPORTANCE Individualized risk prediction tools have an important role as decision aids for use by patients and surgeons before surgery. Patient-centered outcomes should be incorporated into such tools to widen their appeal and improve their usability. OBJECTIVE To develop a patient-centered outcome for the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) Surgical Risk Calculator, a web-based, individualized risk prediction tool. DESIGN, SETTING, AND PARTICIPANTS Retrospective cohort study using data from the ACS NSQIP, a national clinical data registry. A total of 973 211 patients from July 2010 to June 2012, encompassing 392 hospitals, were used in this analysis. MAIN OUTCOMES AND MEASURES Risk of discharge to a postacute care setting. RESULTS The overall rate of discharge to postacute care was 8.8%. Significant predictors of discharge to postacute care included being 85 years or older (odds ratio [OR] = 9.17; 95% CI, 8.84-9.50), the presence of septic shock (OR = 2.43; 95% CI, 2.20-2.69) or ventilator dependence (OR = 2.81; 95% CI, 2.56-3.09) preoperatively, American Society of Anesthesiologists class of 4 or 5 (OR = 3.59; 95% CI, 3.46-3.71), and totally dependent functional status (OR = 2.27; 95% CI, 2.11-2.44). The final model predicted risk of discharge to postacute care with excellent accuracy (C statistic = 0.924) and calibration (Brier score = 0.05). CONCLUSIONS AND RELEVANCE Individualized risk of discharge to postacute care can be predicted with excellent accuracy. This outcome will be incorporated into the ACS NSQIP Surgical Risk Calculator.

Comparison of postoperative complication risk prediction approaches based on factors known preoperatively to surgeons versus patients

BACKGROUND Estimating the risk of postoperative complications can be performed by surgeons with detailed clinical information or by patients with limited information. Our objective was to compare three estimation models: (1) the All Information Model, using variables available from the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP); (2) the Surgeon Assessment Model, using variables available to surgeons preoperatively, and (3) the Patient-Entered Model, using information that patients know about their own health. STUDY DESIGN Using the ACS NSQIP 2011 data for general and colon surgery, standard ACS NSQIP regression methods were used to develop models. Each model examined Overall and Serious Morbidity as outcomes. The models were assessed using the c-statistic, Hosmer-Lemshow statistic, and Akaike Information Criterion. RESULTS The overall morbidity rate was 13.0%, and the serious morbidity rate was 10.5% for patients undergoing general surgery (colon surgery: 31.8% and 26.0%, respectively). There was a small decrement in the c-statistic as the number of predictors decreased. The Akaike Information Criterion likelihood ratio increased between the All Information and Surgeon Assessment models, but decreased in the Patient-Entered Model. The Hosmer-Lemshow statistic suggested good model fit for five colon surgery models and one general surgery model. CONCLUSION Although a small decline in model performance was observed, the magnitude suggests that it may not be clinically meaningful as the risk predictions offered are superior to simply providing unadjusted complications rates. The Surgeon Assessment and Patient-Entered models with fewer predictors can be used with relative confidence to predict a patients risk.

Differences in patients, surgical complexity, and outcomes after cancer surgery at National Cancer Institute-designated cancer centers compared to other hospitals.

Background: Interest in comparing hospital surgical quality continues to increase, particularly with respect to examining certain hospital designations such as National Cancer Institute-designated Cancer Centers (NCI-CC). Our objectives were to compare patients, surgical complexity, and risk-adjusted 30-day outcomes following major cancer surgery at NCI-CC versus non-NCI centers. Methods: From the American College of Surgeons National Surgical Quality Improvement Program, patients were identified who underwent colorectal, pancreatic, or esophagogastric resection for cancer (2007–2011). Regression methods were used to evaluate characteristics associated with undergoing treatment at NCI-CCs and surgical-complexity–adjusted 30-day morbidity, mortality, and prolonged length-of-stay at NCI-CC versus non-NCI centers. Results: NCI-CCs performed 20.2% of colorectal (10,555/52,265), 53.5% of pancreatic (6335/11,838), and 49.8% of esophagogastric (1596/3208) operations for cancer. NCI-CCs were more likely to treat patients who were younger, white, and with fewer comorbidities, but were more likely to perform more complex procedures including synchronous liver resection (eg, colorectal), adjacent organ resections (rectal cancer), and vascular reconstructions (eg, pancreas) (all P<0.05). NCI-CCs had a lower mortality rate for colorectal surgery only (1.2% vs. 1.9%) and increased rates of superficial surgical site infection (SSI) for colorectal (9.8% vs. 7.1%) and pancreatic (10.7% vs. 8.8%) surgery. No differences existed for the remaining complications by NCI-CC designation status. NCI-CCs were distributed throughout hospital quality rankings for all procedures and complications assessed. Conclusions: NCI-CCs treated younger, healthier patients, but performed more complex procedures. Patients treated at NCI-CCs had a lower risk of mortality for colorectal resection, but morbidity was similar to non-NCI centers. Comparison of cancer surgery hospital quality is feasible and should adjust for differences in patient demographics, comorbidities, and surgical complexity.

Comparison of National Operative Mortality in Gastroenterological Surgery Using Web-based Prospective Data Entry Systems

AbstractInternational collaboration is important in healthcare quality evaluation; however, few international comparisons of general surgery outcomes have been accomplished. Furthermore, predictive model application for risk stratification has not been internationally evaluated. The National Clinical Database (NCD) in Japan was developed in collaboration with the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP), with a goal of creating a standardized surgery database for quality improvement. The study aimed to compare the consistency and impact of risk factors of 3 major gastroenterological surgical procedures in Japan and the United States (US) using web-based prospective data entry systems: right hemicolectomy (RH), low anterior resection (LAR), and pancreaticoduodenectomy (PD).Data from NCD and ACS-NSQIP, collected over 2 years, were examined. Logistic regression models were used for predicting 30-day mortality for both countries. Models were exchanged and evaluated to determine whether the models built for one population were accurate for the other population.We obtained data for 113,980 patients; 50,501 (Japan: 34,638; US: 15,863), 42,770 (Japan: 35,445; US: 7325), and 20,709 (Japan: 15,527; US: 5182) underwent RH, LAR, and, PD, respectively. Thirty-day mortality rates for RH were 0.76% (Japan) and 1.88% (US); rates for LAR were 0.43% versus 1.08%; and rates for PD were 1.35% versus 2.57%. Patient background, comorbidities, and practice style were different between Japan and the US. In the models, the odds ratio for each variable was similar between NCD and ACS-NSQIP. Local risk models could predict mortality using local data, but could not accurately predict mortality using data from other countries.We demonstrated the feasibility and efficacy of the international collaborative research between Japan and the US, but found that local risk models remain essential for quality improvement.