Morgan M. Sellers

Validation of new readmission data in the American College of Surgeons National Surgical Quality Improvement Program.

BACKGROUND Hospital readmissions are gathering increasing attention as a measure of health care quality and as a cost-saving target. The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) recently began collecting data related to 30-day postoperative readmissions. Our objectives were to assess the accuracy of the ACS NSQIP readmission variable by comparison with the medical record, and to evaluate the readmission variable against administrative data. STUDY DESIGN Readmission data captured in ACS NSQIP at a single academic institution between January and December 2011 were compared with data abstracted from the medical record and administrative data. RESULTS Of 1,748 cases captured in ACS NSQIP, 119 (6.8%) had an all-cause readmission event identified, and ACS NSQIP had very high agreement with chart review for identifying all-cause readmission events (κ = 0.98). For 1,110 inpatient cases successfully matched with administrative data, agreement with chart review for identifying all-cause readmissions was also very high (κ = 0.97). For identifying unplanned readmission events, ACS NSQIP had good agreement with chart review (κ = 0.67). Overall, agreement with chart review on cause of readmission was higher for ACS NSQIP (κ = 0.75) than for administrative data (κ = 0.46). CONCLUSIONS The ACS NSQIP accurately captured all-cause and unplanned readmission events and had good agreement with the medical record with respect to cause of readmission. Administrative data accurately captured all-cause readmissions, but could not identify unplanned readmissions and less consistently agreed with chart review on cause. The granularity of clinically collected data offers tremendous advantages for directing future quality efforts targeting surgical readmission.

American College of Surgeons NSQIP: quality in-training initiative pilot study.

BACKGROUND Clinical outcomes data are playing an increasingly important role in medical decision-making, reimbursement, and provider evaluation, but there are no documented programs that provide outcomes data to surgical residents as part of a structured curriculum. Our objectives were to develop a national collaborative of training programs to unify the efforts between quality and education personnel and demonstrate the feasibility of generating customized reports of patient outcomes for use in surgical education. STUDY DESIGN The pool of potential hospitals was evaluated by comparing ACS NSQIP participants with the roster of clinical sites for general surgery residency programs maintained by FREIDA Online. A program and user guide was developed to generate custom reports based on institutional data, and a voluntary pilot was conducted, consisting of initial development, implementation, and feedback stages. Programs that successfully completed installation and report generation were queried for feedback on time and resources used. RESULTS Of 245 general surgery residency programs, 47% had a NSQIP-affiliated sponsor institution, and an additional 31% had at least 1 NSQIP-affiliated participant institution. Sixty general surgery residency programs have expressed interest in collaboration. Seventeen pilot sites completed training and installation, and were able to independently generate custom reports. The response rate for the post-report survey was 50%. Participants reported that training and installation typically required one 2-hour phone call, and that total time devoted to the project was less than 8 hours. CONCLUSIONS Collaboration between educators and quality improvement personnel from a diverse group of organizations to integrate outcomes data into surgical education is feasible. Obtaining resident and team reports from ACS NSQIP can be done with minimal effort. Future efforts will be aimed at developing a national data-centered curriculum for general surgery programs.

Quality In-Training Initiative—A Solution to the Need for Education in Quality Improvement: Results from a Survey of Program Directors

BACKGROUND The Next Accreditation System and the Clinical Learning Environment Review Program will emphasize practice-based learning and improvement and systems-based practice. We present the results of a survey of general surgery program directors to characterize the current state of quality improvement in graduate surgical education and introduce the Quality In-Training Initiative (QITI). STUDY DESIGN In 2012, a 20-item survey was distributed to 118 surgical residency program directors from ACS NSQIP-affiliated hospitals. The survey content was developed in collaboration with the QITI to identify program director opinions regarding education in practice-based learning and improvement and systems-based practice, to investigate the status of quality improvement education in their respective programs, and to quantify the extent of resident participation in quality improvement. RESULTS There was a 57% response rate. Eighty-five percent of program directors (n = 57) reported that education in quality improvement is essential to future professional work in the field of surgery. Only 28% (n = 18) of programs reported that at least 50% of their residents track and analyze their patient outcomes, compare them with norms/benchmarks/published standards, and identify opportunities to make practice improvements. CONCLUSIONS Program directors recognize the importance of quality improvement efforts in surgical practice. Subpar participation in basic practice-based learning and improvement activities at the resident level reflects the need for support of these educational goals. The QITI will facilitate programmatic compliance with goals for quality improvement education.

The unmet need for a national surgical quality improvement curriculum: a systematic review.

INTRODUCTION The Accreditation Council for Graduate Medical Education Next Accreditation System will require general surgery training programs to demonstrate outstanding clinical outcomes and education in quality improvement (QI). The American College of Surgeons-National Surgical Quality Improvement Project Quality In-Training Initiative reports the results of a systematic review of the literature investigating the availability of a QI curriculum. METHODS Using defined search terms, a systematic review was conducted in Embase, PubMed, and Google Scholar (January 2000-March 2013) to identify a surgical QI curriculum. Bibliographies from selected articles and other relevant materials were also hand searched. Curriculum was defined as an organized program of learning complete with content, instruction, and assessment for use in general surgical residency programs. Two independent observers graded surgical articles on quality of curriculum presented. RESULTS Overall, 50 of 1155 references had information regarding QI in graduate medical education. Most (n = 24, 48%) described QI education efforts in nonsurgical fields. A total of 31 curricular blueprints were identified; 6 (19.4%) were specific to surgery. Targeted learners were most often post graduate year-2 residents (29.0%); only 6 curricula (19.4%) outlined a course for all residents within their respective programs. Plan, Do, Study, Act (n = 10, 32.3%), and Root Cause Analysis (n = 5, 16.1%) were the most common QI content presented, the majority of instruction was via lecture/didactics (n = 26, 83.9%), and only 7 (22.6%) curricula used validated tool kits for assessment. CONCLUSION Elements of QI curriculum for surgical education exist; however, comprehensive content is lacking. The American College of Surgeons-National Surgical Quality Improvement Project Quality In-Training Initiative will build on the high-quality components identified in our review and develop data-centered QI content to generate a comprehensive national QI curriculum for use in graduate surgical education.

Adherence with postdischarge venous thromboembolism chemoprophylaxis recommendations after colorectal cancer surgery among elderly Medicare beneficiaries.

Objectives:To assess national adherence with extended venous thromboembolism (VTE) chemoprophylaxis guideline recommendations after colorectal cancer surgery. Background:Postoperative VTE remains a major cause of morbidity and mortality after abdominal cancer surgery. On the basis of the results from randomized controlled trials, since 2007, national guidelines have suggested that these patients be discharged on VTE chemoprophylaxis. Methods:Medicare beneficiaries undergoing open colorectal cancer resections in 2008–2009 were identified using the Medicare Provider Analysis and Review data and limited to those who were enrolled and used Part D for their postoperative prescriptions. Postdischarge use of low-molecular-weight-heparin and other anticoagulants was assessed. Results:A total of 5078 patients underwent open colorectal cancer surgery and met the inclusion criteria. Of these, 77% underwent colectomy and 23% underwent proctectomy. A prescription for an anticoagulant was filled immediately after discharge for 77 (1.5%) patients, and a low-molecular-weight-heparin for 60 (1.2%) patients. On multivariable analysis, patients were more likely to receive postdischarge VTE chemoprophylaxis if undergoing rectal cancer surgery [incidence rate ratio (IRR), 1.83; 95% confidence interval, 1.07–3.12; vs colon], if higher educational status (IRR, 2.20; 95% confidence interval, 1.23–3.95; vs low education), or if they had a higher Elixhauser comorbidity index (IRR, 1.13; 95% confidence interval, 1.01–1.25; vs lower index). Conclusions:Although VTE remains a major issue after abdominal cancer surgery, only 1.5% of Medicare beneficiaries undergoing colorectal cancer surgery received care consistent with established guidelines for postdischarge VTE chemoprophylaxis. Barriers to adherence must be elucidated to improve the quality of care for abdominal and pelvic cancer surgery patients.

Development and Participant Assessment of a Practical Quality Improvement Educational Initiative for Surgical Residents

BACKGROUND As patient-safety and quality efforts spread throughout health care, the need for physician involvement is critical, yet structured training programs during surgical residency are still uncommon. Our objective was to develop an extended quality-improvement curriculum for surgical residents that included formal didactics and structured practical experience. METHODS Surgical trainees completed an 8-hour didactic program in quality-improvement methodology at the start of PGY3. Small teams developed practical quality-improvement projects based on needs identified during clinical experience. With the assistance of the hospitals process-improvement team and surgical faculty, residents worked through their selected projects during the following year. Residents were anonymously surveyed after their participation to assess the experience. RESULTS During the first 3 years of the program, 17 residents participated, with 100% survey completion. Seven quality-improvement projects were developed, with 57% completing all DMAIC (Define, Measure, Analyze, Improve, Control) phases. Initial projects involved issues of clinical efficiency and later projects increasingly focused on clinical care questions. Residents found the experience educationally important (65%) and believed they were well equipped to lead similar initiatives in the future (70%). Based on feedback, the timeline was expanded from 12 to 24 months and changed to start in PGY2. CONCLUSIONS Developing an extended curriculum using both didactic sessions and applied projects to teach residents the theory and implementation of quality improvement is possible and effective. It addresses the ACGME competencies of practice-based improvement and learning and systems-based practice. Our iterative experience during the past 3 years can serve as a guide for other programs.

The Quality In-Training Initiative: Giving Residents Data to Learn Clinical Effectiveness.

BACKGROUND Training programs are expected to provide clinical outcomes data to residents. Few systems have the necessary infrastructure. We evaluated initial adoption and use of the Quality In-Training Initiative (QITI) platform linking National Surgical Quality Improvement Program (NSQIP) data to trainees. STUDY DESIGN Proportions of Accreditation Council for Graduate Medical Education general surgery residency programs with differing levels of NSQIP and QITI affiliation were calculated and program characteristics were compared. All NSQIP sites that captured QITI custom field data from July 2013 to June 2016 were included in case analysis. Differences in case collection were compared between participating (P) sites that actively participated in QITI and nonparticipating (NP) sites that did not. Resident participation by procedure type was examined. RESULTS Of 268 accredited general surgery residency programs, 92% (n = 248) is affiliated with a NSQIP hospital and 61% of all clinical months is spent at NSQIP sites. For 42% of all programs (n = 114), the primary teaching hospital is affiliated with the QITI. In all, 74 P sites and 89 NP sites captured a total of 417,816 cases. The median number of cases captured per site was statistically higher for P sites (3063) compared with NP sites (2307, p < 0.001). A total of 68.3% of all cases captured had resident participation indicated by postgraduate year (n = 285,469). The most common procedures with resident participation were laparoscopic appendectomy (n = 17,082, 6.0%) and laparoscopic cholecystectomy (n = 15,502, 5.4%). Percentage coverage rates ranged from 17.3% to 91.8%. CONCLUSION Most general surgery rotations are at NSQIP sites. Identifying resident participation in captured NSQIP cases is feasible on a large scale. Captured cases reflect national case-mix. The platform has the potential to collect data on institutional and program-level variation in resident operative experience that may be used to improve training.

Using Patient Safety Reporting Systems to Understand the Clinical Learning Environment: A Content Analysis

OBJECTIVE To examine patient safety event reporting behavior by trainees caring for surgical patients compared to other clinicians. DESIGN Qualitative analysis of a patient safety event reporting system comparing reports entered by trainees to those entered by attending physicians and nurses. Categorical data associated with reports were compared, and free-text event descriptions underwent content analysis focusing on themes related to report completeness and report focus. SETTING The Hospital of the University of Pennsylvania, an academic tertiary care hospital in Philadelphia, Pennsylvania. PARTICIPANTS All patient safety event reports related to surgical patients from a 6-month period (July-December 2016). RESULTS One thousand four hundred twenty-three reports were entered by trainees (T), attendings (A), and nurses (N). Trainees had a lower number of reports entered per reporter compared to nurses (T median [IQR]: 1 [1-2], N: 2 [1-3]), and the highest percentage of reports entered anonymously for any group (T: 28.7%, N: 9.9%, A: 4.6%). The overall distribution of event location and event type differed significantly between groups (p < 0.001). Trainee reports were found to have a broader range of focus, more elements associated with completeness of reports, and more frequent use of blame language. CONCLUSIONS Surgical trainees report a wide variety of issues in the perioperative, floor, and ICU settings. Their reports often include more details than those entered by other clinicians, but feature higher rates of anonymous reporting and use of blame language. Analysis of patient safety event reports by trainees compared with other healthcare professionals can reveal important insights into the clinical learning environment and areas for safety improvement.

An Instrumental Variables Design for the Effect of Emergency General Surgery

Abstract Confounding by indication is a critical challenge in evaluating the effectiveness of surgical interventions using observational data. The threat from confounding is compounded when using medical claims data due to the inability to measure risk severity. If there are unobserved differences in risk severity across patients, treatment effect estimates based on methods such a multivariate regression may be biased in an unknown direction. A research design based on instrumental variables offers one possibility for reducing bias from unobserved confounding compared to risk adjustment with observed confounders. This study investigates whether a physician’s preference for operative care is a valid instrumental variable for studying the effect of emergency surgery. We review the plausibility of the necessary causal assumptions in an investigation of the effect of emergency general surgery (EGS) on inpatient mortality among adults using medical claims data from Florida, Pennsylvania, and New York in 2012–2013. In a departure from the extant literature, we use the framework of stochastic monotonicity which is more plausible in the context of a preference-based instrument. We compare estimates from an instrumental variables design to estimates from a design based on matching that assumes all confounders are observed. Estimates from matching show lower mortality rates for patients that undergo EGS compared to estimates based in the instrumental variables framework. Results vary substantially by condition type. We also present sensitivity analyses as well as bounds for the population level average treatment effect. We conclude with a discussion of the interpretation of estimates from both approaches.