John G. Demakis

Risk adjustment of the postoperative morbidity rate for the comparative assessment of the quality of surgical care : Results of the National Veterans Affairs Surgical Risk Study

BACKGROUND The National Veterans Affairs Surgical Risk Study was designed to collect reliable, valid data on patient risk and outcomes for major surgery in the Veterans Health Administration and to report comparative risk-adjusted postoperative mortality rates for surgical services in Veterans Health Administration. STUDY DESIGN This cohort study was conducted in 44 Veterans Affairs Medical Centers. Included were 87,078 major noncardiac operations performed under general, spinal, or epidural anesthesia between October 1, 1991, and December 31, 1993. The main outcomes measure was all-cause mortality within 30 days after the index procedure. Multivariable logistic regression risk-adjustment models for all operations and for eight surgical subspecialties were developed. Risk-adjusted surgical mortality rates were expressed as observed-to-expected ratios and were compared with unadjusted 30-day postoperative mortality rates. RESULTS Patient risk factors predictive of postoperative mortality included serum albumin level, American Society of Anesthesia class, emergency operation, and 31 additional preoperative variables. Considerable variability in unadjusted mortality rates for all operations was observed across the 44 hospitals (1.2-5.4%). After risk adjustment, observed-to-expected ratios ranged from 0.49 to 1.53. Rank order correlation of the hospitals by unadjusted and risk-adjusted mortality rates for all operations was 0.64. Ninety-three percent of the hospitals changed rank after risk adjustment, 50% by more than 5 and 25% by more than 10. CONCLUSIONS The Department of Veterans Affairs has successfully implemented a system for the prospective collection and comparative reporting of risk-adjusted postoperative mortality rates after major noncardiac operations. Risk adjustment had an appreciable impact on the rank ordering of the hospitals and provided a means for monitoring and potentially improving the quality of surgical care.

Relation of Surgical Volume to Outcome in Eight Common Operations : Results From the VA National Surgical Quality Improvement Program

OBJECTIVE To examine, in the Veterans Health Administration (VHA), the relation between surgical volume and outcome in eight commonly performed operations of intermediate complexity. SUMMARY BACKGROUND DATA In multihospital health care systems such as VHA, consideration is often given to closing low-volume surgical services, with the assumption that better surgical outcomes are achieved in hospitals with larger surgical volumes. Literature data to support this assumption in intermediate-complexity operations are either limited or controversial. METHODS The VHA National Surgical Quality Improvement Program data on nonruptured abdominal aortic aneurysmectomy, vascular infrainguinal reconstruction, carotid endarterectomy (CEA), lung lobectomy/pneumonectomy, open and laparoscopic cholecystectomy, partial colectomy, and total hip arthroplasty were used. Pearson correlation, analysis of variance, mixed effects hierarchical logistic regression, and automatic interaction detection analysis were used to assess the association of annual procedure/specialty volume with risk-adjusted 30-day death (and stroke in CEA). RESULTS Eight major surgical procedures (68,631 operations) were analyzed. No statistically significant associations between procedure or specialty volume and 30-day mortality rate (or 30-day stroke rate in CEA) were found. CONCLUSIONS In VHA hospitals, the procedure and surgical specialty volume in eight prevalent operations of intermediate complexity are not associated with risk-adjusted 30-day mortality rate from these operations, or with the risk-adjusted 30-day stroke rate from CEA. Volume of surgery in these operations should not be used as a surrogate for quality of surgical care.

Identifying Patient preoperative risk factors and postoperative adverse events in administrative databases: Results from the Department of Veterans Affairs National Surgical Quality Improvement program

BACKGROUND The Department of Veterans Affairs (DVA) National Surgical Quality Improvement Program (NSQIP) employs trained nurse data collectors to prospectively gather preoperative patient characteristics and 30-day postoperative outcomes for most major operations in 123 DVA hospitals to provide risk-adjusted outcomes to centers as quality indicators. It has been suggested that routine hospital discharge abstracts contain the same information and would provide accurate and complete data at much lower cost. STUDY DESIGN With preoperative risks and 30-day outcomes recorded by trained data collectors as criteria standards, ICD-9-CM hospital discharge diagnosis codes in the Patient Treatment File (PTF) were tested for sensitivity and positive predictive value. ICD-9-CM codes for 61 preoperative patient characteristics and 21 postoperative adverse events were identified. RESULTS Moderately good ICD-9-CM matches of descriptions were found for 37 NSQIP preoperative patient characteristics (61%); good data were available from other automated sources for another 15 (25%). ICD-9-CM coding was available for only 13 (45%) of the top 29 predictor variables. In only three (23%) was sensitivity and in only four (31%) was positive predictive value greater than 0.500. There were ICD-9-CM matches for all 21 NSQIP postoperative adverse events; multiple matches were appropriate for most. Postoperative occurrence was implied in only 41%; same breadth of clinical description in only 23%. In only four (7%) was sensitivity and only two (4%) was positive predictive value greater than 0.500. CONCLUSION Sensitivity and positive predictive value of administrative data in comparison to NSQIP data were poor. We cannot recommend substitution of administrative data for NSQIP data methods.

Reinventing VA health care: Systematizing quality improvement and quality innovation

The Veterans Health Administration (VHA) in the US Department of Veterans Affairs (VA) manages the largest fully integrated health care system in the United States. In 1995, the VHA initiated a reinvention effort that included the most radical redesign of VA health care to occur since the veterans health care system was formally established in 1946. The 2 paramount goals of this reinvention effort were to ensure the predictable and consistent provision of high-quality care everywhere in the system and to optimize the value of VA health care. Although still a work in progress, dramatic results have been achieved toward these ends during the past 5 years. This article provides an overview of the veterans health care system, and it highlights selected aspects of the systems reengineering. It also describes various steps that have been taken to better manage performance and to systematize quality improvement and quality innovation. This information provides a global context that should facilitate understanding of the genesis and purposes of the Quality Enhancement Research Initiative that is described in other articles in this issue of Medical Care.

Comparison of Surgical Outcomes Between Teaching and Nonteaching Hospitals in the Department of Veterans Affairs

ObjectiveTo determine whether the investment in postgraduate education and training places patients at risk for worse outcomes and higher costs than if medical and surgical care was delivered in nonteaching settings. Summary Background DataThe Veterans Health Administration (VA) plays a major role in the training of medical students, residents, and fellows. MethodsThe database of the VA National Surgical Quality Improvement Program was analyzed for all major noncardiac operations performed during fiscal years 1997, 1998, and 1999. Teaching status of a hospital was determined on the basis of a background and structure questionnaire that was independently verified by a research fellow. Stepwise logistic regression was used to construct separate models predictive of 30-day mortality and morbidity for each of seven surgical specialties and eight operations. Based on these models, a severity index for each patient was calculated. Hierarchical logistic regression models were then created to examine the relationship between teaching versus nonteaching hospitals and 30-day postoperative mortality and morbidity, after adjusting for patient severity. ResultsTeaching hospitals performed 81% of the total surgical workload and 90% of the major surgery workload. In most specialties in teaching hospitals, the residents were the primary surgeons in more than 90% of the operations. Compared with nonteaching hospitals, the patient populations in teaching hospitals had a higher prevalence of risk factors, underwent more complex operations, and had longer operation times. Risk-adjusted mortality rates were not different between the teaching and nonteaching hospitals in the specialties and operations studied. The unadjusted complication rate was higher in teaching hospitals in six of seven specialties and four of eight operations. Risk adjustment did not eliminate completely these differences, probably reflecting the relatively poor predictive validity of some of the risk adjustment models for morbidity. Length of stay after major operations was not consistently different between teaching and nonteaching hospitals. ConclusionCompared with nonteaching hospitals, teaching hospitals in the VA perform the majority of complex and high-risk major procedures, with comparable risk-adjusted 30-day mortality rates. Risk-adjusted 30-day morbidity rates in teaching hospitals are higher in some specialties and operations than in nonteaching hospitals. Although this may reflect the weak predictive validity of some of the risk adjustment models for morbidity, it may also represent suboptimal processes and structures of care that are unique to teaching hospitals. Despite good quality of care in teaching hospitals, as evidenced by the 30-day mortality data, efforts should be made to examine further the structures and processes of surgical care prevailing in these hospitals.

Quality Enhancement Research Initiative (QUERI): A collaboration between research and clinical practice

This article provides an overview of the Quality Enhancement Research Initiative (QUERI), an ambitious attempt to develop a data-driven national quality-improvement program for the Veterans Health Administration (VHA) that is fully integrated within VHAs Strategic Framework for Quality Management, as discussed elsewhere in this supplement. QUERI is designed to ensure the systematic translation of findings and products (quality tools that promote use of research findings) to promote optimal patient outcomes and system-wide improvements. In developing QUERI, a framework was created to integrate structural elements (organizational characteristics) and process considerations (those actions and action sequences associated with positive change) with outcomes (both at the patient level and at the systems level). In developing this framework, a process for translation of evidence into action was born. The QUERI process depends on having or discovering accurate information about what services are needed, who needs them, how they should be provided, and relevant outcomes and costs. This article describes the 6-step QUERI process and presents an overview of relevant programmatic details, including QUERIs rigorous review process, and VHAs unique qualifications for establishing a national model for quality improvement.

An organizational framework and strategic implementation for system-level change to enhance research-based practice: QUERI Series

BackgroundThe continuing gap between available evidence and current practice in health care reinforces the need for more effective solutions, in particular related to organizational context. Considerable advances have been made within the U.S. Veterans Health Administration (VA) in systematically implementing evidence into practice. These advances have been achieved through a system-level program focused on collaboration and partnerships among policy makers, clinicians, and researchers.The Quality Enhancement Research Initiative (QUERI) was created to generate research-driven initiatives that directly enhance health care quality within the VA and, simultaneously, contribute to the field of implementation science. This paradigm-shifting effort provided a natural laboratory for exploring organizational change processes. This article describes the underlying change framework and implementation strategy used to operationalize QUERI.Strategic approach to organizational changeQUERI used an evidence-based organizational framework focused on three contextual elements: 1) cultural norms and values, in this case related to the role of health services researchers in evidence-based quality improvement; 2) capacity, in this case among researchers and key partners to engage in implementation research; 3) and supportive infrastructures to reinforce expectations for change and to sustain new behaviors as part of the norm. As part of a QUERI Series in Implementation Science, this article describes the frameworks application in an innovative integration of health services research, policy, and clinical care delivery.ConclusionQUERIs experience and success provide a case study in organizational change. It demonstrates that progress requires a strategic, systems-based effort. QUERIs evidence-based initiative involved a deliberate cultural shift, requiring ongoing commitment in multiple forms and at multiple levels. VAs commitment to QUERI came in the form of visionary leadership, targeted allocation of resources, infrastructure refinements, innovative peer review and study methods, and direct involvement of key stakeholders. Stakeholders included both those providing and managing clinical care, as well as those producing relevant evidence within the health care system. The organizational framework and related implementation interventions used to achieve contextual change resulted in engaged investigators and enhanced uptake of research knowledge. QUERIs approach and progress provide working hypotheses for others pursuing similar system-wide efforts to routinely achieve evidence-based care.

Overview of the Veterans Health Administration (VHA) Quality Enhancement Research Initiative (QUERI)

The U.S. Veterans Health Administration (VHA)s Quality Enhancement Research Initiative (QUERI) is an innovative integration of health services research, policy, and clinical care delivery designed to improve the quality, outcomes, and efficiency of VHA health care through the identification and implementation of evidence-based practices in routine care settings. A total of eight condition-specific QUERI centers are currently in operation, each pursuing an integrated portfolio of activities designed to identify and correct gaps in clinical quality and performance and to derive generalizable scientific knowledge regarding quality improvement processes and methods and their effectiveness. This overview article describes QUERIs mission, history, structure, and activities and provides a brief summary of key findings and impacts.

Cooperative Studies in Health Services Research in the Department of Veterans Affairs

The Department of Veterans Affairs, through its Cooperative Studies Program, has a long history of conducting large-scale, multihospital biomedical clinical trials. The agencys Health Services Research and Development Service, although newer, has a distinguished record of mainly single-site research into the organization, delivery, and financing of health services. In 1990, a joint program was initiated to conduct multicenter studies in health services research. This article describes the studies developed in the new program and the research design issues encountered in planning them. Identification of the patient population, specification and measurement of the intervention, and description of the control group, as well as attention to the unit of randomization and analysis, outcome variables and choice of effect size, data quality, and ethical considerations are among the important issues related to the design of these studies and future studies in health services.

Risk Adjustment of the Postoperative Morbidity Rate for the Comparative Assessment of the Quality of Surgical Care: Results of the National Veterans Affairs Surgical Risk Study

BACKGROUND The National Veterans Affairs Surgical Risk Study was designed to collect reliable, valid data on patient risk and outcomes for major surgery in the Veterans Health Administration and to report comparative risk-adjusted postoperative mortality and morbidity rates for surgical services in the Veterans Health Administration. STUDY DESIGN This was a cohort study conducted at 44 Veterans Affairs Medical Centers closely affiliated with university medical centers. Included were 87,078 major noncardiac operations performed under general, spinal, or epidural anesthesia between October 1, 1991, and December 31, 1993. The main outcomes measures in this report are 21 postoperative adverse events (morbidities) occurring within 30 days after the index procedure. Multivariable logistic regression risk-adjustment models for all operations and for eight surgical subspecialties were developed. RESULTS Patient risk factors predictive of postoperative morbidity included serum albumin level, American Society of Anesthesia class, the complexity of the operation, and 17 other preoperative risk variables. Wide variation in the unadjusted rates of one or more postoperative morbidities for all operations was observed across the 44 hospitals (7.4-28.4%). Risk-adjusted observed-to-expected ratios ranged from 0.49 to 1.46. The Spearman rank order correlation between the ranking of the hospitals based on unadjusted morbidity rates and risk-adjusted observed-to-expected ratios for all operations was 0.87. There was little or no correlation between the rank order of the hospitals by risk-adjusted morbidity and risk-adjusted mortality. CONCLUSIONS The Department of Veterans Affairs has successfully implemented a system for the prospective collection and comparative reporting of postoperative mortality and morbidity rates after major noncardiac operations. Risk adjustment had only a modest effect on the rank order of the hospitals.