Julie A. Rainwater

Can administrative data be used to compare postoperative complication rates across hospitals

Background. Several quality assessment systems use administrative data to identify postoperative complications, with uncertain validity. Objectives. To determine how accurately postoperative complications are reported in administrative data, whether accuracy varies systematically across hospitals, and whether serious complications are more consistently reported. Design. Retrospective cohort. Subjects. Nine hundred ninety-one randomly sampled adults who underwent elective lumbar diskectomies at 30 nonfederal acute care hospitals in California in 1990 to 1991. Hospitals with especially low or high risk-adjusted complication rates, and patients who experienced complications, were over sampled. Measures. Postoperative complications were specified by reviewing medical literature and consulting clinical experts; each complication was mapped to ICD-9-CM. Hospital-reported complications were compared with our independent recoding of the same records. Results. The weighted sensitivity, specificity, and positive and negative predictive values for reported complications were 35%, 98%, 82%, and 84%, respectively. The weighted sensitivity was 30% for serious, 40% for minor, and 10% for questionable complications. It varied from 21% among hospitals with fewer complications than expected to 45% among hospitals with more complications than expected. Only reoperation, bacteremia/sepsis, postoperative infection, and deep vein thrombosis were reported with at least 60% sensitivity. Half of the difference in risk-adjusted complication rates between low and high outlier hospitals was attributable to reporting variation. Conclusions. ICD-9-CM complications were underreported among diskectomy patients, especially at hospitals with low risk-adjusted complication rates. The validity of using coded complications to compare provider performance is questionable, even with careful efforts to identify serious events, although these results must be confirmed using more recent data.

Can Administrative Data Be Used to Ascertain Clinically Significant Postoperative Complications

The purpose of this study is to assess whether postoperative complications can be ascertained using administrative data. We randomly sampled 991 adults who underwent elective open diskectomies at 30 nonfederal acute care hospitals in California. Postoperative complications were specified by reviewing medical literature and by consulting clinical experts. We compared hospital-reported ICD-9-CM data and independently recoded ICD-9-CM data with complications abstracted by clinicians using detailed criteria. Recoded ICD-9-CM data were more likely than hospital-reported ICD-9-CM data to capture true complications, when they occurred, but they also mislabeled more patients who never experienced clinically significant complications. This finding was most evident for mild or ambiguous complications, such as atelectasis, posthemorrhagic anemia, and hypotension. Overall, recoded ICD-9-CM data captured 47% and 56% of all mild and severe complications, respectively, whereas hospital reported ICD-9-CM data captured only 37% and 44%, respectively, of all mild and severe complications. These findings raise questions about the validity of using administrative data to ascertain postoperative complications, even if coders are carefully hired, trained, and supervised. ICD-9-CM complication codes are more promising as a tool to help providers identify their own adverse outcomes than as a tool for comparing performance.

Elective cervical discectomy in California : Postoperative in-hospital complications and their risk factors

Study Design. A retrospective cohort study of short‐term outcomes after elective cervical discectomy in California hospitals. Objectives. To compare the frequency of elective cervical discectomy across population strata, to determine the frequency of adverse outcomes in the early postoperative period, and to identify risk factors for such outcomes. Summary of Background Data. Previous cervical discectomy series have been too small to analyze risk factors for early complications, and have originated from centers that may not adequately represent the population. Methods. Computerized hospital discharge abstracts were obtained from the California Office of Statewide Health Planning and Development. Inclusion and exclusion criteria were applied to identify 10,416 routine discectomies at 257 hospitals in 1990‐1991. Several categories of postoperative complications were identified, along with inpatient deaths, early reoperations, and nursing home transfers. Logistic regression was used to estimate the independent effects of patient characteristics on short‐term outcomes. Results. After adjustment for age and gender, blacks were 51% and Hispanics were 24% as likely as whites to undergo elective cervical discectomy. Overall, 6.7% of patients had one or more reported postoperative complications: 1.8% had noninfectious surgical complications, 1.8% had infectious complications, 4.0% had other medical complications, and 0.35% had unplanned reoperations before discharge. Fourteen inpatient deaths were reported (0.13%). Congestive heart failure, alcohol/drug abuse, chronic lung disease, previous spine surgery, psychological disorders, and chronic musculoskeletal disorders were independently associated with postoperative complications. Even after adjustment, risk was higher with advancing age, higher among women than among men, and higher after posterior fusion than after discectomy without fusion. Conclusions. The ethnic disparity in cervical discectomy rates suggests overuse among whites or underuse among minority populations. The complication rates reported here are similar to those synthesized from previous literature, except that the lower incidence of neurologic complications reflects our inability to distinguish preoperative from postoperative deficits. Important comorbidities should be identified and treated, if appropriate, before cervical spine surgery.

Grading the graders: How hospitals in California and New York perceive and interpret their report cards

BACKGROUND Concerns about quality of care are increasing as hospitals struggle to lower costs. Hospital report cards are controversial, but little is known about their impact. OBJECTIVES To determine whether recent hospital report cards are viewed more favorably than pioneering federal efforts; whether a report based on clinical data is viewed more favorably than one based on administrative data; and whether attitudes toward report cards are related to hospital characteristics. DESIGN Mailed survey of chief executives at 374 California hospitals and 31 New York hospitals listed in report cards on myocardial infarction and coronary bypass mortality. SUBJECTS Two-hundred-and-seventy-four hospitals in California (73.3% response) and 27 in New York (87.1% response). California hospitals were categorized on ownership, size, occupancy, risk-adjusted mortality, teaching status, patient volume, and surgical capability. MEASURES Number of hospital units that received or discussed the report card, ratings of its quality, perceptions of its usefulness, and knowledge of its methods. RESULTS In both states, report cards were widely disseminated within hospitals. The mean quality rating was higher (P = 0.0074) in New York than in California; New York respondents appeared to be more knowledgeable about key methods. One or more hospital characteristics was associated with each outcome measure. Leaders at high-mortality hospitals were especially critical and did not find the report useful, despite limited understanding of its methods. CONCLUSIONS Recent hospital report cards were rated better than pioneering federal efforts. A report based on clinical data was rated better, understood better, and disseminated more often to key staff than one that was based on administrative data. Barriers to constructive use of outcomes data persist, especially at high mortality hospitals.

Does Applicant Personality Influence Multiple Mini-Interview Performance and Medical School Acceptance Offers?

Purpose To examine relationships among applicant personality, Multiple Mini-Interview (MMI) performance, and medical school acceptance offers. Method The authors conducted an observational study of applicants who participated in the MMI at the University of California, Davis, School of Medicine during the 2010–2011 admissions cycle and responded to the Big Five Inventory measuring their personality factors (agreeableness, conscientiousness, extraversion, neuroticism, openness). Individuals’ MMI performance at 10 stations was summarized as a total score. Regression analyses examined associations of personality factors with MMI score, and associations of personality factors and MMI score with acceptance offers. Covariates included sociodemographic and academic performance measures. Results Among the 444 respondents, those with extraversion scores in the top (versus bottom) quartile had significantly higher MMI scores (adjusted parameter estimate = 5.93 higher, 95% CI: 4.27–7.59; P < .01). In a model excluding MMI score, top (versus bottom) quartile agreeableness (AOR = 3.22; 95% CI 1.57–6.58; P < .01) and extraversion (AOR = 3.61; 95% CI 1.91–6.82; P < .01) were associated with acceptance offers. After adding MMI score to the model, high agreeableness (AOR = 4.77; 95% CI 1.95–11.65; P < .01) and MMI score (AOR 1.33; 95% CI 1.26–1.42; P < .01) were associated with acceptance offers. Conclusions Extraversion was associated with MMI performance, whereas both extraversion and agreeableness were associated with acceptance offers. Adoption of the MMI may affect diversity in medical student personalities, with potential implications for students’ professional growth, specialty distribution, and patient care.

Implementation of a deidentified federated data network for population-based cohort discovery

Objective The Cross-Institutional Clinical Translational Research project explored a federated query tool and looked at how this tool can facilitate clinical trial cohort discovery by managing access to aggregate patient data located within unaffiliated academic medical centers. Methods The project adapted software from the Informatics for Integrating Biology and the Bedside (i2b2) program to connect three Clinical Translational Research Award sites: University of Washington, Seattle, University of California, Davis, and University of California, San Francisco. The project developed an iterative spiral software development model to support the implementation and coordination of this multisite data resource. Results By standardizing technical infrastructures, policies, and semantics, the project enabled federated querying of deidentified clinical datasets stored in separate institutional environments and identified barriers to engaging users for measuring utility. Discussion The authors discuss the iterative development and evaluation phases of the project and highlight the challenges identified and the lessons learned. Conclusion The common system architecture and translational processes provide high-level (aggregate) deidentified access to a large patient population (>5 million patients), and represent a novel and extensible resource. Enhancing the network for more focused disease areas will require research-driven partnerships represented across all partner sites.

Physician Communication Regarding Prostate Cancer Screening: Analysis of Unannounced Standardized Patient Visits

PURPOSE Prostate cancer screening with prostate-specific antigen (PSA) is a controversial issue. The present study aimed to explore physician behaviors during an unannounced standardized patient encounter that was part of a randomized controlled trial to educate physicians using a prostate cancer screening, interactive, Web-based module. METHODS Participants included 118 internal medicine and family medicine physicians from 5 health systems in California, in 2007–2008. Control physicians received usual education about prostate cancer screening (brochures from the Center for Disease Control and Prevention). Intervention physicians participated in the prostate cancer screening module. Within 3 months, all physicians saw unannounced standardized patients who prompted prostate cancer screening discussions in clinic. The encounter was audio-recorded, and the recordings were transcribed. Authors analyzed physician behaviors around screening: (1) engagement after prompting, (2) degree of shared decision making, and (3) final recommendations for prostate cancer screening. RESULTS After prompting, 90% of physicians discussed prostate cancer screening. In comparison with control physicians, intervention physicians showed somewhat more shared decision making behaviors (intervention 14 items vs control 11 items, P <.05), were more likely to mention no screening as an option (intervention 63% vs control 26%, P <.05), to encourage patients to consider different screening options (intervention 62% vs control 39%, P <.05) and seeking input from others (intervention 25% vs control 7%, P<.05). CONCLUSIONS A brief Web-based interactive educational intervention can improve shared decision making, neutrality in recommendation, and reduce PSA test ordering. Engaging patients in discussion of the uses and limitations of tests with uncertain value can decrease utilization of the tests.

Pairing Physician Education With Patient Activation to Improve Shared Decisions in Prostate Cancer Screening: A Cluster Randomized Controlled Trial

BACKGROUND Most expert groups recommend shared decision making for prostate cancer screening. Most primary care physicians, however, routinely order a prostate-specific antigen (PSA) test with little or no discussion about whether they believe the potential benefits justify the risk of harm. We sought to assess whether educating primary care physicians and activating their patients to ask about prostate cancer screening had a synergistic effect on shared decision making, rates and types of discussions about prostate cancer screening, and the physician’s final recommendations. METHODS Our study was a cluster randomized controlled trial among primary care physicians and their patients, comparing usual education (control), with physician education alone (MD-Ed), and with physician education and patient activation (MD-Ed+A). Participants included 120 physicians in 5 group practices, and 712 male patients aged 50 to 75 years. The interventions comprised a Web-based educational program for all intervention physicians and MD-Ed+A patients compared with usual education (brochures from the Centers for Disease Control and Prevention). The primary outcome measure was patients’ reported postvisit shared decision making regarding prostate cancer screening; secondary measures included unannounced standardized patients’ reported shared decision making and the physician’s recommendation for prostate cancer screening. RESULTS Patients’ ratings of shared decision making were moderate and did not differ between groups. MD-Ed+A patients reported that physicians had higher prostate cancer screening discussion rates (MD-Ed+A = 65%, MD-Ed = 41%, control=38%; P <.01). Standardized patients reported that physicians seeing MD-Ed+A patients were more neutral during prostate cancer screening recommendations (MD-Ed+A=50%, MD-Ed=33%, control=15%; P <.05). Of the male patients, 80% had had previous PSA tests. CONCLUSIONS Although activating physicians and patients did not lead to significant changes in all aspects of physician attitudes and behaviors that we studied, interventions that involved physicians did have a large effect on their attitudes toward screening and in the discussions they had with patients, including their being more likely than control physicians to engage in prostate cancer screening discussions and more likely to be neutral in their final recommendations.

Developing Common Metrics for the Clinical and Translational Science Awards (CTSAs): Lessons Learned

The National Institutes of Health (NIH) Roadmap for Medical Research initiative, funded by the NIH Common Fund and offered through the Clinical and Translational Science Award (CTSA) program, developed more than 60 unique models for achieving the NIH goal of accelerating discoveries toward better public health. The variety of these models enabled participating academic centers to experiment with different approaches to fit their research environment. A central challenge related to the diversity of approaches is the ability to determine the success and contribution of each model. This paper describes the effort by the Evaluation Key Function Committee to develop and test a methodology for identifying a set of common metrics to assess the efficiency of clinical research processes and for pilot testing these processes for collecting and analyzing metrics. The project involved more than one‐fourth of all CTSAs and resulted in useful information regarding the challenges in developing common metrics, the complexity and costs of acquiring data for the metrics, and limitations on the utility of the metrics in assessing clinical research performance. The results of this process led to the identification of lessons learned and recommendations for development and use of common metrics to evaluate the CTSA effort.

Developing the Translational Research Workforce: A Pilot Study of Common Metrics for Evaluating the Clinical and Translational Award KL2 Program

This pilot study describes the career development programs (i.e., NIH KL2 awards) across five Clinical and Translational Science Award (CTSA) institutions within the University of California (UC) system, and examines the feasibility of a set of common metrics for evaluating early outcomes.