Nancy D. Sharp

Organizational readiness to change assessment (ORCA): development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework.

BackgroundThe Promoting Action on Research Implementation in Health Services, or PARIHS, framework is a theoretical framework widely promoted as a guide to implement evidence-based clinical practices. However, it has as yet no pool of validated measurement instruments that operationalize the constructs defined in the framework. The present article introduces an Organizational Readiness to Change Assessment instrument (ORCA), organized according to the core elements and sub-elements of the PARIHS framework, and reports on initial validation.MethodsWe conducted scale reliability and factor analyses on cross-sectional, secondary data from three quality improvement projects (n = 80) conducted in the Veterans Health Administration. In each project, identical 77-item ORCA instruments were administered to one or more staff from each facility involved in quality improvement projects. Items were organized into 19 subscales and three primary scales corresponding to the core elements of the PARIHS framework: (1) Strength and extent of evidence for the clinical practice changes represented by the QI program, assessed with four subscales, (2) Quality of the organizational context for the QI program, assessed with six subscales, and (3) Capacity for internal facilitation of the QI program, assessed with nine subscales.ResultsCronbachs alpha for scale reliability were 0.74, 0.85 and 0.95 for the evidence, context and facilitation scales, respectively. The evidence scale and its three constituent subscales failed to meet the conventional threshold of 0.80 for reliability, and three individual items were eliminated from evidence subscales following reliability testing. In exploratory factor analysis, three factors were retained. Seven of the nine facilitation subscales loaded onto the first factor; five of the six context subscales loaded onto the second factor; and the three evidence subscales loaded on the third factor. Two subscales failed to load significantly on any factor. One measured resources in general (from the context scale), and one clinical champion role (from the facilitation scale).ConclusionWe find general support for the reliability and factor structure of the ORCA. However, there was poor reliability among measures of evidence, and factor analysis results for measures of general resources and clinical champion role did not conform to the PARIHS framework. Additional validation is needed, including criterion validation.

Construction and characteristics of the RxRisk-V: a VA-adapted pharmacy-based case-mix instrument.

Background. Assessment of disease burden is the key to many aspects of health care management. Patient diagnoses are commonly used for case-mix assessment. However, issues pertaining to diagnostic data availability and reliability make pharmacy-based strategies attractive. Our goal was to provide a reliable and valid pharmacy-based case-mix classification system for chronic diseases found in the Veterans Health Administration (VHA) population. Objective. To detail the development and category definitions of a VA-adapted version of the RxRisk (formerly the Chronic Disease Score); to describe category prevalence and reliability; to check category criterion validity against ICD-9 diagnoses; and to assess category-specific regression coefficients in concurrent and prospective cost models. Research Design. Clinical and pharmacological review followed by cohort analysis of diagnostic, pharmacy, and utilization databases. Subjects. 126,075 veteran users of VHA services in Washington, Oregon, Idaho, and Alaska. Methods. We used Kappa statistics to evaluate RxRisk category reliability and criterion validity, and multivariate regression to estimate concurrent and prospective cost models. Results. The RxRisk-V classified 70.5% of the VHA Northwest Network 1998 users into an average of 2.61 categories. Of the 45 classes, 33 classes had good-excellent 1-year reliability and 25 classes had good-excellent criterion validity against ICD-9 diagnoses. The RxRisk-V accounts for a distinct proportion of the variance in concurrent (R2 = 0.18) and prospective cost (R2 = 0.10) models. Conclusions. The RxRisk-V provides a reliable and valid method for administrators to describe and understand better chronic disease burden of their treated populations. Tailoring to the VHA permits assessment of disease burden specific to this population.

The association between nursing factors and patient mortality in the Veterans Health Administration: the view from the nursing unit level.

Context:Nurse staffing is not the same across an entire hospital. Nursing care is delivered in geographically-based units, with wide variation in staffing levels. In particular, staffing in intensive care is much richer than in nonintensive care acute units. Objective:To evaluate the association of in-hospital patient mortality with registered nurse staffing and skill mix comparing hospital and unit level analysis using data from the Veterans Health Administration (VHA). Design, Settings, and Patients:A retrospective observational study using administrative data from 129,579 patients from 453 nursing units (171 ICU and 282 non-ICU) in 123 VHA hospitals. Methods:We used hierarchical multilevel regression models to adjust for patient, unit, and hospital characteristics, stratifying by whether or not patients had an ICU stay during admission. Main Outcome Measure:In-hospital mortality. Results:Of the 129,579 patients, mortality was 2.9% overall: 6.7% for patients with an ICU stay compared with 1.6% for those without. Whether the analysis was done at the hospital or unit level affected findings. RN staffing was not significantly associated with in-hospital mortality for patients with an ICU stay (OR, 1.02; 95% CI, 0.99–1.03). For non-ICU patients, increased RN staffing was significantly associated with decreased mortality risk (OR, 0.91; 95% CI, 0.86–0.96). RN education was not significantly associated with mortality. Conclusions:Our findings suggest that the association between RN staffing and skill mix and in-hospital patient mortality depends on whether the analysis is conducted at the hospital or unit level. Variable staffing on non-ICU units may significantly contribute to in-hospital mortality risk.

Predicting costs of care using a pharmacy-based measure risk adjustment in a veteran population.

Background. Although most widely used risk adjustment systems use diagnosis data to classify patients, there is growing interest in risk adjustment based on computerized pharmacy data. The Veterans Health Administration (VHA) is an ideal environment in which to test the efficacy of a pharmacy-based approach. Objective. To examine the ability of RxRisk-V to predict concurrent and prospective costs of care in VHA and compare the performance of RxRisk-V to a simple age/gender model, the original RxRisk, and two leading diagnosis-based risk adjustment approaches: Adjusted Clinical Groups and Diagnostic Cost Groups/Hierarchical Condition Categories. Methods. The study population consisted of 161,202 users of VHA services in Washington, Oregon, Idaho, and Alaska during fiscal years (FY) 1996 to 1998. We examined both concurrent and predictive model fit for two sequential 12-month periods (FY 98 and FY 99) with the patient-year as the unit of analysis, using split-half validation. Results. Our results show that the Diagnostic Cost Group /Hierarchical Condition Categories model performs best (R2 = 0.45) among concurrent cost models, followed by ADG (0.31), RxRisk-V (0.20), and age/sex model (0.01). However, prospective cost models other than age/sex showed comparable R2: Diagnostic Cost Group /Hierarchical Condition Categories R2 = 0.15, followed by ADG (0.12), RxRisk-V (0.12), and age/sex (0.01). Conclusions. RxRisk-V is a clinically relevant, open source risk adjustment system that is easily tailored to fit specific questions, populations, or needs. Although it does not perform better than diagnosis-based measures available on the market, it may provide a reasonable alternative to proprietary systems where accurate computerized pharmacy data are available.

Use of Outpatient Care in Veterans Health Administration and Medicare among Veterans Receiving Primary Care in Community-Based and Hospital Outpatient Clinics

OBJECTIVE To examine differences in use of Veterans Health Administration (VA) and Medicare outpatient services by VA primary care patients. DATA SOURCES/STUDY SETTING VA administrative and Medicare claims data from 2001 to 2004. STUDY DESIGN Retrospective cohort study of outpatient service use by 8,964 community-based and 6,556 hospital-based VA primary care patients. PRINCIPAL FINDINGS A significant proportion of VA patients used Medicare-reimbursed primary care (>30 percent) and specialty care (>60 percent), but not mental health care (3-4 percent). Community-based patients had 17 percent fewer VA primary care visits (p<.001), 9 percent more Medicare-reimbursed visits (p<.001), and 6 percent fewer total visits (p<.05) than hospital-based patients. Community-based patients had 22 percent fewer VA specialty care visits (p<.0001) and 21 percent more Medicare-reimbursed specialty care visits (p<.0001) than hospital-based patients, but no difference in total visits (p=.80). CONCLUSIONS Medicare-eligible VA primary care patients followed over 4 consecutive years used significant primary care and specialty care outside of VA. Community-based patients offset decreased VA use with increased service use paid by Medicare, suggesting that increasing access to VA primary care via community clinics may fragment veteran care in unintended ways. Coordination of care between VA and non-VA providers and health care systems is essential to improve the quality and continuity of care.

Nurse staffing and patient outcomes in Veterans Affairs hospitals.

Objective: To assess characteristics and perceptions of nurses working in the Veterans Health Administration (VHA), comparing types of nursing personnel, to benchmark to prior studies across healthcare systems. Background: Prior studies have shown relationships between positive registered nurse (RN) perceptions of the practice environment and patient outcomes. To date, no study has reported the comparison of RN perceptions of the practice environment in hospital nursing with those of non-RN nursing personnel. This study is the first to offer a more comprehensive look at perceptions of practice environment from the full range of the nursing work force and may shed light on issues such as the relationship of skill mix to nurse and patient outcomes. Methods: Cross-sectional observational study with a mailed survey administered to all nursing personnel in 125 VA Medical Centers between February and June 2003. Results: Compared with other types of nursing personnel in the VHA, RNs are generally less positive about their practice environments. However, compared with RNs in other countries and particularly with other RNs in the United States (Pennsylvania), VHA RNs are generally more positive about their practice environment and express more job satisfaction. Conclusions: The nursing work force of the VHA has some unique characteristics. The practice environment for nurses in the VHA is relatively positive, and may indicate that the VHA, as a system, provides an environment that is more like magnet hospitals. This is significant for a public sector hospital system.

Frequency of serum low-density lipoprotein cholesterol measurement and frequency of results ≤100 mg/dl among patients who had coronary events (Northwest VA Network Study) ∗

This population-based, cross-sectional analysis targeted all veterans with coronary heart disease (CHD) who were active patients in primary care or cardiology clinics in the Veterans Health Administration Northwest Network from July 1998 to June 1999. We report guideline compliance rates, including whether low-density lipoprotein (LDL) was measured, and if measured, whether the LDL was < or=100 mg/dl. In addition, we utilized multivariate logistic regression to determine patient characteristics associated with LDL measurements and levels. Of 13,891 active patients with CHD, 5,552 (40.0%) did not have a current LDL measurement. Of those with LDL measurements, 39.1% were at the LDL goal of < or =100 mg/dl, whereas 26.5% had LDL > or =130 mg/dl. Male gender, younger age, history of angioplasty or coronary artery bypass grafting, current hypertension, diabetes mellitus, and angina pectoris were associated with increased likelihood of LDL measurement. Older age and current diabetes and angina were associated with increased likelihood of LDL being < or =100 mg/dl, if measured. Although these rates of guideline adherence in the CHD population compare well to previously published results, they continue to be unacceptably low for optimal clinical outcomes. Attention to both LDL measurement and treatment (if elevated) is warranted.

Case‐Mix Adjusting Performance Measures in a Veteran Population: Pharmacy‐ and Diagnosis‐Based Approaches

OBJECTIVE To compare the rankings for health care utilization performance measures at the facility level in a Veterans Health Administration (VHA) health care delivery network using pharmacy- and diagnosis-based case-mix adjustment measures. DATA SOURCES/STUDY SETTING The study included veterans who used inpatient or outpatient services in Veterans Integrated Service Network (VISN) 20 during fiscal year 1998 (October 1997 to September 1998; N = 126,076). Utilization and pharmacy data were extracted from VHA national databases and the VISN 20 data warehouse. STUDY DESIGN We estimated concurrent regression models using pharmacy or diagnosis information in the base year (FY1998) to predict health service utilization in the same year. Utilization measures included bed days of care for inpatient care and provider visits for outpatient care. PRINCIPAL FINDINGS Rankings of predicted utilization measures across facilities vary by case-mix adjustment measure. There is greater consistency within the diagnosis-based models than between the diagnosis- and pharmacy-based models. The eight facilities were ranked differently by the diagnosis- and pharmacy-based models. CONCLUSIONS Choice of case-mix adjustment measure affects rankings of facilities on performance measures, raising concerns about the validity of profiling practices. Differences in rankings may reflect differences in comparability of data capture across facilities between pharmacy and diagnosis data sources, and unstable estimates due to small numbers of patients in a facility.

Importance of health system context for evaluating utilization patterns across systems.

Measuring health services provided to patients can be difficult when patients see providers across multiple health systems and all visits are rarely captured in a single data source covering all systems where patients receive care. Studies that account for only one system will omit the out-of-system health-care use at the patient level. Combining data across systems and comparing utilization patterns across health systems creates complications for both aggregation and accuracy because data-generating processes (DGPs) tend to vary across systems. We develop a hybrid methodology for aggregation across systems, drawing on the strengths of the DGP in each system, and demonstrate its validity for answering research questions requiring cross-system assessments of health-care utilization. Positive and negative predictive probabilities can be useful to assess the impact of the hybrid methodology. We illustrate these issues comparing public sector (administrative records from the US Department of Veterans Affairs system) and private sector (billing records from the US Medicare system) patient level data to identify primary-care utilization. Understanding the context of a particular health system and its effect on the DGP is important in conducting effective valid evaluations.

Use of outpatient care in VA and Medicare among disability-eligible and age-eligible veteran patients

BackgroundMore than half of veterans who use Veterans Health Administration (VA) care are also eligible for Medicare via disability or age, but no prior studies have examined variation in use of outpatient services by Medicare-eligible veterans across health system, type of care or time.ObjectivesTo examine differences in use of VA and Medicare outpatient services by disability-eligible or age-eligible veterans among veterans who used VA primary care services and were also eligible for Medicare.MethodsA retrospective cohort study of 4,704 disability- and 10,816 age-eligible veterans who used VA primary care services in fiscal year (FY) 2000. We tracked their outpatient utilization from FY2001 to FY2004 using VA administrative and Medicare claims data. We examined utilization differences for primary care, specialty care, and mental health outpatient visits using generalized estimating equations.ResultsAmong Medicare-eligible veterans who used VA primary care, disability-eligible veterans had more VA primary care visits (p < 0.001) and more VA specialty care visits (p < 0.001) than age-eligible veterans. They were more likely to have mental health visits in VA (p < 0.01) and Medicare-reimbursed visits (p < 0.01). Disability-eligible veterans also had more total (VA+Medicare) visits for primary care (p < 0.01) and specialty care (p < 0.01), controlling for patient characteristics.ConclusionsGreater use of primary care and specialty care visits by disability-eligible veterans is most likely related to greater health needs not captured by the patient characteristics we employed and eligibility for VA care at no cost. Outpatient care patterns of disability-eligible veterans may foreshadow care patterns of veterans returning from Afghanistan and Iraq wars, who are entering the system in growing numbers. This study provides an important baseline for future research assessing utilizations among returning veterans who use both VA and Medicare systems. Establishing effective care coordination protocols between VA and Medicare providers can help ensure efficient use of taxpayer resources and high quality care for disabled veterans.