Douglas P. Wagner

Comparison of the performance of two comorbidity measures, with and without information from prior hospitalizations.

Objectives.This study compares the performance of two comorbidity risk adjustment methods (the Deyo et al adaptation of the Charlson index and the Elixhauser et al method) in five groups of California hospital patients with common reasons for hospitalization, and assesses the contribution to model performance made by information drawn from prior hospital admissions. Methods.California hospital discharge abstract data for the calendar years 1994 through 1997 were used to create a longitudinal data set for patients in the five disease groups. Eleven logistic regression models were estimated to predict the risk of in-hospital death for patients in each group, with both comorbidity risk adjustment methods applied to patient information available from only the index hospitalization, and to information available from both the index and prior hospitalizations. Results.For every comparison made, the level of statistical performance (area under the receiver operating characteristics curve) demonstrated by models using the Elixhauser et al method was superior to that of models using the Deyo et al adaptation method. Although most patients have information available from prior hospital admissions, this additional information yields only small improvements in the performance of models using either comorbidity risk adjustment method. Conclusions.Better discrimination is achieved with the Elixhauser et al method using only information from the index hospitalization than is achieved with the Deyo et al adaptation using information from all identified hospital admissions. Both comorbidity risk adjustment methods achieve their best performance when information from the index hospitalization and prior admissions is separated into independent indicators of comorbid illness.

A Predictive Risk Model for Outcomes of Ischemic Stroke

BACKGROUND AND PURPOSEnThe great variability of outcome seen in stroke patients has led to an interest in identifying predictors of outcome. The combination of clinical and imaging variables as predictors of stroke outcome in a multivariable risk adjustment model may be more powerful than either alone. The purpose of this study was to determine the multivariable relationship between infarct volume, 6 clinical variables, and 3-month outcomes in ischemic stroke patients.nnnMETHODSnIncluded in the study were 256 eligible patients from the Randomized Trial of Tirilazad Mesylate in Acute Stroke (RANTTAS). Six clinical variables and 1-week infarct volume were the prespecified predictor variables. The National Institutes of Health Stroke Scale, Barthel Index, and Glasgow Outcome Scale were the outcomes. Multivariable logistic regression techniques were used to develop the model equations, and bootstrap techniques were used for internal validation. Predictive performance of the models was assessed for discrimination with receiver operator characteristic (ROC) curves and for calibration with calibration curves.nnnRESULTSnThe predictive models had areas under the ROC curve of 0.79 to 0.88 and demonstrated nearly ideal calibration curves. The areas under the ROC curves were statistically greater (P<0.001) with both clinical and imaging information combined than with either alone for predicting excellent recovery and death or severe disability.nnnCONCLUSIONSnCombined clinical and imaging variables are predictive of 3-month outcome in ischemic stroke patients. Demonstration of this relationship with acute clinical variables and 1-week infarct information supports future attempts to predict 3-month outcome with all acute variables.

Glasgow Coma Scale score in the evaluation of outcome in the intensive care unit: findings from the Acute Physiology and Chronic Health Evaluation III study.

To investigate the ability of the Glasgow Coma Scale score to predict hospital mortality rate for adult medical-surgical intensive care unit (ICU) patients without trauma. Design:A prospective cohort analysis of adult medical-surgical patients from a nationally representative sample of 40 U.S. hospitals. Patients:15,973 consecutive, nontraumatic ICU admissions and a comparison group of 687 head trauma admissions. Interventions:None. Measurements and Main Results:Patients gender, age, treatment location before ICU admission, comorbidities, admission diagnosis, daily physiologic measurements, Glasgow Coma Scale score, Acute Physiology and Chronic Health Evaluation (APACHE IIITM) score, subsequent hospital mortality rate, and unit-specific sedation practices were noted. Hospital mortality rates were stratified by the first ICU day Glasgow Coma Scale score for all admissions. The relationship between the Glasgow Coma Scale score and outcome for two high mortality medical diagnoses (post-cardiac arrest and sepsis) were also examined and compared to the relationship found in patients with head trauma.The Glasgow Coma Scale score on ICU admission had a highly significant (r2 = .922, p < .0001) but nonlinear relationship with subsequent outcome in ICU patients without trauma. Discrimination of patients into high- or low-risk prognostic groups was good, but discrimination in the intermediate levels (Glasgow Coma Scale score of 7 to 11) was reduced. This relationship varied within the operative and nonoperative groups, and also within different disease categories, various age groups, and certain ranges of the Glasgow Coma Scale score. A reduced initial Glasgow Coma Scale score associated with sepsis was a combination of factors associated with a higher mortality rate than that found in patients with head trauma. The proportion of patients who could not be assigned a Glasgow Coma Scale score because of sedation/paralysis varied widely across ICUs. The overall predictive capability of the APACHE III Prognostic Scoring System was improved by incorporating the Glasgow Coma Scale score. Conclusions:We demonstrated the prognostic importance of admission levels of consciousness as measured by the Glasgow Coma Scale score on ICU and hospital mortality rates. We concluded that the Glasgow Coma Scale score may be used to stratify and predict mortality risk in general intensive care patients, but lack of sensitivity in the intermediate range of Glasgow Coma Scale Score should be noted. Ideally, the Glasgow Coma Scale score should also be applied in the context of other physiologic information and the patients specific diagnosis. Variation in the use of sedatives in different ICUs means that imputing or substituting a value other thatn normal for an unobtainable Glasgow Coma Scale score may introduce a substantial treatment bias into subsequent outcome predictions. (Crit Care Med 1993; 21:1459–1465)

Prediction of Survival for Older Hospitalized Patients: The HELP Survival Model

OBJECTIVE: To develop and validate a model estimating the survival time of hospitalized persons aged 80 years and older.

Rurality, Gender, and Mental Health Treatment

Mental health problems are common and costly, yet many individuals with these problems either do not receive care or receive care that is inadequate. Gender and place of residence contribute to disparities in the use of mental health services. The objective of this study was to identify the influence of gender and rurality on mental health services utilization by using more sensitive indices of rurality. Pooled data from 4 panels of the Medical Expenditure Panel Survey (1996–2000) yielded a sample of 32,219 respondents aged 18 through 64. Variables were stratified by residence using rural–urban continuum codes. We used logistic and linear regression to model effects of gender and rurality on treatment rates. We found that rural women are less likely to receive mental health treatment either through the general healthcare system or through specialty mental health systems when compared to women in metropolitan statistical areas (MSA) or urbanized non-MSA areas. Rural men receive less mental health treatment than do rural women and less specialty mental health treatment than do men in MSAs or least rural non-MSA areas. Reported mental health deteriorates as the level of rurality increases. There is a considerable unmet need for mental health services in most rural areas. The general health sector does not seem to contribute remarkably to mental health services for women in these areas.

Validation of an acute ischemic stroke model: does diffusion-weighted imaging lesion volume offer a clinically significant improvement in prediction of outcome?

Background and Purpose— Prediction models for ischemic stroke outcome have the potential to contribute prognostic information in the clinical and/or research setting. The importance of diffusion-weighted magnetic resonance imaging (DWI) in the prediction of clinical outcome, however, is unclear. The purpose of this study was to combine acute clinical data and DWI lesion volume for ischemic stroke patients to determine whether DWI improves the prediction of clinical outcome. Methods— Patients (N=382) with baseline DWI data from the Glycine Antagonist In Neuroprotection and citicoline (010 and 018) trials were used to develop the prediction models by multivariable logistic regression. Data from prospectively collected patients (N=266) from the Acute Stroke Accurate Prediction Study were used to externally validate the model equations. The models predicted either full recovery or nursing home–level disability/death, as defined by the National Institutes of Health Stroke Scale, Barthel Index, or modified Rankin Scale. Results— The full-recovery models with DWI lesion volume had areas under the receiver operating characteristic curves (AUCs) of 0.799 to 0.821, and those without DWI lesion volume had AUCs of 0.758 to 0.798. The nursing home–level disability/death models with DWI had AUCs of 0.832 to 0.882, and those without DWI had AUCs of 0.827 to 0.867. All models had mean absolute errors ≤0.4 for calibration. Conclusions— All 12 models had excellent discrimination and calibration, with 8 of 12 meeting prespecified performance criteria (AUC ≥0.8, mean absolute error ≤0.4). Although DWI lesion volume significantly increased model explanatory power, the magnitude of increase was not large enough to be clinically important.

Predicting Outcome in Ischemic Stroke: External Validation of Predictive Risk Models

Background— Six multivariable models predicting 3-month outcome of acute ischemic stroke have been developed and internally validated previously. The purpose of this study was to externally validate the previous models in an independent data set. Summary of Report— We predicted outcomes for 299 patients with ischemic stroke who received placebo in the National Institute of Neurological Disorders and Stroke rt-PA trial. The model equations used 6 acute clinical variables and head CT infarct volume at 1 week as independent variables and 3-month National Institutes of Health Stroke Scale, Barthel Index, and Glasgow Outcome Scale as dependent variables. Previously developed model equations were used to forecast excellent and devastating outcome for subjects in the placebo tissue plasminogen activator data set. Area under the receiver operator characteristic curve was used to measure discrimination, and calibration charts were used to measure calibration. The validation data set patients were more severely ill (National Institutes of Health Stroke Scale and infarct volume) than the model development subjects. Area under the receiver operator characteristic curves demonstrated remarkably little degradation in the validation data set and ranged from 0.75 to 0.89. Calibration curves showed fair to good calibration. Conclusions— Our models have demonstrated excellent discrimination and acceptable calibration in an external data set. Development and validation of improved models using variables that are all available acutely are necessary.

Rurality and Mental Health Treatment

Diversity within rural areas renders rural–urban comparisons difficult. The association of mental health treatment rates with levels of rurality is investigated here using Rural–Urban Continuum Codes. Data from the 1996–1999 panels of the Medical Expenditure Panel Survey are aggregated to provide annual treatment rates for respondents reporting mental health problems. Data show that residents of the most rural areas receive less mental health treatment than those residing in metropolitan areas. The adjusted odds of receiving any mental health treatment are 47% higher for metropolitan residents than for those living in the most rural settings, and the adjusted odds for receiving specialized mental health treatment are 72% higher. Findings suggest rural community size and adjacency to metropolitan areas influence treatment rates.

Automated intensive care unit risk adjustment: Results from a National Veterans Affairs study

CONTEXTnComparison of outcome among intensive care units (ICUs) requires risk adjustment for differences in severity of illness and risk of death at admission to the ICU, historically obtained by costly chart review and manual data entry.nnnOBJECTIVEnTo accurately estimate patient risk of death in the ICU using data easily available in hospital electronic databases to permit automation.nnnDESIGN AND SETTINGnCohort study to develop and validate a model to predict mortality at hospital discharge using multivariate logistic regression with a split derivation (17,731) and validation (11,646) sample formed from 29,377 consecutive first ICU admissions to medical, cardiac, and surgical ICUs in 17 Veterans Health Administration hospitals between February 1996 and July 1997.nnnMAIN OUTCOME MEASURESnMortality at hospital discharge adjusted for age, laboratory data, diagnosis, source of ICU admission, and comorbid illness.nnnRESULTSnThe overall hospital death rate was 11.3%. In the validation sample, the model separated well between survivors and nonsurvivors (area under the receiver operating characteristic curve = 0.885). Examination of the observed vs. the predicted mortality across the range of mortality showed the model was well calibrated.nnnCONCLUSIONSnAutomation could broaden access to risk adjustment of ICU outcomes with only a small trade-off in discrimination. Broader use might promote valid evaluation of ICU outcomes, encouraging effective practices and improving ICU quality.

Variation in outcomes in Veterans Affairs intensive care units with a computerized severity measure.

Objective:To quantify the variability in risk-adjusted mortality and length of stay of Veterans Affairs intensive care units using a computer-based severity of illness measure. Design:Retrospective cohort study. Setting:A stratified random sample of 34 intensive care units in 17 Veterans Affairs hospitals. Participants:A consecutive sample of 29,377 first intensive care unit admissions from February 1996 through July 1997. Interventions:Standardized mortality ratio (observed/expected deaths) and observed minus expected length of stay (OMELOS) with 95% confidence intervals were estimated for each unit using a hierarchical logistic (standardized mortality ratio) or linear (OMELOS) regression model with Markov Chain Monte Carlo simulation. We adjusted for patient characteristics including age, admission diagnosis, comorbid disease, physiology at admission (from laboratory data), and transfer status. Measurements and Main Results:Mortality across the intensive care units for the 12,088 surgical and 17,289 medical cases averaged 11% (range, 2–30%). Length of stay in the intensive care units averaged 4.0 days (range, mean unit length of stay 3.0–5.9). Standardized mortality ratio of the intensive care units varied from 0.62 to 1.27; the standardized mortality ratio and 95% confidence interval were <1 for four intensive care units and >1.0 for seven intensive care units. OMELOS of the intensive care units ranged from −0.89 to 1.34 days. In a random slope hierarchical model, variation in standardized mortality ratio among intensive care units was similar across the range of severity, whereas variation in length of stay increased with severity. Standardized mortality ratio was not associated with OMELOS (Pearson’s r = .13). Conclusions:We identified intensive care units whose indicators for mortality and length of stay differ substantially using a conservative statistical approach with a severity adjustment model based on data available in computerized clinical databases. Computerized risk adjustment employing routinely available data may facilitate research on the utility of intensive care unit profiling and analysis of natural experiments to understand process and outcome links and quality efforts.