John W. Peabody

Assessing Measurement Properties of Two Single-item General Health Measures

Background: Multi-item health status measures can be lengthy, expensive, and burdensome to collect. Single-item measures may be an alternative. We compared measurement properties of two single-item, general self-rated health (GSRH) questions to assess how well they captured information in a validated, multi-item instrument. Methods: We administered a general health survey (SF-12V) that included “standard” and “comparative” forms of a GSRH. We repeated the survey two weeks later to the same 75 medically stable outpatients to test for GSRH reproducibility, reliability, and validity using SF-12V Physical Functioning and Emotional Health subscales as a reference. Results: At each survey administration, the two GSRH questions demonstrated good alternate forms reliability (first administration: r=0.74, p<0.001; second administration: r=0.74, p<0.001) and good reproducibility (“standard”: ICC 0.69; “comparative”: ICC 0.85). Both GSRH items correlated with physical functioning (“standard”: r=0.66; “comparative”: r=0.56) and emotional health measures (“standard”: r=0.65; “comparative”: r=0.59). Mean subscale measures associated with responses in each GSRH category were significantly different (ANOVA, p<0.001), indicating strong discriminant scale performance. Conclusions: Our single-item, GSRH questions demonstrated good reproducibility, reliability, and strong concurrent and discriminant scale performance with an established health status measure.

Assessing the accuracy of administrative data in health information systems.

Background:Administrative data play a central role in health care. Inaccuracies in such data are costly to health systems, they obscure health research, and they affect the quality of patient care. Objectives:We sought to prospectively determine the accuracy of the primary and secondary diagnoses recorded in administrative data sets. Research Design:Between March and July 2002, standardized patients (SPs) completed unannounced visits at 3 sites. We abstracted the 348 medical records from these visits to obtain the written diagnoses made by physicians. We also examined the patient files to identify the diagnoses recorded on the administrative encounter forms and extracted data from the computerized administrative databases. Because the correct diagnosis was defined by the SP visit, we could determine whether the final diagnosis in the administrative data set was correct and, if not, whether it was caused by physician diagnostic error, missing encounter forms, or incorrectly filled out forms. Subjects:General internal medicine outpatient clinics at 2 Veterans Administration facilities and a large, private medical center participated in this study. Measures:A total of 45 trained SPs presented to physicians with 4 common outpatient conditions. Results:The correct primary diagnosis was recorded for 57% of visits. Thirteen percent of errors were caused by physician diagnostic error, 8% to missing encounter forms, and 22% to incorrectly entered data. Findings varied by condition and site but not by level of training. Accuracy of secondary diagnosis data (27%) was even poorer. Conclusions:Although more research is needed to evaluate the cause of inaccuracies and the relative contributions of patient, provider, and system level effects, it appears that significant inaccuracies in administrative data are common. Interventions aimed at correcting these errors appear feasible.

An assessment of programs to promote improved household stoves in China

In 2002, a team of US and Chinese researchers collaborated on an independent, multidisciplinary review of Chinas improved rural household stove programs that have been carried out since the 1980s. The objectives were to delineate and evaluate the methods used to promote improved stoves, to assess the development of commercial stove production and marketing organizations, and to measure the household impacts of the programs through surveys of health, stoves, and indoor air quality. The team found that China implemented broadly successful programs that delivered better stoves to a majority of households in targeted counties. That success was based on strong administrative, technical, and outreach competence and resources situated at the local level, motivated by sustained national-level attention. Despite overstated claims for penetration of improved stoves, mostbiomass stoves now in use have flues and other “improved” aspects, although field efficiencies are less than design efficiencies. However, most coal stoves, even those using improved fuel (briquettes), lack flues and cannot be considered improved. While in most areas “improved” stove technology became “conventional”, some areas remain significant exceptions and require intervention. Large roles for government oversight of quality control and support of R&D remain inadequately fulfilled. Most results of indoor air quality monitoring and health surveys were not clear-cut, in part because of the wide variety of fuel and stove combinations used by households. For nearly all household stove/fuel groupings, however, PM4 levels were higher than – and sometimes more than twice as high as – the national standard for indoor air (150 μg PM10/m3). If these results are typical, then a large fraction of Chinas rural population is now chronically exposed to levels of pollution far higher than those determined by the Chinese government to harm human health. Improved stoves in the surveyed households did result in reduced PM4 concentrations indoors for biomass fuel combinations. Coal use was associated with elevated levels of CO in exhaled breath, and improved biomass stoves with lower levels. Childhood asthma and adult respiratory disease were positively associated with coal use and negatively associated with improved stoves and good stove maintenance. The benefits of using improved biomass stoves can be outweighed by the use of portable coal stoves without flues. Evaluation of the programs provides important lessons for future initiatives in China and in other countries.

Tobacco control in India

Legislation to control tobacco use in developing countries has lagged behind the dramatic rise in tobacco consumption. India, the third largest grower of tobacco in the world, amassed 1.7 million disability-adjusted life years (DALYs) in 1990 due to disease and injury attributable to tobacco use in a population where 65% of the men and 38% of the women consume tobacco. Indias anti-tobacco legislation, first passed at the national level in 1975, was largely limited to health warnings and proved to be insufficient. In the last decade state legislation has increasingly been used but has lacked uniformity and the multipronged strategies necessary to control demand. A new piece of national legislation, proposed in 2001, represents an advance. It includes the following key demand reduction measures: outlawing smoking in public places; forbidding sale of tobacco to minors; requiring more prominent health warning labels; and banning advertising at sports and cultural events. Despite these measures, the new legislation will not be enough to control the demand for tobacco products in India. The Indian Government must also introduce policies to raise taxes, control smuggling, close advertising loopholes, and create adequate provisions for the enforcement of tobacco control laws.

Health care expenditure prediction with a single item, self-rated health measure.

Background:Prediction models that identify populations at risk for high health expenditures can guide the management and allocation of financial resources. Objective:To compare the ability for identifying individuals at risk for high health expenditures between the single-item assessment of general self-rated health (GSRH), “In general, would you say your health is Excellent, Very Good, Good, Fair, or Poor?,” and 3 more complex measures. Study Design:We used data from a prospective cohort, representative of the US civilian noninstitutionalized population, to compare the predictive ability of GSRH to: (1) the Short Form-12, (2) the Seattle Index of Comorbidity, and (3) the Diagnostic Cost-Related Groups/Hierarchal Condition Categories Relative-Risk Score. The outcomes were total, pharmacy, and office-based annualized expenditures in the top quintile, decile, and fifth percentile and any inpatient expenditures. Data Source:Medical Expenditure Panel Survey panels 8 (2003–2004, n = 7948) and 9 (2004–2005, n = 7921). Results:The GSRH model predicted the top quintile of expenditures, as well as the SF-12, Seattle Index of Comorbidity, though not as well as the Diagnostic Cost-Related Groups/Hierarchal Condition Categories Relative-Risk Score: total expenditures [area under the curve (AUC): 0.79, 0.80, 0.74, and 0.84, respectively], pharmacy expenditures (AUC: 0.83, 0.83, 0.76, and 0.87, respectively), and office-based expenditures (AUC: 0.73, 0.74, 0.68, and 0.78, respectively), as well as any hospital inpatient expenditures (AUC: 0.74, 0.76, 0.72, and 0.78, respectively). Results were similar for the decile and fifth percentile expenditure cut-points. Conclusions:A simple model of GSRH and age robustly stratifies populations and predicts future health expenditures generally as well as more complex models.

An Evaluation of Vignettes for Predicting Variation in the Quality of Preventive Care

OBJECTIVE: Clinical vignettes offer an inexpensive and convenient alternative to the benchmark method of chart audits for assessing quality of care. We examined whether vignettes accurately measure and predict variation in the quality of preventive care. DESIGN: We developed scoring criteria based on national guidelines for 11 prevention items, categorized as vaccine, vascular-related, cancer screening, and personal behaviors. Three measurement methods were used to ascertain the quality of care provided by clinicians seeing trained actors (standardized patients; SPs) presenting with common outpatient conditions: 1) the abstracted medical record from an SP visit; 2) SP reports of physician practice during those visits; and 3) physician responses to matching computerized case scenarios (clinical vignettes). SETTING: Three university-affiliated (including 2 VA) and one community general internal medicine clinics. PATIENTS/PARTICIPANTS: Seventy-one randomly selected physicians from among eligible general internal medicine residents and attending physicians. MEASUREMENTS AND MAIN RESULTS: Physicians saw 480 SPs (120 at each site) and completed 480 vignettes. We calculated the proportion of prevention items for each visit reported or recorded by the 3 measurement methods. We developed a multiple regression model to determine whether site, training level, or clinical condition predicted prevention performance for each measurement method. We found that overall prevention scores ranged from 57% (SP) to 54% (vignettes) to 46% (chart abstraction). Vignettes matched or exceeded SP scores for 3 prevention categories (vaccine, vascular-related, and personal behavior). Prevention quality varied by site (from 40% to 67%) and was predicted similarly by vignettes and SPs. CONCLUSIONS: Vignettes can measure and predict prevention performance. Vignettes may be a less costly way to assess prevention performance that also controls for patient case-mix.

Measuring Compliance with Preventive Care Guidelines

AbstractOBJECTIVE: To determine how accurately preventive care reported in the medical record reflects actual physician practice or competence. DESIGN: Scoring criteria based on national guidelines were developed for 7 separate items of preventive care. The preventive care provided by randomly selected physicians was measured prospectively for each of the 7 items. Three measurement methods were used for comparison: (1) the abstracted medical record from a standardized patient (SP) visit; (2) explicit reports of physician practice during those visits from the SPs, who were actors trained to present undetected as patients; and (3) physician responses to written case scenarios (vignettes) identical to the SP presentations. SETTING: The general medicine primary care clinics of two university-affiliated VA medical centers. PARTICIPANTS: Twenty randomly selected physicians (10 at each site) from among eligible second- and third-year general internal medicine residents and attending physicians. MEASUREMENTS AND MAIN RESULTS: Physicians saw 160 SPs (8 cases × 20 physicians). We calculated the percentage of visits in which each prevention item was recorded in the chart, determined the marginal percentage improvement of SP checklists and vignettes over chart abstraction alone, and compared the three methods using an analysis-of-variance model. We found that chart abstraction underestimated overall prevention compliance by 16% (P < .01) compared with SP checklists. Chart abstraction scores were lower than SP checklists for all seven items and lower than vignettes for four items. The marginal percentage improvement of SP checklists and vignettes to performance as measured by chart abstraction was significant for all seven prevention items and raised the overall prevention scores from 46% to 72% (P < .0001). CONCLUSIONS: These data indicate that physicians perform more preventive care than they report in the medical record. Thus, benchmarks of preventive care by individual physicians and institutions that rely solely on the medical record may be misleading, at best.

Economic reform and health sector policy: Lessons from structural adjustment programs

From a purely economic perspective, structural adjustment programs (SAPs) and economic reform policies are viewed as short-term austerities that lead to long-term growth and development. These intertemporal trade-offs, however, are not always acceptable in health. Unique biologic events such as intrauterine development and neural development cannot be postponed even for a short period. Health policymakers need to understand the expected and unexpected impacts of economic reform on health outcomes in individuals and on the population. The interactions are complex, involve multiple sectors, and can be better understood by looking at the experience of developing countries over almost fifteen years of SAP experience. Health care budgets may be vulnerable to reduced government spending, quality of care deteriorates, nutrition will suffer more likely in urban areas, and cost-effective preventive programs may stop if labor and capital are not properly matched. Health outcomes overall do not appear to suffer but a more detailed look, with better data, shows that the incidence of preventable diseases rises and irreversible deterioration in health status does occur within countries. To prevent this from happening in the future, health policymakers need to take a multidisciplinary focus to first understand the effects of economic reform and then to plan a coordinated response. Better data, alternative financing, and strong political leadership are also important lessons.

Associations between Cognitive Function, Blood Lead Concentration, and Nutrition among Children in the Central Philippines

OBJECTIVE Because little is known about its effects on cognitive function among children in less-developed countries, we determined the impact of lead exposure from other nutritional determinants of cognitive ability. STUDY DESIGN Data were from a cross-sectional population-based stratified random sample of 877 children (age 6 months-5 years) participating in the Quality Improvement Demonstration Study we are conducting in the Philippines. With data from validated psychometric instruments, venous blood samples, and comprehensive survey instruments, we developed multi-stage models to account for endogenous determinants of blood lead levels (BLLs) and exogenous confounders of the association between BLLs and cognitive function. RESULTS A 1 microg/dL increase in BLL was associated with a 3.32 point decline in cognitive functioning in children aged 6 months to 3 years and a 2.47 point decline in children aged 3 to 5 years olds. BLL was inversely associated with hemoglobin and folate levels. Higher folate levels mitigated the negative association between BLL and cognitive function. CONCLUSIONS These population-based data suggest greater lead toxicity on cognitive function than previously reported. Our findings also suggest that folate and iron deficient children are more susceptible to the negative cognitive effects of lead. Folate supplementation may offer some protective effects against lead exposure.

Cardiac risk underestimation in urban, black women.

BACKGROUND: Black women have a disproportionately higher incidence of cardiovascular disease mortality than other groups and the reason for this health disparity is incompletely understood. Underestimation of personal cardiac risk may play a role. OBJECTIVE: We investigated the personal characteristics associated with underestimating cardiovascular disease in black women. DESIGN, SETTING, PARTICIPANTS: Trained surveyors interviewed 128 black women during the baseline evaluation for a randomized controlled trial in an urban, academic continuity clinic affiliated with a public hospital system. They provided information on the presence of cardiac risk factors and demographic and psychosocial characteristics. These self-report data were supplemented with medical record abstraction for weight. MEASUREMENTS AND MAIN RESULTS: The main outcome measure was the accurate perception of cardiac risk. Objective risk was determined by a simple count of major cardiac risk factors and perceived risk by respondent’s answer to a survey question about personal cardiac risk. The burden of cardiac risk factors was high in this population: 77% were obese; 72% had hypertension; 48% had high cholesterol; 49% had a family history of heart disease; 31% had diabetes, and 22% currently used tobacco. Seventy-nine percent had 3 or more cardiac risk factors. Among those with 3 or more risk factors (“high risk”), 63% did not perceive themselves to be at risk for heart disease. Among all patients, objective and perceived cardiac risk was poorly correlated (κ=0.026). In a multivariable model, increased perceived personal stress and lower income were significant correlates of underestimating cardiac risk. CONCLUSIONS: Urban, disadvantaged black women in this study had many cardiac risk factors, yet routinely underestimated their risk of heart disease. We found that the strongest correlates of underestimation were perceived stress and lower personal income.