James A. Shaul

Toward a national consumer survey: evaluation of the CABHS and MHSIP instruments.

This article describes a study evaluating the Consumer Assessment of Behavioral Health Survey (CABHS) and the Mental Health Statistics Improvement Program (MHSIP) surveys. The purpose of the study was to provide data that could be used to develop recommendations for an improved instrument. Subjects were 3,443 adults in six behavioral health plans. The surveys did not differ significantly in response rate or consumer burden. Both surveys reliably assessed access to treatment and aspects of appropriateness and quality. The CABHS survey reliably assessed features of the insurance plan; the MHSIP survey reliably assessed treatment outcome. Analyses of comparable items suggested which survey items had greater validity. Results are discussed in terms of consistency with earlier research using these and other consumer surveys. Implications and recommendations for survey development, quality improvement, and national policy initiatives to evaluate health plan performance are presented.

Similarities and Differences in Choosing Health Plans

Background. Increasingly, consumers have multiple health insurance options. New information is being developed to help consumers with these choices. Objectives. To study similarities and differences in how the publicly and privately insured choose health plans. To explore the effect of traditional enrollment materials and reports developed by the Consumer Assessment of Health Plans Study (CAHPS) on consumers’ perceptions and decision-making. Research Design. Using data from eight CAHPS demonstrations, we tested for significant differences across consumers with employer-sponsored insurance, Medicaid, and Medicare. Subjects. Approximately 10,000 consumers with employer-sponsored, Medicaid, and Medicare health plans. Measures. Perceptions of the health plan selection process, use of information sources, and reactions to and use of traditional enrollment materials and CAHPS reports. Results. Most consumers with all types of insurance thought that choosing a health plan was important and obtained information from multiple sources. Choosing a plan was more difficult for Medicare and Medicaid recipients than for the privately insured. When choosing a plan, Medicaid recipients cared most about convenience and access, whereas the privately insured emphasized providers and costs. The percentage of consumers who looked at and remembered the CAHPS report varied widely from 24% to 77%. In all but one of the demonstration sites, most consumers spent less than 30 minutes looking at the CAHPS report. Conclusions. Group sponsors and the developers of information interventions such as CAHPS may need to invest in developing and testing different reporting approaches for Medicare, Medicaid, and privately insured consumers.

Providing Consumers with Information About the Quality of Health Plans: The Consumer Assessment of Health Plans Demonstration in Washington State

BACKGROUND In 1995 the Agency for Health Care Policy and Research began a five-year project, Consumer Assessment of Health Plans (CAHPS), to create instruments to collect data from consumers about their health care experiences, to develop sophisticated methods to convey these data to consumers, and to evaluate the value of these data to consumers who are selecting health plans. Results were obtained from one of the first CAHPS demonstration sites, the Washington State Health Care Authority. METHODS The survey was distributed in May-June 1997 to 15,885 enrollees in 20 health plans; 8,204 (51.6%) surveys were completed. Survey results were summarized in a report that described the performance of plans, which was distributed to 97,000 enrollees, and reactions to the report were obtained from more than 1,500 individuals. RESULTS Nearly everyone who was mailed the report said they saw it. A large proportion said they read most or all of it, and most thought the report was easy to understand, contained information needed to rate plans, and was helpful to learning about differences between plans. Those who used the CAHPS performance report were more likely to switch plans and to report that they were confident they had selected the best plan for their situation. DISCUSSION The study was unique in that it attempted to evaluate whether employees read the performance report, how they reacted to it, and whether reading it influenced their decision to switch plans or their confidence that they had selected a suitable plan. Choosing a new plan probably stimulated more intense scrutiny of the report than not anticipating switching.

The Impact of Having Parents Report About Both Their Own and Their Children's Experiences With Health Insurance Plans

OBJECTIVES The aim of this study was to determine whether parents rate their childrens care differently when they also rate their own care than when they do not. METHODS Subjects were employees of Washington State who had been enrolled in a health plan for at least 6 months and who had at least one covered child. Subjects were randomly assigned to four study groups that were surveyed using different protocols. To assess the stability of responses over time, a follow-up telephone interview was conducted with individuals in two of the groups. RESULTS Parents or guardians who received both the Adult and Child Surveys were less likely to complete a survey than those who received only one survey. Responses to selected survey questions were quite stable between survey administrations. Parents who rated only their childs health care experiences generally gave more positive responses than those who also rated their own care, although few of these differences were statistically significant. This may have been due, in part, to the lower response rates in the latter group. The pairs of survey questions that ask about the adults and childs experiences with the same aspects of care had moderate to high levels of association. The pair with the weakest association asked how clearly the doctor or nurse explained things to the adult or the child. CONCLUSIONS Sending both an adult and child survey to an adult could have an effect on the pattern of responses and result in lower response rates, but this might be a cost-effective way to collect reports about both adult and child health care.