J. Michael Brick

BIAS IN LIST-ASSISTED TELEPHONE SAMPLES

A number of researchers have suggested list-assisted sampling for the selection of telephone households to overcome some of the operational difficulties associated with the MitofskyWaksberg methods of random digit dialing (RDD). An advantage of a list-assisted method of RDD is that an equal probability systematic sample of telephone numbers can be selected and the variances of estimates from such a sample are usually lower than from a clustered design like the Mitofsky-Waksberg method. The main disadvantage of the list-assisted method is that it excludes some households from the sample, thus creating a coverage bias in the estimates. This article describes research on the coverage bias for a particular method of list-assisted sampling. The two key determinants of coverage bias are the proportion of households that are not eligible for the sample and the differences in the characteristics of the covered and not covered populations. The results show that about 4 percent of all households are excluded in national samples using this method of sampling. Furthermore, they show that the differences between the covered and uncovered populations are generally not large. The coverage bias resulting from these conditions may often be small.

Explaining Rising Nonresponse Rates in Cross-Sectional Surveys:

This review of nonresponse in cross-sectional household surveys in the United States shows trends in nonresponse rates, the main reasons for nonresponse, and changes in the components of nonresponse. It shows that nonresponse is increasing but that existing methods for modeling response mechanisms do not adequately explain these changes.

Exploring nonresponse bias in a health survey using neighborhood characteristics.

OBJECTIVES We examined potential nonresponse bias in a large-scale, population-based, random-digit-dialed telephone survey in California and its association with the response rate. METHODS We used California Health Interview Survey (CHIS) data and US Census data and linked the two data sets at the census tract level. We compared a broad range of neighborhood characteristics of respondents and nonrespondents to CHIS. We projected individual-level nonresponse bias using the neighborhood characteristics. RESULTS We found little to no substantial difference in neighborhood characteristics between respondents and nonrespondents. The response propensity of the CHIS sample was similarly distributed across these characteristics. The projected nonresponse bias appeared very small. CONCLUSIONS The response rate in CHIS did not result in significant nonresponse bias and did not substantially affect the level of data representativeness, and it is not valid to focus on response rates alone in determining the quality of survey data.

Growing Cell‐Phone Population and Noncoverage Bias in Traditional Random Digit Dial Telephone Health Surveys

OBJECTIVE Examine the effect of including cell-phone numbers in a traditional landline random digit dial (RDD) telephone survey. DATA SOURCES The 2007 California Health Interview Survey (CHIS). DATA COLLECTION METHODS CHIS 2007 is an RDD telephone survey supplementing a landline sample in California with a sample of cell-only (CO) adults. STUDY DESIGN We examined the degree of bias due to exclusion of CO populations and compared a series of demographic and health-related characteristics by telephone usage. PRINCIPAL FINDINGS When adjusted for noncoverage in the landline sample through weighting, the potential noncoverage bias due to excluding CO adults in landline telephone surveys is diminished. Both CO adults and adults who have both landline and cell phones but mostly use cell phones appear different from other telephone usage groups. Controlling for demographic differences did not attenuate the significant distinctiveness of cell-mostly adults. CONCLUSIONS While careful weighting can mitigate noncoverage bias in landline telephone surveys, the rapid growth of cell-phone population and their distinctive characteristics suggest it is important to include a cell-phone sample. Moreover, the threat of noncoverage bias in telephone health survey estimates could mislead policy makers with possibly serious consequences for their ability to address important health policy issues.

Unit Nonresponse and Weighting Adjustments: A Critical Review

Abstract This article reviews unit nonresponse in cross-sectional household surveys, the consequences of the nonresponse on the bias of the estimates, and methods of adjusting for it. We describe the development of models for nonresponse bias and their utility, with particular emphasis on the role of response propensity modeling and its assumptions. The article explores the close connection between data collection protocols, estimation strategies, and the resulting nonresponse bias in the estimates. We conclude with some comments on the current state of the art and the need for future developments that expand our understanding of the response phenomenon.

Statistical and Practical Issues in the Design of a National Probability Sample of Births for the Vanguard Study of the National Children's Study

The National Childrens Study is a national household probability sample designed to identify 100,000 children at birth and follow the sampled children for 21 years. Data from the study will support examining numerous hypotheses concerning genetic and environmental effects on the health and development of children. The goals of the study present substantial challenges. For example, the need for preconception, prenatal, and postnatal data requires identifying women in the early stages of pregnancy, the collection of many types of data, and the retention of the children over time. In this paper, we give an overview of the sample design used in a pilot study called the Vanguard Study, and highlight the approaches used to address these challenges. We will also describe the rationale for the sampling choices made at each stage, the unique organizational structure of the NCS and issues we expect to face during implementation.

Estimating Residency Rates for Undetermined Telephone Numbers

The method for estimating residency rates in random digit dial (RDD) telephone surveys is important for computing response rates. This article reviews existing methods of estimating residency rates and introduces a new survival method that takes advantage of more information to provide improved estimates. Examples of applying this to large RDD samples are given along with suggestions for use of the method in other surveys

Chapter 8 - Nonresponse and Weighting

Publisher Summary Nonresponse is the failure to obtain a valid response from a sampled unit. It is of concern to survey methodologists and practitioners because complete response is assumed by the randomization or design-based theory that allows inference from a sample to the target population. Nonresponse has the potential to introduce bias into survey estimates and reduce the precision of survey estimates. As a result, survey practitioners make efforts to minimize nonresponse and its effects on inferences from sample surveys. However, even with the best efforts, there will be nonresponse; hence, it is essential to understand its potential effects and methods that can be used for limiting these effects. This chapter discusses nonresponse in surveys, the reasons for nonresponse, and the methods used for increasing response rates in surveys. Response rates and review methods of computing response rates are defined, and the trends in response rates over time are examined.

Responsive Survey Designs for Reducing Nonresponse Bias

Abstract Survey researchers have been investigating alternative approaches to reduce data collection costs while mitigating the risk of nonresponse bias or to produce more accurate estimates within the same budget. Responsive or adaptive design has been suggested as one means for doing this. Falling survey response rates and the need to find effective ways of implementing responsive design has focused attention on the relationship between response rates and nonresponse bias. In our article, we re-examine the data compiled by Groves and Peytcheva (2008) in their influential article and show there is an important between-study component of variance in addition to the within-study variance highlighted in the original analysis. We also show that theory implies that raising response rates can help reduce the nonresponse bias on average across the estimates within a study. We then propose a typology of response propensity models that help explain the empirical findings, including the relative weak relationship between nonresponse rates and nonresponse bias. Using these results, we explore when responsive design tools such as switching modes, giving monetary incentives, and increasing the level of effort are likely to be effective. We conclude with some comments on the use of responsive design and weighting to control nonresponse bias.

Single-Phase Mail Survey Design for Rare Population Subgroups.

Although using random digit dialing (RDD) telephone samples was the preferred method for conducting surveys of households for many years, declining response and coverage rates have led researchers to explore alternative approaches. The use of address-based sampling (ABS) has been examined for sampling the general population and subgroups, most often using mail or invitations to web surveys. For surveying rare groups, these studies often involve multiple phases to identify eligible population members and then collect responses. We describe a new approach for surveying rare subgroups using ABS with a single-phase mail survey. The study incorporates a new method for using a list frame with the ABS frame to increase the yield of the subgroup of interest while avoiding a potential bias inherent in many dual-frame survey designs. The findings suggest that the new approach is both efficient and effective for identifying and measuring a rare population.