Richard Valliant

Estimation for Volunteer Panel Web Surveys Using Propensity Score Adjustment and Calibration Adjustment

A combination of propensity score and calibration adjustment is shown to reduce bias in volunteer panel Web surveys. In this combination, the design weights are adjusted by propensity scores to correct for selection bias due to nonrandomized sampling. These adjusted weights are then calibrated to control totals for the target population and correct for coverage bias. The final set of weights is comprised of multiple components, and the estimator of a total no longer takes a linear form. Therefore, approximate methods are needed to derive variance estimates. This study compares three variance estimation methods through simulation. The first method resembles what is used in commercial statistical software based on squared residuals. The second approach uses a variance estimator originally derived for the generalized regression estimator. The third method uses jackknife replication. Results indicate bias reduction is crucial for valid variance estimation and favor the replication method over the other approaches.

Poststratification and conditional variance estimation

Poststratification estimation is a technique used in sample surveys to improve efficiency of estimators. Survey weights are adjusted to force the estimated numbers of units in each of a set of estimation cells to be equal to known population totals. The resulting weights are then used in forming estimates of means or totals of variables collected in the survey. For example, in a household survey the estimation cells may be based on age/race/sex categories of individuals, and the known totals may come from the most recent population census. Although the variance of a poststratified estimator can be computed over all possible sample configurations, inferences made conditionally on the achieved sample configuration are desirable. Theory and a simulation study using data from the U.S. Current Population Survey are presented to study both the conditional bias and variance of the poststratified estimator of a total. The linearization, balanced repeated replication, and jackknife variance estimators are also exa...

Estimating Propensity Adjustments for Volunteer Web Surveys

Panels of persons who volunteer to participate in Web surveys are used to make estimates for entire populations, including persons who have no access to the Internet. One method of adjusting a volunteer sample to attempt to make it representative of a larger population involves randomly selecting a reference sample from the larger population. The act of volunteering is treated as a quasi-random process where each person has some probability of volunteering. One option for computing weights for the volunteers is to combine the reference sample and Web volunteers and estimate probabilities of being a Web volunteer via propensity modeling. There are several options for using the estimated propensities to estimate population quantities. Careful analysis to justify these methods is lacking. The goals of this article are (a) to identify the assumptions and techniques of estimation that will lead to correct inference under the quasi-random approach, (b) to explore whether methods used in practice are biased, and (c) to illustrate the performance of some estimators that use estimated propensities. Two of our main findings are (a) that estimators of means based on estimates of propensity models that do not use the weights associated with the reference sample are biased even when the probability of volunteering is correctly modeled and (b) if the probability of volunteering is associated with analysis variables collected in the volunteer survey, propensity modeling does not correct bias.

Generalized Variance Functions in Stratified Two-Stage Sampling

Abstract Generalized variance functions (GVFs) are used in a number of sample surveys as a convenient method of publishing sampling errors. The method consists of estimating the relative variance (relvariance) of an estimator of a total T by using a model with the form a + b/T. Using the prediction approach to finite population sampling, some asymptotic theory is presented for estimators of totals that are linear combinations of sample cluster means from stratified, two-stage cluster samples. One choice of GVF estimator is shown to be consistent under a particular class of prediction models. The theory is illustrated by an empirical study in which two-stage stratified samples are selected from a population of households. The prediction model is one in which units within a stratum have a common mean and variance, units in the same cluster are correlated but units in different clusters are not, and in which the common variance is a quadratic function of the common mean in a stratum. Bernoulli and Poisson r...

An application of mathematical programming to sample allocation

Abstract The problem of sample allocation in multipurpose surveys is complicated by the fact that an efficient allocation for some estimates may be inefficient for others. There may also be precision goals that must be met for certain estimates plus constraints on costs and minimum sample sizes for strata to permit variance estimation. These requirements lead to formulating the allocation problem as one of mathematical programming with an objective function and constraints that are nonlinear in the sample size target variables. We discuss a flexible approach for a two-stage sample allocation that uses multicriteria optimization programming. Software was developed to permit survey designers to easily explore alternative problem formulations and to compare the resulting allocations. The method is illustrated using a business establishment survey that estimates the costs to employers of providing wages and benefits to employees and the percentages of employees that receive certain benefits.

Nonlinear Prediction Theory and the Estimation of Proportions in a Finite Population

Abstract The prediction approach to finite population inference is developed for a general nonlinear model. An estimator of the finite population total and an estimator of its variance are derived, and the asymptotic properties of both are obtained when the random variables in the model are independent. The theory is applied to the problem of estimating the total number of units that have a specified characteristic. An empirical study is presented, which confirms that a nonlinear Bernoulli model is potentially useful for that problem but also illustrates difficulties that may be encountered. Comparisons with the ratio and linear regression estimators are also included.

Partial misspecification of survey design features sufficed to severely bias estimates of health-related outcomes

OBJECTIVE Surveys frequently deviate from simple random sampling through the use of unequal probability sampling, stratified sampling, and multistage sampling. This work uses a survey of public health to systematically illustrate the effects of incompletely accounting for strata, clustering, and weights. STUDY DESIGN AND SETTING Data analysis was based on the Study of Health in Pomerania (n=4,308, 20-79 years), a two-stage regional survey with high sampling fractions at the first stage. Effects of survey design features comprising weights, stratification, clustering, and finite population correction on point and variance estimates of lifestyle indicators and clinical parameters were assessed. RESULTS Misspecifications of the survey design substantially affected both the point estimates of health characteristics and their standard errors (SEs). The strongest bias in SEs concerned the omission of the second sampling stage. Ignoring the sampling design led to minor differences in variance estimates from the complete setup. Weighting predominantly affected point estimates of lifestyle factors. CONCLUSION A partial misspecification of survey design elements may bias variance estimates severely and is sometimes even more harmful compared with completely neglecting design elements. If subgroups are sampled at different rates, weighting is of particular relevance with regard to prevalence estimates of lifestyle indicators.

Efficient Use of Commercial Lists in U.S. Household Sampling

Sampling households using commercial lists has the potential to reduce costs and to efficiently identify some subgroups for which target sample sizes are desired. However, the information on the lists for demographics like age is usually incomplete and inaccurate. We demonstrate that this inexact information can still be used to improve the efficiency with which some, but not all, demographic subgroups can be located during sampling. The paper also illustrates the use of nonlinear programming as a means for finding sample allocations that are subject to a variety of practical constraints. A commercial address list and data from the National Survey of Family Growth and the Health and Retirement Study are used to illustrate the calculation of allocations to strata of housing units defined by information on the list.

Conditional properties of some estimators in stratified sampling

Abstract The prediction properties of the stratified expansion estimator, the separate and combined ratio estimators, and the separate and combined regression estimators are studied under a model appropriate to a population stratified on a size variable. Several estimators of variance for each total estimator are considered, including standard ones from probability sampling theory, alternative choices derived from a superpopulation model, and the jackknife. The theory is tested in an empirical study using a real population. Earlier studies of the ratio and regression estimators under simple random sampling plans have illustrated that conditional properties of those estimators and of the linearization variance estimators that are often used with them can be much different and less desirable than unconditional properties. Whether similar results hold for stratified samples and estimators has been the subject of some debate. This article illustrates both theoretically and empirically that the use of stratifi...

Inference for Nonprobability Samples

Although selecting a probability sample has been the standard for decades when making inferences from a sample to a finite population, incentives are increasing to use nonprobability samples. In a world of “big data”, large amounts of data are available that are faster and easier to collect than are probability samples. Design-based inference, in which the distribution for inference is generated by the random mechanism used by the sampler, cannot be used for nonprobability samples. One alternative is quasi-randomization in which pseudo-inclusion probabilities are estimated based on covariates available for samples and nonsample units. Another is superpopulation modeling for the analytic variables collected on the sample units in which the model is used to predict values for the nonsample units. We discuss the pros and cons of each approach.