Phillip S. Kott

A Note on Handling Nonresponse in Sample Surveys

Abstract Two distinct types of models are used for handling nonresponse in survey sampling theory. In a response (or quasi-randomization) model, the propensity of survey response is modeled as a random process, an additional phase of sample selection. In a parametric (or superpopulation) model, the survey data are themselves modeled. These two models can be used simultaneously in the estimation of a population mean so that one provides some protection against the potential for failure in the other. Two different estimators are discussed in this article. The first is a regression estimator that is both unbiased under the parametric model and nearly quasi-design unbiased under the response model. The second is a direct expansion estimator with imputed missing values. The imputed values are such that the estimator is both nearly quasi-design unbiased and unbiased under the combination of the parametric model and the original sampling design. The article includes a discussion of variance estimation with the g...

Estimating the conditional variance of a design consistent regression estimator

Abstract A design consistent regression estimator provides the means for constructing a finite population parameter estimate that is conditionally unbiased under a model, yet resistant to model failure (i.e., design consistent). In a similar fashion a model unbiased conditional variance estimator for the parameter estimate can be developed that is also a design consistent estimator of design mean squared error.

On voluntary and volunteer government surveys in the United States

Although many US government surveys are mandatory, most are voluntary and more than a few can be considered volunteer, a number that is likely to increase in the future. We will discuss the repercussions of this on the quality of those surveys and discuss how calibration weighting can be used to limit the potential bias on resulting estimators.

Calibration Weighting When Model and Calibration Variables Can Differ

Calibration weighting is an easy-to-implement yet powerful tool for reducing the standard errors of many population estimates derived from a sample survey by forcing the weighted sums of certain “calibration” variables to equal their known (or better-estimated) population totals. Although originally developed to reduce standard errors, calibration weighting can also be used to reduce or remove selection biases resulting from unit nonresponse. To this end, nonrespondents are usually assumed to be “missing at random,” that is, the response mechanism is assumed to be a function of calibration variables with either known values in the entire sample or known population totals. It is possible, however, to use calibration-weighting to compensate for unit nonresponse when response is a function of model variables that need not be calibration variables; in fact, some model variables can have values known only for respondents. We will explore some recent findings connected with this methodology.

Developing Calibration Weights and Standard-Error Estimates for a Survey of Drug-Related Emergency-Department Visits

Abstract This article describes a two-step calibration-weighting scheme for a stratified simple random sample of hospital emergency departments. The first step adjusts for unit nonresponse. The second increases the statistical efficiency of most estimators of interest. Both use a measure of emergency-department size and other useful auxiliary variables contained in the sampling frame. Although many survey variables are roughly a linear function of the measure of size, response is better modeled as a function of the log of that measure. Consequently the log of size is a calibration variable in the nonresponse-adjustment step, while the measure of size itself is a calibration variable in the second calibration step. Nonlinear calibration procedures are employed in both steps. We show with 2010 DAWN data that estimating variances as if a one-step calibration weighting routine had been used when there were in fact two steps can, after appropriately adjusting the finite-population correct in some sense, produce standard-error estimates that tend to be slightly conservative.

Evaluating the Effective Degrees of Freedom of the Delete-a-Group Jackknife

The delete-a-group jackknife is sometimes used when estimating the variances of statistics based on a large sample. We investigate heavily poststratified estimators for a population mean and a simple regression coefficient, where both full-sample and domain estimates are of interest. The delete-a-group (DAG) jackknife employing 30, 60, and 100 replicates is found to be highly unstable, even for large sample sizes. The empirical degrees of freedom of these DAG jackknives are usually much less than their nominal degrees of freedom. This analysis calls into question whether coverage intervals derived from replication-based variance estimators can be trusted for highly calibrated estimates.

Speeding up the asymptotics when constructing one-sided coverage intervals with survey data

SummaryCoverage intervals for a parameter are frequently derived from a survey sample by assuming that the randomization-based parameter estimate is asymptotically normal and that the associated measure of the estimator’s variance is roughly chi-squared. In many situations, however, the size of the sample and the nature of the parameter being estimated render the conventional Wald technique dubious, especially when a one-sided coverage interval is needed. We will propose a method of coverage-interval construction that “speeds up the asymptotics” so that the resulting one-sided intervals can have much better coverage properties than corresponding Wald intervals. For the important case of a mean computed from a stratified, simple random sample with or without replacement, no model need be assumed. A simulation demonstrates the usefulness of our intervals.

Robust variance estimation in linear regression

The standard technique for estimating the variance of a linear regression coefficient is unbiased when the random errors of the observational units are independent and identically distributed. When the unit variances are not all equal, however, as is often the case in practice, this method can be biased. An unbiased variance estimator given uncorrelated, but not necessarily homoscedastic, unit errors is introduced here and compared to the conventional technique using real data.

Calibration Weighting for Nonresponse with Proxy Frame Variables (So that Unit Nonresponse Can Be Not Missing at Random)

Abstract When adjusting for unit nonresponse in a survey, it is common to assume that the response/nonresponse mechanism is a function of variables known either for the entire sample before unit response or at the aggregate level for the frame or population. Often, however, some of the variables governing the response/nonresponse mechanism can only be proxied by variables on the frame while they are measured (more) accurately on the survey itself. For example, an address-based sampling frame may contain area-level estimates for the median annual income and the fraction home ownership in a Census block group, while a household’s annual income category and ownership status are reported on the survey itself for the housing units responding to the survey. A relatively new calibration-weighting technique allows a statistician to calibrate the sample using proxy variables while assuming the response/ nonresponse mechanism is a function of the analogous survey variables. We will demonstrate how this can be done with data from the Residential Energy Consumption Survey National Pilot, a nationally representative web-and-mail survey of American households sponsored by the U.S. Energy Information Administration.