Kent H. Marquis

Response Bias and Reliability in Sensitive Topic Surveys

Abstract Estimates of survey response bias and reliability are presented for six topics: receipt of welfare, income, alcohol use, drug use, criminal history, and embarrassing medical conditions. The estimates are derived from published full-design criterion validity studies. The common assumption that these characteristics are underreported is, in part, based on partial validity studies; the bias in estimating response parameters using partial designs is demonstrated. Evidence from the full-design studies suggests that the response biases for these topics center near zero but that the responses are unreliable or noisy. Implications for survey design and methodological research are considered.

The measurement of expenditures for outpatient physician and dental services: methodological findings from the health insurance study.

Survey data measuring outpatient health expenditure are evaluated for item nonresponse, measurement error, and bias. Item nonresponse is high for persons whose health care is financed through the public sector, but is otherwise manageable. Estimates of mean total and out-of-pocket physician and dental expenditure from two surveys using indirect methods (including the Health Insurance Study) are compared with nonsurvey estimates. Out-of-pocket physician expenditure is overestimated, but the other measures appear unbiased. Estimates using direct, self-administered methods appear biased upward. We demonstrate that commonly used record check methods for evaluating survey data will make random error appear as bias, and a methodology we develop shows substantial random error in the measurement of dental expenditure: 44 per cent of the total variance in survey data and 39 per cent of the variance in records.