Andy Peytchev

Experiments in Mobile Web Survey Design

Self-administered surveys can be conducted on mobile web-capable devices, yet these devices have unique features that can affect response processes. Ninety-two adults were randomly selected and provided with mobile devices to complete weekly web surveys. Experiments were designed to address three main objectives. First, the authors test fundamental findings which have been found robust across other modes, but whose impact may be diminished in mobile web surveys (due largely to the device), by manipulating question order and scale frequencies. Second, the authors test findings from experiments in computer-administered web surveys, altering the presentation of images and the number of questions per page. Third, the authors experiment with the unique display, navigation, and input methods, through the need to scroll, the vertical versus horizontal orientation of scales, and the willingness to provide open-ended responses. Although most findings from other modes are upheld, the small screen and keyboard introduce undesirable differences in responses.

A Typology of Real-Time Validations in Web-Based Surveys

One area in computer-assisted self-interviewing (CASI) where methodological and empirical research have not caught up with technological advancements is real-time validation of respondent input. Some of the literature on computer-assisted interviewing and postsurvey editing is pertinent yet far insufficient—the largest component that could not be deduced being the respondent interaction. This leads to the employment or avoidance of validations by current practitioners based on untested assumptions. This article presents a model that breaks down the different elements in survey validation by type of computation, implementation, and interaction with the respondent. The purpose is twofold: to describe what possibilities exist in validations for web-based surveys and to present a framework in which systematic research could be conducted to evaluate the impact of validations on survey costs and errors. Relevant findings from prior literature are discussed in this context.

The Effects of Differential Interviewer Incentives on a Field Data Collection Effort

Surveys routinely offer incentives to motivate respondents and increase the likelihood of their participation, yet surprisingly little is known about the effectiveness of interviewer incentives. If interviewer incentives increase interviewers’ success in gaining cooperation, they could help address declining survey response rates. In this article, we present the results of an experiment testing the effectiveness of interviewer incentives in the form of cash bonuses for each successfully completed field interview. We did not find evidence that higher payments to interviewers for each completion led to increased effort on the part of interviewers nor did they lead to higher levels of success in securing respondent cooperation. These findings suggest that per complete interviewer incentives may not be cost effective in reducing survey nonresponse.

Coverage Bias in Variances, Associations, and Total Error From Exclusion of the Cell Phone-Only Population in the United States

Although landline telephone household surveys often draw inference about the general population, a proportion with only cell phones is excluded. In the United States, like in much of the world, this proportion is substantial and increasing, providing potential for coverage bias. Studies have looked at bias in means, but undercoverage can affect other essential statistics. The precision of point estimates can be biased, leading to erroneous conclusions. Research examining multivariate relationships will be further affected by bias in associations. A national landline telephone survey was conducted, followed by a survey of adults with only cell phones. In addition to estimates of means and proportions, differences were found for variances and associations. Bias in some point estimates was reduced through poststratification but became larger and in opposite direction for others. Different uses of survey data can be affected by omitting the cell-only population, and reliance on postsurvey adjustments can be misleading.

Financial Record Checking in Surveys Do Prompts Improve Data Quality

Self-reports of financial information in surveys, such as wealth, income, and assets, are particularly prone to inaccuracy. We sought to improve the quality of financial information captured in a survey conducted by phone and in person by encouraging respondents to check records when reporting on income and assets. We investigated whether suggestive prompts influenced unit response, compliance with the request to check records, precision of estimates, and accuracy. We conducted a split sample experiment in the Community Advantage Panel Survey in which half of telephone respondents and half of in-person household interview respondents were encouraged to check the records. We found a modest positive effect of prompts on compliance but no effect on unit response, precision, or accuracy.