Geert Loosveldt

The Effects of Interviewer and Respondent Characteristics on Response Behavior in Panel Surveys A Multilevel Approach

Unit nonresponse in the second wave of a panel survey is related to several respondent characteristics and to the interviewer of that wave. More striking is the effect of the interviewer of the first wave, who is not involved in the second interview. To analyze both interviewer effects simultaneously, the authors use a multilevel cross-classified model. In that analysis, the effect of the interviewer of the second wave almost disappears. That effect turns out to be at least partly spurious due to a correlation of both interviewer effects. The authors conclude that the interviewer of the first interview is very important regarding participation in the subsequent waves of a panel survey.

The effect of personalization on response rates and data quality in web surveys

Personalizing correspondence has often shown to significantly increase survey response rates in mail surveys. This study experimentally tests whether personalization of email invitations acts correspondingly to web survey response rates. Also, it is investigated whether personalization influences data quality. The results of the study, using a large student sample, show that personalization significantly increases the web survey response rate by 8.6 percentage points. The data quality does not appear to be affected in any major way by personalizing the email invitations. However, the analyses do show that respondents of the personalization condition tend to respond with more social desirability bias to sensitive questions. Therefore, it is concluded that personalization has positive effects on the survey response rate, but one should carefully consider whether or not to personalize when a survey on sensitive topics is conducted.

Web surveys: the effect of controlling survey access using PIN numbers

Web surveys generally use some sort of access control to prevent uninvited respondents from taking part in the survey and to prevent multiple completions by the same (invited) respondent. Restriction of survey access can be accomplished in several technically equivalent ways. However, these methods may not be equivalent from a methodological viewpoint. In this article, an exploration of possible effects of access control mechanisms is undertaken. Two different modes of access control were experimentally manipulated. Five hundred respondents were assigned to the automatic login condition. Another 500 respondents were assigned to the manual login condition. It was expected that the automatic login procedure would generate higher response rates but a lower degree of data quality. The results show that using a manual login procedure does not decrease response rates, whereas it does increase the overall degree of data quality.

A Multilevel Multinomial Analysis of Interviewer Effects on Various Components of Unit Nonresponse

This note demonstrates an application of a multilevel multinomial model. We use that model to analyse interviewer effects on various components of unit nonresponse to a face-to-face survey: refusals and noncontacts. The model allows for an analysis of these two interviewer effects and a possible connection between both at the same time. Our results show that both the chances for refusals as for noncontacts are subject to interviewer effects. We also find some evidence for a relation between both interviewer effects: interviewers who obtain more refusals are also more likely to report noncontacts. That result is however at least partly dependent on an outlier, an interviewer with a high number of refusals and noncontacts.

Evaluating Relative Mode Effects in Mixed-Mode Surveys: Three methods to disentangle selection and measurement effects

In order to investigate the advantage of mixed-mode (MM) surveys, selection effects between the modes should be evaluated. Selection effects refer to differences in respondent compositions on the target variables between the modes. However, estimation of selection effects is not an easy task because they may be completely confounded with measurement effects between the modes (differences in measurement error). Publications concerning the estimation of these mode effects are scarce. This article presents and compares three methods that allow measurement effects and selection effects to be evaluated separately. The first method starts from existing publications that avoid the confounding problem by introducing a set of mode-insensitive variables into the analysis model. However, this article will show that this method involves unrealistic assumptions in most practical research. The second and the third methods make use of an MM sample extended by comparable single-mode data. The assumptions, advantages, and disadvantages of all three methods are discussed. Each method will further be illustrated using a set of six variables relating to opinions about surveys among the Flemish population. The results show large differences between the methods.

An evaluation of the semiautomatic login procedure to control web survey access

It is common practice to limit web survey access to individuals in the sample through the use of access codes such as personal identification numbers, user names, and passwords. Different technically equivalent methods exist to control survey access. However, these methods are not necessarily methodologically equivalent. In this study, an experimental design was set up to evaluate the effect of three login procedures: a manual, a semiautomatic, and an automatic login procedure. It was found that the automatic login produced the lowest data quality, whereas the manual and semiautomatic login produced data of a higher and comparable level of quality. Nevertheless. the manual and the semiautomatic login procedures differ from each other in certain respects. The study concludes that the semiautomatic login is a useful alternative to control web survey access.

Explaining the intention to participate in a web survey: a test of the theory of planned behaviour

Even though web surveys have become increasingly popular, considerable efforts are necessary to obtain acceptable response rates. This explains the proliferation of experiments aimed at improving levels of participation in web surveys. The current study is not aimed at increasing web survey response by means of experimental research, but instead uses a general psychological theory to explain web survey response. More specifically, the theory of planned behaviour (TPB) is used to explain the intention to participate in a web survey. Previous studies have used this theory to explain survey participation in specific populations (students), but the current study extends that scope by targeting a general population. The results show that the TPB is capable of explaining people’s intentions to participate in a web survey in the context of a general population.

Evaluating Mode Effects in Mixed-Mode Survey Data Using Covariate Adjustment Models

Abstract The confounding of selection and measurement effects between different modes is a disadvantage of mixed-mode surveys. Solutions to this problem have been suggested in several studies. Most use adjusting covariates to control selection effects. Unfortunately, these covariates must meet strong assumptions, which are generally ignored. This article discusses these assumptions in greater detail and also provides an alternative model for solving the problem. This alternative uses adjusting covariates, explaining measurement effects instead of selection effects. The application of both models is illustrated by using data from a survey on opinions about surveys, which yields mode effects in line with expectations for the latter model, and mode effects contrary to expectations for the former model. However, the validity of these results depends entirely on the (ad hoc) covariates chosen. Research into better covariates might thus be a topic for future studies.

The impact of respondents and interviewers on interview speed in face-to-face interviews

In surveys carried out by interviewers trained according to the key principle of standardized interviewing it is assumed that the interviewer has only limited impact on the time a respondent needs to answer questions. In the paper the effects of interviewers and respondent characteristics on interview speed are analyzed simultaneously by means of a three-level random coefficient model. Data from the fifth round of the European Social Survey (ESS) are used. In twelve participating countries (CAPI) timers were implemented at several places in the questionnaire. Based on this time information the interview speed (number of questions asked per minute) was measured for each respondent during five modules of the questionnaire. The results support most of the expectations concerning the effects of the respondent characteristics. However, the results also indicate that, for all countries, interviewers strongly determine the interview speed and that interview length is not a simple linear function of the number of questions in a questionnaire.

Who Responds to Incentives

In one year’s time, the number of respondents in the Flemish Survey of Cultural Shifts expressing a wish to be paid for participating in surveys doubled (28% in 2002, compared to 14% in 2001). Analyses show this is mainly the result of survey-specific characteristics such as length of the survey, invasion of privacy issues, and cognitive efforts needed to fill in the questionnaire. Person-specific characteristics such as the level of individualistic attitudes and the age of the respondent also have an effect but less so than the survey-related issues.