Annelies G. Blom

Interviewer Effects on Nonresponse in the European Social Survey

In face-to-face surveys interviewers play a crucial role in making contact with and gaining cooperation from sample units. While some analyses investigate the influence of interviewers on nonresponse, they are typically restricted to single-country studies. However, interviewer training, contacting and cooperation strategies as well as survey climates may differ across countries. Combining call-record data from the European Social Survey (ESS) with data from a detailed interviewer questionnaire on attitudes and doorstep behavior we find systematic country differences in nonresponse processes, which can in part be explained by differences in interviewer characteristics, such as contacting strategies and avowed doorstep behavior.

Setting Up an Online Panel Representative of the General Population: The German Internet Panel

This article looks into the processes and outcomes of setting up and maintaining a probability-based longitudinal online survey, which is recruited face-to-face and representative of both the online and the offline population aged 16–75 in Germany. This German Internet Panel studies political and economic attitudes and reform preferences through bimonthly longitudinal online interviews of individuals. The results presented here demonstrate that a carefully designed and implemented online panel can produce high-quality data at lower marginal costs than existing panels that operate solely in a face-to-face mode. Analyses into the representativeness of the online sample showed no major coverage or nonresponse biases. Finally, including offline households in the panel is important as it improves the representation of the older and female segments of the population.

A Comparison of Four Probability-Based Online and Mixed-Mode Panels in Europe

Inferential statistics teach us that we need a random probability sample to infer from a sample to the general population. In online survey research, however, volunteer access panels, in which respondents self-select themselves into the sample, dominate the landscape. Such panels are attractive due to their low costs. Nevertheless, recent years have seen increasing numbers of debates about the quality, in particular about errors in the representativeness and measurement, of such panels. In this article, we describe four probability-based online and mixed-mode panels for the general population, namely, the Longitudinal Internet Studies for the Social Sciences (LISS) Panel in the Netherlands, the German Internet Panel (GIP) and the GESIS Panel in Germany, and the Longitudinal Study by Internet for the Social Sciences (ELIPSS) Panel in France. We compare them in terms of sampling strategies, offline recruitment procedures, and panel characteristics. Our aim is to provide an overview to the scientific community of the availability of such data sources to demonstrate the potential strategies for recruiting and maintaining probability-based online panels to practitioners and to direct analysts of the comparative data collected across these panels to methodological differences that may affect comparative estimates.

Does the Recruitment of Offline Households Increase the Sample Representativeness of Probability-Based Online Panels? Evidence From the German Internet Panel

The past decade has seen a rise in the use of online panels for conducting survey research. However, the popularity of online panels, largely driven by relatively low implementation costs and high rates of Internet penetration, has been met with criticisms regarding their ability to accurately represent their intended target populations. This criticism largely stems from the fact that (1) non-Internet (or offline) households, despite their relatively small size, constitute a highly selective group unaccounted for in Internet panels, and (2) the preeminent use of nonprobability-based recruitment methods likely contributes a self-selection bias that further compromises the representativeness of online panels. In response to these criticisms, some online panel studies have taken steps to recruit probability-based samples of individuals and providing them with the means to participate online. Using data from one such study, the German Internet Panel, this article investigates the impact of including offline households in the sample on the representativeness of the panel. Consistent with studies in other countries, we find that the exclusion of offline households produces significant coverage biases in online panel surveys, and the inclusion of these households in the sample improves the representativeness of the survey despite their lower propensity to respond.

Measuring Interviewer Characteristics Pertinent to Social Surveys: A Conceptual Framework

Interviewer effects are found across all types of interviewer-mediated surveys crossing disciplines and countries. While studies describing interviewer effects are manifold, identifying characteristics explaining these effects has proven difficult due to a lack of data on the interviewers. This paper proposes a conceptual framework of interviewer characteristics for explaining interviewer effects and its operationalization in an interviewer questionnaire. The framework encompasses four dimensions of interviewer characteristics: interviewer attitudes, interviewers’ own behaviour, interviewers’ experience with measurements, and interviewers’ expectations. Our analyses of the data collected from interviewers working on the fourth wave of SHARE Germany show that the above measures distinguish well between interviewers.

Generalization of Classic Question Order Effects across Cultures

Questionnaire design is routinely guided by classic experiments on question form, wording, and context conducted decades ago. This article explores whether two question order effects (one due to the norm of evenhandedness and the other due to subtraction or perceptual contrast) appear in surveys of probability samples in the United States and 11 other countries (Canada, Denmark, Germany, Iceland, Japan, the Netherlands, Norway, Portugal, Sweden, Taiwan, and the United Kingdom; N = 25,640). Advancing theory of question order effects, we propose necessary conditions for each effect to occur, and found that the effects occurred in the nations where these necessary conditions were met. Surprisingly, the abortion question order effect even appeared in some countries in which the necessary condition was not met, suggesting that the question order effect there (and perhaps elsewhere) was not due to subtraction or perceptual contrast. The question order effects were not moderated by education. The strength of the effect due to the norm of evenhandedness was correlated with various cultural characteristics of the nations. Strong support was observed for the form-resistant correlation hypothesis.

The Influence of a Person’s Digital Affinity on Unit Nonresponse and Attrition in an Online Panel:

Research has shown that the non-Internet population is hesitant to respond to online survey requests. However, also subgroups in the Internet population with low digital affinity may hesitate to respond to online surveys. This latter issue has not yet received much attention by scholars despite its potentially detrimental effects on the external validity of online survey data. In this article, we explore the extent to which a person’s digital affinity contributes to nonresponse bias in the German Internet Panel, a probability-based online panel of the general population. With a multidimensional classification of digital affinity, we predict response to the first online panel wave and participation across panel waves. We find that persons who belong to different classes of digital affinity have systematically different sociodemographic characteristics and show different voting behavior. In addition, we find that initial response propensities vary by classes of digital affinity, as do attrition patterns over time. Our results demonstrate the importance of digital affinity for the reduction in nonresponse bias during fieldwork and for postsurvey adjustments.

Can Incentive Effects in Web Surveys Be Generalized to Non-Western Countries? Conditional and Unconditional Cash Incentives in a Web Survey of Ghanaian University Students:

Because research on the impact of web survey incentives has exclusively focused on Western settings, it is unclear to what extent current insights translate and generalize to non-Western societies, which are usually characterized by very different economic conditions, cultural traditions, and survey climates. The current article presents the results of a web survey incentives experiment among almost 4,440 Ghanaian university students who were offered conditional and unconditional incentives of different values (in the form of telephone credit). Our analyses partly replicate Western findings: Higher value incentives produce higher participation rates and unconditional incentives outperform conditional ones in the lower value conditions. In the case of relatively high incentives, however, conditional outperforms unconditional incentives. No differential effects of incentives on response quality were found.

Linking Survey Data to Administrative Records in a Comparative Survey Context

Linking survey data to external databases that contain records pertaining to the same respondents holds substantial promise for survey researchers. There are a number of methodological challenges with generating the links between the data sources, but there may also be administrative challenges in linking the datasets. This issue becomes particularly important when conducting cross-national research where research standards and requirements may vary substantially across countries. This chapter provides an overview of process of linking in a comparative context and highlights several issues that researchers are likely to encounter when conducting this type of work.

Face-to-face Rekrutierung für ein probabilistisches Onlinepanel

Onlineumfragen haben fur die sozialwissenschaftliche Forschung in den vergangen Jahren zunehmend an Bedeutung gewonnen, nicht zuletzt wegen der Moglichkeit, viele Personen innerhalb kurzer Zeit kosteneffizient befragen zu konnen. Im Vergleich zu interviewer-administrierten Befragungen haben Onlineinterviews aufgrund der selbst-administrierten Befragung einen geringeren Messfehler (Kreuter et al. 2008). Hauptkritikpunkt bei Onlineumfragen ist die Reprasentativitat dieser Studien im Hinblick auf die Allgemeinbevolkerung.