E. de Leeuw

Survey measurement and process quality

Partial table of contents: QUESTIONNAIRE DESIGN. From Theoretical Concept to Survey Question (J. Hox). Designing Rating Scales for Effective Measurement in Surveys (J. Krosnick & L. Fabrigar). DATA COLLECTION. Developing a Speech Recognition Application for Survey Research (B. Blyth). Children as Respondents: Methods for Improving Data Quality (J. Scott). POST SURVEY PROCESSING AND OPERATIONS. Integrated Control Systems for Survey Processing (J. Bethlehem). QUALITY ASSESSMENT AND CONTROL. Continuous Quality Improvement in Statistical Agencies (D. Morganstein & D. Marker). ERROR EFFECTS ON ESTIMATION, ANALYSES, AND INTERPRETATION. Categorical Data Analysis and Misclassification (J. Kuha & C. Skinner). Index.

Coverage Bias in European Telephone Surveys: Developments of Landline and Mobile Phone Coverage across Countries and over Time

With the decrease of landline phones in the last decade, telephone survey methodologists face a new challenge to overcome coverage bias. In this study we investigate coverage error for telephone surveys in Europe over time and compare two situations: classical surveys that rely on landline only with surveys that also include mobile phones. We analyzed Eurobarometer data, which are collected by means of face-to-face interviews and contain information on ownership of landline and mobile phones. We show that for the period 2000-2009, time has a significant effect on both mobile phone penetration and coverage bias. In addition, the countries’ development significantly affects the pace of these changes.

Latent Class Analysis of Respondent Scalability

The psychometric literature contains many indices to detect aberrant respondents. A different, promising approach is using ordered latent class analysis with the goal to distinguish latent classes of respondents that are scalable, from latent classes of respondents that are not scalable (i.e., aberrant) according to the scaling model adopted. This article examines seven Latent Class models for a cumulative scale. A simulation study was performed to study the efficacy of different models for data that follow the scale model perfectly. A second simulation study was performed to study how well these models detect aberrant respondents.

Mixed-Mode and Mixed-Device Surveys

Online surveys are one of the most prominent data collection methods in Europe and the USA. Not only are they fast and cheap, data quality in well-designed online surveys is high, especially when sensitive questions are asked. Disadvantages are the threat of undercoverage, as not everyone has Internet access, and high nonresponse. In order to overcome these disadvantages, mixed-mode designs are used in which multiple data collection methods are combined. The strength of mixed-mode surveys is their potential to reduce coverage and nonresponse error at affordable costs. However, survey modes may differ in the effect they have on measurement error, and one critical question is how data from different modes may be combined. A special form of mixed mode design is a mixed-device survey. Web surveys are increasingly completed on a range of different devices. Mobile phones, tablets, and even smart watches are being used in addition to regular desktop PCs. The question arises whether or not answers obtained via smartphones or tablets are comparable to answers via regular desktop PCs. Screen sizes are very different and also the mode of response entry varies between devices. Survey software increasingly adapts to mobile survey responding via responsive survey design. The software detects the device used for completing the survey and adapts the format accordingly. This chapter discusses the most common designs for mixed mode surveys and summarizes the empirical evidence for reducing coverage and nonresponse error and then focuses on measurement error. We pay special attention to careful design and provide rules for doing web surveys that are in fact mixed-device surveys.

Estimating the impact of measurement differences introduced by efforts to reach a balanced response among non-western minorities

This study investigates the impact of different modes and tailor-made response enhancing measures (TMREM)—such as bilingual interviewers with a shared ethnic background and translated questionnaires—on the measurement of substantive variables in surveys among minority ethnic groups in the Netherlands. It also provides insight into the ability to detect mode measurement effects of a recently developed method for disentangling mode measurement and mode selection effects, as well as into the tenability of the assumptions underlying this method. The data used in this study come from a large-scale survey design experiment among the four largest non-Western minority ethnic groups in the Netherlands comparing single-mode computer-assisted personal interviewing (CAPI) and sequential computer-assisted web interviewing, computer-assisted telephone interviewing, and CAPI-MM. The number and intensity of the TMREM varied among the four ethnic groups. The results show that mode measurement effects occur among all ethnic groups and are the result of a combination of the presence or absence of an interviewer and TMREM. Mode measurement effects occur more often on sociocultural questions, but also, on occasion, on more sociostructural or background questions. The method used to disentangle mode measurement and mode selection effects can be applied to detect mode measurement effects, but one should be cautious in interpreting them. Implausible mode measurement effects can be caused by the violation of the assumptions underlying this method.