Peter Lynn

Methodology of longitudinal surveys

This book: Covers all the main stages in the design, implementation and analysis of longitudinal surveys. Reviews recent developments in the field, including the use of dependent interviewing and mixed mode data collection.

Separating refusal bias and non-contact bias: evidence from UK national surveys

Refusal by sample units and failure to contact sample units can both contribute to survey non-response bias. However, the nature of the contribution can be rather different in the two cases. Extended field efforts may reduce either or both components of non-response bias. We examine data from several UK surveys on extended field efforts and the effect that they appear to have on non-response bias and its components. Some consistent patterns are found. We also explore the sensitivity of such analyses to the operational definition of extended efforts. The findings provide some evidence of the relative roles of refusal conversion and repeated contact attempts in reducing non-response bias. The study also has implications for the development of appropriate field strategies to combat non-response bias.

Developing quality standards for cross-national survey research: five approaches

This paper addresses methods for the development of quality standards for cross-national quantitative surveys. Five conceptual approaches to the development of standards are presented and the properties and consequences of each are discussed. The aim is to provide researchers with a rational set of arguments for choosing a particular approach in a given situation. The development of quality standards is particularly complicated in the case of cross-national research, owing to variability between nations in a number of factors that might typically be considered as ‘fixed’ design constraints in the context of single-country studies. This variability is illustrated by a number of examples that arose in the course of development of the European Social Survey. The paper also touches upon some related issues in the implementation and maintenance of quality standards.

PEDAKSI: Methodology for collecting data about survey non-respondents

The effects of unit non-response on survey errors are of great concern to researchers.However, direct assessment of non-response bias in survey estimates is rarely possible.Attempts are often made to adjust for the effects of non-response by weighting, but thisusually relies on the use of frame data or external population data, which are at bestmodestly correlated with the survey variables. This paper reports the development ofa method to collect limited survey data from non-respondents to personal interviewsurveys and a large-scale field test of the method on the British Crime Survey (BCS).The method is shown to be acceptable and low cost, to provide valid data, and to haveno detrimental effect on the main survey. The use of the resultant data to estimatenon-response bias is illustrated and some substantive conclusions are drawn for the BCS.

Methods for Probability-Based Online and Mixed-Mode Panels

This special issue is devoted to discussion of probability-based survey panels that collect data either solely or partly through online questionnaires. Panels of this kind have been around for a long time, though they have been few in number, but recent years have seen several new panels start up in Europe. This has led to renewed interest in the methodology of such panels and also to deeper questioning of the role of these panels. On one hand, the probability-based panels are to some extent competing against cheaper non-probability access panels. On the other hand, the probability-based panels are increasingly being seen as possible alternatives to more expensive probability-based survey methods. In both cases, clients and data users want to better understand the relative advantages and disadvantages of the probability-based panels.

The feasibility of linking household survey and administrative record data: New evidence for Britain

Linkage of household survey responses with administrative data is increasingly on the agenda. Unique individual identifiers have clear benefits for making linkages but are also subject to problems of survey item non-response and measurement error. Our experimental study that linked survey responses to UK government agency records on benefits and tax credits elucidates this trade-off. We compare five linkage criteria: one based on a respon-dent-supplied National Insurance Number (NINO) and the other four using different combinations of sex, name, address and date of birth. As many linkages were made using non-NINO-based matches as were made using matches on NINO and the former were also relatively accurate when assessed in terms of false-positive and false-negative linkage rates. The potential returns from hierarchical and pooled matching are also examined.

Random-digit dialling in the UK: viability revisited

A main obstacle to using telephone methods for social surveys in the UK has been the inability to select a representative and unbiased probability sample of the general population. However, recent changes in the UK telephone numbering system and advances in telephony software appear to provide the means for using random-digit dialling (RDD). This paper presents the results of a study designed to test the feasibility of RDD. The focus is on the sampling method, but we also assess other aspects of the RDD survey method, in the context of a total survey error framework. Although we conclude that RDD is now a viable survey method in the UK, further research and development is advocated to improve the efficiency of fieldwork and response rates.

Extended Field Efforts to Reduce the Risk of Non-response Bias Have the Effects Changed over Time? Can Weighting Achieve the Same Effects?

We examine the effects on survey estimates of extended interviewer efforts to gain survey response, including refusal conversion attempts and attempts to make contact with difficult-to-contact sample members. Previous research on this topic has identified that extended efforts do appear to affect estimates, and in ways that seem consistent with bias reduction. We extend the previous research in three ways. First, we provide the first study of changes over time in the effects of extended efforts on estimates. We study change in the UK over a ten-year period. Second, we use a more precise measure of the difficulty of contact and third, we assess the effects of extended efforts conditional on weight adjustments for non-response estimates as well as on unweighted sample statistics.

On the Bias and Variance of Samples of Individuals: A Comparison of the Electoral Registers and Postcode Address File as Sampling Frames

General population surveys in Britain tend to use either the electoral registers (ERs) or the postcode address file (PAF) as a sampling frame of addresses, with subsequent selection of one unit (household or individual) per address. The PAF has better coverage, and is clearly the superior frame for sampling households, but the ER has an important advantage when sampling individuals: selection probabilities can be made to vary less than with a PAF sample. This paper provides an evaluation of the relative merits of the two frames for sampling individuals. This is achieved by analysis of an attitude survey which employed a split frame sample design. The conclusion is that, for most variables investigated, the PAF provides estimates which are no less accurate than those from the ER, and that other advantages of the PAF suggest that it is likely to be the preferable frame for many purposes.

Sampling frame effects on the British Crime Survey

In 1992, the British Crime Survey used the postcode address file as the sampling frame for the first time, having previously used the electoral registers (ERs). This paper addresses concerns that the change in frame may have affected the survey data, particularly measures of change over time in victimization rates. The general conclusion is that the change may have caused increases in crime rates between 1988 and 1992 to be slightly overestimated, but only by an amount which is very small compared with other sources of survey error. Information is also provided on the nature of the biased coverage of the ERs, and the population subgroups that would be under-represented by surveys involving ER samples.