Representativeness of Individual-Level Data in COVID-19 Phone Surveys: Findings from Sub-Saharan Africa

Abstract

The COVID-19 pandemic has created urgent demand for timely data, leading to a surge in mobile phone surveys for tracking the impacts of and responses to the pandemic. This paper assesses, and attempts to mitigate, selection biases in individual-level analyses based on phone survey data. The research uses data from (i) national phone surveys that have been implemented in Ethiopia, Malawi, Nigeria, and Uganda during the pandemic, and (ii) the pre-COVID-19 national face-to-face surveys that served as the sampling frames for the phone surveys. The availability of pre-COVID-19 face-to-face survey data permits comparisons of phone survey respondents with the general adult population. Phone survey respondents are more likely to be household heads or their spouses and non-farm enterprise owners, and on average, are older and better educated vis-a-vis the general adult population. To improve the representativeness of individual-level phone survey data, the household-level phone survey sampling weights are calibrated based on propensity score adjustments that are derived from a model of an individual’s likelihood of being interviewed as a function of individual- and household-level attributes. Reweighting improves the representativeness of the estimates for the phone survey respondents, moving them closer to those of the general adult population. This holds for women and men and a range of demographic, education, and labor market outcomes. However, reweighting increases the variance of the estimates and fails to overcome selection biases. Obtaining reliable data on men and women through phone surveys requires random selection of adult interviewees within sampled households.