Monica Pratesi

Small area estimation: the EBLUP estimator based on spatially correlated random area effects

This paper deals with small area indirect estimators under area level random effect models when only area level data are available and the random effects are correlated. The performance of the Spatial Empirical Best Linear Unbiased Predictor (SEBLUP) is explored with a Monte Carlo simulation study on lattice data and it is applied to the results of the sample survey on Life Conditions in Tuscany (Italy). The mean squared error (MSE) problem is discussed illustrating the MSE estimator in comparison with the MSE of the empirical sampling distribution of SEBLUP estimator. A clear tendency in our empirical findings is that the introduction of spatially correlated random area effects reduce both the variance and the bias of the EBLUP estimator. Despite some residual bias, the coverage rate of our confidence intervals comes close to a nominal 95%.

M-quantile models with application to poverty mapping

Over the last decade there has been growing demand for estimates of population characteristics at small area level. Unfortunately, cost constraints in the design of sample surveys lead to small sample sizes within these areas and as a result direct estimation, using only the survey data, is inappropriate since it yields estimates with unacceptable levels of precision. Small area models are designed to tackle the small sample size problem. The most popular class of models for small area estimation is random effects models that include random area effects to account for between area variations. However, such models also depend on strong distributional assumptions, require a formal specification of the random part of the model and do not easily allow for outlier robust inference. An alternative approach to small area estimation that is based on the use of M-quantile models was recently proposed by Chambers and Tzavidis (Biometrika 93(2):255–268, 2006) and Tzavidis and Chambers (Robust prediction of small area means and distributions. Working paper, 2007). Unlike traditional random effects models, M-quantile models do not depend on strong distributional assumption and automatically provide outlier robust inference. In this paper we illustrate for the first time how M-quantile models can be practically employed for deriving small area estimates of poverty and inequality. The methodology we propose improves the traditional poverty mapping methods in the following ways: (a) it enables the estimation of the distribution function of the study variable within the small area of interest both under an M-quantile and a random effects model, (b) it provides analytical, instead of empirical, estimation of the mean squared error of the M-quantile small area mean estimates and (c) it employs a robust to outliers estimation method. The methodology is applied to data from the 2002 Living Standards Measurement Survey (LSMS) in Albania for estimating (a) district level estimates of the incidence of poverty in Albania, (b) district level inequality measures and (c) the distribution function of household per-capita consumption expenditure in each district. Small area estimates of poverty and inequality show that the poorest Albanian districts are in the mountainous regions (north and north east) with the wealthiest districts, which are also linked with high levels of inequality, in the coastal (south west) and southern part of country. We discuss the practical advantages of our methodology and note the consistency of our results with results from previous studies. We further demonstrate the usefulness of the M-quantile estimation framework through design-based simulations based on two realistic survey data sets containing small area information and show that the M-quantile approach may be preferable when the aim is to estimate the small area distribution function.

Nonparametric M-quantile regression using penalised splines

Quantile regression investigates the conditional quantile functions of a response variable in terms of a set of covariates. M-quantile regression extends this idea by a ‘quantile-like’ generalisation of regression based on influence functions. In this work, we extend it to nonparametric regression, in the sense that the M-quantile regression functions do not have to be assumed to have a certain parametric form, but can be left undefined and estimated from the data. Penalised splines are employed to estimate them. This choice makes it easy to move to bivariate smoothing and semiparametric modelling. An algorithm based on iteratively reweighted penalised least squares to actually fit the model is proposed. Quantile crossing is addressed using an a posteriori adjustment to the function fits following He [1]. Simulation studies show the finite sample properties of the proposed estimation technique.

Non-parametric bootstrap mean squared error estimation for M-quantile estimators of small area averages, quantiles and poverty indicators

Small area estimation is conventionally concerned with the estimation of small area averages and totals. More recently emphasis has been also placed on the estimation of poverty indicators and of key quantiles of the small area distribution function using robust models, for example, the M-quantile small area model. In parallel to point estimation, Mean Squared Error (MSE) estimation is an equally crucial and challenging task. However, while analytic MSE estimation for small area averages is possible, analytic MSE estimation for quantiles and poverty indicators is difficult. Moreover, one of the main criticisms of the analytic MSE estimator for M-quantile estimates of small area averages is that it can be unstable when the area-specific sample sizes are small. A non-parametric bootstrap framework for MSE estimation for small area averages, quantiles and poverty indicators estimated with the M-quantile small area model is proposed. Emphasis is placed on second order properties of MSE estimators with results suggesting that the bootstrap MSE estimator is more stable than corresponding analytic MSE estimators. The proposed bootstrap is evaluated in a series of simulation studies under different parametric assumptions for the model error terms and different scenarios for the area-specific sample and population sizes. Finally, results from the application of the proposed MSE estimator to real income data from the European Survey of Income and Living Conditions (EU-SILC) in Italy are presented and information on the availability of R functions that can be used for implementing the proposed estimation procedures in practice is provided.

Small Area Model-Based Estimators Using Big Data Sources

Abstract The timely, accurate monitoring of social indicators, such as poverty or inequality, on a finegrained spatial and temporal scale is a crucial tool for understanding social phenomena and policymaking, but poses a great challenge to official statistics. This article argues that an interdisciplinary approach, combining the body of statistical research in small area estimation with the body of research in social data mining based on Big Data, can provide novel means to tackle this problem successfully. Big Data derived from the digital crumbs that humans leave behind in their daily activities are in fact providing ever more accurate proxies of social life. Social data mining from these data, coupled with advanced model-based techniques for fine-grained estimates, have the potential to provide a novel microscope through which to view and understand social complexity. This article suggests three ways to use Big Data together with small area estimation techniques, and shows how Big Data has the potential to mirror aspects of well-being and other socioeconomic phenomena.

Small area estimation of the mean using non-parametric M-quantile regression: a comparison when a linear mixed model does not hold

The demand for reliable statistics in subpopulations, when only reduced sample sizes are available, has promoted the development of small area estimation methods. In particular, an approach that is now widely used is based on the seminal work by Battese et al. [An error-components model for prediction of county crop areas using survey and satellite data, J. Am. Statist. Assoc. 83 (1988), pp. 28–36] that uses linear mixed models (MM). We investigate alternatives when a linear MM does not hold because, on one side, linearity may not be assumed and/or, on the other, normality of the random effects may not be assumed. In particular, Opsomer et al. [Nonparametric small area estimation using penalized spline regression, J. R. Statist. Soc. Ser. B 70 (2008), pp. 265–283] propose an estimator that extends the linear MM approach to the case in which a linear relationship may not be assumed using penalized splines regression. From a very different perspective, Chambers and Tzavidis [M-quantile models for small area estimation, Biometrika 93 (2006), pp. 255–268] have recently proposed an approach for small-area estimation that is based on M-quantile (MQ) regression. This allows for models robust to outliers and to distributional assumptions on the errors and the area effects. However, when the functional form of the relationship between the qth MQ and the covariates is not linear, it can lead to biased estimates of the small area parameters. Pratesi et al. [Semiparametric M-quantile regression for estimating the proportion of acidic lakes in 8-digit HUCs of the Northeastern US, Environmetrics 19(7) (2008), pp. 687–701] apply an extended version of this approach for the estimation of the small area distribution function using a non-parametric specification of the conditional MQ of the response variable given the covariates [M. Pratesi, M.G. Ranalli, and N. Salvati, Nonparametric m-quantile regression using penalized splines, J. Nonparametric Stat. 21 (2009), pp. 287–304]. We will derive the small area estimator of the mean under this model, together with its mean-squared error estimator and compare its performance to the other estimators via simulations on both real and simulated data.

Outlier robust model-assisted small area estimation

Small area estimation with M-quantile models was proposed by Chambers and Tzavidis (). The key target of this approach to small area estimation is to obtain reliable and outlier robust estimates avoiding at the same time the need for strong parametric assumptions. This approach, however, does not allow for the use of unit level survey weights, making questionable the design consistency of the estimators unless the sampling design is self-weighting within small areas. In this paper, we adopt a model-assisted approach and construct design consistent small area estimators that are based on the M-quantile small area model. Analytic and bootstrap estimators of the design-based variance are discussed. The proposed estimators are empirically evaluated in the presence of complex sampling designs.

Resistance to outliers of M-quantile and robust random effects small area models

The presence of outliers is a common feature in real data applications. It has been well established that outliers can severely affect the parameter estimates of statistical models, for example, random effects models, which can in turn affect the small area estimates produced using these models. Two outlier robust methodologies have been recently proposed in the small area literature. These are the M-quantile approach and the robust random effects approach. The M-quantile and robust random effects approaches are two distinct outlier robust small area methods and a comparison between these two methodologies is required. The present paper sets to fulfill this goal. Using model-based simulations and showing an application to real income data we examine how the alternative small area methodologies compare.

Local stationarity in small area estimation models

Small area estimators are often based on linear mixed models under the assumption that relationships among variables are stationary across the area of interest (Fay–Herriot models). This hypothesis is patently violated when the population is divided into heterogeneous latent subgroups. In this paper we propose a local Fay–Herriot model assisted by a Simulated Annealing algorithm to identify the latent subgroups of small areas. The value minimized through the Simulated Annealing algorithm is the sum of the estimated mean squared error (MSE) of the small area estimates. The technique is employed for small area estimates of erosion on agricultural land within the Rathbun Lake Watershed (IA, USA). The results are promising and show that introducing local stationarity in a small area model may lead to useful improvements in the performance of the estimators.

Small Area Estimation of Poverty Indicators

The estimation of poverty, inequality and life condition indicators all over the European Union has become one topic of primary interest. A very common target is the core set of indicators on poverty and social exclusion agreed by the Laeken European Council in December \(2001\) and called Laeken indicators. They include measures of the incidence of poverty, such as the Head Count Ratio (also known as at-risk-of-poverty-rate) and of the intensity of poverty, as the Poverty Gap. Unfortunately, these indicators cannot be directly estimated from EU-SILC survey data when the objective is to investigate poverty at sub-regional level. As local sample sizes are small, the estimation must be done using the small area estimation approach. Limits and potentialities of the estimators of Laeken indicators obtained under EBLUP and M-quantile small area estimation approaches are discussed here, as well as their application to EU-SILC Italian data. The case study is limited to the estimation of poverty indicators for the Tuscany region. However, additional results are available and downloadable from the web site of the SAMPLE project, funded under the 7FP (http://www.sample-project.eu).