Alberto Abadie

On the Failure of the Bootstrap for Matching Estimators

Matching estimators are widely used in empirical economics for the evaluation of programs or treatments. Researchers using matching methods often apply the bootstrap to calculate the standard errors. However, no formal justification has been provided for the use of the bootstrap in this setting. In this article, we show that the standard bootstrap is, in general, not valid for matching estimators, even in the simple case with a single continuous covariate where the estimator is root-N consistent and asymptotically normally distributed with zero asymptotic bias. Valid inferential methods in this setting are the analytic asymptotic variance estimator of Abadie and Imbens (2006a) as well as certain modifications of the standard bootstrap, like the subsampling methods in Politis and Romano (1994).

Semiparametric Difference-in-Differences Estimators

The difference-in-differences (DID) estimator is one of the most popular tools for applied research in economics to evaluate the effects of public interventions and other treatments of interest on some relevant outcome variables. However, it is well known that the DID estimator is based on strong identifying assumptions. In particular, the conventional DID estimator requires that, in the absence of the treatment, the average outcomes for the treated and control groups would have followed parallel paths over time. This assumption may be implausible if pre-treatment characteristics that are thought to be associated with the dynamics of the outcome variable are unbalanced between the treated and the untreated. That would be the case, for example, if selection for treatment is influenced by individual-transitory shocks on past outcomes (Ashenfelters dip). This article considers the case in which differences in observed characteristics create non-parallel outcome dynamics between treated and controls. It is shown that, in such a case, a simple two-step strategy can be used to estimate the average effect of the treatment for the treated. In addition, the estimation framework proposed in this article allows the use of covariates to describe how the average effect of the treatment varies with changes in observed characteristics. Copyright 2005, Wiley-Blackwell.

Bias-Corrected Matching Estimators for Average Treatment Effects

In Abadie and Imbens (2006), it was shown that simple nearest-neighbor matching estimators include a conditional bias term that converges to zero at a rate that may be slower than N1/2. As a result, matching estimators are not N1/2-consistent in general. In this article, we propose a bias correction that renders matching estimators N1/2-consistent and asymptotically normal. To demonstrate the methods proposed in this article, we apply them to the National Supported Work (NSW) data, originally analyzed in Lalonde (1986). We also carry out a small simulation study based on the NSW example. In this simulation study, a simple implementation of the bias-corrected matching estimator performs well compared to both simple matching estimators and to regression estimators in terms of bias, root-mean-squared-error, and coverage rates. Software to compute the estimators proposed in this article is available on the authors’ web pages (http://www.economics.harvard.edu/faculty/imbens/software.html) and documented in Abadie et al. (2003).

Instrumental Variables Estimates Of The Effect Of Subsidized Training On The Quantiles Of Trainee Earnings

This paper reports estimates of the effects of JTPA training programs on the distribution of earnings. The estimation uses a new instrumental variable (IV) method that measures program impacts on quantiles. The quantile treatment effects (QTE) estimator reduces to quantile regression when selection for treatment is exogenously determined. QTE can be computed as the solution to a convex linear programming problem, although this requires first-step estimation of a nuisance function. We develop distribution theory for the case where the first step is estimated nonparametrically. For women, the empirical results show that the JTPA program had the largest proportional impact at low quantiles. Perhaps surprisingly, however, JTPA training raised the quantiles of earnings for men only in the upper half of the trainee earnings distribution.

Semiparametric instrumental variable estimation of treatment response models

This article introduces a new class of instrumental variable (IV) estimators for linear and nonlinear treatment response models with covariates. The rationale for focusing on nonlinear models is that, if the dependent variable is binary or limited, or if the effect of the treatment varies with covariates, a nonlinear model is appropriate. In the spirit of Roehrig (Econometrica 56 (1988) 433), identification is attained nonparametrically and does not depend on the choice of the parametric specification for the response function of interest. One virtue of this approach is that it allows the researcher to construct estimators that can be interpreted as the parameters of a well-defined approximation to a treatment response function under functional form misspecification. In contrast to some usual IV models, heterogeneity of treatment effects is not restricted by the identification conditions. The ideas and estimators in this article are illustrated using IV to estimate the effects of 401(k) retirement programs on savings.

Bootstrap Tests for Distributional Treatment Effects in Instrumental Variable Models

This article considers the problem of assessing the distributional consequences of a treatment on some outcome variable of interest when treatment intake is (possibly) nonrandomized, but there is a binaryinstrument available for the researcher. Such a scenario is common in observational studies and in randomized experiments with imperfect compliance. One possible approach to this problem is to compare the counterfactual cumulative distribution functions of the outcome with and without the treatment. This article shows how to estimate these distributions using instrumental variable methods and a simple bootstrap procedure is proposed to test distributional hypotheses, such as equality of distributions, first-order and second-order stochastic dominance. These tests and estimators are applied to the study of the effects of veteran status on the distribution of civilian earnings. The results show a negative effect of military service during the Vietnam era that appears to be concentrated on the lower tail of the distribution of earnings. First-order stochastic dominance cannot be rejected by the data.

Comparative Politics and the Synthetic Control Method

In recent years, a widespread consensus has emerged about the necessity of establishing bridges between quantitative and qualitative approaches to empirical research in political science. In this article, we discuss the use of the synthetic control method as a way to bridge the quantitative/qualitative divide in comparative politics. The synthetic control method provides a systematic way to choose comparison units in comparative case studies. This systematization opens the door to precise quantitative inference in small‐sample comparative studies, without precluding the application of qualitative approaches. Borrowing the expression from Sidney Tarrow, the synthetic control method allows researchers to put “qualitative flesh on quantitative bones.” We illustrate the main ideas behind the synthetic control method by estimating the economic impact of the 1990 German reunification on West Germany.

The Impact of Presumed Consent Legislation on Cadaveric Organ Donation: A Cross Country Study

In the U.S., Great Britain and in many other countries, the gap between the demand and the supply of human organs for transplantation is on the rise, despite the efforts of governments and health agencies to promote donor registration. In some countries of continental Europe, however, cadaveric organ procurement is based on the principle of presumed consent. Under presumed consent legislation, a deceased individual is classified as a potential donor in absence of explicit opposition to donation before death. This article analyzes the impact of presumed consent laws on donation rates. For this purpose, we construct a dataset on organ donation rates and potential factors affecting organ donation for 22 countries over a 10-year period. We find that while differences in other determinants of organ donation explain much of the variation in donation rates, after controlling for those determinants presumed consent legislation has a positive and sizeable effect on organ donation rates. We use the panel structure of our dataset to test and reject the hypothesis that unmeasured determinants of organ donation rates confound our empirical results.

Estimation of the Conditional Variance in Paired Experiments

In paired randomized experiments units are grouped in pairs, often based on covariate information, with random assignment within the pairs. Average treatment effects are then estimated by averaging the within-pair differences in outcomes. Typically the variance of the average treatment effect estimator is estimated using the sample variance of the within-pair differences. However, conditional on the covariates the variance of the average treatment effect estimator may be substantially smaller. Here we propose a simple way of estimating the conditional variance of the average treatment effect estimator by forming pairs-of-pairs with similar covariate values and estimating the variances within these pairs-of-pairs. Even though these within-pairs-of-pairs variance estimators are not consistent, their average is consistent for the conditional variance of the average treatment effect estimator and leads to asymptotically valid confidence intervals.

A Martingale Representation for Matching Estimators

Matching estimators are widely used in statistical data analysis. However, the large sample distribution of matching estimators has been derived only for particular cases. This article establishes a martingale representation for matching estimators. This representation allows the use of martingale limit theorems to derive the large sample distribution of matching estimators. As an illustration of the applicability of the theory, we derive the asymptotic distribution of a matching estimator when matching is carried out without replacement, a result previously unavailable in the literature. In addition, we apply the techniques proposed in this article to derive a correction to the standard error of a sample mean when missing data are imputed using the “hot deck,” a matching imputation method widely used in the Current Population Survey (CPS) and other large surveys in the social sciences. We demonstrate the empirical relevance of our methods using two Monte Carlo designs based on actual datasets. In these Monte Carlo exercises, the large sample distribution of matching estimators derived in this article provides an accurate approximation to the small sample behavior of these estimators. In addition, our simulations show that standard errors that do not take into account hot-deck imputation of missing data may be severely downward biased, while standard errors that incorporate the correction for hot-deck imputation perform extremely well. This article has online supplementary materials.