Sebastian Calonico

Robust Nonparametric Confidence Intervals for Regression‐Discontinuity Designs

In the regression‐discontinuity (RD) design, units are assigned to treatment based on whether their value of an observed covariate exceeds a known cutoff. In this design, local polynomial estimators are now routinely employed to construct confidence intervals for treatment effects. The performance of these confidence intervals in applications, however, may be seriously hampered by their sensitivity to the specific bandwidth employed. Available bandwidth selectors typically yield a “large” bandwidth, leading to data‐driven confidence intervals that may be biased, with empirical coverage well below their nominal target. We propose new theory‐based, more robust confidence interval estimators for average treatment effects at the cutoff in sharp RD, sharp kink RD, fuzzy RD, and fuzzy kink RD designs. Our proposed confidence intervals are constructed using a bias‐corrected RD estimator together with a novel standard error estimator. For practical implementation, we discuss mean squared error optimal bandwidths, which are by construction not valid for conventional confidence intervals but are valid with our robust approach, and consistent standard error estimators based on our new variance formulas. In a special case of practical interest, our procedure amounts to running a quadratic instead of a linear local regression. More generally, our results give a formal justification to simple inference procedures based on increasing the order of the local polynomial estimator employed. We find in a simulation study that our confidence intervals exhibit close‐to‐correct empirical coverage and good empirical interval length on average, remarkably improving upon the alternatives available in the literature. All results are readily available in R and STATA using our companion software packages described in Calonico, Cattaneo, and Titiunik (2014d, 2014b).

Optimal Data-Driven Regression Discontinuity Plots

Exploratory data analysis plays a central role in applied statistics and econometrics. In the popular regression-discontinuity (RD) design, the use of graphical analysis has been strongly advocated because it provides both easy presentation and transparent validation of the design. RD plots are nowadays widely used in applications, despite its formal properties being unknown: these plots are typically presented employing ad hoc choices of tuning parameters, which makes these procedures less automatic and more subjective. In this article, we formally study the most common RD plot based on an evenly spaced binning of the data, and propose several (optimal) data-driven choices for the number of bins depending on the goal of the researcher. These RD plots are constructed either to approximate the underlying unknown regression functions without imposing smoothness in the estimator, or to approximate the underlying variability of the raw data while smoothing out the otherwise uninformative scatterplot of the data. In addition, we introduce an alternative RD plot based on quantile spaced binning, study its formal properties, and propose similar (optimal) data-driven choices for the number of bins. The main proposed data-driven selectors employ spacings estimators, which are simple and easy to implement in applications because they do not require additional choices of tuning parameters. Altogether, our results offer an array of alternative RD plots that are objective and automatic when implemented, providing a reliable benchmark for graphical analysis in RD designs. We illustrate the performance of our automatic RD plots using several empirical examples and a Monte Carlo study. All results are readily available in R and STATA using the software packages described in Calonico, Cattaneo, and Titiunik. Supplementary materials for this article are available online.

On the Effect of Bias Estimation on Coverage Accuracy in Nonparametric Inference

ABSTRACT Nonparametric methods play a central role in modern empirical work. While they provide inference procedures that are more robust to parametric misspecification bias, they may be quite sensitive to tuning parameter choices. We study the effects of bias correction on confidence interval coverage in the context of kernel density and local polynomial regression estimation, and prove that bias correction can be preferred to undersmoothing for minimizing coverage error and increasing robustness to tuning parameter choice. This is achieved using a novel, yet simple, Studentization, which leads to a new way of constructing kernel-based bias-corrected confidence intervals. In addition, for practical cases, we derive coverage error optimal bandwidths and discuss easy-to-implement bandwidth selectors. For interior points, we show that the mean-squared error (MSE)-optimal bandwidth for the original point estimator (before bias correction) delivers the fastest coverage error decay rate after bias correction when second-order (equivalent) kernels are employed, but is otherwise suboptimal because it is too “large.” Finally, for odd-degree local polynomial regression, we show that, as with point estimation, coverage error adapts to boundary points automatically when appropriate Studentization is used; however, the MSE-optimal bandwidth for the original point estimator is suboptimal. All the results are established using valid Edgeworth expansions and illustrated with simulated data. Our findings have important consequences for empirical work as they indicate that bias-corrected confidence intervals, coupled with appropriate standard errors, have smaller coverage error and are less sensitive to tuning parameter choices in practically relevant cases where additional smoothness is available. Supplementary materials for this article are available online.

Regression Discontinuity Designs Using Covariates

We study regression discontinuity designs when covariates are included in the estimation. We examine local polynomial estimators that include discrete or continuous covariates in an additive separable way, but without imposing any parametric restrictions on the underlying population regression functions. We recommend a covariate-adjustment approach that retains consistency under intuitive conditions and characterize the potential for estimation and inference improvements. We also present new covariate-adjusted mean-squared error expansions and robust bias-corrected inference procedures, with heteroskedasticity-consistent and cluster-robust standard errors. We provide an empirical illustration and an extensive simulation study. All methods are implemented in R and Stata software packages.

New Evidence on the Persistence of Health Spending

Surprisingly little is known about long-term spending patterns in the under-65 population. Such information could inform efforts to improve coverage and control costs. Using the MarketScan claims database, we characterize the persistence of health care spending in the privately insured, under-65 population. Over a 6-year period, 69.8% of enrollees never had annual spending in the top 10% of the distribution and the bottom 50% of spenders accounted for less than 10% of spending. Those in the top 10% in 2003 were almost as likely (34.4%) to be in the top 10% five years later as one year later (43.4%). Many comorbid conditions retained much of their predictive power even 5 years later. The persistence at both ends of the spending distribution indicates the potential for adverse selection and cream skimming and supports the use of disease management, particularly for those with the conditions that remained strong predictors of high spending throughout the follow-up period.

Gender Segregation in the Workplace and Wage Gaps: Evidence from Urban Mexico 1994-2004

This paper analyzes the evolution of gender segregation in the workplace in Mexico between 1994 and 2004, using a matching comparisons technique to explore the role of individual and family characteristics in determining gender segregation and wage gaps. The results suggest that the complete elimination of hierarchical segregation would reduce the observed gender wage gaps by 5 percentage points, while the elimination of occupational segregation would have increased gender wage gaps by approximately 6 percentage points. The results also indicate that the role of occupational segregation in wage gaps has been increasing in magnitude during the period of analysis, while the role of hierarchical segregation in the determination of wage gaps has been decreasing.

Returns to Private Education in Peru

The private provision of educational services has been representing an increasing fraction of the Peruvian schooling system, especially in recent last decades. While there have been many claims about the differences in quality between private and public schools, there is no complete assessment of the different impacts of these two type of providers on the labor markets. This paper is an attempt to provide such a comprehensive overview. We explore private-public differences in the individual returns to education in Urban Peru. Exploiting a rich pair of data sets (ENNIV 1997 and 2000) that include questions on type of education (public vs. private) for each educational level (primary, secondary, technical tertiary and university tertiary) to a representative sample of adults we are able to measure the differences in labor earnings for all possible educational trajectories. The results indicate higher returns to education for those who attended private schools than those who attended the public system. Nonetheless, these higher returns also show higher dispersion, reflecting wider quality heterogeneity within the private system. The private-public differences in returns are more pronounced at the secondary than at any other educational level. On the other hand, the private-public differences in returns from technical education are almost non-existent. A cohort approach paired with a rolling-windows technique allows us to capture generational evolutions of the private-public differences. The results indicate that these differences have been increasing during the last two decades.

The Women of the National Supported Work Demonstration

This paper re-creates three of the samples from LaLonde’s famous 1986 paper that began the literature on “within-study designs” that uses experiments as benchmarks against which to assess the performance of nonexperimental identification strategies. In particular, we recreate the experimental data for the target group of women on welfare from the National Supported Work (NSW) Demonstration and two of the corresponding comparison groups drawn from the Panel Study of Income Dynamics (PSID). The loss of these data resulted in the (sizable) subsequent literature devoting its attention solely to the NSW men. In addition to repeating LaLonde’s analyses on our recreations of his files for the AFDC women, we apply (many of) the estimators from later papers by Dehejia and Wahba and by Smith and Todd to these data. Our findings support the general view in the literature that women on welfare pose a less difficult selection problem when evaluating employment and training programs. They also call into question the generalizability of some of the broad conclusions that Dehejia and Wahba and Smith and Todd draw from their analyses of the NSW men.