Phillip Li

Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say 'Usually Not'

We attempt to replicate 67 papers published in 13 well-regarded economics journals using author-provided replication files that include both data and code. Some journals in our sample require data and code replication files, and other journals do not require such files. Aside from 6 papers that use confidential data, we obtain data and code replication files for 29 of 35 papers (83%) that are required to provide such files as a condition of publication, compared to 11 of 26 papers (42%) that are not required to provide data and code replication files. We successfully replicate the key qualitative result of 22 of 67 papers (33%) without contacting the authors. Excluding the 6 papers that use confidential data and the 2 papers that use software we do not possess, we replicate 29 of 59 papers (49%) with assistance from the authors. Because we are able to replicate less than half of the papers in our sample even with help from the authors, we assert that economics research is usually not replicable. We conclude with recommendations on improving replication of economics research.

Bayesian Analysis of Multivariate Sample Selection Models Using Gaussian Copulas

We consider the Bayes estimation of a multivariate sample selection model with p pairs of selection and outcome variables. Each of the variables may be discrete or continuous with a parametric marginal distribution, and their dependence structure is modeled through a Gaussian copula function. Markov chain Monte Carlo methods are used to simulate from the posterior distribution of interest. The methods are illustrated in a simulation study and an application from transportation economics.

Further Investigation of Parametric Loss Given Default Modeling

We conduct a comprehensive study of some parametric models that are designed to fit the unusual bounded and bimodal distribution of loss given default (LGD). We first examine a smearing estimator, a Monte Carlo estimator and a global adjustment approach to refine transformation regression models that address issues with LGD boundary values. Although these refinements only marginally improve model performance, the smearing and Monte Carlo estimators help to reduce the sensitivity of transformation regressions to the adjustment factor. We then conduct a horse race among the refined transformation methods, five parametric models that are specifically suitable for LGD modeling (two-step, inflated beta, Tobit, censored gamma and two-tiered gamma regressions), fractional response regression and standard linear regression. We find that the sophisticated parametric models do not clearly outperform the simpler ones in either predictive accuracy or rank-ordering ability, in-sample, out-of-sample or out of time. Therefore, it is important for modelers and researchers to choose the model that is appropriate for their particular data set, considering differences in model complexity, computational burden, ease of implementation and model performance.

Measurement Error in Macroeconomic Data and Economics Research: Data Revisions, Gross Domestic Product, and Gross Domestic Income

We analyze the effect of measurement error in macroeconomic data on economics research using two features of the estimates of latent US output produced by the Bureau of Economic Analysis (BEA). First, we use the fact that the BEA publishes two theoretically identical estimates of latent US output that only differ due to measurement error: the more well-known gross domestic product (GDP), which the BEA constructs using expenditure data, and gross domestic income (GDI), which the BEA constructs using income data. Second, we use BEA revisions to previously published releases of GDP and GDI. Using a sample of 23 published economics papers from top economics journals that utilize GDP as a key component of an estimated model, we assess whether using either revised GDP or GDI instead of GDP in the published paper would change reported results. We find that estimating models using revised GDP generates the same qualitative result as the original paper in all 23 cases. Estimatin g models using GDI, both with the GDI data originally available to the authors and with revised GDI, instead of GDP generates larger differences in results than those obtained with revised GDP. For 3 of 23 papers (13%), the results we obtain with GDI are qualitatively different than the original published results.

MEASUREMENT ERROR IN MACROECONOMIC DATA AND ECONOMICS RESEARCH: DATA REVISIONS, GROSS DOMESTIC PRODUCT, AND GROSS DOMESTIC INCOME

We use a preanalysis plan to analyze the effect of measurement error on economics research using the fact that the Bureau of Economic Analysis both revises its gross domestic product (GDP) data and also publishes a second, theoretically identical estimate of U.S. output that only differs from GDP due to measurement error: gross domestic income (GDI). Using a sample of 23 models published in top economics journals, we find that reestimating models using revised GDP always gives the same qualitative result as the original publication. Estimating models using GDI instead of GDP gives a different qualitative result for three of 23 models (13%).

Is Economics Research Replicable? Sixty Published Papers From Thirteen Journals Say “Often Not”

Is Economics Research Replicable? Sixty Published Papers From Thirteen Journals Say “Often Not”

Mortgage Characteristics and the Racial Incidence of Default

Previous research has shown that relative to White borrowers, Black and Hispanic borrowers taking out mortgages at the height of the early-2000s housing boom experienced significantly higher delinquency rates. In this paper we attempt to gain a better understanding of the mechanisms that gave rise to these racial differences in mortgage delinquency. Using a database of nearly 9 million mortgages originated between 2005 and 2009, we find that minority borrowers were significantly more likely to have mortgages with high-risk contract characteristics, such as prepayment penalties, variable interest rates, balloon structures, and negative amortization periods. Results from mortgage default models and a decomposition exercise show that the concentration of minority buyers in such loans explains a significant fraction of the difference in default rates between racial groups. The totality of our results suggest that exotic loan characteristics acted as mortgage default accelerants for many minority homeowners that experienced significant income and equity shocks during the Great Recession.