Raffaella Giacomini

Tests of Conditional Predictive Ability

We argue that the current framework for predictive ability testing (e.g., West, 1996) is not necessarily useful for real-time forecast selection, i.e., for assessing which of two competing forecasting methods will perform better in the future. We propose an alternative framework for out-of-sample comparison of predictive ability which delivers more practically relevant conclusions. Our approach is based on inference about conditional expectations of forecasts and forecast errors rather than the unconditional expectations that are the focus of the existing literature. We capture important determinants of forecast performance that are neglected in the existing literature by evaluating what we call the forecasting method (the model and the parameter estimation procedure), rather than just the forecasting model. Compared to previous approaches, our tests are valid under more general data assumptions (heterogeneity rather than stationarity) and estimation methods, and they can handle comparison of both nested and non-nested models, which is not currently possible. To illustrate the usefulness of the proposed tests, we compare the forecast performance of three leading parameter-reduction methods for macroeconomic forecasting using a large number of predictors: a sequential model selection approach, the diffusion indexes approach of Stock and Watson (2002), and the use of Bayesian shrinkage estimators.

Comparing Density Forecasts via Weighted Likelihood Ratio Tests

We propose a test for comparing the out-of-sample accuracy of competing density forecasts of a variable. The test is valid under general conditions: The data can be heterogeneous and the forecasts can be based on (nested or nonnested) parametric models or produced by semiparametric, nonparametric, or Bayesian estimation techniques. The evaluation is based on scoring rules, which are loss functions defined over the density forecast and the realizations of the variable. We restrict attention to the logarithmic scoring rule and propose an out-of-sample “weighted likelihood ratio” test that compares weighted averages of the scores for the competing forecasts. The user-defined weights are a way to focus attention on different regions of the distribution of the variable. For a uniform weight function, the test can be interpreted as an extension of Vuongs likelihood ratio test to time series data and to an out-of-sample testing framework. We apply the tests to evaluate density forecasts of U.S. inflation produced by linear and Markov-switching Phillips curve models estimated by either maximum likelihood or Bayesian methods. We conclude that a Markov-switching Phillips curve estimated by maximum likelihood produces the best density forecasts of inflation.

A Warp-Speed Method for Conducting Monte Carlo Experiments Involving Bootstrap Estimators

We analyze fast procedures for conducting Monte Carlo experiments involving bootstrap estimators, providing formal results establishing the properties of these methods under general conditions.

How Stable is the Forecasting Performance of the Yield Curve for Output Growth

We provide an extensive evaluation of the predictive performance of the US yield curve for US gross domestic product growth by using new tests for forecast breakdown, in addition to a variety of in‐sample and out‐of‐sample evaluation procedures. Empirical research over the past decades has uncovered a strong predictive relationship between the yield curve and output growth, whose stability has recently been questioned. We document the existence of a forecast breakdown during the Burns–Miller and Volker monetary policy regimes, whereas during the early part of the Greenspan era the yield curve emerged as a more reliable model to predict future economic activity.

The relationship between DSGE and VAR models

This chapter reviews the literature on the econometric relationship between DSGE and VAR models from the point of view of estimation and model validation. The mapping between DSGE and VAR models is broken down into three stages: 1) from DSGE to state-space model; 2) from state-space model to VAR(8); 3) from VAR(8) to finite order VAR. The focus is on discussing what can go wrong at each step of this mapping and on critically highlighting the hidden assumptions. I also point out some open research questions and interesting new research directions in the literature on the econometrics of DSGE models. These include, in no particular order: understanding the effects of log-linearization on estimation and identification; dealing with multiplicity of equilibria; estimating nonlinear DSGE models; incorporating into DSGE models information from atheoretical models and from survey data; adopting flexible modelling approaches that combine the theoretical rigor of DSGE models and the econometric models ability to fit the data.