Michael W. McCracken

Tests of equal forecast accuracy and encompassing for nested models

We examine the asymptotic and finite-sample properties of tests for equal forecast accuracy and encompassing applied to 1-step ahead forecasts from nested parametric models. We first derive the asymptotic distributions of two standard tests and one new test of encompassing. Tables of asymptotically valid critical values are provided. Monte Carlo methods are then used to evaluate the size and power of the tests of equal forecast accuracy and encompassing. The simulations indicate that post-sample tests can be reasonably well sized. Of the post-sample tests considered, the encompassing test proposed in this paper is the most powerful. We conclude with an empirical application regarding the predictive content of unemployment for inflation.

Evaluating Direct Multistep Forecasts

ABSTRACT This paper examines the asymptotic and finite-sample properties of tests of equal forecast accuracy and encompassing applied to direct, multistep predictions from nested regression models. We first derive asymptotic distributions; these nonstandard distributions depend on the parameters of the data-generating process. We then use Monte Carlo simulations to examine finite-sample size and power. Our asymptotic approximation yields good size and power properties for some, but not all, of the tests; a bootstrap works reasonably well for all tests. The paper concludes with a reexamination of the predictive content of capacity utilization for inflation.

Tests of Equal Forecast Accuracy and Encompassing for Nested Models

We examine the asymptotic and finite-sample properties of tests for equal forecast accuracy and encompassing applied to 1-step ahead forecasts from nested parametric models. We first derive the asymptotic distributions of two standard tests and one new test of encompassing. Tables of asymptotically valid critical values are provided. Monte Carlo methods are then used to evaluate the size and power of the tests of equal forecast accuracy and encompassing. The simulations indicate that post-sample tests can be reasonably well sized. Of the post-sample tests considered, the encompassing test proposed in this paper is the most powerful. We conclude with an empirical application regarding the predictive content of unemployment for inflation.

Robust out-of-sample inference

This paper presents analytical, empirical and simulation results concerning inference about the moments of nondifferentiable functions of out-of-sample forecasts and forecast errors. Special attention is given to the measurement of a models predictive ability using the test of equal mean absolute error. Tests for equal mean absolute error and mean square error are used to evaluate predictions of excess returns to the S & P 500 composite. Simulations indicate that appropriately constructed tests for equal mean absolute error can provide more accurately sized and more powerful tests than inappropriately constructed tests for equal mean absolute error and mean square error.

Improving Forecast Accuracy by Combining Recursive and Rolling Forecasts

This paper presents analytical, Monte Carlo, and empirical evidence on combining recursive and rolling forecasts when linear predictive models are subject to structural change. Using a characterization of the bias-variance tradeoff faced when choosing between either the recursive and rolling schemes or a scalar convex combination of the two, we derive optimal observation windows and combining weights designed to minimize mean square forecast error. Monte Carlo experiments and several empirical examples indicate that combination can often provide improvements in forecast accuracy relative to forecasts made using the recursive scheme or the rolling scheme with a fixed window width.

Tests of Equal Predictive Ability With Real-Time Data

This paper examines the asymptotic and finite-sample properties of tests of equal forecast accuracy applied to direct, multi-step predictions from both non-nested and nested linear regression models. In contrast to earlier work in the literature, our asymptotics take account of the real-time, revised nature of the data. Monte Carlo simulations indicate that our asymptotic approximations yield reasonable size and power properties in most circumstances. The paper concludes with an examination of the real-time predictive content of various measures of economic activity for inflation.

Evaluating long-horizon forecasts

This paper examines the asymptotic and finite-sample properties of tests of equal forecast accuracy and encompassing applied to predictions from nested long-horizon regression models. We first derive the asymptotic distributions of a set of tests of equal forecast accuracy and encompassing, showing that the tests have non-standard distributions that depend on the parameters of the data-generating process. Using a simple parametric bootstrap for inference, we then conduct Monte Carlo simulations of a range of data-generating processes to examine the finite-sample size and power of the tests. In these simulations, the bootstrap yields tests with good finite-sample size and power properties, with the encompassing test proposed by Clark and McCracken (2001a) having superior power. The paper concludes with a reexamination of the predictive content of capacity utilization for core inflation.

Fred-Md: A Monthly Database for Macroeconomic Research

This article describes a large, monthly frequency, macroeconomic database with the goal of establishing a convenient starting point for empirical analysis that requires “big data.” The dataset mimics the coverage of those already used in the literature but has three appealing features. First, it is designed to be updated monthly using the Federal Reserve Economic Data (FRED) database. Second, it will be publicly accessible, facilitating comparison of related research and replication of empirical work. Third, it will relieve researchers from having to manage data changes and revisions. We show that factors extracted from our dataset share the same predictive content as those based on various vintages of the so-called Stock–Watson dataset. In addition, we suggest that diffusion indexes constructed as the partial sum of the factor estimates can potentially be useful for the study of business cycle chronology. Supplementary materials for this article are available online.

Nested Forecast Model Comparisons: A New Approach to Testing Equal Accuracy

This paper develops bootstrap methods for testing whether, in a finite sample, competing out-of-sample forecasts from nested models are equally accurate. Most prior work on forecast tests for nested models has focused on a null hypothesis of equal accuracy in population - basically, whether coefficients on the extra variables in the larger, nesting model are zero. We instead use an asymptotic approximation that treats the coefficients as non-zero but small, such that, in a finite sample, forecasts from the small model are expected to be as accurate as forecasts from the large model. Under that approximation, we derive the limiting distributions of pairwise tests of equal mean square error, and develop bootstrap methods for estimating critical values. Monte Carlo experiments show that our proposed procedures have good size and power properties for the null of equal finite-sample forecast accuracy. We illustrate the use of the procedures with applications to forecasting stock returns and inflation.

Reality Checks and Comparisons of Nested Predictive Models

This article develops a simple bootstrap method for simulating asymptotic critical values for tests of equal forecast accuracy and encompassing among many nested models. Our method combines elements of fixed regressor and wild bootstraps. We first derive the asymptotic distributions of tests of equal forecast accuracy and encompassing applied to forecasts from multiple models that nest the benchmark model—that is, reality check tests. We then prove the validity of the bootstrap for these tests. Monte Carlo experiments indicate that our proposed bootstrap has better finite-sample size and power than other methods designed for comparison of nonnested models. Supplementary materials are available online.