Herman Stekler

The state of macroeconomic forecasting

Macroeconomic forecasts are used extensively in industry and government. The historical accuracy of US and UK forecasts are examined in the light of different approaches to evaluating macroforecasts. Issues discussed include the comparative accuracy of macroeconometric models compared to their time series alternatives, whether the forecasting record has improved over time, the rationality of macroeconomic forecasts and how a forecasting service should be chosen. The role of judgement in producing the forecasts is also considered where the evidence unequivocally favors such interventions. Finally the use of macroeconomic forecasts and their effectiveness is discussed. The conclusion drawn is that researchers have paid too little attention to the issue of improving the forecasting accuracy record. Finally, areas where improvements are most likely to be found are discussed.

An evaluation of the predictions of the Federal Reserve

Abstract To successfully implement monetary policy, the Federal Reserve System (FED) must make forecasts about the future state of the economy. This paper examines some of the characteristics of these forecasts. The analysis presents the usual error measures and tests for rationality. The paper compares these predictions with those generated by ARIMA models and the ASA/NBER surveys. In addition, we analyze (1) the relationship between accuracy and the length of the forecast horizon, (2) whether accuracy has improved over time, and (3) the accuracy of the forecasts in the vicinity of turning points. We conclude that the FED predictions tended to yield the same type of errors that private forecasters have displayed: in some periods either real GNP or inflation had systematic errors; turning point errors occurred prior to recessions; the forecasts were unbiased, but showed evidence of inefficiency. However, the FED forecasts were not significantly different from the predictions of the ARIMA models or ASA/NBER surveys.

Evaluating Predictions of Change

This article applies a technique developed by Robert C. Merton (1981) and Roy D. Henriksson and Robert C. Merton (1981) for evaluating the market timing of financial managers to macroeconomic predictions of change. This methodology may be used to determine whether the predictions may be of value to the user. As an illustration, the methodology is applied to a set of real gross national product forecasts. Copyright 1990 by the University of Chicago.

Are sports seedings good predictors?: an evaluation

Abstract Very little attention has been given to predicting outcomes of sporting events. While studies have examined the accuracy of alternative methods of predicting the outcomes of thoroughbred horse races, some obvious predictors of the outcomes of other sporting events have not been examined. In this paper, we evaluate whether rankings (seedings) are good predictors of the actual outcomes in two sports: (1) US collegiate basketball and (2) professional tennis. In this analysis we use statistical probit regressions with the difference in rankings as the predictor of the outcome of games and/or matches. We evaluate both the ex post and ex ante predictions using base rate forecasts and Brier scores. We conclude that the rankings, by themselves, are useful predictors and that the probits improve on this performance.

Predicting the outcomes of National Football League games

Rankings have predictive value for determining the outcomes of basketball games and tennis matches. Rankings, based on power scores, are also available for NFL teams. This paper evaluates power scores as predictors of the outcomes of NFL games for the 1994–2000 seasons. The evaluation involves a comparison of forecasts generated from probit regressions based on power scores published in The New York Times with those of a naive model, the betting market, and the opinions of the sports editor of The New York Times. We conclude that the betting market is the best predictor followed by the probit predictions based on power scores. We analyze the editor’s predictions and find that his predictions were comparable to a bootstrapping model of his forecasts but were inferior to those based on power scores and even worse than naive forecasts.  2001 International Institute of Forecasters. Published by Elsevier Science B.V. All rights reserved.

Directional forecasts of GDP and inflation: a joint evaluation with an application to Federal Reserve predictions

Many studies have undertaken separate analyses of the Feds forecasts of real Gross Domestic Product (GDP) growth and inflation. This article presents a method for jointly evaluating the direction of change predictions of these variables. We conclude that some of the inflation forecasts, examined separately, were not valuable. However, the joint pattern of GDP and inflation projections was generally in accord with the economys movements. ‘… directional forecasting … is now an increasingly popular metric for forecasting performance….’ (Pesaran and Timmermann, 2004, 414)

Macroeconomic forecast evaluation techniques

Abstract This paper examines eight questions which forecast evaluations can and should address. Statistical techniques which can be used to test hypotheses are presented. The use of these techniques is illustrated by a number of concrete examples.

Data Revisions and Economic Forecasting

Abstract The paper determines whether the earliest published indication of changes in GNP provide information which is useful in the interpretation of economic trends or whether subsequent revisions reveal that the true movements were radically different from those initially reported. An analysis of the provisional and advance data indicated that the initial changes differed from the final changes but still approximated the true pattern of the movements. Some of the data revisions, however, where unsuccessful. A further analysis of the temporal discrepancies in the yearly changes of one series indicates that there is not a wide degree of uncertainty about the actual movements. The combined evidence indicates that the early data are useful for economic analysis.

Why did forecasters fail to predict the 1990 recession

Abstract This paper examines the forecasts that were prepared prior to and during the early stages of the recession that occurred in 1990 in the United States. It examines the characteristics of those forecasts, the data that were available and attempts to determine why the forecast errors occurred. Private sector and public sector predictions are compared and the possibility of rational forecast bias is investigated. We conclude that data problems might have contributed to the forecast errors and suggest that individuals might have been able to predict this recession.

Who Forecasts Better

This article suggests that the emphasis on forecast evaluations should be redirected from searching for “best” forecasters to finding those who are “better.” A new methodology is developed, and the results indicate that it is possible to identify “better” forecasters.