Michèle Hibon

The M3-Competition: results, conclusions and implications

Abstract This paper describes the M3-Competition, the latest of the M-Competitions. It explains the reasons for conducting the competition and summarizes its results and conclusions. In addition, the paper compares such results/conclusions with those of the previous two M-Competitions as well as with those of other major empirical studies. Finally, the implications of these results and conclusions are considered, their consequences for both the theory and practice of forecasting are explored and directions for future research are contemplated.

The M2-competition: A real-time judgmentally based forecasting study

The purpose of the M2-Competition is to determine the post sample accuracy of various forecasting methods. It is an empirical study organized in such a way as to avoid the major criticism of the M-Competition that forecasters in real situations can use additional information to improve the predictive accuracy of quantitative methods. Such information might involve inside knowledge (e.g. a machine breakdown, a forthcoming strike in a major competitor, a steep price increase, etc.), be related to the expected state of the industry or economy that might affect the product(s) involved, or be the outcome of a careful study of the historical data and special care in procedure/methods employed while forecasting. The MZCompetition consisted of distributing 29 actual series (23 of these series came from four companies and six were of macro economic nature) to five forecasters. The data covered information including the September figures of the year involved. The objective was to make monthly forecasts covering 1.5 months starting from October and including December of the next year. A year later the forecasters were provided with the new data as they had become available and the process of predicting for 15 months ahead was repeated. In addition to being able to incorporate their predictions about the state of the economy and that of the industry the participating forecasters could ask for any

Generalising about univariate forecasting methods: further empirical evidence

Abstract This paper extends the empirical evidence on the forecasting accuracy of extrapolative methods. The robustness of the major conclusions of the M-Competition data is examined in the context of the telecommunications data of Fildes (1992) . The performance of Robust Trend, found to be a successful method for forecasting the telecommunications data by Fildes, is compared with that of other successful methods using the M-Competition data. Although it is established that the structure of the telecommunications data is more homogeneous than that of the M-Competition data, the major conclusions of the M-Competition continue to hold for this new data set. In addition, while the Robust Trend method is confirmed to be the best performing method for the telecommunications data, for the 1001 M-Competition series, this method is outperformed by methods such as Single or Damped Smoothing. The performance of smoothing methods depended on how the smoothing parameters are estimated. Optimisation at each time origin was more accurate than optimisation at the first time origin, which in turn is shown to be superior to arbitrary (literature based) fixed values. In contrast to the last point, a data based choice of fixed smoothing constants from a cross-sectional study of the time series was found to perform well.

ARMA Models and the Box–Jenkins Methodology

The purpose of this paper is to apply the Box‐Jenkins methodology to ARIMA models and determine the reasons why in empirical tests it is found that the post-sample forecasting the accuracy of such models is generally worse than much simpler time series methods. The paper concludes that the major problem is the way of making the series stationary in its mean (i.e. the method of diAerencing) that has been proposed by Box and Jenkins. If alternative approaches are utilized to remove and extrapolate the trend in the data, ARMA models outperform the models selected through Box‐Jenkins methodology. In addition, it is shown that using ARMA models to seasonally adjusted data slightly improves post-sample accuracies while simplifying the use of ARMA models. It is also confirmed that transformations slightly improve post-sample forecasting accuracy, particularly for long forecasting horizons. Finally, it is demonstrated that AR(1), AR(2) and ARMA(1,1) models can produce more accurate post-sample forecasts than those found through the application of Box‐ Jenkins methodology. #1997 by John Wiley & Sons, Ltd.

Confidence intervals: An empirical investigation of the series in the M-competition

Abstract This paper empirically evaluates the uncertainty of forecasts. It does so using the 1001 series of the M-Competition. The study indicates that although, in model fitting the percentage of observations outside the confidence intervals is close to that postulated theoretically, this is not true for forecasting. In the latter case the percentage of observations outside the confidence intervals is much higher than that postulated theoretically. This is so for the great majority of series, forecasting horizonts, and methods. In addition to evaluating the extent of uncertainty, we provide tables to help users to construct more realistic confidence intervals for their forecasts.

Exponential smoothing: The effect of initial values and loss functions on post-sample forecasting accuracy

This paper describes an empirical investigation aimed at measuring the effect of different initial values and loss functions (both symmetric and asymmetric) on the post-sample forecasting accuracy. The 1001 series of the M-competition are used and three exponential smoothing methods are employed. The results are compared over various types of data and forecasting horizons and validated with additional data. The paper concludes that contrary to expectations, post-sample forecasting accuracies are not affected by the type of initial values used or the loss function employed in the great majority of cases.