Chris Chatfield

Time series forecasting with neural networks: a comparative study using the air line data

Summary. This case-study fits a variety of neural network (NN) models to the well-known airline data and compares the resulting forecasts with those obtained from the Box‐Jenkins and Holt‐ Winters methods. Many potential problems in fitting NN models were revealed such as the possibility that the fitting routine may not converge or may converge to a local minimum. Moreover it was found that an NN model which fits well may give poor out-of-sample forecasts. Thus we think it is unwise to apply NN models blindly in ‘black box’ mode as has sometimes been suggested. Rather, the wise analyst needs to use traditional modelling skills to select a good NN model, e.g. to select appropriate lagged variables as the ‘inputs’. The Bayesian information criterion is preferred to Akaike’s information criterion for comparing different models. Methods of examining the response surface implied by an NN model are examined and compared with the results of alternative nonparametric procedures using generalized additive models and projection pursuit regression. The latter imposes less structure on the model and is arguably easier to understand.

Calculating Interval Forecasts

The importance of interval forecasts is reviewed. Several general approaches to calculating such forecasts are described and compared. They include the use of theoretical formulas based on a fitted probability model (with or without a correction for parameter uncertainty), various “approximate” formulas (which should be avoided), and empirically based, simulation, and resampling procedures. The latter are useful when theoretical formulas are not available or there are doubts about some model assumptions. The distinction between a forecasting method and a forecasting model is expounded. For large groups of series, a forecasting method may be chosen in a fairly ad hoc way. With appropriate checks, it may be possible to base interval forecasts on the model for which the method is optimal. It is certainly unsound to use a model for which the method is not optimal, but, strangely, this is sometimes done. Some general comments are made as to why prediction intervals tend to be too narrow in practice to encompas...

The M2-competition: A real-time judgmentally based forecasting study

The purpose of the M2-Competition is to determine the post sample accuracy of various forecasting methods. It is an empirical study organized in such a way as to avoid the major criticism of the M-Competition that forecasters in real situations can use additional information to improve the predictive accuracy of quantitative methods. Such information might involve inside knowledge (e.g. a machine breakdown, a forthcoming strike in a major competitor, a steep price increase, etc.), be related to the expected state of the industry or economy that might affect the product(s) involved, or be the outcome of a careful study of the historical data and special care in procedure/methods employed while forecasting. The MZCompetition consisted of distributing 29 actual series (23 of these series came from four companies and six were of macro economic nature) to five forecasters. The data covered information including the September figures of the year involved. The objective was to make monthly forecasts covering 1.5 months starting from October and including December of the next year. A year later the forecasters were provided with the new data as they had become available and the process of predicting for 15 months ahead was repeated. In addition to being able to incorporate their predictions about the state of the economy and that of the industry the participating forecasters could ask for any

The Holt-Winters Forecasting Procedure

SUMMARY The Holt-Winters forecasting procedure is a simple widely used projection method which can cope with trend and seasonal variation. However, empirical studies have tended to show that the method is not as accurate on average as the more complicated Box-Jenkins procedure. This paper points out that these empirical studies have used the automatic version of the method, whereas a non-automatic version is also possible in which subjective judgement is employed, for example, to choose the correct model for seasonality. The paper re-analyses seven series from the Newbold-Granger study for which Box-Jenkins forecasts were reported to be much superior to the (automatic) Holt-Winters forecasts. The series do not appear to have any common properties, but it is shown that the automatic Holt-Winters forecasts can often be improved by subjective modifications. It is argued that a fairer comparison would be that between Box-Jenkins and a non-automatic version of Holt-Winters. Some general recommendations are made concerning the choice of a univariate forecasting procedure. The paper also makes suggestions regarding the implementation of the Holt-Winters procedure, including a choice of starting values.

Holt-Winters Forecasting: Some Practical Issues

The Holt-Winters forecasting procedure is a variant of exponential smoothing which is simple, yet generally works well in practice, and is particularly suitable for producing short-term forecasts for sales or demand time-series data. Some practical problems in implementing the method are discussed, including the normalisation of seasonal indices, the choice of starting values and the choice of smoothing parameters. There is an important distinction between an automatic and a nonautomatic approach to forecasting and detailed suggestions are made for implementing Holt-Winters in both ways. The question as to what underlying model, if any, is assumed by the method is also addressed. Some possible areas for future research are then outlined.

Analysing sequences of behavioural events.

Abstract This paper reviews techniques for examining sequential dependencies in a series of behavioural events and points out the relationship between the χ 2 goodness-of-fit test and information theory. The paper also considers how the techniques should be modified when the behavioural events are not immediately repeated or when events are often repeated a large number of times. Some examples are given to illustrate the use of information theory.

Model uncertainty and forecast accuracy

In time-series analysis, a model is rarely pre-specified but rather is typically formulated in an iterative, interactive way using the given time-series data. Unfortunately the properties of the fitted model, and the forecasts from it, are generally calculated as if the model were known in the first place. This is theoretically incorrect, as least squares theory, for example, does not apply when the same data are used to formulates and fit a model. Ignoring prior model selection leads to biases, not only in estimates of model parameters but also in the subsequent construction of prediction intervals. The latter are typically too narrow, partly because they do not allow for model uncertainty. Empirical results also suggest that more complicated models tend to give a better fit but poorer ex-ante forecasts. The reasons behind these phenomena are reviewed. When comparing different forecasting models, the BIC is preferred to the AIC for identifying a model on the basis of within-sample fit, but out-of-sample forecasting accuracy provides the real test. Alternative approaches to forecasting, which avoid conditioning on a single model, include Bayesian model averaging and using a forecasting method which is not model-based but which is designed to be adaptable and robust.

Statistical Inference Regarding Markov Chain Models

The paper reviews different techniques for examining sequential dependencies in a series of observations each of which can have c possible outcomes. The relationship between the likelihood ratio test and information theory is described. The paper also considers how the techniques need to be modified in situations where two successive outcomes are always different.

A new look at models for exponential smoothing

Exponential smoothing (ES) forecasting methods are widely used but are often dis-cussed without recourse to a formal statistical framework. This paper reviews and compares a variety of potential models for ES. As well as autoregressive integrated moving average and structural models, a promising class of dynamic non-linear state space models is described that allows for a changing variance. The richness of possible models helps to explain why ES methods seem to be robust in practice. A modelling approach can enhance the forecasters ability to identify pertinent components of time series variation, and to obtain more reliable estimates of prediction error variances. The paper should be of particular interest to those engaged in forecasting appli-cations where strategies that allow for risk and uncertainty are needed.

Prediction intervals for the Holt-Winters forecasting procedure

Abstract Prediction interval formulae are derived for the Holt-Winters forecasting procedure with an additive seasonal effect. The formulae make no assumptions about the ‘true’ underlying model. The results are contrasted with those obtained from various alternative approaches to the calculation of prediction intervals. Some large discrepancies are noted and it is suggested that the formulae presented here should be preferred to those which depend on an inappropriate deterministic model or which depend on invalid generalised approximations which take no account of the particular properties of the given series. Results for cumulative forecasts and for a damped trend model are also given. For completeness we also give results for one- and two-parameter exponential smoothing. Finally, we make some general comments as to why prediction intervals tend to be too narrow in practice.