Allan Timmermann

Data-Snooping, Technical Trading Rule Performance, and the Bootstrap

In this paper we utilize Whites Reality Check bootstrap methodology (White (1997)) to evaluate simple technical trading rules while quantifying the data-snooping bias and fully adjusting for its effect inthe context of the full universe form which the trading rules are drawn. Henxe, for the first time, the paper presents a comrehensive test of perfomance across all technical trading rules examined. We consider the study of brock, Lakonishok and LeBaron (1992), expand their universe of 26 trading rules, apply the rules to 100 years of daily data on the Dow Jone Industrial Average, and determine the effects of data-snooping.

Can Mutual Fund 'Stars' Really Pick Stocks? New Evidence from a Bootstrap Analysis

We apply an innovative bootstrap statistical technique to examine the performance of the U.S. equity mutual fund industry during the 1962 to 1994 period. Using this new method, we bootstrap the distribution of the performance measure (the “alpha”) across mutual funds to determine whether funds with the best alphas are simply lucky, or whether managers of these funds possess genuine stockpicking skills—this bootstrap technique is necessary because of the complicated form of the distribution of alphas across funds and the non-normal nature of individual funds’ alphas. Our results indicate that, controlling for luck, fund managers that pick stocks well enough to more than cover their costs do exist. That is, the distribution of alphas computed from bootstrapped fund returns (and assuming that no stockpicking talent exists) has a much smaller right tail than the distribution of alphas computed from actual fund returns. Unfortunately for investors, our bootstrap results also show strong evidence of funds with significant inferior performance. Further, our evidence suggests that stockpicking skills are most clearly evident among growth-fund managers. In general, our study supports the value of active mutual fund management, although it also highlights the drawbacks of funds actively managed by those who cannot pick stocks.

A Simple Nonparametric Test of Predictive Performance

This paper derives a distribution free procedure for testing the accuracy of forecasts when the focus of the analysis is on the correct prediction of the direction of change in the variable under consideration. The test applies to a general m x n contingency table and it is shown that the standard null hypothesis of independence in a contingency table implies the null hypothesis of the proposed test of predictive failure but not vice versa. As a test of predictive performance the chi-squared test of independence will, in general, be more conservative than the suggested test of predictive failure. The paper also contains two applications: A dichotomous version of the test is applied to the CBIs Industrial Trends Surveys of actual and expected price changes in the manufacturing sector, and a trichotomous version of the test is applied to the demand data from business surveys of French manufacturing industry conducted by INSEE.

Efficient Market Hypothesis and Forecasting

The efficient market hypothesis gives rise to forecasting tests that mirror those adopted when testing the optimality of a forecast in the context of a given information set. However, there are also important differences arising from the fact that market efficiency tests rely on establishing profitable trading opportunities in ‘real time’. Forecasters constantly search for predictable patterns and affect prices when they attempt to exploit trading opportunities. Stable forecasting patterns are therefore unlikely to persist for long periods of time and will self-destruct when discovered by a large number of investors. This gives rise to nonstationarities in the time series of financial returns and complicates both formal tests of market efficiency and the search for successful forecasting approaches.

Moments of Markov switching models

This paper derives the moments for a range of Markov switching models. We characterise in detail the patterns of volatility, skewness and kurtosis that these models can produce as a function of the transition probability and parameters of the underlying state densities entering the switching process. The autocovariance of the level and squares of time series generated by Markov switching processes is also derived and we use these results to shed light on the relationship between volatility clustering, regime switches and structural breaks in time series models.

Dangers of data mining : The case of calendar effects in stock returns

Economics is primarily a non-experimental science. Typically, we cannot generate new data sets on which to test hypotheses independently of the data that may have led to a particular theory. The common practice of using the same data set to formulate and test hypotheses introduces data-mining biases that, if not accounted for, invalidate the assumptions underlying classical statistical inference. A striking example of a data-driven discovery is the presence of calendar effects in stock returns. There appears to be very substantial evidence of systematic abnormal stock returns related to the day of the week, the week of the month, the month of the year, the turn of the month, holidays, and so forth. However, this evidence has largely been considered without accounting for the intensive search preceding it. In this paper we use 100 years of daily data and a new bootstrap procedure that allows us to explicitly measure the distortions in statistical inference induced by data mining. We find that although nominal p-values for individual calendar rules are extremely significant, once evaluated in the context of the full universe from which such rules were drawn, calendar effects no longer remain significant.

Business cycle asymmetries in stock returns: evidence from higher order moments and conditional densities

Markow switching models with time-varying means, variances and mixing weights are applied to characterise business cycle variation in the probability distribution and higher order moments of stock returns. This allows us to provide a comprehensive characterization of risk that goes well beyond the mean and variance of returns. Several mixture models with different specifications of the state transition are compared and we propose a new mixture of Gaussian and student-t distributions that captures outliers in returns. The models produce very similar expected returns and volatilities but imply very different time series for conditional skewness, kurtosis and predictive density. Consistent with economic theory, the gains in predictive accuracy from considering two-state mixture models rather than a single-state specification are higher for small firms than for large firms. JEL Classification: C22, C52

Disagreement and Biases in Inflation Expectations

Disagreement in inflation expectations observed from survey data varies systematically over time in a way that reflects the level and variance of current inflation. This paper offers a simple explanation for these facts based on asymmetries in the forecasters’ costs of over- and under-predicting inflation. Our model implies (i) biased forecasts; (ii) positive serial correlation in forecast errors; (iii) a cross-sectional dispersion that rises with the level and the variance of the inflation rate; and (iv) predictability of forecast errors at different horizons by means of the spread between the short- and long-term variance of inflation. We find empirically that these patterns are present in inflation forecasts from the Survey of Professional Forecasters. A constant bias component, not explained by asymmetric loss and rational expectations, is required to explain the shift in the sign of the bias observed for a substantial portion of forecasters around 1982.

Duration Dependence in Stock Prices: An Analysis of Bull and Bear Markets

This article studies time series dependence in the direction of stock prices by modeling the (instantaneous) probability that a bull or bear market terminates as a function of its age and a set of underlying state variables, such as interest rates. A random walk model is rejected both for bull and bear markets. Although it fits the data better, a generalized autoregressive conditional heteroscedasticity model is also found to be inconsistent with the very long bull markets observed in the data. The strongest effect of increasing interest rates is found to be a lower bear market hazard rate and hence a higher likelihood of continued declines in stock prices.

How Costly is it to Ignore Breaks When Forecasting the Direction of a Time Series

Empirical evidence suggests that many macroeconomic and financial time series are subject to occasional structural breaks. In this paper we present analytical results quantifying the effects of such breaks on the correlation between the forecast and the realization and on the ability to forecast the sign or direction of a time-series that is subject to breaks. Our results suggest that it can be very costly to ignore breaks. Forecasting approaches that condition on the most recent break are likely to perform better over unconditional approaches that use expanding or rolling estimation windows provided that the break is reasonably large.