Olivier Ledoit

A well-conditioned estimator for large-dimensional covariance matrices

Many applied problems require a covariance matrix estimator that is not only invertible, but also well-conditioned (that is, inverting it does not amplify estimation error). For large-dimensional covariance matrices, the usual estimator--the sample covariance matrix--is typically not well-conditioned and may not even be invertible. This paper introduces an estimator that is both well-conditioned and more accurate than the sample covariance matrix asymptotically. This estimator is distribution-free and has a simple explicit formula that is easy to compute and interpret. It is the asymptotically optimal convex linear combination of the sample covariance matrix with the identity matrix. Optimality is meant with respect to a quadratic loss function, asymptotically as the number of observations and the number of variables go to infinity together. Extensive Monte Carlo confirm that the asymptotic results tend to hold well in finite sample.

Improved estimation of the covariance matrix of stock returns with an application to portfolio selection

This paper proposes to estimate the covariance matrix of stock returns by an optimally weighted average of two existing estimators: the sample covariance matrix and single-index covariance matrix. This method is generally known as shrinkage, and it is standard in decision theory and in empirical Bayesian statistics. Our shrinkage estimator can be seen as a way to account for extra-market covariance without having to specify an arbitrary multi-factor structure. For NYSE and AMEX stock returns from 1972 to 1995, it can be used to select portfolios with significantly lower out-of-sample variance than a set of existing estimators, including multi-factor models.

Honey, I Shrunk the Sample Covariance Matrix

The central message of this paper is that nobody should be using the sample covariance matrix for the purpose of portfolio optimization. It contains estimation error of the kind most likely to perturb a mean-variance optimizer. In its place, we suggest using the matrix obtained from the sample covariance matrix through a transformation called shrinkage. This tends to pull the most extreme coefficients towards more central values, thereby systematically reducing estimation error where it matters most. Statistically, the challenge is to know the optimal shrinkage intensity, and we give the formula for that. Without changing any other step in the portfolio optimization process, we show on actual stock market data that shrinkage reduces tracking error relative to a benchmark index, and substantially increases the realized information ratio of the active portfolio manager.

Gain, Loss, and Asset Pricing

We develop an approach to asset pricing in incomplete markets that bridges the gap between the two fundamental approaches in finance: model‐based pricing and pricing by no arbitrage. We strengthen the absence of arbtrage assumption by precluding investment opportunities whose attractiveness to a benchmark investor exceeds a specified threshold. In our framework, the attractiveness of an investment opportunity is measured by the gain‐loss ratio. We show that a restriction on the maximum gain‐loss ratio is equivalent to a restriction on the ratio of the maximum to minimum values of the pricing kernel. By limiting the maximum gainloss ratio, we can restrict the admissible set of pricing kernels, which in turn allows us to restrict the set of prices that can be assigned to assets. We illustrate our methodology by computing price bounds for call options in a Black‐Scholes economy without intermediate trading. When we vary the maximum permitted gainloss ratio, these bounds can range from the exact prices implied by a model‐based pricing approach to the loose price bounds implied by the no‐arbitrage approach.

CRASHES AS CRITICAL POINTS

We study a rational expectation model of bubbles and crashes. The model has two components: (1) our key assumption is that a crash may be caused by local self-reinforcing imitation between noise traders. If the tendency for noise traders to imitate their nearest neighbors increases up to a certain point called the “critical” point, all noise traders may place the same order (sell) at the same time, thus causing a crash. The interplay between the progressive strengthening of imitation and the ubiquity of noise is characterized by the hazard rate, i.e. the probability per unit time that the crash will happen in the next instant if it has not happened yet. (2) Since the crash is not a certain deterministic out come of the bubble, it remains rational for traders to remain invested provided they are compensated by a higher rate of growth of the bubble for taking the risk of a crash. Our model distinguishes between the end of the bubble and the time of the crash: the rational expectation constraint has the specific implication that the date of the crash must be random. The theoretical death of the bubble is not the time of the crash because the crash could happen at any time before, even though this is not very likely. The death of the bubble is the most probable time for the crash. There also exists a finite probability of the attaining the end of the bubble without crash. Our model has specific predictions about the presence of certain critical log-periodic patterns in pre-crash prices, associated with the deterministic components of the bubble mechanism. We provide empirical evidence showing that these patterns were indeed present before the crashes of 1929, 1962 and 1987 on Wall Street and the 1997 crash on the Hong Kong Stock Exchange. These results are compared with the statistical tests on synthetic data.

Some hypothesis tests for the covariance matrix when the dimension is large compared to the sample size

This paper analyzes whether standard covariance matrix tests work when dimensionality is large, and in particular larger than sample size. In the latter case, the singularity of the sample covariance matrix makes likelihood ratio tests degenerate, but other tests based on quadratic forms of sample covariance matrix eigenvalues remain well-defined. We study the consistency property and limiting distribution of these tests as dimensionality and sample size go to infinity together, with their ratio converging to a finite non-zero limit. We find that the existing test for sphericity is robust against high dimensionality, but not the test for equality of the covariance matrix to a given matrix. For the latter test, we develop a new correction to the existing test statistic that makes it robust against high dimensionality.

Flexible Multivariate GARCH Modeling With an Application to International Stock Markets

The goal of this paper is to estimate time-varying covariance matrices. Since the covariance matrix of financial returns is known to change through time and is an essential ingredient in risk measurement, portfolio selection, and tests of asset pricing models, this is a very important problem in practice. Our model of choice is the Diagonal-Vech version of the Multivariate GARCH(1,1) model. The problem is that the estimation of the general Diagonal-Vech model model is numerically infeasible in dimensions higher than 5. The common approach is to estimate more restrictive models which are tractable but may not conform to the data. Our contribution is to propose an alternative estimation method that is numerically feasible, produces positive semi-definite conditional covariance matrices, and does not impose unrealistic a priori restrictions. We provide an empirical application in the context of international stock markets, comparing the new estimator to a number of existing ones.

Nonlinear Shrinkage Estimation of Large-Dimensional Covariance Matrices

Many statistical applications require an estimate of a covariance matrix and/or its inverse. When the matrix dimension is large compared to the sample size, which happens frequently, the sample covariance matrix is known to perform poorly and may suffer from ill-conditioning. There already exists an extensive literature concerning improved estimators in such situations. In the absence of further knowledge about the structure of the true covariance matrix, the most successful approach so far, arguably, has been shrinkage estimation. Shrinking the sample covariance matrix to a multiple of the identity, by taking a weighted average of the two, turns out to be equivalent to linearly shrinking the sample eigenvalues to their grand mean, while retaining the sample eigenvectors. Our paper extends this approach by considering nonlinear transformations of the sample eigenvalues. We show how to construct an estimator that is asymptotically equivalent to an oracle estimator suggested in previous work. As demonstrated in extensive Monte Carlo simulations, the resulting bona fide estimator can result in sizeable improvements over the sample covariance matrix and also over linear shrinkage.

Flexible Multivariate GARCH Modeling with an Application to International Stock Markets

This paper offers a new approach to estimating time-varying covariance matrices in the framework of the diagonal-vech version of the multivariate GARCH(1,1) model. Our method is numerically feasible for large-scale problems, produces positive semidefinite conditional covariance matrices, and does not impose unrealistic a priori restrictions. We provide an empirical application in the context of international stock markets, comparing the new estimator with a number of existing ones.

Eigenvectors of Some Large Sample Covariance Matrices Ensembles

We consider sample covariance matrices constructed from real or complex i.i.d. variates with finite 12th moment. We assume that the population covariance matrix is positive definite and its spectral measure almost surely converges to some limiting probability distribution as the number of variables and the number of observations go to infinity together, with their ratio converging to a finite positive limit. We quantify the relationship between sample and population eigenvectors, by studying the asymptotics of a broad family of functionals that generalizes the Stieltjes transform of the spectral measure. This is then used to compute the asymptotically optimal bias correction for sample eigenvalues, paving the way for a new generation of improved estimators of the covariance matrix and its inverse.