Agnostic Risk Parity: Taming Known and Unknown-Unknowns
Raphael Benichou, Yves Lempérière, Emmanuel Sérié, Julien Kockelkoren, Philip Seager, Jean-Philippe Bouchaud, Marc Potters
aa r X i v : . [ q -f i n . P M ] O c t Agnostic Risk Parity:Taming Known and Unknown-Unknowns
Raphael Benichou, Yves Lempérière, Emmanuel Sérié,Julien Kockelkoren, Philip Seager,Jean-Philippe Bouchaud & Marc Potters
Capital Fund Management23 rue de l’Université, 75007 Paris, France
Abstract
Markowitz’ celebrated optimal portfolio theory generally fails to de-liver out-of-sample diversification. In this note, we propose a new port-folio construction strategy based on symmetry arguments only, leadingto “Eigenrisk Parity” portfolios that achieve equal realized risk on allthe principal components of the covariance matrix. This holds truefor any other definition of uncorrelated factors. We then specialize ourgeneral formula to the most agnostic case where the indicators of futurereturns are assumed to be uncorrelated and of equal variance. This“Agnostic Risk Parity” (AGP) portfolio minimizes unknown-unknownrisks generated by over-optimistic hedging of the different bets. AGP isshown to fare quite well when applied to standard technical strategiessuch as trend following.
Introduction
Diversification is the mantra of rational investment strategies. Harry Markowitzproposed a mathematical incarnation of that mantra which is common lorein the professional world. Unfortunately, the practical implementation ofMarkowitz’ ideas is fraught with difficulties and yields very disappointingresults. This has been known for long, with many papers attempting toidentify its flaws and suggesting remedies [1, 2, 3, 4, 5, 6]. The most im-portant problems are well understood: the optimally diversified Markowitzportfolio often ends up – somewhat paradoxically – being very concentratedon a few assets only, which inevitably leads to disastrous out-of-sample risks.The optimal portfolio is also unstable in time and sensitive to small changesin parameters and/or expected future gains. In the face of these difficulties,two distinct branches of research have emerged.The first one concerns the determination of the covariance matrix of the N different assets eligible in the portfolio, for example all the stocks belong-ing to a given index. This covariance matrix is specified by a large number ofentries ( N × ( N + 1) / ) for which only a limited amount of data is available( N × T , where T is the length of the time series at one’s disposal). When1 is not extremely large compared to N , the empirically determined covari-ance matrix is highly unreliable and leads to severe instabilities when used inthe Markowitz optimisation program. Recently, some powerful mathematicaltools have been proposed to optimally “clean” the empirical covariance ma-trix, leading to a very significant improvement in the efficiency of Markowitzdiversification using the so-called “Rotationally Invariant Estimator” (RIE);for a short review see [7] and refs. therein.Another crucial step, of course, is to specify a list of expected returns foreach asset. These expected returns result either from quantitative signals(such as trend following) or from other form of analysis (quantitative orsubjective). These signals are usually extremely noisy and unreliable, soone should rather speak, as we will do below, of indicators , i.e. possiblysuboptimal and biased predictions of future returns.Once all this is done, however, a time-worn but fundamental problemremains [8, 9]. Even when sophisticated statistical tools can adequately dealwith risk , they cannot handle uncertainty , i.e. the intrinsic propensity offinancial markets to behave in a way that is not consistent with prior prob-abilities. For example, although the future “true” covariance matrix is oftenreasonably close to the cleaned (RIE) covariance matrix, correlations can alsosuddenly shift to a new regime that was never observed in the past. This isin fact worse for expected returns that are even more exposed to unknown-unknowns than volatility or correlations. One therefore needs an extra layerof control, beyond Markowitz’ optimisation, that acts as a safeguard againststatistically unexpected events.This is what the second strand of research mentioned above attempts toaddress. The idea is to add to the standard risk-return objective functionsome extra penalty terms that enforce diversification, typically in the form ofgeneralized Herfindahl indices or entropy functions [10, 5, 11]. This has ledto important breakthroughs, such as the concept of “Maximally DiversifiedPortfolios” (MDP) [3], or more recently, of “Principal Risk Parity Portfolios”(PRP) (with several variations on this theme, see Refs. [12, 6, 13, 14, 15]). Diversification and Isotropy
Although interesting, there is a hidden assumption in these penalty termsthat is far from neutral, which is the choice of the assets one considers as“fundamental”, among which risk should be as diversified as possible in theportfolio. These assets are chosen to be physical stocks for MDP’s or theprincipal components of the correlation matrix in the case of PRP’s. In thecase of long-only portfolios and traditional asset management, the choiceof physical assets as the natural “basis” for portfolio construction might bereasonable. But for – say – a portfolio of futures contracts with long and shortpositions, any linear combination of these assets is a priori feasible (at leastwithin some overall leverage constraint). In mathematical terms, one can“rotate” the natural asset basis into any a priori equivalent one. The point,however, is that a maximally diversified portfolio in one basis can in factbecome maximally concentrated in another! Take for example a portfolio of2tocks with equal weights w i = 1 /N on all N stocks. From the point of viewof the (neg)entropy S = P i w i ln w i or of the Herfindahl index H = P i w i ,this is clearly optimal. But since the leading risk factor associated with thecorrelation matrix is itself very close to an equi-weighted allocation on allstocks, a rotation onto the principal component basis α leads to the worsepossible values for both the entropy and the Herfindahl index. In otherwords, the very concept of maximal diversification is not invariant under aredefinition of the assets considered as “fundamental”. Another vivid exampleof the arbitrariness in the definition of fundamental assets is provided by theinterest rate curve or more generally, of contracts with different maturities.Should one consider the physical contracts, or only one of them and allassociated calendar spreads?Are there special directions in asset space that play a special role? Canone unambiguously identify risk factors that are more fundamental than oth-ers? This is an old problem in quantitative finance, with a long list of papersattempting to identify these factors, in particular in the equity space. How-ever, as recently reviewed by Roll [16], there is no consensus on this point.If risk is associated to volatility (or variance), then the problem is in factcompletely degenerate or, using mathematical parlance, isotropic .To make this clear, let us consider asset returns r i ( i = 1 , . . . , N ) asrandom variables with zero mean and (true) covariance matrix C , with C ij = E [ r i r j ] . One can then build N linear combinations of assets suchthat their returns b r α are all uncorrelated and of unit variance. But thischoice is not unique: in fact, any further rotation in the space of assets(i.e. an orthogonal combination of the synthetic assets returns b r α ) leadsto another set of uncorrelated, unit variance assets – see below. Amongthis infinite choice of potential “factors”, is there any one that stands out,that would justify applying a maximum diversification criterion among thesespecial assets? This is the path followed in, e.g. [17], where the furthernotion of “Minimum Torsion Bets” was introduced. Symmetries
We want to propose here a related, but different route based on symmetryarguments, which fully exploits rotation and dilation invariance at the levelof indicators as well as at the level of returns. First, let us note that one canrescale the returns of each asset i by an arbitrary factor without changing theportfolio allocation problem. Investing in a stock is the same as investing on a fictitious “2-stock” contract, with twice the returns as the original stock.So we can always choose to work with returns with unit variance, a choicethat we will make henceforth. In this case, the covariance matrix C is in fact Here and below, we assume that any non zero average return (coming for examplefrom predictive signals) is small compared to the volatility, and can be neglected in ourdiscussion. Still, of course, this non zero average returns is what motivates the portfolioconstruction to start with! What we call rotations in this paper in fact includes both proper and improper rota-tions, i.e. rotations plus inversions. b r i = X j (cid:0) C − / (cid:1) ij r j (1)is such that E [ b r i b r j ] = δ ij , i.e., to a set of uncorrelated assets. Here C − / isdefined as the positive-definite square root of C , namely: C − / = X a √ λ a u a u Ta , (2)where λ a and u a are the eigenvalues and eigenvectors of C . This is themeaning we will give throughout this paper to the square-root of a symmetricmatrix. As noted above, there is a large degeneracy in the constructionof the set of uncorrelated assets: any rotation of b r would do. A naturalchoice at this point is to insist that the ˆ r i ’s are “as close as possible” to theoriginal normalized returns, so that the financial intuition about the resultingsynthetic assets is preserved (to wit, d SPX ≈ SPX). This is the case for the ˆ r i ’s defined in Eq. (1) (see Appendix for a proof of this statement) .The same construction can be applied for statistical indicators of futurereturns that we call p i , i = 1 , . . . , N . We insist that p i is not necessarily the“true” expectation value of the future r i , but simply the best guess of theinvestor based on his information/skill set/biases, etc. A standard exampleconsidered below is a trend indicator based on a moving average of pastreturns, but any quantitative indicator based on information or intuitionwould do. These indicators fluctuate in time and are also characterized bysome covariance matrix Q ij = E [ p i p j ] . This matrix is in general non trivial,as one may systematically predict similar returns for two different assets i and j , leading to Q ij > . In any case, one can as above build N uncorrelatedlinear combinations of indicators, given by: b p i = X j (cid:0) Q − / (cid:1) ij p j , (3)with the above interpretation for Q − / . The ˆ p i ’s are then all uncorrelatedand of unit variance, i.e. with the same scale of predictability in all direc-tions, and “as close as possible” to the original p i ’s, which is again financiallymeaningful. At this stage, any rotation in the space of (synthetic) assets alsorotates the new indicators b p i while keeping them all uncorrelated and of unitvariance. The portfolio construction problem has thus become completelyisotropic. Rotationally Invariant Portfolios
How does all this help us to construct a truly agnostic Risk Parity portfolio,with no reference to a specific set of assets deemed fundamental? A simple The choice of normalization for the returns r is important here. Indeed working withnon-normalized returns would lead to a different result for b r . The choice we made is inline with our isotropy assumption. The usual case of static “long-only” indicators is special since the corresponding cor-relation matrix is ill-defined. This will be the subject of a forthcoming work. G of a portfolio invested in the syntheticasset α proportionally to b p α is given by: G = N X α =1 b p α · b r α := N X α =1 G α . (4)This portfolio has several very desirable properties: • The risk associated with each synthetic asset is the same: E [ G α ] = E [ b p α ] E [ b r α ] = 1 , provided one neglects E [ G α ] – see footnote 1. • The gains associated with different synthetic assets are uncorrelated: E [ G α G β ] = δ α,β – see previous footnote 4. • Most importantly, the total gain G is invariant under any further si-multaneous rotation R of the assets and the indicators, as should befor a scalar product: G R = N X α =1 N X β =1 R α,β b p β · N X γ =1 R α,γ b r γ = N X β =1 N X γ =1 b p β · b r γ N X α =1 R α,β R α,γ = N X β =1 b p β · b r β ≡ G (5)where we have used the fundamental property of rotation matrices RR T = I .The last property means that any arbitrary choice of uncorrelated, unit vari-ance synthetic assets with its corresponding set of indicators leads to the verysame gain, so one does not need to decide on supposedly more fundamentalinvestment factors.Why should one invest in the synthetic asset α proportionally to b p α ? Onthe basis of symmetry arguments, this is the only rational choice. All invest-ments directions are made statistically equivalent, any other choice would cor-respond to an arbitrary breaking of isotropy. In the language of Markowitzoptimisation, this corresponds to the optimal portfolio of synthetic assetswhen the expected future return of α is S b p α , where the expected Sharperatio S is independent of α . Note that this in fact relies on the assumptionthat E [ b p α b r β ] = Sδ α,β , i.e. that at the level of uncorrelated factors, thereis no significant cross-prediction left. This is, we believe, a very plausibleassumption in practice – see below. This implicitly assumes that the cross-correlations between the b p α and the b r β = α aresmall, which is in fact an important hypothesis underlying our rotational symmetry prin-ciple. b r α and b p α : G = N X α =1 N X i,j =1 (cid:0) Q − / (cid:1) αj p j (cid:0) C − / (cid:1) αi r i = N X i =1 N X α =1 N X j =1 (cid:0) C − / (cid:1) αi (cid:0) Q − / (cid:1) αj p j ! r i := N X i =1 π i r i , (6)where the last equation defines the physical position π i in a asset i , which isthus found to be: π i = ω N X α =1 N X j =1 (cid:0) C − / (cid:1) αi (cid:0) Q − / (cid:1) αj p j , (7)where ω is a constant that sets the overall risk of the portfolio, or, in vectorialform (using the symmetry of C ): π = ω C − / Q − / p (8)This is the central result of this paper, that we now comment and spe-cialize to several situations. First, let us notice that the above portfolioconstruction is such that the expected risk along any eigen-direction of C isthe same, hence the name “Eigenrisk Parity Portfolio” (ERP) – on this topic,see also [12, 13]. Indeed, the expected risk along the a th principal componentis given by: R a = E [( π · v a ) ] λ a , (9)where λ a is the a th eigenvalue of C and v a the corresponding eigenvector.Simple algebra then leads to: R a = ω (cid:18) √ λ a (cid:19) E [( v a · Q − / p ) ] λ a = ω ∀ a, (10)where we have used the fact that the expected covariance of the indicatoris Q . Note that although for any given day the allocation π points in aspecific direction and is thus “fully concentrated” in that sense, this directionis expected to change over time – provided the indicators themselves are notstatic. Isotropy is thus statistically restored on long enough time scales. Agnostic Risk Parity
Now, the naive choice for the indicator covariance matrix Q should be pro-portional to the return covariance matrix itself, i.e. Q ∝ C . In a stationary6orld where the indicators would really statistically predict future returns,i.e. p i = E [ r fut. i ] , this assumption would be natural, at least when C is com-puted on the time scale of the predicted returns, which is usually much longerthan a day. Interestingly, plugging Q ∝ C in Eq. (8) above precisely leadsto the standard Markowitz optimal portfolio: π = ω C − E [ r fut. ] . However,this is a highly over-optimistic view of the world that only deals with “knownunknowns”. Directional predictions are extremely uncertain, much more sothan risk predictions. In fact, directional predictions should not even be pos-sible in an efficient market. If one insists that some signals may (weakly)predict future returns, it is wiser not to assume any particular structureon the correlation matrix of these indicators that any optimizer would useto hedge some bets with other bets. The most agnostic choice, less proneto unknown unknowns, is to choose Q = σ p I , i.e. no reliable correlationsbetween the realized predictions, and the same amount of predictability (orexpected Sharpe ratio) on all assets. This leads to a very interesting portfolioconstruction: π ∗ = ω C − / RIE p (11)coined henceforth as “Agnostic Risk Parity” (ARP) because this specificasset allocation allows one to precisely balance the risk between all the prin-cipal components of the (cleaned) covariance matrix C RIE , in the worst-casescenario where the realized correlations between indicators would completelybreak down.Note that there is no explicit optimisation used in this argument – rather,we look for a rotationally invariant portfolio construction with the minimalamount of information on the correlation structure of the indicators. Therisk distribution per eigen-mode for various portfolio allocations is drawn inFig. 1, when the realized covariance of the indicators is Q = σ p I . Notethat, as is well known, the Markowitz optimisation scheme tends to over-allocate on small eigen-modes, which can lead to significant out-of-sample(bad) surprises [2], a bias that is corrected within the ARP framework.Finally, one might believe that although uncertain, part of the returncorrelations could be inherited by the indicators. A simple way to encodethis is to use for Q a shrinkage estimator, i.e. Q ∝ ϕ C RIE + (1 − ϕ ) I , where ϕ ∈ [0 , allows one to smoothly interpolate between complete uncertainly( ϕ = 0 ), corresponding to ARP, and the standard Markowitz prescription( ϕ = 1 ). Agnostic Trend Following
The previous discussion was rather formal. As an example, we considerhere the universal “Trend” indicator, based on a 1-year flat moving averageof past returns of a collection of 110 futures contracts (commodities, FX,indices, bonds and interest rates) – see the discussion in [18]. We normalizethe returns of all futures and all the predictors to have unit variance. We thenuse three different portfolio constructions: equal /N risk on each physical7igure 1: Realized risk carried by different eigen-modes resulting from threeportfolio constructions: /N on futures contracts, Markowitz, and AgnosticRisk Parity, all in the case where indicators are such that their realizedcovariance is Q = σ p I . 8igure 2: Profit & Loss (P&L) curves for universal trend following for fourportfolio constructions: /N on futures contracts, Markowitz with or withouta cleaned RIE correlation matrix, and Agnostic Risk Parity, again with RIE.The universe here is composed of 110 contracts (commodities, FX, indices,bonds and interest rates). The trend indicator is a 1-year flat moving averageof past returns. All P&L’s are rescaled such that their realized volatility isthe same.asset, Markowitz optimal portfolio with either the raw empirical correlationmatrix or a cleaned version C RIE (using the RIE estimator detailed in [7],and no future information) and the Agnostic Risk Parity, again using theRIE estimator for C RIE . The P&L’s of the different portfolio since 1998 areshown in Figure 2. While part of the improvement comes – as expected – fromusing a cleaned correlation matrix, we see that Agnostic Risk Parity yieldsthe best result. Clearly the true correlation of predicted yearly returns Q isnearly impossible to measure without centuries of data, hence motivating thechoice Q = σ p I . We have observed similar results for other standard CTAstrategies. 9 erspectives In summary, we have offered a new perspective on portfolio allocation, whichavoids any explicit optimisation but rather takes the point of view of symme-try . In a context where linear combinations of assets can easily be synthesizedin a portfolio whose risk is measured through volatility, the asset space can bemade fully “isotropic”, in the sense that no preferred directions (correspond-ing to specific risk factors) can be identified. Therefore, in the absence ofextra information, portfolio construction should respect this symmetry. Thisonly requirement leads to a precise allocation formula, Eq. (8), that gen-eralizes Markowitz’ prescription such as to take into account the expectedcorrelation between the predicted returns of each asset in the portfolio. Wehave argued that the most agnostic choice, which is probably the most ro-bust one out-of-sample, is to assume that these correlations are zero, i.e. thatone should refrain from trying to hedge different bets if there is no certaintyabout the correlations between these bets. This leads to an Agnostic ParityPortfolio that realizes an equal risk over all principal components of the co-variance matrix. We found that such an allocation over-performs Markowitz’portfolios when applied to classic technical (CTA) strategies, such as (uni-versal) trend following. There are several routes that should be exploredfurther. For example, non-quadratic measures of risk, such as skewness orkurtosis, would break rotational symmetry and possibly lead to meaningfulfundamental risk factors that should be maximally diversified (see e.g. [19]).We leave this for future work.We thank N. Bercot, J. Bun, R. Chicheportiche, S. Ciliberti, C. Deremble,L. Duchayne, L. Laloux, A. Rej for many useful discussions on these issues.
Appendix
We consider random variables r of mean zero and unit variance, with acorrelation matrix given by C . We are looking for b r , a linear transformationof r such that E [ b r i b r j ] = δ ij . Clearly b r = C − / r satisfy this property, andany solution is of the form b r = R C − / r where R is a rotation matrix. Wefurther demand that the following Mahalanobis distance d ( R ) is minimized: d ( R ) = E (cid:2) ( b r − r ) · C − ( b r − r ) (cid:3) . (12)Expanding the square, one readily sees that the only term that depends on R is − Tr [ R C − / ] , which must be maximized. Since C − / is a positivedefinite matrix, it is immediate to show that the optimal solution is R = I ,i.e. C − / is diagonal in the basis of C . Although natural in the present con-text, we note that changing the Mahalanobis distance to any other positivedefinite quadratic form where C − is replaced by any (matrix) function of C – including the identity matrix – leads to the same result.10 eferences [1] F Black, R Litterman, Global portfolio optimization , Financial AnalystsJournal, 1992[2] R Michaud,
Efficient Asset Management: A Practical Guide to StockPortfolio Optimization and Asset Allocation . Cambridge, MA: HarvardBusiness School Press, 1998.[3] Y Choueifaty, Y Coignard,
Towards Maximal Diversification , JournalPortfolio Management, Fall, 40-51 (2008).[4] T Roncalli,
Introduction to Risk Parity and Budgeting , Chapman andHall, (2013)[5] A Meucci,
Managing diversification , Risk, 22, 74-79 (2009)[6] R Deguest, L Martellini, and A Meucci,
Risk parity andbeyond - from asset allocation to risk allocation decisions http://ssrn.com/abstract=2355778. EDHEC-Risk Working Paper(2013).[7] J Bun, J P Bouchaud, M Potters,
Cleaning correlation matrices ,Risk magazine (March 2016) & J Bun, J P Bouchaud, M Potters,arXiv:1610.08104, to appear in Physics Reports, 2016.[8] J M Keynes,
The General Theory of Employment, Interest and Money ,1934[9] N N Taleb,
The Black Swan , Random House Trade (2010)[10] J P Bouchaud, M Potters, J P Aguilar,
Missing information and assetallocation , arXiv preprint cond-mat/9707042 (1997)[11] G Frahm, C Wiechers,
On the diversification of portfolios of risky assets ,University of Cologne (2011)[12] M H Partovi, M Caputo,
Principal Portfolios: Recasting the EfficientFrontier , Economics Bulletin, 7, 1-10 (2004)[13] C Kind,
Risk-Based Allocation of Principal Portfolios ,http://ssrn.com/abstract=2240842 (2012)[14] H Lohre, U Neugebauer, C Zimmer,
Diversified Risk Parity Strategiesfor Equity Portfolio Selection , Journal of Investing, 21, 111-128, (2012)[15] D H Bailey, M Lopez de Prado,
Balanced Baskets: A New Approachto Trading and Hedging Risks
Journal of Investment Strategies, 1, 21-62(2012)[16] K Pukthuanthong, R Roll,
A Protocol for Factor Identification ,http://ssrn.com/abstract=2517944 (2014)1117] A Meucci, A Santangelo, R Deguest,
Risk Budgeting andDiversication Based on Optimized Uncorrelated Factors ,http://ssrn.com/abstract=2276632, Risk Magazine (November 2015).[18] Y Lempérière, C Deremble, P Seager, M Potters and J P Bouchaud,
Two Centuries of Trend Following , Journal of Investing Strategies 3, 41-61, (2014), and refs. therein.[19] E Baitinger, A Dragoschy, A Topalovaz,