Test of Covariance and Correlation Matrices
Longyang Wu, Chengguo Weng, Xu Wang, Kesheng Wang, Xuefeng Liu
aa r X i v : . [ s t a t . M E ] D ec Test of Covariance and Correlation Matrices
Longyang Wu , Chengguo Weng , Xu Wang , Kesheng Wang , and Xuefeng Liu ThermoFisher Scientific, San Clara, CA, 95051 Department of Statistics and Actuarial Science, University of Waterloo, Waterloo, Canada,N2L 3G1 Department of Mathematics, Wilfrid Laurier University, Waterloo, Canada, N2L 3G5 Department of Biostatistics and Epidemiology, East Tennessee State University, Johnson City,TN, 37614 Department of Systems, Populations and Leadership, University of Michigan, Ann Arbor, MI,48109 Department of Biostatistics, University of Michigan, Ann Arbor, MI, 48109
December 5, 2018
Abstract
Based on a generalized cosine measure between two symmetric matri-ces, we propose a general framework for one-sample and two-sample tests ofcovariance and correlation matrices. We also develop a set of associated per-mutation algorithms for some common one-sample tests, such as the testsof sphericity, identity and compound symmetry, and the K -sample tests ofmultivariate equality of covariance or correlation matrices. The proposedmethod is very flexible in the sense that it does not assume any under-lying distributions and data generation models. Moreover, it allows datato have different marginal distributions in both the one-sample identity and K -sample tests. Through real datasets and extensive simulations, we demon-strate that the proposed method performs well in terms of empirical type Ierror and power in a variety of hypothesis testing situations in which dataof different sizes and dimensions are generated using different distributionsand generation models. KEY WORDS:
Sphericity test; Identity test; Compound symmetry test;Two-sample test; K -sample test Testing covariance and correlation matrices is an enduring topic in statistics. Inaddition to its apparent importance in classical multivariate analysis such as multi-1ariate analysis of variance, Fisher’s linear discriminant analysis, principle compo-nent analysis, and repeated measure analysis (
Johnson and
Wichern
Green
Annaert et al.
Singh et al.
Djauhari and
Gan χ test for the equality of two correlation matrices of multivariate normaldistributions. Jennrich’s test has been adopted in several statistical softwaresand applied in financial studies. Likelihood ratio based tests for covariance ma-trices of multivariate normal distributions are well documented in the classicalmultivariate analysis literature (e.g. Muirhead (2005), Chp. 8). Since the clas-sical likelihood ratio based tests are generally not applicable in high-dimensionaldata due to the singularity of sample covariance matrices, the recent methodol-ogy developments focus on the tests of high-dimensional covariance and corre-lation matrices, where the dimension p is much bigger than the sample size n .Among others, Ledoit and Wolf (2002) studied the performance of two classicalstatistics for the one-sample test of sphericity and identity in a high-dimensionalsetting. Chen, Zhang, and Zhong (2010) adopted the similar test statistics as inLedoit and Wolf (2002) but used U-statistics to estimate the population covariancematrix. Li and Chen (2012) proposed a two-sample equality test of covariancematrices. Fujikoshi, Ulyanov, and Shimizu (2011) provided a summary of someclassical and high-dimensional tests of covariance matrices of multivariate normaldistributions. Yao, Zheng, and Bai (2015) discussed the recent developments onestimation and hypothesis tests of covariance matrices in high dimensions fromthe random matrix theory viewpoint. Cai (2017) presented an up-to-date reviewof methodology developments of the equality test of high-dimensional covariancematrices with sparse signals.There obviously lacks a unified and flexible hypothesis testing framework forpractitioners to handle both covariance and correlation matrices without switchingbetween different test statistics or procedures. In this study we propose such aunified and easy-to-use approach for many common tests of covariance and cor-relation matrices. We exploit a simple geometric property of symmetric matricesto construct test statistics and develop the associated permutation algorithms tocompute p-values under a variety of null hypotheses. Our proposed method canhandle a variety of one-sample tests including the tests of sphericity, identity andcompound symmetry, and two- and K -sample equality tests ( K > K -sampletests, even for high-dimensional data where the dimension p is much bigger thanthe sample size n .This paper is organized as follows. Section 2 introduces a generalized cosinemeasure between two symmetric matrices and, based on this notion of generalizedcosine, the test statistics for one-sample and two-sample tests are proposed. Sec-tion 3 studies the one-sample tests of sphericity, identity and compound symmetry.Section 4 discusses the two-sample equality test of covariance or correlation matri-ces and extends the method to handle the K -sample problem. Section 5 containsextensive simulation studies on the performance of the proposed one-sample andtwo-sample tests. In addition, Section 5 probes into factors that affect the perfor-mance of the proposed methods. The paper concludes with discussions in Section6. Below is the definition of the generalized cosine measure between two symmetricmatrices.
Definition.
Let R p × p be a vector space of symmetric matrices of size p × p and f : R p × p → R m × n . The angle between two symmetric matrices M and M in R p × p with respect to f is defined as arccos ( M , M ) and cos ( M , M ) = h f ( M ) , f ( M ) ik f ( M ) kk f ( M ) k , (1) where h· , ·i is an inner product and k · k is the corresponding norm in R m × n . We use two symmetric matrices A and B to illustrate some possible cosinevalues with different mappings, their associated inner products and norms. Matrix A is .
00 0 .
50 0 .
33 0 .
25 0 . .
50 1 .
00 0 .
25 0 .
20 0 . .
33 0 .
25 1 .
00 0 .
17 0 . .
25 0 .
20 0 .
17 1 .
00 0 . .
20 0 .
17 0 .
14 0 .
12 1 . , A ( i, j ) = ( i + j − , i = j, , i = j, (2)for i, j = 1 , . . .
5. Matrix B is .
00 0 .
74 0 .
83 0 .
54 0 . .
74 1 .
00 0 .
55 0 .
60 0 . .
83 0 .
55 1 .
00 0 .
28 0 . .
54 0 .
60 0 .
28 1 .
00 0 . .
41 0 .
34 0 .
58 0 .
48 1 . , which is a realization of the random matrix A ∗ , constructed from A , as follows: A ∗ ( i, j ) = A ∗ ( j, i ) = ( i + j − + ǫ, i = j, , i = j, (3)where ǫ is generated from a uniform distribution over the interval (0, 0.5).Case 1: Suppose f is an identity mapping, i.e., f ( A ) = A . We considerthe Frobenius inner product h A , B i F = tr( A T B ) and its corresponding norm k A k F = tr( A T A ) . By Equation (1), the cosine between A and B is calculatedas cos ( A , B ) = tr( A T B )tr( A T A ) tr( B T B ) , (4)which is 0.92, corresponding to an angle of 0.41 radian or 23 ◦ .Case 2: In this case, we construct a two-step mapping f : R p × p → R m . In stepone we apply the Cholesky decomposition to obtain A = L L T and B = L L T ,where L and L are lower triangle matrices with positive diagonal elements. Instep two we apply the half-vectorization operator (which vectorizes or stacks onlythe lower trianguler portion of a symmetric matrix) to L and L separately toobtain two vectors vech( L ) and vech( L ), each with a length of m = p ( p + 1) / cos ( A , B ) = vech( L ) T vech( L ) k vech( L ) kk vech( L ) k , (5)which is 0.87, corresponding to an angle of 0.52 radian or 29 . ◦ . This mapping isfeasible only if A and B are positive definite matrices.Case 3: The mapping f can also be constructed by first applying the eigen-decomposition to obtain Q Λ Q T for A and Q Λ Q T for B , respectively. Q and Q are orthogonal matrices of eigenvectors, while Λ and Λ are diagonal matrices4hose elements are the eigenvalues of A and B , respectively. Provided that notall diagonal elements in Λ and Λ are zero, we can extract the diagonal elementsfrom Λ and Λ to create vectors v and v , respectively. Thus, we construct amapping f : R p × p → R p . To apply Equation (1), we adopt the Euclidean innerproduct and norm to compute the cosine as cos ( A , B ) = v T v k v kk v k , (6)which is 0.93, corresponding to a angle of 0.37 radian or approximately 21 ◦ .Case 4: Since a p × p symmetric matrix is completely determined by its lowertriangular elements together with the symmetry, we construct the mapping f byapplying the half-vectorization operator on A and B directly. The cosine between A and B can be obtained from Equation (1) as follows: cos ( A , B ) = vech( A ) T vech( A ) k vech( A ) kk vech( B ) k , (7)which is 0.94, corresponding to a angle of 0.35 radian or 20 ◦ .For correlation matrices, the computation can be simplified by introducing amodified half-vectorization operator vech ∗ ( · ) from vech( · ) by excluding the diago-nal elements of the matrix. Suppose A and B are two correlation matrices, thecosine can be computed by Equation (1) as cos ( A , B ) = vech ∗ ( A ) T vech ∗ ( A ) k vech ∗ ( A ) kk vech ∗ ( B ) k , (8)which is 0.95, corresponding to 0.31 radian or 18 ◦ .In summary, we have considered four different mappings to compute the gener-alized cosine between the two modified Hilbert matrices A and B . The mappingbased on the Cholesky decomposition gives the lowest cosine value of 0.87. Themappings based on the Frobenius inner product and the eigen-decomposition givethe similar cosine values of 0.92 and 0.93, respectively. When the redundant in-formation is removed, the half-vectorization and the modified half-vectorizationoperators give the largest cosine values of 0.94 and 0.95, respectively. In thisstudy we choose the half-vectorization operator, vech( · ), for covariance matricesand the modified half-vectorization operator, vech ∗ ( · ), for correlation matrices,because these two operators completely remove the redundancy in symmetric ma-trices and are very easy to compute. Moreover, these two operators demonstratebetter statistical power in our pilot simulation studies. The cosine value com-puted from Equation (1) measures the similarity between two symmetric matrices.When this value is one, the two matrices are identical. We exploit this propertyto propose new test statistics for one-sample and two-sample tests of covarianceand correlation matrices. 5 ne-sample test statistic. Let S be the sample covariance (correlation) matrixwith the corresponding population covariance (correlation) matrix Σ . The teststatistic for the equality of Σ and Σ , which is either known or specified up tosome unknown parameters, is − cos ( S , Σ ) , (9) where the cosine is computed according to Equation (1) under a pre-determinedmapping f with a properly chosen inner product and norm. Two-sample test statistic.
Let S and S be the sample covariance (correlation)matrices with the corresponding population covariance (correlation) matrices Σ and Σ , respectively. The test statistic for the equality of Σ and Σ is − cos ( S , S ) , (10) where the cosine is computed according to Equation (1) under a pre-determinedmapping f with a properly chosen inner product and norm. It is worth pointing out that the two newly proposed test statistics exploita simple geometric property of symmetric matrices and require no distributionalassumptions of data. We further adopt a permutation approach to obtain thenull distributions of the two test statistics and compute p-values under the nullhypotheses of different tests.
Suppose a matrix, D n × p = { x ij } , i = 1 , . . . , n and j = 1 , . . . , p , contains datacollected on n subjects and each with p covariates. Its i th row is a realization ofrandom vector X = ( X , X , . . . , X p ) T , which has an unknown distribution with acovariance matrix Σ and a correlation matrix R . In the one-sample problem onemay use the data matrix D n × p to test if the covariance matrix Σ is proportionalto an identity matrix, known as the sphericity test, or to test if Σ or R equals toan identity matrix, known as the identity test. The hypotheses in the sphericity test for a covariance matrix are as follows: H : Σ = σ I vs . H : Σ = σ I , (11)where σ is the unknown population variance under the null. In this case, Σ inthe one-sample test statistic (9) is σ I and S is the sample covariance matrix of6he data matrix D n × p . To test the sphericity of a covariance matrix, using vech( · ),the Euclidean inner product and norm, the test statistic (9) can be shown to be1 − tr( S ) √ p k vech( S ) k , (12)where tr( · ) denotes the trace. The derivation of (12) is provided in Appendix I.Note that the test statistic (12) does not depend on the unknown parameter σ .The sphericity test (11) is a combination of two tests ( Fujikoshi et al. H : Σ = σ I vs. H : Σ = diag( σ , σ , . . . , σ p ) and 2) H : Σ = diag( σ , σ , . . . , σ p )vs. H : Σ = diag( σ , σ , . . . , σ p ). To handle these two tests simultaneously, wepropose Algorithm 1 to compute p-values under the null of sphericity. This al-gorithm first permutes the elements in each row of D n × p to generate a permuteddata matrix D † n × p , then the elements in each column of D † to generate D ‡ n × p . Thepermutation on each row of the data matrix is to test if the diagonal elements in Σ are identical; the permutation on each column is to test if the off-diagonal elementsof Σ are zero. Algorithm 1 assumes that the elements in the random vector X areexchangeable in the sense that Σ is invariant to the permutations of the elementsin X under the null hypothesis of (11). This algorithm requires that the elementsin X have the same distribution so that the elements in each row of D n × p can bepermuted. Algorithm 1:
Test the sphericity of a covariance matrix. r ← the number of permutations T ( i ) ← i = 1 , . . . , r S ← Compute the sample covariance matrix of D n × p T o ← Compute the test statistic (12) using S For i = 1 to i = r D † n × p ← Permute the elements in each row of D n × p D ‡ n × p ← Permute the elements in each column of D † n × p S ∗ ← Compute the sample covariance matrix of D ‡ n × p T ( i ) ← Compute the test statistic (12) using S ∗ End ForReport p -value ← ( T ( i ) ≥ T o ) + 1) / ( r + 1)We illustrate the proposed sphericity test and Algorithm 1 using the bfi data inthe R package pysch ( Revelle bfi data contain twenty-five personal-ity self reported items from 2800 subjects who participated in the Synthetic Aper-ture Personality Assessment (SAPA) web based personality assessment project.7hese twenty-five items are organized by five putative factors: Agreeableness, Con-scientiousness, Extraversion, Neuroticism, and Openness. Each subject answeredthese personality questions using a six point response scale ranging from 1 (VeryInaccurate) to 6 (Very Accurate). After removing the missing data, we apply theproposed sphericity test and Algorithm 1 to test the sphericity of the 25 ×
25 covari-ance matrix of personality measurements of 2436 subjects. We obtain a p-value of0.01 based on 100 permutations using Algorithm 1. Therefore, the null hypothesisof sphericity is rejected at the significance level of 0.05. We will further investigatethe empirical type I error and power of our proposed sphericity test in Section 5.
The identity test of a covariance matrix can be formulated as (
Ledoit and
Wolf H : Σ = I vs . H : Σ = I , (13)which includes the more general null hypothesis H : Σ = M , where M is aknown covariance matrix, since we can replace M by I and multiply data by M − / .Because I in the hypothesis (13) and σ I in the sphericity hypothesis (11) onlydiffer by a constant σ and the test statistic (12) does not depend on σ , we canadopt the test statistic (12) to test the hypothesis (13).For the identity test of a correlation matrix, H : R = I vs . H : R = I , (14)for which, the test statistic (12) can be simplified into1 − √ p k vech( S ) k . (15)The derivation of (15) is provided in Appendix I.Since the identity test mainly concerns the off-diagonal elements of a covari-ance or correlation matrix, we propose Algorithm 2, which permutes the elementsin each column of the data matrix D n × p , to compute p-values under the null.Permuting the elements in columns can disrupt the covariance structure, but itdoes not change the sample variance of each column of the data matrix, i.e. thediagonal elements of the sample covariance matrix remain unchanged in differentpermutations. This implies that, for a given D n × p , tr( S ) remains a constant in(12) in different permutations. For a correlation matrix S , we have tr( S )= p . Sincethe column-wise permutations can sufficiently disassociate every pair of randomvariables, and hence, disrupt the covariance structure of the p random variables,8he off-diagonal elements of the sample covariance matrix of the permuted dataare much closer to zero than the original sample covariance (correlation) matrixunder the alternative. Therefore, the sample covariance and correlation matriceswith a good portion of non-zero off-diagonal elements have larger values of thestatistics of (12) and (15), respectively, to reject the null hypothesis of identity.Moreover, Algorithm 2 does not require each column of a data matrix to have thesame distribution. Algorithm 2:
Test the identity of covariance or correlation matrix. r ← the number of permutations T ( i ) ← i = 1 , . . . , r S ← Compute the sample covariance or correlation matrix of D n × p T o ← Compute the test statistic (12) or (15) using S For i = 1 to i = r D ∗ n × p ← Permute the elements in each column of D n × p S ∗ ← Compute the sample covariance or correlation matrix of D ∗ n × p T ( i ) ← Compute the test statistic (12) or (15) using S ∗ End ForReport p -value ← ( T ( i ) ≥ T o ) + 1) / ( r + 1)In addition to the Pearson’s correlation matrix, the test statistic defined in(15) and its associated Algorithm 2 can handle the Spearman’s and the Kendall’srank correlation matrices, because this test statistic is based on the cosine measurebetween a sample correlation matrix S and I , and does not require any particulartypes of correlation matrices.We use the bfi data as described in section 3.1 to illustrate the proposed iden-tity test and its associated Algorithm 2 for correlation matrices. For a Pearson’scorrelation matrix of twenty-five personality self reported items, Algorithm 2 pro-duces a p-value of 0.01 based on 100 permutations. The Bartlett’s identity testimplemented in the same R package gives a p-value of zero. We also apply theidentity test to the Spearman’s rank and Kendell’s rank correlation matrices. Al-gorithm 2 produces a p-value of 0.01 for both types of correlation matrices basedon 100 permutations . Therefore, we reject the identity of these three types of cor-relation matrices. We will further study the performance of our proposed identitytest in Section 5. The hypothesis of compound symmetry is H : Σ = σ [(1 − ρ ) I + ρ J ] for acovariance matrix or H : R = [(1 − ρ ) I + ρ J ] for a correlation matrix, where J is9 square matrix of ones and ρ > in the one-sample test statistic (9) is σ [(1 − ρ ) I + ρ J ] for a covarianceand [(1 − ρ ) I + ρ J ] for a correlation matrix, respectively. Using the modified half-vectorization operator, vech ∗ ( · ), a unified test statistic for testing the compoundsymmetry of both covariance and correlation matrices can be obtained from (9) as1 − vech ∗ ( S ) T vech ∗ ( J ) p . p ( p − k vech ∗ ( S ) k , (16)where S is either a sample covariance or correlation matrix. The derivation of (16)is provided in Appendix II. Note that this test statistic does not depend on theunknown parameters σ and ρ .Suppose a data matrix D n × p contains rows that are the realizations of a p × X = ( X , . . . , X p ) T , in which all X i ’s have the same distribution.Under the null hypothesis of compound symmetry, it is true that V ar ( X i ) = σ , forall i ’s and Cov ( X i , X j ) = σ (1 − ρ ) for all pairs of i, j , provided i = j . This impliesthat the random permutations of the elements in X do not alter its covariancematrix under the assumption of compound symmetry. Using this fact, we proposeAlgorithm 3 to compute p-values for the test statistic (16) by randomly permutingthe elements in each row of the data matrix. For this algorithm to be applicable,each column of the data matrix must have the same distribution. Algorithm 3:
Test the compound symmetry of covariance (correlation) ma-trix. r ← the number of permutations T ( i ) ← i = 1 , . . . , r S ← Compute the sample covariance (correlation) matrix of D n × p T o ← Compute the test statistic (16) using S For i = 1 to i = r D ∗ n × p ← Permute the elements in each row of D n × p S ∗ ← Compute the sample covariance (correlation) matrix of D ∗ n × p T ( i ) ← Compute the test statistic (16) using S ∗ End ForReport p -value ← ( T ( i ) ≥ T o ) + 1) / ( r + 1)For an illustrative purpose, we consider the Cork data (Rencher and Christensen (2012),Table 6.21). Four cork borings, one in each of the four cardinal directions, were10aken from each of twenty-eight trees and their weights were measured in integers.The goal is to test the compound symmetry of the covariance matrix of theseweight measurements taken in the four cardinal directions. Using the modifiedhalf-vectorization operator vech ∗ ( · ) and the Euclidean inner product and norm,we obtain a cosine of 0.99 (0.998) between the sample covariance (correlation) ma-trix and the compound symmetry, indicating a strong similarity. We apply ourproposed compound symmetry test (16) to this dataset and, using Algorithm 3, ob-tain a p-value of 0.099 for the compound symmetry test of the sample covariancematrix and 0.069 for the sample correlation matrix based on 100 permutations.Hence, the null hypothesis of compound symmetry can not be rejected at thesignificance level of 0.05. Rencher and Christensen (2012), however, rejected thecompound symmetry based on a χ -approximation to a likelihood ratio test us-ing a multivariate normal distribution (Rencher and Christensen (2012), Example7.2.3). This discrepancy is due to their inappropriate adoption of the multivariatenormal distribution, since three of the four marginal distributions failed to passthe normality test. The test statistic (16) and Algorithm 3, on the other hand, re-quire no distributional assumptions and are robust to the underlying distributionof Cork borings weights. K -sample test ( K > ) In this section we will first discuss the test statistics and their associated algo-rithms for the two-sample problem ( K =2), then extend the method to handle the K -sample problem. Let D n × p and D n × p be two data matrices with sample co-variance (correlation) matrices S and S , respectively. Suppose the rows of D n × p are the realizations of a p × X = ( X , . . . , X p ) T and the rows of D n × p are of random vector X = ( X , . . . , X p ) T . Let Σ ( R ) be the covariance(correlation) matrix of X and Σ ( R ) be the covariance (correlation) matrix of X .In the two-sample problem, the null hypotheses are H : Σ = Σ for covariancematrix and H : R = R for correlation matrix. We adopt vech( · ), the Euclideaninner product and norm in the two-sample test statistic (10) to test the equalityof covariance matrices, leading to a test statistic1 − vech( S ) T vech( S ) k vech( S ) kk vech( S ) k , (17)and vech ∗ ( · ), the Euclidean inner product and norm to test the equality of corre-lation matrices, yielding a test statistic1 − vech ∗ ( S ) T vech ∗ ( S ) k vech ∗ ( S ) kk vech ∗ ( S ) k , (18)11here S and S are the sample covariance or correlation matrices.We propose Algorithm 4 to compute p-values for these test statistics under thenull hypothesis of equality. In this algorithm two data matrices D n × p and D n × p are stacked to form a new data matrix D n × p ( n = n + n ). In each permutationthe rows of D n × p are randomly permuted to generate a permuted data matrix D ∗ n × p , which is then split into two data matrices of D ∗ n × p and D ∗ n × p to computethe test statistic (17) or (18). Algorithm 4 assumes that X i in X and X i in X have the same distribution for all i ’s. One advantage of our proposed two-sampletest is that X i and X j , i = j , need not to have the same distribution.Under the null hypothesis of equality Σ = Σ = Σ , the sample covariancematrices S ∗ of D ∗ n × p and S ∗ of D ∗ n × p can be considered as the realizations ofmatrix Σ . The rationale behind Algorithm 4 is that the cosine value between S ∗ and S ∗ is similar to that of S and S under the null hypothesis and thepermutations provide a good control of the type I error. Under the alternative,the repeated random-mixing rows of D n × p and D n × p produce S ∗ and S ∗ suchthat the cosine value between the two is bigger than that of S and S , therefore thetest statistics (17) and (18) have good power to reject the null at a pre-determinedsignificance level. Algorithm 4:
Test for the equality of two covariance (correlation) matrices. r ← the number of permutations T ( i ) ← i = 1 , . . . , r S ← Compute the sample covariance (correlation ) matrix from D n × p S ← Compute the sample covariance (correlation ) matrix from D n × p T o ← Compute (17) for covariance or (18) for correlation matrix D ( n + n ) × p ← stack D n × p and D n × p For i = 1 to i = r D ∗ ( n + n ) × p ← randomly shuffle the rows of D ( n + n ) × p D ∗ n × p ← the first n rows of D ∗ ( n + n ) × p D ∗ n × p ← the remaining n rows of D ∗ ( n + n ) × p S ∗ ← Compute the sample covariance (correlation ) matrix from D ∗ n × p S ∗ ← Compute the sample covariance (correlation ) matrix from D ∗ n × p T ( i ) ← Compute (17) for covariance or (18) for correlation matrixEnd ForReport p -value = ( T ( i ) ≥ T o ) + 1) / ( r + 1)We use the flea beetles data (Rencher and Christensen (2012), Table 5.5) toillustrate our proposed two-sample equality test. This dataset contains four mea-surements of two species of flea beetles, Haltica oleraces and
Haltica cardorum . To12est the equality of two covariance matrices of the four measurements of the twoflea beetle species, we apply the proposed two-sample test (17) and obtain a p-value of 0.37 using Algorithm 4 with 100 permutations. Assuming normality andusing χ - and F - approximation, the Box ′ s M - test gives a similar scale p-value of0.56 ( Rencher and
Christensen
Box ′ s M - test , and the null hypothesisof the equality of two covariance matrices is not rejected at the significance levelof 0.05.To test the multivariate equality of several covariance or correlation matrices,we consider H : Σ = Σ = · · · = Σ K for the equality of multiple covariancematrices and H : R = R = · · · = R K for the equality of multiple correlationmatrices. We propose the test statistic T = max { T , T , . . . , T ( K − K } , (19)where T ij is the test statistic (17) or (18) for the pairwise two-sample compari-son of Σ i ( R i ) and Σ j ( R j ) for all possible unique pairs of K populations. Let D n × p , D n × p , . . . , D n K × p be the data matrices from K populations. Algorithm 5provides a permutation approach to compute p-values under the null hypothesisof the multivariate equality of covariance or correlation matrices. Algorithm 5:
Test for the equality of K covariance (correlation) matrices. r ← the number of permutations T ( i ) ← i = 1 , . . . , rT o ← Compute the test statistic (19) for D n × p , D n × p , . . . , D n K × p D ← stack D n × p , D n × p , . . . , D n K × p For i = 1 to i = r D ∗ ← randomly shuffle the rows of DD ∗ n × p ← the first n rows of D ∗ D ∗ n × p ← the second n rows of D ∗ ... ... ... D ∗ n K × p ← the remaining n K rows of D ∗ T ( i ) ← Compute the test statistic (19) for D ∗ n × p , D ∗ n × p , . . . , D ∗ n K × p End ForReport p -value = ( T ( i ) ≥ T o ) + 1) / ( r + 1)We use the Rootstock data (Rencher and Christensen (2012), Table 6.2) toillustrate our proposed K -sample test. Eight apple trees from each of the six13ootstocks were measured on four variables: 1) trunk girth at 4 years; 2) extensiongrowth at 4 years; 3) trunk girth at 15 years; 4) weight of the tree above groundat 15 years. The null hypothesis is H : Σ = Σ = Σ = Σ = Σ = Σ , (20)for the equality of six covariance matrices of the six rootstocks. The likelihoodratio based Box ′ s M - test yields a p-value of 0.71 and 0.74 based on χ - and F -approximation ( Rencher and
Christensen
Box ′ s M - test , and the null hypothesis of the multivariate equality of these sixcovariance matrices is not rejected at the significance level of 0.05. In this section we carry out extensive simulations to investigate the empirical typeI error and power of our proposed one-sample and two-sample tests. In the one-sample setting we investigate the proposed sphericity test and identity test. In thetwo-sample setting, we investigate the proposed equality test of two covariancematrices. We design and conduct these simulation studies with the backdrop ofhigh dimensions. We also probe into factors that may affect the performance of theproposed methods. All simulations are conducted in the R computing environment(
R Core Team
In this section we evaluate the empirical type I error and power of our proposedsphericity test. We adapt the simulation design and model of
Chen et al. (2010)for a p -dimensional random vector X = ( X , . . . , X p ) T as X = Γ Z , (21)where Γ is a p × m constant matrix with p ≤ m and Z = ( Z , · · · , Z m ) T is a m -dimensional random vector with V ar ( Z ) = I . We further let ΓΓ T = Σ = V ar ( X ). The elements Z i ’s in Z are IID random variables with a pre-specifieddistribution. It is worth noting that Model (21) is more general than Model (2.4) in Chen et al. (2010), where they additionally require that E ( Z i ) = 0 and a uniformbound for the 8th moment for all Z i ’s. 14e consider two distributions for Z i : 1) N (0 , Chen et al. (2010)also considered these two distributions but forcing the mean of Gamma(4,0.5) tobe zero in order to meet their requirement of data generation model for theirsphericity test.To evaluate the empirical type I error, we set Γ = I to simulate data under thenull with Σ = I . To evaluate the power, following Chen et al. (2010), we considertwo forms of alternatives: 1) Σ = σ I + σ A , where A = diag( I ⌊ λp ⌋ , p −⌊ λp ⌋ ), σ = σ = 1 and ⌊·⌋ is the floor function. In this case Γ = diag (cid:0) √ I ⌊ λp ⌋ , I p −⌊ λp ⌋ ) (cid:1) and Z is a p × N (0 ,
1) or Gamma(4,0.5)random variables. 2) Σ = (1 − ρ ) σ I + ρσ J with σ = 1 and σ = 2. In thiscase Γ = diag (cid:0) √ − ρ I , √ (cid:1) and Z is a ( p + 1) × N (0 ,
1) or Gamma(4,0.5) random variables. We choose the challengingcases in
Chen et al. (2010) by setting λ = 0 .
125 in the first alternative and ρ = 0 . Chen et al. (2010) had lower power intheir simulation study.We apply the sphericity test (12) to a collection of simulated datasets with thevarying samples sizes and dimensions as considered in
Chen et al. (2010), andcompute p-values with 100 permutations using Algorithm 1. We replicate eachsimulation setting 2000 times and reject the null hypothesis at the significancelevel of 0.05. Table 1 shows that the proposed sphericity test produces p-valuescompatible with the nominal level of 0.05 under the null. Table 2 presents theempirical power of our proposed sphericity test under the first alternative, wherefor a fixed dimension, the empirical power increases quickly with the sample sizefor both distributions. For a fixed sample size, except for the case with the samplesize of twenty, the empirical power increases with the dimension. Table 3 showsthat our proposed sphericity test demonstrates superb empirical power, almost100%, under the second alternative. 15able 1: Empirical type I error (%) of the sphericity test at the 5% significance levelSample Size ( n ) 20 40 60 80 20 40 60 80Dimension ( p ) N (0 ,
1) Gamma(4, 0.5)38 4.9 5.2 5.1 5.2 6.2 5.3 4.6 4.755 4.8 5.6 5.0 4.5 4.7 5.6 4.2 4.789 5.2 5.1 4.5 5.0 5.0 5.2 4.5 4.9159 4.8 5.2 5.3 4.2 6.0 5.3 4.5 5.1181 5.2 5.6 5.0 4.4 4.9 4.4 4.7 5.9331 4.4 4.6 5.0 4.1 4.9 5.5 5.1 4.7343 5.3 3.4 5.0 5.4 5.8 4.2 5.8 4.2642 4.8 4.9 5.4 4.8 5.5 4.4 5.6 4.9 able 2: Empirical power (%) of the sphericity test of H : Σ = σ I vs H : Σ = σ I + σ A at the 5% significance level with A = diag( I [ λp ] , p − [ λp ] ), σ = σ = 1, and λ = 0 . n ) 20 40 60 80 20 40 60 80Dimension ( p ) N (0 ,
1) Gamma(4, 0.5)38 32 66 91 98 17 48 73 9055 32 73 94 99 16 52 83 9589 36 83 98 100 17 64 92 98159 37 85 99 100 21 67 95 100181 37 86 99 100 20 68 96 100331 38 90 100 100 20 74 97 100343 38 88 99 100 21 73 97 100642 38 89 100 100 22 73 98 100 able 3: Empirical power (%) of the sphericity test H : Σ = σ I vs H : Σ = (1 − ρ ) σ I + ρσ J at the 5% significance level with ρ = 0 . σ = 1 and σ = 2Sample Size ( n ) 20 40 60 80 20 40 60 80Dimension ( p ) N (0 ,
1) Gamma(4, 0.5)38 97 100 100 100 93 100 100 10055 99 100 100 100 97 100 100 10089 100 100 100 100 99 100 100 100159 100 100 100 100 100 100 100 100181 100 100 100 100 100 100 100 100331 100 100 100 100 100 100 100 100343 100 100 100 100 100 100 100 100642 100 100 100 100 100 100 100 100 .1.2 Identity test In this section we evaluate the performance of the proposed identity test (12)for covariance or correlation matrices. We design a block-diagonal model for a p × X = ( X T , X T , X T , X T ) T , where X i is a q × i = 1 , , ,
4, such that
V ar ( X ) = Σ , V ar ( X i ) = Σ i and Cov ( X i , X j ) = for i = j . The covariance matrix in this block-diagonal model takes the form of Σ = diag( Σ , · · · , Σ ), where Σ i is a q × q matrix for all i ’s and Σ has dimensionsof p × p with p = 4 q . We first simulate each X i = LU i , where U i is a q × L is a lower triangle matrix from the Cholesky decomposition ˜Σ q = LL T , where ˜Σ q has a structure of (1 − ρ ) I + ρ J . Thus Σ i = L Λ i L T , where Λ i is a diagonal matrix whose diagonal elements are the variances of the elementsof U i . Then these four X i ’s are stacked to obtain the random vector X .We consider two configurations for X . In the first configuration, the four U i ’sare IID and the elements in all U i ’s have one of the following four distributions:1) N (0 , t ; 4) Gumbel(10, 2) to generate therandom vector X . In the second configuration, the four U i ’s are independently, butnot identically, distributed and each U i has a different distribution. The secondconfiguration is referred as a hybrid configuration. The reason to choose these fourdistributions is to investigate the impact of different types of distributions, such asasymmetric distributions, heavy-tail distributions, and extreme value distributions,on the performance of the proposed identity test (12) and its associated Algorithm2. We also consider two scenarios for the sample size: 1) a low-dimensional casewith n > p ; 2) a high-dimensional case with n ≪ p . Using the block-diagonalmodel, we generate data under the null with ρ = 0 to evaluate the empirical typeI error and under the alternative with ρ = 0 .
15 to evaluate the power. UsingAlgorithm 2, we compute p-values based on 100 permutations and reject the nullhypothesis at the significance level of 0.05. We replicate each simulation setting2000 times. Table 4 summarizes the results of the empirical type I error and powerof our proposed identity test. It is obvious that the p-values are compatible withthe nominal level of 0.05 under the null. The empirical power is close to 100%when the sample size is bigger than the dimension of the data. When the samplesize is much smaller than the dimension, the empirical power increases with theincrease of the dimension for this block-diagonal model, though there are somevariations across different distributions.19able 4: Empirical type I error (%) and power (%) of the identity test using block-diagonal modelfor H : Σ = I and H : Σ = I ⊗ Σ q , where Σ q = (1 − ρ ) I + ρ J with ρ = 0 . n ) n = 100 n = 4Dimension ( p ) 24 32 64 76 92 100 200 300 500 700 1000Type I error Type I error N (0 ,
1) 5.5 5.0 5.5 5.0 4.8 5.3 5.5 4.5 5.0 4.1 4.9Log-normal(0,1) 5.0 4.2 4.4 4.9 5.4 5.8 4.9 4.4 5.2 5.1 5.0Student t N (0 ,
1) 99 100 100 100 100 13 24 31 47 57 70Log-normal(0,1) 97 100 100 100 100 39 69 82 93 97 99Student t
99 100 100 100 100 18 32 43 58 68 80Gumbel(10,2) 99 100 100 100 100 14 28 40 59 68 79Hybrid 99 100 100 100 100 22 43 57 74 84 90 .2 Two-sample test In this section we study the empirical type I error and power of our proposedtwo-sample test (17) for the equality of two covariance matrices. We considertwo dimensionality configurations: 1) min( n , n ) > p and 2) max( n , n ) ≪ p ,where n and n are the two sample sizes and p is the dimension. We adopt theblock-diagonal model as in Section 5.1.2 to simulate data in these two dimensionalconfigurations. Under the null hypothesis we set ρ = 0 .
15 in both samples andunder the alternative we set ρ = 0 .
15 for sample one and ρ = 0 .
30 for sampletwo. We apply Algorithm 4 with 100 permutations to compute p-values and rejectthe null hypothesis at the significance level of 0.05. We replicate each simulationsetting 2000 times.Table 5 shows that the proposed equality test maintains a good control ofthe empirical type I error at the nominal level of 0.05 under the null hypothesisregardless of the underlying distributions and data dimensions.Note that 75% of the elements in the covariance matrix are zero under theblock-diagonal model. This implies that at least 75% of the elements of Σ and Σ are the same regardless of the value of ρ , which makes it rather difficult to testthe equality of two covariance matrices. In fact, for the same data, the two-sample Box ′ s M - test , which approximates the two-sample likelihood ratio test based on amultivariate normal distribution, produces empirical power: 30%, 30%, 66%, 96%and 100% for the sample sizes of n = n =100 and the dimensions p = 24, 32, 64,76, 92, respectively. Clearly, the empirical power is not high even for the Box ′ sM - test based on a correct model. The results of our proposed two-sample testsare reported in Table 5. For the multivariate normal distribution, our proposedmethod, on average, outperforms the two-sample Box ′ s M - test . The model withlog-normal(0,1) gives the lowest power in the first dimensionality configurationwhere n = n =100 ( > p ). The model with Gumbel(10, 2) gives the highest poweramong the four distributions in both configurations. Furthermore, it is interestingto notice that the model with hybrid distributions gives the second highest powerin both dimensionality configurations. We discover that the empirical power ofour proposed two-sample test is sensitive to the signal-to-noise ratio (SNR) of theunderlying distribution, defined as the reciprocal of the coefficient of variation.The model with Gumbel(10,2) has the highest empirical power accompanied withthe highest theoretical SNR of 1.70, followed by the model with log-normal(0,1)which has the second highest power with a theoretical SNR of 0.35. The modelswith N (0 ,
1) and Student t have similar low empirical power and both have thetheoretical SNR of zero. We will further investigate the impact of SNR in Section5.3. 21able 5: Empirical type 1 error and power of two-sample test H : Σ = Σ with ρ = ρ = 0 .
15 vs. Σ = Σ with ρ = 0 .
15 and ρ = 0 . n ) n = 100, n = 100 n = 20, n = 20Dimension ( p ) 24 32 64 76 92 100 200 300 500 700 1000Type I error Type I error N (0 ,
1) 5.5 4.3 4.2 4.8 4.5 4.6 5.5 4.6 5.1 4.8 4.8Log-normal(0,1) 4.9 5.0 4.8 5.5 5.3 5.8 4.9 4.4 5.2 5.1 5.0Student t N (0 ,
1) 46 65 90 92 95 23 30 30 33 34 35Log-normal(0,1) 9.5 13 30 42 50 24 37 45 47 49 49Student t
29 43 81 85 90 18 28 27 30 33 31Gumbel(10,2) 90 100 100 100 100 100 100 100 100 100 100Hybrid 37 65 99 99 100 100 100 100 100 100 100 .3 Factors affecting the performance of the proposed two-sample test In this section, we probe into some factors that may affect the performance of ourproposed two-sample test (17) and its associated Algorithm 4. Define a randomvector X = ( X , . . . , X p ) T with V ar ( X ) = Σ for the population one and X = ( X , . . . , X p ) T with V ar ( X ) = Σ for the population two. We recall theassumption required for the proposed two-sample test as X j and X j have the same distribution for all j ′ s : j = 1 , · · · , p. (22)We first investigate the impact of violating assumption (22) on the empiricaltype I error. Secondly, we use distributions with different SNR’s to examine thesensitivity of the proposed two-sample test to the change of SNR. Finally, westudy the power of the proposed two-sample test under a sparse alternative as inCai, Liu, and Xia (2013), where the two covariance matrices only differ in a veryfew and fixed number of off-diagonal elements. In this section we focus on thescenarios where the sample size n is smaller than the dimension p . To investigate the impact of violating assumption (22) on our proposed two-sampletest, we adapt the moving average models as in Li and Chen (2012). To generate X and X under the null hypothesis, we use the model X ij = Z ij + 2 Z i ( j +1) , (23)where j = 1 , · · · , p and i = 1 , { Z k } and { Z k } , k = 1 , · · · , p + 1, consist of IID random variables.Under the alternative, we generate the sample one using Model (23) and sampletwo using the following model X j = Z j + 2 Z j +1) + Z j +2) , (24)where the sequence { Z j } consists of ( p + 2) IID random variables.To examine the impact of violating assumption (22) under the null hypothesis,we set { Z k } to be IID Gamma(4, 0.5) random variables and { Z k } to be IIDGamma(0.5, √
2) random variables in Model (23). We choose the smallest samplesize of twenty as used in Li and Chen (2012). We replicate the simulation 2000times. In each replicate we compute p-values using 100 permutations in Algorithm4 and reject the null hypothesis at the significance level of 0.05. Table 6 showsthat the violation of assumption (22) can result in incorrect empirical type I errorsin our proposed two-sample test. On the other hand, when the two samples aregenerated using the same distribution, the proposed two-sample test producescorrect empirical type I errors compatible with the nominal level of 0.05.23 .3.2 Impact of SNR
To investigate the impact of SNR on the power of our proposed two-sample test,we generate X using Model (23) in the sample one and X using Model (24) insample two. The sequences { Z k } and { Z k } have the same distribution of eitherGamma(4, 0.5) or Gamma(0.5, √ √ √ N (5 ,
1) and N (5 , ), respectively, and the former has a SNRthat is five times bigger than the latter. We use n = n = 20 for the samplesizes and p = 50 for the dimension. Simulation results in Table 6 show thatthe empirical power of the model with N (5 ,
1) is about 100%, whereas the powerreduces significantly to 27% using the sample generated by the model with N (5 , ). To investigate the impact of sparse signals, we adopt a model from the supplemen-tal simulation study of Cai, Liu, and Xia (2013). Under the null hypothesis, weset Σ = Σ = Σ null , where Σ null = ( Σ ∗ + δ I ) / (1 + δ ) + δ I . (25)Under the alternative, we set Σ = Σ null and Σ = Σ alt , where Σ alt = ( Σ ∗ + δ I ) / (1 + δ ) + ∆ + δ I . (26)In models (25) and (26), Σ ∗ = ( σ ij ) is a symmetric matrix, in which σ ii = 1and σ ij = 0 . ∗ Bernoulli(1 , .
05) for i < j . δ = | λ min ( Σ ∗ ) | + 0.05 and δ = | min { λ min (( Σ ∗ + δ I ) / (1 + δ )) , λ min (( Σ ∗ + δ I ) / (1 + δ ) + ∆ ) }| + 0 .
05, where λ min ( · ) returns the smallest eigenvalue of the matrix which it applies to. ∆ isa square matrix containing thirty-two entries of 0.9 and all the other entries arezero. The locations of sixteen nonzero entries are randomly selected in the lowertriangle portion and the other sixteen nonzero entries are located in the uppertriangle portion accordingly to make ∆ symmetric. Regardless of the dimensionof the covariance matrix, the number of nonzero entries in ∆ remains thirty-twoto keep the signals sparse. Note that the matrix ∆ changes from one replicate toanother in this simulation study. 24hrough the Cholesky decomposition we obtain Σ null = Γ Γ T and Σ alt = Γ Γ T .Under the null we generate X = Γ U and X = Γ U to evaluate the empiricaltype I error, where U is a random vector whose elements are IID random variables.Under the alternative we generate X = Γ U and X = Γ U to evaluate the powerof our proposed two-sample test.Following Cai, Liu, and Xia (2013), we set the sample size to thirty for eachsample. We use Algorithm 4 to compute p-values with 100 permutations. Wereplicate each simulation setting 2000 times. We also study several distributionsfor the elements in U . Due to the sampling variations in small samples, thesample SNR’s may differ from their corresponding theoretic SNR’s. We thereforeconsider the sample signal-to-noise (SSNR) computed using the sample mean andthe sample standard deviation. Table 7 presents the influence of sparse signals andSSNR’s on the performance of our proposed two-sample test and its associatedAlgorithm 4. The results show that, under the null hypothesis, our approachproduces p-values compatible with the nominal level of 0.05, regardless of thedimension of data and the level of SSNR’s. In terms of the empirical power, ourmethod is not sensitive to different distributions but to the level of SSNR’s: thehigher SSNR is, the higher the empirical power is. Moreover, Table 7 shows thatdistributions with similar SSNR’s tend to have similar power.25able 6: Empirical type I error (%) and power (%)of the proposed two-sample testDimension p
50 100 200 300 600 800Sample 1 Sample 2 Type I errorGamma(4, 0.5) Gamma(0.5, √
2) 100 100 100 100 100 100Gamma(0.5, √
2) Gamma(0.5, √
2) 5.8 5.7 4.3 5.6 5.0 5.0Gamma(4, 0.5) Gamma(4, 0.5) 5.3 5.7 6.0 5.2 5.3 5.5Sample 1 Sample 2 PowerGamma(4, 0.5) Gamma(0.5, √
2) 100 100 100 100 100 100Gamma(0.5, √
2) Gamma(0.5, √
2) 13 20 37 51 87 97Gamma(4, 0.5) Gamma(4, 0.5) 99 100 100 100 100 100 able 7: Impact of sparse signal and sample signal-to-noise ratio (SSNR) on the proposed two-sample testDimension ( p ) 50 100 200 400 800 50 100 200 400 800SSNR Type I error (%) SSNR power (%) N (0 ,
1) 0.0 (0.0) * N (2 ,
1) 2.4 (0.1) 5.5 4.7 4.7 5.0 5.3 2.5 (0.1) 48 27 17 13 9 N (4 ,
1) 4.9 (0.1) 5.9 4.7 4.7 5.0 5.3 5.0 (0.1) 99 96 89 80 76Gamma(5,1) 2.7 (0.1) 4.6 5.3 5.3 5.0 5.3 2.8 (0.1) 55 34 21 15 11Gamma(10,1) 3.9 (0.1) 4.8 5.3 5.4 5.6 5.2 4.0 (0.1) 89 74 57 42 36Poisson(5) 2.7 (0.1) 5.1 5.3 4.9 5.0 5.3 2.8 (0.1) 53 35 22 14 10Poisson(10) 3.8 (0.1) 4.7 5.1 5.2 5.4 4.3 3.9 (0.1) 88 76 57 42 34Log-normal(0, 0.4) 3.0 (0.1) 4.9 5.0 4.3 5.5 4.8 3.0 (0.1) 58 38 25 17 13Log-normal(0, 0.3) 4.0 (0.1) 5.1 5.4 4.1 5.8 4.9 4.1 (0.1) 91 77 61 47 39 * Standard deviation of SSNR Discussion
In this study we introduce a generalized cosine measure between two symmetricmatrices and, based on this geometry, we propose new test statistics for one-sampleand two-sample tests of covariance and correlation matrices. These test statisticsand their companioned algorithms for computing p-values can be applied to a va-riety of hypothesis tests of covariance and correlation matrices. In the one-samplesetting, we implement the test statistics and algorithms for the hypothesis tests ofsphericity, identity and compound symmetry. We also propose new test statisticsand algorithms to test the equality of two or multiple covariance or correlationmatrices. We demonstrate the effectiveness of these tests and algorithms throughseveral real datasets and extensive simulation studies, where data are generatedusing a variety of models and distributions adopted in the previous literature.In order for the proposed algorithms to work properly, the number of permuta-tions is important and usually can be determined by the pre-specified significancelevel. For instance, we recommend at least one hundred permutations for the sig-nificance level of 0.05, for this number can provide the required precision to twodecimal places. Moreover, the extensive simulations show that one hundred per-mutations appear to be sufficient to control the empirical type I error and providegood power for a variety of models and distributions at the significance level of0.05. If a greater significance level is desirable, for example 0.001 in a multipletesting problem, then at least one thousand permutations is recommended. Fur-thermore, the dimension and the sample size of data also impact on the numberof recommended permutations. For algorithms associated with the proposed one-sample tests, the dimension plays a more prominent role than the sample size, sincethe number of permutations is often determined by the number of columns in thedata matrix, i.e., the dimension of data. We show in Table 4 that the proposedalgorithm can provide a correct empirical type I error for data with sample size assmall as four in identity test, provided that the dimension of data is sufficientlylarge. In the K -sample test the sample size may put a restriction on the numberof possible permutations, since the algorithms for the K -sample tests randomlyshuffle rows of the stacked data matrix. For example, in the two-sample test,the minimum recommended sample size for the combined sample is ten with thesmaller sample containing at least three subjects, for (cid:0) (cid:1) is 120 and exceeds thenumber of permutations recommended for the significance level of 0.05. Generallyfor a K -sample test, we recommend that (cid:0) nn (1) (cid:1) ≥ k , where n (1) is the smallestsample size of the K samples and k is the required number of permutations for apre-specified significance level.Our proposed test statistics are general and flexible so that they can be easilyextended to handle other complex situations. As an example, we outline one28ossible extension as follows. It is often of interest to test the independence ofseveral normal random vectors as in Fujikoshi, Ulyanov, and Shimizu (2011) ormore generally uncorrelation of multiple random vectors. Suppose a random vector X can be partitioned into k subvectors of X , . . . , X k with the length of p , . . . , p k ,respectively. Below we let k = 2 for an illustrative purpose. Let the covariancematrix of X be Σ = (cid:20) Σ X X Σ X X Σ X X Σ X X (cid:21) , (27)where Σ X i X j is the covariance matrix of X i and X j . The null hypothesis ofuncorrelation between X and X is H : Σ = (cid:20) Σ X X Σ X X (cid:21) . (28)To test this null hypothesis, we specify Σ in the one-sample test statistic (9) as Σ = diag( J , J ) , where J i is a p i × p i matrix of ones for i = 1 ,
2, respectively. Using the mappingvech( · ), the Euclidean inner product and norm, it can be shown that test statistic(9) in this case can be expressed as1 − vech( S ) T vech( Σ ) q . P i =1 p i ( p i + 1) k vech( S ) k , (29)where S is the sample covariance matrix. Below we outline an associated algo-rithm to compute the p-value of (29) under the null hypothesis of uncorrelation.Analogous to the identity test, we consider permutations of the elements in eachcolumn of the data matrix. In this case the data matrix has two “group-column”’s:the first “group-column” consists of the first p columns in the data matrix andthe second “group-column” includes the remaining p columns. We then permutethe elements in each of the “group-column”’s to compute the p-value under thenull, for this permutation procedure retains the sample covariance matrices foreach of the two sub-vectors, while disrupting the covariance between the two. Infuture study we plan to extend the proposed tests and permutation algorithmsto handle other covariance structures, for example, those frequently adopted inrepeated measure data analysis such as AR(1) and Toeplitz. We would like to thank the computing facility SHARCNET (Ontario, Canada) fora partial computation support. 29 ppendix
Appendix I Derivation of sphericity test statistic (12) andidentity test statistic (15)
Let S be a p × p covariance matrix and I a p × p identity matrix. For thesphericity test, Σ in the one-sample test statistic (9) is σ I . We adopt themapping vech( · ), the Euclidean inner product and norm and apply Equation (1)to compute cos ( S , σ I ) as follows: cos ( S , σ I ) = vech( S ) T vech( σ I ) k vech( S ) kk vech( σ I ) k = vech( S ) T vech( I ) k vech( S ) kk vech( I ) k = P pi =1 s ii √ p k vech( S ) k = tr( S ) √ p k vech( S ) k , (30)where s ii , i = 1 , . . . , p , are the diagonal elements of S . Then the sphericity teststatistic (12) follows from 1 − cos ( S , σ I ).For the identity test of a correlation matrix S , the Σ in the one-sample teststatistic (9) is I . We adopt the mapping vech( · ), the Euclidean inner productand norm and apply Equation (1) to compute cos ( S , I ), which leads to equation(30). Since S is a sample correlation matrix and tr( S )= p , the cos ( S , I ) can besimplified into cos ( S , I ) = √ p k vech( S ) k , (31)then the identity test statistic (15) follows from 1 − cos ( S , I ). Appendix II Derivation of the compound symmetry teststatistic (16)
Let S be a p × p sample covariance or correlation matrix and J a p × p matrix ofone’s. For the compound symmetry test, Σ in the one-sample test statistic (9) is σ [(1 − ρ ) I + ρ J ] for the covariance matrix and [(1 − ρ ) I + ρ J ] for the correlationmatrix, where ρ > σ and ρ . 30e adopt the modified half-vectorization vech ∗ ( · ), the Euclidean inner productand norm to compute cos ( S , σ [(1 − ρ ) I + ρ J ]) according to Equation (1) as follows: cos ( S , σ [(1 − ρ ) I + ρ J ]) = vech ∗ ( S ) T vech ∗ ( σ [(1 − ρ ) I + ρ J ] k vech ∗ ( S ) kk vech ∗ ( σ [(1 − ρ ) I + ρ J ]) k = vech ∗ ( S ) T vech ∗ ( J ) k vech ∗ ( S ) kk vech ∗ ( J ) k = vech ∗ ( S ) T vech ∗ ( J ) p . p ( p − k vech ∗ ( S ) k . (32)which does not involve the unknown parameters of σ and ρ . The test statistic(16) for the covariance matrix S follows from 1 − cos ( S , σ [(1 − ρ ) I + ρ J ]).Suppose S is a sample correlation matrix, we adopt the modified half-vectorizationvech ∗ ( · ), the Euclidean inner product and norm to compute cos ( S , [(1 − ρ ) I + ρ J ])according to Equation (1). Following the above calculation for the sample covari-ance matrix, we can show that cos ( S , [(1 − ρ ) I + ρ J )]) yields the same equation(32) and the test statistic (16) follows from 1 − cos ( S , [(1 − ρ ) I + ρ J ]). References
Annaert, J. , A. G. Claes , and
M. J. De Ceuster , 2006 Intertemporalstability of the European credit spread co-movement structure. The EuropeanJournal of Finance (1) : Bartlett, M. , 1951 The effect of standardization on a chi-squared approxima-tion in factor analysis. Biometrika (2) : Cai, T. , W. Liu , and
Y. Xia , 2013 Two-sample covariance matrix testingand support recovery in high-dimensional and sparse settings. Journal of theAmerican Statistical Association (501) : Cai, T. T. , 2017 Global Testing and Large-Scale Multiple Testing for High-Dimensional Covariance Structures. Annual Review of Statistics and Its Appli-cation Chen, S. X. , L.-X. Zhang , and
P.-S. Zhong , 2010 Tests for high-dimensionalcovariance matrices. Journal of the American Statistical Association (490) : Djauhari, M. A. and
S. L. Gan , 2014 Dynamics of correlation structure instock market. Entropy (1) : Fujikoshi, Y. , V. V. Ulyanov , and
R. Shimizu , 2011
Multivariate statistics:High-dimensional and large-sample approximations . John Wiley & Sons.31 reen, J. A. , 1992 Testing whether correlation matrices are different from eachother. Developmental Psychology (2) : Jennrich, R. I. , 1970 An asymptotic χ (330) : Johnson, R. A. and
D. W. Wichern , 2007
Applied multivariate statisticalanalysis (6 ed.). Prentice Hall Englewood Cliffs, NJ.
Ledoit, O. and
M. Wolf , 2002 Some hypothesis tests for the covariance matrixwhen the dimension is large compared to the sample size. Annals of Statistics : Li, J. and
S. X. Chen , 2012 Two sample tests for high-dimensional covariancematrices. The Annals of Statistics (2) : Muirhead, R. J. , 2005
Aspects of Multivariate Statistical Theory . John Wiley& Sons.
R Core Team , 2018 R: A Language and Environment for Statistical Computing.
Rencher, A. C. and
W. F. Christensen , 2012
Methods of multivariate anal-ysis , Volume 709. John Wiley & Sons.
Revelle, W. , 2018 psych: Procedures for Psychological, Psychometric, andPersonality Research. R package version 1.8.4.
Singh, D. , P. G. Febbo , K. Ross , D. G. Jackson , J. Manola , C. Ladd , P. Tamayo , A. A. Renshaw , A. V. D’Amico , J. P. Richie , and others ,2002 Gene expression correlates of clinical prostate cancer behavior. Cancercell (2) : Yao, J. , S. Zheng , and
Z. Bai , 2015