Exact Multivariate Two-Sample Density-Based Empirical Likelihood Ratio Tests Applicable to Retrospective and Group Sequential Studies
11 Exact Multivariate Two-Sample Density-Based Empirical Likelihood Ratio Tests Applicable to Retrospective and Group Sequential Studies
Ablert Vexler a , Gregory Gurevich b and Li Zou c a Department of Biostatistics, The State University of New York at Buffalo, Buffalo, NY 14214, U.S.A, [email protected] b Department of Industrial Engineering and Management, SCE- Shamoon College of Engineering, Ashdod, Israel c Department of Statistics and Biostatistics, California State University, East Bay, Hayward, CA 94542, U.S.A.
ABSTRACT Nonparametric tests for equality of multivariate distributions are frequently desired in research. It is commonly required that test-procedures based on relatively small samples of vectors accurately control the corresponding Type I Error (TIE) rates. Often, in the multivariate testing, extensions of null-distribution-free univariate methods, e.g., Kolmogorov-Smirnov and Cramér-von Mises type schemes, are not exact, since their null distributions depend on underlying data distributions. The present paper extends the density-based empirical likelihood technique in order to nonparametrically approximate the most powerful test for the multivariate two-sample (MTS) problem, yielding an exact finite-sample test statistic. We rigorously establish and apply one-to-one-mapping between the equality of vectors’ distributions and the equality of distributions of relevant univariate linear projections. In this framework, we prove an algorithm that simplifies the use of projection pursuit, employing only a few of the infinitely many linear combinations of observed vectors’ components. The displayed distribution-free strategy is employed in retrospective and group sequential manners. The asymptotic consistency of the proposed technique is shown. Monte Carlo studies demonstrate that the proposed procedures exhibit extremely high and stable power characteristics across a variety of settings. Supplementary materials for this article are available online.
Keywords:
Density-based empirical likelihood; Exact test; Multivariate two-sample test; Nonparametric test; Projection pursuit Introduction
In many practical studies, data consist of multiple outcomes in the forms of random vectors realizations. We consider an illustrative example based on data from a study that has evaluated the association between biomarkers and myocardial infarction (MI). The study was focused on the residents of Erie and Niagara counties, 35-79 years of age (Schisterman et al., 2001). The New York State department of Motor Vehicles drivers’ license rolls was used as the sampling frame for adults between the age of 35 and 65 years, while the elderly sample (age 65-79) was randomly chosen from the Health Care Financing Administration database. This research has examined the diagnostic ability of the biomarkers “thiobarbituric acid-reactive substances” (TBARS), “vitamin E”, “glucose” and “high-density lipoprotein (HDL)-cholesterol”, using samples that were collected on cases, who recently survived on MI disease, and controls, who had no previous MI disease. This practical issue requires formal developing of an MTS, “case-control”, test in a retrospective manner. The difficulty in analyzing the TBARS, “vitamin E” and “glucose” biomarkers is that values of these biomarkers have been shown to be dependent and non-normal distributed. A common approach in statistics is to generalize univariate testing mechanisms in multivariate setting. For example, the Hotelling T test successfully extends Student’s t -test to an MTS location decision-making strategy, when data follow a multivariate normal distribution. A departure of the underlying data distribution from being normal can imply a critical issue in controlling the TIE rate of the Hotelling T test. The classical Hotelling T test cannot be applied when the dimension of observed vectors exceeds the sample size (Biswas and Ghosh, 2014). Perhaps, the difficulty to develop an exact test in the multivariate setting is due to the fact that, in general, the empirical estimator of the multivariate distribution function is not distribution free as in the univariate case (Simpson, 1951). Note also that Kolmogorov-Smirnov and Cramér-von Mises type two sample procedures may significantly suffer from a lack of power under various alternatives or when the size of one sample is relatively greater than the other sample (Gordon and Klebanov, 2010). We can find within the modern statistical literature a line of research around constructions of MTS tests. For example, Biswas and Ghosh (2014) developed an MTS test based on inter-point distances that can be conveniently used in the high-dimensional low sample size setting (see also, e.g., Baringhaus and Franz, 2004, in this context). The authors showed the testing method that is asymptotically exact and consistent under general alternatives, when the sample size tends to infinity. Jurečková and Kalina (2012) derived nonparametric MTS rank tests, considering location/scale alternatives. The special attention to distances associated with the samples’ means and dispersion matrices afforded the authors to focus on Wilcoxon test type mechanisms (see also Marozzi,2016, in this context). Jurečková and Kalina (2012) concentrated on the MTS Wilcoxon, Psi and Savage exact (distribution-free) tests, taking into account some alternatives of Lehman type. The restriction that the dimension of the data is smaller than the sample size is employed in several rank tests’ constructions presented in Jurečková and Kalina (2012: e.g., p.235). Zhou et al. (2017) developed smooth MTS tests for special alternatives. In this development, the idea of projection pursues based on linear combinations of observed vectors’ components was applied, reducing the MTS testing statement to the univariate problem. The authors assumed that observed samples contain realizations of independent p -dimensional vectors ,..., Tp X X X and ,..., Tp Y Y Y , when every linear combination of X ’s components and every linear combination of Y ’s components distributed with respect to a broad family of parametric exponential type forms that can be formulated via d parameters (see Sections 1.1 and 2.1 of Zhou et al., 2017, for details). Then allowing the number of parameters, d , to tend to infinity (along with the sample size), Zhou et al. (2017) proposed the test strategy for a large array of alternatives. The consideration of large sample sizes is essential in Zhou et al. (2017)’s developments. Assumption 4.3. of Zhou et al. (2017) restricts the sample sizes to be exceedingly superior to the vectors dimension, p . Unfortunately, the assumption that distributions of every linear combinations of vector components can be approximated by parametric distribution functions has not a direct interpretation in terms of restrictions related to the multivariate distribution of the vector, in general. We can also anticipate a difficulty in practical implementations of the proposed smooth tests when every linear combinations of vectors components should be employed in computations of the test statistics presented by Zhou et al. (2017) (see Shao and Zhou, 2010, in this context). In this case, evaluations based on all subsets ( ) r p of the vectors components , , p X X and , , p Y Y together with the use of an infinity number of linear combinations of them may not insure the correct decision regarding the equality of vectors X and Y distributions (Hamedani, 1984). Zhou et al. (2017) proposed to substitute the null hypothesis H : “ X and Y are identically distributed”, say d X Y , by the hypothesis H : “ T Td u X u Y , for all vectors u in the unit p-1 dimensional sphere”, applying the union-intersection principle. For a review, discussion and limitations related to the union-intersection principle, we can refer the reader to Olkin and Tomsky (1981). Perhaps, due to a possible strong dependence between sub-hypotheses contained in H and their infinity number, the form of the null hypothesis applied in Zhou et al. (2017) needs a substantial analysis. In the present paper, we provide rigorous probabilistic arguments to characterize the equality of vectors’ distributions via the equality of distributions of relevant univariate linear projections. This implies constructing of precise multivariate extensions to powerful techniques developed in the univariate two-sample settings. Note that, developments and correct applications of univariate characterizations of vector distributions through the use of linear combinations of the vector components are not straightforward tasks even when observations are normally distributed (e.g., Hamedani, 1984; Shao and Zhou, 2010; Vexler, 2020). Zhou et al. (2017) provided the multiplier bootstrap method to compute the critical value of the MTS smooth tests, which are not exact. According to Zhou et al. (2017), the limiting null distributions of the MTS smooth test statistics may not exist. We can remark that the smooth testing technique may suffer from a lack of power when underlying data are from nonexponential distributions (Vexler et al., 2014a). In general, in the nonparametric MTS framework, there are no most powerful decision-making mechanisms. Although several techniques for the MTS problem have been proposed, there is still a demand for MTS tests developments based on strong statistical paradigms. The main aim of this paper is to develop an exact and consistent approach for MTS testing of general alternatives, rigorously evaluating an applicability of the univariate projection pursuit in the MTS decision-making statement. The proposed testing strategy is derived without the assumption that the sample size exceeds the vectors dimension, p. Note that, according to Friedman (1987), since linear projections are the simplest and most interpretable dimension-reducing methods, they are among most commonly used in theoretical and practical multivariate exploratory data analysis procedures. In practice, technical reasons restrict the number of the linear combinations related to projection pursuit to be considered. Then, it is important to show that a procedure supposedly based on the infinitely many linear combinations of multivariate variates can be conducted correctly by using a few relevant linear combinations of components. In the present paper we prove an MTS testing algorithm based on realizations of the linear combinations T u X and T u Y with selected finite values of the vector 's u components. In order to construct the proposed MTS test, we employ the density-based empirical likelihood ratio technique. The likelihood ratio methodology provides a basis for many important procedures and methods in statistical inference. By virtue of the Neyman-Pearson lemma, when functional forms of data distributions are completely specified the parametric likelihood approach is unarguably the most powerful tool. The parametric likelihood methods cannot be applied properly if assumptions on the forms of distributions of data do not hold. In this paper, we use a distribution-free strategy to approximate an optimal parametric likelihood ratio test-statistic via an empirical likelihood methodology. Empirical likelihood (EL) concepts were introduced as nonparametric alternatives to parametric likelihood methods. The EL principle has been dealt with extensively across a variety of settings (e.g., Owen, 2001; Vexler et al. , 2014b, 2016). Commonly, the EL function has the form n ii EL p , where the probability weights, , 1,..., i p i n satisfy the assumptions n ii p i p
1, , i n and the values of , 1,..., i p i n , are derived by maximizing the EL function under empirical constraints. For example, when we draw a univariate sample of iid data points 𝑍 (cid:2869) , … , 𝑍 (cid:3041) under the null hypothesis, H , that E 0 Z , the corresponding constraint is n i ii p Z , an empirical version of the H -statement. The density-based EL (DBEL) approach can represent nonparametric test statistics that approximate parametric Neyman-Pearson statistics (e.g., Nanda and Chowdhury, 2020; Vexler et al. , 2014a; Gurevich and Vexler, 2011). The DBEL method proposes to consider the likelihood in the form of ( )1 1 ( ) , n nf i i i ii i L f Z f f f Z , where )( f is a density function of observations ,..., n Z Z , and (1) ( ) ... n Z Z are the corresponding order statistics. The DBEL approach then approximates values of j f via maximization of f L given a constraint related to the empirical version of the density property of the form ( ) 1 f u du . We introduce an MTS DBEL ratio decision-making mechanism with high and stable power properties for detecting general cases of nonequalities of two vectors’ distributions. The proposed method is null-distribution-free, robust to model structures and highly efficient. This approach is applied in the retrospective setting with fixed sample sizes, as well as in the group sequential manner (e.g., Jennison and Turnbull, 1999; Zou, et al., 2019). For the last 30 years, there has been growing interest in the use of group sequential designs in clinical studies. This is because such designs allow early stopping for either efficacy or futility, reducing the cost associated with data collections. In this context, for an extensive review and examples related to the group sequential methodology and its applications, we refer the reader to Jennison and Turnbull (1999). According to Jennison and Turnbull (1999), sequential comparisons of multiple outcomes’ distributions in clinical trials belong to a first cohort of biostatistical targets. Thus, the MTS testing method developed in this paper is a valuable addition to statistical inference. Commonly, sequential multivariate statistical procedures require to assume parametric forms of the underlying data distributions. Performances of parametric sequential tests strongly depend on the correctness of the distribution assumptions. Retrospective, non-sequential, studies are generally based on already collected datasets. In contrast to the analysis of data obtained retrospectively, we have the following issues related to sequential analysis. It can be difficult to specify the parametric distribution forms of the underlying data distributions before data points are observed. In a case we have strong reasons to assume the parametric forms of the data distributions, it could be extremely difficult, e.g., to test the corresponding parametric assumptions after the execution of sequential procedures. Sequential tests are based on random numbers of observations, and then data obtained after sequential analyses cannot be evaluated for goodness-of-fit using the conventional retrospective testing methods. In this paper we focus on a nonparametric MTS sequential procedure.
The paper is organized as follows. In Section 2, we characterize the multivariate statement of d X Y by studying the distributions of the univariate projections of X and Y . The DBEL method is introduced and extended to test for the MTS hypothesis. To simplify and correctly use the proposed decision-making mechanism, we prove an algorithm for computing extreme values of the MTS DBEL ratio test statistic that theoretically employs infinitely many linear combinations of underlying components of multivariate observations. This result can be associated with methods for conducting multivariate procedures developed via a Kolmogorov Smirnov type manner or ranks based concepts, when the univariate projection pursuit is employed. In Sections 3 and 4, the proposed method is used for constructing the retrospective and group sequential MTS tests. The asymptotic consistency of the developed tests is presented. We present exact mechanisms to control the TIE rates of the MTS DBEL ratio tests. An extensive Monte Carlo comparison between the proposed testing scheme and the modern MTS procedures is shown in Section 5. We discuss the performance of the tests under various alternative designs involving, e.g., cases when observed X ’s and Y ’s components are identically distributed, whereas X and Y have different distributions. It turns out that the proposed method significantly outperforms the known tests in almost all of the scenarios we considered. In Section 6, the real-world applicability of the proposed decision-make technique is illustrated using data from a study of biomarkers associated with myocardial infarction. We conclude with remarks in Section 7. Proofs of the theoretical results and algorithms presented in this paper are outlined in the supplementary materials. Method
This section displays the main stages in the MTS DBEL test statistic development. We begin by considering the statement that the observed data consists of two samples of realizations of independent p -dimensional random vectors ,..., Tp X X X and ,..., Tp Y Y Y with unknown multivariate distribution functions X F and Y F , say ~ X F X and ~ Y F Y , respectively. We test the null hypothesis : X Y
H F F against the general alternative hypothesis : X Y
H F F . To reduce the dimension of the MTS testing problem, we consider the univariate linear combinations ( ) pT i ii X u X u u X and ( ) pT i ii Y u Y u u Y , where vector ,..., . Tp u u u Assuming that values of X and Y are measured on a continuous scale, we have the following characterization. Proposition 1.
The joint distribution functions X F and Y F are equal if and only if ( ) X u and ( ) Y u are identically distributed for all ,..., p u u R . Proof.
The proof treats the corresponding characteristic functions and is outlined in the supplementary materials. To simplify the application of Proposition 1 to the further test development, we represent Proposition 1 in the form below.
Proposition 2.
The next two statements are equivalent: (a)
X Y
F F , and (b) ( ) X u and ( ) Y u are identically distributed in each of the following scenarios regarding values of , 1,..., , , j u j p selections:
11 1 1 ,..., : for 1, ... 0; 1; , 1,.., s p s s j
A u u s u u u u R j s p , s p , and ,..., : 0, 1, 1,..., 1 . p p i p A u u u u i p
These results allow to use univariate outcomes from ( ) X u and ( ) Y u in constructing the test statistic for : X Y
H F F . We begin by outlining basic ingredients of the univariate two-sample DBEL test construction, introducing a principle of notations we use in this paper. Assume we observe two independent samples ,..., n Z Z and ,..., m V V , where ,..., n Z Z R are iid data points, ,..., m V V R are iid data points and , n m are the sample sizes. Define the order statistics ... n Z Z and ... m V V based on the observations ,..., n Z Z and ,..., m V V , respectively. In order to test for the hypothesis : Z V
H F F , where Z F and V F denote the unknown distribution functions of Z and , V respectively, Gurevich and Vexler (2011) developed the nonparametric DBEL approach for approximating the likelihood ratio
1( ) ( ) ( ) ( )1 1 1 1 n m n mZ i V i i ii i i i
L f Z f V f Z f V , where density functions Z f , V f , f correspond to Z F , V F , and an H -distribution of Z and , V respectively. In the DBEL approach, values of ( ) , 1,..., , Zi Z i f f Z i n and ( ) , 1,..., ,
Vj V j f f V j m can be estimated by maximizing the likelihoods n mZi Vii i f f , given empirical constraints to control the assumptions / 1, , D f u f u f u du D Z V . According to Gurevich and Vexler (2011), for all integers / 2 r n and / 2 s m , the corresponding empirical constrains have the forms ( )(1)
11 2 n Z nZ Zi ZiriZ i f u ff u duf u r f Z and ( )(1)
11 2 m V mV Vi VisiV i f u ff u duf u s f V , where ( ) ( ) | , | , Zir n m i r n m i r
F Z Z V F Z Z V , ( ) ( ) | , | , Vis n m i s n m i s
F V Z V F V Z V ; ( ) ( ) i r n Z Z , if i r n ; ( ) (1) i r Z Z , if i r ; ( ) ( ) i s m V V , if i s m ; ( ) (1) i s V V , if i s ; | , n mn m i ji j F t Z V n m I Z t I V t defines the H -empirical distribution function based on ,..., n Z Z and ,..., m V V ; and . I is the indicator function. Then, the method of Lagrange multipliers yields the DBEL estimator of the likelihood ratio L in the form
111 1 n mZir Vjsi j r n s m
The DBEL method incorporates an aspect of maximum likelihood methodology to state the test statistic , , ( , ) , nm Z n V m TS Z V ELR ELR min 2 , min 2 , n n m m n mZ n a r b Zir V m a s b Visi i ELR r n ELR s m j a j , min , 0.5 j b j j , , j n m ,
0, 0.25 . Thus, by virtue of Propositions 1 and 2, in order to test for : X Y
H F F based on observed vectors ,..., Ti i pi
X X X , i n and ,..., Tj j pj
Y Y Y , j m realizations of ,..., ~ Tp X
X X F X and ,..., ~ Tp Y
Y Y F Y , we can focus on the statistic ( ), ( ), ( ), ( ) nm X n Y m TS X Y ELR ELR u u u u with min 2 , min 2 , n n m m n mX n a r b X ir Y m a s b Y isi i ELR r n ELR s m u u u u where ( ) ( ) ( ) ( ) | ( ), ( ) ( ) | ( ), ( )
X ir n m i r n m i r
F X X Y F X X Y u u u u u u u , ( ) ( ) ( ) ( ) | ( ), ( ) ( ) | ( ), ( ) Y is n m i s n m i s
F Y X Y F Y X Y u u u u u u u ; ( ) , pj i iji X u X u j n ( ) , pk i iki Y u Y u k m ; ( ) ... ( ), ( ) ... ( ); n m X X Y Y u u u u | ( ), ( ) ( ) ( ) n mn m i ji j
F t X Y n m I X t I Y t u u u u ; ( ) ( ) ( ) ( ) i r n X X u u , if ; i r n ( ) (1) ( ) ( ) i r X X u u , if i r ; ( ) ( ) ( ) ( ) i s m Y Y u u , if i s m ; ( ) (1) ( ) ( ) i s Y Y u u , if i s . Note that, defining the function k g t I t tI t k kI t k , we can simplify the notations
1( ) ( )1 ( ) | ( ), ( ) ( ) ( ) mn m i n j ij
F X X Y n m g i I Y X u u u u u ,
1( ) ( )1 ( ) | ( ), ( ) ( ) ( ) nn m j i j mi
F Y X Y n m I X Y g j u u u u u . In order to make decision regarding the MTS problem, it is reasonable to employ a concept associated with the Kolmogorov-Smirnov principle for measuring distances between nonparametric hypotheses. To this end, we can use large values of ,..., max ( ), ( ) pp jj nm nmu u A TS TS X Y u u , discriminating H and its alternative hypothesis. The requirement for computing the maximum of the statistic ( ), ( ) nm TS X Y u u over all ,..., p u u R is not at least user-friendly. Note, for example, that, commonly, to implement Kolmogorov-Smirnov type statistics, algorithms for conducting maximums involved in the statistics can be performed by using finite numbers of arguments based on observations. Towards this, the following results are obtained. For clarity of explanation, we begin by exemplifying the case with p , where we are interested in conducting the statistic , max ( ), ( ) nm nmu u A A TS TS X Y u u . The sets
11 1 2 1 2 , : 1,
A u u u u R and , : 0, 1
A u u u u are defined in Proposition 2. Define the data points
11 1 2 2 ( , ) i j j i
W i j X Y Y X ,
11 1 2 2 ( , ) i r r i
U i r X X X X and
11 1 2 2 ( , ) j s s j
V j s Y Y Y Y in order to show that , max ( ), ( ) nm nmu u B A TS TS X Y u u , where the set , : 1, ( , ), ( , ), ( , ), , ,1 , , 1 , B u u u u W i j U i r V j s i r j s i r n j s m . To establish this result, we will apply the following scheme. The statistic ( ), ( ) nm TS X Y u u is based on the variables ( ) ( ) i j
I X Y u u , ( ) ( ) j i I Y X u u , ( ) ( ) i k I X X u u , ( ) ( ) j r I Y Y u u , i k n j r m . Thus, two vectors , T u u u , , T v v v satisfy ( ), ( ) ( ), ( ) nm nm TS X Y TS X Y u u v v , if ( ) ( ) ( ) ( ) i j i j I X Y I X Y u u v v , ( ) ( ) ( ) ( ) i k i k I X X I X X u u v v , ( ) ( ) ( ) ( ) j r j r I Y Y I Y Y u u v v , for all i k n and j r m . In the supplementary materials, we display details of the proof that, for A A u and B A v , ( ) ( ) ( ) ( ) i j i j I X Y I X Y u u v v , ( ) ( ) ( ) ( ) i k i k I X X I X X u u v v , ( ) ( ) ( ) ( ) j r j r I Y Y I Y Y u u v v , for all i k n and j r m . In order to consider the general case with dimension p , we recursively define the following notations. Let , 1 1 2 2 , ,..., , j j W j
J i j i j , , 2 1 2 1 2 2 , ,..., , j j j j cW j J i j i j , , 1 1 2 2 , ,..., , j j U j
J i r i r , , 2 1 2 1 2 2 , ,..., , j j j j cU j J i r i r , , 1 1 2 2 , ,..., , j j V j
J j s j s and , 2 1 2 1 2 2 , ,..., , j j j j cV j J j s j s denote integer row-vectors with the components q q i r n q q j s m , where 1,..., 2 j q and j p . For example, ,3 1 1 2 2 3 3 4 4 , , , , , , , W J i j i j i j i j , ,3 5 5 8 8 , ,..., , cW J i j i j . We can write ,1 , W J i j , ,1 , U J i r and ,1 , V J j s , for the sake of simplicity. When , , cW j W j J J , , , , cU j U j J J , , , cV j V j J J we denote, for k p , the sets of the random variables k W ki kj pj pi W J X Y Y X , k U ki kr pr pi U J X X X X , k V kj ks ps pj V J Y Y Y Y , and then, for j p , k p j ,
1, , 1 , 1 1 , 1 1 , 1 c ck W j k W j k W j p j W j p j W j
W J W J W J W J W J ,
1, , 1 , 1 1 , 1 1 , 1 c ck U j k U j k U j p j U j p j U j
U J U J U J U J U J ,
1, , 1 , 1 1 , 1 1 , 1 c ck V j k V j k V j p j V j p j V j
V J V J V J V J V J . Note, for example, that the notation ,1 k W W J means the sequence
11 1 1 1 , k k p p X Y Y X
11 2 2 1 ,..., k k p p
X Y Y X . kn km pm pn X Y Y X Proposition 3.
We have ,..., ,..., max ( ), ( ) max ( ), ( ) p pp j p jj j nm nm nmu u A u u B
TS TS X Y TS X Y u u u u , where sets ,..., p B B contain elements defined via the following algorithms: for s p ,..., : for 1, ... 0; 1; s p s s B u u s u u u for d p s , given ,..., s d u u , select , s ds d h W p d s hh s u W J u , s d s dh U p d s h h V p d s hh s h s U J u V J u and p p
B A . We exemplify details of the notations used in Proposition 3, when p , in the supplementary materials. The proof scheme for deriving the statement of Proposition 3 associates the indicators ( ) ( ) , i j I X Y u u ( ) ( ) , i j I X X u u ( ) ( ) i j I Y Y u u with ( ) ( ) , i j I X Y v v ( ) ( ) , i j I X X v v ( ) ( ) , i j I Y Y v v for all , i j , ,..., T pp j j u u A u and ,..., T pp j j v v B v . This proof algorithm can be applied to execute different multivariate procedures developed by using a Kolmogorov Smirnov type manner or ranks based concepts, when the univariate projection pursuit is employed (see the supplementary materials, for details). Remark 1.
Various Monte Carlo experiments based on realizations of , X Y with a variety of sample sizes (n,m) showed that multiple uses of the R function ‘ optim ’ executed with initial values equating to different empirical quantiles of ,..., p k u u B , k p can significantly reduce the computation time of the proposed procedure.
3. The MTS DBEL test based on pre-collected samples with nonrandom sizes
Often, practical studies consider retrospectively collected p -dimensional independent outcomes from two groups, say ,..., Tp X X X and ,..., Tp Y Y Y , following multivariate distributions X F and Y F , respectively. Let ,..., Ti i pi
X X X be the outcomes of the i- th subject. i n from the X -sample as well as let ,..., Tj j pj
Y Y Y be the outcomes of the j- th subject, j m from the Y -sample. The null hypothesis being tested is : X Y
H F F . Towards this end, we define the linear combinations ( ) , 1,.. , pTj j i iji X u X j n u u X and ( ) pTk k i iki Y u Y u u Y , obtaining the test statistic . nm TS We then propose to reject H if nm TS C , where C is an -level test threshold. It is clear that, the proposed test is exact. In Section 3.1 we discuss computing of the critical values of the proposed testing strategy. The next result points out that the MTS DBEL procedure is an asymptotic power one test. Let Pr k H and E k H denote the probability measure and expectation under , 0,1 k H k . Denote the density functions ( ; ) Pr ( ) / Pr ( ) / H H H f t d X t dt d Y t dt u u u and ( ) 1 ( ) 1 ( ; ) Pr ( ) / , ( ; ) Pr ( ) /
X H Y H f t d X t dt f t d Y t dt u u u u u u and a ssume that / n m as , n m , where a constant . Then the following proposition shows the consistency of the proposed test. Proposition 4.
Let X F and Y F be absolute continuous distribution function defined on p R . Assume there exists a vector ,..., Tp u u u such that, for X u and Y u , the expectations E log ; H f u , E log ; X f u u and E log ; Y f u u are finite. Then, for a positive threshold C , we have Pr log 1
H nm n m TS C , whereas Pr log 0
H nm n m TS C , as , n m . Proof.
The proof uses the theorem of Dvoretzky, Kiefer and Wolfowitz (Serfling, 2009, p. 59) and is outlined in the supplementary materials.
In this section, we point out that the proposed test statistic is exact, i.e., H -distribution-free. We then present the critical values for the proposed test for different sample sizes and p=2 as well as an R code to derive the test critical values in practice. Note that, under H , the test statistic nm TS depends only on the empirical distribution function | ( ), ( ) n m F t X Y u u , for all p R u , which in turn depend only on certain indicator functions. By virtue of Proposition 1, under H , we have the distribution function ( ) Pr ( ) Pr ( ) H i H j
F t X t Y t u u , for all p R u and i n j m , Then, ( ) ( ) ( ) ( ) I X Y I F X F Y I U U u u u u , where , U U are Uniform (0, 1) distributed. (For details regarding distribution-free test constructions based on empirical distribution functions see Crouse, 1966). Hence, it follows that Pr logPr log | ,..., , ,..., ~ (0, ), 1,..., , 1,..., ,
H nm T Tnm i i pi j j pj p
TS CTS C X X Y Y N I i n j m X Y where p I is a p -dimensional identity matrix. Therefore, the proposed method is exact, and then the critical values for the proposed DBEL test can be accurately approximated using Monte Carlo techniques. Note that extensive Monte Carlo simulations confirmed the robustness of the proposed test with respect to the values of
0, 0.25 at the definition of the test log nm TS C . For practical purposes, we suggest a value of that is used in our applications. It was shown in the DBEL literature that power values of DBEL type test statistics do not differ substantially for values of (0, 0.25) (e.g., Gurevich and Vexler, 2011; Tsai et al., 2013; Vexler et al., 2014a). In the case with p=2 , we tabulated the percentiles of the null distribution of the test statistic based on 20,000 samples of , ~ 0, , Ti i i
X X N I X i n and , ~ (0, ), Tj j j
Y Y N I Y j m to calculate values of nm TS at each pair ( n, m ). The generated values of the test statistic log nm TS were used to determine the critical values C of the null distribution of log nm TS at the significance level . The results of this Monte Carlo study are presented in Table 1. Table 1.
Critical Values of the Proposed Test Statistic, log nm TS Significance level Significance level ( , ) n m ( , ) n m (10, 10) 11.063 11.983 13.832 (15, 40) 15.592 16.538 18.397 (10, 15) 11.878 12.656 14.470 (15, 50) 16.400 17.304 19.311 (10, 30) 14.276 15.111 16.874 (20, 20) 13.877 14.714 16.560 (10, 40) 15.128 16.026 17.822 (20, 30) 15.226 16.108 18.029 (10, 50) 15.918 16.863 18.904 (20, 40) 16.112 17.021 18.978 (15, 15) 12.510 13.374 15.168 (20, 50) 16.896 17.853 19.855 (15, 20) 13.249 14.090 15.959 (50,50) 19.538 20.317 22.340 (15, 30) 14.714 15.612 17.475 (65, 70) 22.677 23.428 24.850 An R function (R Development Core Team, 2012) for the Monte Carlo computations related to the critical values C of the null distribution of log nm TS is shown in the supplementary materials. This function can be easily modified to prepare the proposed test in a real study. We can remark that, since the statistic log nm TS is H -distribution-free, Monte Carlo or Bootstrap type procedures can be employed to estimate different probabilistic characteristics of log nm TS , under H , e.g., var log nm TS . The group sequential MTS DBEL test
Investigators can design a clinical study in group sequential manners that offer the possibility to stop an experiment early with a statistically significant test result. The sequential trial can require need less observations than the trial with fixed sample sizes in which a test decision can be made only at the end of the trial. Following Jennison and Turnbull (2000), we assume that K groups of subjects are available from two sets, say set A and set B . Let ,..., ~ Ti i pi X
X X F X and ,..., ~ Ti i pi Y
Y Y F Y , i be independent and represent the measures of subjects allocated to sets A and B , respectively. For simplicity, according to a constrained randomization scheme of the basic two-sample comparison, we assume m subjects from A and m subjects from B can provide their measurements in every group k K . Consider the problem of testing the null hypothesis : X Y
H F F , when the distribution functions , X Y
F F are unknown. For example, sets A and B can be used to indicate individuals who receive two different treatments. In this case, H states that no treatment difference. To arrange the sequential decision-making procedure, say K SP , we fix a threshold , K C and execute the next algorithm: starting from k , (a) for group k , compute the statistic ,..., ( ), ( ), max p km u u X km Y km R ELR ELR u u ; (b) if , log km K R C , stop and reject H otherwise, if k K , continue to step (a), employing group k ; (c) in the case with , log km K R C , for all k K , do not reject H . The critical value , K C can be chosen to give overall TIE , i.e., Pr rejects
H K
SP H . In a similar manner to Section 3.1, we note that the null distribution of log km R does not depend on underlying data distributions. Then, for different values of , e.g., and m , , K C can be computed numerically that is exemplified in Section 4.1. Note that
0 ,1
Pr rejects Pr log
KK jm Kj
SP H R C . Proposition 5 below displays that the nonparametric procedure K SP is consistent. Proposition 5.
Let the assumptions of Proposition 4 be held and, for a constant d , 1 ( ) d K O m as m . Then, for a fixed C , we have Pr max log 1
H k K km
R mC , whereas Pr max log 0
H k K km
R mC , as m . Proof.
The proof uses the proof scheme of Proposition 4 and is shown in the supplementary materials. In this section, we exemplify evaluations of the critical values , K C for the proposed procedure , K SP considering different values of K and m . An R code for the corresponding computations is presented in the supplementary materials. Since
1 , 1 , ,
Pr max log Pr log ,..., log
H k K km K H m K Km K
R C R C R C , in a similar manner to Section 3.1, we can write the TIE rate of K SP in the form Pr rejects
H K
SP H
Pr max log | , ~ (0, ),1 , . k K km K i j p
R C N I i j Km X Y
In the case with p=2 , we tabulated the percentiles of the null distribution of the test statistic max log k K km R based on 20,000 generated samples of , ~ 0, , Ti i i
X X N I X and , ~ (0, ), Tj j j
Y Y N I Y i j Km when the number of groups K =2, 3, 4, 5, and the sample size per group m =5, 10. The results of this Monte Carlo study are displayed in Table 2. Table 2.
Critical values , K C of the group sequential MTS DBEL test based on K (=2, 3, 4, 5) groups of observations with m (=5, 10) data points per group. The significance level is . K M =0.2 =0.1 =0.05 =0.01 2 5 10.413 11.053 11.987 13.872 10 13.067 13.918 14.737 16.452 3 5 11.818 12.660 13.525 15.410 10 15.663 16.502 17.321 19.207 4 5 13.185 14.103 14.938 16.859 10 17.277 18.136 19.004 20.864 5 5 14.458 15.338 15.338 18.018 10 18.734 19.569 20.405 22.401 We can remark that, in the context of group sequential testing, Monte Carlo approximations to the critical values are often applied in practice (e.g., Jennison and Turnbull, 1999). Monte Carlo Experiments
We carried out an extensive Monte Carlo study to explore the performance of the proposed testing strategy in comparison to the modern MTS tests developed by Jurečková and Kalina (2012), Zhou et al. (2017), Biswas and Ghosh (2014). We are grateful for the authors of the papers, Jurečková and Kalina (2012) and Zhou et al. (2017), who provided us relevant programming codes and/or discussed details regarding implementations of their tests. As suggested, we used the Jurečková and Kalina (2012) type MTS test in the form proposed by Marozzi (2016). We considered generating data from various scenarios, evaluating more than 210 designs of sampling X and Y , under H . In this study, a case, where the proposed method demonstrated a relatively weak power, was not detected. Table 3 displays Designs D ,…, D and their explanations, exemplifying the treated scenarios. Table 3.
Distributions for , T X X X and , T Y Y Y used in the power study. Designs
Models/Descriptions D ~ 0,1 , ~ 0,1 X N X N vs. Y from a multivariate t -distribution with scale matrix 1 0.90.9 1 and degrees of freedom df=7, the corresponding R function (R Development Core Team, 2012) is ‘rmvt’. D ~ 0,1 , ~ (1) X N X Exp vs. ~ 0,1 , ~ 0,1
Y N Y LogN , where , , ,
X X Y Y are independent. D uses the skewed distributions. D
21 2 ~ ( 1,1), ~ 0,1.5
X Unif X N vs. ~ 0,1 , ~ 0,1 Y N Y N , where , , ,
X X Y Y are independent. D represents a case of a uniform distribution vs. a normal distribution as well as different variances of X - and Y -samples. (Oftentimes, it is difficult to goodness-of-fit-detect departures from normal distributions based on uniformly distributed data.) D ~ 0,1 , ~ 0,1 X N X N vs. ~ 0.5,1 , ~ 0.7,1
Y N Y N , where , , ,
X X Y Y are independent. D shows an obvious location-difference in X - and Y -distributions. D ~ , X N X μ vs. ~ , Y N Y μ , where
T X Y μ . D shows an obvious scale-difference in X - and Y -distributions. D ~ , X N X μ vs. ~ (0.5, 0.7) ,
T Y N Y , where , , X Y μ are defined in D . D shows a location/scale-difference in X - and Y -distributions. D ~ , X N X μ vs.
1, ~ (1), ~ 0,1
Y Exp Y N , where , Y Y are independent. D ~ , X N X μ vs. Y ~ N , , Y Y , where random variable and is independent of Y , Pr , μ is defined above. In this case, Y ~ N , , but Y is not bivariate normal. D ~ , X N X μ vs.
Y Y , where ~ 0,1
Unif and , , are independent (0,1) N -distributed random variables, μ is defined in D . In this case, Y ~ N , ,Y ~ N , , but Y is not bivariate normal, Stoyanov (2014), p. 97-98. Assuming that reasonable MTS tests based on large samples provide relatively equivalent and powerful outputs, in this Monte Carlo study, we focused on 10,000 replicates of ,..., n X X and ,..., m Y Y for each pair , n m (10,15), (15,10), (30,50), (50,30), (50,50), (65,70) and (70,65). Table 4 shows the results of the power evaluations of the proposed test (“
Log(TS nm ) ”), Zhou et al. (2017)’s test (“Z”), Jurečková and Kalina (2012)’s and Marozzi (2016)’s test (“JKM”) as well as Biswas and Ghosh (2014)’s test (“BG”). The significance level of the tests, , was supposed to be fixed at 5%. This study demonstrates the DBEL MTS test is superior to the considered Z, JKM and BG tests in almost all scenarios under the designs of D ,…, D . Only in the case D with (n,m) =(10,15), the Monte Carlo estimated power of Log(TS nm ) is less than that that of BG as 0.1049<0.1057, respectively. In contrast, under D with (n,m) =(30,50), the DBEL test and BG test have power levels of 0.3205 and 0.1265, respectively. Table 4.
The Monte Carlo power of the tests ( p , ). Design Test /(n,m) (10,15) (15,10) (30,50) (50,30) (50,50) (65,70) (70,65) D Log(TS nm ) Log(TS nm ) Log(TS nm ) . Log(TS nm ) JKM 0.1962 0.191 0.365 0.357 0.419 0.486 0.484 BG 0.1814 0.189 0.404 0.376 0.454 0.609 0.537 D Log(TS nm ) Log(TS nm ) Log(TS nm ) Log(TS nm ) Log(TS nm ) with (n,m) =(10,15). The H -design D considered samples from a t -distribution versus samples from a normal distribution. In this case, the power of the proposed test is roughly three times larger than that of the Z, JKM and BG tests. Note that, the JKM test is developed for location/scale alternatives that are exemplified by the designs D and D . Under D , D and D it is clear that the proposed procedure dramatically outperforms the JKM test. D and D present cases when , X X , , Y Y are identically distributed, but X and Y have different distributions. Under D and D , the DBEL test is significantly superior to the Z, JKM and BG tests. In this section we also demonstrate the Monte Carlo power evaluations of the Log(TS nm ) , Z, JKM and BG tests under the following alternative designs, where p . S : ~ , X N X μ vs., ~ (0.5, 0.7, 0.5) ,
T X N Y , where (0, 0, 0) , T μ , X ij ( ),1 , 3 ij I i j i j ; S : ~ , X N X μ vs. ~ , Y N Y μ , where , Y ij ( ) 0.5 ( ),1 , 3 ij I i j I i j i j ; S : ~ , X N X μ vs. ~ (0.5, 0.7, 0.5) ,
T Y N Y , where , , X Y μ are defined above. Table 5 shows the results of the power evaluations, supporting the experimental conclusions based on the outputs of Table 4. Table 5.
The Monte Carlo power of the tests ( p , ). Design Test /(n,m) (10,15) (15,10) (30,50) (50,30) (50,50) S Log(TS nm ) Log(TS nm ) Log(TS nm ) Z 0.0971 0.0301 0.5115 0.5010 0.6995 JKM 0.1510 0.1341 0.2962 0.2890 0.3165 BG 0.1683 0.1995 0.3845 0.3995 0.5205 Based on the Monte Carlo results, we conclude that the proposed test exhibits very high and stable power characteristics in comparison to the known modern procedures. Data Analysis
In this section, we present a data example to illustrate the practical application of the proposed method. The use of thiobarbituric acid-reactive substances (TBARS) as a value to summarize total circulating oxidative stress in individuals is common in laboratory research (Armstrong, 1994), but its use as a discriminant factor between individuals with and without myocardial infarction (MI) disease is still controversial (e.g., Schisterman et al ., 2001). Some authors have found a positive association between TBARS and MI disease (e.g., Jayakumari et al ., 1992; Miwa et al ., 1995), while others did not find corresponding significant associations (e.g., Karmansky et al ., 1996). The biomarkers, HDL-cholesterol, glucose and vitamin E, are historically known to be significantly associated with MI disease (e.g., Schisterman et al ., 2001). The aim of our study is to investigate the joint discriminative properties of the vector biomarker [TBARS, HDL, glucose, vitamin E] T , with regard to MI disease. Towards this end, we implemented the tests: Log(TS nm ) , Z, JKM and BG described in Section 5, using the following data. A sample of randomly selected residents of Erie and Niagara counties, 35 to 79 years of age, was employed in this investigation. The New York State department of Motor Vehicles drivers’ license rolls was used as the sampling frame for adults between the age of 35 and 65, while the elderly sample (age 65 to 79) was randomly selected from the Health Care Financing Administration database. The study evaluated 70 measurements of TBARS, HDL-cholesterol, glucose and vitamin E biomarkers. Half of them were collected on cases, who recently survived on MI disease (say MI=1), and the other half on controls, who had no previous MI disease (say MI=0). The p -values obtained via the Log(TS nm ) , Z, JKM and BG procedures are 0.0050, 0.0579, 0.0501 and 0.0235, respectively. The JKM test provides p -values that is slightly larger than a significance level of 5%. The proposed test reveals a strong evidence of an association between a joint distribution of the vector [TBARS, HDL, glucose, vitamin E] T and MI disease. Then, we organized a Bootstrap/Jackknife type study to examine the power performances of the test-statistics. The conducted strategy was that two samples with sizes n m were randomly selected from the data with MI=1 and MI=0 to be tested for the hypothesis H : the vector of biomarkers values, [TBARS, HDL, glucose, vitamin E] T , is distributed identically with respect to MI=0 and MI=1. The Log(TS nm ) , Z, JKM and BG tests were conducted at 5% level of significance. We repeated this strategy 5,000 times calculating the frequencies of the events { Log(TS nm ) rejects H }, {Z rejects H }, {JKM rejects H } and {BG rejects H }. The obtained experimental powers of the four tests are 0.77, 0.19, 0.51, 0.65, respectively. In this study, the proposed test outperforms the known Z, JKM and BG procedures in terms of the power properties when detecting the joint discriminative properties of the biomarkers values, [TBARS, HDL, glucose, vitamin E], with regard to MI disease. Concluding Remarks
In this article we developed a novel density-based empirical likelihood ratio mechanism for testing the equality of two multivariate distributions based on observed vectors. The proposed approach is exact and distribution-free. Our method employs a univariate projection pursuit-based procedure. It is indicated that correctness of applications of projection pursuit are critically important for constructing and performing multivariate decision-making procedures. We proved one-to-one-mapping between the equality of vectors’ distributions and the equality of distributions of relevant univariate linear projections. It turns out that an algorithm can be provided to simplify the use of projection pursuit via using only a few of the infinitely many linear combinations of observed vectors’ components. In this framework, the demonstrated proof algorithm can be applied to execute different multivariate procedures developed by using Kolmogorov Smirnov type testing mechanisms or ranks based concepts, when the univariate projection pursuit is employed. The displayed testing strategy was presented in retrospective and group sequential manners. The asymptotic consistency of the proposed technique was shown. Through extensive Monte Carlo experimental studies, we demonstrated that the proposed procedure has significantly higher power as compared with the modern methods of Jurečková and Kalina (2012), Marozzi (2016), Zhou et al. (2017) and Biswas and Ghosh (2014) across a variety of experiments’ scenarios, including, e.g., cases when observed X ’s and Y ’s components are identically distributed, whereas X and Y have different distributions. This study shows that the proposed test can efficiently detect relatively small departures from the null hypothesis that treats two multivariate distributions to be identical. Supplementary Materials Theoretical Results:
Proofs of Propositions 1,3,4,5 and the algorithm contained in Section 2 regarding the outcome , max ( ), ( ) nm nmu u A A TS TS X Y u u , max ( ), ( ) nmu u B A TS X Y u u ; an illustration of the statement of Proposition 3, when p R Codes:
Codes for Monte Carlo computing the critical values of the null distributions of the proposed tests.
References
Armstrong, D. (1994),
Free radicals in diagnostic medicine: a systems approach to laboratory, technology, clinical correlations, and antioxidant therapy , New York: Plenum Press. Baringhaus, L., and Franz, C. (2004), “On a new multivariate two-sample test,”
Journal of Multivariate Analysis,
88, 190-206 . Biswas, M., and Ghosh, A. K. (2014), “A nonparametric two sample test applicable to high dimensional data,”
Journal of Multivariate Analysis , 123, 160-171. Crouse, C. F. (1966), “Distribution Free Tests Based on the Sample Distribution Function,”
Biometrika , 53, 99-108. Friedman, J. H. (1987), “Exploratory projection pursuit,”
Journal of the American Statistical Association , 82, 249-252. Gordon, A.Y., and Klebanov, L.B. (2010), “On a paradoxical property of the Kolmogorov-Smirnov two-sample test,” In
Nonparametrics and Robustness in Modern Statistical Inference and Time Series Analysis: A Festschrift in Honor of Professor Jana Jurečková ; Antoch, J., Hušková, M., Sen, P.K., Eds.; Institute of Mathematical Statistics: Beachwood, OH, USA, Volume 7, 70–74. Gurevich, G., and Vexler, A. (2011), “A two-sample empirical likelihood ratio test based on samples entropy,”
Statistics and Computing , 21, 657–670. Hamedani, G. G. (1984), “Nonnormality of Linear Combinations of Normal Random Variables,”
The American Statistician,
38, 295-296. Jennison, C., and Turnbull, B. W. (1999),
Group sequential methods with applications to clinical trials.
CRC Press, New York. Jayakumari, N., Ambikakumari, V., Balakrishnan, K. G., and Subramonia Lyer, K. (1992), “Antioxidant status in relation to free radical production during stable and unstable angina syndromes,”
Atherosclerosis , 94, 183-190. Jurečková, J., and Kalina, J. (2012), “Nonparametric multivariate rank tests and their unbiasedness,”
Bernoulli , 18, 229-251. Karmansky, I., Shnaider, H., Palant, A., and Gruener, N. (1996), “Plasma lipid oxidation and susceptibility of low-density lipoproteins to oxidation in male patients with stable coronary artery disease,”
Clin Biochem , 29, 573-579. Marozzi, M. (2016), “Multivariate tests based on interpoint distances with application to magnetic resonance imaging,”
Statistical Methods in Medical Research , 6, 2593–2610. Miwa, K., Miyagi, U., and Fujita, M. (1995), “Susceptibility of plasman low density liporprotein to cupric ion-induced peroxidation in patients with variant angina,”
J Am Coll Cardiol , 26, 632-638. Nanda, A. K., and Chowdhury, S. (2020), “Shannon’s Entropy and Its Generalisations Towards Statistical Inference in Last Seven Decades,”
International Statistical Review.
In Press. doi:10.1111/insr.12374 Olkin, I., and Tomsky, J. L. (1981), “A new class of multivariate tests based on the union-intersection principle,”
The Annals of Statistics , 9. 792-802. Owen A. B. (2001),
Empirical Likelihood , Chapman and Hall/CRC, New York. R Development Core Team. (2012),
R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing.
J Cardiovasc Risk , , 219-225. Serfling, R. J. (2009), Approximation Theorems of Mathematical Statistics . Wiley: New York. Shao, Y., and Zhou, M. (2010), “A characterization of multivariate normality through univariate projections,”
Journal of Multivariate Analysis , 101, 2637-2640. Simpson, P. B. (1951), “Note on the estimation of a bivariate distribution function,”
Annals of Math. Stat ., 22, 476-478. Stoyanov, J. M. (2014),
Counterexamples in Probability : Third Edition. Dover Publications. New York. Tsai, W-M., Vexler, A., and Gurevich, G. (2013), “An extensive power evaluation of a novel two-sample density-based empirical likelihood ratio test for paired data with an application to a treatment study of Attention-Deficit/Hyperactivity Disorder and Severe Mood Dysregulation. Journal of Applied Statistics,”
40, 1189-1208. Vexler, A. (2020), “Univariate likelihood projections and characterizations of the multivariate normal distribution,”
Journal of Multivariate Analysis . In press. doi: 10.1016/j.jmva.2020.104643 Vexler, A., Tsai, W-M., and Hutson, A. D. (2014a), “A simple density-based empirical likelihood ratio test for independence,” Amer. Statist., 68, 158-169. doi:10.1080/00031305.2014.901922. Vexler, A., Tao, G., Hutson, A. D. (2014b), “Posterior expectation based on empirical likelihoods,” Biometrika . .
70 (3), 243-249 . Zhou, W-X, Zheng, C., and Zhang, Z. (2017), “Two-sample smooth tests for the equality of distributions,”
Bernoulli , 23, 951-989. Zou, L., Vexler, A., Yu, J., and Wan, H. (2019), “A Sequential Density-Based Empirical Likelihood Ratio Test for Treatment Effects,”
Statistics in Medicine , 38, 2115-2125. doi: 10.1002/sim.8095
Supplementary materials to Exact Multivariate Two-Sample Density-Based Empirical Likelihood Ratio Tests Applicable to Retrospective and Group Sequential Studies
Ablert Vexler , Gregory Gurevich and Li Zou Department of Biostatistics, The State University of New York at Buffalo, Buffalo, NY 14214, U.S.A, [email protected] Department of Industrial Engineering and Management, SCE- Shamoon College of Engineering Department of Statistics and Biostatistics, California State University, East Bay, Hayward, CA 94542, U.S.A.
Supplementary Materials: - Proofs of Proposition 1 -
To Section 2: The case with p , , max ( ), ( ) nm nmu u A A TS TS X Y u u , max ( ), ( ) nmu u B A TS X Y u u . - Proof of Proposition 3. -
Notations related to Proposition 3’s statement with p . - Proof of Proposition 4. -
R code to calculate the critical values of the proposed test (see Section 3.1) -
R code to calculate the critical values of the proposed test (see Section 4.1) -
References
Proof of Proposition 1.
Assume
Pr ( ) Pr ( )
X z Y z u u , for all , ,..., p z u u R . Then the characteristic function of X , ,..., E exp pX p j jj t t i t X with ,..., p t t R , satisfies ,..., exp Pr exp Pr ,..., p mX p j j j j Y pj j t t iz d t X z iz d t Y u t t , where i and ,..., Y p t t is the characteristic function of Y . This implies X F = Y F . Assume X F = Y F . We lose nothing in generality if we suppose that there are density functions X f and Y f of X and Y , respectively. The characteristic function of ( ) X u can be presented as
1( ) 11 1 11 1 ( )1 1 ... ,...,E exp ... exp ......... ,...,... exp ... ,... p p X pX u j j j j pj j pp Y pj j p Y uj p d dF x xt it u X it u x dx dxdx dxd dF x xit u x dx dx tdx dx where t R and ( ) Y u t is the characteristic function of ( ) Y u . Then we conclude that ( ) X u and ( ) Y u are identically distributed for all ,..., p u u R . The proof of Proposition 1 is complete. Section 2: The case with p , , max ( ), ( ) nm nmu u A A TS TS X Y u u , max ( ), ( ) nmu u B A TS X Y u u . To establish this result, we will apply the following scheme. The statistic ( ), ( ) nm TS X Y u u is based on the variables ( ) ( ) i j
I X Y u u , ( ) ( ) j i I Y X u u , ( ) ( ) i k I X X u u , ( ) ( ) j r I Y Y u u , i k n j r m . Thus, two vectors , T u u u , , T v v v satisfy ( ), ( ) ( ), ( ) nm nm TS X Y TS X Y u u v v , if ( ) ( ) ( ) ( ) i j i j I X Y I X Y u u v v , ( ) ( ) ( ) ( ) i k i k I X X I X X u u v v , ( ) ( ) ( ) ( ) j r j r I Y Y I Y Y u u v v , for all i k n and j r m . Let us focus on the equation ( ) ( ) i j
I X Y u u ( ) ( ) i j I X Y v v . We assume , T u u A u and consider three scenarios related to locations of values of u with respect to the ( , ) W i j , ( , ) U i r , ( , ) V j s -based order statistics ...
Q Q Q , where m n m n m . (1) Suppose that u Q , then
12 1 1 2 2 i j j i u X Y Y X , for all i n and j m . In n nm this case, since
11 1 2 2 2 1 1 2 2 2 2 2 , 0 i j j i i j j i j i
I X Y u Y X I X Y Y X u Y X
11 1 2 2 2 2 2 , 0 , i j j i j i
I X Y Y X u Y X we have i j j i I X Y u Y X j i I Y X , i.e. i i j j
I X u X Y u Y i i i i
I X X Y Y , where , 0, 1 T v . This implies ( ), ( ) ( ), ( ) nm nm TS X Y TS X Y u u v v . The scenario with
0, 1 T v corresponds to testing that X and Y are identically distributed, which is equivalent to testing the distributions of X and Y . Thus, according to the form of the DBEL ratio test statistic, we have ( ), ( ) ( ), ( ) nm nm TS X Y TS X Y v v v v , where T A v . (2) Now, we suppose that d d
Q u Q , d . In this case, the event
11 1 2 2 2 i j j i
X Y Y X u equals to the event
11 1 2 2 i j j i d
X Y Y X Q and then the events
11 1 2 2 2 i j j i
X Y Y X u and
11 1 2 2 i j j i d
X Y Y X Q are equivalent. This leads to , 0 i j j i i j j i j i I X Y u Y X I X Y u Y X Y X , 0 i j j i j i
I X Y u Y X Y X i j d j i
I X Y Q Y X , for all i n j m . This implies i i j j i i j jd d I X u X Y u Y I X Q X Y Q Y , that is ( ), ( ) ( ), ( ) nm nm
TS X Y TS X Y u u v v , where T d
Q B v . (3) Suppose that u Q , i.e.
11 1 2 2 2 i j j i
X Y Y X u , for all i n , j m . In this case, , 0 i j j i i j j i j i I X Y u Y X I X Y u Y X Y X , 0 i j j i j i
I X Y u Y X Y X
11 1 2 2 2 2 2 , 0 i j j i j i
I X Y Y X u Y X
11 1 2 2 2 2 2 2 2 , 0 0 i j j i j i j i
I X Y Y X u Y X I Y X . This can be rewritten as i i j j i i j j
I X u X Y u Y I X X Y Y with , 0,1 T v that leads to ( ), ( ) ( ), ( ) nm nm TS X Y TS X Y u u v v , where T A v . It is clear that, when , u Q we have ( ), ( ) ( ), ( ) nm nm TS X Y TS X Y u u v v , where T Q B v . The concept outlined above can be easily applied to evaluate the equations ( ) ( ) ( ) ( ) i k i k I X X I X X u u v v , ( ) ( ) ( ) ( ) j r j s I Y Y I Y Y u u v v , where , A A B A u v . Therefore, in order to compute nm TS , we can treat the linear combinations, ( ), ( ) X Y v v , with , v v B A only. Proof of Proposition 3.
For key principles used in the proof below, we refer the reader to the analysis presented with respect to the case with p=2.
These principles will be recursively employed to prove Proposition 3. We will establish that for each vector ,...,
T pp j j u u A u , there exists some vector ,..., T pp j j v v B v such that ( ), ( ) ( ), ( ) mn mn TS X Y TS X Y u u v v . To this end, we consider the indicators ( ) ( ) , i j I X Y u u ( ) ( ) , i j I X X u u ( ) ( ) , i j I Y Y u u ( ) ( ) , i j I X Y v v ( ) ( ) , i j I X X v v ( ) ( ) , i j I Y Y v v on which the statistics ( ), ( ) mn TS X Y u u and ( ), ( ) mn TS X Y v v are based. We begin with ,..., Tp u u A u . Denote the order statistics ... p Q Q Q based on the set of the variables , , W p U p V p
W J U J V J . In this case, p can be calculated via the recursive formula h h h with h p and n m n m . We assume that
22 2 1 d d
Q u Q , for some p d . In this case, c cW p W p W p W p dc cW p W p W p W p W J W J W J W JI u I QW J W J W J W J and then , c cW p W p W p W pc cW p W p W p W pd I W J W J u W J W JI W J W J Q W J W J . c cW p W p W p W pp c cW p W p W p W pd d I W J W J u W J W J uI W J W J Q W J W J Q
Defining d v Q B , we obtain . c cW p W p W p W pc cW p W p W p W p I W J W J u W J W J uI W J W J v W J W J v
This equation means that, for any fixed , 2
W p J and fixed , d d u Q Q , the rank of W p W p
W J u W J in the sequence of the variables c cW p W p W J u W J , for all different , 2 cW p J , equals to the rank of c cW p W p W J v W J in the sequence of the variables c cW p W p W J v W J , for all different , 2 cW p J . Now we define ... p O O O to be the order statistics based on W p W p
W J u W J , U p U p
U J u U J , V p V p
V J u V J . Assume that
33 3 1 d d
O u O , for some p d , then pW p W p W p W p I W J u W J u I W J v W J u , for any u that satisfies
33 3 1 d d
Q u Q , where ... p Q Q Q are the order statistics based on W p W p
W J v W J , U p U p
U J v U J , V p V p
V J v V J . Thus, we have W p W p W p W p d
I W J u W J u I W J v W J Q , and denoting d v Q , we obtain W p W p W p W p
I W J u W J u I W J v W J v . (A.1) This leads to c cW p W p W p W pc cW p W p W p W p
W J W J W J W JI u uW J W J W J W J c cW p W p W p W pc cW p W p W p W p
W J W J W J W JI v vW J W J W J W J , where the definitions of
W p
W J and W p
W J are employed, that implies c c cW p W p W p W p W p W p I W J W J u W J W J u W J W J c c cW p W p W p W p W p W p
I W J W J v W J W J v W J W J , and then c c cW p W p W p W p W p W p
I W J u W J u W J W J u W J u W J p c c cW p W p W p W p W p W p
I W J v W J v W J W J v W J v W J . (See the analysis presented regarding the case with p=2, for details . ) In a recursive manner, we can employ the scheme shown above in order to obtain that p p p pc ci i W i i W i i W i i Wi i i i I u W J u W J I v W J v W J , where u v ; it is assumed that i i ii d i d Q u Q , for some i p i d , i p ; ... p i i i i O O O are the order statistics based on the variable i j j W p ij u W J , i j j U p ij u U J , i j j V p ij u V J ; i i i d v Q ; ... p i i i i Q Q Q are the order statistics based on i j j W p ij v W J , i j j U p ij v U J , i j j V p ij v V J . In the next step of the proof, we assume that p p pp d p d O u O for p d where ... p p p O O O are the order statistics based on p j j Wj u W J , p j j Uj u U J , p j j Vj u V J . Denote p p p d v Q , where ... p p p Q Q Q are the order statistics based on p j j Wj v W J , p j j Uj v U J , p j j Vj v V J . It is clear that we can apply the concept used for obtaining Equation (A.1) to conclude that p pj j W p j j W pj j I u W J u I v W J v . This result can be rewritten as p pk ki kj pj pi p k ki kj pj pi pk k
I u X Y Y X u I v X Y Y X v . Then, p p p pk ki k ki k ki k kik k k k
I u X u Y I v X v Y for all i n j m . That is, it turns out that if ,..., Tp u u A u when i i ii d i d O u O with i p i d , i i i d v Q , i p , then ( ) ( ) ( ) ( ) i j i j I X Y I X Y u u v v for all i n and j m , where ,..., Tp v v B v . Now, we consider the scenario with p p u O . In this case, p k k W pk u W J u , i.e. p k ki kj pj pi pk u X Y Y X u , for all i n and j m . Hence, we have p k ki kj p pj pi pj pik I u X Y u Y X I Y X , This can be represented as p p p pk ki k kj k ki k kjk k k k I u X u Y I v X v Y with , ,..., 0,..., 0,1
T p v v that leads to ( ), ( ) ( ), ( ) nm nm TS X Y TS X Y u u v v , where T p p
B A v . In a similar manner to the algorithm shown above, we have that if p k k W pk u W J u , i.e. p k ki kj pj pi pk u X Y Y X u , for all i n and j m . We obtain p k ki kj p pj pi pj pik I u X Y u Y X I Y X , This implies p p p pk ki k kj k ki k kjk k k k I u X u Y I v X v Y with , ,..., 0,..., 0, 1
T p v v and then ( ), ( ) ( ), ( ) nm nm TS X Y TS X Y u u v v , where T v . The scenario with T v corresponds to testing that p X and p Y are identically distributed, which is equivalent to testing distributions of p X and p Y . Thus, according to the form of the DBEL ratio test statistic, we have ( ), ( ) ( ), ( ) nm nm TS X Y TS X Y v v v v , where T p p
B A v . We next consider the cases with i i ii d i d O u O and p j j j u O , where i p i d , i j p and j p . In these cases, j k k W p j jk u W J u . Then,
11 , , , ,1 j c ck k W p j k W p j j W p j j W p j jk u W J W J W J W J u . This provides j c ck k W p j k W p j j j W p j j W p jk I u W J W J u W J W J , , cj W p j j W p j I W J W J , i.e. , ,1 1 j j ck k W p j k k W p jk k
I u W J u W J , , cj W p j j W p j I W J W J , for a fixed vector , W p j J and different , cW p j J . That is, for a fixed vector , W p j J , the tank of ,1 j k k W p jk u W J in the sequence of the variables ,1 j ck k W p jk u W J , for all different , cW p j J , equals to that of , j W p j W J in the sequence of , cj W p j W J , for all different , cW p j J . For , s s s s d s d u O O , we have , ,1 j k k W p j s j W p j sk I u W J u I W J u , where s j , ( ) ( 1) , s s s d d u Z Z , (1) (2) ... Z Z are the order statistics based on , j W p j W J . Hence , 1 , 11 j k k W p j j j W p j jk I u W J u I W J v , where ( ) , 1 s s d v Z s j . By virtue of the definitions of , k W p j W J , we obtain , 1 , 1 11 1 , 1 1 , 1 cj k W p j k W p jk jck j W p j j W p j W J W JI u uW J W J , 1 , 1 11 , 1 1 , 1 cj W p j j W p j jcj W p j j W p j W J W JI vW J W J that gives j j ck k W p j k k W p jk k
I u W J u W J , 1 1 1 , 1 , 1 1 1 , 1 ck W p j j j W p j k W p j j j W p j I W J v W J W J v W J . The same arguments employed in dealing with the proof scheme shown above immediately lead to p p ck k W k k Wk k
I u W J u W J p pc cj W k k W j W k k Wk j k j I W J v W J W J v W J that means, for all s , p pk ki k kik k I u X u Y p pji k ki js k ksk j k j I X v X Y v Y with ( ) j k j k d v Q , where (1) (2) .... Q Q are the order statistic based on
1, 2 , 21 j kj W p j k i i W p j ki j
W J v W J , j k p j k d . k p j . That is, ( ), ( ) ( ), ( ) nm nm TS X Y TS X Y u u v v , where T j p j v v B v . By similar techniques, we can obtain the following results: If, for all i j p , we have i i ii d i d O u O with i p i d , and j j u O , where j p , then we can find a vector T j p w w w that satisfies ( ), ( ) ( ), ( ) nm nm TS X Y TS X Y u u w w and T j p j w w B v . Since testing that random variables and are identically distributed is equivalent to testing equality of distributions of and , according to the form of the DBEL ratio test statistic, we have ( ), ( ) ( ), ( ) nm nm TS X Y TS X Y w w v v . It can be shown that if u Q or p u Q then ( ), ( ) ( ), ( ) , nm nm TS X Y TS X Y u u v v where T B v . The proof scheme displayed above can be directly applied to analyze the equations ( ) ( ) ( ) ( ) i r i r
I X X I X X u u v v , ( ) ( ) ( ) ( ) j s j s I Y Y I Y Y u u v v , i r n , j s m as well as to consider the situations with ,..., Tp j u u A u , j p . The proof of Proposition 3 is complete. Notations related to Proposition 3’s statement with p . , , : B u u u
1; , , , when , , ; c c cW U V W W U U V V u u W J U J V J J J J J J J given chosen above, , ,
W W U U V V u u W J u W J U J u U J V J u V J , , , : 0; 1; , , W U V
B u u u u u u W J U J V J ,
0, 0, 1
B A u u u , where we have: cW WW cW W
W J W JW J W J W J , i jW j i X YW J Y X , i jcW j i X YW J Y X , i jW j i X YW J Y X and i jcW j i
X YW J Y X ; i j i jj i j iW i j i jj i j i X Y X YY X Y XW J X Y X YY X Y X , , 1,..., i i n , , 1,..., j j m , , , i j i j , i j i jW W j i j i X Y X YW J u W J uY X Y X , i n , j m , cU UU cU U U J U JU J U J U J , i rU r i X XU J X X , i rcW r i X XU J X X , i rU r i X XU J X X and i rcW r i
X XU J X X ; i r i rr i r iU i r i rr i r i X X X XX X X XU J X X X XX X X X , , , , 1,..., i i r r n , , , i r i r , j j i r ,
1, 2 j , i r i rU U r i r i X X X XU J u U J uX X X X , , 1,..., i r n , i r ; p p cV VV p c pV V V J V JV J V J V J , j sW s j Y YV J Y X , j scV s j Y YV J Y Y , j sV s j Y YV J Y X , j scV s j Y YV J Y Y ; j s j ss j s jV j s j ss j s j Y Y Y YY Y Y YV J Y Y Y YY Y Y Y , , , , 1,..., j j s s m , , , j s j s , i i j s ,
1, 2 i , j s j sV V s j s j Y Y Y YV J u V J uY Y Y Y , , 1,..., j s m , j s . Proof of Proposition 4.
In order to simplify the needed proof scheme of Proposition 3, it is clear that, we can focus on the case with bivariate observations and, by virtue of Proposition 2, we can state the problem in the following form. Let ( ) , 1,..., , i i i
X a X aX i n and ( ) , 1,..., , i i i
Y a Y aY i m where a R . Assume that ( ) ~ ( ; ) i H X a f u a , under H ; ( ) ( ) ~ ( ; ) i X a X a f u a , under H ; ( ) ~ ( ; ) i H Y a f u a , under H ; ( ) ( ) ~ ( ; ) i Y a Y a f u a , under H . Let a distribution function ( ; ) H F u a correspond to the H -density ( ; ) H f u a . Define the DBEL ratios , 1
2( ) min ( ) ( ) n n nX n a k b i n k n ki k i k kELR a n F X a F X a , , 1 ( ) ( )
2( ) min ( ) ( ) m m mY m a r b i n k i r n k i r rELR a m F Y a F Y a , j a j , min , 0.5 j b j j , , j n m ,
0, 0.25 , where ( ) ( ) n mn m i ji j F t n m I X a t I Y a t is the H -empirical distribution function, (1) (2) ( ) ( ) ( ) ... ( ) n X a X a X a and (1) (2) ( ) ( ) ( ) ... ( ) m Y a Y a Y a are the order statistics based on observations ( ), 1,..., i X a i n and ( ), 1,..., i Y a i m , respectively. Here ( ) ( ) ( ) ( ) i r n X a X a , if ; i r n ( ) (1) ( ) ( ) i r X a X a , if i r ( ) ( ) ( ) ( ) i s m Y a Y a , if i s m ; ( ) (1) ( ) ( ) i s Y a Y a , if i s . Note that, we can present ( ) ( ) n m X a n Y a m F t n m nF t mF t , where the empirical distribution functions
1( ) 1 ( ) , nX a n ii
F t n I X a t
1( ) 1 ( ) mY a m ii
F t m I Y a t . We consider the statistic , , max ( ) ( ) nm a X n Y m TS ELR a ELR a , aiming to prove that there is a positive threshold C , such that Pr log 1
H nm n m TS C and Pr log 0
H nm n m TS C , when / 0 n m as n . Define
01 001 0 ( ) 1 0 00 ( ) 1 0 0( ) 1 0 0( ) 1 0 0 ( );1E log1 1 1 ( );( );1 1 E log1 1 1 ( );
YH XXH Y f Xf Xf Yf Y uuuu u uu u uu uu u and note that
01 001 0 ( ) 1 0 00 ( ) 1 0 0( ) 1 0 0( ) 1 0 0 ( );1log E1 1 1 ( );( );1 1log E 0.1 1 1 ( );
YH XXH Y f Xf Xf Yf Y uuuu u uu u uu uu u Lemma A.1.
For a fixed a , T a and a threshold C , which satisfies C we have Pr log 1
H nm n m TS C , as n . Proof.
Pr log Pr log ( ) ( ) 1
H nm H X n Y m n m TS C n m ELR a ELR a C as n , since Proposition 4.1 of Gurevich and Vexler (2011). This completes the proof of Lemma A.1. Now, we assume that the null hypothesis is true and obtain the following result. Lemma A.2.
Under H , for all C , we have Pr log 0
H nm n m TS C , as n . Proof.
We begin with the trivial inequality: since the definitions of , ( ) X n
ELR a and , ( ) Y m
ELR a involve the minimums, we have
Pr log 2Pr max log ( ) ( )2 ( ) ( ) ,
H nm nH a n m n mi n i nim n m n mi m i mi n m TS C nn m F X a F X anm F Y a F Y a Cm where
1/ 4 . Then, using the definitions of ( ) ( ) n m X a n Y a m
F t n m nF t mF t and the order statistics ( ) ( ) ( ) ( ) ( ), ( ), ( ), ( ) i r i r i s i r X a X a Y a Y a , we can represent
Pr log Pr min ( ) ( )
H nm H a kn mn n m TS C n m A a A a C with ( ) log ( ) ( ) ,2 ( )( ) log ( ) ( )2 ( ) nmn n Y a m n Y a mi n i nimn n X a n n X a nj m i m nA a g i n mF X a g i n mF X an n mmA a g j m nF Y a g j m nF Y am n m mj and the function n g t I t tI t n nI t n . Now, to replace the empirical distribution functions with the corresponding theoretical distributions, we define ( ) ( ) ( ); , ( ) ( ); ( ) ,( ) ( ); ( ) , ( ) ( ) imn Y a m H imn H X a ni n i n i n i nimn H Y a m imn X a n Hi n i n i n
G a F X a F X a a G a F X a a F X aG a F X a a F X a G a F X a F X
11 1 1 10 01 10 ( ); ,( ) ( ) ( ); , ( ) ( ); ( ) ,( ) ( ); ( ) , ( ) i njmn X a n H jmn H Y a mj m j m j m j mjmn H X a n jmn Y a mj m j m a aa F Y a F Y a a a F Y a a F Y aa F Y a a F Y a a F Y ( ) ( ); ,
Hi m i m a F Y a a and rewrite ( ) log 2 ( )( ) ( ) ( ) ,( ) log 2 ( )( ) nmn n niX a n X a n kimni n i n kmmn n njY a m Y a mj m j m nA a g i n g i nn n mmF X a mF X a m G amA a g j m g j mm n mnF Y a nF Y ( ) ( ) . kjmnk a n a Then, we obtain the following inequality. For , Pr logPr min ( ) ( ) , max ( ) , max ( )Pr max ( ) Pr max ( ) ,
H nmH a nm nm a m a nH a m H a n n m TS Cn m A a A a C G a m G a nG a m G a n where ( ) ( ) ( ) sup ( ) ; , ( ) sup ; m t Y a m H n t H X a n
G a F t F t a G a F t a F t provide ( ) 2 ( ) 2 ( ) kimn m nk G a G a G a and ( ) 2 ( ) 2 ( ). kjmn m nk a G a G a (The event max ( ) , max ( ) a m a n G a m G a n insures that the arguments of the log(.) ’s appeared in the inequalities below are positive for large values of n . ) This implies that, for ( ) log 1 2 ( ) 2 ( ) ,2 ( )( ) log 1 2 ( ) 2 ( ) ,2 ( ) nnm n n m ni mnm n n m nj nJ a g i n g i n mn mG a mG an n mmJ a g j m g j m nm nG a nG am n m we have Pr log
H nm n m TS C
00 0
Pr min ( ) ( ) , max ( ) , max ( )Pr max ( ) Pr max ( ) .
H a nm nm a m a nH a m H a n n m J a J a C G a m G a nG a m G a n
Now, we can evaluate probabilistic characteristics of notations containing ( ) m G a and ( ) n G a using Kolmogorov-Smirnov type inequalities. To this end, noting that n g t I t tI t n nI t n , ( ) log 1 1 2 ( ) 2 ( )2 ( )log 1 2 ( ) 2 ( )2 ( )log 2 1 2 ( ) 2 ( )2 ( ) nnm m nin m ni n nn n m ni n nJ a i n mn mG a mG an n mn n i n mn mG a mG an n mn n mn mG a mG an n m and ( ) log 1 1 2 ( ) 2 ( )2 ( ) mnm m nj mJ a j m nm nG a nG am n m log 1 2 ( ) 2 ( )2 ( )log 2 1 2 ( ) 2 ( ) ,2 ( ) m m nj m mm m m nj m m n j m nm nG a nG am n mm m nm nG a nG am n m we obtain Pr logPr , max ( ) , max ( )Pr max ( ) Pr max ( )
H nmH nm nm a m a nH a m H a n n m TS Cn m J J C G a m G a nG a m G a n with
11 11 1111 nnm a m a nin a m a ni n nn n a m a ni n i n n mJ G a G an n mn i n n m G a G an n mn m G a G an m
12 11 1111 mnm a m a njm a m a nj m mm m a m a nj m j m nmJ G a G am n mn j m nm G a G am n mnm G a G an m Thus, it is clear that, defining , where
1/ 4 , we have, e.g., ( ) 0 n m n m as n , where ( ) m O n , and then, for relatively large n , we obtain Pr log Pr max ( ) Pr max ( ) ,
H nm nm nm H a mH a n n m TS CI n m C C C G a mG a n (A.2) where nnm ini n n i n n m n mC n n m n mn i n n m n m n m n mn nn n m n m n m n m n m n m n m n mn nn m n m n m n mn m n mn n n m n m mnm jm m mj m m j m j m nm n mC m n m n mn j m nm n m nm n mm n m n m n m n m nm n m nm n mm mn m n m n m n mnm n mm m n m n m and nm nm n m C C as n . Therefore, we conclude that Pr log (1) Pr max ( )
H nm H a m n m TS C o G a m Pr max ( ) .
H a n
G a n
Proposition 3 implies that we can define the set
11 1 2 2 11 1 2 211 1 2 2 , 1,..., , 1,..., , 1 , 1 nm ij i j j iij i j j iij i j j i
D W X Y Y X i n j mU X X X X i j nV Y Y Y Y i j m to present
Pr log Pr log max ( ) ( )Pr log max ( ) ( ) . nm H nm H a X n Y mH X n Y ma D n m TS C n m ELR a ELR a Cn m ELR a ELR a C
Thus, Pr log (1)
H nm n m TS C o Pr , max nm n m H l ij l ij la Dl m n i j G W l G W G a Pr , max nm n n H l ij l ij la Di j G U l G U G a Pr , max nm m m H l ij l ij la Di j G V l G V G a (1) Pr Pr n m n nH l ij H l ijl m n i j i j o G W l G U l Pr m m H l iji j G V l . (A.3) We consider Pr n m H l ijl m n i j G W l that is a summand presented in the upper bound shown in (A.3). Note that, denoting F as the inverse or quantile function of a distribution function F , such that F F u u , we have ( ); ( ); ( ) sup ( ) , G ( ) sup H H m u n uF Y a a m F X a a n
G a F u u a u F u and then n m mH u H j ij ij H s ij iji j s s jm H s ij ijs s j
I F Y W W u I F Y W W um m mI F Y W W u u mm n m nH u H i ij ij H s ij iji j s s in H s ij ijs s i I F X W W u I F X W W un n nI F X W W u u nn where
11 1 2 2 ij i j j i
W X Y Y X . This leads to Pr 2 n m H jm iji j
G W m m Pr 2 n m H in iji j
G W n n , where
11 1, 12 1,
1( ) sup ( ); ,11( ) sup ( ); .1 mjm u H ss s jnin u H ss s i
G a I F Y a a u umG a I F X a a u un It is clear that, when and
1/ 4 , for relatively large n , such that m m bm and n n bn ( b is a constant), the inequality obtained above yields E Pr | n m H H jm ij iji j
G W bm W E Pr | n m H H in ij iji j
G W bn W , where E H means the expectation derived under H . For a fixed
11 1 2 2 ij i j j i
W X Y Y X , the statistics , jm ij in ij G W G W contain the empirical distribution functions m H s ij ijs s j m I F Y W W u and n H s ij ijs s i n I F X W W u that are based on independent and identically distributed random variables. By virtue of the theorem of Dvoretzky, Kiefer and Wolfowitz (Serfling, 2009, p. 59), we have, for , Pr | exp 2 ( 1) and Pr | exp 2 ( 1) ,
H jm ij ijH in ij ij
G W bm W C b m mG W bn W C b n n where C , C are finite positive constants (not depending on distributions of ( ) X a and ( )
Y a ). Then exp 2 ( 1) exp 2 ( 1)
C b m m nm C b n n nm . It is clear that the terms presented in the upper bound obtained in (A.3) can be evaluated in a similar manner to the ’s evaluation shown above. This implies that, for constants
0, 0
C C , we have Pr log (1) exp 2 ( 1) exp 2 ( 1)
H nm n m TS Co C b m m nm C b n n nm (A.4) that means Pr log 0
H nm n m TS C as n and Lemma A.2 is proven. Lammas A.1 and A.2 provide the statement of Proposition 4. Proof of Proposition 5.
Without loss of generality, we can assume the framework that is presented above Lemma A.1 in the proof of Proposition 4. In this case, ( ) , 1,..., , i i i
X a X aX i km and ( ) , 1,..., , i i i
Y a Y aY i km where a R and k K . Then, , , max ( ) ( ) km a X km Y km R ELR a ELR a . Under H , by virtue of Lemma A.1, we can find a constant C such that, for s K Pr max log Pr log 1
H k K km H sm
R mC R mC . Under H , we have the inequality
1 1
Pr max log Pr log .
KH k K km H sms
R mC R mC Now, we reconsider the result (A.4), noting that the term (1) o in Inequality (A.4) means nm nm I n m C C C used in (A.2), when the notations of the proof of Proposition 4 are in effect. Employing Result (A.4), where , n m are redefined to equal to sm , we can show that, for r sm ,
1 11 2 2 21 2 112 2 22
Pr log exp 2 ( 1) exp 2 ( 1)
H sm rr rr m R CI m C C C C b sm sm s mC b sm sn s m with the deterministic term rr rr m C C , as m , where , b is a constant and C , C are finite positive constants. Then, for large values of sm and positive constants , M M , we can write Pr log exp
H sm m R C M s m M sm . This leads to the inequality Pr max log Pr log exp 0
KH k K km H sms
R mC R mC M K m M m that completes the proof of Proposition 5. R code to calculate the critical values of the proposed test (see Section 3.1) library(MASS) x1<- txx1[1:n1] x2<- txx2[1:n2] sx<-sort(x1) sy<-sort(x2) a<-replicate(n2,m) rm<-as.vector(t(a)) L<-c(1:n2)- rm LL<-replace(L, L <= 0, 1 ) U<-c(1:n2)+ rm UU<-replace(U, U > n2, n2) yL<-sy[LL] yU<-sy[UU] F<- n1*(ecdf(sx)(yU)-ecdf(sx)(yL)) + (UU-LL) F<-F/(n1+n2) F[F==0]<-1/(n1+n2) I<-2*rm/ ( n2*F ) uy<-array(I, c(n2,length(m))) tstat2<- log(min(apply(uy,2, prod))) Test_Stat<- tstat1+tstat2 return(Test_Stat) } XXX12<-rbind(XXX1,XXX2) for (i in 1:length(XXX12[,1])) {
R code to calculate the critical values of the proposed test (see Section 4.1) xL<-sx[LL] xU<-sx[UU] F<-(UU-LL)+n2*(ecdf(sy)(xU)-ecdf(sy)(xL)) F<-F/(n1+n2) I<-2*rm/ ( n1*F ) ux<-array(I, c(n1,length(m))) tstat1<-log(min(apply(ux,2,prod))) XXX12<-rbind(XXX1,XXX2) for (i in 1:length(XXX12[,1])) { References
Gurevich, G. and Vexler, A. (2011). A two-sample empirical likelihood ratio test based on samples entropy.
Statistics and Computing , 21, 657–670. R Development Core Team. (2012).
R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing.