Nonparametric Tests in Linear Model with Autoregressive Errors
aa r X i v : . [ m a t h . S T ] J u l Nonparametric Tests in Linear Model withAutoregressive Errors
Olcay Arslan , Yes¸im G¨uneyJana Jureˇckov´a ∗ , Yetkin Tuac¸
University of Ankara and Charles University, Prague
Abstract
In the linear regression model with possibly autoregressive errors, wepropose a family of nonparametric tests for regression under a nuisanceautoregression. The tests avoid the estimation of nuisance parameters, incontrast to the tests proposed in the literature. A simulation study, as wellas an application of tests to real data, illustrate their good performance.Keywords: Autoregression Linear regression Rank test Regression rankscores Autoregression rank scores
The standard assumption of the independent and identically distributed errorsin the linear regression model is often violated. Some authors admit the au-toregressive structure of model errors. McKnight et al. (2000) applied a doublebootstrap method to analyze linear models with autoregressive errors. The au-thors mostly estimated the regression parameters under autoregressive errorswith a known innovation distribution. Alpuim and El-Shaarawi (2008) usedthe ordinary least squares (OLS) estimator under the p -order autoregressive(AR(p)) error term and the normal innovations. Tua¸c et al. (2018) consid-ered linear regression model with AR(p) errors with Student’s t-distributionand used conditional maximum likelihood estimation of model parameters. In(2020), Tua¸c et al. proposed an autoregressive regression procedure based onthe skew-normal and skew- t distributions. G¨uney et al. (2020a) considered theconditional maximum Lq-likelihood (CMLq) estimation method for the autore-gressive error terms regression models under normality assumption.In the real life applications, the data sets may contain outliers and theirdistribution can be heavy-tailed. Then we should take recourse to nonpara-metric models without a specific distribution assumptions. The most powerful ∗ The research of J. Jureˇckov´a was supported by the Grant GAˇCR 18-01137S
We consider the linear regression model of order s, whose model errors follow astationary autoregressive process of order p : y t = β + x ⊤ t β ∗ + ε t = β + x t β + ... + x tp β p + ε t , (2.1) ε t = ϕ + ϕ ε t − + u t + . . . + ϕ p ε t − p , t = 1 , , ..., n, (2.2) β ∗ = ( β , . . . , β s ) ⊤ . Here y t is the response variable, x t = ( x t , . . . , x ts ) ⊤ are the regressors and β j , j = 0 , . . . , s are unknown regression parameters. We assume that x t = 1for all t, hence that β is an intercept.Moreover, ϕ , ϕ , . . . , ϕ p are unknown autoregression parameters, where theintercept ϕ is added for mathematical convenience and can be 0. The inno-vations u t are assumed being independently and identically distributed ( i.i.d. )with a continuous distribution function F and density f, generally unknown butsatisfying E ( u t ) = 0 , V ar ( u t ) = σ < ∞ . (2.3)The stationarity condition requires that all roots of the equation z − ϕ z =0 − ϕ z p − − ... − ϕ p z t − p = 0 are inside the unit circle (Brockwell and Davis,1987). Note that the error model given in equation (2.2) is a strictly stationaryprocess, and so the ε t share a common marginal distribution and thus sharethe same quantiles. The intercept term ϕ in model (2.1) is included for the2dentifiability of the autoregression quantiles, and can be equal to 0. The dis-tribution function F of u t is unknown, but we assume that it is increasing onthe set { u : 0 < F ( u ) < } . Because of the identifiability, we assume that thestarting observations ( y − p +1 , . . . , y ) are known.For the convenience, we also write (2.2) in the form u t = Φ ( B ) ε t (2.4)where B is called the backshift operator. Then the linear regression model withAR(p) error term given in equation (2.1) can be expressed asΦ ( B ) y t = y t − ϕ − ϕ y t − + ...ϕ p y t − p , (2.5) Φ ( B ) x t = x t − ϕ − ϕ x t − + . . . ϕ p x t − p , hence Φ ( B ) y t = ( Φ ( B ) x t ) T β + u t . (2.6)In the model (2.1), we construct the tests of the hypothesis: H : β ∗ = , with β , ( ϕ , ϕ , . . . , ϕ p ) ⊤ = unspecified.Our tests are nonparametric; the test of H is based on the autoregression rankscores and on the linear autoregression rank statistics for the model (2.1), (2.2)without regression. H : β ∗ = , β , ϕ , ϕ , . . . , ϕ p unspecified . The usual alternative is the local (Pitman) regression K n : β ∗ = β ∗ n = n − / β ∗ x , with β ∗ x ∈ IR p fixed. (3.1)Under H , the observations follow the model y t = β + ε t , t = 1 , . . . , n. The hypothesis H in fact means the hypothetical autoregressive model H : y t = β + ϕ + ϕ y t − + ϕ y t − + . . . + ϕ p y t − p + u t , t = 1 , . . . , n (3.2)which we like to test against the alternative K n . Let y ∗ t = ( y t , y t − , . . . , y t − p ) ⊤ , y t = (1 , y t , y t − , . . . , y t − p ) ⊤ , t = 0 , . . . , n − n × p and n × ( p + 1) Y ∗ n = y ∗⊤ · · · y ∗⊤ n , Y n = y ⊤ · · · y ⊤ n . (3.4)3or convenience, denote also x ∗ t = ( x t , . . . , x tp ) ⊤ , x t = (1 , x t , . . . , x ts ) ⊤ = (1 , x ∗⊤ t ) ⊤ X ∗ n = x ∗⊤ · · · x ∗⊤ n , X n = x ⊤ · · · x ⊤ n . The autoregression rank scores b a n ( α ) = (ˆ a n ( α ) , . . . , ˆ a nn ( α )) ⊤ under hy-pothesis H are defined as the solution vector of the linear programming prob-lem P nt =1 y t ˆ a nt ( α ) : = max P nt =1 (ˆ a nt ( α ) − (1 − α )) = 0 Y ∗⊤ n ( b a n ( α ) − (1 − α ) n ) = b a n ( α ) ∈ [0 , n , ≤ α ≤ . (3.5)The autoregression rank scores are autoregression-invariant . More precisely,(3.5) implies that b a n ( α ) can be also formally written as a solution of the linearprogram P nt =1 u t ˆ a nt ( α ) : = max P nt =1 (ˆ a nt ( α ) − (1 − α )) = 0 Y ∗⊤ n ( b a n ( α ) − (1 − α ) n ) = b a n ( α ) ∈ [0 , n , ≤ α ≤ . where u n = ( u , . . . , u n ) ⊤ is the unobservable white noise process.We shall construct a family of new tests of the hypothesis H for the model(2.1), based on autoregression rank scores, and analyze the asymptotic distribu-tion of the test criterion under the null hypothesis as well as under contiguousalternatives. Surprisingly, no preliminary estimation of ϕ is needed in order tocompute autoregression rank score statistics, in contrast with the aligned rankmethods (Puri and Sen and others).The (unknown) density f of u t is assumed to belong to the family F ofexponentially tailed densities, satisfying (2.3) and the following conditions onthe tails: (F1) f is positive and absolutely continuous, with a.e. derivative f ′ and finiteFisher information I ( f ) = R (cid:16) f ′ ( x ) f ( x ) (cid:17) f ( x )d x < ∞ ; moreover, there exists K f ≥ f has two bounded derivatives f ′ and f ′′ for all | x | > K f ; (F2) f is monotonically decreasing to 0 as x → ±∞ andlim x →−∞ − log F ( x ) b | x | r = lim x →∞ − log(1 − F ( x )) b | x | r = 1for some b > r ≥
1. 4ther properties of densities in F are summarized in [8].Moreover, we impose the following conditions on the regression matrix X ∗ n : (X1) The matrix A n = n − P nt =1 X ∗⊤ n X ∗ n is positive definite of order s for n ≥ n . (X2) n − P nt =1 k x nt k = O (1) as n → ∞ . (X3) lim n →∞ max ≤ t ≤ n (cid:8) n − x ⊤ nt A − n x nt (cid:9) = . Define D n = n − Y ⊤ n Y n and H n = Y ⊤ n ( Y ⊤ n Y n ) − Y n , b X ∗ n = H n X ∗ n (3.6)the projection matrix and the projection of X ∗ n on the space spanned by thecolumns of Y n , respectively. Moreover, let Q n = n − ( X ∗ n − b X ∗ n ) ⊤ ( X ∗ n − b X ∗ n ) . (3.7)The (random) matrices D n and Q n are of the respective orders ( p + 1) × ( p + 1)and s × s . We shall assume thatlim n →∞ D n = D , lim n →∞ IE ( Q n ) = Q (3.8)where D and Q are positive definite matrices.Choose a nondecreasing, square integrable score generating function J :(0 , → IR , such that J (1 − u ) = − J ( u ) , < u < J ′ ( u ) existsfor u ∈ (0 , α ) ∪ (1 − α ,
1) and, in this domain, satisfies the Chernoff-Savagecondition | J ′ ( u ) | ≤ c ( u (1 − u )) − − δ , < δ < . (3.9)Define the scores generated by J as b b n = (ˆ b n , . . . , ˆ b nn ) ⊤ withˆ b nt = − Z J ( u )dˆ a nt ( u ) , t = 1 , . . . , n. (3.10)The proposed tests of H are based on the linear autoregression rank statisticsof the form S n = n − ( X ∗ n − b X ∗ n ) ⊤ b b n , (3.11)and for testing H against K n we propose the criterion T n = S ⊤ n Q − n S n /A ( J ) (3.12)where A ( J ) = Z ( J ( t ) − ¯ J ) d t, ¯ J = Z J ( t )d t. (3.13)The typical choices of J are: 5i) Wilcoxon scores (optimal for f logistic) : J ( u ) = u − , < u < . Thescores are ˆ b n ; t = − R ( J ( u ) − ) d ˆ a t ( u ) = R J ( u ) du − while A ( J ) = and γ ( J, F ) = R f ( x ) dx. (ii) Normal (van der Waerden) scores (asymptotically optimal for f normal): J ( t ) = Φ − ( u ) , < u < , Φ being the d.f. of standard normal distribu-tion. Here A ( J ) = 1 and γ ( J, F ) = R f ( F − ( J ( x ))) dx .(iii) Median (sign) scores: J ( u ) = sign ( u − ) , < u < . Notice that the test statistic T n requires no estimation of nuisance parameters,since the functional A ( J ) depends only on the score function and not on (theunknown) F. We shall show that the asymptotic distribution of T n under H is central χ with s degrees of freedom, hence it is asymptotically distributionfree. Under K n it is noncentral χ with s degrees of freedom and noncentralityparameter dependent on J and F but not on the nuisance parameters. In thisway, it is asymptotically equivalent to the rank test of H in the situationwithout nuisance autoregression. X ∗ n and Y ∗ n satisfyconditions (3.6)–(3.8). We want to test the hypothesis H : β ∗ = ( β , ϕ unspecified )against the alternative K n : β ∗ = n − / β ∗ x ( β ∗ x ∈ IR s fixed ) . Let b a n ( α ) = (ˆ a n ( α ) , . . . , ˆ a nn ( α )) be the autoregression rank scores correspond-ing to the submodel under H , i.e. y t = β + ϕ + ϕ y t − + ϕ y t − + . . . + ϕ p y t − p + u t , t = 1 , . . . , n. Let J : (0 , IR be a nondecreasing and square integrable score-generatingfunction such that J (1 − u ) = − J ( u ) , < u <
1, satisfying (3.9). Define thescores b b n = (ˆ b n , . . . , ˆ b nn ) ⊤ by the relation (3.10). Consider the test statis-tics T n = S ⊤ n Q − n S n /A ( J ) defined in (3.11)–(3.13). The test is based on theasymptotic distribution of T n under H O , described in the following theorem. Theorem 4.1
Assume that the distribution F of the innovations u t satisfies(F1)–(F4) and the regression matrix X n satisfies (X1)–(X4). Let T n be gener-ated by the function J satisfying (3.9), nondecreasing and square integrable on (0 , . (i) Then, under H , the asymptotic distribution of T n is central χ with s degrees of freedom. ii) Under K n , the asymptotic distribution of T n is noncentral χ with s degreesof freedom and the noncentrality parameter η = β ⊤ x Q β x · γ ( J, F ) /A ( J ) where (4.1) γ ( J, F ) = − Z J ( v )d f ( F − ( v )) . Hence, the test rejects H on the significance level τ ∈ (0 ,
1) if T n > χ s (1 − τ )where χ s (1 − τ ) is the 100(1 − τ )%-quantile of the χ distribution with s degreesof freedom. The asymptotic distribution under K n also shows that the Pitmanefficiency of the test coincides with that of the classical rank test in the situationwithout the autoregressive errors. Proof. (i) It follows from [8], Theorem 3.3 and [7], Theorem 4.1, that under H , thelinear autoregression rank scores statistic admits the representation Q − / n S n = n − / Q − / n ( X ∗ n − X ∗ n ) ⊤ e b n + o p (1)as n → ∞ , where e b n = (˜ b n , . . . , ˜ b nn ) ⊤ and ˜ b nt = J ( F ( u nt )) , t = 1 , . . . , n. Then (i) follows from the central limit theorem.(ii) The same representation holds also under the sequence of alternatives K n , which is contiguous with respect to the sequence of null distributions withthe densities Q nt =1 f ( u t ) . References [1] Alpuim, T. & El-Shaarawi, A. (2008). On the efficiency of regression anal-ysis with AR(p) errors.
J. Applied Statistics
35, 717–737.[2] Brockwell, P.J. and R.A. Davis (1991).
Time Series: Theory and Methods ,2nd edition. Springer Verlag, New York.[3] El Bantli, F. and Hallin, M. (2001). Kolmogorov-Smirnov tests for AR mod-els based on autoregression rank scores.
Lecture Notes-Monograph Series,Vol. 37
Selected Proceedings of the Symposium on Inference for StochasticProcesses (2001), pp. 111-124.[4] G¨uney, Y., Tua¸c, Y., ¨Ozdemir, S¸. & Arslan, O. (2020a). Conditional max-imum L q -likelihood estimation for regression model with autoregressive er-ror terms. Metrika
39. 75] G¨uney, Y., Jureˇckov´a, J. and Arslan, O. (2020b). Averaged AutoregressionQuantiles in Autoregressive Model. In:
Analytical Methods in Statistics (M.Maciak et al. (Eds.)), Springer Proceedings in Mathematics and Statistics,Vol. 329, ISBN 978-3-030-48813-0.[6] Gutenbrunner, C. and J. Jureˇckov´a (1992). Regression rank scores andregression quantiles.
Ann. Statist. , 305-330.[7] Gutenbrunner, C., J. Jureˇckov´a, R. Koenker and S. Portnoy (1993). Testsof linear hypotheses based on regression rank scores. J. Nonpar. Statist. ,307-331.[8] Hallin, M. and Jureˇckov´a, J. (1991). Optimal tests for autoregressive mod-els based on autoregression rank scores. Ann. Statist.
27, 1385–1414.[9] Jureˇckov´a, J. (1991). Tests of Kolmogorov-Smirnov type based on regres-sion rank scores.
Trans. 11th Prague Conf. on Inform. Theory, Statist. De-cision Functions and Random Processes (J. ´A. V´ıˇsek, ed.), 41-49. Academia,Prague.[10] Koenker, R. and G. Bassett (1978). Regression quantiles.
Econometrica , 33-50.[11] Koenker, R. and Xiao, Z. (2004). Unit Root Quantile Autoregression In-ference. J. Amer. Statist. Assoc.
99, 775–787.[12] Koenker, R. and Xiao, Z. (2006). Quantile Autoregression.
J. Amer. Statist.Assoc.
Ann. Statist.
23, 670–689.[14] McKnight, S., McKean, J., Huitema, B. (2000). A double bootstrap methodto analyze linear models with autoregressive error terms.
PsychologicalMethods
5, 87–101[15] Tua¸c, Y., G¨uney, Y., Senoglu, B., Arslan, O. (2018). Robust parameterestimation of regression model with AR(p) error terms.
Communicationsin Statistics-Simulations and Computations
47, 2343–2359[16] Tua¸c, Y., G¨uney, Y., Arslan, O. (2020). Parameter estimation of regres-sion model with AR(p) error terms based on skew distributions with EMalgorithm.