Strong Law of Large Numbers for Functionals of Random Fields With Unboundedly Increasing Covariances
SSTRONG LAW OF LARGE NUMBERS FOR FUNCTIONALS OF RANDOM FIELDSWITH UNBOUNDEDLY INCREASING COVARIANCESIllia Donhauzer † Andriy Olenko ‡ Andrei Volodin § Key Words: strong law of large numbers, random field, integral functional, non-stationary,functional data, long-range dependence, non-central limit theorem.ABSTRACTThe paper proves the Strong Law of Large Numbers for integral functionals of randomfields with unboundedly increasing covariances. The case of functional data and increasingdomain asymptotics is studied. Conditions to guarantee that the Strong Law of Large Num-bers holds true are provided. The considered scenarios include wide classes of non-stationaryrandom fields. The discussion about application to weak and long-range dependent randomfields and numerical examples are given.
The recent evolution of technology and measuring tools provided a tremendous amount offunctional data and substantively motivated a development of new statistical models andmethods. Numerous classical results that originally were obtained for discretely sampleddata require extensions to new types of observations collected as functional curves, rasterimages or spatial data. These high-dimensional data structures often do not have such niceproperties as stationarity or uniform boundedness of their moment characteristics.The aim of this paper is to derive the Strong Law of Large Numbers (SLLN) for func-tional data in R d , d ≥ , under not restrictive assumptions on moments and dependenciesbetween observations. We consider realizations of random fields X ( s ) , s ∈ R d , d ≥ , (not † La Trobe University, Melbourne, Australia. email: [email protected] ‡ (cid:0) La Trobe University, Melbourne, Australia. email: [email protected] § University of Regina, Saskatchewan, Canada. email: [email protected] a r X i v : . [ m a t h . P R ] N ov ecessary homogeneous and isotropic) with weak restrictions on their covariance functions B ( s , s ) , s , s ∈ R d . Namely, the covariance functions B ( s , s ) can unboundedly increase,as || s || , || s || → ∞ , when the distance || s − s || between the locations s , s is preservedbounded. By this condition, the variances V ar ( X ( s )) are not necessary uniformly boundedon R d . Moreover, as we will see later, this condition allows to consider random fields withlong-range dependence and their nonlinear transformations.We are interested in the asymptotic behaviour of ξ ( µ ) = 1 µ d (cid:90) ∆( µ ) X ( s ) ds, when µ → ∞ , where ∆( µ ) is a homothetic transformation with the coefficient µ of a set ∆ ∈ R d whichplays a role of an observation window in statistical applications. For the case of identicallydistributed X ( s ) the statistic ξ ( µ ) is a classical estimator of the mean. This paper studies thecase of correlated observations and conditions on the behaviour of the covariance function B ( s , s ) that guarantee the convergence ξ ( µ ) a.s. −−→ , µ → ∞ . In Lyons (1988) SLLN was obtained for sequences of weakly dependent random variables { X ( n ) , n ≥ } such that V ar ( X ( n )) = O (1) . Other results on the SLLN can be found inM´oricz (1977), M´oricz (1985), Serfling (1970).The multidimensional versions of the results were obtained in Gaposhkin (1977) andParker and Rosalsky (2019). Parker and Rosalsky (2019) proved the SLLN for 2-dimensionalarrays { X ( n, m ) , n ≥ , m ≥ } of independent random elements with values in Banachspaces. In Gaposhkin (1977), the multidimensional SLLN for stationary random processesand homogeneous random fields was established. Necessary and sufficient conditions for theSLLN were found.Later Hu, Rosalsky, and Volodin (2005) proved the SLLN for sequences of random vari-ables { X ( n ) , n ≥ } with less restrictive conditions on moments and dependencies betweenobservations comparing with the results in Lyons (1988). More precisely, they studied thecase of V ar ( X ( n )) < H ( n ) , n ≥ , (cid:88) n =1 H ( n ϕ ) n < ∞ , where ϕ is a golden ratio, and used the dependency assumptionsup n | cov ( X ( n ) , X ( n + m )) | ≤ ρ ( m ) , ∞ (cid:88) m =1 ρ ( m ) m ϕ − < ∞ . From these conditions, one may see that there are sequences of random variables possessinglong-range dependence that satisfy SLLN.The results presented in Hu, Rosalsky, and Volodin (2005) were extended by the sameauthors to the case of more general normalization of partial sums in Hu, Rosalsky, andVolodin (2008). These results found numerous applications. We provide a few examplesthat illustrate areas that employed such results:- in Ashikhmin, Li, and Marzetta (2018), the results from Hu, Rosalsky, and Volodin (2005)were applied to an investigation of wireless massive multiple-input multiple-output systementails a large number of base station antennas serving a much smaller number of users, withlarge gains in spectral efficiency and energy efficiency;- in Li, Mukhopadhyay, and Dunson (2017), to data lying in a high dimensional ambientspace that commonly thought to have a much lower intrinsic dimension;- in Baron (2014), to a study of a Bayesian multichannel change-point detection problem ina general setting;- in Shu and Nan (2019), to the estimation of large covariance and precision matrices fromhigh-dimensional sub-Gaussian or heavier-tailed observations with slowly decaying temporaldependence;- in Kumar, Wenzel, Ellis, ElBsat, Drees, and Zavala (2019), to show that stochastic pro-gramming provides a framework to design hierarchical model predictive control schemes forperiodic systems;- in Pumi, Schaedler, and Souza (2020), to a class of dynamic models for time series takingvalues on the unit interval; 3 in Cousido-Rocha, de U˜na ´Alvarez, and Hart (2019), to a recurring theme in modernstatistics that is dealing with high-dimensional data whose main feature is a large numberof variables but a small sample size;- in Vega and Rey (2013), to a construction of adaptive algorithms that leads to the so calledStochastic Gradient algorithms;- in Hojjatinia, Lagoa, and Dabbene (2020), to introduce novel methodologies for the iden-tification of coefficients of switching autoregressive moving average with exogenous inputsystems and switched autoregressive exogenous linear models.The main novelties of the obtained in this paper results are in investigating– case of random fields with multidimensional observation windows;– case of functional data and their integral functionals;– weaker conditions on variances and dependencies then in the above publications;– weakly and strongly dependent random fields.Specifically, the conditions on the variances and covariance functions are relaxed. Thevariances are bounded by the functions H ( s ) which can increase faster than in Hu, Rosalsky,and Volodin (2005), as || s || → ∞ . The covariance functions can unboundedly increase, whenthe locations are getting far away from the origin, but a distance between them is bounded.We show that wide classes of long-range dependent random fields and their nonlinear trans-formations satisfy the SLLN. Moreover, observation windows ∆ can be taken from a wideclass of sets.This paper is organized as follows. Section 2 provides required definitions and notations.The main results of this article are proved in Section 3. Numerical studies confirming thetheoretical findings are given in Section 5. Conclusions and some open problems are presentedin Section 6. 4
Definitions and notations
The main definitions and notations about random fields are given in this section.In what follows we use the symbol C to denote constants which are not important forour discussion. Moreover, the same symbol C may be used for different constants appealingin the same proof. Let d = denote the equality of finite-dimensional distributions.For ν > − , we denote by J ν ( z ) = ∞ (cid:88) m =0 ( − m ( z/ m + ν m !Γ( m + ν + 1) , z ≥ , the Bessel’s function of the first kind of order ν , where z ≥ . Definition 1.
A random field X ( s ) , s ∈ R n , is called strictly homogeneous, if finite-dimensional distributions of Y ( s ) are invariant with respect to the group of motion trans-formations P ( X ( s ) < a , ..., X ( s k ) < a k ) = P ( X ( s + h ) < a , ..., X ( s k + h ) < a k )for all h ∈ R n , where s , .., s k ∈ R n and a , .., a k ∈ R . Definition 2.
A random field X ( s ) , s ∈ R n , is called isotropic, if its finite-dimensionaldistributions are invariant with respect to rotation transformations P ( X ( s ) < a , ..., X ( s k ) < a k ) = P ( X ( As ) < a , ..., X ( As k ) < a k )for all rotation transformations, i.e. orthogonal matrices A with the absolute value of deter-minant of A equals to 1, s , .., s k ∈ R n and a , .., a k ∈ R d . Let X ( s ) be a constant mean random field defined on R d , d ≥ . Without loss of generality,let EX ( s ) = 0 . The function B ( r ) , r ≥ , is a correlation function of an isotropic random field if andonly if there exists a measure G on { R + , B + } such that B ( r ) allows the following integralrepresentation B ( r ) = E (cid:0) X ( s ) X ( s ) (cid:1) = 2 n − Γ (cid:18) n (cid:19) (cid:90) R + J n − ( ur )( ur ) − n G ( du ) , r = || s − s || , || · || denotes the Euclidean distance in R d . Definition 3.
A measurable function L : (0 , ∞ ) → (0 , ∞ ) is called slowly varying at theinfinity if for all λ > t →∞ L ( λt ) L ( t ) = 1 . Definition 4.
The function H m ( u ) = ( − m e u / d m du m e − u is a Hermite polynomial of order m . The first few Hermite polynomials are H ( u ) = 1 , H ( u ) = u, H ( u ) = u − . It is known that the Hermite polynomials form a complete orthogonal system in the space L ( R , φ ( u ) du ), i.e. (cid:90) R H m ( u ) H m ( u ) φ ( u ) du = δ m m m ! , where δ m m is a Kronecker delta function. Definition 5.
A random field X ( s ) , s ∈ R d , is called long-range dependent if its covariancefunction B ( s , s ) = E ( X ( s ) X ( s )) is not absolutely integrable for each s ∈ R d , i.e. (cid:90) R d | B ( s , s + s ) | ds = + ∞ . In this section we introduce dependencies assumptions and prove the SLLN for random fields.
Assumption 1.
The absolute value of the covariance function of X ( s ) is bounded as | B ( s , s ) | = | cov ( X ( s ) , X ( s )) | ≤ C (1 + || s || γ + || s || γ ) ρ ( || s − s || ) , γ ≥ , where ρ ( u ) , u ∈ R + , is a such positive bounded function that for some β > ρ ( u ) ≤ /u β , u ≥ . X ( s ) is bounded by the function H ( s ) := C (1 + 2 || s || γ ) ρ (0) , which can unboundedly increase, when || s || → ∞ . Remark 1.
It follows from Assumption 1 that for any fixed s ∈ R d the covariance functionis bounded by C ( s ) || s || γ − β if || s || → + ∞ and (cid:90) R d B ( s , s ) ds ≤ C (cid:18) C ( s ) (cid:90) || s ||≥ || s || γ − β ds (cid:19) = C (cid:18) C ( s ) (cid:90) ∞ r d + γ − β − dr (cid:19) . Therefore, in the case β − γ ≥ d the random field X ( s ) is weakly dependent and in thecase β − γ < d the random field X ( s ) can have a non-integrable covariance function and belong-range dependent.We consider the random variables ξ ( µ ) = 1 µ d (cid:90) ∆( µ ) X ( s ) ds, µ > , (3.1)where ∆( µ ) is a homothetic transformation with the parameter µ of a simply connected d -dimensional set ∆ ⊂ R d , which is a compact set containing the origin and the Lebesguemeasure | ∆ | >
0. The integral in (3.1) is well-defined because of the measurability of X ( s ),see Theorem 1.1.1. in Ivanov and Leonenko (2012).We will use the notation diam(A) for the diameter of the set A ⊂ R d , i.e. diam(A) =sup x , y ∈ A || x − y || . To establish conditions for ξ ( µ ) a.s. −−→ , as µ → ∞ , we use the classical method ofsubsequences. By this method, the existance of the increasing subsequence { µ n , n ≥ } suchthat ξ ( µ n ) a.s. −−→ , as n → ∞ , and the convergence of the deviations sup µ ∈ [ µ n ,µ n +1 ) | ξ ( µ ) − ξ ( µ n ) | a.s. −−→ , as n → ∞ , are enough for the convergence ξ ( µ ) a.s. −−→ , as µ → ∞ . Lemma 1.
Let Assumption 1 be satisfied and there exist an increasing sequence of positivenumbers { µ n , n ≥ } such one of the following conditions holds7i) for β < d ∞ (cid:88) n =1 µ β − γn < + ∞ , (3.2)(ii) for β ≥ d ∞ (cid:88) n =1 µ d − γn < + ∞ . (3.3)Then, the sequence of random variables ξ ( µ n ) a.s. −−→ , as n → ∞ . Proof.
Note that as E ( X ( s )) = 0 , then the variances of ξ ( µ ) can be estimated as V ar ( ξ ( µ )) = 1 µ d (cid:90) ∆( µ ) (cid:90) ∆( µ ) cov ( X ( s ) , X ( s )) ds ds ≤ Cµ d (cid:90) ∆( µ ) (cid:90) ∆( µ ) (1 + || s || γ + || s || γ ) ρ ( || s − s || ) ds ds ≤ Cµ d (cid:90) ∆( µ ) (cid:90) ∆( µ ) (1 + || s || γ ) ρ ( || s − s || ) ds ds + Cµ d (cid:90) ∆( µ ) (cid:90) ∆( µ ) (1 + || s || γ ) ρ ( || s − s || ) ds ds = 2 Cµ d (cid:90) ∆( µ ) (cid:90) ∆( µ ) (1 + || s || γ ) ρ ( || s − s || ) ds ds . After the change of the variables ˜ s = s , ˜ s = s − s one gets V ar ( ξ ( µ )) = 2 Cµ d (cid:90) ∆( µ ) (cid:90) ∆( µ ) − ∆( µ ) (1 + || ˜ s || γ ) ρ ( || ˜ s || ) d ˜ s ˜ ds , where ∆( µ ) − ∆( µ ) := { s − ˜ s : s, ˜ s ∈ ∆( µ ) } denotes the Minkowski difference of sets.The set ∆( µ ) is bounded, so there exists a centered ball B ( µ · diam(∆)) = { s ∈ R n : || s || ≤ µ · diam(∆) } such that ∆( µ ) − ∆( µ ) ⊂ B ( µ · diam(∆)) . Then, by converting theintegrals to the spherical coordinates, it follows from Assumption 1 that
V ar ( ξ ( µ )) ≤ Cµ d (cid:90) ∆( µ ) (1 + || ˜ s || γ ) d ˜ s (cid:90) B ( µ · diam(∆)) ρ ( || ˜ s || ) ˜ ds ≤ Cµ d µ · diam∆(1) (cid:90) ˜ r d − (1 + ˜ r γ ) d ˜ r µ · diam(∆) (cid:90) ˜ r d − ρ (˜ r ) d ˜ r Cµ d (cid:0) µ d + µ d + γ (cid:1)(cid:18) C + µ · diam(∆) (cid:90) ˜ r d − ρ (˜ r ) d ˜ r (cid:19) ≤ Cµ d (cid:0) µ d + µ d + γ (cid:1)(cid:0) C + µ d − β (cid:1) . Now the Borel–Cantelli lemma is used to find conditions for ξ ( µ n ) a.s. −−→ . By Chebyshev’s inequality one gets ∞ (cid:88) n =1 P ( | ξ ( µ n ) | > ε ) ≤ ε ∞ (cid:88) n =1 V ar ( ξ ( µ n )) ≤ C ∞ (cid:88) n =0 (cid:0) µ dn + µ d + γn (cid:1)(cid:0) C + µ d − βn (cid:1) µ dn . Let β < d, then (cid:0) µ dn + µ d + γn (cid:1)(cid:0) C + µ d − βn (cid:1) µ dn ≤ Cµ β − γn , n → ∞ . Thus, for β < d the sequence ξ ( µ n ) a.s. −−→ , when n → ∞ , if (cid:80) ∞ n =1 µ γ − βn < + ∞ . Let β ≥ d. Then (cid:0) µ dn + µ d + γn (cid:1)(cid:0) C + µ d − βn (cid:1) µ dn ≤ Cµ d − γn . Thus, for β ≥ d the sequence ξ ( µ n ) a.s. −−→ , when n → ∞ , if ∞ (cid:88) n =1 µ d − γn < + ∞ . (cid:4) Lemma 2.
Let Assumption 1 be satisfied. If there exists an increasing sequence of positivenumbers { µ n , n ≥ } such that ∞ (cid:88) n =1 ( µ dn +1 − µ dn ) µ γn µ dn < + ∞ (3.4)and ∞ (cid:88) n =1 (cid:18) µ dn +1 − µ dn (cid:19) µ d + γn +1 < + ∞ , (3.5)then the sequence of random variables η ( µ n ) := sup µ ∈ [ µ n ,µ n +1 ) (cid:12)(cid:12) ξ ( µ ) − ξ ( µ n ) (cid:12)(cid:12) a.s. −−→ , n → ∞ . Proof.
It is enough to show that the random variables η ( µ n ) are bounded by a sequence ofrandom variables converging a.s. to 0, when n → ∞ .9he random variables η ( µ n ) allow the following estimation from abovesup µ ∈ [ µ n ,µ n +1 ) (cid:12)(cid:12) ξ ( µ ) − ξ ( µ n ) (cid:12)(cid:12) = sup µ ∈ [ µ n ,µ n +1 ) (cid:12)(cid:12)(cid:12)(cid:12) µ d (cid:90) ∆( µ ) X ( s ) ds − µ dn (cid:90) ∆( µ n ) X ( s ) ds (cid:12)(cid:12)(cid:12)(cid:12) ≤ sup µ ∈ [ µ n ,µ n +1 ) (cid:12)(cid:12)(cid:12)(cid:12) µ dn (cid:90) ∆( µ ) \ ∆( µ n ) X ( s ) ds (cid:12)(cid:12)(cid:12)(cid:12) + sup µ ∈ [ µ n ,µ n +1 ) (cid:12)(cid:12)(cid:12)(cid:12)(cid:18) µ d − µ dn (cid:19) (cid:90) ∆( µ ) X ( s ) ds (cid:12)(cid:12)(cid:12)(cid:12) . The above supremums can be estimated assup µ ∈ [ µ n ,µ n +1 ) (cid:12)(cid:12)(cid:12)(cid:12) µ dn (cid:90) ∆( µ ) \ ∆( µ n ) X ( s ) ds (cid:12)(cid:12)(cid:12)(cid:12) ≤ µ dn (cid:90) ∆( µ n +1 ) \ ∆( µ n ) | X ( s ) | ds := I (1) µ n , sup µ ∈ [ µ n ,µ n +1 ) (cid:12)(cid:12)(cid:12)(cid:12)(cid:18) µ d − µ dn (cid:19) (cid:90) ∆( µ ) X ( s ) ds (cid:12)(cid:12)(cid:12)(cid:12) ≤ (cid:18) µ dn − µ dn +1 (cid:19) (cid:90) ∆( µ n +1 ) | X ( s ) | ds := I (2) µ n . The next step is to find conditions that guarantee that I (1) µ n a.s. −−→ I (2) µ n a.s. −−→ , when n → ∞ . Using Markov’s inequality, the following series can be estimated as ∞ (cid:88) n =1 P ( I (1) µ n > ε ) = ∞ (cid:88) n =1 P (cid:18) µ dn (cid:90) ∆( µ n +1 ) \ ∆( µ n ) | X ( s ) | ds > ε (cid:19) ≤ ε ∞ (cid:88) n =1 µ dn E (cid:18) (cid:90) ∆( µ n +1 ) \ ∆( µ n ) | X ( s ) | ds (cid:19) = 1 ε ∞ (cid:88) n =1 µ dn (cid:90) (cid:90)(cid:0) ∆( µ n +1 ) \ ∆( µ n ) (cid:1) E | X ( s ) || X ( s ) | ds ds , where (cid:82)(cid:82) A denotes the double integral (cid:82) A (cid:82) A . By using H¨older’s inequality for p = q = 2 one gets ∞ (cid:88) n =1 P ( I (1) µ n > ε ) ≤ ε ∞ (cid:88) n =1 µ dn (cid:90) (cid:90)(cid:0) ∆( µ n +1 ) \ ∆( µ n ) (cid:1) (cid:18) EX ( s ) EX ( s ) (cid:19) / ds ds = 1 ε ∞ (cid:88) n =1 µ dn (cid:18) (cid:90) ∆( µ n +1 ) \ ∆( µ n ) (cid:0) EX ( s ) (cid:1) / ds (cid:19) ≤ | ∆(1) | ε ∞ (cid:88) n =1 ( µ dn +1 − µ dn ) sup s ∈ ∆( µ n +1 ) \ ∆( µ n ) H ( s ) µ dn ≤ Cε ∞ (cid:88) n =1 ( µ dn +1 − µ dn ) (1 + Cµ γn +1 ) µ dn . I (1) µ n a.s. −−→
0, when n → ∞ , if ∞ (cid:88) n =1 ( µ dn +1 − µ dn ) µ γn +1 µ dn < + ∞ . Now we derive conditions for the convergence I (2) µ n a.s. −−→ , when n → ∞ . Using Markov’s and H¨older’s inequalities one obtains ∞ (cid:88) n =1 P ( I (2) µ n > ε ) = ∞ (cid:88) n =1 P (cid:18)(cid:18) µ dn − µ dn +1 (cid:19) (cid:90) ∆( µ n +1 ) | X ( s ) | ds > ε (cid:19) ≤ ε ∞ (cid:88) n =1 (cid:18) µ dn +1 − µ dn (cid:19) × E (cid:18) (cid:90) ∆( µ n +1 ) | X ( s ) | ds (cid:19) = 1 ε ∞ (cid:88) n =1 (cid:18) µ dn +1 − µ dn (cid:19) (cid:90) (cid:90)(cid:0) ∆( µ n +1 ) (cid:1) E | X ( s ) || X ( s ) | ds ds ≤ ε ∞ (cid:88) n =1 (cid:18) µ dn +1 − µ dn (cid:19) (cid:18) (cid:90) ∆( µ n +1 ) (cid:18) EX ( s ) (cid:19) / ds (cid:19) ≤ Cε ∞ (cid:88) n =1 (cid:18) µ dn +1 − µ dn (cid:19) (1+ Cµ γn +1 ) µ dn +1 . Hence, by the Borel-Cantelli Lemma, the sequence I (2) µ n a.s. −−→
0, when n → ∞ , if ∞ (cid:88) n =1 (cid:18) µ dn +1 − µ dn (cid:19) µ γ +2 dn +1 < + ∞ . From the positiveness of η ( µ n ) = sup µ ∈ [ µ n ,µ n +1 ) (cid:12)(cid:12) ξ ( µ ) − ξ ( µ n ) (cid:12)(cid:12) and the boundedness η ( µ n ) ≤ I (1) µ n + I (2) µ n such that I (1) µ n a.s. −−→ I (2) µ n a.s. −−→ µ ∈ [ µ n ,µ n +1 ) (cid:12)(cid:12) ξ ( µ ) − ξ ( µ n ) (cid:12)(cid:12) a.s. −−→ , n → ∞ . (cid:4) Theorem 1.
If for β > γ > { µ n , n ≥ } satisfying theassumptions in Lemmas 1 and 2, then ξ ( µ ) a.s. −−→ , as µ → ∞ . Proof. As { µ n , n ≥ } ⊂ R + is an increasing sequence and µ n → ∞ , n → ∞ , then for each µ ∈ R + there exists µ n such that µ ∈ [ µ n , µ n +1 ) . It follows from ξ ( µ ) = ξ ( µ n ) + ξ ( µ ) − ξ ( µ n )that ξ ( µ n ) − sup µ ∈ [ µ n ,µ n +1 ) (cid:12)(cid:12)(cid:12)(cid:12) ξ ( µ ) − ξ ( µ n ) (cid:12)(cid:12)(cid:12)(cid:12) ≤ ξ ( µ ) ≤ ξ ( µ n ) + sup µ ∈ [ µ n ,µ n +1 ) (cid:12)(cid:12)(cid:12)(cid:12) ξ ( µ ) − ξ ( µ n ) (cid:12)(cid:12)(cid:12)(cid:12) . Thus, by Lemma 1 and by Lemma 2 we get ξ ( µ ) a.s. −−→ , as µ → ∞ . (cid:4) heorem 2. Let Assumption 1 be satisfied. The SLLN holds true, if one of the followingconditions is satisfied(i) β ∈ (0 , d ) and 2 γ < β, (ii) β ≥ d and 2 γ < d. Proof.
Let µ n = n α . Consider the first case and check the conditions of Lemmas 1 and 2.The condition (3.2) becomes ∞ (cid:88) n =1 n α ( β − γ ) < ∞ , and is satisfied if α > β − γ . The conditions (3.4) and (3.5) hold if αγ < . Indeed, from the asymptotic behavior ofthe terms in (3.4) and (3.5)( µ dn +1 − µ dn ) µ γn µ dn = (( n + 1) αd − n αd ) n αγ n αd ∼ (cid:0) ( n + 1) αd − (cid:1) n αγ n αd = 1 n − αγ and (cid:18) µ dn +1 − µ dn (cid:19) µ γ +2 dn +1 = (cid:18) n + 1) αd − n αd (cid:19) ( n + 1) αγ +2 αd ∼ ( n + 1) αγ +2 αd ( n + 1) αd +2 = 1( n + 1) − αγ . Thus, the conditions of Lemmas 1 and 2 are satisfied if 0 < β < d, α > β − γ and αγ < . Then, it follows from β − γ < α < γ that the required α exists if 2 γ < β. Using the same approach, we derive that for β ≥ d the conditions of Lemmas 1 and 2are satisfied if β ≥ d and d − γ < α < γ . Thus, the required α exists if 2 γ < d. (cid:4) Remark 2.
The upper bound in Assumption 1 can be replaced by another one that guaran-tees for || s || , || s || → + ∞ the covariance cov ( X ( s ) , X ( s )) sufficiently fast decays to zerofor s and s which are getting further away from each other, and it increases not too fastfor s and s that are close. For instance, one can use the conditions | B ( s , s ) | ≤ C (1 + || s || + || s || ) γ ρ ( || s − s || )or | B ( s , s ) | ≤ C (1 + || s || γ )(1 + || s || γ ) ρ ( || s − s || ) . emark 3. Homogeneous isotropic random fields satisfy Assumption 1 with γ = 0 , if theircovariance functions have hyberbolic bounded decays of order β. As SLLN holds for homogeneous isotropic random fields with hyperbolically bounded co-variance functions, it would be interesting to provide a simple example of non-homogeneousand non-isotropic random field for which the result holds true.
Example.
Let X ( s ) = g ( s ) H k ( Z ( s )) , s ∈ R d , d ≥ , where g ( · ) is a deterministic function, H k ( · ) , k ∈ N , is the Hermite polynomial of degree k and Z ( · ) is a homogeneous isotropicGaussian random field with EZ ( s ) = 0 and the covariance function B ( s ) , such that B (0) = 1and B Z ( s ) = E (cid:0) Z ( s ) Z (0) (cid:1) = L ( || s || ) || s || β , β > , where L ( s ) is a slowly varying function.By properties of the Hermite polynomials of Gaussian random variables, see, for example,(2 . .
8) in Ivanov and Leonenko (2012) EX ( s ) = g ( s ) EH k ( Z ( s )) = 0 ,B ( s , s ) = g ( s ) g ( s ) E (cid:0) H k ( Z ( s )) H k ( Z ( s )) (cid:1) = g ( s ) g ( s ) k ! B k ( || s − s || ) . By properties of slowly varying functions, see Proposition 1 . . v ) in Bingham, Goldie,and Teugels (1989), for any β > kβ there is a constant C such that B kZ ( || s || ) ≤ C || s || β , || s || ≥ . Thus, if | g ( s ) g ( s ) | ≤ C (1 + || s || γ + || s || γ ) (4.1)and kβ < β, then Assumption 1 holds true and by Theorem 21 µ d (cid:90) ∆( µ ) g ( s ) H k ( Z ( s )) ds a.s. −−→ , µ → + ∞ . | g ( s ) | ≤ C (1 + || s || γ ) , γ > , (4.2)that | g ( s ) g ( s ) | ≤ C (1 + || s || γ + || s || γ ) . Thus, if (4.2) holds, then (4.1) is true with γ = 2 γ . Some example of functions g ( · ) satisfying (4.1) are(i) g ( s ) ≡ C > . This case corresponds to the classical equally-weighted average func-tionals of homogeneous isotropic process or field;(ii) g ( s ) = (cid:81) di =1 s l i i , where s = ( s , .., s d ) , l i > , i = 1 , .., d. Note that | g ( s ) | = d (cid:89) i =1 | s i | l i ≤ || s || (cid:80) di =1 l i and (4.1) is satisfied with γ = 2 (cid:80) di =1 l i ;(iii) g ( s ) = (cid:81) di =1 s i ln( q i + | s i | ) , where s = ( s , ..., s d ) and q i > , i = 1 , .., d. By using the logarithm inequality ln( x ) ≤ x − , one obtains that | g ( s ) | ≤ C d (cid:89) i =1 | s i | + d (cid:89) i =1 | s i | and the upper bound follows from the estimate in (ii) and (4.2).The weight functions in (ii) and (iii) are often used in non-linear regression and M estimators applications.It follows from results in Alodat and Olenko (2020); Ivanov and Leonenko (2012) that forthe field X ( s ) in the examples above one can obtain not only SLLN, but also limit theoremsabout the convergence of distributions. Namely, the following result holds true.14 heorem 3. Alodat and Olenko (2020) Let a function g ( s ) , s ∈ R d , satisfy the condition µ d − β k g ( µ · d ) L k ( µ ) → ∞ , when µ → ∞ , and there exists a function g ∗ ( · ) such thatlim µ →∞ (cid:12)(cid:12)(cid:12)(cid:12) g ( µs ) g ( µ d ) − g ∗ ( s ) (cid:12)(cid:12)(cid:12)(cid:12) → s ∈ ∆(1 + ε ) for some ε > , (cid:90) (cid:90)(cid:0) ∆(1+ ε ) (cid:1) | g ∗ ( s ) g ∗ ( s ) ||| s − s || β κ ds ds < + ∞ , (cid:90) R dk k (cid:89) j =1 || λ j || β − d | K ∆ ( λ j , g ∗ ) | k (cid:89) j =1 dλ j < + ∞ , andlim µ →∞ (cid:90) R dk (cid:12)(cid:12)(cid:12)(cid:12) (cid:90) ∆ e i ( λ + .. + λ k ,s ) (cid:18) g ( µ || s || ) g ( µ · d || s || ) k (cid:89) j =1 (cid:115) L ( µ/ || λ j || ) L ( µ ) − g ∗ ( s ) (cid:19) ds (cid:12)(cid:12)(cid:12)(cid:12) k (cid:89) j =1 || λ β − d || k (cid:89) j =1 dλ j = 0 . Then, for β ∈ (cid:0) , min (cid:0) dk , d +12 (cid:1)(cid:1) the random variables1 µ d − β k/ g ( µ · d ) L k/ ( µ ) c k/ ( d, β ) (cid:90) ∆( µ ) g ( s ) H k ( Z ( s )) ds converge weakly to the random variable ξ ∗ := (cid:90) (cid:48) R dk K ∆ ( λ + .. + λ k , g ∗ ) (cid:81) kj =1 W ( dλ j ) (cid:81) kj =1 || λ j || ( d − β ) / , where W ( · ) is the complex Gaussian white noise random measure on R d , (cid:82) (cid:48) R d denotes the mul-tiple Wiener-Itˆo integral, where the diagonal hyperplanes λ i = ± λ j , i, j = 1 , .., k, i (cid:54) = j, areexcluded from the domain of integration, 1 d = (1 , .., ∈ R d , K ∆ ( λ, g ∗ ) = (cid:82) ∆ e i ( λ,s ) g ∗ ( s ) ds,c ( d, β ) = Γ(( d − β ) / / β π d/ Γ( β / . Remark 4.
For the three functions g ( · ) introduced in the Example of it is easy to see that(i) g ∗ ( s ) ≡ C, (ii) g ∗ ( s ) = (cid:81) di =1 s l i i , (iii) g ∗ ( s ) = (cid:81) di =1 s i . emark 5. For β ∈ (0 , min( dk , d +12 )) the random field Z ( s ) is long-range dependent andthe limit ξ ∗ has a non-Gaussian distribution if k ≥ . Remark 6.
If the random field X ( s ) is weak-dependent, one can derive the Central LimitTheorem for the integral functionals of the form (3.1), see, for example, Theorems 1.7.1-1.7.3in Ivanov and Leonenko (2012). In this section, we provide a numerical example confirming the obtained theoretical results.By simulations of random fields, we show that for the function g ( · ) satisfying (ii) in the Ex-ample in Section 4 the integral functional in (3.1) converges to 0, as µ → ∞ . A reproducibleversion of the code in this paper is available in the folder ”Research materials” from thewebsite https://sites.google.com/site/olenkoandriy/ .We consider d = 2 , the random variables in (3.1) and the random field X(s) given by theformula X ( s ) = (cid:89) k =1 | s k | γ H ( Z ( s )) , where H ( x ) = x − Z ( s ) , s = ( s , s ) ∈ R , is ahomogeneous isotropic Gaussian random field with the Cauchy type covariance function B Z ( r ) = 1(1 + r ) β , r ≥ . The observation window ∆(1) = (cid:3) (1) := { ( s , s ) : | s | ≤ , | s | ≤ } is a square.For the simulations we used the values of β = 0 . γ = 0 . . Theorems 2 and 3 holdtrue for these values.As the simulations of random fields can be done only on a discrete grid, we used thedense grid of points { ( ih, jh ) : i, j = − N, − N + 1 , ..., N − , N } , N ∈ N , where h is a smallfixed step. The integrals in (3.1) were approximated by the Riemann’s sums (cid:90) ∆( µ ) X ( s ) ds ≈ N (cid:88) i = − N N (cid:88) j = − N X ( ih, jh ) h = N (cid:88) i = − N N (cid:88) j = − N | ih | . | jh | . H ( Z ( ih, jh )) h . X ( s ) in the square region (cid:3) (300) = { ( s , s ) : | s i | ≤ , i = 1 , } were generated. A realization of the random fields X ( s ) in the square (cid:3) (300) on the 2D grid with the step h = 0 .
25 and the corresponding values of ξ ( µ ) for µ = 10 , , , ...,
300 are given in Figure 1. The Q-Q plot of the simulated values of ξ (300)is shown in Figure 2a. As µ = 300 is sufficiently large the distribution is close to theasymptotic one. As expected, it is not Gaussian. (a) Realization of X ( s ) (b) Realization of ξ ( µ ) Figure 1: Realizations of the random field and its integral functionalUsing the obtained realizations of X ( s ) , the random variables ξ ( µ ) were computed for µ = 10 , , , .., . The box plots of the simulated values of ξ ( µ ) are given in Figure 2b.Table 1 shows the corresponding Root Mean Square Error (RMSE) of ξ ( µ ) for different valuesof µ. Figure 2b and the table confirm the convergence of ξ ( µ ) to zero when µ increases. µ
10 50 100 150 200 250 300RMSE 0.217 0.106 0.079 0.068 0.057 0.052 0.048Table 1: RMSE of ξ ( µ ).17 a) QQplot of ξ (300) (b) Boxplots of ξ ( µ ) Figure 2: Empirical distributions of ξ ( µ ) The SLLN for random fields with unboundedly increasing variances and covariance functionswas obtained. The conditions of the obtained results allow to consider the case of nonlineartransformations of long-range dependent random fields. The results were derived for a verygeneral class of simply connected observation windows ∆ . In the future studies, it would be interesting to obtain:- Laws of Large Numbers with the complete convergence, see Hu, Rosalsky, and Volodin(2012), for multidimensional functional data;- Necessary and sufficient conditions for the SLLN for non-homogeneous and non-isotropicrandom fields Gaposhkin (1977);- Rate of convergence in the SLLN, see Anh, Leonenko, Olenko, and Vaskovych (2019);Hu and Sun (2020). 18 cknowledgements
The authors would like to thank the anonymous reviewers for their suggestions that helpedto improve the style of the paper.
Funding
This research was supported under La Trobe University SEMS CaRE Grant ”Asymptoticanalysis for point and interval estimation in some statistical models”. The research of thelast listed author was partially funded by the subsidy allocated to Kazan Federal Universityfor the state assignment in the sphere of scientific activities, project 1.13556.2019/ 13.1.
References
Alodat, T. and A. Olenko (2020). On asymptotics of discretized functionals of long-rangedependent functional data.
To appear in Communications in Statistics - Theory andMethods .Anh, V., N. Leonenko, A. Olenko, and V. Vaskovych (2019). On rate of convergence innon-central limit theorems.
Bernoulli 25 (4A), 2920–2948.Ashikhmin, A., L. Li, and T. Marzetta (2018). Interference reduction in multi-cell massiveMIMO systems with large-scale fading precoding.
IEEE Transactions on InformationTheory 64 (9), 6340–6361.Baron, M. (2014). Asymptotically pointwise optimal change detection in multiple channels.
Sequential Analysis 33 (4), 440–457.Bingham, N., C. Goldie, and J. Teugels (1989).
Regular variation . Cambridge UniversityPress. 19ousido-Rocha, M., J. de U˜na ´Alvarez, and J. Hart (2019). A two-sample test for the equalityof univariate marginal distributions for high-dimensional data.
Journal of MultivariateAnalysis 174 , 104537.Gaposhkin, V. (1977). Criteria for the strong law of large numbers for classes of stationaryand homogeneous random fields.
Theory of Probability and its Applications 22 (2), 286–310.Hojjatinia, S., C. Lagoa, and F. Dabbene (2020). Identification of switched autoregressiveexogenous systems from large noisy datasets.
To appear in International Journal of Robustand Nonlinear Control .Hu, T., A. Rosalsky, and A. Volodin (2005). On the golden ratio, strong law, and firstpassage problem.
Mathematical Scientist 30 (2), 77–86.Hu, T., A. Rosalsky, and A. Volodin (2008). On convergence properties of sums of dependentrandom variables under second moment and covariance restrictions.
Statistics & ProbabilityLetters 78 (14), 1999–2005.Hu, T., A. Rosalsky, and A. Volodin (2012). A complete convergence theorem for rowsums from arrays of rowwise independent random elements in Rademacher type p Banachspaces.
Stochastic Analysis and Applications 30 (2), 343–353.Hu, Z. and W. Sun (2020). Convergence rates in the law of large numbers and new kindsof convergence of random variables.
To appear in Communications in Statistics - Theoryand Methods .Ivanov, A. and N. Leonenko (2012).
Statistical analysis of random fields . Springer.Kumar, R., M. Wenzel, M. Ellis, M. ElBsat, K. Drees, and V. Zavala (2019). HierarchicalMPC schemes for periodic systems using stochastic programming.
Automatica Journal ofIFAC 107 , 306–316. 20i, D., M. Mukhopadhyay, and D. Dunson (2017). Efficient manifold and subspace approxi-mations with spherelets. arXiv preprint arXiv:1706.08263 .Lyons, R. (1988). Strong laws of large numbers for weakly correlated random variables.
Michigan Mathematical Journal 35 (3), 353–359.M´oricz, F. (1977). The strong laws of large numbers for quasi-stationary sequences.
Zeitschrift f¨ur Wahrscheinlichkeitstheorie und Verwandte Gebiete 38 (3), 223–236.M´oricz, F. (1985). Slln and convergence rates for nearly orthogonal sequences of randomvariables.
Proceedings of the American Mathematical Society 95 (2), 287–294.Parker, R. and A. Rosalsky (2019). Strong laws of large numbers for double sums of Banachspace valued random elements.
Acta Mathematica Sinica(English Series) 35 (5), 583–596.Pumi, G., P. Schaedler, and R. Souza (2020). A dynamic model for double bounded timeseries with chaotic driven conditional averages.
To appear in Scandinavian Journal ofStatistics .Serfling, R. (1970). Convergence properties of S n under moment restrictions. The Annals ofMathematical Statistics 41 (4), 1235–1248.Shu, H. and B. Nan (2019). Estimation of large covariance and precision matrices fromtemporally dependent observations.
The Annals of Statistics 47 (3), 1321–1350.Vega, L. and H. Rey (2013).