aa r X i v : . [ s t a t . O T ] A p r On R´enyi entropy convergence of the max domain of attraction
Ali Saeb Theoretical Statistics and Mathematics Unit,Indian Statistical Institute, Delhi Center,7 S.J.S Sansanwal Marg, New Delhi 110016, India
Abstract:
In this paper, we prove that the R´enyi entropy of linearly normalized partialmaxima of independent and identically distributed random variables is convergent to thecorresponding limit R´enyi entropy when the linearly normalized partial maxima convergesto some nondegenerate random variable.
Keywords:
R´enyi entropy, Max stable laws, Max domain of attraction.
MSC 2010 classification: Corresponding author: [email protected] Introduction
The limit laws of linearly normalized partial maxima M n = max( X , · · · , X n ) of inde-pendent and identically distributed (iid) random variables (rvs) X , X , . . . , with commondistribution function (df) F, namely,lim n →∞ Pr( M n ≤ a n x + b n ) = lim n →∞ F n ( a n x + b n ) = G ( x ) , x ∈ C ( G ) , (1.1)where, a n > , b n ∈ R , are norming constants, G is a non-degenerate distribution function, C ( G ) is the set of all continuity points of G, are called max stable laws. If, for some non-degenerate distribution function G, a distribution function F satisfies (1.1) for some normingconstants a n > , b n ∈ R , then we say that F belongs to the max domain of attraction of G under linear normalization and denote it by F ∈ D ( G ) . Limit distribution functions G satisfying (1.1) are the well known extreme value types of distributions, or max stable laws,namely, the Fr´echet law: Φ α ( x ) = (cid:26) , x < , exp( − x − α ) , ≤ x ;the Weibull law: Ψ α ( x ) = (cid:26) exp( −| x | α ) , x < , , ≤ x ;and the Gumbel law: Λ( x ) = exp( − exp( − x )); x ∈ R ; α > φ α ( x ) = (cid:26) , x ≤ ,αx − ( α +1) e − x − α , < x ;the Weibull density: ψ α ( x ) = (cid:26) α | x | α − e −| x | α , x < , , ≤ x ;and the Gumbel density: λ ( x ) = e − x e − e − x , x ∈ R . Note that (1.1) is equivalent tolim n →∞ n (1 − F ( a n x + b n )) = − log G ( x ) , x ∈ { y : G ( y ) > } . We shall denote the left extremity of distribution function F by l ( F ) = inf { x : F ( x ) > } ≥−∞ and the right extremity of F by r ( F ) = sup { x : F ( x ) < } ≤ ∞ . Criteria for F ∈ D ( G )are well known (see, for example, Galambos, 1987; Resnick, 1987; Embrechts et al., 1997).The Shannon entropy of a continuous rv X with density function f ( x ) is defined as H ( X ) = − Z A f ( x ) log f ( x ) dx, where A = { x ∈ R : f ( x ) > } . R´enyi entropy is a generalization of Shannon entropy (R´enyi, 1961). It is one of a family offunctional for quantifying the diversity, uncertainty or randomness of a system. The R´enyientropy of order β is defined as H β ( X ) = 11 − β log (cid:18)Z A ( f ( x )) β dx (cid:19) , (1.2)where, 0 < β < ∞ , β = 1 . By L’Hopital’s rule, the R´enyi entropy tends to Shannon entropy,as β → . The R´enyi entropy are important in ecology and statistics as indices of diversity.R´enyi entropy appear also in several important contexts such as those of information theory,statistical estimation.The idea of tracking the central limit theorem using Shannon entropy goes back to Linnik(1959) and Shimizu (1975), who used it to give a particular proof of the central limit theorem.Brown (1982), Barron (1986) and Takano (1987) discuss the central limit theorem withconvergence in the sense of Shannon entropy and relative entropy. Artstein et al.(2004)and Johnson and Barron (2004) obtained the rate of convergence under some conditions on the density. Johnson (2006) is a good reference to the application of information theoryto limit theorems, especially the central limit theorem. Cui and Ding (2010) show thatthe convergence of the R´enyi entropy of the normalized sums of iid rvs and obtain thecorresponding rates of convergence. Saeb (2014) study the rate of convergence of R´enyientropy for the max domain of attraction.In this article, our main interest is to investigate conditions under which the R´enyi entropyof the normalized partial maxima of iid rvs converges to the corresponding limit R´enyientropy. In the other hand, our problem of interest to see if normalized partial maximaconverges to a nondegenerate rv, does the R´enyi entropy of the normalized partial maximaconverges to the R´enyi entropy of the limit rv? In the next section we give our main results,followed by a section on Proofs. The R´enyi entropies of the extreme value distributions aregiven in the appendix A with proof and appendix B containing results used in this article.2.
Main Result
Theorem 2.1.
Suppose F ∈ D ( G ) is absolutely continuous with pdf f which is eventuallypositive and decreasing in left neighbourhood of r ( F ) . If R r ( F ) −∞ ( f ( x )) β dx < ∞ for β > , and(a) G = Φ α and r ( F ) = ∞ , then lim n →∞ H β ( g n ) = H β ( φ α );(b) G = Ψ α and r ( F ) < ∞ , then lim n →∞ H β ( g n ) = H β ( ψ α );(c) G = Λ and r ( F ) ≤ ∞ , then lim n →∞ H β ( g n ) = H β ( λ ) . Remark 2.1.
Lemma A.2 show that, the R´enyi entropy do not depend on the location andscale parameters.The proof of the above theorem is different from the proof for the R´enyi entropy of thenormalized sums of iid rvs. In our proofs, the properties of normalized partial maxima suchas, von Mises condition and density convergence, plays an important role.3.
Proofs
Lemma 3.1.
If ¯ F ∈ RV − α then a ( · ) ∈ RV /α . Proof.
We know that 1 − F is regularly varying so that n ¯ F ( a n x ) → x − α as n → ∞ . Set U = 1 / ¯ F and V = U ← . Therefore U ( a n x ) /n → x α , for x > , and inverting we have V ( ny ) /a n → y − /α . Since, a n ≃ ( − F ) ← ( n ) = V ( n ) and a switch to a continuous variable we have V ( ty ) − V ( t ) a ( t ) → y /α − . Now we have, lim t →∞ a ( tx ) a ( t ) = lim t →∞ a ( tx )( V ( tx ) − V ( t )) − a ( t )( V ( tx x − − V ( tx ))) , = ( x /α − / (1 − x − /α ) , = x /α . (cid:3) Proof of Theorem 2.1-(a).
Suppose F ∈ D (Φ α ) , and 1 − F is regularly varying so thatlim n →∞ F ( a n x ) F ( a n ) = x − α , x >
0; andlim n →∞ F n ( a n x ) = Φ α ( x ) , x ∈ R , with a n = F ← (1 − n ) = inf { x : F ( x ) > − n } , n ≥ b n = 0 . From Theorem B.4, F satisfies the von Mises condition: lim t →∞ tf ( t )1 − F ( t ) = α. (3.1)Now, by Theorem B.6, implies the following density convergence on compact sets:lim n →∞ g n ( x ) = φ α ( x ) , x ∈ K ⊂ (0 , ∞ ) (3.2)where K is a compact set, and g n ( x ) = na n f ( a n x ) F n − ( a n x ) . From definition of R´enyi entropy, we write, H β ( g n ) = 11 − β log [ I A ( n, v ) + I B ( n, v ) + I C ( n, v )] . (3.3)where, I A ( n, v ) = R ∞ v ( g n ( x )) β dx, and I B ( n, v ) = R v − −∞ ( g n ( x )) β dx, and I C ( n, v ) = R vv − ( g n ( x )) β dx. It is enough to show that lim v →∞ lim n →∞ ( I A ( n, v ) + I B ( n, v )) = 0 . We set, 0 < I A ( n, v ) = Z ∞ v ( g n ( x )) β − dF n ( a n x ) , = L ( n, β ) Z ∞ v (cid:18) f ( a n x ) f ( a n ) F ( n − ( a n x ) (cid:19) β − g n ( x ) dx,< L ( n, β ) (cid:18) f ( a n v ) f ( a n ) (cid:19) β − (1 − F n ( a n v )) , where, L ( n, β ) = (cid:18) a n f ( a n ) n ¯ F ( a n )¯ F ( a n ) (cid:19) β − , and, n ¯ F ( a n ) = 1 and using von Mises conditionsin (3.1) f is decreasing function, L ( n, β ) → α β − , as n → ∞ , and f ( a n x ) f ( a n ) < x ≥ . Hence, for β > , lim v →∞ lim n →∞ I A ( n, v ) = 0 . (3.4)Now, we choose ξ n by − log F ( ξ n ) ≃ n − / , and t n = ξ n a n . If ξ n a n → c > n / ≃− n (log F ( t n a n )) → c − α and this is contradict the fact that n / → ∞ . Therefore, t n → ξ n → ∞ for large n. We have, I B ( n, u ) = Z t n −∞ ( g n ( x )) β dx + Z v − t n ( g n ( x )) β dx = I B ( n ) + I B ( n, v ) . (3.5) We set, I B ( n ) = n β a β − n Z ξ n −∞ ( F n − ( s ) f ( s )) β ds, (where, a n x = s ) , ≤ n β a β − n F β ( n − ( ξ n ) Z ∞−∞ ( f ( s )) β ds, ≃ n β a β − n exp { ( n − βn − / } Z ∞−∞ ( f ( s )) β ds, Since a n ≃ (cid:16) − F (cid:17) ← ( n ) from Lemma 3.1, a n ∈ RV α and (B.2) for n > N given ǫ > ρ ( n ) < ǫα then a n = c ( n ) exp n Z nN ρ ( t ) t − dt o ,< ( n/N ) ǫα c, Therefore, 0 < I B ( n ) ≤ c n β ( n/N ) ǫα ( β − exp { β ( n / − n − / } Z ∞−∞ ( f ( s )) β ds. If R ∞−∞ ( f ( x )) β dx < ∞ then, lim n →∞ I B ( n ) = 0 . (3.6)From (B.3), for given ǫ > f ( a n x ) ≤ − ( α + ǫ ) log F ( a n x ) a n x ultimately and fromTheorem B.2 for sufficiently large n given ǫ > n ¯ F ( a n x ) < − ǫ x − ( α − ǫ ) and given ǫ > n > − n − n < − (1 − ǫ ) . Hence, for sufficiently large n such that n > n we have g n ( x ) = na n f ( a n x ) F n − ( a n x ) ,< − ( α + ǫ ) n log F ( a n x ) x − exp n n − n n log F ( a n x ) o ,< α + ǫ − ǫ x − − α + ǫ exp n − − ǫ − ǫ x − ( α − ǫ ) o ,< cα ′ x − α ′ − exp n − cx − α ′ o . where, α ′ = α − ǫ , and c is positive constant. We define h ( x ) = cα ′ x − α ′ − exp n − cx − α ′ o . (3.7)Set, I B ( n, v ) = R v − t n ( g n ( x )) β dx. From (3.7) for large n, g n ( x ) < h ( x ) we have0 < I B ( n, v ) < Z v − ( h ( x )) β dx, Since R ∞ ( h ( x )) β < ∞ so that, lim v →∞ lim n →∞ I B ( n, v ) = 0 . (3.8)From, (3.5), (3.6) and (3.8) lim v →∞ lim n →∞ I B ( n, v ) = 0 . (3.9) Next, I C ( n, v ) = R vv − ( g n ( x )) β dx, and from (3.7) for large n, we have g n ( x ) < h ( x ) , and R vv − h ( x ) dx < ∞ , and using (3.2) lim n →∞ g n ( x ) = φ α ( x ) , locally uniformly convergence in x ∈ [ v − , v ] by DCT, lim v →∞ lim n →∞ Z vv − ( g n ( x )) β dx = Z ∞ ( φ α ( x )) β dx. (3.10)And from (3.3), (3.4), (3.9) and (3.10) imply,lim n →∞ H β ( g n ) = H β ( φ α ) . (cid:3) Let Y , Y , . . . are iid rvs with common distribution function F Y and r ( F Y ) < ∞ and X i = 1 / ( r ( F Y ) − Y i ) with common distribution function F X . In following lemma is easilyshow that the relationship between the domain attraction of Φ α and Ψ α . Lemma 3.2. If F Y ∈ D (Ψ α ) then F X ∈ D (Φ α ) with a n = δ n and b n = 0 . Proof.
From F Y ∈ (.Ψ α ) with δ n > r ( F Y ) < ∞ , we have, ∨ ni =1 Y i − r ( F Y ) δ n d −→ M, where, M is rv with distribution function Ψ α . Therefore, − (cid:18) ∨ ni =1 Y i − r ( F Y ) δ n (cid:19) − = (cid:18) ∨ ni =1 δ n r ( F Y ) − Y i (cid:19) , = ∨ ni =1 X i a n d −→ − M , (3.11)and − M is a rv with distribution function Φ α and left of (3.11) is equivalent to F X ∈ D (Φ α )with a n = δ n and b n = 0 . (cid:3) Proof of Theorem 2.1-(b).
Suppose F ∈ D (Ψ α ) , iff r ( F ) < ∞ and F ( r ( F ) − x − ) ∈ RV − α , is regularly varying. In this case we may set τ n = F − (1 − n ) , and δ n = ( r ( F ) − τ n ) , and then lim n →∞ F n ( δ n x + r ( F )) = Ψ α ( x ) , x ∈ R , From Theorem B.4, F satisfies the von Mises condition:lim x ↑ r ( F ) ( r ( F ) − x ) f ( x ) F ( x ) = α. Now, by Theorem B.6, implies the following density convergence on compact sets:lim n →∞ g n ( x ) = ψ α ( x ) , x ∈ K ⊂ ( −∞ , , where K is a compact set, and g n ( x ) = nδ n f ( δ n x + r ( F )) F n − ( δ n x + r ( F )) . From definitionof R´enyi entropy, we write, H β ( g n ) = 11 − β log (cid:18)Z −∞ ( g n ( x )) β dx (cid:19) . Put x = − /y, and using Lemma 3.2, we have, H β ( g n ) = 11 − β log (cid:18)Z ∞ (˜ g n ( x )) β dyy (cid:19) , where, ˜ g n ( x ) = nδ n f ( r ( F ) − δ n /y ) F n − ( r ( F ) − δ n /y ) . From Lemma 3.2, F X ( a n y ) = F ( r ( F ) − δ n /y ) ∈ D (Φ α ) , and with conditions Theorem 2.1-(a), if R r ( F ) −∞ ( f ( s )) β ds < ∞ and f is de-creasing function for β > n →∞ H β ( g n ) = H β ( ψ α ) . (cid:3) Lemma 3.3.
Suppose F ∈ D (Λ) with auxiliary function u and ǫ > . There exists a large N such that for x ≥ , and n > Nn (1 − F ( a n x + b n )) ≤ (1 + ǫ ) (1 − ǫx ) ǫ − , and for x < , n (1 − F ( a n x + b n )) ≤ (1 + ǫ ) (1 + ǫ | x | ) ǫ − . Proof.
From Theorem B.8 and for x ≥ , and sufficient large n such that | u ′ ( a n t + b n ) | ≤ ǫ (cid:12)(cid:12)(cid:12)(cid:12) u ( a n x + b n ) u ( b n ) − (cid:12)(cid:12)(cid:12)(cid:12) = (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)Z a n x + b n b n u ′ ( s ) u ( b n ) ds (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) , ≤ Z x | u ′ ( a n t + b n ) | dt, where a n s + b n = t, ≤ ǫ x. (3.12)Consequently, u ( b n ) u ( a n x + b n ) < − ǫ x . Since, for large n, and ǫ > (1 − ǫ )¯ F ( b n ) < n < (1+ ǫ )¯ F ( b n ) then n (1 − F ( a n x + b n )) ≤ (1 + ǫ ) 1 − F ( b n + a n x )1 − F ( b n ) , = (1 + ǫ ) c ( b n + a n x ) c ( b n ) exp n − Z b n + a n xb n dyu ( y ) o , ≤ (1 + ǫ ) exp n − Z x u ( t ) dsu ( b n + a n s ) o , where, y = b n + a n s and lim t →∞ c ( t + xu ( t )) c ( t ) = 1 and from (3.12), then n (1 − F ( a n x + b n )) ≤ (1 + ǫ ) exp n − Z x dt − ǫt o , = (1 + ǫ ) (1 − ǫx ) ǫ − . For the second statement, for x < n (cid:12)(cid:12)(cid:12)(cid:12) − u ( a n x + b n ) u ( b n ) (cid:12)(cid:12)(cid:12)(cid:12) = (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)Z b n a n x + b n u ′ ( s ) u ( b n ) ds (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) , ≤ Z x | u ′ ( a n t + b n ) | dt, where a n s + b n = t, ≤ ǫ | x | . (3.13)Consequently, u ( b n ) u ( a n x + b n ) < ǫ | x | . Then, n (1 − F ( a n x + b n )) ≤ (1 + ǫ ) c ( b n + a n x ) c ( b n ) exp n − Z b n + a n xb n dyu ( y ) o , ≤ (1 + ǫ ) exp n Z x u ( t ) dsu ( b n + a n s ) o , where, y = b n + a n s and lim t →∞ c ( t + xu ( t )) c ( t ) = 1 and from (3.13), then n (1 − F ( a n x + b n )) ≤ (1 + ǫ ) exp n Z x dt ǫ | t | o , = (1 + ǫ ) (1 + ǫ | x | ) ǫ − . (cid:3) Proof of Theorem 2.1-(c).
Suppose F ∈ D (Λ) and 1 − F is Γ varying so thatlim n →∞ F ( b n + xa n ) F ( b n ) = e − x , x ∈ R , where the function a n = u ( b n ) = R r ( F ) b n F ( s ) ds/F ( b n ) is called an auxiliary function. From(1.1) lim n →∞ F n ( a n x + b n ) = Λ( x ) , x ∈ R , with a n = u ( b n ) and b n = F ← (1 − n ) = inf { x : F ( x ) > − n } , n ≥ . From Theorem B.5, F satisfies the von Mises condition: lim n →∞ f ( b n ) u ( b n ) F ( b n ) = 1 . (3.14)Now, by Theorem B.6 implies the following density convergence on compact sets:lim n →∞ g n ( x ) = λ ( x ) , x ∈ K ⊂ R , (3.15)where K is a compact set, and g n ( x ) = n a n f ( a n x + b n ) F n − ( a n x + b n ) . We write H β ( g n ) = − β ) log hR v − v ( g n ( x )) β dx + R − v −∞ ( g n ( x )) β dx + R ∞ v ( g n ( x )) β dx i . It is enough to show,lim v →∞ lim n →∞ (cid:20)Z − v −∞ ( g n ( x )) β dx + Z ∞ v ( g n ( x )) β dx (cid:21) = 0 . Set, I E ( n, v ) = R ∞ v ( g n ( x )) β − g n ( x ) dx, we have0 < I E ( n, v ) < J ( n, β ) (cid:18) f ( a n v + b n ) f ( b n ) (cid:19) β − (1 − F n ( a n v + b n )) . where, J ( n, β ) = (cid:16) n ¯ F ( b n ) f ( b n ) a n ¯ F ( b n ) (cid:17) β − → n → ∞ and, 0 ≤ F ( . ) ≤ , for β > f isdecreasing function then f ( a n v + b n ) f ( b n ) < v ≥ . Therefore,lim v →∞ lim n →∞ I E ( n, v ) = 0 . (3.16)Now, we choose ξ n satisfying − log F ( ξ n ) ≃ n − / . If t n = ξ n − b n a n → c, then n / ≃− n log F ( a n t n + b n ) → e − c , and this is contradict with n / → ∞ . Therefore, t n = ξ n − b n a n →−∞ , as ξ n → r ( F ) for large n. We now decompose the integral, I F ( n, v ) = Z t n −∞ ( g n ( x )) β dx + Z − vt n ( g n ( x )) β dx = I F ( n ) + I F ( n, v ) . Set 0 < I F ( n ) < n β a β − n F β ( n − ( ξ n ) Z R ( f ( x )) β dx, ≃ n β a β − n exp { n / β } exp { β n − / } Z ∞−∞ ( f ( x )) β dx, where, F n − ( ξ n ) = exp n ( n −
1) log F ( ξ n ) o ≃ exp {− n / + n − / } . From Theorem B.1, a n ∈ RV and from Lemma B.1, for n > N given ǫ > ǫ ( n ) < ǫ then a n = c ( n ) exp n Z nN ρ ( t ) t − dt o < ( n/N ) ǫ c, Therefore, 0 < I F ( n ) ≤ cn β ( n/N ) ( β − ǫ exp { β ( n / − n − / ) } Z ∞−∞ ( f ( s )) β ds. If R ∞−∞ ( f ( x )) β dx < ∞ then lim n →∞ I F ( n ) = 0 . (3.17)From (3.14), for given ǫ > f ( a n x + b n ) ≤ − (1 + ǫ ) log F ( a n x + b n ) u ( a n x + b n ) ultimatelyand from Theorem B.7 given ǫ > x < a n x + b n ≥ n , u ( b n ) u ( a n x + b n ) < − ǫ h − log F ( a n x + b n ) − log F ( b n ) i ǫ and for ǫ > n ≥ − ǫ < n − n , and from Lemma3.3 for ǫ > x < n ¯ F ( a n x + b n ) < (1 + ǫ ) (1 + ǫ | x | ) ǫ − , and for ǫ > , F ( b n ) < n − ǫ . Therefore, g n ( x ) = na n f ( a n x + b n ) F n − ( a n x + b n ) ,< − (1 + ǫ ) u ( b n ) u ( a n x + b n ) n log F ( a n x + b n ) exp n n − n n log F ( a n x + b n ) o ,< ǫ (1 − ǫ )(1 − ǫ ) ǫ ( − n log F ( a n x + b n )) ǫ exp n (1 − ǫ ) n log F ( a n x + b n ) o ,< (1 + ǫ )(1 + ǫ ) ǫ ǫ (1 − ǫ )(1 − ǫ ) ǫ (1 + ǫ | x | ) ǫ ǫ exp n − (1 − ǫ )(1 + ǫ ) (1 + ǫ | x | ) ǫ − o , = c (1 + ǫ | x | ) c exp n − c (1 + ǫ | x | ) ǫ − o , where, c = (1+ ǫ )(1+ ǫ ) ǫ ǫ (1 − ǫ )(1 − ǫ ) ǫ and c = ǫ ǫ , and c = (1 − ǫ )(1 + ǫ ) . We define h ( x ) = c (1 + ǫ | x | ) c exp n − c (1 + ǫ | x | ) ǫ − o . (3.18)And Z −∞ h ( x ) dx = Z −∞ c (1 + ǫ | x | ) c exp n − c (1 + ǫ | x | ) ǫ − o dx. Putting, c (1 + ǫ | x | ) ǫ − = y, and − c (1 + ǫ | x | ) ǫ − − dx = dy we have, Z −∞ h ( x ) dx = c (cid:18) c (cid:19) ǫ ( c +1) Z ∞ c y ǫ ( c +1) − e − y dy < ∞ . Similarly, from (3.14) and from Theorem B.7 given ǫ > x > a n x + b n ≥ n , u ( b n ) u ( a n x + b n ) < − ǫ h − log F ( a n x + b n ) − log F ( b n ) i − ǫ and from Lemma 3.3 for ǫ > x > n ¯ F ( a n x + b n ) < (1 + ǫ ) (1 − ǫ x ) ǫ − , we have g n ( x ) = na n f ( a n x + b n ) F n − ( a n x + b n ) ,< − (1 + ǫ ) u ( b n ) u ( a n x + b n ) n log F ( a n x + b n ) exp n n − n n log F ( a n x + b n ) o ,< ǫ (1 − ǫ )(1 − ǫ ) − ǫ ( − n log F ( a n x + b n )) − ǫ exp n (1 − ǫ ) n log F ( a n x + b n ) o ,< (1 + ǫ )(1 + ǫ ) − ǫ ǫ (1 − ǫ )(1 − ǫ ) − ǫ (1 − ǫ x ) − ǫ ǫ exp n − (1 − ǫ )(1 + ǫ ) (1 − ǫ x ) ǫ − o , = c (1 − ǫ x ) c exp n − c (1 − ǫ x ) ǫ − o , where, c = (1+ ǫ )(1+ ǫ ) − ǫ ǫ (1 − ǫ )(1 − ǫ ) − ǫ and c = − ǫ ǫ , and c = (1 − ǫ )(1 + ǫ ) . We define h ( x ) = c (1 − ǫ x ) c exp n − c (1 − ǫ x ) ǫ − o . (3.19)And Z ∞ h ( x ) dx = Z ∞ c (1 − ǫ x ) c exp n − c (1 − ǫ x ) ǫ − o dx. Putting c (1 − ǫ x ) ǫ − = y, and − c (1 − ǫ x ) ǫ − − dx = dy we have, Z ∞ h ( x ) dx = c (cid:18) c (cid:19) ǫ ( c +1) Z c y ǫ ( c +1) − e − y dy < ∞ . where, ǫ > ǫ . Now, we set I F ( n, v ) = R − vt n ( g n ( x )) β dx. From (3.18) for large n, we have g n ( x ) < h ( x ) , then I F ( n, v ) < Z − v −∞ c β (1 + ǫ | x | ) βc exp n − c β (1 + ǫ | x | ) ǫ − o dx. (3.20)Putting c β (1 + ǫ | x | ) ǫ − = y, and − c β (1 + ǫ | x | ) ǫ − − dx = dy we have,0 < I F ( n, v ) < c β (cid:18) c (cid:19) ǫ ( c β +1) Z ∞ c β (1+ ǫ v ) ǫ y ǫ ( c β +1) − e − y dy. Therefore, lim v →∞ lim n →∞ I F ( n, v ) = 0 . (3.21)From, (3.17) and (3.21) lim v →∞ lim n →∞ I F ( n, v ) = 0 . (3.22)Set I G ( n, v ) = R v − v ( g n ( x )) β dx. From (3.18) and (3.19) for large n, we have g n ( x ) < h ( x ) + h ( x ) , for and R − v h ( x ) dx + R v h ( x ) dx < ∞ , and using (3.15) lim n →∞ g n ( x ) = λ ( x ) , for x ∈ [ − v, v ] by DCT, lim v →∞ lim n →∞ Z v − v ( g n ( x )) β dx = Z ∞−∞ ( λ ( x )) β dx. (3.23)From, (3.16), (3.22) and (3.23), lim n →∞ H β ( g n ) = H β ( λ ) . (cid:3) Appendix A. Lemma A.1.
The R´enyi entropy of(i) Fr´echet law: H β ( φ α ) = − log α + α + 1 α log β − − β (cid:18) log β − log Γ (cid:18) α + 1 α ( β −
1) + 1 (cid:19)(cid:19) ;where, α +1 < β. (ii) Weibull law: H β ( ψ α ) = − log α + α − α log β − − β (cid:18) log β − log Γ (cid:18) α − α ( β −
1) + 1 (cid:19)(cid:19) ;where, max (cid:16) , β − β (cid:17) < α, for β > . (iii) Gumbel law: H β ( λ ) = 11 − β log Γ( β ) β β ;where, β > . Proof. (i) The R´enyi entropy of Fr´echet distribution is H β ( φ α ) = 11 − β log Z ∞ (cid:16) αx − α − e − x − α (cid:17) β dx. Putting, βx − α = u, − βαx − α − dx = du,H β ( φ α ) = 11 − β log Z ∞ α β − u ( β − α +1 α ) β − (( β − α +1 α )+1) e − u du, = 11 − β (cid:18) ( β −
1) log α − (cid:18) ( β − α + 1 α + 1 (cid:19) log β + log Γ (cid:18) α + 1 α ( β −
1) + 1 (cid:19)(cid:19) . where, α +1 α ( β −
1) + 1 > , so β < α + 1 . (ii) The R´enyi entropy of Weibull distribution H β ( ψ α ) = 11 − β log Z −∞ (cid:16) α ( − x ) α − e − ( − x ) α (cid:17) β dx. Putting, β ( − x ) α = u, − βα ( − x ) α − dx = du,H β ( ψ α ) = 11 − β log Z ∞ α β − u ( β − α − α ) β − (( β − α − α )+1) e − u du, = 11 − β (cid:18) ( β −
1) log α − (cid:18) ( β − α − α + 1 (cid:19) log β + log Γ (cid:18) α − α ( β −
1) + 1 (cid:19)(cid:19) , where, α − α ( β −
1) + 1 > . If β > α > β − β , and for β < , we have 0 < α, therefore,max (cid:16) , β − β (cid:17) < α, for all β > . (iii) The R´enyi entropy of Gumbel distribution H β ( λ ) = 11 − β log Z ∞−∞ (cid:16) e − x e − e − x (cid:17) β dx, Taking uβ = e − x and duβ = − e − x dxH β ( λ ) = 11 − β log Z ∞ u β − e − u β − β du, = 11 − β (log Γ( β ) − β log β ) . where, β > . (cid:3) Lemma A.2. If Y = X − ba , for b ∈ R and a > , then the R´enyi’s entropy of Y is given by H β ( f Y ) = − log a + H β ( f X ) . Proof.
We have F Y ( y ) = Pr ( X ≤ ay + b ) = F X ( ay + b ) , and f Y ( y ) = af X ( ay + b ) , so thatfrom (1.2), H β ( f Y ) = 11 − β log Z ∞−∞ ( af X ( ay + b )) β dy = 11 − β log Z ∞−∞ f βX ( z ) a β − dz, = − log a + H β ( f X ) . (cid:3) Appendix B. Definition 1. (Definition, Page 27, Resnick (1987)) A nonegative, nondecreasing function V ( x ) defined on a semi infinite interval ( z, ∞ ) is Π varying (written V ∈ Π) if there existfunctions a ( t ) > , b ( t ) ∈ R such that for x > t →∞ V ( tx ) − b ( t ) a ( t ) = log x. Theorem B.1. (Proposition 0.12, Resnick (1987)) If V ∈ Π with auxiliary function a ( t )then a ( · ) ∈ RV . Theorem B.2. (Proposition 0.8, Resnick (1987)) Suppose U ∈ RV ρ , ρ ∈ R . Take ǫ > . Then there exists t such that for x ≥ t ≥ t (1 − ǫ ) x ρ − ǫ < U ( tx ) U ( t ) < (1 + ǫ ) x ρ + ǫ . Lemma B.1. (Corollary, Page 17, Resnick (1987)) If L is slowly varying iff L can berepresented as L ( x ) = c ( x ) exp n Z x t − ǫ ( t ) dt o , (B.1)for x > x →∞ c ( x ) = c and lim t →∞ ǫ ( t ) = 0 . Remark B.1. (Remark, Page 19, Resnick (1987)) If U ∈ RV ρ then U has representation U ( x ) = c ( x ) exp n Z x t − ρ ( t ) dt o , (B.2)where, lim x →∞ c ( x ) = c and lim t →∞ ρ ( t ) = ρ. Theorem B.3. (Proposition 2.1, Resnick (1987)) Let F ∈ D ( G ) . (i) If G = Φ α , then a n = (1 / (1 − F )) ← ( n ) , b n = 0 , and if for some integer 0 < k < α, Z −∞ | x | k F ( dx ) < ∞ , then lim n →∞ E (cid:18) M n a n (cid:19) k = R ∞ x k Φ α ( dx ) = Γ (cid:0) − kα (cid:1) . (ii) If G = Ψ α , then δ n = r ( F ) − (1 / (1 − F )) ← ( n ) , r ( F ) < ∞ , and if for some integer k > , Z r ( F ) −∞ | x | k F ( dx ) < ∞ , then lim n →∞ E (cid:18) M n − r ( F ) δ n (cid:19) k = R −∞ x k Ψ α ( dx ) = ( − k Γ (cid:0) kα (cid:1) . (iii) If G = Λ , then b n = (1 / (1 − F )) ← ( n ) , a n = u ( b n ) , and if for some integer k > , Z −∞ | x | k F ( dx ) < ∞ , then lim n →∞ E (cid:18) M n − b n a n (cid:19) k = R ∞−∞ x k Λ( dx ) = ( − k Γ ( k ) (1) , where Γ ( k ) (1) is thek-th derivative of the Gamma function at x = 1 . Theorem B.4. (Proposition 1.15 and 1.16, Resnick (1987))(i) Suppose that distribution function F is absolutely continuous with density f which iseventually positive.(a) If for some α > x →∞ xf ( x ) F ( x ) = α, (B.3)then F ∈ D (Φ α ) . (b) If f is nonincreasing and F ∈ D (Φ α ) then (B.3) holds.(ii) Suppose F has right endpoint r ( F ) finite and density f positive in a left neighbourhoodof r ( F ) . (a) If for some α > x → r ( F ) ( r ( F ) − x ) f ( x ) F ( x ) = α (B.4)then F ∈ D (Ψ α ) . (b) If f is nonincreasing and F ∈ D (Ψ α ) then (B.4) holds. Theorem B.5. (Proposition 1.17, Resnick (1987)) Let F be absolutely continuous in a leftneighborhood of r ( F ) with density f. Iflim x ↑ r ( F ) f ( x ) Z r ( F ) x F ( t ) dt/F ( x ) = 1 , (B.5)then F ∈ D (Λ) . In this case we may take, u ( x ) = Z r ( F ) x F ( t ) dt/F ( x ) , b n = F ← (1 − /n ) , a n = u ( b n ) . Theorem B.6. (Theorem 2.5, Resnick (1987)), Suppose that F is absolutely continuouswith pdf f. If F ∈ D ( G ) and(i) G = Φ α , then g n ( x ) → φ α ( x ) locally uniformly on (0 , ∞ ) iff (B.3) holds;(ii) G = Ψ α , then g n ( x ) → ψ α ( x ) locally uniformly on ( −∞ ,
0) iff (B.4) holds;(iii) G = Λ , then g n ( x ) → λ ( x ) locally uniformly on R iff (B.5) holds. Theorem B.7. (Lemma 2, De Haan and Resnick (1982)) Suppose F ∈ D (Λ) with auxiliaryfunction u and ǫ > . There exists a t such that for x ≥ , t ≥ t (1 − ǫ ) (cid:20) − log F ( t ) − log F ( t + xu ( t )) (cid:21) − ǫ ≤ u ( t + xu ( t )) u ( t ) ≤ (1 + ǫ ) (cid:20) − log F ( t ) − log F ( t + xu ( t )) (cid:21) ǫ , and for x < , t + xu ( t ) ≥ t (1 − ǫ ) (cid:20) − log F ( t + xu ( t )) − log F ( t ) (cid:21) − ǫ ≤ u ( t + xu ( t )) u ( t ) ≤ (1 + ǫ ) (cid:20) − log F ( t + xu ( t )) − log F ( t ) (cid:21) ǫ . Theorem B.8. (Corollary, Balkema and De Haan (1972)) A distribution function F ∈ D (Λ)if and only if there exist a positive function c satisfying lim x → r ( F ) c ( x ) = 1 and a positivedifferentiable function u ( t ) satisfying lim x → r ( F ) u ′ ( x ) = 0 such that¯ F ( x ) = c ( x ) exp n − Z x −∞ dtu ( t ) o for x < r ( F ) . References [1]
Artstein, S. and Ball, K.M. and Barthe, F. and Naor, A., (2004),
On the rate of convergence inthe entropic central limit theorem , Probability Theory Related Fields , Vol. 129, Pages 381-390.[2]
Barron A.R., (1986),
Entropy and the Central Limit Theorem, The Annals of Probability , Vol. 14, No.1, Pages 336-342.[3]
Balkema, A. A. and De Haan, L., (1972),
On R. von Mises condition for the domain of attraction of exp( − e − x ) , The Annals of Mathematical Statistics , Vol. 43, No. 4, Pages 1352-1354.[4] Brown, L. D., (1982),
A proof of the central limit theorem motivated by the Cramer-Rao inequality , Statistics and probability: Essays in Honour of C.R. Rao , Pages 141-148.[5]
Cui, Hongfei and Ding, Yiming, (2010),
The convergence of the R´enyi entropy of the normalized sumsof IID random variables, Statistics and Probability Letters , Vol. 80, Pages 1167-1173.[6]
De Haan, Laurens and Resnick, Sidney I., (1982),
Local limit theorems for sample extremes, TheAnnals of Probability , Vol. 10, No. 2, Pages 396-413.[7]
Embrechts P. and Kl¨uppelberg C. and Mikosch, T., (1997),
Modelling Extremal Events for Insur-ance and Finance,
Springer Verlag.[8]
Galambos J., (1987),
The Asymptotic Theory of Extreme Order Statistics,
Krieger Pub. Co.[9]
Johnson O., (2006),
Information Theory and the Central Limit Theorem,
Imperical College Press.[10]
Johnson, O. and Barron, A., (2004),
Fisher information inequalities and the central limit theorem,Probab. Theory Relat. Fields , Vol. 129, Pages 391-409.[11]
Linnik, J. V., (1959),
An information theoretic proof of the central limit theorem with Lindebergconditions , Theory Probability Application , Vol. 4, Pages 288-299.[12]
Resnick Sidney I., (1987),
Extreme Values, Regular Variation, and Point Processes,
Springer Verlag.[13]
Renyi, A., (1961),
On Measures of Entropy and Information, Proceedings of the Fourth BerkeleySymposium on Mathematics, Statistics and Probability, University of California Press , Vol. 1, Pages 547-561.[14]
Saeb, A., (2014),
Rates of convergence for R´enyi entropy in extreme value theory, (Preprint).[15]
Shimizu, R., (1975),
On fisher’s amount of information for location family , Statistical distributions inscientific work , Vol. 3, Pages 305-312.[16]
Takano, Seiji, (1987),