Improving Coverage Accuracy of Block Bootstrap Confidence Intervals
aa r X i v : . [ s t a t . M E ] A p r Improving Coverage Accuracy of BlockBootstrap Confidence Intervals
Stephen M.S. Lee e-mail: [email protected]
P.Y. Lai e-mail: [email protected] of Statistics and Actuarial Science, The University of Hong KongPokfulam Road, Hong Kong
BSTRACT
The block bootstrap confidence interval based on dependent data can outper-form the computationally more convenient normal approximation only withnon-trivial Studentization which, in the case of complicated statistics, callsfor highly specialist treatment. We propose two different approaches to im-proving the accuracy of the block bootstrap confidence interval under verygeneral conditions. The first calibrates the coverage level by iterating theblock bootstrap. The second calculates Studentizing factors directly fromblock bootstrap series and requires no non-trivial analytic treatment. Bothapproaches involve two nested levels of block bootstrap resampling and yieldhigh-order accuracy with simple tuning of block lengths at the two resam-pling levels. A simulation study is reported to provide empirical support forour theory.Key words and phrases: block bootstrap; coverage calibration; Studenti-zation; weakly dependent.
Introduction
The block bootstrap has been developed as a completely model-free proce-dure for handling inference problems concerning dependent data. A majorcriticism that impedes widespread acceptance of the procedure in applica-tions is that it lacks second-order accuracy and that empirical selection ofblock length is critical yet difficult. Although intensive work has been done onthe second issue, remedies thus far proposed for the first drawback are ratherrestrictive in the sense that they require either non-trivial, and sometimesalgebraically formidable, Studentization or assumptions of more stringentmodel structures. Those well-established techniques, such as the iterativebootstrap and the bootstrap- t , designed for enhancing bootstrap accuracy forindependent data appear to have lost their appeal in the context of dependentdata, because the block bootstrap series typically exhibits undesirable arte-facts as a consequence of pasting randomly selected data blocks together.An important question is whether the block bootstrap can be made moreaccurate, by an order asymptotically as well as for finite samples, withoutanalytically cumbersome Studentization nor having to confine applicationsto dependent data generated by specific processes.We investigate formally the applications of two general resampling-basedtechniques, namely coverage calibration and bootstrap Studentization, to2he block bootstrap confidence intervals based on dependent data. A noveldouble bootstrap procedure is proposed for either coverage calibration orbootstrap Studentization to improve coverage accuracy of the block bootstrapbeyond the first order. The procedure enables both techniques to retainthe simplicity and generality they have already enjoyed when applied toindependent data.Hall (1985) and K¨unsch (1989) introduce the block bootstrap as a fullynonparametric extension of the bootstrap to handle dependent data. Itsconsistency for distributional estimation is verified by K¨unsch (1989) and Liuand Singh (1992). Lahiri (1992) proves for m -dependent data that the blockbootstrap distribution of an adjusted Studentized sample mean is accurateto second order. Davison and Hall (1993) achieve similar results by kernel-based Studentization. Hall, Horowitz and Jing (1995), G¨otze and K¨unsch(1996) and Zvingelis (2003) sharpen the results by giving explicit orders forthe estimation error.Variants of the block bootstrap include circular block resampling (Politisand Romano, 1992), the stationary bootstrap (Politis and Romano, 1993),the matched-block bootstrap (Carlstein, Do, Hall, Hesterberg and K¨unsch,1998) and the tapered bootstrap (Paparoditis and Politis, 2001). Lahiri(1999) compares the first two with the block bootstrap and confirms supe-riority of the latter. Davison and Hall (1993), Choi and Hall (2000) and3¨uhlmann (2002) remark on the distortion of dependence structures in blockbootstrap series and, for that reason, express doubt over effectiveness of cov-erage calibration by bootstrap iterations.The subsampling method, as studied by Politis and Romano (1994), ismore generally applicable than the block bootstrap, but has inferior asymp-totic properties: see Hall and Jing (1996) and Bertail (1997). Nonparamet-ric methods more accurate than the block bootstrap have been found undermore stringent assumptions on the data generating processes. Examples in-clude the sieve bootstrap (B¨uhlmann, 1997; Choi and Hall, 2000) for linearprocesses, the Markov bootstrap (Rajarshi, 1990) and the local bootstrap(Paparoditis and Politis, 2002) for Markov processes.We introduce in Section 2 a double bootstrap procedure for either cov-erage calibration or Studentization of the overlapping block bootstrap. Sec-tion 3 establishes asymptotic expansions for the coverage probabilities ofboth the iterated block bootstrap and Studentized block bootstrap confi-dence intervals under sufficiently general regularity conditions, derives theoptimal second-level block length in relation to the first-level block lengthand proves asymptotic superiority of our procedures. Section 4 reports asimulation study which compares our methods with the conventional blockbootstrap and two alternative bootstrap- t approaches. Section 5 concludesour findings. All technical proofs are given in Appendix 6.1.4 Coverage calibration and Studentization
Let X = ( X , . . . , X n ) be a series of d -variate observations from the se-quence { X i : −∞ < i < ∞} , which is a realization of a strictly stationary,discrete-time, stochastic process with finite mean µ = E [ X ]. Denote by¯ X = P ni =1 X i /n the sample mean.We briefly review the block bootstrap construction of a level α upper con-fidence bound for a scalar parameter of interest θ = H ( µ ), for some smoothfunction H : R d → R . A natural plug-in estimator of θ is ˆ θ = H ( ¯ X ). Thissmooth function model setup encompasses a wide variety of estimators, ortheir high-order asymptotic approximations, providing a sufficiently generalplatform for investigating the block bootstrap confidence procedure.For a block length ℓ (1 ≤ ℓ ≤ n ), let n ′ = n − ℓ + 1 and define overlappingblocks Y j,ℓ = ( X j , X j +1 , . . . , X j + ℓ − ), j = 1 , . . . , n ′ . A generic first-level blockbootstrap series X ∗ = ( X ∗ , . . . , X ∗ bℓ ), where b = h n/ℓ i and h x i denotes theinteger part of x , is given by sampling b blocks randomly with replacementfrom { Y j,ℓ : 1 ≤ j ≤ n ′ } and pasting them end-to-end in the order sampled,so that ( X ∗ ( j − ℓ +1 , . . . , X ∗ jℓ ) denotes the j th block sampled, j = 1 , . . . , b .Let P ∗ and E ∗ denote the probability measure and expectation operatorinduced by block bootstrap sampling, conditional on X , respectively. Define5 X ∗ = P bℓi =1 X ∗ i / ( bℓ ) and the block bootstrap distribution function G ∗ ( x ) = P ∗ (cid:0) ( bℓ ) / [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] ≤ x (cid:1) , x ∈ R . Then I ( α ) = ˆ θ − n − / G ∗− (1 − α ) defines a level α block bootstrap upper confidence bound for θ . Note thatsampling of overlapping blocks incurs an edge effect which explains the useof H ( E ∗ ¯ X ∗ ), rather than the more conventional ˆ θ = H ( ¯ X ), for centering thebootstrap estimator in the definition of G ∗ . Under regularity conditions tobe detailed in Section 3, the choice ℓ ∝ n / yields the smallest coverageerror, of order O ( n − / ), for I ( α ). For independent and identically distributed data, coverage calibration andStudentization provide two well-known techniques for improving coverageaccuracy of bootstrap confidence intervals. We consider applications of thetwo techniques in the present context of dependent data. Both coveragecalibration and the version of Studentization proposed herein call for a doublebootstrap procedure as described below.Based on X ∗ , define blocks Y ∗ i,j,k = ( X ∗ ( i − ℓ + j , X ∗ ( i − ℓ + j +1 , . . . , X ∗ ( i − ℓ + j + k − ),each of length k (1 ≤ k ≤ ℓ ), for i = 1 , . . . , b and j = 1 , . . . , ℓ ′ , where ℓ ′ = ℓ − k + 1. Note that for each fixed i = 1 , . . . , b , Y ∗ i, ,k , . . . , Y ∗ i,ℓ ′ ,k repre-sent overlapping blocks within the block ( X ∗ ( i − ℓ +1 , . . . , X ∗ iℓ ), which is itself6ampled randomly from the blocks { Y j,ℓ : 1 ≤ j ≤ n ′ } . The second-levelblock bootstrap series, denoted by X ∗∗ = ( X ∗∗ , . . . , X ∗∗ ck ), for c = h n/k i , issampled from the bℓ ′ blocks { Y ∗ i,j,k : 1 ≤ i ≤ b, ≤ j ≤ ℓ ′ } in the same wayas is X ∗ from { Y j,ℓ : 1 ≤ j ≤ n ′ } . That Y ∗ i,j,k is a subseries of k consecu-tive observations within X eliminates the possibility of drawing second-levelblocks that run across joints of the first-level block bootstrap series, therebyavoiding the discontinuity problem which has aroused forejudged criticismsabout the very usefulness of the double block bootstrap.Denote by P ∗∗ and E ∗∗ respectively the probability measure and expecta-tion operator induced by second-level block bootstrap sampling, conditionalon X ∗ . Define ¯ X ∗∗ = P cki =1 X ∗∗ i / ( ck ) and G ∗∗ ( x ) = P ∗∗ (cid:0) ( ck ) / [ H ( ¯ X ∗∗ ) − H ( E ∗∗ ¯ X ∗∗ )] ≤ x (cid:1) , x ∈ R . The second-level block bootstrap distribution G ∗∗ can be used in two differentways, namely coverage calibration and Studentization, to correct I ( α ):1. Coverage calibration —The coverage calibration method adjusts the nominal level α to ˆ α , ob-tained as solution to the equation P ∗ (cid:0) ( bℓ ) / [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] ≤ G ∗∗− (1 − ˆ α ) (cid:1) = 1 − α. The coverage-calibrated upper confidence bound is then I C ( α ) = I ( ˆ α ) =ˆ θ − n − / G ∗− (1 − ˆ α ). 7. Studentization —Let ˆ τ be the conditional standard deviation of ( bℓ ) / H ( ¯ X ∗ ) given X ,and τ ∗ be that of ( ck ) / H ( ¯ X ∗∗ ) given X ∗ . Define, for x ∈ R , J ∗ ( x ) = P ∗ (cid:0) ( bℓ ) / [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] /τ ∗ ≤ x (cid:1) . The level α Studentized upper con-fidence bound is then given by I S ( α ) = ˆ θ − n − / ˆ τ J ∗− (1 − α ).We show in Section 3 that under regularity conditions, I C ( α ) and I S ( α )are asymptotically equivalent up to order O p (cid:0) k − n − / + ℓn − / (cid:1) . Bothmethods enjoy a reduced coverage error of order O ( n − / ) if we set, for ex-ample, 2 k = ℓ ∝ n / . Our results rebut the criticisms expressed by, for ex-ample, Davison and Hall (1993), Choi and Hall (2000) and B¨uhlmann (2002)over the effectiveness of coverage calibration. Indeed, I C ( α ) is the first evernon-Studentized block bootstrap interval having the same order of coverageaccuracy as has previously been shown to be possible only with Studentiza-tion under the present regularity conditions. This has especially importantimplications for problems in which Studentization is found to be numericallyunstable and therefore results in highly variable interval endpoints. On theother hand, construction of I S ( α ) makes unnecessary all those non-trivial,problem-specific, algebraic manipulations which are instrumental to calcu-lation of the Studentizing factors suggested by Lahiri (1992), Davison andHall (1993) and G¨otze and K¨unsch (1996). Indeed, both ˆ τ and τ ∗ are readilyobtained by direct Monte Carlo simulation from the bootstrap distributions8 ∗ and G ∗∗ respectively, thus adhering most closely to the celebrated plug-inprinciple underlying the very bootstrap methodology. Higher-order asymptotic investigation of coverage accuracy of the block boot-strap confidence bounds is possible if we assume regularity conditions thatfacilitate Edgeworth expansions of the distribution functions of n / (ˆ θ − θ )and n / (ˆ θ − θ ) / ˆ τ . The set of conditions considered by G¨otze and Hipp(1983) has generally been accepted as the standard assumptions underpin-ning a high-order asymptotic theory of the block bootstrap. Importantly,previous studies have shown that the block bootstrap can be made accurateto second order only with non-trivial Studentization or substantial strength-ening of the G¨otze and Hipp conditions. We shall establish asymptotic resultsfor our coverage calibration and Studentization approaches under the G¨otzeand Hipp conditions, as modified by Lahiri (2003, Section 6.5) below, with k · k denoting the usual Euclidean norm:(A1) E k X k δ < ∞ for some δ > n →∞ Cov (cid:0) n − / P ni =1 X i (cid:1) exists and is nonsingular.9A3) There exists a constant C ∈ (0 ,
1) such that for i, j > /C ,inf ( s T Cov i + j X r = i +1 X r ! s : k s k = 1 ) > Cj. (A4) There exist a constant C > σ -fields D , D ± , . . . of the σ -fieldunderlying the probability space induced by X such that for i, j =1 , , . . . ,(i) there exist D i + ji − j -measurable random vectors ˜ X i,j satisfying E k X i − ˜ X i,j k ≤ C − e − Cj for j > /C , where D sr denotes the sigma-fieldgenerated by {D t : r ≤ t ≤ s } ;(ii) | P ( A ∩ B ) − P ( A ) P ( B ) | ≤ C − e − Cj for any A ∈ D i −∞ and B ∈ D ∞ i + j ;(iii) E (cid:12)(cid:12)(cid:12) E h exp( ιs T P i + jr = i − j X r ) | {D t : t = i } i(cid:12)(cid:12)(cid:12) ≤ e − C for i > j > /C and s ∈ R d with k s k ≥ C , where ι = − E | P ( A | {D t : t = i } ) − P ( A | {D t : 0 < | t − i | ≤ j + r } ) | ≤ C − e − Cj for r = 1 , , . . . and A ∈ D i + ri − r .Note that (A4) introduces an auxiliary set of sub- σ -fields D t to bring a widevariety of weakly dependent processes under a common framework. Specialexamples include linear processes, m -dependent shifts, stationary homoge-neous Markov chains and stationary Gaussian processes.Bhattacharya and Ghosh’s (1978) smooth function model supplies a richclass of estimators and has been extensively studied in the bootstrap liter-10ture: see, for example, Hall (1992). In the dependent data context, it en-compasses estimators such as sample autocovariances, sample autocorrelationcoefficients, sample partial autocorrelation coefficients and Yule-Walker esti-mators for autoregressive processes. Importantly, the model admits highly-structured asymptotic expansions to facilitate establishment of Edgeworthexpansions and their block bootstrap versions. We adopt the smooth func-tion model as described by G¨otze and K¨unsch (1996) under the assumption(A5) H : R d → R is four times continuously differentiable with non-vanishinggradient at µ and fourth-order derivatives at x ∈ R d bounded in mag-nitude by C (1 + k x k D ) for fixed constants C, D > x = ( x (1) , . . . , x ( d ) ) for each x ∈ R d . Define, for r , r , . . . = 1 , . . . , d and i , i , . . . = 0 , , , . . . , γ r ,r ,...i ,i ,... = E (cid:2) ( X − µ ) ( r ) ( X i − µ ) ( r ) ( X i − µ ) ( r ) · · · (cid:3) . For r, s, . . . = 1 , . . . , d , define H r = ( ∂/∂x ( r ) ) H ( x ) (cid:12)(cid:12) x = µ , H rs = ( ∂ /∂x ( r ) ∂x ( s ) ) H ( x ) (cid:12)(cid:12) x = µ , etc. Under con-ditions (A1)–(A4), we can expand the variance-covariance matrix of S n = n − / P ni =1 ( X i − µ ) such that Cov (cid:16) S ( r ) n , S ( s ) n (cid:17) = χ r,s , + n − χ r,s , + O ( n − ), r, s = 1 , . . . , d , for constants χ r,s , and χ r,s , not depending on n . In particular,we have χ r,s , = P ∞ i = −∞ γ r,si . Define σ = Var (cid:16)P dr =1 H r S ( r ) n (cid:17) , which, underthe above conditions, is positive and has order O (1). Let φ ( · ) and z ξ be thestandard normal density function and ξ th quantile respectively.11ur main theorem below derives expansions for the coverage probabilitiesof the various block bootstrap upper confidence bounds. Theorem 1
Let { X i : −∞ < i < ∞} be a strictly stationary, discrete-time, stochastic process with finite mean µ = E [ X ] . Let α ∈ (0 , be fixed.Assume that conditions (A1)–(A5) hold. Then,(i) for ℓ = O ( n / ) and ℓ/n ǫ → ∞ for some ǫ ∈ (0 , , P ( θ ≤ I ( α ))= α + ℓ − − σ − z α φ ( z α ) d X r,s =1 H r H s χ r,s , − n − / − σ − z α φ ( z α ) × ( d X r,s,t =1 H r H s H t ∞ X i,j = −∞ γ r,s,ti,j + 2 d X r,s,t,u =1 H r H s H tu χ r,t , χ s,u , ) + O (cid:0) ℓ − + ℓn − (cid:1) ; (1) (ii) for k ≤ ℓ = O ( n / ) and k/n ǫ → ∞ for some ǫ ∈ (0 , , the confidencelimits I C ( α ) and I S ( α ) differ by O p (cid:0) k − n − / + ℓn − / (cid:1) and have cov-erage probability α + (2 ℓ − − k − )2 − σ − z α φ ( z α ) d X r,s =1 H r H s χ r,s , + O (cid:0) k − + ℓn − (cid:1) . (2)It is clear from Theorem 1 that I ( α ) has coverage error of order O ( ℓ − + ℓn − ),which can be reduced by either coverage calibration or Studentization to O ( ℓ − + ℓn − ) if we set k = ℓ/
2. Heuristically, a chief source of coverageerror of I ( α ) stems from the large bias, of order 1 /ℓ , of the block bootstrap12ariance estimator. The second-level block bootstrap variance estimator hasleading bias of order 1 /k − /ℓ when viewed as an estimator of the first-levelblock bootstrap variance estimate. Existence of such second-level bias termenables either the coverage calibration or Studentization strategies to auto-matically offset the first-level bias of order 1 /ℓ , provided that k is set to ℓ/ ℓ and k for achieving the best coverage error rates. We seefrom (1) that, in the absence of coverage calibration or Studentization, theoptimal block length ℓ should have order n / in order to yield the smallestcoverage error, of order O ( n − / ), for I ( α ). With k = ℓ/ ℓ ∝ n / ,the coverage error of both I C ( α ) and I S ( α ) has order O ( n − / ), a signifi-cant improvement over that of the unmodified I ( α ). The following corollarysummarizes the above results. Corollary 1
Under the conditions of Theorem 1,(i) I ( α ) has coverage error of order O ( n − / ) , achieved by setting ℓ ∝ n / ;(ii) I C ( α ) and I S ( α ) are asymptotically equivalent up to order O p ( n − / ) and have coverage error of order O ( n − / ) , achieved by setting k = ℓ ∝ n / . Corollary 1 confirms that second-order correction of the block bootstrap in-terval can be achieved by straightforward application of either coverage cal-13bration or Studentization. Previous approaches proposed in the literatureto such second-order correction rely invariably on explicit computation ofa non-trivial expression of the Studentizing factor, which must be analyti-cally derived for each smooth function model under study. See, for example,H¨ardle, Horowitz and Kreiss (2003) for a review of such approaches. At theexpense of computational efficiency incurred by the double bootstrap proce-dure, calculation of I C ( α ) or I S ( α ) involves no analytic formula and can becarried out by brute force Monte Carlo simulation. Perhaps surprising is theextremely simple relationship ( k = ℓ/
2) between the optimal first-level andsecond-level block lengths, which relieves us of the notoriously difficult taskof determining the best block length for the double block bootstrap, in so faras the selection of k is concerned. We conducted a simulation study to investigate the empirical performanceof I C ( α ) and I S ( α ) in comparison with I ( α ). Two other Studentized blockbootstrap confidence bounds, based on constructions of Davison and Hall(1993) and G¨otze and K¨unsch (1996) and denoted by I DH ( α ) and I GK ( α )respectively, were also included in the study for reference: see Appendix 6.2for details of these two latter approaches. Time series data were generated14nder the following three models:(a) ARCH(1) process: X i = e i (1 + 0 . X i − ) / ,(b) MA(1) process: X i = e i + 0 . e i − ,(c) AR(1) process: X i = 0 . X i − + e i ,where the e i are independent N (0 ,
1) variables. The parameter θ was taken tobe the mean, variance and lag 1 autocorrelation, and the nominal level α wasset to be 0 .
05, 0.10, 0.90 and 0.95. For each method, the coverage probabilityof the level α upper confidence bound was approximated by averaging over1000 independent time series of length n = 500 and 1000. Construction ofeach confidence bound was based on 1000 first-level block bootstrap seriesusing block length ℓ = h n / i , in addition to which 1000 second-level seriesbased on block length k = h ℓ/ i were generated from each first-level series toconstruct I C ( α ) and I S ( α ). Specifically, we have ( ℓ, k ) = (8 ,
4) and (10 , n = 500 and 1000 respectively. The constant c was set to be 0.5 in thecalculation of the Studentizing factor for I GK ( α ): see Appendix 6.2.The coverage results are given in Tables 1–3 for the mean, the varianceand the lag 1 autocorrelation cases respectively. In general, coverage calibra-tion and all three Studentization methods succeed in reducing coverage errorof I ( α ) when the latter is noticeably inaccurate such as for θ = Var( X ). Ourproposed I C ( α ) and I S ( α ) either outperform or are comparable to I DH ( α )15nd I GK ( α ) in the variance and lag 1 autocorrelation cases. Note that I GK ( α )is exceptionally poor for small α in the autocorrelation case. All five confi-dence bounds have similar performance when θ = E [ X ]. We have proposed two double bootstrap approaches, one for calibrating thenominal coverage and the other for calculating the Studentizing factor, toimproving accuracy of the block bootstrap confidence interval. The mainadvantage of the proposed approaches lies in the ease with which the second-level block length k can be determined, namely half the first-level blocklength, and the Studentizing factor can be computed, essentially by a trivialapplication of the plug-in principle. Not in the literature has the same degreeof improvement been achieved without analytic derivation of the Studentiz-ing factor in a highly problem-specific manner. The problem of empiricaldetermination of the first-level block length ℓ has been dealt with by var-ious authors but methods which have proven satisfactory performance arenot yet available. Both theoretical and empirical findings suggest that ourproposed coverage calibration or Studentization approaches are effective inreducing coverage error even in the absence of a sophisticated data-basedscheme for selecting ℓ in the confidence procedure. While implementation16f the approaches is analytically effortless, the only price to pay is the extracomputational cost induced by the second level of block bootstrapping.Although our focus is confined to the smooth function model setting, itis believed that similar results extend also to von Mises-type functionals aswell as to estimating functions, after appropriate modifications of the proofof our main theorem. Extension to dependence structures outside the presentframework, such as series exhibiting long-range dependence, is less trivial andworth investigating in future studies. We first state a few lemmas concerning moments of centred sums of stationaryobservations and their bootstrap counterparts.Define S ∗ n ≡ ( bℓ ) / (cid:0) ¯ X ∗ − E ∗ ¯ X ∗ (cid:1) and S ∗∗ n ≡ ( ck ) / (cid:0) ¯ X ∗∗ − E ∗∗ ¯ X ∗∗ (cid:1) . De-fine, for i = 0 , ± , . . . and r = 1 , , . . . , Z i = X i − µ and V i,r = r − / P i + r − s = i Z s . Lemma 1
Under the conditions of Theorem 1, we have, for r = 1 , , , and s , s , . . . , s r = 1 , . . . , d , Var (cid:16)P n ′ i =1 V ( s ) i,ℓ · · · V ( s r ) i,ℓ /n ′ (cid:17) = O ( ℓn − ) . Lemma 1 follows immediately from Lemma 3.1 of Lahiri (2003, Section 3.2.1).17 emma 2
Under the conditions of Theorem 1, we have, for r, s, t, u =1 , . . . , d and as m → ∞ , E h V ( r )1 ,m V ( s )1 ,m i = χ r,s , + m − χ r,s , + O ( m − ) , E h V ( r )1 ,m V ( s )1 ,m V ( t )1 ,m i = m − / χ r,s,t , + O ( m − / ) , E h V ( r )1 ,m V ( s )1 ,m V ( t )1 ,m V ( u )1 ,m i = χ r,s,t,u , + O ( m − ) , where χ r,s , , χ r,s , and χ r,s,t , are constants independent of m , and χ r,s,t,u , = χ r,s , χ t,u , + χ r,t , χ s,u , + χ r,u , χ s,t , . Lemma 2 can be established using arguments similar to those for proving theunivariate case: see G¨otze and Hipp (1983).A generic first-level block bootstrap series X ∗ can be represented as theordered sequence of observations in ( Y N ,ℓ , . . . , Y N b ,ℓ ), where N , . . . , N b areindependent random variables uniformly distributed over { , , . . . , n ′ } . De-fine, for i = 0 , ± , . . . , r = 1 , , . . . and s , s , . . . , s r = 1 , . . . , d , Q s ,...,s r i = ℓ ′ X j =1 V ( s ) i + j − ,k · · · V ( s r ) i + j − ,k /ℓ ′ − E h V ( s )1 ,k · · · V ( s r )1 ,k i ,Q s ,...,s r = ( n ′ ) − n ′ X i =1 Q s ,...,s r i , ˜ Q s ,...,s r = b − b X j =1 Q s ,...,s r N j − Q s ,...,s r and P s ,...,s r = P n ′ i =1 V ( s ) i,ℓ · · · V ( s r ) i,ℓ /n ′ − E h V ( s )1 ,ℓ · · · V ( s r )1 ,ℓ i . Write ˘ ℓ = min( ℓ ′ , k )and ¯ ℓ = max( ℓ ′ , k ). 18 emma 3 Under the conditions of Theorem 1, we have Q s ,...,s r = O p (cid:0) n − / k / (cid:1) and ˜ Q s ,...,s r = O p (cid:16) n − / ( ℓ ˘ ℓ/ℓ ′ ) / (cid:17) for r = 1 , , , and s , s , . . . , s r =1 , . . . , d .Proof of Lemma 3 .For r, s, . . . = 1 , . . . , d and −∞ < i , i , . . . < ∞ , write ξ r,s,...i ,i ,... = Z ( r ) i Z ( s ) i · · · and ¯ ξ r,s,...i ,i ,... = ξ r,s,...i ,i ,... − E ξ r,s,...i ,i ,... . Consider first E ( Q s ,...,s r ) = k − r ( ℓ ′ ) − ℓ ′ X i,j =1 k − X i ,...,i r ,j ,...,j r =0 E (cid:2) ¯ ξ s ,...,s r i + i ,...,i + i r ¯ ξ s ,...,s r j + j ,...,j + j r (cid:3) = O ( k − r +1 ( ℓ ′ ) − ℓ ′ X j =1 k − X i ,...,i r ,j ,...,j r =0 E (cid:2) ¯ ξ s ,...,s r ,i ,...,i r ¯ ξ s ,...,s r j + j ,...,j + j r (cid:3)) , which follows by stationarity of the series { Z j : −∞ < j < ∞} and abackward shift of i + i units. Under the assumed mixing conditions, the lastexpectation has order O ( n − K ) for arbitrarily large K if the observations in¯ ξ s ,...,s r ,i ,...,i r and ¯ ξ s ,...,s r j + j ,...,j + j r are at least K log n units apart. We can thereforerestrict, up to O ( n − K ), the first sum to that over j = 1 , . . . , ˘ ℓ , so that E ( Q s ,...,s r ) has order O ( k − r +1 ( ℓ ′ ) − ˘ ℓ k − X i ,...,i r ,j ,...,j r =0 E (cid:12)(cid:12) ξ s ,...,s r ,s ,...,s r ,i ,...,i r ,j ,...,j r (cid:12)(cid:12)) = O ( ( ℓ ′ ) − ˘ ℓ max p,q ∈{ s ,...,s r } ∞ X j = −∞ E (cid:12)(cid:12) ξ p,q ,j (cid:12)(cid:12)! r ) = O (cid:16) ˘ ℓ/ℓ ′ (cid:17) . (3)Noting thatVar ∗ ( ˜ Q s ,...,s r ) = ( bn ′ ) − n ′ X i =1 ( Q s ,...,s r i − Q s ,...,s r ) = O p (cid:16) E ( Q s ,...,s r ) /b (cid:17) ,
19e have ˜ Q s ,...,s r = O p (cid:16) b − / (˘ ℓ/ℓ ′ ) / (cid:17) = O p (cid:16) n − / ( ℓ ˘ ℓ/ℓ ′ ) / (cid:17) .Using similar arguments, we see that E ( Q s ,...,s r ) = O ( k − r +1 ( n ′ ) − ℓ − X t =0 k − X i ,...,i r ,j ,...,j r =0 E (cid:2) ¯ ξ s ,...,s r ,i ,...,i r ¯ ξ s ,...,s r t + j ,...,t + j r (cid:3)) = O ( k − r +2 ( n ′ ) − k − X i ,...,i r ,j ,...,j r =0 E (cid:12)(cid:12) ξ s ,...,s r ,s ,...,s r ,i ,...,i r ,j ,...,j r (cid:12)(cid:12)) = O ( k/n ′ ) , so that Q s ,...,s r = O p (cid:0) ( k/n ′ ) / (cid:1) . Lemma 4
Under the conditions of Theorem 1, we have, for r, s, t, u =1 , . . . , d , E ∗ (cid:2) S ∗ ( r ) n S ∗ ( s ) n (cid:3) = E (cid:2) S ( r ) n S ( s ) n (cid:3) + P r,s + ℓ − χ r,s , + O p ( ℓn − + ℓ − ) , E ∗ (cid:2) S ∗ ( r ) n S ∗ ( s ) n S ∗ ( t ) n (cid:3) = E (cid:2) S ( r ) n S ( s ) n S ( t ) n (cid:3) + O p ( ℓn − + ℓ − n − / ) , E ∗ (cid:2) S ∗ ( r ) n S ∗ ( s ) n S ∗ ( t ) n S ∗ ( u ) n (cid:3) = E (cid:2) S ( r ) n S ( s ) n S ( t ) n S ( u ) n (cid:3) + O p ( ℓ / n − / + ℓ − ) . Proof of Lemma 4 .Note first that, by Lemma 1, P r , P r,s , P r,s,t and P r,s,t,u have order O p ( ℓ / n − / )for r, s, t, u = 1 , . . . , d . Lemma 2 then implies that E ∗ (cid:2) S ∗ ( r ) n S ∗ ( s ) n (cid:3) = P r,s + E h V ( r )1 ,ℓ V ( s )1 ,ℓ i − P r P s = P r,s + χ r,s , + ℓ − χ r,s , + O p ( ℓn − + ℓ − ) . By Lemma 2 again, we have E h S ( r ) n S ( s ) n i = χ r,s , + O ( n − ) and the first resultfollows. 20imilarly, the second and third results follow by noting Lemma 2 and that E ∗ (cid:2) S ∗ ( r ) n S ∗ ( s ) n S ∗ ( t ) n (cid:3) = b − / E h V ( r )1 ,ℓ V ( s )1 ,ℓ V ( t )1 ,ℓ i + O p ( b − / ℓ / n − / )= n − / χ r,s,t , + O p ( ℓ − n − / + ℓn − )and E ∗ (cid:2) S ∗ ( r ) n S ∗ ( s ) n S ∗ ( t ) n S ∗ ( u ) n (cid:3) = b − (cid:8) χ r,s,t,u , + O ( ℓ − ) + O p ( ℓ / n − / ) (cid:9) + (1 − b − ) (cid:8) χ r,s , χ t,u , + χ r,t , χ s,u , + χ r,u , χ s,t , + O p ( ℓ − + ℓ / n − / ) (cid:9) = χ r,s,t,u , + O p ( ℓ − + ℓ / n − / ) . A generic second-level block bootstrap series X ∗∗ can be identified as theordered sequence of observations in ( Y ∗ I ,J ,k , . . . , Y ∗ I c ,J c ,k ) = ( Y N I + J ,k , . . . , Y N Ic + J c ,k ),where the I j and J j are independent random numbers distributed uniformlyover { , , . . . , b } and { , , . . . , ℓ ′ } respectively, both independently of ( N , . . . , N b ).Thus we can write S ∗∗ n = c − / P ci =1 V N Ii + J i − ,k − c / ( bℓ ′ ) − P bi =1 P ℓ ′ j =1 V N i + j − ,k . Lemma 5
Under the conditions of Theorem 1, we have, for r, s, t, u =21 , . . . , d , E ∗∗ (cid:2) S ∗∗ ( r ) n S ∗∗ ( s ) n (cid:3) = E ∗ (cid:2) S ∗ ( r ) n S ∗ ( s ) n (cid:3) − P r,s + ˜ Q r,s + Q r,s + ( k − − ℓ − ) χ r,s , + O p ( ℓn − + k − ) , E ∗∗ (cid:2) S ∗∗ ( r ) n S ∗∗ ( s ) n S ∗∗ ( t ) n (cid:3) = E ∗ (cid:2) S ∗ ( r ) n S ∗ ( s ) n S ∗ ( t ) n (cid:3) + O p ( ℓn − + k − n − / ) , E ∗∗ (cid:2) S ∗∗ ( r ) n S ∗∗ ( s ) n S ∗∗ ( t ) n S ∗∗ ( u ) n (cid:3) = E ∗ (cid:2) S ∗ ( r ) n S ∗ ( s ) n S ∗ ( t ) n S ∗ ( u ) n (cid:3) + O p ( ℓ / n − / + k − ) . Proof of Lemma 5 .It follows from Lemmas 2 and 3 that E ∗∗ (cid:2) S ∗∗ ( r ) n S ∗∗ ( s ) n (cid:3) = ˜ Q r,s + Q r,s + E h V ( r )1 ,k V ( s )1 ,k i − (cid:16) ˜ Q r + Q r (cid:17) (cid:16) ˜ Q s + Q s (cid:17) = ˜ Q r,s + Q r,s + E (cid:2) S ( r ) n S ( s ) n (cid:3) + k − χ r,s , + O p (cid:16) k − + n − k + n − ℓ ˘ ℓ/ℓ ′ (cid:17) . The first result then follows by subtracting the expression for E ∗ h S ∗ ( r ) n S ∗ ( s ) n i stated in Lemma 4.Similar arguments show that E ∗∗ (cid:2) S ∗∗ ( r ) n S ∗∗ ( s ) n S ∗∗ ( t ) n (cid:3) = c − / n E h V ( r )1 ,k V ( s )1 ,k V ( t )1 ,k i + O p ( ˜ Q r,s,t + Q r,s,t ) o = E (cid:2) S ( r ) n S ( s ) n S ( t ) n (cid:3) + O p (cid:16) k − n − / + kn − + n − ( kℓ ˘ ℓ/ℓ ′ ) / (cid:17) E ∗∗ (cid:2) S ∗∗ ( r ) n S ∗∗ ( s ) n S ∗∗ ( t ) n S ∗∗ ( u ) n (cid:3) = χ r,s , χ t,u , + χ r,t , χ s,u , + χ r,u , χ s,t , + O p (cid:16) k − + k / n − / + n − / ( ℓ ˘ ℓ/ℓ ′ ) / (cid:17) , which, on subtracting E ∗ h S ∗ ( r ) n S ∗ ( s ) n S ∗ ( t ) n i and E ∗ h S ∗ ( r ) n S ∗ ( s ) n S ∗ ( t ) n S ∗ ( u ) n i as ex-pressed in Lemma 4, yield the other two results.Set L n = K log n for some large K >
0. For r, s, . . . = 1 , . . . , d and −∞ < p, q, i, i , i , . . . < ∞ , recall the definitions of ξ r,s,...i ,i ,... and ¯ ξ r,s,...i ,i ,... in theproof of Lemma 3, and split the sum S n = S i,p,q + ¯ S i,p,q = S i,p + ¯ S i,p such that S i,p,q = n − / X | j − ( i + p ) |∨| j − ( i + q ) |≤ L n Z j and S i,p = n − / X | j − ( i + p ) |≤ L n Z j . Lemma 6
Under the conditions of Theorem 1, we have, for r, s, t, u =1 , . . . , d , ( n ′ ℓ ) − n ′ X i =1 ℓ − X p =0 Z ( r ) i + p S ( s ) i,p = n − / χ r,s , + O p ( ℓn − / + n − L / n ) , (4)( n ′ ℓ ) − n ′ X i =1 ℓ − X p =0 Z ( r ) i + p S ( s ) i,p S ( t ) i,p = O p ( n − ) , (5)( n ′ ℓ ) − n ′ X i =1 ℓ − X p,q =0 ¯ ξ r,si + p,i + q S ( t ) i,p,q = n − / ∞ X i,j = −∞ γ r,s,ti,j + O p ( ℓ − n − / L n + ℓn − ) , (6)23 n ′ ℓ ) − n ′ X i =1 ℓ − X p,q =0 ¯ ξ r,si + p,i + q S ( t ) i,p,q S ( u ) i,p,q = O p ( ℓn − ) . (7) Proof of Lemma 6 .We outline the proof of (6) and (7); that of (4) and (5) follows by similar,albeit simpler, arguments.Consider first Π r,st ≡ P n ′ i =1 P ℓ − p,q =0 P ( i + p,i + q ) j ¯ ξ r,si + p,i + q Z ( t ) j , where P ( i ,i ) j denotes summation over j satisfying | j − i | ∨ | j − i | ≤ L n . Note that thevariance of Π r,st has leading term n ′ ℓ X | q |≤ ℓ X j (0 ,q ) X | i ′ |≤ n ′ X | p ′ |∨| q ′ |≤ ℓ X j ′ ( i ′ + p ′ ,i ′ + q ′ ) E (cid:16) ¯ ξ r,s ,q Z ( t ) j − E [ ¯ ξ r,s ,q Z ( t ) j ] (cid:17) (cid:16) ¯ ξ r,si ′ + p ′ ,i ′ + q ′ Z ( t ) j ′ − E [ ¯ ξ r,si ′ + p ′ ,i ′ + q ′ Z ( t ) j ′ ] (cid:17) ∼ n ′ ℓ X | q |≤ ℓ X j (0 ,q ) X | p ′ |∨| q ′ |≤ ℓ X j ′ ( p ′ ,q ′ ) E (cid:16) ¯ ξ r,s ,q Z ( t ) j − E [ ¯ ξ r,s ,q Z ( t ) j ] (cid:17) (cid:16) ¯ ξ r,sp ′ ,q ′ Z ( t ) j ′ − E [ ¯ ξ r,sp ′ ,q ′ Z ( t ) j ′ ] (cid:17) = O n ′ ℓ X | q |≤ ℓ X j (0 ,q ) X | p ′ |∨| q ′ |≤ ℓ X j ′ ( p ′ ,q ′ ) E (cid:12)(cid:12) ξ r,s,t,r,s,t ,q,j,p ′ ,q ′ ,j ′ (cid:12)(cid:12) = O n ′ ℓ max u,v ∈{ r,s,t } ∞ X j = −∞ E (cid:12)(cid:12) ξ u,v ,j (cid:12)(cid:12)! = O ( ℓ n ) , using stationarity properties and the fact that if both i ′ + p ′ and i ′ + q ′ differby at least 3 L n from 0 and q , then E (cid:16) ¯ ξ r,s ,q Z ( t ) j − E [ ¯ ξ r,s ,q Z ( t ) j ] (cid:17) (cid:16) ¯ ξ r,si ′ + p ′ ,i ′ + q ′ Z ( t ) j ′ − E [ ¯ ξ r,si ′ + p ′ ,i ′ + q ′ Z ( t ) j ′ ] (cid:17) = O ( n − K )for arbitrarily large K > r,st has mean( n + O ( ℓ )) ℓ − X p =0 ℓ − − p X q = − p X j (0 ,q ) γ r,s,tq,j = ( n + O ( ℓ ))( ℓ + O ( L n )) X | q |≤ L n X | j − q |≤ L n γ r,s,tq,j . It follows that Π r,st has expansion nℓ P ∞ i,j = −∞ γ r,s,ti,j + O ( nL n ) + O p ( ℓ n / ),which yields (6) on multiplying it by n − / ( n ′ ℓ ) − .Consider next Π r,st,u ≡ P n ′ i =1 P ℓ − p,q =0 P ( i + p,i + q ) j ,j ¯ ξ r,si + p,i + q ξ t,uj ,j , which has meanof order n ′ ℓ X | q |≤ ℓ X j ,j (0 ,q ) (cid:12)(cid:12) E (cid:2) ¯ ξ r,s ,q ξ t,uj ,j (cid:3)(cid:12)(cid:12) = O ( n ′ ℓ ∞ X j = −∞ E (cid:12)(cid:12) ξ r,t ,j (cid:12)(cid:12) ∞ X i = −∞ E (cid:12)(cid:12) ξ s,u ,i (cid:12)(cid:12)) = O ( ℓ n ) , and variance of order n ′ ℓ X | q |≤ ℓ X j ,j (0 ,q ) X | p ′ |∨| q ′ |≤ ℓ X j ′ ,j ′ ( p ′ ,q ′ ) E (cid:0) ¯ ξ r,s ,q ξ t,uj ,j − E [ ¯ ξ r,s ,q ξ t,uj ,j ] (cid:1) (cid:16) ¯ ξ r,sp ′ ,q ′ ξ t,uj ′ ,j ′ − E [ ¯ ξ r,sp ′ ,q ′ ξ t,uj ′ ,j ′ ] (cid:17) = O n ′ ℓ X | q | , | p ′ | , | q ′ |≤ ℓ X j ,j (0 ,q ) X j ′ ,j ′ ( p ′ ,q ′ ) E (cid:12)(cid:12)(cid:12) ξ r,s,t,u,r,s,t,u ,q,j ,j ,p ′ ,q ′ ,j ′ ,j ′ (cid:12)(cid:12)(cid:12) = O ( ℓ n ) . Thus (7) follows by multiplying Π r,st,u by ( n ′ ℓ ) − n − .Consider next the decomposition S n = S i,j,p,q + ¯ S i,j,p,q = S i,j,p + ¯ S i,j,p suchthat S i,j,p,q = n − / P ( i + j − p,i + j − q ) t Z t and S i,j,p = n − / P | t − ( i + j − p ) |≤ L n Z t ,for −∞ < p, q, i, j < ∞ . Arguments similar to those for proving Lemma 6can be used to establish: Lemma 7
Under the conditions of Theorem 1, we have, for r, s, t, u =25 , . . . , d , ( n ′ ℓ ′ k ) − n ′ X i =1 ℓ ′ X j =1 k − X p =0 Z ( r ) i + j − p S ( s ) i,j,p = n − / χ r,s , + O p ( ℓn − / + n − L / n ) , ( n ′ ℓ ′ k ) − n ′ X i =1 ℓ ′ X j =1 k − X p =0 Z ( r ) i + j − p S ( s ) i,j,p S ( t ) i,j,p = O p ( n − ) , ( n ′ ℓ ′ k ) − n ′ X i =1 ℓ ′ X j =1 k − X p,q =0 ¯ ξ r,si + j − p,i + j − q S ( t ) i,j,p,q = n − / ∞ X i,j = −∞ γ r,s,ti,j + O p ( k − n − / L n + kn − + ℓn − / ) , ( n ′ ℓ ′ k ) − n ′ X i =1 ℓ ′ X j =1 k − X p,q =0 ¯ ξ r,si + j − p,i + j − q S ( t ) i,j,p,q S ( u ) i,j,p,q = O p ( kn − ) . We now proceed with the proof of Theorem 1.Define H r ( x ) = ( ∂/∂x ( r ) ) H ( x ), H rs = ( ∂ /∂x ( r ) ∂x ( s ) ) H ( x ), etc., for r, s, . . . = 1 , . . . , d . Recall that we write H r = H r ( µ ), H rs = H rs ( µ ), etc. forconvenience. Note that ˆ µ ( r ) ≡ E ∗ ¯ X ∗ ( r ) = ℓ − / P r + µ ( r ) . Write ˆ H r = H r (ˆ µ ),ˆ H rs = H rs (ˆ µ ), etc. Taylor expansion shows that Var( n / ˆ θ ) has leadingterm σ = P dr,s =1 H r H s E h S ( r ) n S ( s ) n i . Define ˆ σ = P dr,s =1 ˆ H r ˆ H s E ∗ h S ∗ ( r ) n S ∗ ( s ) n i ,which can, by Lemmas 2 and 4, be Taylor expanded to giveˆ σ = σ + d X r,s =1 H r H s (cid:0) ℓ − χ r,s , + P r,s (cid:1) +2 ℓ − / d X r,s,t =1 H r H st χ r,s , P t + O p ( ℓn − + ℓ − ) . (8)Lahiri (2003, Section 6.4.3) provides an Edgeworth expansion for the distri-bution function G of n / (ˆ θ − θ ): G ( x ) = Φ( x/σ ) − n − / (cid:2) K + K ( x /σ − (cid:3) φ ( x/σ ) + O ( n − ) , (9)26here K and K are smooth functions, both of order O (1), of the moments E h S ( s ) n · · · S ( s r ) n i , for s , . . . , s r = 1 , . . . , d and r = 2 , ,
4, and Φ denotes thestandard normal distribution function. Lahiri’s (2003) Theorem 6.7 derivesa block bootstrap version of (9) under the conditions of our Theorem 1: G ∗ ( x ) = Φ( x/ ˆ σ ) − n − / h ˆ K + ˆ K ( x / ˆ σ − i φ ( x/ ˆ σ ) + O p ( ℓn − ) , (10)where ˆ K and ˆ K have the same expressions as K and K with the popu-lation moments E h S ( s ) n · · · S ( s r ) n i replaced by E ∗ h S ∗ ( s ) n · · · S ∗ ( s r ) n i . With theaid of Lemma 4 and the expressions (8), (9) and (10), we can expand thedifference between G ∗− and G − , so that, for ξ ∈ (0 , P (cid:16) n / (ˆ θ − θ ) ≤ G ∗− ( ξ ) (cid:17) = P ( T n ≤ y ) + O ( ℓn − + ℓ − ) , (11)where T n = n / (ˆ θ − θ ) − z ξ (2 σ ) − (cid:16)P dr,s =1 H r H s P r,s + 2 ℓ − / P dr,s,t =1 H r H st χ r,s , P t (cid:17) and y = G − ( ξ ) + ℓ − z ξ (2 σ ) − P dr,s =1 H r H s χ r,s , . Noting that P r and P r,s are O p ( ℓ / n − / ) by Lemma 3, that n / (ˆ θ − θ ) = P du =1 H u S ( u ) n + O p ( n − / ) andexpanding the characteristic function of T n about that of n / (ˆ θ − θ ), we get,for β ∈ R , E e ιβT n − E e ιβn / (ˆ θ − θ ) = − ιβz ξ (2 σ ) − d X r,s =1 H r H s E " P r,s exp ιβ d X u =1 H u S ( u ) n ! − ιβz ξ σ − ℓ − / d X r,s,t =1 H r H st χ r,s , E " P t exp ιβ d X u =1 H u S ( u ) n ! + O ( ℓn − ) . (12)27ote that for s , s , s = 1 , . . . , d ,( n ′ ℓ ) − (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) n ′ X i =1 ℓ − X p,q =0 E h ¯ ξ r,si + p,i + q S ( s ) i,p,q S ( s ) i,p,q S ( s ) i,p,q i(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) ≤ ( n ′ ℓ ) − n − / n ′ X i =1 ℓ − X p,q =0 X i ,i ,i ( i + p,i + q ) E (cid:12)(cid:12) ¯ ξ r,si + p,i + q ξ s ,s ,s i ,i ,i (cid:12)(cid:12) = O n − / X | q |≤ ℓ X i ,i ,i (0 ,q ) E (cid:12)(cid:12) ¯ ξ r,s ,q ξ s ,s ,s i ,i ,i (cid:12)(cid:12) = O ( ℓn − / L n ) . (13)It follows by expansion of the exponential function, (6), (7) and (13) that E " P r,s exp ιβ d X u =1 H u S ( u ) n ! = ( n ′ ℓ ) − n ′ X i =1 ℓ − X p,q =0 E " ¯ ξ r,si + p,i + q exp ιβ d X u =1 H u S ( u ) n !( ιβ d X u =1 H u S ( u ) i,p,q + 2 − β d X t,u =1 H t H u S ( t ) i,p,q S ( u ) i,p,q + exp − ιβ d X u =1 H u S ( u ) i,p,q !) + O ( ℓn − / L n )= ( n ′ ℓ ) − n ′ X i =1 ℓ − X p,q =0 E " ¯ ξ r,si + p,i + q exp ιβ d X u =1 H u ¯ S ( u ) i,p,q ! + ιβn − / d X t =1 H t ∞ X i,j = −∞ γ r,s,ti,j E h e ιβn / (ˆ θ − θ ) i + O ( ℓ − n − / L n + ℓn − )= ιβn − / d X t =1 H t ∞ X i,j = −∞ γ r,s,ti,j E h e ιβn / (ˆ θ − θ ) i + O ( ℓ − n − / L n + ℓn − ) . (14)The last equality follows by the assumed mixing properties and noting thatobservations defining ¯ S ( u ) i,p,q and ¯ ξ r,si + p,i + q are at least L n units apart on the seriesand that E ¯ ξ r,si + p,i + q = 0. Noting that ( n ′ ℓ ) − (cid:12)(cid:12)(cid:12)P n ′ i =1 P ℓ − p =0 E h Z ( r ) i + p S ( s ) i,p S ( s ) i,p S ( s ) i,p i(cid:12)(cid:12)(cid:12) =28 ( n − / L n ) for s , s , s = 1 , . . . , d , the same arguments show that ℓ − / E " P t exp ιβ d X u =1 H u S ( u ) n ! = ιβn − / d X u =1 H u χ t,u , E h e ιβn / (ˆ θ − θ ) i + O ( ℓn − / + n − L / n ) . (15)Substitution of (14) and (15) into (12) gives E e ιβT n / E e ιβn / (ˆ θ − θ ) = 1 + n − / (2 σ ) − β z ξ d X r,s,t =1 H r H s H t ∞ X i,j = −∞ γ r,s,ti,j + n − / σ − β z ξ d X r,s,t,u =1 H r H u H st χ r,s , χ t,u , + O ( ℓn − + ℓ − n − / L n ) . (16)It follows by inverse Fourier-transforming E e ιβT n that P ( T n ≤ x ) = G ( x ) + n − / − σ − z ξ xφ ( x/σ ) × ( d X r,s,t =1 H r H s H t ∞ X i,j = −∞ γ r,s,ti,j + 2 d X r,s,t,u =1 H r H u H st χ r,s , χ t,u , ) + O ( ℓn − + ℓ − n − / L n ) . (17)It then follows by combining (11) and (17), setting x = y and noting that y = σz ξ + O ( n − / + ℓ − ) that P (cid:16) n / (ˆ θ − θ ) ≤ G ∗− ( ξ ) (cid:17) = ξ + ℓ − − σ − z ξ φ ( z ξ ) d X r,s =1 H r H s χ r,s , + n − / − σ − z ξ φ ( z ξ ) × ( d X r,s,t =1 H r H s H t ∞ X i,j = −∞ γ r,s,ti,j + 2 d X r,s,t,u =1 H r H u H st χ r,s , χ t,u , ) + O ( ℓn − + ℓ − ) , ξ = 1 − α and taking complement.For proving (2), write µ ∗ = E ∗∗ ¯ X ∗∗ , H ∗ r = H r ( µ ∗ ), H ∗ rs = H rs ( µ ∗ ) etc.and define σ ∗ = P dr,s =1 H ∗ r H ∗ s E ∗∗ h S ∗∗ ( r ) n S ∗∗ ( s ) n i . Note that, for r = 1 , . . . , d , µ ∗ ( r ) − ˆ µ ( r ) = k − / (cid:16) ˜ Q r + Q r (cid:17) − ℓ − / P r = O p ( n − / ) by Lemmas 1 and 3.It follows by Lemma 5 and Taylor expansion that σ ∗ = ˆ σ + d X r,s =1 H r H s h ˜ Q r,s + Q r,s − P r,s + ( k − − ℓ − ) χ r,s , i + 2 d X r,s,t =1 H r H st χ r,s , h k − / (cid:16) ˜ Q t + Q t (cid:17) − ℓ − / P t i + O p ( n − ℓ + k − ) , (18)using the fact that ˆ µ = µ + O p ( n − / ). Denote by K ∗ and K ∗ the versions of K and K with the moments E h S ( s ) n · · · S ( s r ) n i replaced by E ∗∗ h S ∗∗ ( s ) n · · · S ∗∗ ( s r ) n i in their definitions. Thus, by analogy with (10), we have G ∗∗ ( x ) = Φ( x/σ ∗ ) − n − / (cid:2) K ∗ + K ∗ ( x /σ ∗ − (cid:3) φ ( x/σ ∗ )+ O p ( kn − ) . (19)The expansions (10), (18), (19) and the results in Lemma 5 enable us toexpand G ∗∗− ( ξ ) about G ∗− ( ξ ) and write P ∗ (cid:0) ( bℓ ) / [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] ≤ G ∗∗− ( ξ ) (cid:1) = P ∗ (cid:0) ( bℓ ) / [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] − ∆ ∗ n ≤ ˆ y (cid:1) + O p ( ℓn − + k − ) , (20)where ∆ ∗ n = b − P bj =1 R ∗ j , R ∗ j = (2ˆ σ ) − z ξ ( d X r,s =1 H r H s ( Q r,sN j − Q r,s ) + 2 k − / d X r,s,t =1 H r H st χ r,s , ( Q tN j − Q t ) ) y = G ∗− ( ξ ) + (2ˆ σ ) − z ξ ( d X r,s =1 H r H s (cid:2) Q r,s − P r,s + ( k − − ℓ − ) χ r,s , (cid:3) + 2 d X r,s,t =1 H r H st χ r,s , (cid:0) k − / Q t − ℓ − / P t (cid:1)) . Define also Y ∗ j = P dr =1 (cid:16) V ( r ) N j ,ℓ − P r (cid:17) ˆ H r for j = 1 , . . . , b , so that ( bℓ ) / [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] = b − / P bj =1 Y ∗ j + O p ( n − / ) by Taylor expansion. Note that theobservations ( Y ∗ j , R ∗ j ) are independent, zero-mean and identically distributedwith respect to first-level block bootstrap sampling, conditional on X . Wesee by Lemma 3 that ∆ ∗ n = O p ( k / n − / ) and by (3) that R ∗ j = O p ((˘ ℓ/ℓ ′ ) / ),whereas Y ∗ j = O p (1) by Lemmas 1 and 2. It follows that, conditional on X ,( bℓ ) / [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] − ∆ ∗ n and ( bℓ ) / [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] have iden-tical means, variances differing by − b − / E ∗ [ Y ∗ R ∗ ] + O p ( kn − ) and thirdcumulants differing by − b − E ∗ [ Y ∗ R ∗ ] + O p ( kn − ) = O p ( ℓn − ). Such cu-mulant differences can be employed to establish an Edgeworth expansion for( bℓ ) / [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] − ∆ ∗ n analogous to (10), bearing in mind that ˆ K and ˆ K stem from the first and third cumulants respectively: P ∗ (cid:0) ( bℓ ) / [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] − ∆ ∗ n ≤ x (cid:1) = G ∗ ( x ) + b − / ˆ σ − xφ ( x/ ˆ σ ) E ∗ [ Y ∗ R ∗ ] + O p ( ℓn − ) . (21)31ote by Lemmas 1, 2 and 3 that for r, s, t = 1 , . . . , d ,Cov ∗ (cid:16) V ( r ) N ,ℓ , Q s,tN (cid:17) = ( n ′ ℓ ′ ) − n ′ X i =1 ℓ ′ X j =1 V ( r ) i,ℓ V ( s ) i + j − ,k V ( t ) i + j − ,k − P r E [ V ( s )1 ,k V ( t )1 ,k ] − P r Q s,t = ( n ′ ℓ ′ k ) − ℓ − / n ′ X i =1 ℓ ′ X j =1 ℓ − X a =0 k − X p,q =0 ξ r,s,ti + a,i + j − p,i + j − q + O p ( ℓ / n − / ) . (22)Consider ℓ ′ X j =1 ℓ − X a =0 k − X p,q =0 E ξ r,s,ta,j − p,j − q = k − X p,q =0 ℓ − X a =1 − ℓ ′ { ( ℓ − a ) ∧ ℓ ′ − (1 − a ) ∨ } E ξ r,s,ta,p,q = X | p | , | q |≤ L n k + O ( L n ) X a = O ( L n ) { ( ℓ − a ) ∧ ℓ ′ − (1 − a ) ∨ } γ r,s,tp,q = (cid:8) kℓ ′ + O ( ℓ ′ L n + L n ) (cid:9) ∞ X p,q = −∞ γ r,s,tp,q andVar ( n ′ ) − n ′ X i =1 ℓ ′ X j =1 ℓ − X a =0 k − X p,q =0 ξ r,s,ti + a,i + j − p,i + j − q ! ∼ ( n ′ ) − kℓ ′ X | i ′ |≤ n ′ X | j ′ |≤ ℓ ′ X | a | , | a ′ |≤ ℓ X | q | , | p ′ | , | q ′ |≤ k E (cid:2) ¯ ξ r,s,ta, ,q ¯ ξ r,s,ti ′ + a ′ ,i ′ + j ′ − p ′ ,i ′ + j ′ − q ′ (cid:3) = O n − kℓ ′ ℓ X | j ′ |≤ ℓ ′ X | i ′ | , | a |≤ ℓ X | q | , | p ′ | , | q ′ |≤ k E (cid:12)(cid:12) ξ r,s,t,r,s,ta, ,q,i ′ ,i ′ + j ′ − p ′ ,i ′ + j ′ − q ′ (cid:12)(cid:12) = O n − ( kℓ ′ ℓ ) X | q |≤ k E (cid:12)(cid:12) ξ s,t ,q (cid:12)(cid:12) X | a |≤ ℓ E (cid:12)(cid:12) ξ r,ra, (cid:12)(cid:12) X | q ′ |≤ k E (cid:12)(cid:12) ξ s,t ,q ′ (cid:12)(cid:12) = O ( n − ( kℓ ′ ℓ ) ) ,
32o that ( n ′ ) − n ′ X i =1 ℓ ′ X j =1 ℓ − X a =0 k − X p,q =0 ξ r,s,ti + a,i + j − p,i + j − q = kℓ ′ ∞ X i,j = −∞ γ r,s,ti,j + O p ( ℓ ′ L n + L n + kℓ ′ ℓn − / ) . (23)Similar arguments show thatCov ∗ (cid:16) V ( r ) N ,ℓ , Q tN (cid:17) = ( k/ℓ ) / ∞ X a = −∞ γ r,ta + O p (cid:8) ( kℓ ) − / L n + ( ℓ ′ ) − ( kℓ ) − / L n + k / n − / (cid:9) . (24)Combining (22)–(24), we have E ∗ [ Y ∗ R ∗ ] = ℓ − / (2ˆ σ ) − z ξ ( d X r,s,t =1 H r H s H t ∞ X i,j = −∞ γ r,s,ti,j + 2 d X r,s,t,u =1 H r H s H tu χ r,t , χ s,u , ) + O p ( ℓ − / k − L n + ( kℓ ′ ) − ℓ − / L n + ℓ / n − / ) . (25)Substitution of (25) into (21), setting x = ˆ y and noting (20), we have P ∗ (cid:0) ( bℓ ) / [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] ≤ G ∗∗− ( ξ ) (cid:1) = ξ + 2 − σ − z ξ φ ( z ξ ) ( d X r,s =1 H r H s (cid:2) Q r,s − P r,s + ( k − − ℓ − ) χ r,s , (cid:3) + 2 d X r,s,t =1 H r H st χ r,s , (cid:0) k − / Q t − ℓ − / P t (cid:1)) + n − / − σ − z ξ φ ( z ξ ) ( d X r,s,t =1 H r H s H t ∞ X i,j = −∞ γ r,s,ti,j + 2 d X r,s,t,u =1 H r H s H tu χ r,t , χ s,u , ) + O p ( ℓn − + k − ) , α = α + δ n + B n + O p ( ℓn − + k − ), where δ n = − ( k − − ℓ − )2 − σ − z α φ ( z α ) d X r,s =1 H r H s χ r,s , + n − / − σ − z α φ ( z α ) × d X r,s,t =1 H r H s H t ∞ X i,j = −∞ γ r,s,ti,j + 2 d X r,s,t,u =1 H r H s H tu χ r,t , χ s,u , ! ,B n = − − σ − z α φ ( z α ) × ( d X r,s =1 H r H s ( Q r,s − P r,s ) + 2 d X r,s,t =1 H r H st χ r,s , (cid:0) k − / Q t − ℓ − / P t (cid:1)) . It follows from (11) that the coverage probability of I C ( α ) is1 − P (cid:16) n / (ˆ θ − θ ) ≤ G ∗− (1 − ˆ α ) (cid:17) = 1 − P ( ˜ T n ≤ ˜ y ) + O ( ℓn − + k − ) , (26)where˜ T n = n / (ˆ θ − θ ) + z α (2 σ ) − d X r,s =1 H r H s P r,s + 2 ℓ − / d X r,s,t =1 H r H st χ r,s , P t ! + σφ ( z α ) − B n = n / (ˆ θ − θ ) + z α (2 σ ) − × ( d X r,s =1 H r H s (2 P r,s − Q r,s ) + 2 d X r,s,t =1 H r H st χ r,s , (2 ℓ − / P t − k − / Q t ) ) and ˜ y = G − (1 − α − δ n ) − ℓ − z α (2 σ ) − P dr,s =1 H r H s χ r,s , . Similar to (14) and(15), E h Q r,s exp (cid:16) ιβ P du =1 H u S ( u ) n (cid:17)i and k − / E h Q t exp (cid:16) ιβ P du =1 H u S ( u ) n (cid:17)i can be expanded by invoking Lemma 7, so that the difference between thecharacteristic functions of ˜ T n and n / (ˆ θ − θ ) can be established as in the proofof (16). This enables us to derive an Edgeworth expansion for ˜ T n analogous34o (17): P ( ˜ T n ≤ x ) = G ( x ) − n − / − σ − z α xφ ( x/σ ) × ( d X r,s,t =1 H r H s H t ∞ X i,j = −∞ γ r,s,ti,j + 2 d X r,s,t,u =1 H r H u H st χ r,s , χ t,u , ) + O ( ℓn − + k − n − / L n ) . (27)The coverage expansion (2) for I C ( α ) then follows by noting (26), setting x = ˜ y in (27) and Taylor expansion.It remains to prove (2) for the Studentized I S ( α ). We see by Taylorexpanding the smooth function H ( · ) and the moment relations asserted inLemmas 4 and 5 that ˆ τ = ˆ σ + O p ( n − + ℓn − / ) and τ ∗ = σ ∗ + O p ( n − + ℓn − / ). Expanding τ ∗ about ˆ σ based on (18), we have, for ξ ∈ (0 , J ∗ ( z ξ ) = P ∗ (cid:0) ( bℓ ) / [ H ( ¯ X ∗ ) − H ( E ∗ ¯ X ∗ )] − ∆ ∗ n ≤ ˆ w (cid:1) + O p ( ℓn − + k − ) , where ∆ ∗ n is defined as in (20) andˆ w = ˆ σz ξ + (2ˆ σ ) − z ξ ( d X r,s =1 H r H s (cid:2) Q r,s − P r,s + ( k − − ℓ − ) χ r,s , (cid:3) + 2 d X r,s,t =1 H r H st χ r,s , (cid:0) k − / Q t − ℓ − / P t (cid:1)) . Noting (25) and (21), we have J ∗ ( z ξ ) = G ∗ ( ˆ w ) + n − / − ˆ σ − z ξ φ ( z ξ ) × ( d X r,s,t =1 H r H s H t ∞ X i,j = −∞ γ r,s,ti,j + 2 d X r,s,t,u =1 H r H s H tu χ r,t , χ s,u , ) + O p ( ℓn − + n − / k − L n + ( kℓ ′ ) − n − / L n ) . (28)35ecall the expression for ˆ α = α + δ n + B n + O p ( ℓn − + k − ). Putting z ξ = G ∗− (1 − ˆ α ) / ˆ σ in (28), we verify that J ∗ ( G ∗− (1 − ˆ α ) / ˆ σ ) = 1 − α + O p ( ℓn − + k − ), so that ˆ τ J ∗− (1 − α ) = G ∗− (1 − ˆ α ) + O p ( ℓn − + k − ). Thus I S ( α ) isequivalent asymptotically to I C ( α ) up to O p (cid:8) n − / ( ℓn − + k − ) (cid:9) , yieldingfor its coverage probability the same expression as given by (26) up to order O ( ℓn − + k − ). This completes the proof of part (ii). Under the smooth function model setting, Davison and Hall (1993) and G¨otzeand K¨unsch (1996) suggest Studentizing the block bootstrap based on closed-form expressions. Their constructions are similar to that of our I S ( α ), ex-cept that ˆ τ and τ ∗ are replaced by closed-form expressions depending onpartial derivatives { H r } of H . Specifically, Davison and Hall (1993) defineˆ τ = P dr,s =1 H r ( ¯ X ) H s ( ¯ X ) ˆΣ rs , where ˆΣ rs = n − P ni =1 ( X i − ¯ X ) ( r ) ( X i − ¯ X ) ( s ) + n − P ℓ − j =1 P n − ji =1 ( X i − ¯ X ) ( r ) ( X i + j − ¯ X ) ( s ) , and τ ∗ analogously with X replacedby the block bootstrap series X ∗ in the above definition of ˆ τ . G¨otze andK¨unsch’s (1996) Studentizing factors have similar expressions except thatthey define ˆΣ rs = P ℓ − j =0 w j n − P n − ℓi =1 ( X i − ¯ X ) ( r ) ( X i + j − ¯ X ) ( s ) , where w = 1and w j = 2 { − c ( j/ℓ ) } for 1 ≤ j ≤ ℓ − c >
0, and its bootstrapversion by b − P bj =1 ℓ − nP ℓi =1 ( X ∗ ( j − ℓ + i − ¯ X ∗ ) ( r ) o nP ℓi =1 ( X ∗ ( j − ℓ + i − ¯ X ∗ ) ( s ) o .36 eferences [1] Bertail, P. (1997). Second order properties of an extrapolated bootstrapwithout replacement: the i.i.d. and the strong mixing cases. Bernoulli , , 149-179.[2] Bhattacharya, R.N. and Ghosh, J.K. (1978). On the validity of the for-mal Edgeworth expansion. Annals of Statistics , , 434-451.[3] B¨uhlmann, P. (1997). Sieve bootstrap for time series. Bernoulli , , 123-148.[4] B¨uhlmann, P. (2002). Bootstraps for time series. Statistical Science , ,52-72.[5] Carlstein, E., Do, K.-A., Hall, P., Hesterberg, T. and K¨unsch, H.R.(1998). Matched-block bootstrap for dependent data. Bernoulli , , 305-328.[6] Choi, E. and Hall, P. (2000). Bootstrap confidence regions computedfrom autoregressions of arbitrary order. Journal of the Royal StatisticalSociety, Series B , , 461-477. 377] Davison, A.C. and Hall, P. (1993). On Studentizing and blocking meth-ods for implementing the bootstrap with dependent data. AustralianJournal of Statistics , , 215-224.[8] G¨otze, F. and Hipp, C. (1983). Asymptotic expansions for sums ofweakly dependent random vectors. Zeitschrift f¨ur Wahrscheinlichkeit-stheorie und Verwandte Gebiete , , 211-239.[9] G¨otze, F. and K¨unsch, H.R. (1996). Blockwise bootstrap for depen-dent observations: higher order approximations for Studentized statis-tics. Annals of Statistics , , 1914-1933.[10] Hall, P. (1985). Resampling a coverage process. Stochastic Process Ap-plications , , 259-269.[11] Hall, P. (1992). The Bootstrap and Edgeworth Expansion . Springer-Verlag: New York.[12] Hall, P., Horowitz, J.L. and Jing, B.-Y. (1995). On blocking rules forthe bootstrap with dependent data.
Biometrika , , 561-574.[13] Hall, P. and Jing, B.-Y. (1996). On sample reuse methods for dependentdata. Journal of the Royal Statistical Society, Series B , , 727-737.[14] H¨ardle, W., Horowitz, J. and Kreiss, J.-P. (2003). Bootstrap methodsfor time series. International Statistical Review , , 435-459.3815] K¨unsch, H.R. (1989). The jackknife and the bootstrap for general sta-tionary observations. Annals of Statistics , , 1217-1241.[16] Lahiri, S.N. (1992). Edgeworth correction by ’moving block’ bootstrapfor stationary and nonstationary data. In Exploring the Limits of Boot-strap , Eds. R. LePage and L. Billard, Wiley: New York, pp. 183-214.[17] Lahiri, S.N. (1999). Theoretical comparisons of block bootstrap meth-ods.
Annals of Statistics , , 386-404.[18] Lahiri, S.N. (2003). Resampling Methods for Dependent Data . Springer:New York.[19] Liu, R.Y. and Singh, K. (1992). Moving blocks jackknife and bootstrapcapture weak dependence. In
Exploring the Limits of Bootstrap , Eds. R.LePage and L. Billard, Wiley: New York, pp. 225-248.[20] Paparoditis, E. and Politis, D.N. (2001). Tapered block bootstrap.
Biometrika , , 1105-1119.[21] Paparoditis, E. and Politis, D.N. (2002). The local bootstrap for Markovprocesses. Journal of Statistical Planning and Inference , , 301-328.[22] Politis, D.N. and Romano, J.P. (1992). A general resampling schemefor triangular arrays of alpha-mixing random variables with application39o the problem of spectral density estimation. Annals of Statistics , ,1985-2007.[23] Politis, D.N. and Romano, J.P. (1993). The stationary bootstrap. Jour-nal of the American Statistical Association , , 1303-1313.[24] Politis, D.N. and Romano, J.P. (1994). Large sample confidence regionsbased on subsamples under minimal assumptions. Annals of Statistics , , 2031-2050.[25] Rajarshi, M.B. (1990). Bootstrap in Markov-sequences based on esti-mates of transition density. Annals of the Institute of Statistical Mathe-matics , , 253-268.[26] Zvingelis, J. (2003). On bootstrap coverage probability with dependentdata. In Computer-Aided Econometrics , Ed. D. Giles, Marcel Dekker:New York, pp. 69-90. 40 = 500 n = 1000nominal level α I ( α ) 0.053 0.099 0.897 0.943 0.034 0.110 0.903 0.939 I C ( α ) 0.056 0.096 0.897 0.942 0.037 0.113 0.903 0.936 I S ( α ) 0.052 0.097 0.898 0.944 0.034 0.109 0.901 0.937 I DH ( α ) 0.053 0.100 0.899 0.944 0.033 0.106 0.902 0.939 I GK ( α ) 0.052 0.102 0.899 0.941 0.036 0.107 0.902 0.935(b) MA(1) series I ( α ) 0.059 0.088 0.904 0.948 0.050 0.104 0.899 0.952 I C ( α ) 0.056 0.086 0.912 0.951 0.048 0.096 0.902 0.952 I S ( α ) 0.053 0.085 0.914 0.954 0.044 0.098 0.904 0.952 I DH ( α ) 0.052 0.087 0.912 0.955 0.048 0.097 0.902 0.951 I GK ( α ) 0.053 0.087 0.908 0.953 0.046 0.098 0.900 0.954(c) AR(1) series I ( α ) 0.059 0.104 0.894 0.937 0.049 0.108 0.891 0.934 I C ( α ) 0.045 0.096 0.902 0.941 0.043 0.104 0.899 0.939 I S ( α ) 0.045 0.095 0.902 0.942 0.041 0.103 0.899 0.942 I DH ( α ) 0.046 0.100 0.902 0.941 0.045 0.105 0.896 0.938 I GK ( α ) 0.046 0.101 0.902 0.940 0.043 0.104 0.899 0.939 Table 1: Mean example — coverage probabilities of nominal level α upperconfidence bounds for mean, approximated from 1,000 independent series oflength n . = 500 n = 1000nominal level α I ( α ) 0.025 0.090 0.840 0.902 0.028 0.089 0.828 0.889 I C ( α ) 0.053 0.111 0.887 0.943 0.054 0.109 0.869 0.926 I S ( α ) 0.052 0.112 0.888 0.944 0.052 0.106 0.871 0.926 I DH ( α ) 0.054 0.114 0.881 0.940 0.055 0.108 0.864 0.924 I GK ( α ) 0.058 0.113 0.883 0.942 0.053 0.110 0.867 0.927(b) MA(1) series I ( α ) 0.046 0.090 0.883 0.930 0.056 0.097 0.872 0.921 I C ( α ) 0.059 0.099 0.906 0.946 0.064 0.105 0.884 0.936 I S ( α ) 0.058 0.100 0.909 0.948 0.065 0.105 0.883 0.935 I DH ( α ) 0.059 0.101 0.905 0.944 0.064 0.105 0.881 0.938 I GK ( α ) 0.061 0.103 0.907 0.944 0.064 0.105 0.877 0.937(c) AR(1) series I ( α ) 0.042 0.091 0.885 0.928 0.047 0.097 0.863 0.916 I C ( α ) 0.053 0.106 0.903 0.950 0.059 0.107 0.881 0.936 I S ( α ) 0.052 0.104 0.902 0.953 0.058 0.110 0.880 0.932 I DH ( α ) 0.054 0.104 0.899 0.952 0.057 0.109 0.883 0.930 I GK ( α ) 0.055 0.107 0.901 0.949 0.057 0.108 0.883 0.927 Table 2: Variance example — coverage probabilities of nominal level α upperconfidence bounds for variance, approximated from 1,000 independent seriesof length n . = 500 n = 1000nominal level α I ( α ) 0.063 0.103 0.878 0.930 0.056 0.098 0.888 0.930 I C ( α ) 0.054 0.096 0.885 0.937 0.049 0.098 0.891 0.934 I S ( α ) 0.054 0.099 0.884 0.936 0.052 0.098 0.890 0.933 I DH ( α ) 0.057 0.100 0.883 0.934 0.058 0.100 0.888 0.930 I GK ( α ) 0.049 0.095 0.878 0.934 0.054 0.094 0.887 0.935(b) MA(1) series I ( α ) 0.056 0.095 0.905 0.952 0.041 0.084 0.888 0.947 I C ( α ) 0.052 0.087 0.901 0.953 0.039 0.078 0.886 0.944 I S ( α ) 0.052 0.087 0.899 0.949 0.037 0.075 0.884 0.947 I DH ( α ) 0.051 0.090 0.903 0.952 0.041 0.079 0.886 0.947 I GK ( α ) 0.022 0.058 0.920 0.966 0.026 0.051 0.914 0.962(c) AR(1) series I ( α ) 0.067 0.110 0.882 0.941 0.045 0.101 0.873 0.936 I C ( α ) 0.057 0.100 0.880 0.941 0.042 0.093 0.877 0.935 I S ( α ) 0.055 0.101 0.881 0.937 0.039 0.091 0.878 0.937 I DH ( α ) 0.060 0.103 0.882 0.946 0.044 0.095 0.873 0.941 I GK ( α ) 0.025 0.065 0.897 0.959 0.023 0.054 0.899 0.955 Table 3: Autocorrelation example — coverage probabilities of nominal level α upper confidence bounds for lag 1 autocorrelation, approximated from 1,000independent series of length nn