Decompositions of finite high-dimensional random arrays
DDECOMPOSITIONS OF FINITE HIGH-DIMENSIONAL RANDOMARRAYS
PANDELIS DODOS, KONSTANTINOS TYROS AND PETROS VALETTAS
Abstract. A d -dimensional random array on a nonempty set I is a stochastic process X = (cid:104) X s : s ∈ (cid:0) Id (cid:1) (cid:105) indexed by the set (cid:0) Id (cid:1) of all d -element subsets of I . We obtainstructural decompositions of finite, high-dimensional random arrays whose distribu-tion is invariant under certain symmetries.Our first main result is a distributional decomposition of finite, (approximately)spreadable, high-dimensional random arrays whose entries take values in a finite set;the two-dimensional case of this result is the finite version of an infinitary decompo-sition due to Fremlin and Talagrand. Our second main result is a physical decom-position of finite, spreadable, high-dimensional random arrays with square-integrableentries which is the analogue of the Hoeffding/Efron–Stein decomposition. All proofsare effective.We also present applications of these decompositions in the study of concentrationof functions of finite, high-dimensional random arrays. Contents
1. Introduction 22. Approximation by a random array of lower complexity 83. A coding for distributions 154. Proofs of Theorems 1.4 and 1.5 185. Orbits 286. Comparing two-point correlations of spreadable random arrays 297. Proof of Theorem 1.6 318. Connection with concentration 39Appendix A. Proof of Lemma 3.4 44References 45
Mathematics Subject Classification : 60G07, 60G09, 60G42, 60E15.
Key words : exchangeable random arrays, spreadable random arrays, decompositions, martingales.P.V. is supported by Simons Foundation grant 638224. a r X i v : . [ m a t h . P R ] F e b PANDELIS DODOS, KONSTANTINOS TYROS AND PETROS VALETTAS Introduction
Overview.
Our topic is probability with symmetries , a classical theme in probabilitytheory which originates in the work of de Finetti and whose basic objects of study arethe following classes of stochastic processes.
Definition 1.1 (Random arrays, and their subarrays) . Let d be a positive integer, andlet I be a ( possibly infinite ) set with | I | (cid:62) d . A d -dimensional random array on I is astochastic process X = (cid:104) X s : s ∈ (cid:0) Id (cid:1) (cid:105) indexed by the set (cid:0) Id (cid:1) of all d -element subsets of I .If J is a subset of I with | J | (cid:62) d , then the subarray of X determined by J is the d -dimensional random array X J := (cid:104) X s : s ∈ (cid:0) Jd (cid:1) (cid:105) . The infinitary branch of the theory was developed in a series of foundational papersby Aldous [Ald81], Hoover [Hoo79] and Kallenberg [Kal92], with important earlier con-tributions by Fremlin and Talagrand [FT85]. The subject is presented in the monographsof Aldous [Ald85] and Kallenberg [Kal05]; more recent expositions, which also discussseveral applications, are given in [Ald10, Au08, Au13, DJ08].However, the focus of this paper is on the finitary case which is significantly lessdeveloped (see, e.g. , [Au13, page 16] for a discussion on this issue). Our motivation stemsfrom certain applications, in particular, from the concentration results obtained in thecompanion paper [DTV20] which are inherently finitary; we shall comment further onthese connections in Section 8.1.2.
Notions of symmetry.
Arguably, the most well-known notion of symmetry of ran-dom arrays is exchangeability. Let d be a positive integer, and recall that a d -dimensionalrandom array X = (cid:104) X s : s ∈ (cid:0) Id (cid:1) (cid:105) on a (possibly infinite) set I is called exchangeable iffor every (finite) permutation π of I , the random arrays X and X π := (cid:104) X π ( s ) : s ∈ (cid:0) Id (cid:1) (cid:105) have the same distribution. Another well-known notion of symmetry, which is weaker than exchangeability, is spreadability: a d -dimensional random array X on I is called spreadable if for every pair J, K of finite subsets of I with | J | = | K | (cid:62) d , the subarrays X J and X K have the same distribution.Beyond these two notions, in this paper we will consider yet another notion of symmetrywhich is a natural weakening of spreadability (see also [DTV20, Definition 1.3]). Definition 1.2 (Approximate spreadability) . Let X be a d -dimensional random array ona ( possibly infinite ) set I , and let η (cid:62) . We say that X is η -spreadable ( or approximatelyspreadable if η is understood ) , provided that for every pair J, K of finite subsets of I with Some authors refer to this notion as joint exchangeability . Actually, the relation between these two notions is more subtle. For infinite sequences of randomvariables, spreadability coincides with exchangeability (see [Kal05]), but it is a weaker notion for higher-dimensional random arrays. We note that this is not standard terminology. In particular, in [FT85] spreadable random arraysare referred to as deletion invariant , while in [Kal05] they are called contractable . ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 3 | J | = | K | (cid:62) d we have (1.1) ρ TV ( P J , P K ) (cid:54) η where P J and P K denote the laws of the random subarrays X J and X K respectively, and ρ TV stands for the total variation distance. The following proposition—whose proof is a fairly straightforward application of Ram-sey’s theorem [Ra30]—justifies Definition 1.2 and shows that approximately spreadablerandom arrays are ubiquitous.
Proposition 1.3.
For every triple m, n, d of positive integers with n (cid:62) d , and every η > ,there exists an integer N (cid:62) n with the following property. If X is a set with |X | = m and X is an X -valued, d -dimensional random array on a set I with | I | (cid:62) N , then thereexists a subset J of I with | J | = n such that the random array X J is η -spreadable. Random arrays with finite-valued entries.
Our first main result is a distribu-tional decomposition of finite, approximately spreadable, high-dimensional random arrayswhose entries take values in a finite set. In order to state this decomposition we needto recall a canonical way for defining finite-valued spreadable random arrays. In whatfollows, by N = { , , . . . } we denote the set of positive integers, and for every positiveinteger n we set [ n ] := { , . . . , n } .1.3.1. Let X be a finite set; to avoid degenerate cases, we will assume that |X | (cid:62)
2. Alsolet d be a positive integer, let (Ω , Σ , µ ) be a probability space, and let Ω d be equippedwith the product measure. We say that a collection H = (cid:104) h a : a ∈ X (cid:105) of [0 , d is an X -partition of unity if Ω d = (cid:80) a ∈X h a almost surely. Withevery X -partition of unity H we associate an X -valued, spreadable, d -dimensional randomarray X H = (cid:104) X H s : s ∈ (cid:0) N d (cid:1) (cid:105) on N whose distribution satisfies the following: for everynonempty finite subset F of (cid:0) N d (cid:1) and every collection ( a s ) s ∈F of elements of X , we have(1.2) P (cid:16) (cid:92) s ∈F [ X H s = a s ] (cid:17) = (cid:90) (cid:89) s ∈F h a s ( ω s ) d µ ( ω )where µ stands for the product measure on Ω N and, for every s = { i < · · · < i d } ∈ (cid:0) N d (cid:1) and every ω = ( ω i ) ∈ Ω N , by ω s = ( ω i , . . . , ω i d ) ∈ Ω d we denote the restriction of ω onthe coordinates determined by s .These distributions were considered by Fremlin and Talagrand who showed that if“ d = 2” and “ X = { , } ”, then they are precisely the extreme points of the compact con-vex set of all distributions of boolean, spreadable, two-dimensional random arrays on N ;see [FT85, Theorem 5H]. This striking probabilistic/geometric fact together with Cho-quet’s representation theorem yield that the distribution of an arbitrary boolean, spread-able, two-dimensional random array on N is a mixture of distributions of the form (1.2). See [FT85, Section 1G] for a justification of the existence of this random array.
PANDELIS DODOS, KONSTANTINOS TYROS AND PETROS VALETTAS d and anyfinite set X —is the finite analogue of the Fremlin–Talagrand decomposition. Of course,instead of mixtures, we will consider finite convex combinations. Specifically, let J be anonempty finite index set, let λ = (cid:104) λ j : j ∈ J (cid:105) be convex coefficients (that is, positivecoefficients which sum-up to 1) and let H = (cid:104)H j : j ∈ J (cid:105) be X -partitions of unity suchthat each H j = (cid:104) h aj : a ∈ X (cid:105) is defined on Ω dj where Ω j is the sample space of a probabilityspace (Ω j , Σ j , µ j ). Given these data, we define an X -valued, spreadable, d -dimensionalrandom array X λ , H = (cid:104) X λ , H s : s ∈ (cid:0) N d (cid:1) (cid:105) on N whose distribution satisfies(1.3) P (cid:16) (cid:92) s ∈F [ X λ , H s = a s ] (cid:17) = (cid:88) j ∈ J λ j (cid:90) (cid:89) s ∈F h a s j ( ω s ) d µ j ( ω )for every nonempty finite subset F of (cid:0) N d (cid:1) and every collection ( a s ) s ∈F of elements of X .1.3.3. We are now in a position to state the first main result of this paper. Theorem 1.4 (Distributional decomposition) . Let d, m, k be positive integers with m (cid:62) and k (cid:62) d , let < ε (cid:54) , and set (1.4) C = C ( d, m, k, ε ) := exp (2 d ) (cid:16) m k d ε (cid:17) where for every positive integer (cid:96) by exp ( (cid:96) ) ( · ) we denote the (cid:96) -th iterated exponential .Also let n (cid:62) C be an integer, let X be a set with |X | = m , and let X = (cid:104) X s : s ∈ (cid:0) [ n ] d (cid:1) (cid:105) be an X -valued, (1 /C ) -spreadable, d -dimensional random array on [ n ] . Then there exist • two nonempty finite sets J and Ω with | J | , | Ω | (cid:54) C , • convex coefficients λ = (cid:104) λ j : j ∈ J (cid:105) , and • for every j ∈ J a probability measure µ j on the set Ω and an X -partition of unity H j = (cid:104) h aj : a ∈ X (cid:105) defined on Ω d such that, setting H := (cid:104)H j : j ∈ J (cid:105) and letting X λ , H be as in (1.3) , the following holds.If L is a subset of [ n ] with | L | = k , and P L and Q L denote the laws of the subarrays of X and X λ , H determined by L respectively, then we have (1.5) ρ TV ( P L , Q L ) (cid:54) ε. An immediate consequence of Theorem 1.4 is that every, not too large, subarray ofa finite, finite-valued, approximately spreadable random array is “almost extendable” toan infinite spreadable random array.Closely related to Theorem 1.4 is the following theorem. Theorem 1.5.
Let the parameters d, m, k, ε be as in Theorem , and let the constant C = C ( d, m, k, ε ) be as in (1.4) . Also let n (cid:62) C be an integer, let X be a set with |X | = m , Thus, we have exp (1) ( x ) = exp( x ), exp (2) ( x ) = exp (cid:0) exp( x ) (cid:1) , exp (3) ( x ) = exp (cid:0) exp(exp( x )) (cid:1) , etc. This fact can also be proved using an ultraproduct argument but, of course, this sort of reasoning isnot effective.
ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 5 and let X = (cid:104) X s : s ∈ (cid:0) [ n ] d (cid:1) (cid:105) be an X -valued, (1 /C ) -spreadable, d -dimensional randomarray on [ n ] . Then there exists a Borel measurable function f : [0 , d +1 → X with thefollowing property. Let X f = (cid:104) X fs : s ∈ (cid:0) N d (cid:1) (cid:105) be the X -valued, spreadable, d -dimensionalrandom array on N defined by setting for every s = { i < · · · < i d } ∈ (cid:0) N d (cid:1) , (1.6) X fs = f ( ζ, ξ i , . . . , ξ i d ) where ( ζ, ξ , . . . ) are i.i.d. random variables uniformly distributed in [0 , . Then, forevery subset L of [ n ] with | L | = k , denoting by P L and Q L the laws of the subarrays of X and X f determined by L respectively, we have ρ TV ( P L , Q L ) (cid:54) ε . Theorem 1.5 is akin to the Aldous–Hoover–Kallenberg representation theorem. Themain difference is that in Theorem 1.5 the number of variables which are needed in orderto represent the random array X is d + 1, while the corresponding number of variablesrequired by the Aldous–Hoover–Kallenberg theorem is 2 d . This particular information is agenuinely finitary phenomenon, and it is important for the results related to concentrationwhich are presented in Section 8.1.4. Random arrays with square-integrable entries.
Our second main result is aphysical decomposition of finite, spreadable, high-dimensional random arrays with squareintegrable entries which is in the spirit of the classical Hoeffding/Efron–Stein decompo-sition [Hoe48, ES81]. It is less informative than Theorem 1.4, but this is offset by thefact that it applies to a fairly large class of distributions (including bounded, gaussian,subgaussian, etc.).1.4.1. At this point it is appropriate to introduce some terminology and notation whichwill be used throughout the paper. Given two subsets
F, L of N , by PartIncr( F, L ) wedenote the set of strictly increasing partial maps p whose domain, dom( p ), is containedin F and whose image, Im( p ), is contained in L . (The empty partial map is included inPartIncr( F, L ), and it is denoted by ∅ .) For every p ∈ PartIncr(
F, L ) and every subset G of dom( p ) by p (cid:22) G ∈ PartIncr(
F, L ) we denote the restriction of p on G .Next, let p , p ∈ PartIncr(
F, L ) be distinct partial maps. We say that the pair { p , p } is aligned if there exists a (possibly empty) subset G of dom( p ) ∩ dom( p ) such that:(i) p (cid:22) G = p (cid:22) G , and (ii) p (cid:0) dom( p ) \ G (cid:1) ∩ p (cid:0) dom( p ) \ G (cid:1) = ∅ . We shall refer tothe (necessarily unique) set G as the root of { p , p } and we shall denote it by r ( p , p );moreover, we set p ∧ p := p (cid:22) r ( p , p ) ∈ PartIncr(
F, L ).1.4.2. Whenever necessary, we identify subsets of N with strictly increasing partial mapsas follows. Let L be a nonempty finite subset of N , set (cid:96) := | L | , and let { i < · · · < i (cid:96) } denote the increasing enumeration of L . We define the canonical isomorphism I L : [ (cid:96) ] → L associated with L by setting I L ( j ) := i j for every j ∈ [ (cid:96) ]. Note that I L ∈ PartIncr([ (cid:96) ] , L ). PANDELIS DODOS, KONSTANTINOS TYROS AND PETROS VALETTAS r ( p , p )dom( p )dom( p ) p (cid:22) r ( p , p ) = p (cid:22) r ( p , p ) p (dom( p ) \ r ( p , p )) p (dom( p ) \ r ( p , p )) Figure 1.
Aligned pairs of partial maps.1.4.3. After this preliminary discussion, and in order to motivate our second decomposi-tion, let us consider the model case of a spreadable, d -dimensional random array X on N whose entries are of the form X s = h ( ξ i , . . . , ξ i d ) for every s = { i < · · · < i d } ∈ (cid:0) N d (cid:1) ,where h : [0 , d → [0 ,
1] is Borel measurable and ( ξ i ) are i.i.d. random variables uniformlydistributed in [0 , F of N let A F denote the σ -algebra generated bythe random variables (cid:104) ξ i : i ∈ F (cid:105) . (In particular, A ∅ is the trivial σ -algebra.) Since therandom variables ( ξ i ) are independent, the σ -algebras (cid:104)A F : F ⊆ N (cid:105) generate a latticeof projections: for every pair F, G of subsets of N and every random variable Z we have E (cid:2) E [ Z | A F ] | A G (cid:3) = E [ Z | A F ∩ G ]. This lattice of projections can be used, in turn, todecompose the random array X in a natural (and standard) way.Specifically, for every p ∈ PartIncr([ d ] , N ) we select s ∈ (cid:0) N d (cid:1) such that I s (cid:22) dom( p ) = p ,and we set Y p := E [ X s | A dom( p ) ]. (Notice that Y p is independent of the choice of s .) Viainclusion-exclusion, the process Y = (cid:104) Y p : p ∈ PartIncr([ d ] , N ) (cid:105) induces the “increments” ∆ = (cid:104) ∆ p : p ∈ PartIncr([ d ] , N ) (cid:105) defined by∆ p := (cid:88) G ⊆ dom( p ) ( − | dom( p ) \ G | Y p (cid:22) G . Then, for every s ∈ (cid:0) N d (cid:1) , we have X s = (cid:88) F ⊆ [ d ] ∆ I s (cid:22) F . More importantly, the fact that the random variables ( ξ i ) are independent yields thatif p , p ∈ PartIncr([ d ] , N ) are distinct and the pair { p , p } is aligned, then the randomvariables ∆ p and ∆ p are orthogonal; in particular, we have (cid:107) X s (cid:107) L = (cid:80) F ⊆ [ d ] (cid:107) ∆ I s (cid:22) F (cid:107) L . Note that this selection is not always possible, but it is certainly possible if Im( p ) ⊆ { d, d + 1 , . . . } .Here, we ignore this minor technical issue for the sake of exposition. ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 7
Theorem 1.6 (Physical decomposition) . Let d be a positive integer, let ε > , and set c = c ( d, ε ) := 2 − ε / ( d +1) (1.7) n = n ( d, ε ) := 2 d +1) ε − ( d +5) . (1.8) Then for every integer n (cid:62) n there exists a subset N of [ n ] with | N | (cid:62) c d +1 √ n andsatisfying the following property. If X = (cid:104) X s : s ∈ (cid:0) [ n ] d (cid:1) (cid:105) is a real-valued, spreadable, d -dimensional random array on [ n ] such that (cid:107) X s (cid:107) L = 1 for all s ∈ (cid:0) [ n ] d (cid:1) , then thereexists a real-valued stochastic process ∆ = (cid:104) ∆ p : p ∈ PartIncr([ d ] , N ) (cid:105) such that thefollowing hold true. (i) (Decomposition) For every s ∈ (cid:0) Nd (cid:1) we have (1.9) X s = (cid:88) F ⊆ [ d ] ∆ I s (cid:22) F . (ii) (Approximate zero mean) If p ∈ PartIncr([ d ] , N ) with p (cid:54) = ∅ , then (1.10) (cid:12)(cid:12) E [∆ p ] (cid:12)(cid:12) (cid:54) ε. (iii) (Approximate orthogonality) If p , p ∈ PartIncr([ d ] , N ) are distinct and the pair { p , p } is aligned, then (1.11) (cid:12)(cid:12) E [∆ p ∆ p ] (cid:12)(cid:12) (cid:54) ε. (iv) (Uniqueness) The process ∆ is unique in the following sense. There exists asubset L of N with | L | (cid:62) (cid:0) ( ε − + 2 d ) d (cid:1) − | N | such that for every real-valuedstochastic process Z = (cid:104) Z p : p ∈ PartIncr([ d ] , N ) (cid:105) which satisfies (i) and (iii) above, we have (cid:107) ∆ p − Z p (cid:107) L (cid:54) d +22 ) √ ε for all p ∈ PartIncr([ d ] , L ) . Outline of the proofs/Structure of the paper.
The proofs of Theorems 1.4, 1.5and 1.6 are a blend of analytic, probabilistic and combinatorial ideas.1.5.1. The proofs of Theorems 1.4 and 1.5 rely on two preparatory steps which are largelyindependent of each other and can be read separately.The first step is to approximate, in distribution, any finite-valued, approximatelyspreadable random array by a random array of “lower-complexity”. We note that a similarapproximation is used in the proof of the Aldous–Hoover theorem; see, e.g. , [Au13, Sec-tion 5]. However, our argument is technically different since we work with approximatelyspreadable, instead of exchangeable, random arrays. The details of this approximationare given in Section 2.The second step, which is presented in Section 3, is a coding lemma for distributions ofthe form (1.2). It asserts that the laws of their finite subarrays can be approximated, witharbitrary accuracy, by the laws of subbarrays of distributions of the form (1.2) which are
PANDELIS DODOS, KONSTANTINOS TYROS AND PETROS VALETTAS generated by genuine partitions instead of partitions of unity. The proof of this coding isbased on a random selection of uniform hypergraphs.Section 4 is devoted to the proofs of Theorems 1.4 and 1.5. We actually prove a slightlystronger result—Theorem 4.1 in the main text—which encompasses both Theorem 1.4and Theorem 1.5 and it is more amenable to an inductive scheme.1.5.2. The proof of Theorem 1.6 is somewhat different, and it is based exclusively on L methods. The main goal is to construct an appropriate collection of σ -algebras forwhich the corresponding projections behave like the lattice of projections described inParagraph 1.4.3.This goal boils down to classify all two-point correlations of finite, spreadable randomarrays with square-integrable entries. Sections 5 and 6 are devoted to the proof of thisclassification; we note that this material is of independent interest, and it can also beread independently. The proof of Theorem 1.6 is completed in Section 7.1.5.3. Finally, as we have already mentioned, in the last section of this paper, Section 8,we present an application of Theorem 1.4 which is related to the concentration resultsobtained in [DTV20].2. Approximation by a random array of lower complexity
The main result in this section—Proposition 2.1 below—asserts that any large subarrayof a finite-valued, approximately spreadable random array X can be approximated, indistribution, by a random array which is obtained by projecting the entries of X oncertain σ -algebras of “lower complexity”.2.1. The σ -algebras Σ( G s(cid:96) , X ) . Our first goal is to define the aforementioned σ -algebras.To this end we need to introduce some notation which will be used throughout this sectionand Section 4. Let n (cid:62) d be positive integers, and let G be a nonempty subset of (cid:0) [ n ] d (cid:1) .For every finite-valued, d -dimensional random array X = (cid:104) X s : s ∈ (cid:0) [ n ] d (cid:1) (cid:105) on [ n ] we set(2.1) Σ( G , X ) := σ (cid:0) { X s : s ∈ G} (cid:1) ;that is, Σ( G , X ) denotes the σ -algebra generated by the random variables (cid:104) X s : s ∈ G(cid:105) .Moreover, for every pair F = { i < · · · < i k } and G = { j < · · · < j k } of nonemptysubsets of N with | F | = | G | = k , we define I F,G : F → G by setting(2.2) I F,G ( i r ) = j r for every r ∈ [ k ]. Notice that I F,G = I G ◦ I − F where I F and I G denote the canonicalisomorphisms associated with the sets F and G respectively. (See Paragraph 1.4.2.) ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 9 n, d, (cid:96) be positive integers with n (cid:62) d . Also let F be a nonempty subsetof [ n ], and let { j < · · · < j | F | } denote the increasing enumeration of F . We say that F is (cid:96) -sparse provided that • (cid:96) (cid:54) min( F ), • max( F ) (cid:54) n − (cid:96) , and • if | F | (cid:62)
2, then j i +1 − j i (cid:62) (cid:96) for all i ∈ { , . . . , | F | − } .2.1.1.1. Now assume that d (cid:62)
2. For every (cid:96) -sparse x = { j < · · · < j d − } ∈ (cid:0) [ n ] d − (cid:1) we set(2.3) R x(cid:96) := (cid:18)(cid:0) (cid:83) d − r =1 { j r − (cid:96) + 1 , . . . , j r } (cid:1) ∪ { n − (cid:96) + 1 , . . . , n } d (cid:19) . j ‘ n‘j d − ‘ Figure 2.
The set R x(cid:96) .Moreover, for every (cid:96) -sparse s ∈ (cid:0) [ n ] d (cid:1) we define(2.4) G s(cid:96) := (cid:91) x ∈ ( sd − ) R x(cid:96) . Finally, if X is a finite-valued, d -dimensional random array on [ n ], then Σ( G s(cid:96) , X ) denotesthe corresponding σ -algebra defined via formula (2.1); notice that(2.5) Σ( G s(cid:96) , X ) = (cid:95) x ∈ ( sd − ) Σ( R x(cid:96) , X ) . d = 1, then for every (cid:96) -sparse s ∈ (cid:0) [ n ]1 (cid:1) we set(2.6) G s(cid:96) := (cid:18) { n − (cid:96) + 1 , . . . , n } (cid:19) . Of course, for every finite-valued, d -dimensional random array X on [ n ], the corresponding σ -algebra Σ( G s(cid:96) , X ) is also defined via formula (2.1).2.2. The approximation.
We have the following proposition.
Proposition 2.1.
Let n, d, m, k be positive integers with k (cid:62) d and m (cid:62) , and let θ > .Assume that (2.7) n (cid:62) ( k + 1) k m (cid:98) /θ (cid:99) +1 and set (cid:96) := k m (cid:98) /θ (cid:99) . Then every ( k(cid:96) ) -sparse subset L of [ n ] of cardinality k has thefollowing property. For every set X with |X | = m , every η (cid:62) and every X -valued, η -spreadable, d -dimensional random array X = (cid:104) X s : s ∈ (cid:0) [ n ] d (cid:1) (cid:105) on [ n ] there exists (cid:96) ∈ [ (cid:96) ] such that for every nonempty subset F of (cid:0) Ld (cid:1) and every collection ( a s ) s ∈F of elementsof X we have (2.8) (cid:12)(cid:12)(cid:12) P (cid:16) (cid:92) s ∈F [ X s = a s ] (cid:17) − E (cid:104) (cid:89) s ∈F E (cid:2) [ X s = a s ] | Σ( G s(cid:96) , X ) (cid:3)(cid:105)(cid:12)(cid:12)(cid:12) (cid:54) k d (cid:113) θ + 15 ηm ( k(cid:96) ( d +1)) d . The rest of this section is devoted to the proof of Proposition 2.1.2.2.1.
Step 1.
We start with the following lemma which is a consequence of spreadability.
Lemma 2.2 (Shift invariance of projections) . Let n, d be positive integers with n (cid:62) d ,let s ∈ (cid:0) [ n ] d (cid:1) , and let F be a nonempty subset of (cid:0) [ n ] d (cid:1) . Set F := s ∪ ( ∪F ) . Also let G be asubset of [ n ] with | F | = | G | , and set (2.9) t := I F,G ( s ) and G := (cid:8) I F,G ( s ) : s ∈ F (cid:9) . Finally, let X be a finite set, let η (cid:62) , and let X = (cid:104) X s : s ∈ (cid:0) [ n ] d (cid:1) (cid:105) be an X -valued, η -spreadable, d -dimensional random array on [ n ] . Then for every a ∈ X we have (2.10) (cid:12)(cid:12)(cid:12)(cid:13)(cid:13) E [ [ X s = a ] | Σ( F , X )] (cid:13)(cid:13) L − (cid:13)(cid:13) E [ [ X t = a ] | Σ( G , X )] (cid:13)(cid:13) L (cid:12)(cid:12)(cid:12) (cid:54) η |X | |F| . Proof.
Fix a ∈ X . For every collection a = ( a u ) u ∈F of elements of X we set(2.11) B a := (cid:92) u ∈F [ X u = a u ] and C a := (cid:92) u ∈F [ X I F,G ( u ) = a u ] . Since the random array X is η -spreadable, for every a ∈ X F we have(2.12) (cid:12)(cid:12) P ( B a ) − P ( C a ) (cid:12)(cid:12) (cid:54) η and (cid:12)(cid:12) P ([ X s = a ] ∩ B a ) − P ([ X t = a ] ∩ C a ) (cid:12)(cid:12) (cid:54) η. Set B := { a ∈ X F : P ( B a ) > P ( C a ) > } . By (2.12), for every a ∈ X F \ B we have P ( B a ) (cid:54) η and P ( C a ) (cid:54) η and, consequently,(2.13) (cid:12)(cid:12) P (cid:0) [ X s = a ] | B a (cid:1) P ( B a ) − P (cid:0) [ X t = a ] | C a (cid:1) P ( C a ) (cid:12)(cid:12) (cid:54) η. Next, let a ∈ B be arbitrary, and observe that (cid:12)(cid:12) P (cid:0) [ X s = a ] | B a (cid:1) − P (cid:0) [ X t = a ] | C a (cid:1)(cid:12)(cid:12) =(2.14) = (cid:12)(cid:12)(cid:12) P (cid:0) [ X s = a ] ∩ B a (cid:1) P ( B a ) − P (cid:0) [ X t = a ] ∩ C a (cid:1) P ( C a ) (cid:12)(cid:12)(cid:12) (cid:54) P ( B a ) (cid:12)(cid:12) P (cid:0) [ X s = a ] ∩ B a (cid:1) − P (cid:0) [ X t = a ] ∩ C a (cid:1)(cid:12)(cid:12) + P ( C a ) (cid:12)(cid:12)(cid:12) P ( B a ) − P ( C a ) (cid:12)(cid:12)(cid:12) (2.12) (cid:54) η P ( B a ) + 1 P ( B a ) (cid:12)(cid:12) P ( C a ) − P ( B a ) (cid:12)(cid:12) (2.12) (cid:54) η P ( B a ) . On the other hand, we have P (cid:0) [ X s = a ] | B a (cid:1) + P (cid:0) [ X t = a ] | C a (cid:1) (cid:54) (cid:12)(cid:12) P (cid:0) [ X s = a ] | B a (cid:1) − P (cid:0) [ X t = a ] | C a (cid:1) (cid:12)(cid:12) (cid:54) η P ( B a ) . ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 11
Therefore, for every a ∈ B , (cid:12)(cid:12) P (cid:0) [ X s = a ] | B a (cid:1) P ( B a ) − P (cid:0) [ X t = a ] | C a (cid:1) P ( C a ) (cid:12)(cid:12) (cid:54) (2.16) (cid:54) P ( B a ) (cid:12)(cid:12) P (cid:0) [ X s = a ] | B a (cid:1) − P (cid:0) [ X t = a ] | C a (cid:1) (cid:12)(cid:12) ++ P (cid:0) [ X t = a ] | C a (cid:1) (cid:12)(cid:12) P ( B a ) − P ( C a ) (cid:12)(cid:12) (2.15) , (2.12) (cid:54) η. By (2.13) and (2.16), we conclude that (cid:12)(cid:12)(cid:12)(cid:13)(cid:13) E [ [ X s = a ] | Σ( F , X )] (cid:13)(cid:13) L − (cid:13)(cid:13) E [ [ X t = a ] | Σ( G , X )] (cid:13)(cid:13) L (cid:12)(cid:12)(cid:12) =(2.17) = (cid:12)(cid:12)(cid:12) (cid:88) a ∈X F P (cid:0) [ X s = a ] | B a (cid:1) P ( B a ) − P (cid:0) [ X t = a ] | C a (cid:1) P ( C a ) (cid:12)(cid:12)(cid:12) (cid:54) η |X | |F| as desired. (cid:3) Step 2.
The next lemma follows from elementary properties of martingale differencesequences.
Lemma 2.3 (Basic approximation) . Let n, d, m, k be positive integers with k (cid:62) d and m (cid:62) , and let θ > . Assume that (2.18) n (cid:62) ( d + 1) k m (cid:98) /θ (cid:99) +1 and set (cid:96) := k m (cid:98) /θ (cid:99) . Moreover, let X be a set with |X | = m , let η (cid:62) , and let X = (cid:104) X s : s ∈ (cid:0) [ n ] d (cid:1) (cid:105) be an X -valued, η -spreadable, d -dimensional random array on [ n ] .Then for every ( k(cid:96) ) -sparse t ∈ (cid:0) [ n ] d (cid:1) there exists (cid:96) ∈ [ (cid:96) ] such that for every a ∈ X , (2.19) (cid:13)(cid:13) E (cid:2) [ X t = a ] | Σ( G tk(cid:96) , X ) (cid:3) − E (cid:2) [ X t = a ] | Σ( G t(cid:96) , X ) (cid:3)(cid:13)(cid:13) L (cid:54) √ θ. Proof.
Fix t ∈ (cid:0) [ n ] d (cid:1) which is ( k(cid:96) )-sparse. For every a ∈ X and r ∈ [ m (cid:98) /θ (cid:99) + 1], we set(2.20) D ar := E (cid:2) [ X t = a ] | Σ( G tk r , X ) (cid:3) − E (cid:2) [ X t = a ] | Σ( G tk r − , X ) (cid:3) . Clearly, it enough to show that there exists r ∈ [ m (cid:98) /θ (cid:99) + 1] such that (cid:107) D ar (cid:107) L (cid:54) √ θ for every a ∈ X . Assume, towards a contradiction, that for every r ∈ [ m (cid:98) /θ (cid:99) + 1] thereexists a r ∈ X such that (cid:107) D a r r (cid:107) L > √ θ . Since |X | = m , by the pigeonhole principle,there exist b ∈ X and a subset R of [ m (cid:98) /θ (cid:99) + 1] with | R | = (cid:98) /θ (cid:99) + 1 such that a r = b ,which is equivalent to saying that (cid:107) D br (cid:107) L > √ θ for every r ∈ R . Now, observe that thesequence ( G t , . . . , G tκ m (cid:98) /θ (cid:99) +1 ) is increasing with respect to inclusion, which in turn implies,by (2.1), that the sequence ( D b , . . . , D bm (cid:98) /θ (cid:99) +1 ) is a martingale difference sequence. Bythe contractive property of conditional expectation, we obtain that(2.21) 1 (cid:62) (cid:107) [ X t = b ] (cid:107) L (cid:62) m (cid:98) /θ (cid:99) +1 (cid:88) r =1 (cid:107) D br (cid:107) L (cid:62) (cid:88) r ∈ R (cid:107) D br (cid:107) L > | R | θ > (cid:3) We will need the following consequence of Lemma 2.3.
Corollary 2.4.
Let n, d, m, k, (cid:96) , X , η, X be as in Lemma . Then there exists (cid:96) ∈ [ (cid:96) ] such that for every ( k(cid:96) ) -sparse s ∈ (cid:0) [ n ] d (cid:1) and every a ∈ X we have (2.22) (cid:13)(cid:13) E (cid:2) [ X s = a ] | Σ( G sk(cid:96) , X ) (cid:3) − E (cid:2) [ X s = a ] | Σ( G s(cid:96) , X ) (cid:3)(cid:13)(cid:13) L (cid:54) (cid:113) θ + 10 ηm ( k(cid:96) ( d +1)) d . Proof.
Fix a ( k(cid:96) )-sparse t ∈ (cid:0) [ n ] d (cid:1) . By Lemma 2.3, there exists (cid:96) ∈ [ (cid:96) ] such that forevery a ∈ X we have(2.23) (cid:13)(cid:13) E (cid:2) [ X t = a ] | Σ( G sk(cid:96) , X ) (cid:3) − E (cid:2) [ X t = a ] | Σ( G s(cid:96) , X ) (cid:3)(cid:13)(cid:13) L (cid:54) √ θ. Since the set G t(cid:96) is contained in G tk(cid:96) , we see that Σ( G t(cid:96) , X ) is a sub- σ -algebra of Σ( G tk(cid:96) , X ).Hence, by (2.23), for every a ∈ X we have(2.24) (cid:12)(cid:12)(cid:12)(cid:13)(cid:13) E (cid:2) [ X t = a ] | Σ( G tk(cid:96) , X ) (cid:3)(cid:13)(cid:13) L − (cid:13)(cid:13) E (cid:2) [ X t = a ] | Σ( G t(cid:96) , X ) (cid:3)(cid:13)(cid:13) L (cid:12)(cid:12)(cid:12) (cid:54) θ. Now let s ∈ (cid:0) [ n ] d (cid:1) be an arbitrary ( k(cid:96) )-sparse subset of [ n ]. Set F := t ∪ ( ∪G t(cid:96) ) and G := s ∪ ( ∪G s(cid:96) ), and notice that(2.25) s = I F,G ( t ) and G s(cid:96) = (cid:8) I F,G ( u )) : u ∈ G t(cid:96) (cid:9) where I F,G is as in (2.2). By Lemma 2.2 and the fact that |G t(cid:96) | (cid:54) (( d +1) (cid:96) ) d (cid:54) ( k(cid:96) ( d +1)) d ,for every a ∈ X we have(2.26) (cid:12)(cid:12)(cid:12)(cid:13)(cid:13) E (cid:2) [ X t = a ] | Σ( G t(cid:96) , X ) (cid:3)(cid:13)(cid:13) L − (cid:13)(cid:13) E (cid:2) [ X s = a ] | Σ( G s(cid:96) , X ) (cid:3)(cid:13)(cid:13) L (cid:12)(cid:12)(cid:12) (cid:54) ηm ( k(cid:96) ( d +1)) d . With identical arguments we obtain that(2.27) (cid:12)(cid:12)(cid:12)(cid:13)(cid:13) E (cid:2) [ X t = a ] | Σ( G tk(cid:96) , X ) (cid:3)(cid:13)(cid:13) L − (cid:13)(cid:13) E (cid:2) [ X s = a ] | Σ( G sk(cid:96) , X ) (cid:3)(cid:13)(cid:13) L (cid:12)(cid:12)(cid:12) (cid:54) ηm ( k(cid:96) ( d +1)) d . Finally, the fact that G s(cid:96) is contained G sk(cid:96) yields that Σ( G s(cid:96) , X ) is a sub- σ -algebra ofΣ( G sk(cid:96) , X ), and so, for every a ∈ X we have (cid:13)(cid:13) E (cid:2) [ X s = a ] | Σ( G sk(cid:96) , X ) (cid:3) − E (cid:2) [ X s = a ] | Σ( G s(cid:96) , X ) (cid:3)(cid:13)(cid:13) L =(2.28) = (cid:12)(cid:12)(cid:12)(cid:13)(cid:13) E (cid:2) [ X s = a ] | Σ( G sk(cid:96) , X ) (cid:3)(cid:13)(cid:13) L − (cid:13)(cid:13) E (cid:2) [ X s = a ] | Σ( G s(cid:96) , X ) (cid:3)(cid:13)(cid:13) L (cid:12)(cid:12)(cid:12) . The desired estimate (2.22) follows from (2.24), (2.26), (2.27), (2.28) and the triangleinequality. (cid:3)
Step 3.
For the next step of the proof of Proposition 2.1 we need to introduce someauxiliary σ -algebras. Let n, d, (cid:96) be positive integers with n (cid:62) (cid:96) ( d + 1). Also let L be an (cid:96) -sparse subset of [ n ] of cardinality at least d , set k := | L | and let { i < · · · < i k } denotethe increasing enumeration of L . Moreover, let s = { i l < · · · < i l d } ∈ (cid:0) Ld (cid:1) . First, wedefine the following subsets of [ n ].( D
1) We set R s,L, (cid:96) := (cid:83) l u =1 { i u − (cid:96) + 1 , . . . , i u } .( D
2) If we have that d (cid:62)
2, then we set R s,L,r(cid:96) := (cid:83) l r u = l r − +1 { i u − (cid:96) + 1 , . . . , i u } forevery r ∈ { , . . . , d } .( D
3) If l d < k , then we set ∆ s,L,n(cid:96) := { n − (cid:96) + 1 , . . . , n } ∪ (cid:83) ku = l d +1 { i u − (cid:96) + 1 , . . . , i u } ;otherwise, we set ∆ s,L,n(cid:96) = { n − (cid:96) + 1 , . . . , n } . ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 13
Next, we set(2.29) G s,L(cid:96) := (cid:91) x ∈ ( [ d ] d − ) (cid:18) ∆ s,L,n(cid:96) ∪ (cid:83) r ∈ x R s,L,r(cid:96) d (cid:19) . Finally, for every d -dimensional random array X on [ n ] we define the corresponding σ -algebra Σ( G s,L(cid:96) , X ) via formula (2.1).We have the following lemma. Lemma 2.5 (Absorbtion) . Let n, d, m, k be positive integers with k (cid:62) d , and let θ > .Assume that (2.30) n (cid:62) ( k + 1) k m (cid:98) /θ (cid:99) +1 and set (cid:96) := k m (cid:98) /θ (cid:99) . Then every ( k(cid:96) ) -sparse subset L of [ n ] with | L | = k has thefollowing property. For every set X with |X | = m , every η (cid:62) and every X -valued, η -spreadable, d -dimensional random array X = (cid:104) X s : s ∈ (cid:0) [ n ] d (cid:1) (cid:105) on [ n ] , there exists (cid:96) ∈ [ (cid:96) ] such that for every a ∈ X and every s ∈ (cid:0) Ld (cid:1) we have (2.31) (cid:13)(cid:13) E (cid:2) [ X s = a ] | Σ( G s,L(cid:96) , X ) (cid:3) − E (cid:2) [ X s = a ] | Σ( G s(cid:96) , X ) (cid:3)(cid:13)(cid:13) L (cid:54) (cid:113) θ + 15 ηm ( k(cid:96) ( d +1)) d . Proof.
For notational convenience, we will assume that d (cid:62)
2. The case “ d = 1” is similar.At any rate, in order to facilitate the reader, we shall indicate the necessary changes.Let L, X , η, X be as in the statement of the lemma. We apply Corollary 2.4 and weobtain (cid:96) ∈ [ (cid:96) ] such that for every a ∈ X and every ( k(cid:96) )-sparse s ∈ (cid:0) [ n ] d (cid:1) we have theestimate (2.22). In what follows, this (cid:96) will be fixed.Let s = { j < · · · < j d } ∈ (cid:0) Ld (cid:1) be arbitrary; notice that s is ( k(cid:96) )-sparse. Let∆ , R , . . . , R d denote the unique subintervals of [ n ] with the following properties. • We have | ∆ | = | ∆ s,L,n(cid:96) | and max(∆) = n , where ∆ s,L,n(cid:96) is as in ( D • For every r ∈ [ d ] we have | R r | = | R s,L,r(cid:96) | and max( R r ) = j r , where R s,L, (cid:96) is as in( D
1) and R s,L,r(cid:96) is as in ( D
2) if r (cid:62) G := (cid:91) x ∈ ( [ d ] d − ) (cid:18) ∆ ∪ (cid:83) r ∈ x R r d (cid:19) . (If “ d = 1”, then we set G := (cid:0) ∆1 (cid:1) .) Next, we define ∆ (cid:96) := { n − (cid:96) + 1 , . . . , n } and∆ k(cid:96) := { n − k(cid:96) + 1 , . . . , n } ; moreover, for every r ∈ [ d ] we set R r(cid:96) := { j r − (cid:96) + 1 , . . . , j r } and R rk(cid:96) := { j r − k(cid:96) + 1 , . . . , j r } . With these choices, by (2.4) and (2.29), we have(2.33) G s(cid:96) = (cid:91) x ∈ ( [ d ] d − ) (cid:18) ∆ (cid:96) ∪ (cid:83) r ∈ x R r(cid:96) d (cid:19) and G sk(cid:96) = (cid:91) x ∈ ( [ d ] d − ) (cid:18) ∆ k(cid:96) ∪ (cid:83) r ∈ x R rk(cid:96) d (cid:19) . (If “ d = 1”, then we have G s(cid:96) = (cid:0) ∆ (cid:96) (cid:1) and G sk(cid:96) = (cid:0) ∆ k(cid:96) (cid:1) .) Observing that(2.34) (cid:96) (cid:54) | ∆ s,L,n(cid:96) | , | R s,L, (cid:96) | , . . . , | R s,L,d(cid:96) | (cid:54) | L | (cid:96) = k(cid:96) (cid:54) k(cid:96) , we see that ∆ ⊆ ∆ k(cid:96) and R r ⊆ R rk(cid:96) for every r ∈ [ d ], and moreover, ∆ (cid:96) ⊆ ∆ and R r(cid:96) ⊆ R r for every r ∈ [ d ]. By (2.32) and (2.33), we obtain that G s(cid:96) ⊆ G ⊆ G sk(cid:96) which, in turn,implies that(2.35) Σ( G s(cid:96) , X ) ⊆ Σ( G , X ) ⊆ Σ( G sk(cid:96) , X ) . By (2.22) and (2.35), for every a ∈ X we have(2.36) (cid:12)(cid:12)(cid:12)(cid:13)(cid:13) E (cid:2) [ X s = a ] | Σ( G , X ) (cid:3)(cid:13)(cid:13) L − (cid:13)(cid:13) E (cid:2) [ X s = a ] | Σ( G s(cid:96) , X ) (cid:3)(cid:13)(cid:13) L (cid:12)(cid:12)(cid:12) (cid:54) θ + 10 ηm ( k(cid:96) ( d +1)) d . On the other hand, setting F := ∆ ∪ (cid:83) dj =1 R j and G := ∆ s,L,n(cid:96) ∪ (cid:83) dr =1 R s,L,r(cid:96) , we have(2.37) s = I F,G ( s ) and G s,L(cid:96) = (cid:8) I F,G ( t ) : t ∈ G (cid:9) where I F,G is as in (2.2). By Lemma 2.2 and the fact that |G| (cid:54) (( k +1) (cid:96) ) d (cid:54) ( k(cid:96) ( d +1)) d ,for every a ∈ X we have(2.38) (cid:12)(cid:12)(cid:12)(cid:13)(cid:13) E (cid:2) [ X s = a ] | Σ( G , X ) (cid:3)(cid:13)(cid:13) L − (cid:13)(cid:13) E (cid:2) [ X s = a ] | Σ( G s,L(cid:96) , X ) (cid:3)(cid:13)(cid:13) L (cid:12)(cid:12)(cid:12) (cid:54) ηm ( k(cid:96) ( d +1)) d . Finally recall that, by (2.35), we have Σ( G s(cid:96) , X ) ⊆ Σ( G s,L(cid:96) , X ). Therefore, the estimate(2.31) follows from (2.36), (2.38) and the triangle inequality. (cid:3) Completion of the proof.
For every positive integer d let < lex denote the lexico-graphical order on (cid:0) N d (cid:1) . Specifically, for every distinct s = { i < · · · < i d } ∈ (cid:0) N d (cid:1) and t = { j < · · · < j d } ∈ (cid:0) N d (cid:1) , setting r := min (cid:8) r ∈ [ d ] : i r (cid:54) = j r (cid:9) , we have(2.39) s < lex t ⇔ i r < j r . We also isolate, for future use, the following fact. Although it is an elementary obser-vation which follows readily from the relevant definitions, it is quite crucial for the proofof Proposition 2.1 and, to a large extend, it justifies the definition of the families of setsin (2.4) and (2.29).
Fact 2.6.
Let n, d, (cid:96) be positive integers with n (cid:62) (cid:96) ( d + 1) . Also let L be an (cid:96) -sparsesubset of [ n ] with | L | (cid:62) d . Then the following hold. (i) For every s, t ∈ (cid:0) Ld (cid:1) we have G s(cid:96) ⊆ G t,L(cid:96) . (ii) For every s ∈ (cid:0) Ld (cid:1) we have { t ∈ (cid:0) Ld (cid:1) : s < lex t } ⊆ G s,L(cid:96) . We are now ready to give the proof of Proposition 2.1.
Proof of Proposition . Fix a ( k(cid:96) )-sparse subset L of [ n ] of cardinality k , and let X , η, X be as in the statement of the proposition. By Lemma 2.5, there exists (cid:96) ∈ [ (cid:96) ]such that for every a ∈ X and every s ∈ (cid:0) Ld (cid:1) we have(2.40) (cid:13)(cid:13) E (cid:2) [ X s = a ] | Σ( G s,L(cid:96) , X ) (cid:3) − E (cid:2) [ X s = a ] | Σ( G s(cid:96) , X ) (cid:3)(cid:13)(cid:13) L (cid:54) (cid:113) θ + 15 ηm ( k(cid:96) ( d +1)) d . We claim that (cid:96) is as desired.Indeed, let F be subset of (cid:0) Ld (cid:1) and let ( a s ) s ∈F be a collection of elements of X . Set κ := |F| and let { s < lex · · · < lex s κ } denote the lexicographical increasing enumeration ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 15 of F . Notice that κ = |F| (cid:54) (cid:12)(cid:12)(cid:0) Ld (cid:1)(cid:12)(cid:12) (cid:54) k d . Thus, in order to verify (2.8), by a telescopicargument, it is enough to show that for every r ∈ [ κ ] we have (cid:12)(cid:12)(cid:12) E (cid:104)(cid:16) r − (cid:89) i =1 E (cid:2) [ X si = a si ] | Σ( G s i (cid:96) , X ) (cid:3)(cid:17) · (cid:16) κ (cid:89) i = r [ X si = a si ] (cid:17)(cid:105) − (2.41) − E (cid:104)(cid:16) r (cid:89) i =1 E (cid:2) [ X si = a si ] | Σ( G s i (cid:96) , X ) (cid:3)(cid:17) · (cid:16) κ (cid:89) i = r +1 [ X si = a si ] (cid:17)(cid:105)(cid:12)(cid:12)(cid:12) (cid:54) (cid:113) θ + 15 ηm ( k(cid:96) ( d +1)) d . (Here, we use the convention that the product of an empty family of functions is equalto the constant function 1.) So, fix r ∈ [ κ ]. By Fact 2.6, we see that E (cid:104)(cid:16) r − (cid:89) i =1 E (cid:2) [ X si = a si ] | Σ( G s i (cid:96) , X ) (cid:3)(cid:17) · (cid:16) κ (cid:89) i = r [ X si = a si ] (cid:17)(cid:105) =(2.42)= E (cid:104) E (cid:104)(cid:16) r − (cid:89) i =1 E (cid:2) [ X si = a si ] | Σ( G s i (cid:96) , X ) (cid:3)(cid:17) · (cid:16) κ (cid:89) i = r [ X si = a si ] (cid:17) (cid:12)(cid:12)(cid:12) Σ( G s r ,L(cid:96) , X ) (cid:105)(cid:105) == E (cid:104)(cid:16) r − (cid:89) i =1 E (cid:2) [ X si = a si ] | Σ( G s i (cid:96) , X ) (cid:3)(cid:17) · E (cid:2) [ X sr = a sr ] | Σ( G s r ,L(cid:96) , X ) (cid:3) · (cid:16) κ (cid:89) i = r +1 [ X si = a si ] (cid:17)(cid:105) . Inequality (2.41) follows from (2.40), (2.42) and the Cauchy–Schwarz inequality. Theproof of Proposition 2.1 is completed. (cid:3) A coding for distributions
The following proposition is the main result in this section.
Proposition 3.1.
Let d, m, κ be positive integers with d, m (cid:62) , let ε > , and set (3.1) u = u ( d, m, κ , ε ) := 5 d d ! mκ d +1 ε − d +1 . Let X be a set with |X | = m , and let H = (cid:104) h a : a ∈ X (cid:105) be an X -partition of unity definedon Y d where ( Y , ν ) is a finite probability space. ( See Paragraph . ) Then there existsa partition (cid:104) E a : a ∈ X (cid:105) of ( Y × [ u ]) d such that for every nonempty subset F of (cid:0) N d (cid:1) with |F| (cid:54) κ and every collection ( a s ) s ∈F of elements of X we have (3.2) (cid:12)(cid:12)(cid:12) (cid:90) (cid:89) s ∈F h a s ( y s ) d ν ( y ) − (cid:90) (cid:89) s ∈F E as ( ω s ) d µ ( ω ) (cid:12)(cid:12)(cid:12) (cid:54) ε where: (i) ν denotes the product measure on Y N obtained by equipping each factor with themeasure ν , (ii) µ denotes the product measure on ( Y × [ u ]) N obtained by equipping eachfactor with the product of ν and the uniform probability measure on [ u ] , and (iii) for every y ∈ Y N , every ω ∈ ( Y × [ u ]) N and every s ∈ (cid:0) N d (cid:1) by y s and ω s we denote the restrictionson the coordinates determined by s of y and ω respectively. ( See also Paragraph . )Proposition 3.1 immediately yields the following corollary. Corollary 3.2.
Let d, m, k (cid:62) be integers with k (cid:62) d , let ε > , and set (3.3) u (cid:48) = u (cid:48) ( d, m, k, ε ) := m k d +1 k d +1 ε − d +1 . Let X , H , ( Y , ν ) be as in Proposition . Then there exists a partition (cid:104) E a : a ∈ X (cid:105) of ( Y × [ u (cid:48) ]) d with the following property. Set E := (cid:104) E a : a ∈ X (cid:105) , and let X H and X E denote the spreadable, d -dimensional random arrays on N defined via (1.2) for H and E respectively. Then for every subset L of N of cardinality at most k we have (3.4) ρ TV ( P L , Q L ) (cid:54) ε where P L and Q L denote the laws of the subarrays X H and X E determined by L respec-tively. Corollary 3.2 asserts that the finite pieces of all distributions of the form (1.2) areessentially generated by genuine partitions instead of partitions of unity. Besides itsintrinsic interest, this information is important for the proof of Theorem 1.4.The rest of this section is devoted to the proof of Proposition 3.1. We start by pre-senting some preparatory material.3.1. Box norms.
We will use below—as well as in Section 8—the box norms introducedby Gowers [Go07]. We shall recall the definition of these norms and a couple of their basicproperties; for proofs, and a more complete presentation, we refer to [GT10, Appendix B]and [DKK20, Section 2].Let d (cid:62) , Σ , µ ) be a probability space, and let Ω d be equippedwith the product measure. For every integrable random variable h : Ω d → R we define its box norm (cid:107) h (cid:107) (cid:3) by setting(3.5) (cid:107) h (cid:107) (cid:3) := (cid:16) (cid:90) (cid:89) (cid:15) ∈{ , } d h ( ω (cid:15) ) d µ ( ω ) (cid:17) / d where µ denotes the product measure on Ω d and, for every ω = ( ω , ω , . . . , ω d , ω d ) ∈ Ω d and every (cid:15) = ( (cid:15) , . . . , (cid:15) d ) ∈ { , } d we have ω (cid:15) := ( ω (cid:15) , . . . , ω (cid:15) d d ) ∈ Ω d ; by convention, weset (cid:107) h (cid:107) (cid:3) := + ∞ if the integral in (3.5) does not exist.The quantity (cid:107)·(cid:107) (cid:3) is a norm on the vector space { h ∈ L : (cid:107) h (cid:107) (cid:3) < + ∞} , and it satisfiesthe following H¨older-type inequality, known as the Gowers–Cauchy–Schwarz inequality :for every collection (cid:104) h (cid:15) : (cid:15) ∈ { , } d (cid:105) of integrable random variables on Ω d we have(3.6) (cid:12)(cid:12)(cid:12) (cid:90) (cid:89) (cid:15) ∈{ , } d h (cid:15) ( ω (cid:15) ) d µ ( ω ) (cid:12)(cid:12)(cid:12) (cid:54) (cid:89) (cid:15) ∈{ , } d (cid:107) h (cid:15) (cid:107) (cid:3) . We will need the following simple fact which follows from Fubini’s theorem and theGowers–Cauchy–Schwarz inequality. We have stated Proposition 3.1 and Corollary 3.2 for finite probability spaces mainly because thisis the context of Theorem 1.4. But of course, by an approximation argument, one easily sees that theseresults hold true in full generality.
ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 17
Fact 3.3.
Let (Ω , Σ , µ ) be a probability space, and let µ denote the product measureon Ω N . Let d, k be positive integers with d (cid:62) , and let f, g, h , . . . , h k : Ω d → [ − , berandom variables. Also let s , s , . . . , s k ∈ (cid:0) N d (cid:1) with s (cid:54) = s i for every i ∈ [ k ] . Then, (3.7) (cid:12)(cid:12)(cid:12) (cid:90) (cid:0) f ( ω s ) − g ( ω s ) (cid:1) k (cid:89) i =1 h i ( ω s i ) d µ ( ω ) (cid:12)(cid:12)(cid:12) (cid:54) (cid:107) f − g (cid:107) (cid:3) . ( Here, we follow the notational conventions in Paragraph . )3.2. Random selection.
We will also need the following lemma.
Lemma 3.4.
Let d, m (cid:62) be integers, let ε > , and set (3.8) n = n ( d, m, ε ) := 5 d d ! mε − d +1 . Also let λ , . . . , λ m (cid:62) such that λ + · · · + λ m = 1 . Then for every finite set V with | V | (cid:62) n there exists a partition (cid:104) E , . . . , E m (cid:105) of V d into nonempty symmetric sets suchthat (cid:107) E j − λ j (cid:107) (cid:3) (cid:54) ε for every j ∈ [ m ] . ( Here, we view V as a probability space equippedwith the uniform probability measure. )Lemma 3.4 is based on a (standard) random selection and the bounded differencesinequality. We present the details in Appendix A.3.3. Proof of Proposition 3.1.
Let X , H = (cid:104) h a : a ∈ X (cid:105) , ( Y , ν ) be as in the statementof the proposition. Without loss of generality, we may assume that ν ( y ) > y ∈ Y , and consequently, we have (cid:80) a ∈X h a ( y ) = 1 for every y ∈ Y d . By Lemma 3.4 andthe choice of u in (3.1), for every y ∈ Y d there exists a partition (cid:104) E a y : a ∈ X (cid:105) of [ u ] d such that for every a ∈ X we have(3.9) (cid:107) E a y − h a ( y ) (cid:107) (cid:3) (cid:54) εκ . For every a ∈ X we set(3.10) E a := (cid:91) y ∈Y d { y } × E a y , and we observe that the family (cid:104) E a : a ∈ X (cid:105) is a partition of ( Y × [ u ]) d into nonemptysets. We claim that this partition (cid:104) E a : a ∈ X (cid:105) is as desired.Indeed, let F be a nonempty subset of (cid:0) N d (cid:1) with |F| (cid:54) κ and let ( a s ) s ∈F be a collectionof elements of X . Set κ := |F| , and let { s , . . . , s κ } be an enumeration of F . Also let λ denote the product measure on [ u ] N obtained by equipping each factor with the uniform Recall that a subset E of a Cartesian product V d is called symmetric if for every ( v , . . . , v d ) ∈ V d and every permutation π of [ d ] we have that ( v , . . . , v d ) ∈ E if and only if ( v π (1) , . . . , v π ( d ) ) ∈ E ; inparticular, for any symmetric set E , the set { ( v , . . . , v d ) ∈ E : v , . . . , v d are mutually distinct } can beidentified with a d -uniform hypergraph on V . probability measure. First observe that, by Fact 3.3 and (3.9), for every y ∈ Y N andevery j ∈ [ κ ] we have(3.11) (cid:12)(cid:12)(cid:12) (cid:90) (cid:16) (cid:89) i
Proofs of Theorems 1.4 and 1.5
In this section we present the proofs of Theorems 1.4 and 1.5. As already noted, wewill actually prove a slightly stronger theorem—Theorem 4.1 below—whose proof occupiesSubsections 4.1 up to 4.6. The deduction of Theorems 1.4 and 1.5 from Theorem 4.1 isgiven in Subsection 4.7.4.1.
Initializing various numerical invariants.
We start by introducing some numer-ical invariants. The reader is advised to skip this section at first reading.4.1.1. First, we define θ : N × R + → R + , (cid:96) : N × R + → N , m : N × R + → N and ε : N × R + → R + by setting θ ( d, k, ε ) := ε · k d (4.1) (cid:96) ( d, m, k, ε ) := k m (cid:98) /θ ( d,k,ε ) (cid:99) (4.2) m ( d, m, k, ε ) := m (cid:0) (cid:96) ( d,m,k,ε ) d (cid:1) d (4.3) ε ( d, m, k, ε ) := ε m ( d, m, k, ε ) − k d − . (4.4)4.1.2. By recursion on d , for every pair m, k of positive integers with k (cid:62) d and every ε > η ( d, m, k, ε ) , n ( d, m, k, ε ) and v ( d, m, k, ε ). For “ d = 1” we set η (1 , m, k, ε ) := ε · k m − (cid:96) (1 ,m,k,ε ) (4.5) n (1 , m, k, ε ) := ( k + 1) k (cid:96) (1 , m, k, ε )(4.6) v (1 , m, k, ε ) := 2 m (cid:96) (1 ,m,k,ε )+1 k ε − . (4.7) ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 19
Next, let d (cid:62) η ( d − , m, k, ε ), n ( d − , m, k, ε ) and v ( d − , m, k, ε ) have been defined for every choice of admissible parameters. For notationalsimplicity set m := m ( d, m, k, ε ) and ε := ε ( d, m, k, ε ), and define η ( d, m, k, ε ) := min (cid:110) ε · k d m − (cid:0) k ( d +1) (cid:96) ( d,m,k,ε ) (cid:1) d , η ( d − , m, k, ε ) (cid:111) (4.8) n ( d, m, k, ε ) := k (cid:96) ( d, m, k, ε ) · (cid:0) n ( d − , m, k, ε ) + 1 (cid:1) (4.9) v ( d, m, k, ε ) := 4 m k d d +2 ε − d +2 v ( d − , m, k, ε ) . (4.10)4.2. The main result.
We are ready to state the main result in this section.
Theorem 4.1.
Let d, m, k be positive integers with m (cid:62) and k (cid:62) d , let ε > , andlet η ( d, m, k, ε ) , n ( d, m, k, ε ) and v ( d, m, k, ε ) be the quantities defined in Subsection .Also let n (cid:62) n ( d, m, k, ε ) be a positive integer, let X be a set with |X | = m , and let X = (cid:104) X s : s ∈ (cid:0) [ n ] d (cid:1) (cid:105) be an X -valued, η ( d, m, k, ε ) -spreadable, d -dimensional randomarray on [ n ] . Then there exist a finite probability space (Ω , µ ) with | Ω | (cid:54) v ( d, m, k, ε ) anda partition (cid:104) E a : a ∈ X (cid:105) of Ω { }∪ [ d ] such that for every M ∈ (cid:0) [ n ] k (cid:1) , every nonempty subset F of (cid:0) Md (cid:1) and every collection ( a s ) s ∈F of elements of X we have (4.11) (cid:12)(cid:12)(cid:12) P (cid:16) (cid:92) s ∈F [ X s = a s ] (cid:17) − (cid:90) (cid:89) s ∈F E as ( ω { }∪ s ) d µ ( ω ) (cid:12)(cid:12)(cid:12) (cid:54) ε where µ denotes the product measure on Ω { }∪ N and, for every s = { j < · · · < j d } ∈ (cid:0) N d (cid:1) and every ω = ( ω i ) i ∈{ }∪ N ∈ Ω { }∪ N , by ω { }∪ s := ( ω , ω j , . . . , ω j d ) we denote therestriction of ω on the coordinates determined by { } ∪ s . Toolbox.
Our next goal is to collect some preliminary results which are part of theproof of Theorem 4.1, but they are not related with the main argument. Specifically, wehave the following lemma.
Lemma 4.2.
Let n, d, m, (cid:96) be positive integers with m (cid:62) and n (cid:62) ( d + 1) (cid:96) , and let s, t ∈ (cid:0) [ n ] d (cid:1) be (cid:96) -sparse. ( See Paragraph . ) Also let X be a set with |X | = m , let η (cid:62) ,and let X = (cid:104) X u : u ∈ (cid:0) [ n ] d (cid:1) (cid:105) be an X -valued, η -spreadable, d -dimensional random arrayon [ n ] . Set F := s ∪ ( ∪G s(cid:96) ) and G := t ∪ ( ∪G t(cid:96) ) where G s(cid:96) and G t(cid:96) are as Subsection .Moreover, for every collection a = ( a u ) u ∈G s(cid:96) of elements of X set (4.12) B a := (cid:92) u ∈G s(cid:96) [ X u = a u ] and C a := (cid:92) u ∈G s(cid:96) [ X I F,G ( u ) = a u ] where I F,G is as in (2.2) . ( Note that I F,G ( s ) = t . ) Finally, for every a ∈ X define (4.13) f a := (cid:88) a ∈X G s(cid:96) P (cid:0) [ X t = a ] | C a (cid:1) B a . Then, for every a ∈ X we have (4.14) (cid:13)(cid:13) f a − E (cid:2) [ X s = a ] | Σ( G s(cid:96) , X ) (cid:3)(cid:13)(cid:13) L (cid:54) (cid:113) η / m ( (cid:96) ( d +1)) d . Proof.
Observe that for every a ∈ X we have(4.15) f a − E (cid:2) [ X s = a ] | Σ( G s(cid:96) , X ) (cid:3) = (cid:88) a ∈X G s(cid:96) (cid:16) P (cid:0) [ X t = a ] | C a (cid:1) − P (cid:0) [ X s = a ] | B a (cid:1)(cid:17) B a . Hence, if η = 0, then (4.14) follows immediately by (4.15) and the spreadability of X .So in what follows we may assume that η >
0. Set A := (cid:8) a ∈ X G s(cid:96) : P ( B a ) (cid:54) η / (cid:9) and B := X G s(cid:96) \ A . Notice that for every a ∈ X and every a ∈ A we have the trivial estimate(4.16) (cid:12)(cid:12) P (cid:0) [ X t = a ] | C a (cid:1) − P (cid:0) [ X s = a ] | B a (cid:1)(cid:12)(cid:12) (cid:54) . Moreover, by the η -spreadability of X , for every a ∈ X and every a ∈ X G s(cid:96) we have(4.17) | P ( C a ) − P ( B a ) | (cid:54) η and (cid:12)(cid:12) P (cid:0) [ X t = a ] ∩ C a (cid:1) − P (cid:0) [ X s = a ] ∩ B a (cid:1)(cid:12)(cid:12) (cid:54) η. Hence, for every a ∈ X and every a ∈ B we have (cid:12)(cid:12) P (cid:0) [ X t = a ] | C a (cid:1) − P (cid:0) [ X s = a ] | B a (cid:1)(cid:12)(cid:12) = (cid:12)(cid:12)(cid:12) P (cid:0) [ X t = a ] ∩ C a (cid:1) P ( C a ) − P (cid:0) [ X s = a ] ∩ B a (cid:1) P ( B a ) (cid:12)(cid:12)(cid:12) (4.18) (cid:54) P ( B a ) (cid:12)(cid:12) P (cid:0) [ X t = a ] ∩ C a (cid:1) − P (cid:0) [ X s = a ] ∩ B a (cid:1)(cid:12)(cid:12) + P ( C a ) (cid:12)(cid:12)(cid:12) P ( C a ) − P ( B a ) (cid:12)(cid:12)(cid:12) (cid:54) η / + 1 P ( B a ) | P ( B a ) − P ( C a ) | (cid:54) η / . By (4.15), (4.16) and (4.18), we conclude that for every a ∈ X ,(4.19) (cid:13)(cid:13) f a − E (cid:2) [ X s = a ] | Σ( G s(cid:96) , X ) (cid:3)(cid:13)(cid:13) L (cid:54) (cid:88) a ∈B η / P ( B a ) + |A| η / (cid:54) η / m |G s(cid:96) | . Inequality (4.14) follows from (4.19) after observing that |G s(cid:96) | (cid:54) ( (cid:96) ( d + 1)) d . (cid:3) We will also need the following consequence of Proposition 3.1.
Corollary 4.3.
Let d, m, κ be positive integers with m (cid:62) , let ε > , and set (4.20) u = u ( d, m, κ ) := 5( d + 1) ( d + 1)! mκ d +2 ε − d +2 . Let X be a set with |X | = m , and let H = (cid:104) h a : a ∈ X (cid:105) be an X -partition of unity definedon Y { }∪ [ d ] where ( Y , ν ) is a finite probability space. Then there exist a finite probabilityspace (Ω , µ ) with | Ω | (cid:54) u |Y| and a partition (cid:104) E a : a ∈ X (cid:105) of Ω { }∪ [ d ] such that for everynonempty subset F of (cid:0) N d (cid:1) with |F| (cid:54) κ and every collection ( a s ) s ∈F of elements of X we have (4.21) (cid:12)(cid:12)(cid:12) (cid:90) (cid:89) s ∈F h a s ( y { }∪ s ) d ν ( y ) − (cid:90) (cid:89) s ∈F E as ( ω { }∪ s ) d µ ( ω ) (cid:12)(cid:12)(cid:12) (cid:54) ε. ( Here, we follow the conventions in Proposition and Theorem . ) ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 21
The inductive hypothesis.
We have already mentioned in the introduction thatthe proof of Theorem 4.1 proceeds by induction on d . Specifically, for every positiveinteger d by P( d ) we shall denote the following statement. Let the parameters m, k, ε , n ( d, m, k, ε ) , η ( d, m, k, ε ) , v ( d, m, k, ε ) and the notation be asin Theorem . Then for every integer n (cid:62) n ( d, m, k, ε ) , every set X with |X | = m andevery X -valued, η ( d, m, k, ε ) -spreadable, d -dimensional random array X = (cid:104) X s : s ∈ (cid:0) [ n ] d (cid:1) (cid:105) on [ n ] there exist a finite probability space (Ω , µ ) with | Ω | (cid:54) v ( d, m, k, ε ) and a partition (cid:104) E a : a ∈ X (cid:105) of Ω { }∪ [ d ] such that for every M ∈ (cid:0) [ n ] k (cid:1) , every nonempty subset F of (cid:0) Md (cid:1) and every collection ( a s ) s ∈F of elements of X we have (4.22) (cid:12)(cid:12)(cid:12) P (cid:16) (cid:92) s ∈F [ X s = a s ] (cid:17) − (cid:90) (cid:89) s ∈F E as ( ω { }∪ s ) d µ ( ω ) (cid:12)(cid:12)(cid:12) (cid:54) ε. Notice that Theorem 4.1 is equivalent to the validity of P( d ) for every integer d (cid:62) The base case “ d = 1 ”. In this subsection we establish P(1). We note that thiscase is, essentially, the analogue of the results of Diaconis and Freedman [DF80] forapproximately spreadable random vectors. The proofs, however, are rather different, andthe bounds we obtain are quite weaker than those in [DF80]; this is mainly due to thefact that we are dealing with random vectors whose distribution is much less symmetric.We proceed to the details. Let m, k be positive integers with m (cid:62)
2, and let ε > θ := θ (1 , k, ε ), (cid:96) := (cid:96) (1 , m, k, ε ) and η := η (1 , k, m, ε )where θ (1 , k, ε ), (cid:96) (1 , m, k, ε ) and η (1 , k, m, ε ) are as in (4.1), (4.2) and (4.5) respectively.Let n, X and X = (cid:104) X s : s ∈ (cid:0) [ n ]1 (cid:1) (cid:105) be as in P(1), and define(4.23) L := (cid:8) jk(cid:96) : j ∈ [ k ] (cid:9) . Notice that L is a ( k(cid:96) )-sparse subset of [ n ] with | L | = k . Since n (cid:62) n (1 , m, k, ε ), by thechoice of n (1 , m, k, ε ) in (4.6), Proposition 2.1 and the choice of (cid:96) , there exists (cid:96) ∈ [ (cid:96) ]such that for every nonempty subset F of (cid:0) L (cid:1) and every collection ( a s ) s ∈F of elementsof X we have(4.24) (cid:12)(cid:12)(cid:12) P (cid:16) (cid:92) s ∈F [ X s = a s ] (cid:17) − E (cid:104) (cid:89) s ∈F E (cid:2) [ X s = a s ] | Σ( G s(cid:96) , X ) (cid:3)(cid:105)(cid:12)(cid:12)(cid:12) (cid:54) k (cid:113) θ + 15 ηm k m (cid:98) /θ (cid:99) +1 . Fix t ∈ (cid:0) L (cid:1) and set G := G t (cid:96) . By (2.6), we see that G s(cid:96) = G for every s ∈ (cid:0) L (cid:1) . By Lemma4.2, the previous observation and the fact that (cid:96) (cid:54) (cid:96) , for every s ∈ (cid:0) L (cid:1) and every a ∈ X we have(4.25) (cid:13)(cid:13) E (cid:2) [ X s = a ] | Σ( G s(cid:96) , X ) (cid:3) − E (cid:2) [ X t = a ] | Σ( G , X ) (cid:3)(cid:13)(cid:13) L (cid:54) η / m (cid:96) . Moreover, notice that(4.26) k (cid:113) θ + 15 ηm k m (cid:98) /θ (cid:99) +1 + 2 kη / m (cid:96) (cid:54) ε . By (4.24)–(4.26), the Cauchy–Schwarz inequality, the fact that | (cid:0) L (cid:1) | = k and a telescopicargument, we conclude that for every nonempty subset F of (cid:0) L (cid:1) and every collection( a s ) s ∈F of elements of X ,(4.27) (cid:12)(cid:12)(cid:12) P (cid:16) (cid:92) s ∈F [ X s = a s ] (cid:17) − E (cid:104) (cid:89) s ∈F E (cid:2) [ X t = a s ] | Σ( G , X ) (cid:3)(cid:105)(cid:12)(cid:12)(cid:12) (cid:54) ε . Next, set Y := X G and define a probability measure ν on Y by the rule(4.28) ν ( a ) := P (cid:16) (cid:92) s ∈G [ X s = a s ] (cid:17) for every a = ( a s ) s ∈G ∈ Y . Moreover, for every a ∈ X define h (cid:48) a : Y → [0 ,
1] by setting forevery a = ( a s ) s ∈G ∈ Y ,(4.29) h (cid:48) a ( a ) := P (cid:16) [ X t = a ] (cid:12)(cid:12)(cid:12) (cid:92) s ∈G [ X s = a s ] (cid:17) . Observe that (cid:104) h (cid:48) a : a ∈ X (cid:105) is an X -partition of unity, and for every nonempty subset F of (cid:0) L (cid:1) and every collection ( a s ) s ∈F of elements of X we have(4.30) E (cid:104) (cid:89) s ∈F E (cid:2) [ X t = a s ] | Σ( G , X ) (cid:3)(cid:105) = (cid:90) (cid:89) s ∈F h (cid:48) a s ( a ) dν ( a ) . This information is already strong enough, but we need to write it in a form which issuitable for the induction.Specifically, for every a ∈ X we define the function h a : Y { }∪ [1] → [0 ,
1] by setting h a ( a , a ) := h (cid:48) a ( a ) for every ( a , a ) ∈ Y { }∪ [1] . Again observe that (cid:104) h a : a ∈ X (cid:105) isan X -partition of unity, and for every nonempty subset F of (cid:0) L (cid:1) and every collection( a s ) s ∈F of elements of X we have(4.31) (cid:90) (cid:89) s ∈F h (cid:48) a s ( a ) dν ( a ) = (cid:90) (cid:89) s ∈F h a s ( y { }∪ s ) d ν ( y )where ν denotes the product measure on Y { }∪ N obtained by equipping each factor withthe measure ν . By the choice of v (1 , k, m, ε ) in (4.7), the fact that |Y| (cid:54) m k m (cid:98) /θ (cid:99) andCorollary 4.3 applied for “ κ = k ”, “ d = 1” and “ ε = ε/ , µ ) with | Ω | (cid:54) v (1 , k, m, ε ) and a partition (cid:104) E a : a ∈ X (cid:105) of Ω { }∪ [1] such that forevery nonempty subset F of (cid:0) L (cid:1) and every collection ( a s ) s ∈F of elements of X we have(4.32) (cid:12)(cid:12)(cid:12) (cid:90) (cid:89) s ∈F h a s ( y { }∪ s ) d ν ( y ) − (cid:90) (cid:89) s ∈F E as ( ω { }∪ s ) d µ ( ω ) (cid:12)(cid:12)(cid:12) (cid:54) ε (cid:12)(cid:12)(cid:12) P (cid:16) (cid:92) s ∈F [ X s = a s ] (cid:17) − (cid:90) (cid:89) s ∈F E as ( ω { }∪ s ) d µ ( ω ) (cid:12)(cid:12)(cid:12) (cid:54) ε . ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 23
Finally, by (4.33) and the η -spreadability of X , we see that if M ∈ (cid:0) [ n ] k (cid:1) is arbitrary, thenfor every nonempty subset F of (cid:0) M (cid:1) and every collection ( a s ) s ∈F of elements of X ,(4.34) (cid:12)(cid:12)(cid:12) P (cid:16) (cid:92) s ∈F [ X s = a s ] (cid:17) − (cid:90) (cid:89) s ∈F E as ( ω { }∪ s ) d µ ( ω ) (cid:12)(cid:12)(cid:12) (cid:54) ε η (4.5) (cid:54) ε. The proof of the case “ d = 1” is completed.4.6. The general inductive step.
Let d (cid:62) d − d ) also holds true. Clearly, this is enough to completethe proof of Theorem 4.1.We fix a pair m, k of positive integers with k (cid:62) d and m (cid:62)
2, and ε >
0. As in theprevious subsection, for notation convenience, we set(4.35) (cid:96) := (cid:96) ( d, m, k, (cid:96) ) , m := m ( d, m, k, (cid:96) ) and ε := ε ( d, m, k, (cid:96) ) . where (cid:96) ( d, m, k, (cid:96) ), m ( d, m, k, (cid:96) ) and ε ( d, m, k, (cid:96) ) are as in (4.2), (4.3) and (4.4) respec-tively. Also let the parameters n ( d, m, k, ε ) and(4.36) η := η ( d, m, k, ε )be as in Subsection 4.1, and let n, X and X = (cid:104) X s : s ∈ (cid:0) [ n ] d (cid:1) (cid:105) be as in P( d ). We set(4.37) Q := (cid:8) jk(cid:96) : j ∈ [ n ( d − , m, k, ε )] (cid:9) and(4.38) L := (cid:8) jk(cid:96) : j ∈ [ k ] (cid:9) . Notice that both L and Q are ( k(cid:96) )-sparse subsets of [ n ]. All these data will fixed in therest of this subsection.
Step 1: application of the approximation.
First observe that, by the selection inSubsection 4.1, we have(4.39) k d (cid:113) θ ( d, k, ε ) + 15 η ( d, m, k, ε ) m ( κ ( d +1) (cid:96) ) d (cid:54) ε . Since L is a ( k(cid:96) )-sparse subset of [ n ] with | L | = k and n (cid:62) n ( d, m, k, ε ), by Proposition2.1 and (4.39), there exists (cid:96) ∈ [ (cid:96) ] such that for every nonempty subset F of (cid:0) Ld (cid:1) andevery collection ( a s ) s ∈F of elements of X we have(4.40) (cid:12)(cid:12)(cid:12) P (cid:16) (cid:92) s ∈F [ X s = a s ] (cid:17) − E (cid:104) (cid:89) s ∈F E (cid:2) [ X s = a s ] | Σ( G s(cid:96) , X ) (cid:3)(cid:105)(cid:12)(cid:12)(cid:12) (cid:54) ε . Step 2: application of the shift invariance property.
Fix t ∈ (cid:0) Qd (cid:1) , and set G := G t (cid:96) and M t := t ∪ ( ∪G t (cid:96) ). Moreover, for every s ∈ (cid:0) Qd (cid:1) , every a = ( a u ) u ∈G ∈ X G and every a ∈ X set(4.41) M s := s ∪ ( ∪G s(cid:96) ) , B s a := (cid:92) u ∈G [ X I Mt ,Ms ( u ) = a u ] and λ a a := P (cid:0) [ X t = a ] | B t a (cid:1) where I M t ,M s is as in (2.2). Finally, for every s ∈ (cid:0) Qd (cid:1) and every a ∈ X set(4.42) f as := (cid:88) a ∈X G λ a a B s a and notice that, by Lemma 4.2 and the fact that (cid:96) (cid:54) (cid:96) ,(4.43) (cid:13)(cid:13) f as − E (cid:2) [ X s = a ] | Σ( G s(cid:96) , X ) (cid:3)(cid:13)(cid:13) L (cid:54) (cid:113) η / m ( (cid:96) ( d +1)) d . On the other hand, it is easy to see that(4.44) 2 k d (cid:113) η / m ( (cid:96) ( d +1)) d (cid:54) ε . By (4.43) and (4.44), the Cauchy–Schwartz inequality, the observation that (cid:107) f as (cid:107) L ∞ (cid:54) | (cid:0) Ld (cid:1) | (cid:54) k d and a telescopic argument, we obtain that for every nonemptysubset F of (cid:0) Ld (cid:1) and every collection ( a s ) s ∈F of elements of X ,(4.45) (cid:12)(cid:12)(cid:12) E (cid:104) (cid:89) s ∈F E (cid:2) [ X s = a s ] | Σ( G s(cid:96) , X ) (cid:3)(cid:105) − E (cid:104) (cid:89) s ∈F f a s s (cid:105)(cid:12)(cid:12)(cid:12) (cid:54) ε . Step 3: application of the inductive hypothesis.
Fix y ∈ (cid:0) Qd − (cid:1) , and set R := R y (cid:96) and L y := ∪R y (cid:96) where R y (cid:96) is as in (2.3). Furthermore, for every x ∈ (cid:0) Qd − (cid:1) and every b = ( b u ) u ∈R ∈ X R set(4.46) L x := ∪R x(cid:96) and C x b := (cid:92) u ∈R [ X I Ly ,Lx ( u ) = b u ] . (Here, I L y ,L x is as in (2.2).) Next, set(4.47) Z := X R and let Y := (cid:104) Y x : x ∈ (cid:0) Qd − (cid:1) (cid:105) denote the Z -valued, ( d − Q defined by setting [ Y x = b ] = C x b for every x ∈ (cid:0) Qd − (cid:1) and every b ∈ Z . Since therandom array X is η -spreadable and η = η ( d, m, k, ε ) (cid:54) η ( d − , m, k, ε ), we see that Y is η ( d − , m, k, ε )-spreadable. Moreover, by (4.37), we have that | Q | = n ( d − , m, k, ε ).Therefore, by the fact that |Z| (cid:54) m and our inductive hypothesis that property P( d − Y , ν ) with |Y| (cid:54) v ( d − , m, k, ε ) and apartition (cid:104) E (cid:48) b : b ∈ Z(cid:105) of Y { }∪ [ d − such that for every nonempty subset Γ of (cid:0) Ld − (cid:1) andevery collection ( b x ) x ∈ Γ of elements of Z we have(4.48) (cid:12)(cid:12)(cid:12) P (cid:16) (cid:92) x ∈ Γ [ Y x = b x ] (cid:17) − (cid:90) (cid:89) x ∈ Γ E (cid:48) b x ( y { }∪ x ) d ν ( y ) (cid:12)(cid:12)(cid:12) (cid:54) ε. ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 25
Step 4: compatibility.
It is convenient to introduce the following notation. Forevery s ∈ (cid:0) N d (cid:1) set(4.49) ∂s := (cid:18) sd − (cid:19) . Next, given a = ( a u ) u ∈G ∈ X G and β = ( b z ) z ∈ ∂t = (cid:104) b zu : u ∈ R , z ∈ ∂t (cid:105) ∈ Z ∂t , we saythat the pair ( a , β ) is compatible provided that for every u ∈ G , every u (cid:48) ∈ R and every z ∈ ∂t , if we have I L y ,L z ( u (cid:48) ) = u , then a u = b zu (cid:48) . Set(4.50) B := (cid:8) β ∈ Z ∂t : there exists a ∈ X G such that the pair ( a , β ) is compatible (cid:9) . Notice that for every β ∈ B there exists a unique a ∈ X G such that the pair ( a , β ) iscompatible, and conversely, for every a ∈ X G the exists a unique β ∈ B such that thepair ( a , β ) is compatible. This observation enables us to define the map T : B → X G bysetting T ( β ) to be the unique element of X G such that the pair (cid:0) T ( β ) , β (cid:1) is compatible.Observe that for every β = ( b z ) z ∈ ∂t ∈ B and every s ∈ (cid:0) Qd (cid:1) we have the identity(4.51) B sT ( β ) = (cid:92) x ∈ ∂s C x b I s,t x ) where B sT ( β ) is as in (4.41), and for every x ∈ ∂s the event C x b I s,t x ) is as in (4.46); onthe other hand, notice that for every ( b z ) z ∈ ∂t ∈ Z ∂t \ B and every s ∈ (cid:0) Qd (cid:1) we have(4.52) (cid:92) x ∈ ∂s C x b I s,t x ) = ∅ . Having these observations in mind, for every a ∈ X and every β ∈ Z ∂t we define(4.53) λ a β := λ aT ( β ) if β ∈ B , λ aT ( β ) is as in (4.41). By (4.51), (4.52), (4.53) and (4.42), it follows in particularthat for every s ∈ (cid:0) Qd (cid:1) and every a ∈ X we have(4.54) f as = (cid:88) β =( b z ) z ∈ ∂t ∈Z ∂t λ a β (cid:89) x ∈ ∂s C x b I s,t x ) . Step 5: definition of the partition of unity.
Now, for every a ∈ X we define afunction h a : Y { }∪ [ d ] → [0 ,
1] by setting for every y ∈ Y { }∪ [ d ] ,(4.55) h a ( y ) := (cid:88) β =( b z ) z ∈ ∂t ∈Z ∂t λ a β (cid:89) x ∈ ∂ [ d ] E (cid:48) b I t x ) ( y { }∪ x ) . For every β = ( b z ) z ∈ ∂t ∈ Z ∂t the map(4.56) Y { }∪ [ d ] (cid:51) y (cid:55)→ (cid:89) x ∈ ∂ [ d ] E (cid:48) b I t x ) ( y { }∪ x )is clearly boolean, and so, it is equal to the indicator function of some subset of Y { }∪ [ d ] which we shall denote by D β . Using the fact that the family (cid:104) E (cid:48) b : b ∈ Z(cid:105) is a partition of Y { }∪ [ d − , we see that the family (cid:104) D β : β ∈ Z ∂t (cid:105) is also a partition of Y { }∪ [ d ] withpossibly empty parts; therefore, by (4.53) and (4.41), we conclude that the collection H = (cid:104) h a : a ∈ X (cid:105) is an X -partition of unity.4.6.6. Step 6: application of the coding.
Recall that |Y| (cid:54) v ( d − , m, k, ε ). Therefore,by (4.10) and Corollary 4.3 applied for “ κ = (cid:0) kd (cid:1) ” and “ ε = ε/ , µ ) with | Ω | (cid:54) v ( d, m, k, ε ) and a partition (cid:104) E a : a ∈ X (cid:105) of Ω { }∪ [ d ] such that for every nonempty subset F of (cid:0) Ld (cid:1) and every collection ( a s ) s ∈F of elementsof X we have(4.57) (cid:12)(cid:12)(cid:12) (cid:90) (cid:89) s ∈F h a s ( y { }∪ s ) d ν ( y ) − (cid:90) (cid:89) s ∈F E as ( ω { }∪ s ) d µ ( ω ) (cid:12)(cid:12)(cid:12) (cid:54) ε . Step 7: verification of the inductive hypothesis.
Let F be an arbitrary nonemptysubset of (cid:0) Ld (cid:1) and let ( a s ) s ∈F be an arbitrary collection of elements of X . We set(4.58) Γ := (cid:110) x ∈ (cid:18) Ld − (cid:19) : x ∈ ∂s for some s ∈ F (cid:111) . By (4.54), we have (cid:89) s ∈F f a s s = (cid:89) s ∈F (cid:88) β =( b z ) z ∈ ∂t ∈Z ∂t λ a s β (cid:89) x ∈ ∂s C x b I s,t x ) (4.59) = (cid:88) (cid:104) b z,s : z ∈ ∂t ,s ∈F(cid:105)∈ ( Z ∂t ) F (cid:89) s ∈F (cid:16) λ a s ( b z,s ) z ∈ ∂t (cid:89) x ∈ ∂s C x b I s,t x ) ,s (cid:17) = (cid:88) (cid:104) b z,s : z ∈ ∂t ,s ∈F(cid:105)∈ ( Z ∂t ) F (cid:16) (cid:89) s ∈F λ a s ( b z,s ) z ∈ ∂t (cid:17) (cid:89) s ∈F (cid:89) x ∈ ∂s C x b I s,t x ) ,s . = (cid:88) (cid:104) b z,s : z ∈ ∂t ,s ∈F(cid:105)∈ ( Z ∂t ) F (cid:16) (cid:89) s ∈F λ a s ( b z,s ) z ∈ ∂t (cid:17) (cid:89) x ∈ Γ (cid:89) { s ∈F : x ∈ ∂s } C x b I s,t x ) ,s . Fix x ∈ Γ and let s , s ∈ F such that x ∈ ∂s and x ∈ ∂s . Note that if we have b I s ,t ( x ) ,s (cid:54) = b I s ,t ( x ) ,s then the events C x b I s ,t x ) ,s and C x b I s ,t x ) ,s are disjoint; inparticular, all these terms in the above sum vanish. On the other hand, in the remainingterms, the last product collapses since it consists of indicators of the same event. Thus,we have (cid:89) s ∈F f a s s (4.59) = (cid:88) ( b x ) x ∈ Γ ∈Z Γ (cid:16) (cid:89) s ∈F λ a s ( b I t ,s ( z ) ) z ∈ ∂t (cid:17) (cid:89) x ∈ Γ C x b x (4.60) = (cid:88) ( b x ) x ∈ Γ ∈Z Γ (cid:16) (cid:89) s ∈F λ a s ( b I t ,s ( z ) ) z ∈ ∂t (cid:17) (cid:89) x ∈ Γ [ Y x = b x ] . Since |Z Γ | (cid:54) ( m ) k d − and ε = ( ε/ m ) − k d − , by (4.60) and (4.48), we have(4.61) (cid:12)(cid:12)(cid:12) E (cid:104) (cid:89) s ∈F f a s s (cid:105) − (cid:88) ( b x ) x ∈ Γ ∈Z Γ (cid:16) (cid:89) s ∈F λ a s ( b I t ,s ( z ) ,s ) z ∈ ∂t (cid:17) (cid:90) (cid:89) x ∈ Γ E (cid:48) b x ( y { }∪ x ) d ν ( y ) (cid:12)(cid:12)(cid:12) (cid:54) ε . ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 27
Moreover, arguing precisely as above and using the fact that the family (cid:104) E (cid:48) b : b ∈ Z(cid:105) isa partition, we obtain that (cid:88) ( b x ) x ∈ Γ ∈Z Γ (cid:16) (cid:89) s ∈F λ a s ( b I t ,s ( z ) ,s ) z ∈ ∂t (cid:17) (cid:90) (cid:89) x ∈ Γ E (cid:48) b x ( y { }∪ x ) d ν ( y ) =(4.62)= (cid:90) (cid:88) (cid:104) b z,s : z ∈ ∂t ,s ∈F(cid:105)∈ ( Z ∂t ) F (cid:16) (cid:89) s ∈F λ a s ( b z,s ) z ∈ ∂t (cid:17) (cid:89) s ∈F (cid:89) x ∈ ∂s E (cid:48) b I s,t x ) ,s ( y { }∪ x ) d ν ( y )= (cid:90) (cid:88) (cid:104) b z,s : z ∈ ∂t ,s ∈F(cid:105)∈ ( Z ∂t ) F (cid:89) s ∈F (cid:16) λ a s ( b z,s ) z ∈ ∂t (cid:89) x ∈ ∂s E (cid:48) b I s,t x ) ,s ( y { }∪ x ) (cid:17) d ν ( y )= (cid:90) (cid:89) s ∈F (cid:88) ( b z ) z ∈ ∂t ∈Z ∂t λ a s ( b z ) z ∈ ∂t (cid:89) x ∈ ∂s E (cid:48) b I s,t x ) ( y { }∪ x ) d ν ( y )= (cid:90) (cid:89) s ∈F (cid:88) ( b z ) z ∈ ∂t ∈Z ∂t λ a s ( b z ) z ∈ ∂t (cid:89) x ∈ ∂ [ d ] E (cid:48) b I t x ) ( y { }∪ I s ( x ) ) d ν ( y ) (4.55) = (cid:90) (cid:89) s ∈F h a s ( y { }∪ s ) d ν ( y ) . By (4.40), (4.45), (4.61), (4.62) and (4.57), for every nonempty subset F of (cid:0) Ld (cid:1) and everycollection ( a s ) s ∈F of elements of X we have(4.63) (cid:12)(cid:12)(cid:12) P (cid:16) (cid:92) s ∈F [ X s = a s ] (cid:17) − (cid:90) (cid:89) s ∈F E as ( ω { }∪ s ) d µ ( ω ) (cid:12)(cid:12)(cid:12) (cid:54) ε . Finally, using the η -spreadability of X and (4.63), we conclude that for every M ∈ (cid:0) [ n ] k (cid:1) ,every nonempty subset F of (cid:0) M (cid:1) and every collection ( a s ) s ∈F of elements of X we have(4.64) (cid:12)(cid:12)(cid:12) P (cid:16) (cid:92) s ∈F [ X s = a s ] (cid:17) − (cid:90) (cid:89) s ∈F E as ( ω { }∪ s ) d µ ( ω ) (cid:12)(cid:12)(cid:12) (cid:54) ε η (4.8) (cid:54) ε. This shows that property P( d ) is satisfied, and so the entire proof of Theorem 4.1 iscompleted.4.7. Proofs of Theorems 1.4 and 1.5.
Invoking the definition of all relevant parame-ters in Subsection 4.1 and proceeding by induction on d , it is not hard to see that η ( d, m, k, ε ) − (cid:54) exp (2 d ) (cid:16) m k d ε (cid:17) (4.65) n ( d, m, k, ε ) (cid:54) exp (2 d ) (cid:16) m k d ε (cid:17) (4.66) v ( d, m, k, ε ) (cid:54) exp (2 d ) (cid:16) m k d ε (cid:17) (4.67)for every triple d, m, k of positive integers with m (cid:62) k (cid:62) d , and every 0 < ε (cid:54) C ( d, m, k, ε ) in (1.4). Remark . The tower type dependence of the parameters η ( d, m, k, ε ), n ( d, m, k, ε ) and v ( d, m, k, ε ) with respect to the dimension d is, of course, a byproduct of the inductivenature of the proof of Theorem 4.1. It would be very interesting—and also importantfor certain applications—if these bounds could be improved to a single, or even double,exponential behavior. 5. Orbits
The following definition plays an important role in the proof of Theorem 1.6.
Definition 5.1 (Orbits) . Let X = (cid:104) X i : i ∈ I (cid:105) be a family of real-valued random variablesdefined on a common probability space, indexed by a set I with | I | (cid:62) , and such that (cid:107) X i (cid:107) L = 1 for every i ∈ I . Also let η (cid:62) . We say that X is an η -orbit ( and simply an orbit if η = 0) if for every pair { i , j } and { i , j } of doubletons of I we have (cid:12)(cid:12) E [ X i X j ] − E [ X i X j ] (cid:12)(cid:12) (cid:54) η. (5.1)Arguably, the simplest example of an orbit is a (finite or infinite) sequence of indepen-dent random variables with zero mean and unit variance. Note, however, that the notionof an orbit is significantly less restrictive than independence—we will see several morerefined examples of orbits in Sections 6 and 7.It is intuitively clear that an orbit is a stochastic process which “everywhere looks thesame”. We formalize this basic intuition in the following proposition. Proposition 5.2 (Universality) . Let η (cid:62) , and let X = (cid:104) X i : i ∈ I (cid:105) be an η -orbit. Alsolet F , G be finite subsets of I with |F| , |G| (cid:62) , and set Z F := 1 |F| (cid:88) i ∈F X i and Z G := 1 |G| (cid:88) i ∈G X i . (5.2) Then we have (cid:107) Z F − Z G (cid:107) L (cid:54) (cid:16) {|F| , |G|} + η (cid:17) / . (5.3) Proof.
Fix κ, (cid:96) ∈ I with κ (cid:54) = (cid:96) , and set δ := E [ X κ X (cid:96) ]. Since X is an η -orbit, we see that δ − η (cid:54) E [ X i X j ] (cid:54) δ + η for every i, j ∈ I with i (cid:54) = j ; also recall that E [ X i ] = 1 for every i ∈ I . Therefore, E [ Z F ] = 1 |F| (cid:88) i,j ∈F E [ X i X j ] = 1 |F| (cid:88) i ∈F E [ X i ] + 1 |F| (cid:88) i,j ∈F i (cid:54) = j E [ X i X j ](5.4) (cid:54) |F| + (cid:16) − |F| (cid:17) ( δ + η ) . Similarly, we obtain that(5.5) E [ Z G ] (cid:54) |G| + (cid:16) − |G| (cid:17) ( δ + η ) . ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 29
On the other hand, we have E [ Z F Z G ] = 1 |F| · |G| (cid:88) i ∈F ,j ∈G E [ X i X j ](5.6) = 1 |F| · |G| (cid:88) i ∈F∩G E [ X i ] + 1 |F| · |G| (cid:88) i ∈F ,j ∈G i (cid:54) = j E [ X i X j ] (cid:62) |F ∩ G||F| · |G| + (cid:16) − |F ∩ G||F| · |G| (cid:17) ( δ − η ) . Finally, by the Cauchy–Schwarz inequality, we have | δ | (cid:54)
1. Using this observation, theestimate (5.3) follows from the identity (cid:107) Z F − Z G (cid:107) L = E [ Z F ] + E [ Z G ] − E [ Z F Z G ] andinequalities (5.4)–(5.6). (cid:3) Proposition 5.2 will mostly be used in the following form. (The proof follows immedi-ately from Proposition 5.2 and the Cauchy–Schwarz inequality.)
Corollary 5.3.
Let η (cid:62) , let X = (cid:104) X i : i ∈ I (cid:105) be an η -orbit, and let F , G be finitesubsets of I with |F| , |G| (cid:62) . Then for every random variable Y with (cid:107) Y (cid:107) L (cid:54) we have (cid:12)(cid:12)(cid:12) |F| (cid:88) i ∈F E [ X i Y ] − |G| (cid:88) i ∈G E [ X i Y ] (cid:12)(cid:12)(cid:12) (cid:54) (cid:16) {|F| , |G|} + η (cid:17) / . (5.7) In particular, if ϑ (cid:62) is such that (a) (cid:12)(cid:12) E [ X i Y ] − E [ X j Y ] (cid:12)(cid:12) (cid:54) ϑ for every i, j ∈ F , and (b) (cid:12)(cid:12) E [ X i Y ] − E [ X j Y ] (cid:12)(cid:12) (cid:54) ϑ for every i, j ∈ G ,then for every i ∈ F and every j ∈ G we have (cid:12)(cid:12) E [ X i Y ] − E [ X j Y ] (cid:12)(cid:12)(cid:12) (cid:54) (cid:16) {|F| , |G|} + η (cid:17) / + 2 ϑ. (5.8)6. Comparing two-point correlations of spreadable random arrays
Motivation.
Let n (cid:62) X = (cid:104) X s : s ∈ (cid:0) [ n ]2 (cid:1) (cid:105) is areal-valued, two-dimensional random array on [ n ] such that (cid:107) X s (cid:107) L = 1 for all s ∈ (cid:0) [ n ]2 (cid:1) .We wish to compare the correlations α := E [ X { , } X { , } ] and β := E [ X { , } X { , } ] . Of course, if X is exchangeable, then α = β . On the other hand, in X is spreadable,then Kallenberg’s representation theorem [Kal92] and an ultraproduct argument yieldthat α = β + o n →∞ (1) but with an ineffective error term. We will see, however, that(6.1) | α − β | (cid:54) √ n . In other words, the symmetries of a finite, spreadable, high-dimensional random arraywith square-integrable entries, impose explicit restrictions on its two-point correlations.
In order to see that (6.1) is satisfied, let n (cid:62)
10 be an integer (if n (cid:54)
9, then (6.1)is straightforward), let (cid:96) be the largest positive integer such that 2 (cid:96) + 3 < n , and noticethat (cid:96) (cid:62) n/
4. Also observe that, by the spreadability of X , for every i ∈ { , . . . , (cid:96) + 1 } we have β = E [ X { ,(cid:96) +2 } X { i,n } ], and on the other hand for every j ∈ { (cid:96) + 3 , . . . , (cid:96) + 3 } we have α = E [ X { ,(cid:96) +2 } X { j,n } ]. Therefore, setting Y := (1 /(cid:96) ) (cid:96) +1 (cid:88) i =2 X { i,n } and Z := (1 /(cid:96) ) (cid:96) +3 (cid:88) j = (cid:96) +3 X { j,n } , we obtain that(6.2) α = E [ X { ,(cid:96) +2 } Z ] and β = E [ X { ,(cid:96) +2 } Y ] . The main observation, which follows readily from the spreadability of X , is that theprocess (cid:104) X { k,n } : k ∈ { , . . . , (cid:96) + 1 } ∪ { (cid:96) + 3 , . . . , (cid:96) + 3 }(cid:105) is an orbit in the sense ofDefinition 5.1. This information together with (6.2) and Corollary 5.3 yield (6.1).6.2. Our goal in this section is to study of the phenomenon outlined above and to char-acterize, combinatorially, when two two-point correlations of a spreadable random arrayessentially coincide. To this end, we need the following analogue of the notion of analigned pair of partial maps which was introduced in Paragraph 1.4.1. Definition 6.1 (Aligned pair of sets) . Let d be a positive integer, and let s , s ∈ (cid:0) N d (cid:1) bedistinct. We say that the pair { s , s } is aligned if there exists a proper ( possibly empty ) subset G of [ d ] such that: (i) I s (cid:22) G = I s (cid:22) G , and (ii) I s (cid:0) [ d ] \ G (cid:1) ∩ I s (cid:0) [ d ] \ G (cid:1) = ∅ . ( Here, I s and I s denote the canonical isomorphisms associated with the sets s and s ;see Paragraph . ) We call the set G the root of { s , s } and we denote it by r ( s , s ) . We have the following proposition.
Proposition 6.2.
Let n, d be positive integers with n (cid:62) d + 2 , and let s , s , t , t ∈ (cid:0) [ n ] d (cid:1) with s (cid:54) = s and t (cid:54) = t . Assume that the pairs { s , s } and { t , t } are aligned and havethe same root. If X = (cid:104) X s : s ∈ (cid:0) [ n ] d (cid:1) (cid:105) is a real-valued, spreadable, d -dimensional randomarray on [ n ] such that (cid:107) X s (cid:107) L = 1 for all s ∈ (cid:0) [ n ] d (cid:1) , then (6.3) (cid:12)(cid:12) E [ X s X s ] − E [ X t X t ] (cid:12)(cid:12) (cid:54) d √ n . Remark . By considering spreadable, high-dimensional random arrays whose distri-bution is of the form (1.2), it is not hard to see that the assumption in Proposition 6.2(namely, the fact that the pairs { s , s } and { t , t } are aligned and have the same root)is essentially optimal.The proof of Proposition 6.2 is based on the following lemma. Lemma 6.4.
Let n, d, s , s , t , t be as in Proposition . Assume that the pairs { s , s } and { t , t } are aligned, and that there exists i ∈ [ d ] with the following properties. ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 31 (i)
We have I s ( i ) < I s ( i ) and I t ( i ) < I t ( i ) . (ii) We have I s (cid:22) ([ d ] \ { i } ) = I t (cid:22) ([ d ] \ { i } ) and I s (cid:22) ([ d ] \ { i } ) = I t (cid:22) ([ d ] \ { i } ) .If X = (cid:104) X s : s ∈ (cid:0) [ n ] d (cid:1) (cid:105) is a real-valued, spreadable, d -dimensional random array on [ n ] such that (cid:107) X s (cid:107) L = 1 for all s ∈ (cid:0) [ n ] d (cid:1) , then (6.4) (cid:12)(cid:12) E [ X s X s ] − E [ X t X t ] (cid:12)(cid:12) (cid:54) √ n . Proof.
Since X is spreadable, we may assume that there exist subintervals L and L of [ n ] with | L | = | L | = (cid:98) ( n − d + 1) / (cid:99) and satisfying the following properties.( P
1) We have max( L ) < I s ( i ) < min( L ).( P
2) If i >
1, then I s ( i − < min( L ) and I s ( i − < min( L ).( P
3) If i < d , then max( L ) < I s ( i + 1) and max( L ) < I s ( i + 1).Set L := L ∪ L and (cid:96) := (cid:98) ( n − d + 1) / (cid:99) ; also set g j := (cid:0) s \ { I s ( i ) } (cid:1) ∪ { I L ( j ) } ∈ (cid:18) [ n ] d (cid:19) for every j ∈ [2 (cid:96) ]. Using the spreadability of X again, we obtain that( P E [ X s X g j ] = E [ X t X t ] for every j ∈ [ (cid:96) ], and( P E [ X s X g j ] = E [ X s X s ] for every j ∈ [2 (cid:96) ] \ [ (cid:96) ].By the spreadability of X once again, we see that the collection (cid:104) X g j : j ∈ [2 (cid:96) ] (cid:105) is anorbit and, moreover, E [ X s X g i ] = E [ X s X g j ] if either i, j ∈ [ (cid:96) ], or i, j ∈ [2 (cid:96) ] \ [ (cid:96) ]. Theresult follows using the previous remarks, Corollary 5.3 and the fact that n (cid:62) d + 2. (cid:3) We are ready to give the proof of Proposition 6.2.
Proof of Proposition . Note that, by applying successively Lemma 6.4 at most d times, we obtain the following. Let s , s , s , s ∈ (cid:0) [ n ] d (cid:1) with s (cid:54) = s and s (cid:54) = s . Assume that the pairs { s , s } and { s , s } are aligned and have the same root G . Assume, moreover, that (i) I s (cid:22) G = I s (cid:22) G , (ii) I s (cid:22) G = I s (cid:22) G , and (iii) for every interval I of [ d ] \ G we have max (cid:0) I s ( I ) (cid:1) < min (cid:0) I s ( I ) (cid:1) .Then we have (cid:12)(cid:12) E [ X s X s ] − E [ X s X s ] (cid:12)(cid:12) (cid:54) d / √ n . The estimate (6.3) follows using this observation, the triangle inequality and the spread-ability of the random array X . (cid:3) Proof of Theorem 1.6
In this section we give the proof of Theorem 1.6. As already noted, the proof is basedon the results obtained in Sections 5 and 6; in particular, the reader is advised to reviewthis material, as well as the terminology and notation introduced in Paragraphs 1.4.1 and1.4.2, before reading this section.
Existence of decomposition.
The main step is the following proposition whichestablishes the existence of the desired decomposition.
Proposition 7.1.
Let n, d, κ, k be positive integers with κ (cid:62) and n (cid:62) κ d ( k + 1) d +1 ,and set (7.1) γ = γ ( n, d, κ ) := (cid:16) κ + 8 d √ n (cid:17) / . Then there exists a subset N of [ n ] with | N | = k and satisfying the following property.If X = (cid:104) X s : s ∈ (cid:0) [ n ] d (cid:1) (cid:105) is a real-valued, spreadable, d -dimensional random array on [ n ] such that (cid:107) X s (cid:107) L = 1 for all s ∈ (cid:0) [ n ] d (cid:1) , then there exists a real-valued stochastic process ∆ = (cid:104) ∆ p : p ∈ PartIncr([ d ] , N ) (cid:105) such that the following hold true. (i) For every s ∈ (cid:0) Nd (cid:1) we have X s = (cid:80) F ⊆ [ d ] ∆ I s (cid:22) F . (ii) For every p ∈ PartIncr([ d ] , N ) with p (cid:54) = ∅ we have | E [∆ p ] | (cid:54) d γ . (iii) If p , p ∈ PartIncr([ d ] , N ) are distinct and the pair { p , p } is aligned, then wehave (cid:12)(cid:12) E [∆ p ∆ p ] (cid:12)(cid:12) (cid:54) d +2 γ . The bulk of this section is devoted to the proof of Proposition 7.1—it spans Paragraphs7.1.1 up to 7.1.4. The proof of Theorem 1.6 is completed in Subsection 7.2.7.1.1.
Definitions/Notation.
This is the heart of the proof of Proposition 7.1. Our goalis to define the set N and the process ∆ . This task is combinatorially delicate, andit requires a number of preparatory steps. In what follows, let d, n, κ, k, X be as inProposition 7.1.7.1.1.1 . We start by selecting two sequences ( L , . . . , L k ) and ( D , . . . , D k , D k +1 ) of subin-tervals of [ n −
1] with the following properties. • For every i ∈ [ k ] we have | L i | = κ . • For every i ∈ [ k + 1] we have | D i | = dκ ( k + 1) d . • For every i ∈ [ k ] we have max( D i ) < min( L i ) (cid:54) max( L i ) < min( D i +1 ).We set(7.2) N := (cid:8) min( L i ) : i ∈ [ k ] (cid:9) and (cid:101) N := N ∪ { n } . Moreover, for every i ∈ [ k + 1] we select a collection (cid:104) Γ i,p : p ∈ PartIncr([ d ] , N ) (cid:105) ofpairwise disjoint subintervals of D i of length dκ . Next, for every i ∈ [ k + 1] and every p ∈ PartIncr([ d ] , N ) let (Θ i,p, , . . . , Θ i,p,d ) denote the unique finite sequence of successivesubintervals of Γ i,p of length κ . Finally, for every i ∈ [ k + 1], every p ∈ PartIncr([ d ] , N )and every j ∈ [ d ] by ( H i,p,j, , . . . , H i,p,j,κ ) we denote the unique finite sequence of succes-sive subintervals of Θ i,p,j of length κ . ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 33 L i − dκ ( k + 1) d L i D i dκ Γ i,p Γ i,p Γ i,p κ Θ i,p,j Θ i,p, Θ i,p,d κH i,p,j, κH i,p,j,κ Figure 3.
The sets D i , Γ i,p , Θ i,p,j and H i,p,j,r .7.1.1.2 . Next, for every p ∈ PartIncr([ d ] , N ) we define a subset O p of (cid:0) [ n ] d (cid:1) as follows. Ifdom( p ) = [ d ], then we set(7.3) O p := (cid:8) Im( p ) (cid:9) . Otherwise, if dom( p ) (cid:32) [ d ], then let ( K , . . . , K b ) denote the unique finite sequence ofsubintervals of [ d ] \ dom( p ) of maximal length which cover the set [ d ] \ dom( p )—that is,[ d ] \ dom( p ) = K ∪· · ·∪ K b —and such that max( K a ) < min( K a +1 ) if b (cid:62) a ∈ [ b − b (cid:62)
2, then for every a ∈ [ b −
1] we have that max( K a ) + 1 ∈ dom( p ).)Also let (cid:101) p : dom( p ) ∪ { d + 1 } → (cid:101) N denote the extension of p which satisfies (cid:101) p ( d + 1) = n .For every r ∈ [ κ ] we set s p,r := Im( p ) ∪ (cid:8) min( H i,p,j,r ) : a ∈ [ b ] , i = I − (cid:101) N (cid:0)(cid:101) p (max( K a ) + 1) (cid:1) and j ∈ { , . . . , | K a |} (cid:9) and we define(7.4) O p := (cid:8) s p,r : r ∈ [ κ ] (cid:9) . We also set(7.5) G p := (cid:91) G ⊆ dom( p ) O p (cid:22) G and(7.6) O := (cid:91) p ∈ PartIncr([ d ] ,N ) O p . Note that if p, p (cid:48) ∈ PartIncr([ d ] , N ) are distinct, then O p ∩ O p (cid:48) = ∅ .Finally, for every s ∈ O and every G ⊆ dom( p )—where p ∈ PartIncr([ d ] , N ) denotesthe unique partial map such that s ∈ O p —we define a subset O (cid:48) s,G of (cid:0) [ n ] d (cid:1) as follows. Forevery r ∈ [ κ ] we set t s,G,r := p ( G ) ∪ (cid:8) v + r − v ∈ s \ p ( G ) (cid:9) ∈ (cid:0) [ n ] d (cid:1) and we define(7.7) O (cid:48) s,G := (cid:8) t s,G,r : r ∈ [ κ ] (cid:9) . . We are now in a position to introduce the stochastic process ∆ . First, for every p ∈ PartIncr([ d ] , N ) we set(7.8) Y p := 1 |O p | (cid:88) s ∈O p X s (notice that, by (7.3), we have Y I s = X s for every s ∈ (cid:0) [ n ] d (cid:1) ), and we define(7.9) ∆ p := (cid:88) G ⊆ dom( p ) ( − | dom( p ) \ G | Y p (cid:22) G . We also set(7.10) A p := σ (cid:0) { X s : s ∈ G p (cid:9)(cid:1) ;that is, A p is the σ -algebra generated by the random variables (cid:104) X s : s ∈ G p (cid:105) .7.1.2. Basic properties.
We isolate, for future use, the following basic properties of theconstruction presented in Paragraph 7.1.1.( P
1) For every p ∈ PartIncr([ d ] , N ) and every subset G of dom( p ) the random variable Y p (cid:22) G is A p -measurable.( P
2) Let p , p ∈ PartIncr([ d ] , N ) be distinct and such that the pair { p , p } is aligned,and assume that r ( p , p ) (cid:54) = dom( p ). Then for every s ∈ O p the family ofrandom variables (cid:104) X t : t ∈ O p ∧ p ∪ O (cid:48) s,r ( p ,p ) (cid:105) is an (8 d / √ n )-orbit in the senseof Definition 5.1.( P
3) Let p , p ∈ PartIncr([ d ] , N ) be distinct and such that the pair { p , p } is aligned,and assume that r ( p , p ) (cid:54) = dom( p ). Also let s ∈ O p be arbitrary. Then forevery s (cid:48) ∈ O (cid:48) s,r ( p ,p ) we have that E [ X s (cid:48) | A p ] = E [ X s | A p ].Property ( P
1) follows immediately by (7.8) and the fact that O p (cid:22) G ⊆ G p . In order to seethat property ( P
2) is satisfied notice that, since p (cid:54) = p ∧ p , if t , t ∈ O p ∧ p ∪O (cid:48) s,r ( p ,p ) are distinct, then I t (cid:22) r ( p , p ) = I t (cid:22) r ( p , p ) = p ∧ p and the pair { t , t } isaligned with r ( t , t ) = r ( p , p ). Using this observation, property ( P
2) follows fromProposition 6.2. Finally, for property ( P
3) we first observe that for every F ⊆ dom( p )we have that p (cid:54) = ( p (cid:22) F ) and dom( p ∧ p (cid:22) F ) ⊆ r ( p , p ). Therefore, for every i ∈ [ d ] \ r ( p , p ) and every F ⊆ dom( p ) we have that I s ( i ) (cid:54)∈ ∪O p (cid:22) F and, consequently, ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 35 I s ( i ) (cid:54)∈ ∪G p . On the other hand, by the definition of the set O (cid:48) s,r ( p ,p ) in (7.7), thereexists a collection (cid:104) J i : i ∈ [ d ] \ r ( p , p ) (cid:105) of disjoint intervals of length κ such that forevery i ∈ [ d ] \ r ( p , p ) we have that I s ( i ) = min( J i ), I s (cid:48) ( i ) ∈ J i and J i ∩ ( ∪G p ) = ∅ .Taking into account these remarks, property ( P
3) follows from the spreadability of X .7.1.3. Compatibility of projections.
The following lemma shows that the projections as-sociated with the σ -algebras defined in (7.10) behave like a lattice of projections whenapplied to the random variables defined in (7.8). Lemma 7.2.
Let d, n, κ, k, X be as in Proposition , and let γ be as in (7.1) . Alsolet N ⊆ [ n ] , Y = (cid:104) Y p : p ∈ PartIncr([ d ] , N ) (cid:105) and (cid:104)A p : p ∈ PartIncr([ d ] , N ) (cid:105) be asin Paragraph . Then for every distinct p , p ∈ PartIncr([ d ] , N ) such that the pair { p , p } is aligned we have (7.11) (cid:13)(cid:13) E [ Y p | A p ] − Y p ∧ p (cid:13)(cid:13) L (cid:54) γ. Proof.
If dom( p ) = r ( p , p ), then p = p ∧ p and, by property ( P Y p is A p -measurable; hence, in this case, the result is straightforward. Therefore,we may assume that dom( p ) \ r ( p , p ) (cid:54) = ∅ . By (7.8), it is enough to show that for every s ∈ O p we have(7.12) (cid:13)(cid:13) E [ X s | A p ] − Y p ∧ p (cid:13)(cid:13) L (cid:54) γ. To this end, let s ∈ O p be arbitrary. Since p ∧ p = p (cid:22) r ( p , p ), using property ( P Y p ∧ p is A p -measurable and, consequently,(7.13) Y p ∧ p = E [ Y p ∧ p | A p ] . By property ( P (cid:104) X t : t ∈ O p ∧ p ∪ O (cid:48) s,r ( p ,p ) (cid:105) is an (8 d / √ n )-orbit in thesense of Definition 5.1. Moreover, |O p ∧ p | = |O (cid:48) s,r ( p ,p ) | = κ . By Proposition 5.2, thedefinition of Y p ∧ p in (7.8) and the choice of γ , we have(7.14) (cid:13)(cid:13)(cid:13) Y p ∧ p − κ (cid:88) t ∈O (cid:48) s,r ( p ,p X t (cid:13)(cid:13)(cid:13) L (cid:54) γ and so, by the contractive property of conditional expectation and (7.13),(7.15) (cid:13)(cid:13)(cid:13) Y p ∧ p − κ (cid:88) t ∈O (cid:48) s,r ( p ,p E [ X t | A p ] (cid:13)(cid:13)(cid:13) L (cid:54) γ. The estimate (7.12) follows from (7.15) and property ( P (cid:3) The following corollary of Lemma 7.2 is the last ingredient needed for the proof ofProposition 7.1.
Corollary 7.3.
Let d, n, κ, k, X be as in Proposition , and let γ be as in (7.1) . Alsolet N ⊆ [ n ] and Y = (cid:104) Y p : p ∈ PartIncr([ d ] , N ) (cid:105) be as in Paragraph . Then for everydistinct p , p ∈ PartIncr([ d ] , N ) such that the pair { p , p } is aligned we have (7.16) (cid:12)(cid:12) E [ Y p Y p ] − E [ Y p ∧ p ] (cid:12)(cid:12) (cid:54) γ. Proof.
We first observe that(7.17) (cid:12)(cid:12) E [ Y p Y p ] − E [ Y p ∧ p ] (cid:12)(cid:12) (cid:54) (cid:12)(cid:12) E [ Y p ( Y p − Y p ∧ p )] (cid:12)(cid:12) + (cid:12)(cid:12) E [ Y p ∧ p ( Y p − Y p ∧ p )] (cid:12)(cid:12) . By (7.8), we see that (cid:107) Y p (cid:107) L (cid:54)
1. Since Y p and Y p ∧ p are A p -measurable—whichfollows from property ( P (cid:12)(cid:12) E [ Y p ( Y p − Y p ∧ p )] (cid:12)(cid:12) = (cid:12)(cid:12) E (cid:2) E [ Y p ( Y p − Y p ∧ p ) | A p ] (cid:3)(cid:12)(cid:12) (7.18) = (cid:12)(cid:12) E [ Y p ( E [ Y p | A p ] − Y p ∧ p )] (cid:12)(cid:12) (cid:54) (cid:13)(cid:13) E [ Y p | A p ] − Y p ∧ p (cid:13)(cid:13) L (7.11) (cid:54) γ. Similarly, we obtain that(7.19) (cid:12)(cid:12) E [ Y p ∧ p ( Y p − Y p ∧ p )] (cid:12)(cid:12) (cid:54) (cid:13)(cid:13) E [ Y p | A p ∧ p ] − Y p ∧ p (cid:13)(cid:13) L (7.11) (cid:54) γ. Inequality (7.16) follows by combining (7.17), (7.18) and (7.19). (cid:3)
Proof of Proposition . Let N be as in (7.2). Moreover, given the random ar-ray X , let Y = (cid:104) Y p : p ∈ PartIncr([ d ] , N ) (cid:105) and ∆ = (cid:104) ∆ p : p ∈ PartIncr([ d ] , N ) (cid:105) be thereal-valued stochastic processes defined in (7.8) and (7.9) respectively.We claim that N and ∆ are as desired. To this end we first observe that | N | = k . Forpart (i), let s ∈ (cid:0) Nd (cid:1) be arbitrary. Notice that for every G ⊆ [ d ] the quantity(7.20) (cid:88) G ⊆ F ⊆ [ d ] ( − | F \ G | is equal to 1 if G = [ d ], and 0 otherwise. Therefore, (cid:88) F ⊆ [ d ] ∆ I s (cid:22) F (7.9) = (cid:88) F ⊆ [ d ] (cid:16) (cid:88) G ⊆ F ( − | F \ G | Y I s (cid:22) G (cid:17) (7.21) = (cid:88) G ⊆ [ d ] Y I s (cid:22) G (cid:16) (cid:88) G ⊆ F ⊆ [ d ] ( − | F \ G | (cid:17) (7.20) = Y I s (7.8) = X s . For part (ii), fix p ∈ PartIncr([ d ] , N ) with p (cid:54) = ∅ and observe that(7.22) (cid:88) G ⊆ dom( p ) ( − | dom( p ) \ G | = 0 . ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 37
Since (cid:107) Y ∅ (cid:107) L (cid:54)
1, by the Cauchy–Schwarz inequality and Lemma 7.2, we obtain that (cid:12)(cid:12) E [∆ p ] (cid:12)(cid:12) (7.9) = (cid:12)(cid:12)(cid:12) (cid:88) G ⊆ dom( p ) ( − | dom( p ) \ G | E [ Y p (cid:22) G ] (cid:12)(cid:12)(cid:12) (7.23) = (cid:12)(cid:12)(cid:12) (cid:88) G ⊆ dom( p ) ( − | dom( p ) \ G | E (cid:2) E [ Y p (cid:22) G | A ∅ ] (cid:3)(cid:12)(cid:12)(cid:12) (cid:54) (cid:12)(cid:12)(cid:12) (cid:88) G ⊆ dom( p ) ( − | dom( p ) \ G | E [ Y ∅ ] (cid:12)(cid:12)(cid:12) + (cid:88) ∅(cid:54) = G ⊆ dom( p ) (cid:12)(cid:12) E (cid:2) E [ Y p (cid:22) G | A ∅ ] − Y ∅ (cid:3)(cid:12)(cid:12) (7.22) (cid:54) (cid:88) ∅(cid:54) = G ⊆ dom( p ) (cid:13)(cid:13) E [ Y p (cid:22) G | A ∅ ] − Y ∅ (cid:13)(cid:13) L (7.11) (cid:54) d γ. Finally, for part (iii), let p , p ∈ PartIncr([ d ] , N ) be distinct such that the pair { p , p } isaligned. Without loss of generality we may assume that dom( p ) \ r ( p , p ) (cid:54) = ∅ . (If not,then we will work with p .) By Corollary 7.3, we have (cid:12)(cid:12) E [∆ p ∆ p ] (cid:12)(cid:12) (7.9) = (cid:12)(cid:12)(cid:12) (cid:88) G ⊆ dom( p ) H ⊆ dom( p ) ( − | dom( p ) \ G | ( − | dom( p ) \ H | E [ Y p (cid:22) G Y p (cid:22) H ] (cid:12)(cid:12)(cid:12) (7.16) (cid:54) (cid:12)(cid:12)(cid:12) (cid:88) G ⊆ dom( p ) H ⊆ dom( p ) ( − | dom( p ) | + | dom( p ) | + | G | + | H | E [ Y p (cid:22) G ∩ H ∩ r ( p ,p ) ] (cid:12)(cid:12)(cid:12) + 2 d +2 γ ;on the other hand, our assumption that dom( p ) \ r ( p , p ) (cid:54) = ∅ yields that (cid:88) (cid:101) G ⊆ dom( p ) \ r ( p ,p ) ( − | (cid:101) G | = 0 , and so, (cid:12)(cid:12)(cid:12) (cid:88) G ⊆ dom( p ) H ⊆ dom( p ) ( − | dom( p ) | + | dom( p ) | + | G | + | H | E [ Y p (cid:22) G ∩ H ∩ r ( p ,p ) ] (cid:12)(cid:12)(cid:12) == (cid:12)(cid:12)(cid:12) (cid:88) K ⊆ r ( p ,p ) G,H ⊆ r ( p ,p ) \ KG ∩ H = ∅ E [ Y p (cid:22) K ] ( − | G | + | H | (cid:88) (cid:101) H ⊆ dom( p ) \ r ( p ,p ) ( − | (cid:101) H | (cid:88) (cid:101) G ⊆ dom( p ) \ r ( p ,p ) ( − | (cid:101) G | (cid:12)(cid:12)(cid:12) = 0 . Therefore, (cid:12)(cid:12) E [∆ p ∆ p ] (cid:12)(cid:12) (cid:54) d +2 γ . The proof of Proposition 7.1 is completed.7.2. Proof of Theorem 1.6.
Let d be a positive integer, let ε >
0, and let c and n beas in (1.7) and (1.8) respectively. Fix an integer n (cid:62) n . We set(7.24) κ := (cid:108) d +5 ε (cid:109) and k := (cid:106) (cid:16) ε d (cid:17) d +1 d +1 √ n (cid:107) and we observe that κ (cid:62) n (cid:62) κ d ( k + 1) d +1 . Let N be the subset of [ n ] obtainedby Proposition 7.1 applied for n, d and the positive integers κ, k defined above. By thechoices of c, n , k and the fact that | N | = k , it is easy to see that | N | (cid:62) c d +1 √ n . Next,let X be a d -dimensional, spreadable, random array on [ n ] as in Theorem 1.6, and let ∆ be the real-valued stochastic process obtained by Proposition 7.1 when applied to X . Bythe definition of the constant γ in (7.1) and using again the choices of κ and k , it is nothard to check that parts (i), (ii) and (iii) of Proposition 7.1 yield the corresponding partsof Theorem 1.6. Thus, we only need to verify part (iv), that is, the fact that the process ∆ is (essentially) unique.Indeed, set(7.25) (cid:96) := (cid:100) ε − +2 d (cid:101) , k := (cid:106) k − (cid:96) ( d − (cid:96) ( d −
1) + 1 (cid:107) and L := (cid:8) I N (cid:0) ( (cid:96) ( d − j (cid:1) : j ∈ [ k ] (cid:9) and observe that L is a subset of N with | L | (cid:62) (cid:0) ε − + 2 d ) d (cid:1) − k = (cid:0) ε − + 2 d ) d (cid:1) − | N | .We will show that the set L is as desired. To this end, we first observe the followingproperty which follows from the definition of the set L .( A ) For every p ∈ PartIncr([ d ] , L ) there exists a sequence ( s pj ) (cid:96)j =1 in (cid:0) Nd (cid:1) such that forevery distinct i, j ∈ [ (cid:96) ] the pair { s pi , s pj } is aligned in the sense of Definition 6.1,and satisfies I s pi ∧ I s pj = p .Now, let Z = (cid:104) Z p : p ∈ PartIncr([ d ] , N ) (cid:105) be a real-valued stochastic process whichsatisfies parts (i) and (iii) of Theorem 1.6. By part (i) applied for ∆ and Z , for every s ∈ (cid:0) Nd (cid:1) we have 1 = (cid:107) X s (cid:107) L = (cid:88) F ⊆ [ d ] (cid:107) ∆ I s (cid:22) F (cid:107) L + (cid:88) F,G ⊆ [ d ] F (cid:54) = G E [∆ I s (cid:22) F ∆ I s (cid:22) G ]and 1 = (cid:107) X s (cid:107) L = (cid:88) F ⊆ [ d ] (cid:107) Z I s (cid:22) F (cid:107) L + (cid:88) F,G ⊆ [ d ] F (cid:54) = G E [ Z I s (cid:22) F Z I s (cid:22) G ]and therefore, by part (iii), for every F ⊆ [ d ] we have(7.26) (cid:107) ∆ I s (cid:22) F (cid:107) L (cid:54) d ε and (cid:107) Z I s (cid:22) F (cid:107) L (cid:54) d ε. Claim 7.4.
Let p ∈ PartIncr([ d ] , L ) , and let ( s pj ) (cid:96)j =1 be the corresponding sequence in (cid:0) Nd (cid:1) described in property ( A ) . Then for every F ⊆ [ d ] the following hold. (i) If F ⊆ dom( p ) , then we have (7.27) 1 (cid:96) (cid:96) (cid:88) j =1 ∆ I spj (cid:22) F = ∆ p (cid:22) F and (cid:96) (cid:96) (cid:88) j =1 Z I spj (cid:22) F = Z p (cid:22) F . (ii) If F \ dom( p ) (cid:54) = ∅ , then we have (7.28) (cid:13)(cid:13)(cid:13) (cid:96) (cid:96) (cid:88) j =1 ∆ I spj (cid:22) F (cid:13)(cid:13)(cid:13) L (cid:54) √ ε and (cid:13)(cid:13)(cid:13) (cid:96) (cid:96) (cid:88) j =1 Z I spj (cid:22) F (cid:13)(cid:13)(cid:13) L (cid:54) √ ε. Proof of Claim . By property ( A ), for every j ∈ [ n ] we have that I s pj (cid:22) dom( p ) = p ;(7.27) follows from this observation. On the other hand, invoking property ( A ) again,we see that if F \ dom( p ) (cid:54) = ∅ , then for every distinct j , j ∈ [ (cid:96) ] the partial maps ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 39 I s pj and I s pj are distinct and the pair { I s pj , I s pj } is aligned. Taking into account thisremark, (7.28) follows from (7.26), the fact that the processes ∆ and Z satisfy part (iii)of Theorem 1.6, and the choice of (cid:96) in (7.25). (cid:3) After this preliminary discussion, for every p ∈ PartIncr([ d ] , L ) we will show that(7.29) (cid:107) ∆ p − Z p (cid:107) L (cid:54) | dom( p ) | +12 ) + d +1 √ ε with the convention that (cid:0) (cid:1) = 0; clearly, this is enough to complete the proof. Wewill proceed by induction on the cardinality of dom( p ). If “dom( p ) = 0”, then this isequivalently to saying that p = ∅ ; in this case, by (7.27) and using the fact that part (i)of Theorem 1.6 is satisfied for ∆ and Z , we see that1 (cid:96) (cid:96) (cid:88) j =1 X s ∅ j = ∆ ∅ + (cid:88) ∅(cid:54) = F ⊆ [ d ] (cid:96) (cid:96) (cid:88) j =1 ∆ I s ∅ j (cid:22) F and 1 (cid:96) (cid:96) (cid:88) j =1 X s ∅ j = Z ∅ + (cid:88) ∅(cid:54) = F ⊆ [ d ] (cid:96) (cid:96) (cid:88) j =1 Z I s ∅ j (cid:22) F and so, by (7.28), we obtain that (cid:107) ∆ ∅ − Z ∅ (cid:107) L (cid:54) d +1 √ ε. Next, let u ∈ [ (cid:96) ] and assume that (7.29) has been proved for every partial map whosedomain has size strictly less than u . Fix p ∈ PartIncr([ d ] , L ) with | dom( p ) | = u . Usingagain (7.27) and the validity of part (i) of Theorem 1.6 for ∆ and Z , we see that1 (cid:96) (cid:96) (cid:88) j =1 X s pj = (cid:88) F ⊆ dom( p ) ∆ p (cid:22) F + (cid:88) F ⊆ [ d ] F \ dom( p ) (cid:54) = ∅ (cid:96) (cid:96) (cid:88) j =1 ∆ I spj (cid:22) F = (cid:88) F ⊆ dom( p ) Z p (cid:22) F + (cid:88) F ⊆ [ d ] F \ dom( p ) (cid:54) = ∅ (cid:96) (cid:96) (cid:88) j =1 Z I spj (cid:22) F . Invoking this identity, (7.28) and the inductive assumptions, we conclude that (cid:107) ∆ p − Z p (cid:107) L = (cid:13)(cid:13)(cid:13) (cid:88) F (cid:32) dom( p ) ( Z p (cid:22) F − ∆ p (cid:22) F ) + (cid:88) F ⊆ [ d ] F \ dom( p ) (cid:54) = ∅ (cid:96) (cid:96) (cid:88) j =1 ( Z I spj (cid:22) F − ∆ I spj (cid:22) F ) (cid:13)(cid:13)(cid:13) L (cid:54) (2 u − u ) + d +1 √ ε + 2(2 d − u ) √ ε (cid:54) u +12 ) + d +1 √ ε. This completes the proof of the general inductive step, and consequently, the entire proofof Theorem 1.6 is completed.8.
Connection with concentration
Overview.
We are about to present an application of Theorem 1.4 which supple-ments the concentration results obtained in [DTV20]. To put things in a proper context,we first recall the main problem addressed in [DTV20].
Problem 8.1.
Let n (cid:62) d (cid:62) be integers, and let X = (cid:104) X s : s ∈ (cid:0) [ n ] d (cid:1) (cid:105) be an approximatelyspreadable, d -dimensional random array on [ n ] whose entries take values in a finite set X .Also let f : X ( [ n ] d ) → R be a function, and assume that E [ f ( X )] = 0 and (cid:107) f ( X ) (cid:107) L p = 1 for some p > . Under what condition on X can we find a large subset I of [ n ] such that,setting F I := σ (cid:0) { X s : s ∈ (cid:0) Id (cid:1) } (cid:1) , the random variable E [ f ( X ) | F I ] is concentrated aroundits mean? Note that Problem 8.1 is somewhat distinct from the traditional setting of concen-tration of smooth functions (see, e.g. , [Le01, BLM13]). It is particularly relevant in acombinatorial context since functions on discrete sets are, usually, highly nonsmooth. Werefer the reader to the introduction of [DTV20] for further motivation, and to [DK16] fora broader discussion on this “conditional concentration” and its applications.8.1.1.
The box independence condition.
In [DTV20] it was shown that an affirmativeanswer to Problem 8.1 can be obtained if—and essentially only if—the random array X satisfies a certain correlation condition to which we refer as the box independencecondition . In order to state this condition we need to introduce some terminology. Let n, d be integers with n (cid:62) d and d (cid:62)
2; we say that a subset of (cid:0) [ n ] d (cid:1) is a d -dimensionalbox of [ n ] if it is of the form(8.1) (cid:110) s ∈ (cid:18) [ n ] d (cid:19) : | s ∩ H i | = 1 for all i ∈ [ d ] (cid:111) . where H , . . . , H d are 2-element subsets of [ n ] which satisfy max( H i ) < min( H i +1 ) forevery i ∈ [ d − Definition 8.2 (Box independence condition) . Let n, d be integers with n (cid:62) d and d (cid:62) , let X be a finite set with |X | (cid:62) , and let X = (cid:104) X s : s ∈ (cid:0) [ n ] d (cid:1) (cid:105) be a d -dimensionalrandom array on [ n ] with X -valued entries. Also let ϑ (cid:62) . We say that X satisfies the ϑ -box independence condition if there exists a subset S of X with |S| = |X | − such thatfor every d -dimensional box B of [ n ] and every a ∈ S we have (8.2) (cid:12)(cid:12)(cid:12) P (cid:16) (cid:92) s ∈ B [ X s = a ] (cid:17) − (cid:89) s ∈ B P (cid:0) [ X s = a ] (cid:1)(cid:12)(cid:12)(cid:12) (cid:54) ϑ. Thus, for instance, if “ d = 2” and “ X = { , } ”, then the ϑ -box independence conditionis equivalent to saying that for every i, j, k, (cid:96) ∈ [ n ] with i < j < k < (cid:96) we have(8.3) (cid:12)(cid:12) E [ X { i,k } X { i,(cid:96) } X { j,k } X { j,(cid:96) } ] − E [ X { i,k } ] E [ X { i,(cid:96) } ] E [ X { j,k } ] E [ X { j,(cid:96) } ] (cid:12)(cid:12) (cid:54) ϑ. See, in particular, [DTV20, Theorems 1.5 and 2.3].
ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 41 that a random array X satisfies condition (8.2) if and only if its distribution is close toa distribution of the form (1.3) where for “almost every” j ∈ J and every a ∈ X therandom variable h aj is box uniform and its average E [ h aj ] is roughly equal to the expectedvalue P (cid:0) [ X [ d ] = a ] (cid:1) .8.1.3. Box uniformity.
The aforementioned box uniformity is a well-known pseudoran-domness property—see, e.g. , [R˝o15]—which is defined using the box norms. Specifically,let d (cid:62) , Σ , µ ) be a probability space, and let Ω d be equippedwith the product measure. Also let (cid:37) >
0. We say that an integrable random variable h : Ω d → R is (cid:37) -box uniform provided that(8.4) (cid:13)(cid:13) h − E [ h ] (cid:13)(cid:13) (cid:3) (cid:54) (cid:37) where (cid:107) · (cid:107) (cid:3) denotes the corresponding box norm. (See Subsection 3.1.)8.2. The characterization.
We have the following proposition.
Proposition 8.3.
Let d, m (cid:62) be integers, and let < ε (cid:54) . Let C = C ( d, m, d, ε ) beas in (1.4) , let n, X , X be as in Theorem , and set δ a := P (cid:0) [ X [ d ] = a ] (cid:1) for every α ∈ X .Finally, let J, Ω , λ = (cid:104) λ j : j ∈ J (cid:105) and H = (cid:104) h aj : j ∈ J, a ∈ X (cid:105) be as in Theorem when applied to the random array X for the parameters d, m, ε and k = 2 d . Then thefollowing hold. (i) Let (cid:37) > , and set (8.5) ϑ = ϑ ( d, ε, (cid:37) ) := 2 d (2 ε + 4 (cid:37) ) . Assume that there is a subset G of J such that: (a) (cid:80) j ∈ G λ j (cid:62) − (cid:37) , and (b) forevery j ∈ G and every a ∈ X we have | E [ h aj ] − δ a | (cid:54) (cid:37) and (cid:13)(cid:13) h aj − E [ h ja ] (cid:13)(cid:13) (cid:3) (cid:54) (cid:37) .Then, X is ϑ -box independent. (ii) Conversely, let ϑ > , and set (8.6) (cid:37) = (cid:37) ( d, m, ε, ϑ ) := 2 d +7 m ( ε / d + ϑ / d ) . Assume that X is ϑ -box independent. Then there exists a subset G of J suchthat: (a) (cid:80) j ∈ G λ j (cid:62) − (cid:37) , and (b) for every j ∈ G and every a ∈ X we have | E [ h aj ] − δ a | (cid:54) (cid:37) and (cid:13)(cid:13) h aj − E [ h ja ] (cid:13)(cid:13) (cid:3) (cid:54) (cid:37) .Proof. First we argue for part (i). Let B be a d -dimensional box of [ n ], and fix a ∈ X .By (1.6) and part (a) of our assumptions, we have(8.7) (cid:12)(cid:12)(cid:12) P (cid:16) (cid:92) s ∈ B [ X s = a ] (cid:17) − (cid:88) j ∈ G λ j (cid:90) (cid:89) s ∈ B h aj ( ω s ) d µ j ( ω ) (cid:12)(cid:12)(cid:12) (cid:54) ε + (cid:37). Let j ∈ G be arbitrary, and observe that (cid:107) h aj (cid:107) (cid:3) (cid:54)
1. By the (cid:37) -box uniformity of h aj , theGowers–Cauchy–Schwarz inequality (3.6) and a telescopic argument, we see that(8.8) (cid:12)(cid:12)(cid:12) (cid:90) (cid:89) s ∈ B h aj ( ω s ) d µ j ( ω ) − (cid:89) s ∈ B E [ h aj ] (cid:12)(cid:12)(cid:12) (cid:54) d (cid:37) and so, using the fact | E [ h aj ] − δ a | (cid:54) (cid:37) , we obtain that(8.9) (cid:12)(cid:12)(cid:12) (cid:90) (cid:89) s ∈ B h aj ( ω s ) d µ j ( ω ) − δ d a (cid:12)(cid:12)(cid:12) (cid:54) d +1 (cid:37). On the other hand, since X is (1 /C )-spreadable, we have(8.10) (cid:12)(cid:12)(cid:12) δ d a − (cid:89) s ∈ B P (cid:0) [ X s = a ] (cid:1)(cid:12)(cid:12)(cid:12) (cid:54) d C .
By (8.7)–(8.10), assumption (a), the fact that 1 /C (cid:54) ε and the choice of ϑ in (8.5), weconclude that (cid:12)(cid:12)(cid:12) P (cid:16) (cid:92) s ∈ B [ X s = a ] (cid:17) − (cid:89) s ∈ B P (cid:0) [ X s = a ] (cid:1)(cid:12)(cid:12)(cid:12) (cid:54) ϑ which yields that the random array X is ϑ -box independent.We proceed to the proof of part (ii). We will need the following fact which followsfrom [DTV20, Lemma 4.6 and Subsection 5.2] and the fact that n (cid:62) C (cid:62) ε − . It showsthat the box independence condition is inherited to subsets of d -dimensional boxes. Fact 8.4.
Let the notation and assumptions be as in part (ii) of Proposition , and set (8.11) Θ := 100 2 d m d (2 ε / d + ϑ / d ) . Then for every d -dimensional box B of [ n ] , every nonempty subset F of B and every a ∈ X we have (8.12) (cid:12)(cid:12)(cid:12) P (cid:16) (cid:92) s ∈ F [ X s = a ] (cid:17) − (cid:89) s ∈ F P (cid:0) [ X s = a ] (cid:1)(cid:12)(cid:12)(cid:12) (cid:54) Θ . Now, fix a ∈ X , and set s := { i − i ∈ [ d ] } ∈ (cid:0) [ n ] d (cid:1) and s := { i : i ∈ [ d ] } ∈ (cid:0) [ n ] d (cid:1) .By (1.6), we have (cid:12)(cid:12)(cid:12) δ a − (cid:88) j ∈ J λ j E [ h aj ] (cid:12)(cid:12)(cid:12) (cid:54) ε, (8.13) (cid:12)(cid:12)(cid:12) P (cid:0) [ X s = a ] ∩ [ X s = a ] (cid:1) − (cid:88) j ∈ J λ j E [ h aj ] (cid:12)(cid:12)(cid:12) (cid:54) ε. (8.14)By Fact 8.4, the (1 /C )-spreadability of X and (8.14), we see that(8.15) (cid:12)(cid:12)(cid:12) δ a − (cid:88) j ∈ J λ j E [ h aj ] (cid:12)(cid:12)(cid:12) (cid:54) ε + Θ + 2 C .
Thus, setting(8.16) (cid:37) := 2 m (4 ε + Θ) / , by (8.13), (8.15), the fact that 1 /C (cid:54) ε , Chebyshev’s inequality and a union bound, weobtain a subset G of J such that (cid:80) j ∈ G λ j (cid:62) − (cid:37) and | E [ h aj ] − δ a | (cid:54) (cid:37) for every j ∈ G and every a ∈ X . ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 43
Again, let a ∈ X be arbitrary. We shall estimate the quantity (cid:88) j ∈ G λ j (cid:13)(cid:13) h aj − E [ h aj ] (cid:13)(cid:13) d (cid:3) (3.5) = (cid:88) H ⊆{ , } d ( − d −| H | × (8.17) × (cid:16) (cid:88) j ∈ G λ j E [ h aj ] d −| H | (cid:90) (cid:89) (cid:15) ∈ H h aj ( ω (cid:15) ) d µ ( ω ) (cid:17) . (Here, as in Section 2, we use the convention that the product of an empty family offunctions is equal to the constant function 1.) To this end, let H be an arbitrary subsetof { , } d . Notice first that, by the choice of G , we have (cid:12)(cid:12)(cid:12) (cid:88) j ∈ G λ j E [ h aj ] d −| H | (cid:90) (cid:89) (cid:15) ∈ H h aj ( ω (cid:15) ) d µ ( ω ) − (8.18) − δ d −| H | a (cid:88) j ∈ J λ j (cid:90) (cid:89) (cid:15) ∈ H h aj ( ω (cid:15) ) d µ ( ω ) (cid:12)(cid:12)(cid:12) (cid:54) (2 d + 1) (cid:37) . Next observe that if H is nonempty, then, by (1.6), we may select a d -dimensional box B of [ n ] and a nonempty subset F of B with | F | = | H | and such that (cid:12)(cid:12)(cid:12) P (cid:16) (cid:92) s ∈ F [ X s = a ] (cid:17) − (cid:88) j ∈ J λ j (cid:90) (cid:89) (cid:15) ∈ H h aj ( ω (cid:15) ) d µ ( ω ) (cid:12)(cid:12)(cid:12) (cid:54) ε. (8.19)Thus, by Fact 8.4, the (1 /C )-spreadability of X and the fact that | F | = | H | , we have (cid:12)(cid:12)(cid:12) δ | H | a − (cid:88) j ∈ J λ j (cid:90) (cid:89) (cid:15) ∈ H h aj ( ω (cid:15) ) d µ ( ω ) (cid:12)(cid:12)(cid:12) (cid:54) ε + Θ + 2 d C . (8.20)By (8.18) and (8.20), we see that for every (possibly empty) subset H of { , } d , (cid:12)(cid:12)(cid:12) (cid:88) j ∈ G λ j E [ h aj ] d −| H | (cid:90) (cid:89) (cid:15) ∈ H h aj ( ω (cid:15) ) d µ ( ω ) − δ d a (cid:12)(cid:12)(cid:12) (cid:54) ε + Θ + 2 d C + (2 d + 1) (cid:37) . (8.21)By (8.17) and (8.21), we conclude that for every a ∈ X we have (cid:88) j ∈ G λ j (cid:13)(cid:13) h aj − E [ h aj ] (cid:13)(cid:13) d (cid:3) (cid:54) d (cid:16) ε + Θ + 2 d C + (2 d + 1) (cid:37) (cid:17) . (8.22)Using once again the fact that 1 /C (cid:54) ε , (8.22), the choice of (cid:37) , Θ and (cid:37) in (8.6), (8.11)and (8.16) respectively, Markov’s inequality and a union bound, we may select a subset G of G with (cid:80) j ∈ G λ j (cid:62) − (cid:37) and such that (cid:13)(cid:13) h aj − E [ h aj ] (cid:13)(cid:13) (cid:3) (cid:54) (cid:37) for every j ∈ G and every a ∈ X . Since G ⊆ G and (cid:37) (cid:54) (cid:37) , we see that G is as desired. The proof of Proposition8.3 is completed. (cid:3) Appendix A. Proof of Lemma 3.4
Let d, m, ε be as in the statement of Lemma 3.4, and let n be as in (3.8). Fixcoefficients λ , . . . , λ m (cid:62) λ + · · · + λ m = 1, and let V be a finite set with | V | (cid:62) n .Observe that, by the choice of n , we have(A.1) (cid:16) | V | d/ + (1 + d ! m )2 d | V | (cid:17) / d (cid:54) ε and 1 − m exp (cid:16) − d ! 4 d | V | d/ (cid:17) > . We define an equivalence relation ∼ on V d by setting( v , . . . , v d ) ∼ ( v (cid:48) , . . . , v (cid:48) d ) ⇔ there exists a permutation π of [ d ](A.2) such that v (cid:48) i = v π ( i ) for all i ∈ [ d ] , and for every e ∈ V d by [ e ] := { e (cid:48) ∈ V d : e (cid:48) ∼ e } we denote the ∼ -equivalence class of e .We also set Sym( V d ) := V d / ∼ .Next, we fix a collection X = (cid:104) X e : e ∈ Sym( V d ) (cid:105) of [ m ]-valued, independent randomvariables defined on some probability space (Ω , Σ , P ) which satisfy P (cid:0) [ X e = j ] (cid:1) = λ j for every e ∈ Sym( V d ) and every j ∈ [ m ]. Moreover, for every j ∈ [ m ] we define f j : [ m ] Sym( V d ) → R + by setting for every x = ( x e ) e ∈ Sym( V d ) ∈ [ m ] Sym( V d ) ,(A.3) f j ( x ) = (cid:107) { e ∈ V d : x [ e ] = j } − λ j (cid:107) d (cid:3) . Recall that for every v = ( v , v , . . . , v d , v d ) ∈ V d and every (cid:15) = ( (cid:15) , . . . , (cid:15) d ) ∈ { , } d we set v (cid:15) = ( v (cid:15) , . . . , v (cid:15) d d ) ∈ V d . Let I denote the subset of V d consisting of all stringswith distinct entries. Observe that for every x = ( x e ) e ∈ Sym( V d ) ∈ [ m ] Sym( V d ) and every j ∈ [ m ] we have f j ( x ) (3.5) = 1 | V | d (cid:88) v ∈ V d (cid:89) (cid:15) ∈{ , } d ( { j } ( x [ v (cid:15) ] ) − λ j )(A.4) = 1 | V | d (cid:88) v ∈I (cid:88) H ⊆{ , } d ( − λ j ) d −| H | (cid:89) (cid:15) ∈ H { j } ( x [ v (cid:15) ] ) ++ 1 | V | d (cid:88) v ∈ V d \I (cid:89) (cid:15) ∈{ , } d ( { j } ( x [ v (cid:15) ] ) − λ j )and(A.5) (cid:12)(cid:12)(cid:12) | V | d (cid:88) v ∈ V d \I (cid:89) (cid:15) ∈{ , } d ( { j } ( x [ v (cid:15) ] ) − λ j ) (cid:12)(cid:12)(cid:12) (cid:54) | V d \ I|| V | d (cid:54) d | V | . On the other hand, by the definition of I , for every v = ( v , v , . . . , v d , v d ) ∈ I and everydistinct (cid:15) , (cid:15) ∈ { , } d we have [ v (cid:15) ] (cid:54) = [ v (cid:15) ]. Therefore, by the independence of theentries of X and linearity of expectation, for every v = ( v , v , . . . , v d , v d ) ∈ I and every j ∈ [ m ] we have(A.6) E (cid:104) (cid:88) H ⊆{ , } d ( − λ j ) d −| H | (cid:89) (cid:15) ∈ H { j } ( X [ v (cid:15) ] ) (cid:105) = (cid:88) H ⊆{ , } d ( − λ j ) d −| H | λ | H | j = 0 . ECOMPOSITIONS OF HIGH-DIMENSIONAL ARRAYS 45
By (A.4), (A.5) and (A.6), for every j ∈ [ m ] we obtain that(A.7) E [ f j ( X )] (cid:54) d | V | . Moreover, by (A.4) again, if x = ( x e ) e ∈ Sym( V d ) and y = ( y e ) e ∈ Sym( V d ) in [ m ] Sym( V d ) aresuch that |{ e ∈ Sym( V d ) : x e (cid:54) = y e }| (cid:54)
1, then for every j ∈ [ m ] we have(A.8) | f j ( x ) − f j ( y ) | (cid:54) d ! (cid:16) | V | (cid:17) d . By (A.8) and the bounded differences inequality (see, e.g. , [BLM13, Theorem 6.2]), forevery j ∈ [ m ] and every t > P (cid:16)(cid:12)(cid:12) f j ( X ) − E [ f j ( X )] (cid:12)(cid:12) > t (cid:17) (cid:54) (cid:16) − t | V | d d ! 4 d (cid:17) . Applying (A.9) for “ t = 1 / | V | d/ ” and using (A.7), for every j ∈ [ m ] we have(A.10) P (cid:16) f j ( X ) (cid:54) | V | d/ + 2 d | V | (cid:17) (cid:62) − (cid:16) − d !4 d | V | d/ (cid:17) . By (A.10), a union bound and (A.1), we select ω ∈ Ω which belongs to the event(A.11) (cid:92) j ∈ [ m ] (cid:104) f j ( X ) (cid:54) | V | d/ + 2 d | V | (cid:105) . We select a partition (cid:104) D , . . . , D m (cid:105) of Sym( V d ) into nonempty sets such that(A.12) | D j (cid:52) { e ∈ Sym( V d ) : X e ( ω ) = j }| (cid:54) m for every j ∈ [ m ]. Finally, we set E j := { e ∈ V d : [ e ] ∈ D j } for every j ∈ [ m ]. By (A.8),the choice of ω and (A.1), we see that the partition (cid:104) E , . . . , E m (cid:105) is as desired. The proofof Lemma 3.4 is completed. References [Ald81] D. J. Aldous,
Representations for partially exchangeable arrays of random variables , J. Multi-variate Anal. 11 (1981), 581–598.[Ald85] D. J. Aldous.
Exchangeability and related topics , In “´Ecole d’ ´et´e de probabilit´es de Saint-FlourXIII, 1983”, Lecture Notes in Mathematics, vol. 1117, Springer, 1985, 1–198.[Ald10] D. J. Aldous,
More uses of exchangeability: representations of complex random structures , in“Probability and mathematical genetics. Papers in honour of Sir John Kingman”, London Mathe-matical Society Lecture Note Series, Vol. 378, Cambridge University Press, 2010, 35–63.[Au08] T. Austin,
On exchangeable random variables and the statistics of large graphs and hypergraphs ,Probability Surveys 5 (2008), 80–145.[Au13] T. Austin,
Exchangeable random arrays , preprint (2013), available at .[BLM13] S. Boucheron, G. Lugosi and P. Massart,
Concentration Inequalities. A Nonasymptotic Theoryof Independence , Oxford University Press, 2013.[DJ08] P. Diaconis and S. Janson,
Graph limits and exchangeable random graphs , Rend. Mat. Appl. 28(2008), 33–61.[DF80] P. Diaconis and D. Freedman,
Finite exchangeable sequences , Ann. Probab. 8 (19180), 745–764. [DK16] P. Dodos and V. Kanellopoulos,
Ramsey Theory for Product Spaces , Mathematical Surveys andMonographs, Vol. 212, American Mathematical Society, 2016.[DKK20] P. Dodos, V. Kanellopoulos and Th. Karageorgos, L p regular sparse hypergraphs: box norms ,Fund. Math. 248 (2020), 49–77.[DTV20] P. Dodos, K. Tyros and P. Valettas, Concentration estimates for functions of finite high-dimensional random arrays , preprint (2021).[ES81] B. Efron and C. Stein,
The jackknife estimate of variance , Ann. Statist. 9 (1981), 586–596.[FT85] D. H. Fremlin and M. Talagrand,
Subgraphs of random graphs , Trans. Amer. Math. Soc. 291(1985), 551–582.[Go07] W. T. Gowers,
Hypergraph regularity and the multidimensional Szemer´edi theorem , Ann. Math.166 (2007), 897–946.[GT10] B. Green and T. Tao,
Linear equations in primes , Ann. Math. 171 (2010), 1753–1850.[Hoe48] W. Hoeffding,
A class of statistics with asymptotically normal distribution , Ann. Math. Stat. 19(1948), 293–325.[Hoo79] D. N. Hoover,
Relations on probability spaces and arrays of random variables , preprint (1979),available at .[Kal92] O. Kallenberg,
Symmetries on random arrays and set-indexed processes , J. Theor. Probab. 5(1992), 727–765.[Kal05] O. Kallenberg,
Probabilistic symmetries and invariance principles , Probability and its Applica-tions (New York), Springer, 2005.[Le01] M. Ledoux,
The Concentration of Measure Phenomenon , Mathematical Surveys and Monographs,Vol. 89, American Mathematical Society, 2001.[Ra30] F. P. Ramsey,
On a problem of formal logic , Proc. London Math. Soc. 30 (1930), 264–286.[R˝o15] V. R˝odl,
Quasi-randomness and the regularity method in hypergraphs , in “Proceedings of theInternational Congress of Mathematicians” Vol. I, 571–599, 2015.
Department of Mathematics, University of Athens, Panepistimiopolis 157 84, Athens, Greece
Email address : [email protected] Department of Mathematics, University of Athens, Panepistimiopolis 157 84, Athens, Greece
Email address : [email protected] Mathematics Department, University of Missouri, Columbia, MO, 65211
Email address ::