Sequential Selection of a Monotone Subsequence from a Random Permutation
aa r X i v : . [ m a t h . P R ] S e p SEQUENTIAL SELECTION OF A MONOTONE SUBSEQUENCEFROM A RANDOM PERMUTATION
PEICHAO PENG AND J. MICHAEL STEELE
Abstract.
We find a two term asymptotic expansion for the optimal expectedvalue of a sequentially selected monotone subsequence from a random permu-tation of length n . A striking feature of this expansion is that tells us that theexpected value of optimal selection from a random permutation is quantifi-ably larger than optimal sequential selection from an independent sequencesof uniformly distributed random variables; specifically, it is larger by at least(1 /
6) log n + O (1). Key Words.
Monotone subsequence problem, sequential selection, onlineselection, Markov decision problem, nonlinear recursion, asymptotics
Mathematics Subject Classification (2010).
Primary: 60C05, 60G40,90C40; Secondary: 60F99, 90C27, 90C39 Sequential Subsequence Problems
In the classical monotone subsequence problem, one chooses a random permu-tation π : [1 : n ] → [1 : n ], and one considers the length of its longest increasingsubsequence, L n = max { k : π [ i ] < π [ i ] < · · · < π [ i k ] where 1 ≤ i < i · · · < i k ≤ n } . On the other hand, in the sequential monotone subsequence problem one views thevalues π [1], π [2], ... as though they were presented over time to a decision makerwho, when shown the value π [ i ] at time i , must decide (once and for all) either toaccept or reject π [ i ] as element of the selected increasing subsequence.The decision to accept or reject π [ i ] at time i is based on just the knowledge of thetime horizon n and the observed values π [1] , π [2] , . . . , π [ i ]. Thus, in slightly moreformal language, the sequential selection problems amounts to the consideration ofrandom variables of the form(1) L τ n = max { k : π [ τ ] < π [ τ ] < · · · < π [ τ k ] where 1 ≤ τ < τ · · · < τ k ≤ n } , where the indices τ i , i = 1 , , . . . are stopping times with respect to the increasingsequence of σ -fields F k = σ { π [1] , π [2] , . . . , π [ k ] } , 1 ≤ k ≤ n . We call a sequence ofsuch stopping times a feasible selection strategy , and, if we use τ as a shorthand forsuch a strategy, then the quantity of central interest here can be written as(2) s ( n ) = sup τ E [ L τ n ] , where one takes the supremum over all feasible selection strategies. Peichao Peng: Department of Statistics, The Wharton School, University of Pennsylvania, 3730Walnut Street, Philadelphia, PA, 19104. Email address: [email protected] .J. M. Steele: Department of Statistics, The Wharton School, University of Pennsylvania, 3730Walnut Street, Philadelphia, PA, 19104. Email address: [email protected] . It was conjectured in Baer and Brock (1968) that(3) s ( n ) ∼ √ n as n → ∞ , and a proof of this relation was first given in Samuels and Steele (1981). A muchsimpler proof of (3) was later given by Gnedin (2000) who made use of a recursionthat had been used for numerical computations by Baer and Brock (1968). Themain purpose of this note is to show that by a more sustained investigation of thatrecursion one can obtain a two term expansion. Theorem 1 (Sequential Selection from a Random Permutation) . For n → ∞ onehas the asymptotic relation (4) s ( n ) = √ n + 16 log n + O (1) . Given what is known for some closely related problems, the explicit second orderterm (log n ) / n independently uniformly distributedrandom variables X , X , ..., X n . In this problem a feasible selection strategy τ isagain expressed by an increasing sequence of stopping times τ j , j = 1 , , . . . , but nowthe stopping times are adapted to the increasing σ -fields b F j = σ { X , X , . . . , X j } .The analog of (1) is then(5) b L τ n = max { k : X τ < X τ < · · · < X τ k where 1 ≤ τ < τ · · · < τ k ≤ n } , and the analog of (2) is given by b s ( n ) = sup τ E [ b L τ n ] . It was proved by Bruss and Robertson (1991) that for b s ( n ) one has a uniformupper bound(6) b s ( n ) ≤ √ n for all n ≥ , so, by comparison with (4), we see there is a sense in which sequential selectionof a monotone subsequence from a permutation is easier than sequential selectionfrom an independent sequence. In part, this is intuitive; each successive observationfrom a permutation gives useful information about the subsequent values that canbe observed. By (4) one quantifies how much this information helps.Since (6) holds for all n and since (4) is only asymptotic, it also seems naturalto ask if there is a relation between b s ( n ) and s ( n ) that is valid for all n . There issuch a relation if one gives up the logarithmic gap. Theorem 2 (Selection for Random Permutations vs Random Sequences) . One hasfor all n = 1 , , . . . that b s ( n ) ≤ s ( n ) . Here we should also note that much more is known about b s ( n ) than just (6); inparticular, there are several further connections between s ( n ) and b s ( n ). These aretaken up in a later section, but first it will be useful to give the proofs of Theorems1 and 2.The proof of Theorem 1 takes most of our effort, and it is given over the next fewsections. Section 2 develops the basic recurrence relations, and Section 3 developsstability relations for these recursions. In Section 4 we then do the calculations thatsupport a candidate for the asymptotic approximation of s ( n ), and we compete the proof of Theorem 1. Our arguments conclude in Section 5 with the brief — andalmost computation free — proof of Theorem 2. Finally, in Section 6 we discussfurther relations between s ( n ), b s ( n ), and some other closely related quantities thatmotivate two open problems.2. Recurrence Relations
One can get a recurrence relation for s ( n ) by first step analysis. Specifically, wetake a random permutation π : [1 : n + 1] → [1 : n + 1], and we consider its initialvalue π [1] = k . If we reject π [1] as an element of our subsequence, we are faced withthe problem of sequential selection from the reduced random permutation π ′ on an n -element set. Alternatively, if we choose π [1] = k as an element of our subsequence,we are then faced with the problem of sequential selection for a reduced randompermutation π ′′ of the set { k + 1 , k + 2 , . . . , n + 1 } that has n + 1 − k elements. Bytaking the better of these two possibilities, we get from the uniform distribution of π [1] that s ( n + 1) = 1 n + 1 n +1 X k =1 max { s ( n ) , s ( n + 1 − k ) } . (7)From the definition (2) of s ( n ) one has s (1) = 1, so subsequent values can then becomputed by (7). For illustration and for later discussion, we note that one has theapproximate values: n s ( n ) 1 1 . .
375 2 .
725 3 .
046 3 .
333 3 .
601 3 .
857 4 . √ n .
414 2 2 .
449 2 .
828 3 .
162 3 .
464 3 .
742 4 4 .
243 4 . s ( n ) ≤ √ n , and, infact, this relation persists for all 1 ≤ n ≤ n = 175 one has √ n < s ( n ), just as (4) requires for all sufficiently large values of n .We also know from (2) that the map n s ( n ) is strictly monotone increasing,and, as a consequence, the recursion (7) can be written a bit more simply as s ( n + 1) = 1 n + 1 max ≤ k ≤ n ( ( n − k + 1) s ( n ) + n X i = n − k +1 { s ( i ) + 1 } ) (8) = 1 n + 1 max ≤ k ≤ n ( ( n − k + 1) s ( n ) + k + n X i = n − k +1 s ( i ) ) . In essence, this recursion goes back to Baer and Brock (1968, p. 408), and it is thebasis of most of our analysis.3.
Comparison Principles
Given a map g : N → R and 1 ≤ k ≤ n , it will be convenient to set(9) H ( n, k, g ) = k + ( n − k + 1) g ( n ) + n X i = n − k +1 g ( i ) , so the optimality recursion (8) can be written more succinctly as(10) s ( n + 1) = 1 n + 1 max ≤ k ≤ n H ( n, k, s ) . The next two lemmas make rigorous the idea that if g is almost a solution of (10)for all n , then g ( n ) is close to s ( n ) for all n . Lemma 3 (Upper Comparison) . If δ : N → R + , ≤ g (1) + δ (1) , and (11) 1 n + 1 max ≤ k ≤ n H ( n, k, g ) ≤ g ( n + 1) + δ ( n + 1) for all n ≥ , then one has (12) s ( n ) ≤ g ( n ) + n X i =1 δ ( i ) for all n ≥ . Proof.
We set ∆( i ) = δ (1)+ δ (2)+ · · · + δ ( i ), and we argue by induction. Specifically,using (12) for 1 ≤ i ≤ n we have H ( n, k, s ) = k + ( n − k + 1) s ( n ) + n X i = n − k +1 s ( i ) ≤ k + ( n − k + 1)( g ( n ) + ∆( n )) + n X i = n − k +1 { g ( i ) + ∆( i ) } so by monotonicity of ∆( · ) we have1 n + 1 H ( n, k, s ) ≤ n + 1 H ( n, k, g ) + ∆( n ) . Now, when we take the maximum over k ∈ [1 : n ], the recursion (8) and theinduction condition (11), give us s ( n + 1) ≤ n + 1 max ≤ k ≤ n H ( n, k, g ) + ∆( n ) ≤ g ( n + 1) + δ ( n + 1) + ∆( n ) = g ( n + 1) + ∆( n + 1) , so induction establishes (12) for all n ≥ (cid:3) Naturally, there is a lower bound comparison principle that parallels Lemma 3.The statement has several moving parts, so we frame it as a separate lemma eventhough its proof can be safely omitted.
Lemma 4 (Lower Comparison) . If δ : N → R + , g (1) − δ (1) ≤ , and g ( n + 1) − δ ( n + 1) ≤ n + 1 max ≤ k ≤ n H ( n, k, g ) for all n ≥ , then one has g ( n ) − n X i =1 δ ( i ) ≤ s ( n ) for all n ≥ . An Approximation Solution
We now argue that the function f : N → R defined by(13) f ( n ) = √ n + 16 log n, gives one an approximate solution of the recurrence equation (8) for n s ( n ). Proposition 5.
There is a constant < B < ∞ such that for all n ≥ , one has (14) − Bn − / ≤ n + 1 (cid:26) max ≤ k ≤ n H ( n, k, f ) (cid:27) − f ( n + 1) ≤ Bn − / . First Step: Localization of the Maximum
To deal with the maximum in (14), we first estimate k ∗ ( n ) = locmax k H ( n, k, f ) . From the definition (9) of H ( n, k, f ) we find H ( n, k + 1 , f ) − H ( n, k, f ) = 1 − f ( n ) + f ( n − k ) , and, from the definition (13) of f , we see this difference is monotone decreasingfunction of k ; accordingly, we also have the representation(15) k ∗ ( n ) = 1 + max { k : 0 ≤ − f ( n ) + f ( n − k ) } . Now, for each n = 1 , , . . . we then consider the function D n : [0 , n ] → R defined bysetting D n ( x ) = 1 − f ( n ) + f ( n − x ) = 1 − {√ n − p n − x ) } − { log n − log( n − x ) } . This function is strictly decreasing with D n (0) = 1 and D n ( n ) = −∞ , so there isa unique solution of the equation D n ( x ) = 0. For x ∈ [0 , n ) we also have the easybound D n ( x ) = 1 − Z n n − x ) √ u du −
16 log( n/ ( n − x )) ≤ − x √ n . This gives us D n ( √ n ) ≤
0, so by monotonicity we have x n ≤ √ n .To refine this bound to an asymptotic estimate, we start with he equation D n ( x n ) = 0 and we apply Taylor expansions to get1 = √ n n − (1 − x n /n ) / o −
16 log(1 − x n /n )= √ n n x n n + O ( x n /n ) o + O ( x n /n ) . By simplification, we then get(16) √ n = x n + O ( x n /n ) + O ( x n /n / ) = x n + O (1) , where in the last step we used our first bound x n ≤ √ n .Finally, by (16) and the characterization (15), we immediately find the estimatethat we need for k ∗ ( n ). Lemma 6.
There is a constant
A > such that for all n ≥ , we have (17) √ n − A ≤ k ∗ ( n ) ≤ √ n + A. Remark 7.
The relations (16) and (17) can be sharpened. Specifically, if we use atwo-term Taylor series with integral remainders, then one can show √ n − ≤ x n .Since we already know that x n ≤ √ n , we then see from the characterization (15)and integrality of k ∗ ( n ) that we can take A = 2 in Lemma 6. This refinement doesnot lead to a meaningful improvement in Theorem 1, so we omit the details of theexpansions with remainders. Completion of Proof of Proposition 5
To prove Proposition 5, we first note that the definition (9) of H ( n, k, f ) one hasfor all 1 ≤ k ≤ n that(18) 1 n + 1 H ( n, k, f ) = f ( n ) + 1 n + 1 ( k − k − X i =1 ( f ( n ) − f ( n − i )) ) The task is to estimate the right-hand side of (18) when k = k ∗ ( n ) and k ∗ ( n ) isgiven by (15).For the moment, we assume that one has k ≤ D √ n where D > k − X i =1 ( f ( n ) − f ( n − i )) = k − X i =1 (cid:16) √ n − p n − i ) (cid:17) + k − X i =1 (cid:18) log n − log( n − i )6 (cid:19) = k − X i =1 (cid:18) i √ n + i n √ n + O (cid:18) i n / (cid:19)(cid:19) + k − X i =1 (cid:18) i n + O (cid:18) i n (cid:19)(cid:19) = ( k − k √ n + ( k − k (2 k − n √ n + ( k − k n + O ( n − / ) , (19)where the implied constant of the remainder term depends only on D .We now define r ( n ) by the relation k ∗ ( n ) = √ n + r ( n ), and we note by (17)that | r ( n ) | ≤ A . Direct algebraic expansions then give us the elementary estimates( k ∗ ( n ) − k ∗ ( n )12 n = 16 + O ( n − / )and ( k ∗ ( n ) − k ∗ ( n )(2 k ∗ ( n ) − n √ n = 16 + O ( n − / ) , where in each case the implied constant depends only on A .Estimation of the first summand of (19) is slightly more delicate than this sincewe need to account for the dependence of this term on r ( n ); specifically we have( k ∗ ( n ) − k ∗ ( n )2 √ n = (cid:0) √ n + r ( n ) − (cid:1) (cid:0) √ n + r ( n ) (cid:1) √ n = p n/ r ( n ) −
12 + O ( n − / ) . Now, for a pleasing surprise, we note from the last estimate and from the definitionof k ∗ ( n ) and r ( n ) that we have cancelation of r ( n ) when we then compute thecritical sum; thus, one has simply(20) k ∗ ( n ) − k ∗ ( n ) − X i =1 ( f ( n ) − f ( n − i )) = p n/ O ( n − / ) . Finally, from the formula (13) for f ( · ), we have the Taylor expansion(21) f ( n + 1) − f ( n ) = 1 √ n + 16 n + O ( n − / ) , so, when we return to the identity (18), we see that the estimates (20) and (21)give us the estimate1 n + 1 (cid:26) max ≤ k ≤ n H ( n, k, f ) (cid:27) − f ( n + 1)= 1 n + 1 (cid:18)p n/ O ( n − / ) (cid:19) + f ( n ) − f ( n + 1) = O ( n − / ) . Here the implied constant is absolute, and the proof of Proposition 5 is complete.
Completion of Proof of Theorem 1
Lemmas 3 and 4 combine with Proposition 5 to tell us that by summing thesequence n − / , n = 1 , , . . . and by writing ζ ( z ) = 1 + 2 − z + 3 − z + · · · one has | s ( n ) − f ( n ) | ≤ ζ (3 / B ≤ (2 . B for all n ≥ . This is slightly more than one needs to complete the proof of Theorem 1.5.
Proof of Theorem 2
The sequential monotone selection problem is a finite horizon Markov decisionproblem with bounded rewards and finite action space, and for such problems it isknown one cannot improve upon an optimal deterministic strategy by the use ofstrategies that incorporate randomization, (cf. Bertsekas and Shreve, 1978, Corol-lary 8.5.1). The proof Theorem 2 exploits this observation by constructing a ran-domized algorithm for the sequential selection of a monotone subsequence from arandom permutation.We first recall that if e i , i = 1 , , . . . , n + 1 are independent exponentially dis-tributed random variables with mean 1 and if one sets Y i = e + e + · · · + e i e + e + · · · + e n +1 , then the vector ( Y , Y , . . . , Y n ) has the same distribution as the vector of orderstatistics ( X (1) , X (2) , . . . , X ( n ) ) of an i.i.d. sample of size n from the uniform distri-bution. Next we let A denote an optimal algorithm for sequential selection from anindependent sample X , X , . . . , X n from the uniform distribution, and we let τ ( A )denote the associated sequence of stopping times. If b L τ ( A ) n denotes the length of thesubsequence that is chosen from from X , X , . . . , X n when one follows the strategy τ ( A ) determined by A , then by optimality of A for selection from X , X , . . . , X n we have b s ( n ) = sup τ E [ b L τ n ] = E [ b L τ ( A ) n ] . We use the algorithm A to construct a new randomized algorithm A ′ for se-quential selection of an increasing from a random permutation π : [ n ] [ n ].First, the decision maker generates independent exponential random variables e i , i = 1 , , . . . , n + 1 as above. This is done off-line, and this step can be viewed as aninternal randomization.Now, for i = 1 , , . . . , n , when we are presented with π [ i ] at time i , we compute X i = Y π [ i ] . Finally, if at time i the value X i would be accepted by the algorithm A , then the algorithm A ′ accepts π [ i ]. Otherwise the newly observed value π [ i ] isrejected. By our construction we have(22) E [ L τ ( A ′ ) n ] = E [ b L τ ( A ) n ] = b s ( n ) . Moreover, A ′ is a randomized algorithm for construction an increasing subse-quence of a random permutation π . By definition, s ( n ) is the expected length ofa monotone subsequence selected from a random permutation by an optimal de-terministic algorithm, and by our earlier observation, the randomized algorithm A ′ cannot do better. Thus, from (22) one has b s ( n ) ≤ s ( n ), and the proof of Theorem1 is complete. 6. Further Connections and Considerations
As we noted before, Bruss and Robertson (1991) discovered the uniform bound(23) b s ( n ) ≤ √ n for all n ≥ , and their proof depended on a general bound for the expected value of the partialsums of the smallest order statistics of a uniformly distributed random sample.Gnedin (1999) later gave a much different proof of (23) and generalized the boundin a way that accommodate random samples with random sizes. More recently,Arlotto, Mossel and Steele (2015) obtained yet another proof (23) as a corollary tobounds on the quickest selection problem , which is an informal dual to the tradi-tional selection problem.Since the bound (23) is now well understood from several points of view, it isreasonable to ask about the possibility of some corresponding uniform bound on s ( n ). The numerical values that we noted after the recursion (23) and the relation(24) s ( n ) = √ n + 16 log n + O (1)from Theorem 1 both tell us that one cannot expect a uniform bound for s ( n ) thatis as simple as that for b s ( n ) given by (23). Nevertheless, numerical evidence suggestthat the O (1) term in (24) is always negative. The tools used here cannot confirmthis conjecture, but the multiple perspectives available for (23) give one hope.A closely related issue arises for b s ( n ) when one considers lower bounds. Here thefirst steps were taken by Bruss and Delbaen (2001) who considered i.i.d. samples ofsize N ν where N ν is an independent random variable with the Poisson distributionwith mean ν . In the natural (but slightly overburdened) notation, they proved thatthere is a constant c > √ ν − c log ν ≤ b s ( ν );moreover, Bruss and Delbaen (2004) subsequently proved that for the optimal fea-sible strategy τ ∗ = ( τ , τ , . . . ) the random variable b L τ ∗ N ν = max { k : X τ < X τ < · · · < X τ k where 1 ≤ τ < τ · · · < τ k ≤ N ν } , also satisfies a central limit theorem. Arlotto, Nguyen and Steele (2015) consideredthe de-Poissonization of these results, and it was found that one has the corre-sponding CLT for b L τ ∗ n where the sample size n is deterministic. In particular, onehas the bounds √ n − c log n ≤ b s ( n ) ≤ √ n. Now, by analogy with (24), one strongly expects that there is a constant c > b s ( n ) = √ n − c log n + O (1) . Still, a proof this conjecture is reasonably remote, since, for the moment, there isnot even a compelling candidate for the value of c .For a second point of comparison, one can recall the non-sequential selectionproblem where one studies ℓ ( n ) = E [max { k : X i < X i < . . . < X i k , ≤ i < i < · · · < i k ≤ n } ] . Through a long sequence of investigations culminating with Baik, Deift and Jo-hansson (1999), it is now known that one has(26) ℓ ( n ) = 2 √ n − αn / + o ( n / ) , where the constant α = 1 . ... is determined numerically in terms of solutionsof a Painlev´e equation of type II. Romik (2014) gives an elegant account of theextensive technology behind (26), and there are interesting analogies between ℓ ( n )and s ( n ). Nevertheless, a proof of the conjecture (25) seems much more likely tocome from direct methods like those used here to prove (24).Finally, one should note that the asymptotic formulas for n ℓ ( n ), n s ( n ),and n b s ( n ) all suggest that these maps are concave, but so far only n b s ( n )has been proved to be concave (cf. Arlotto, Nguyen and Steele (2015, p. 3604)). References
Arlotto, A., Mossel, E. and Steele, J. M. (2015), ‘Quickest online selection of an increasingsubsequence of specified size’, to appear in
Random Structures and Algorithms (eprintarXiv:1412.7985) .Arlotto, A., Nguyen, V. V. and Steele, J. M. (2015), ‘Optimal online selection of a monotonesubsequence: a central limit theorem’,
Stochastic Process. Appl. (9), 3596–3622.Baer, R. M. and Brock, P. (1968), ‘Natural sorting over permutation spaces’,
Math. Comp. , 385–410.Baik, J., Deift, P. and Johansson, K. (1999), ‘On the distribution of the length of the longestincreasing subsequence of random permutations’, J. Amer. Math. Soc. (4), 1119–1178.Bertsekas, D. P. and Shreve, S. E. (1978), Stochastic optimal control: the discrete time case ,Vol. 139 of
Mathematics in Science and Engineering , Academic Press Inc. [Harcourt BraceJovanovich Publishers], New York, NY.Bruss, F. T. and Delbaen, F. (2001), ‘Optimal rules for the sequential selection of monotonesubsequences of maximum expected length’,
Stochastic Process. Appl. (2), 313–342.Bruss, F. T. and Delbaen, F. (2004), ‘A central limit theorem for the optimal selection process formonotone subsequences of maximum expected length’, Stochastic Process. Appl. (2), 287–311.Bruss, F. T. and Robertson, J. B. (1991), “‘Wald’s lemma” for sums of order statistics of i.i.d.random variables’,
Adv. in Appl. Probab. (3), 612–623.Gnedin, A. V. (1999), ‘Sequential selection of an increasing subsequence from a sample of randomsize’, J. Appl. Probab. (4), 1074–1085.Gnedin, A. V. (2000), ‘A note on sequential selection from permutations’, Combin. Probab. Com-put. (1), 13–17.Romik, D. (2014), The Surprising Mathematics of Longest Increasing Subsequences , CambridgeUniversity Press, Cambridge.Samuels, S. M. and Steele, J. M. (1981), ‘Optimal sequential selection of a monotone sequencefrom a random sample’,
Ann. Probab.9