A refined and asymptotic analysis of optimal stopping problems of Bruss and Weber
AA refined and asymptotic analysis of optimal stoppingproblems of Bruss and Weber
Guy Louchard ∗ August 22, 2018
Abstract
The classical secretary problem has been generalized over the years into several directions. Inthis paper we confine our interest to those generalizations which have to do with the more generalproblem of stopping on a last observation of a specific kind. We follow Dendievel [10], [11], (where abibliography can be found) who studies several types of such problems, mainly initiated by Bruss [3]and [5], Weber [17] and [18]. Whether in discrete time or continuous time, whether all parametersare known or must be sequentially estimated, we shall call such problems simply
Bruss-Weberproblems.
Our contribution in the present paper is a refined analysis of several problems in thisclass and a study of the asymptotic behaviour of solutions.The problems we consider center around the following model. Let X , X , . . . , X n be a sequenceof independent random variables which can take three values: { +1 , − , } . Let p := P ( X i = 1) , p (cid:48) := P ( X i = − , ˜ q := P ( X i = 0) , p ≥ p (cid:48) , where p + p (cid:48) + ˜ q = 1. The goal is to maximize the probabilityof stopping on a value +1 or − p known, n unknown, then n known, p unknown and finally n, p unknown are considered. We also presentsimulations of the corresponding complete selection algorithm. Keywords : Stopping times, Unified Approach to best choice, Odds-algorithm, Optimal solutions,x-Strategy, Asymptotic expansions, Incomplete information. : 60G40 (68W27,62L12)
The classical secretary problem has been generalized over the years into several directions. In thispaper we confine our interest to those generalizations which have to do with the more general problemof stopping on a last observation of a specific kind. We follow Dendievel [10], [11], (where a bibliographycan be found) who studies several types of such problems, mainly initiated by Bruss [3], [5] and Weber[17], [18]. Whether in discrete time or continuous time, whether all parameters are known or must besequentially estimated, we shall call such problems simply
Bruss-Weber problems.
Bruss [5] studied the case of stopping on a last 1 in a sequence of n independent random variables X , X , . . . , X n , taking values { , } . This led to the versatile odds-algorithm and also to a similarmethod in continuous-time, allowing for interesting applications in different domains, as e.g. in in-vestment problems studied in Bruss and Ferguson [7]. See also Szajowski and (cid:32)Lebek [15]. Moreover,Bruss and Louchard [8] studied the case where the odds are unknown and have to be sequentiallyestimated, showing a convincing stability for applications.Weber (R.R. Weber, University of Cambridge), considered the model of iid random variablestaking values in { +1 , − , } . The goal is to maximize the probability of stopping on a value +1 or − ∗ Universit´e Libre de Bruxelles, D´epartement d’Informatique, CP 212, Boulevard du Triomphe, B-1050 Bruxelles,Belgium, email: [email protected] a r X i v : . [ c s . PF ] M a y hen teaching the odds-algorithm in his course (see section 6 of his course on optimization andcontrol [17]), Weber proposed the following problem to his students: A financial advisor can impress his clients if immediately following a week in which the FTSEindex moves by more than in some direction he correctly predicts that this is the last week duringthe calendar year that it moves more than in that directionSuppose that in each week the change in the index is independently up by at least , down by atleast or neither of these, with probabilities p , p and − p respectively ( p ≤ / ). He makes atmost one prediction this year. With what strategy does he maximize the probability of impressing hisclients? The solution of this interesting problem is easy but can only be partially retrieved from the odds-algorithm.Weber [18] then discussed with Bruss several more difficult versions of this problem, some of themstudied in Dendievel’s PhD thesis [11].Let us also mention shortly related work: Hsiau and Yang [12] have studied the problem of stoppingon a last 1 in a sequence of Bernoulli trials in a Markovian framework, where the value taken by the k th variable is influenced by the value of the the ( k − p . Tamaki [16] generalized the odds-algorithm by introducingmultiplicative odds in order to solve the problem of optimal stopping on any of a fixed number oflast successes. Surprising coincidences of lower bounds for odds-problems with multiple stopping havebeen discovered by Matsui and Ano [14], generalizing Bruss [6]. A more specific interesting problem ofmultiple stopping in Bernoulli trials with a random number of observations was studied by Kurushimaand Ano [13].Let p := P ( X i = 1) , p (cid:48) := P ( X i = − , ˜ q := P ( X i = 0) , p ≥ p (cid:48) , where p + p (cid:48) + ˜ q = 1.A first problem studied in [10] is to maximize for a fixed number n of variables the success prob-ability w j,k , j ≥ k with the following strategy: we observe X , X , . . . . Wait until i = k . From k on, if X i = − X i and stop. If not we proceed to the next random variable and start thealgorithm again. If no − j , then, from j on, if X i = +1 or X i = − X i = 0 from j to n ) then we fail. The goal is to find j ∗ , k ∗ such that w j ∗ ,k ∗ is maximum. In [10], explicit expressions for w j,k , w j,j are given and j ∗ , k ∗ arenumerically computed for given n . Dendievel also proves that the problem is monotone in the senseof Assaf and Samuel-Cahn [2]: if at a certain time it is optimal to stop on a 1 (respectively on a − −
1) at any later time index. Also, it is proved in[10], that if p ≥ p (cid:48) then j ∗ ≥ k .Our contribution is the following: in Section 2, we provide explicit optimal solutions in a continuousmodel and in the present discrete case for p > p (cid:48) and p = p (cid:48) .Another problem, initiated by a model of Bruss in continuous time, and leading to the 1/e-law ofbest choice (Bruss [3]) is a problem in continuous time, now with a fixed total number of variables n with possible values in 0 , − ,
1. More precisely, let U i , i = 1 , , . . . , n be independent random variablesuniformly distributed on the interval [0 , T i = U { i } : T i is the i th order statistic of the U i ’s. T i is the arrival time of X i . The strategy is to wait until some time x ∗ n and from x ∗ n on, we selectthe first X i = +1 or X i = −
1, using the previous algorithm with p = p (cid:48) . Following Bruss [5], wecall this strategy an x-strategy. In [10], for this problem, the author gives the optimal x ∗ n and thecorresponding success probability P ∗ n .In Section 3 we provide some asymptotic expansions for this x-strategy’s parameters, for p = p (cid:48) .We also consider the success probability for small p and for the case p > p (cid:48) .In Section 4, following a suggestion by Bruss, we have analyzed an x-strategy with incompleteinformation: the cases p known, n unknown, then n known, p unknown and finally n, p unknown areconsidered. We also present simulations of the complete selection algorithm.2 The optimal solution
In this Section, we analyze explicitly the optimal solutions in the continuous and discrete case for p > p (cid:48) and p = p (cid:48) . The following notations will be used in the sequel: q := 1 − p, q (cid:48) = 1 − p (cid:48) , ˜ q = 1 − p − p (cid:48) . p > p (cid:48) Let us first consider p > p (cid:48) , j ≥ k . The success probabilities satisfy the following forward recurrenceequations (these are easily obtained from the stopping times characterizations): w j,j = pq n − j + p (cid:48) q (cid:48) n − j + ˜ qw j +1 ,j +1 , w n,n = p + p (cid:48) , (1) w j,k = p (cid:48) q (cid:48) n − k + q (cid:48) w j,k +1 . (2)The solutions, already given in Dendievel [10], are w jj = ( p q n − j +1 − p ˜ q n − j +1 + p (cid:48) q (cid:48) n − j +1 − p (cid:48) ˜ q n − j +1 ) / ( p (cid:48) p ) , (3) w j,k = ( j − k ) p (cid:48) q (cid:48) n − k + q (cid:48) j − k (cid:18) p ( q n − j +1 − ˜ q n − j +1 ) p (cid:48) + p (cid:48) ( q (cid:48) n − j +1 − ˜ q n − j +1 ) p (cid:19) . (4)If j ≤ k , we use w k , j := ( k − j ) p q n − j + q k − j (cid:32) p (cid:48) ( q (cid:48) n − k +1 − ˜q n − k +1 ) p + p ( q n − k +1 − ˜q n − k +1) p (cid:48) (cid:33) . Simplification using generating functions
We shall show that these expressions can be nicely derived by using backward generating functions.Let F ( z ) := (cid:80) n − j = −∞ z n − j w j,j . From (1), we have F ( z ) − p − p (cid:48) − p (cid:48) q (cid:48) z − z + p (cid:48) z − p q z − z + z p − ˜ q z F ( z ) = 0 , the solution of which is F ( z ) = − p (cid:48) z + p + p (cid:48) + 2 p p (cid:48) z − z p (1 − z + z p ) (1 − z + p (cid:48) z ) (1 − z + z p + p (cid:48) z )= − ( p + p (cid:48) ) ˜ qp (cid:48) p (1 − z + z p + p (cid:48) z ) + p (cid:48) q (cid:48) p (1 − z + p (cid:48) z ) + p qp (cid:48) (1 − z + z p ) . This immediately leads to (3). Similarly, let F j ( z ) := (cid:80) j − k = −∞ z j − k w j,k . From (2) this satisfies F j ( z ) − (cid:0) p q n − j +1 − p ˜ q n − j +1 + p (cid:48) q (cid:48) n − j +1 − p (cid:48) ˜ q n − j +1 (cid:1) / ( p (cid:48) p ) − p (cid:48) zq (cid:48) − n + j − (1 − z + p (cid:48) z ) − q (cid:48) z F j ( z ) = 0 , the solution of which, expanded into partial fractions, leads to F j ( z ) = (cid:0) − p (cid:48) q (cid:48) n − j + p (cid:48) ˜ q n − j − p p (cid:48) q (cid:48) n − j + p (cid:48) q (cid:48) n − j + p p (cid:48) ˜ q n − j − p (cid:48) ˜ q n − j + p (cid:48) p ˜ q n − j − p ˜ q n − j + p q n − j − p q n − j + p ˜ q n − j (cid:1) / ((1 − z + p (cid:48) z ) p (cid:48) p ) + p (cid:48) q (cid:48) n − j (1 − z + p (cid:48) z ) . This simplifies as F j ( z ) = (cid:0) p qq n − j + p (cid:48) ˜ qq (cid:48) n − j − ( p + p (cid:48) )˜ q n − j +1 (cid:1) / ((1 − z + p (cid:48) z ) pp (cid:48) ) + p (cid:48) q (cid:48) n − j (1 − z + p (cid:48) z ) . F th j ( z ) = p (cid:48) zq (cid:48) ( − n + j − ( − q (cid:48) z ) − q (cid:48) (cid:18) pp (cid:48) (cid:18) q ( − n + j − − q ( − n + j − (cid:19) + p (cid:48) p (cid:18) q (cid:48) ( − n + j − − q ( − n + j − (cid:19)(cid:19) z ( − q (cid:48) z ) . Identification with F j ( z ) is immediate. Computation of the optimal values j ∗ , k ∗ Let us now turn to the main object of this Section which is the computation of the optimal values j ∗ , k ∗ . It is proved in [10] that, if p > p (cid:48) then j ∗ ≥ j ∗ . Actually, setting j = n − C, k = n − D in (3),(4),we see that w j,k , w k,j do not depend on n . We have, with C ≤ D , and using C, D as continuousvariables, w C,D := ( − C + D ) p (cid:48) q (cid:48) D + q (cid:48)− C + D (cid:32) p ( q C +1 − ˜ q C +1 ) p (cid:48) + p (cid:48) ( q (cid:48) C +1 − ˜ q C +1) p (cid:33) , and if D ≤ C , w D,C := ( − D + C ) p q C + q − D + C (cid:18) p (cid:48) ( q (cid:48) D +1 − ˜ q D +1 ) p + p ( q D +1 − ˜ q D +1 ) p (cid:48) (cid:19) . The optimal value C ∗ is the (unique) solution of φ ( C ∗ ) = 0 , (5) φ ( C ) := ∂w C,D ∂C q (cid:48) C − D pp (cid:48) = − ˜ q ( p + p (cid:48) ) ( − ln( q (cid:48) ) + ln(˜ q ))˜ q C + p q ( − ln( q (cid:48) ) + ln( q )) q C − p (cid:48) pq (cid:48) C . (6)First of all, we have ˜ q < q < q (cid:48) , p (cid:48) < p for 0 ≤ p ≤ / p (cid:48) < − p for 1 / ≤ p ≤
1. DividingEq. (6) by q (cid:48) C , we see that φ ( C ) ∼ φ as ( C ) = − p (cid:48) pq (cid:48) C , C → ∞ which is negative. A plot of φ ( C ), for p = 0 . , p (cid:48) = 0 .
05 is given in Figure 1, together with φ as ( C ), showing numerically a uniquemaximum, but we need a formal proof.We would like to have φ (0) >
0, this would imply the existence of C ∗ . A plot of φ (0) (satisfyingthe constraints on ( p, p (cid:48) )) is given in Figure 2. We see that there exists a curve p (cid:48) = γ ( p ), given in Fig-ure 3, such that φ (0) < p (cid:48) > γ ( p ). In this case, we must choose C ∗ = 0. Otherwise, we know that C ∗ does exist. The extremal points of γ ( p ) are (0 . . . . , . . . . ) , (0 . . . . , C ∗ . By dividing Eq.(5) by q (cid:48) C , we obtain, with ˜ r :=˜ q/q (cid:48) , r := q/q (cid:48) , ˜ r < r , A ˜ r C = A r C + A , where A , A , A do not depend on C . On both sides, we have strictly convex/concave functions of C which ensure the uniqueness of C ∗ .Interestingly, C ∗ does not depend on D . The optimal value D ∗ is the solution, for C = C ∗ , of ∂w C,D ∂D q (cid:48)− D pp (cid:48) = p (cid:48) p − p (cid:48) ln( q (cid:48) ) C p + p (cid:48) ln( q (cid:48) ) D p + ln( q (cid:48) ) q (cid:48)− C p q C +1 − ln( q (cid:48) ) q (cid:48)− C p ˜ q C +1 + p (cid:48) q (cid:48) ln( q (cid:48) ) − ln( q (cid:48) ) q (cid:48)− C p (cid:48) ˜ q C +1 = 0 , this gives D = φ ( C ) :== (cid:32) − p q q C p (cid:48) + ˜ q ( p + p (cid:48) ) p (cid:48) p ˜ q C (cid:33) q (cid:48)− C + − p + ln( q (cid:48) ) C p − q (cid:48) ln( q (cid:48) )ln( q (cid:48) ) p , (7)and D ∗ = φ ( C ∗ ). 4 Figure 1: φ ( C ), p = 0 . , p (cid:48) = 0 .
05, together with φ as ( C ) (lower curve) Figure 2: A plot φ (0) defined in Eq.6 as a function of p, p (cid:48) Figure 3: The graphic shows the functions γ ( p ) (circles), γ ( p ) (box), γ ( p ) (cross), γ ( p ) (diamonds)defined in the text with the constraints on ( p, p (cid:48) ) The acceptance regions
1. Curiously enough, even if we must choose C ∗ = 0 (see above), D ∗ is not necessarily non-negative! If we solve φ (0) = 0 w.r.t p (cid:48) for each p , we obtain a second curve p (cid:48) = γ ( p ) also given in Figure3. The extremal points of γ ( p ) are (0 . . . . , . . . . ) , (1 , p (cid:48) > γ ( p ),then we must choose D ∗ = 0 which means waiting until X n . Notice that the two curves do cross.2. Even more interesting, even if C ∗ > D ∗ is not necessarily > C ∗ . If we solve { φ ( C ∗ ) =0 , φ ( C ∗ ) = C ∗ } w.r.t. { C ∗ , p (cid:48) } , we obtain a third curve p (cid:48) = γ ( p ) also given in Figure 3. If p (cid:48) > γ ( p ), we must choose the optimal point on the diagonal: see the remark below at theend of Section 2.3. The intersection of γ , γ , γ is given by p • = 0 . . . . , p (cid:48)• =0 . . . . .3. Finally, if we stay above the curve γ ( p ), we obtain C ∗ <
0. For instance, for C ∗ = − .
3, if wesolve φ ( − .
3) = 0 w.r.t p (cid:48) for each p , we obtain a fourth curve p (cid:48) = γ ( p ) also given in Figure 3.The extremal points of γ ( p ) are (0 . . . . , . . . . ) , (0 . , γ ( p )is of course not practically useful in our analysis ( we must have C ∗ ≥ useful table summarizing acceptance regions The following table 1 shows the different { p, p (cid:48) } regions and their corresponding C ∗ , D ∗ character-istics. p, p (cid:48) Theoretical C ∗ , D ∗ Practical C ∗ , D ∗ p (cid:48) > γ ( p ) , p (cid:48) > γ ( p ) C ∗ < , φ (0) < C ∗ = 0 , D ∗ = 0 p (cid:48) = γ ( p ) , p > p • C ∗ < , φ (0) = 0 C ∗ = 0 , D ∗ = 0 p (cid:48) > γ ( p ) , p (cid:48) < γ ( p ) , p > p • C ∗ < , φ (0) > C ∗ = 0 , D ∗ = φ (0) p (cid:48) = γ ( p ) , p < p • C ∗ = 0 , φ (0) < C ∗ = 0 , D ∗ = 0 p = p • , p (cid:48) = p (cid:48)• C ∗ = 0 , φ (0) = 0 C ∗ = 0 , D ∗ = 0 p (cid:48) = γ ( p ) , p > p • C ∗ = 0 , φ (0) > C ∗ = 0 , D ∗ = φ (0) p (cid:48) > γ ( p ) , p (cid:48) < γ ( p ) , p < p • C ∗ > , φ (0) < C ∗ , D ∗ = C ∗ p (cid:48) = γ ( p ) , p (cid:48) > γ ( p ) , p < p • C ∗ > , φ (0) = 0 C ∗ , D ∗ = C ∗ p (cid:48) < γ ( p ) , p (cid:48) > γ ( p ) , p < p • C ∗ > , φ (0) > , φ ( C ∗ ) < C ∗ C ∗ , D ∗ = C ∗ p (cid:48) < γ ( p ) , p (cid:48) < γ ( p ) , p (cid:48) < γ ( p ) C ∗ > , φ ( C ∗ ) > C ∗ C ∗ , D ∗ = φ ( C ∗ )Table 1: { p, p (cid:48) } regions and their corresponding C ∗ , D ∗ characteristicsAs an illustration of the last line of Table 1, a plot of w C,D , p = 0 . , p (cid:48) = 0 . , C ≤ D is given inFigure 4 as well as w D,C , C ≥ D . Also w C ∗ ,D ∗ = 0 . . . . , p = 0 . , p (cid:48) = 0 . Figure 4: w C,D , C ≤ D , w D,C , C ≥ D , p = 0 . , p (cid:48) = 0 . see the remark below at the end of Section 2.3 .2 The optimal solution in the discrete case for p > p (cid:48) We must now investigate the discrete values, close to C ∗ , D ∗ , leading to the optimal success probabili-ties. Of course, it is not the discrete values just closest to C ∗ , D ∗ . We must compute the correspondingnumerical values of w C,D . For instance, with p = 0 . , p (cid:48) = 0 . C ∗ = 6 . . . . , D ∗ =11 . . . . . The Figure 5 shows C ∗ , φ ( C ) and some closest discrete points. It appears that,numerically, the discrete solution is C ∗ d = 7 , D ∗ d = 12. This fits with the numerical experiments donein [10], with w j,k , n = 40. This gives w C ∗ d ,D ∗ d = 0 . . . . , not far from the continuous value w C ∗ ,D ∗ . Figure 5: C ∗ (vertical line), φ ( C ) (curved line) , p = 0 . , p (cid:48) = 0 .
05, and some closest discrete pointsNotice that two discrete couples can lead to the same optimal solution. For instance, with p (cid:48) =0 . w , − w , is null for p = 0 . . . . . p = p (cid:48) Notice that, if p = p (cid:48) , the coefficient of q C in (6) is null and the coefficient of ˜ q C becomes T :=2˜ qp (cid:48) (ln( q ) − ln(˜ q )). Hence we have the explicit solution C ∗ eq = ln (cid:18) (1 − p ) (2 ln(1 − p ) p − p ln(1 − p )) p (cid:19) ln (cid:18) − p − p (cid:19) . (8)From (7), we obtain φ ,eq ( C ) = (cid:0) − p + p ln( q ) C − q ) + 2 ln( q ) p + 2 ln( q ) q − C (1 − p ) C +1 (cid:1) / ( p ln( q )) , and again, D ∗ eq = φ ,eq ( C ∗ eq ). w C,D , w
C,C become now w eq,C,D = ( D − C ) pq D + q D − C (cid:0) q D +1 − ˜ q C +1 (cid:1) ,w eq,C,C = 2 (cid:0) q C +1 − ˜ q C +1 (cid:1) . (9)8f course, we must use w eq,C,C in our case, and the solution of ∂w eq,C,C ∂C = 0 is given by C ∗ diag = − (ln(ln( q ) / ln(˜ q )) + ln( q ) − ln(˜ q )) / (ln( q ) − ln(˜ q )) . Figure 6 shows, for p = p (cid:48) = 0 . , C ∗ eq = 6 . . . . , φ ,eq ( C ) , D ∗ eq = 6 . . . . , C ∗ diag =6 . . . . the point (6 ,
6) and the diagonal. Notice that the point ( C ∗ eq , D ∗ eq ) is below thediagonal . Of course, only the part C ≤ D is relevant. Figure 6: C ∗ eq (vertical line), φ ,eq ( C ) (curved line), D ∗ eq (circle), C ∗ diag (square), (6 ,
6) (cross) and thediagonal, p = p (cid:48) = 0 . w C ∗ eq ,D ∗ eq = 0 . . . . , this the maximum, but we can not use it. w C ∗ eq ,C ∗ eq =0 . . . . , w C ∗ diag ,D ∗ diag = 0 . . . . is the optimal diagonal continuous value. w , =0 . . . . is the optimal useful discrete value. We observe the order: w C ∗ eq ,D ∗ eq > w ∗ C ∗ diag ,D diag >w C ∗ eq ,C ∗ eq > w , .We notice that, even if p > p (cid:48) , we can have a similar situation. If we choose for instance p =0 . , p (cid:48) = 0 . ,
6) is on the diagonal. This confirms to the existence of γ ( p ) definedabove.A plot of w C,D , C ≤ D and w D,C , C ≥ D , p = p (cid:48) = 0 .
09 is given in Figure 9. This surface issymmetric w.r.t. the diagonal.
We recall the notion of an x-strategy given in the Introduction: let U i , i = 1 , , . . . , n be independentrandom variables uniformly distributed on the interval [0 , T i = U { i } : T i is the i th order statisticof the U i ’s. T i is the arrival time of X i . The strategy is to wait until some time x ∗ n and from x ∗ n on,we select the first X i = +1 or X i = −
1, using the previous algorithm with p = p (cid:48) . Following Bruss [3],we call this strategy an x-strategy. In [10], the author gives, for this problem, the optimal x ∗ n and thecorresponding success probability P ∗ n . In this Section, we analyze accordingly asymptotic expansionsfor p = p (cid:48) . We also consider the success probability for small p , and also the case p > p (cid:48) .9 Figure 7: C ∗ (vertical line), φ ( C ) (curved line) , p = 0 . , p (cid:48) = 0 . Figure 8: Closer look at Fig. 7, with optimal point (6 , Figure 9: w C,D , C ≤ D and w D,C , C ≥ D , p = p (cid:48) = 0 . p = p (cid:48) Let first recall a few results from [10]. If we denote by (cid:96) the number of observed variables, startingfrom x , we must set, in (9), C = (cid:96) −
1. This leads to the success probability P n ( x, p ) = n (cid:88) (cid:18) n(cid:96) (cid:19) (1 − x ) (cid:96) x n − (cid:96) (cid:16) q (cid:96) − ˜ q (cid:96) (cid:17) = 2 (( q + p x ) n − (2 q − p x ) n ) . The optimal value x ∗ n is solution of dP n ( x,p ) dx = 0, which leads to x ∗ n := − q + 2 β n q − be − q − β n + 2 be q , β n := 2 / ( n − . This gives P ∗ n := P n ( x ∗ n , p ) := 2 (2 2 ( n − ) − (1 − n ) . Notice that P ∗ n is independent of p . Open Problem 1: why is it so? It appears that, for p = ˜ p n , wehave x ∗ n = 0, with ˜ p n = β n − β n − . We can also check that P n (0 , ˜ p n ) = P ∗ n .Let us now turn to the the asymptotic analysis of the case p = p (cid:48) and the corresponding behaviourfor small p .Asymptotically, we obtain, for n → ∞ , x ∗ n = 1 − ln(2) np + 12 ln(2) ( − pn + O (cid:18) n (cid:19) , (10) P ∗ n = 12 + 12 ln(2) n + 14 ln(2) (2 − ) n + O (cid:18) n (cid:19) , p n = ln(2) n + −
12 ln(2) ( − n + O (cid:18) n (cid:19) . (11) P ∗ n converges to 1 / n → ∞ .For instance, P ∗ = 0 . . . . . An interesting question is:what is the behaviour of P ∗ n for p ≤ ˜ p n ? Following (11), we tentatively set q = 1 − y/n, x = 0 in P n ( x, p ). This leads to P n ( y ) = 2 e − y − e − y + − e − y y + 4 e − y y n + 2 e − y ( − y + 18 y ) − e − y (cid:18) − y + 2 y (cid:19) n + O (cid:18) n (cid:19) . In order to check, we put the first term of ˜ p n i.e. y = ln(2) into P n ( y ). Expanding, this leads to the firsttwo terms of P ∗ n . Similarly, putting the first two terms of ˜ p n , i.e. y = ln(2) + −
12 ln(2) ( − n into P n ( y ) gives the first three terms of P ∗ n . p > p (cid:48) This case was not considered before. We can still use the x-strategy, but now we must set D = (cid:96) − D ≥ C ∗ d , we use w C ∗ d ,D and if D ≤ C ∗ d , we use w D,D (we must stay above the diagonal). Thisleads to P ∗ n = n (cid:88) (cid:96) = C ∗ d (cid:18) n(cid:96) (cid:19) (1 − x ) (cid:96) x n − (cid:96) w C ∗ d ,(cid:96) − + C ∗ d (cid:88) l =0 (cid:18) n(cid:96) (cid:19) (1 − x ) (cid:96) x n − (cid:96) w (cid:96) − ,(cid:96) − = n (cid:88) (cid:18) n(cid:96) (cid:19) (1 − x ) (cid:96) x n − (cid:96) w C ∗ d ,(cid:96) − + C ∗ d (cid:88) l =0 (cid:18) n(cid:96) (cid:19) (1 − x ) (cid:96) x n − (cid:96) [ w (cid:96) − ,(cid:96) − − w C ∗ d ,(cid:96) − ] . The first summation leads to S + S , with S := (cid:18) (1 − x ) q (cid:48) x + 1 (cid:19) n x n ( − C ∗ d − p (cid:48) q (cid:48) + (cid:18) (1 − x ) q (cid:48) x + 1 (cid:19) n (1 − x ) q (cid:48) n (cid:18) − x n C ∗ d p (cid:48) q (cid:48) − x n ( − C ∗ d − p (cid:48) q (cid:48) (cid:19) x (cid:18) (1 − x ) (1 − p (cid:48) ) x + 1 (cid:19) ,S := (cid:18) (1 − x ) q (cid:48) x + 1 (cid:19) n x n q (cid:48)− C ∗ d − (cid:18) p ( q C ∗ d +1 − ˜ q C ∗ d +1 ) p (cid:48) + p (cid:48) ( q (cid:48) C ∗ d +1 − ˜ q C ∗ d +1 ) p (cid:19) . The second summation leads to a complicated expression, involving binomials and hypergeometricterms that we do not display here. However, if we plug in numerical values, for instance p = 0 . , p (cid:48) =0 . , n = 40 , C ∗ d = 7, we obtain a tractable function P ( x ) that we can differantiate, leading to x ∗ = 0 . . . . . This gives P ( x ∗ ) = 0 . . . . .12 The x-strategy with incomplete information
Bruss suggested to analyze this strategy because incomplete information has an increased appeal forapplications.We will only consider the case p = p (cid:48) . The other cases can similarly analyzed, with more com-plicated algebra. We will consider the cases p known, n unknown, then n known, p unknown andfinally n, p unknown. Some simulations are also provided. In all our numerical expressions, we willuse n = 500 , p = 0 . p known, n unknown We will always denote by m the number of observed variables up to time x and by k the number of { +1 , − } observed variables up to time x . From (10), we have x ∗ n ∼ − ln(2) np and we will use thenatural estimate ˜ n = mx . Hence we start from the formal equation resulting from (10), hence x = 1 − x ln(2) mp , from which we deduce the two functions x = g ( m, p ) = mpmp + ln(2) ,m = f ( x, p ) = ln(2) xp (1 − x ) . Our algorithm proceeds as follows: wait until m crosses the function f ( x, p ) at value m ∗ . It followsfrom Bruss and Yor [9] ,Thm 5.1 that all optimal actions are confined to the interval [ x ,
1] for some x < x equals 1 / x ∗ = g ( m ∗ , p ). Wewill use this value in the x-strategy. First of all we notice that, asymptotically, m corresponds to aBrownian bridge of order √ n with a drift nx . On the other side, f (cid:48) ( x ∗ n ) ∼ pn / ln(2). Hence, withhigh probability, m crosses f ( x, p ) only once in the neighbourhood of x ∗ n . Let G ( n, m, x ) := (cid:18) nm (cid:19) x m (1 − x ) n − m be the distribution of m at time x . We have ϕ ( n, µ, p ) := P ( m ∗ = µ ) ∼ G ( n, µ, g ( µ, p )) , and using P eq ( (cid:96), p ) := 2[ q (cid:96) − ˜ q (cid:96) ] , we obtain the success probability P ( n, p ) = n (cid:88) ϕ ( n, µ, p ) P eq ( n − µ, p ) . For instance, we show in Figure 10 an illustration of a typical crossing and in Figure 11, the function ϕ ( n, µ, p ) (line) together with G ( n, µ, x ∗ n ) (circles) (the classical x-strategy µ distribution ).The distributions are quite similar. Open Problem 2: why? We obtain P ( n, p ) ∼ . . . . (Inthe numerical summations, we sum µ from some value ˜ µ to avoid any problems near the origin)13 Figure 10: The case p known, n unknown: a typical crossing because it occurs close to 1 Figure 11: The case p known, n unknown: ϕ ( n, µ, p ) (line) , with G ( n, µ, x ∗ n ) (circles)14 .2 The case n known, p unknown Now we use the following estimate for p : ˜ p = k/ (2 m ). The formal starting equation is x = 1 − ln(2) np . Hence the two functions x = u ( n, p ) = 1 − ln(2) np ,p = h ( n, x ) = ln(2) n (1 − x ) . The algorithm waits until ˜ p crosses function h ( n, x ) at value p ∗ , giving a value x ∗ = u ( n, p ∗ ). Again,with high probability, ˜ p crosses h ( n, x ) only once in the neighbourhood of x ∗ n . The joint distributionof m, k at time x is given, with k ≤ m by H ( n, m, k, x, p ) = G ( n, m, x ) (cid:18) mk (cid:19) (2 p ) k (1 − p ) m − k . The joint distribution of m = µ, k given that ˜ p has just crossed h ( n, x ) is given byΠ( n, µ, k, p ) ∼ H ( n, µ, k, u ( n, ˜ p ) , p ) . We have ϕ ( n, µ, p ) := P ( m ∗ = µ ) ∼ µ (cid:88) k =1 Π( n, µ, k, p ) , and finally the success probability is given by P ( n, p ) = n (cid:88) ϕ ( n, µ, p ) P eq ( n − µ, p ) . As an example, we show in Figure 12 the function ϕ ( n, µ, p ). Also P ( n, p ) ∼ . . . . n, p unknown The estimates are now ˜ p = k/ (2 m ) , ˜ n = mx . This leads to formal starting equation x = 1 − x ln(2) k . Hence the two functions x = v ( k ) = kk + 2 ln(2) ,k = w ( x ) = 2 ln(2) x (1 − x ) . The algorithm waits until k crosses function w ( x ) at value k ∗ , giving a value x ∗ = v ( k ∗ ). Again,with high probability, k crosses w ( x ) only once in the neighbourhood of x ∗ n . The joint distribution of m = µ, k given that k has just crossed w ( x ) is given byΠ( n, µ, k, p ) ∼ H ( n, µ, k, v ( k ) , p ) . We have ϕ ( n, µ, p ) := P ( m ∗ = µ ) ∼ µ (cid:88) k =1 Π( n, µ, k, p ) , Figure 12: The case n known, p unknown: ϕ ( n, µ, p )and finally the success probability is given by P ( n, p ) = n (cid:88) ϕ ( n, µ, p ) P eq ( n − µ, p ) . For instance, we show in Figure 13 the function ϕ ( n, µ, p ) together with the corresponding distribu-tion in the the case n known, p unknown (circles). Curiously enough, the distributions are quite similarbut different from the case p known, n unknown. Open Problem 3: why? Also P ( n, p ) ∼ . . . . . We have made three simulations of the crossing value µ distribution compared with ϕ ( n, µ, p ). Eachtime we made 500 simulated paths. For the case p known, n unknown, a typical path is given in Figure14 and, in Figure 15 , we show the empirical observed distribution, together with ϕ ( n, µ, p ) ( For thepurpose of smoothing, we have grouped two successive observed probabilities together). Numerically,this gives P sim ( n, p ) = 0 . . . . .Similarly, for the case n known, p unknown, a typical path is given in Figure 16 and, in Figure17 , we show the empirical observed distribution, together with ϕ ( n, µ, p ). Numerically, this gives P sim ( n, p ) = 0 . . . . .For the case n, p unknown, a typical path is given in Figure 18 and, in Figure 19 , we showthe empirical observed distribution, together with ϕ ( n, µ, p ). Numerically, this gives P sim ( n, p ) =0 . . . . .All fits are satisfactory. 16 Figure 13: The case n, p unknown: ϕ ( n, µ, p ) (line) together with the corresponding distribution inthe the case n known, p unknown (circles) Figure 14: The case p known, n unknown: a typical path17 Figure 15: The case p known, n unknown: the empirical observed distribution, together with ϕ ( n, µ, p ) Figure 16: The case n known, p unknown: a typical path18 Figure 17: The case n known, p unknown: the empirical observed distribution, together with ϕ ( n, µ, p ) Figure 18: The case n, p unknown: a typical path19
Figure 19: The case n, p unknown: the empirical observed distribution, together with ϕ ( n, µ, p ) Using a continuous model, some asymptotic expansions and an incomplete information strategy, wehave obtained a refined and asymptotic analysis of the extended Weber problem and several versionsof Bruss-Weber problems. Three problems remain open: why is P ∗ n independent of p ? Can we justifythe similarities in the distributions of the crossing value m ∗ ? An interesting problem would be toconsider the case with several values {− k, − ( k − , . . . , − , , , . . . , k } with corresponding stoppingtimes. If moreover values can be associated with relative ranks, such problems (Bruss calls them“ basket ” problems ) are partially studied in Dendievel [11]. γ ( p ) Some numerical experiments show that, for C ∗ near − γ ( p ) is very close to p (cid:48) = 1 − p , and that novalue C ∗ < − γ ( p ) for C ∗ near − • for p near 1, we set p (cid:48) = w . For w = 0, φ ( C ) is identically 0. So we expand (5) near w = 0 andkeep the w term. This gives p (1 − p ) C ∗ (1 + C ∗ ln(1 − p ) + ln(1 − p )) = 0 . Setting p = 1 − ξ, C ∗ = − η , we obtain( − ξ ) ξ − η (1 + η ln( ξ )) = 0 , hence η ( ξ ) ∼ − / ln( ξ ) , ξ → ,ξ ( η ) ∼ exp( − /η ) , η → . For instance, for C ∗ = − .
09 we have (ˆ x always denotes some solution of (5))ˆ ξ = 0 . . . . and η ( ˆ ξ ) = 0 . . . . , ξ (0 .
09) = 0 . . . . .20 on the diagonal p (cid:48) = p , we set p = p (cid:48) = 1 / − ε, C ∗ = − η . From (8), expand w.r.t. ξ , weobtain C ∗ ∼ ln( −
16 ln(2) − ε )) + ln( ε ) − − ln( ε ) ∼ − − ln(2) + ln( − ln( ε ))ln( ε ) , hence η ( ξ ) ∼ − ln(2) + ln( − ln( ε ))ln( ε ) , ε → . To obtain ε as a function of η , we set A := − ln( ε ). We derive, to first order,ln(2) + ln( A ) − ηA = 0 ,A exp( − ηA ) = 1 / , − ηA exp( − ηA ) = − η/ , − ηA = W − ( − η/ ,A = − W − ( − η/ /η, for − η/ > − /e = − . . . . ,ε ( η ) ∼ exp( W − ( − η/ /η ) , η → , where W ( x ) is the Lambert-W function and the lower branch has W ≤ − W − ( x ). It decreases from W − ( − /e ) = − W − (0) = −∞ . For instance, for ε = 10 − , ˆ η =0 . . . . and η (ˆ ε ) = . . . . , ε (ˆ η ) = 4 . − .Now W − ( x ) ∼ ln( − x ) , x ↑
0. Hence ε ( η ) ∼ exp(ln( η/ /η ) , η → . • in the neighbourhood of p (cid:48) = 1 − p , we set p (cid:48) = 1 − p − δ, C ∗ = − η . Hence ˜ q = δ, q (cid:48) = p − δ .As δ →
0, we have p (cid:48) ∼ − p, q (cid:48) ∼ p . So we expand (5) to first order. We obtain C δ η + C q η + C ( p − δ ) − η = 0 , with C = C + C ln( δ ) , C = ( p +(1 − p ) ) ln( p ) , C = − p − (1 − p ) , C = − p ( − ln( q )+ln( p )) , C = − (1 − p ) p. This leads to η ( δ ) ∼ ln( C ) − ln(ln( δ ))ln( δ ) , δ → ,C = − /pq ) p − /pq ) p + ln(1 /pq ) p p (2 p + 1 − p )( p − . Setting B := − ln( δ ) , C = − C , this leads to Be − ηB ∼ C , − ηBe − ηB ∼ − ηC , − ηB ∼ W − ( − ηC ) ,B ∼ − W − ( − ηC ) /η,δ ( η ) ∼ exp(ln( ηC ) /η ) , η → ,η ( δ ) ∼ (ln( B ) − ln( C )) /B, δ → . For instance, for p = 0 . , η = 0 . p (cid:48) = 0 . − . . . . − , ˆ δ = . . . . − , C = − . . . . and η (ˆ δ ) = . . . . . − ln( δ ) = 139 . . . is not large enough, compared with C = 34 . . . in order to use η (ˆ δ ). However, ln( B ) /B = . ηC = 1 . . . . , which is too large( > /e ) in our case for allowing using − W − ( − ηC ) /η .21 cknowledgement. We would like to thank F.T. Bruss for many illuminating discussions.
References [1] K. Ano and M. Ando. A note on Bruss’ stopping problem with random availability. In
Papers inhonor of Thomas S. Ferguson, IMS Lectures Notes- Monograph Series , volume 35, pages 71–82.2000.[2] D. Assaf and E. Samuel-Cahn. Simple ratio prophet inequalities for a mortal with multiplechoices.
Journal of Applied Probability , 37(4):1084–1091, 2000.[3] F.T. Bruss. A unified approach to a class of best choice problems with an unknown number ofoptions.
Annals of Probability , 12(3):882–889, 1984.[4] F.T. Bruss. On an optimal selection problem of Cowan and Zabczyk.
Journal of Applied Proba-bility , 24(4):918–928, 1987.[5] F.T. Bruss. Sum the odds to one and stop.
Annals of Probability , 28(3):1384–1391, 2000.[6] F.T. Bruss. A note on bounds for the odds-theorem of optimal stopping.
Annals of Probability ,31(4):1859–1861, 2003.[7] F.T. Bruss and T. S. Ferguson. High-risk and competitive investment models.
Annals of AppliedProbability , 12(4):1202–1226, 2002.[8] F.T. Bruss and G. Louchard. The odds-algorithm based on sequential updating and its perfor-mance.
Advances in Applied Probability , 41:131–153, 2009.[9] F.T. Bruss and M. Yor. Stochastic processes with proportional increments and the last arrivalproblem.
Stochastic Processes and Their Applications , 122(9):3239–3261, 2012.[10] R. Dendievel. Weber’s optimal stopping problem and generalizations.
Statistics and ProbabilityLetters , 97:176–184, 2015.[11] R. Dendievel. Sequential stopping under different environments of weak information. Ph.D. dis-sertation. Technical report, Universit´e Libre de Bruxelles, 2016.[12] S.R. Hsiau and J.R. Yang. Selecting the last success in Markov-dependent trials.
Journal ofApplied Probability , 39(2):271–281, 2002.[13] A. Kurishima and K. Ano. Multiple stopping odds problem in Bernoulli trials with randomnumber of observations.
Mathematica Applicanda , 44(1):209–220, 2016.[14] T. Matsui and K. Ano. Lower bounds for Bruss’ odds problem with multiple stoppings.
Mathe-matics of Operations Research , 41(2):700–714, 2016.[15] K. Szajowski and D. (cid:32)Lebek. Optimal strategies in high risk investments.
Bulletin of the BelgianMathematical Society-Simon Stevin , 14(1):143–155, 2007.[16] M. Tamaki. Sum the multiplicative odds to one and stop.
Journal of Applied Probability ,47(3):761–777, 2010.[17] R.R. Weber. Optimization and control, Section 6.