Equilibrium concepts for time-inconsistent stopping problems in continuous time
aa r X i v : . [ q -f i n . M F ] S e p ON THE NOTIONS OF EQUILIBRIA FOR TIME-INCONSISTENT STOPPINGPROBLEMS IN CONTINUOUS TIME
ERHAN BAYRAKTAR, JINGJIE ZHANG, AND ZHOU ZHOU
Abstract. A new notion of equilibrium, which we call strong equilibrium , is introduced for time-inconsistent stopping problems in continuous time. Compared to the existing notions introducedin [4] and [3], which in this paper are called mild equilibrium and weak equilibrium respectively, astrong equilibrium captures the idea of subgame perfect Nash equilibrium more accurately. Whenthe state process is a continuous-time Markov chain and the discount function is log sub-additive,we show that an optimal mild equilibrium is always a strong equilibrium. Moreover, we provide anew iteration method that can directly construct an optimal mild equilibrium and thus also proveits existence. Introduction
On a filtered probability space (Ω , F , ( F t ) t ∈ [0 , ∞ ) , P ) consider an optimal stopping problem incontinuous time sup τ ∈T E x [ δ ( τ ) X τ ] , (1.1)where X = ( X t ) t ∈ [0 , ∞ ) is a time-homogeneous Markov process taking values in some space X ⊂ R , T is a set of stopping times, δ is a discount function, and E x is the expectation given X = x . Itis well known that when δ is not exponential, the problem (1.1) may be time-inconsistent. Thatis, the optimal stopping strategy obtained today may not be optimal in the eyes of future selves.One way to approach such time inconsistency is consistent planning (see [10]), which is formulatedas a subgame perfect Nash equilibrium: once an equilibrium strategy is enforced over the planninghorizon, the current self has no incentive to deviate from it, given all future selves will follow theequilibrium strategy.There are two general notions of equilibrium stopping strategies in continuous time in the liter-ature. The first notion is proposed in [4] and further studied in [5, 7, 6], which we will call mildequilibrium in this paper. Following [4, Definition 3.3] and [7, Definition 2.2], we have the followingdefinition of mild equilibrium. Definition 1.1.
A measurable set S ⊂ X is said to be a mild equilibrium, if (cid:26) x ≤ E x [ δ ( τ S ) X τ S ] , ∀ x / ∈ S, (1.2) x ≥ E x [ δ ( τ + S ) X τ S ] , ∀ x ∈ S, (1.3) where τ S := inf { t ≥ X t ∈ S } , and τ + S := inf { t > X t ∈ S } . (1.4) Key words and phrases.
Time-inconsistency, optimal stopping, strong equilibria, weak equilibria, mild equilibria,non-exponential discounting, subgame perfect Nash equilibrium.E. Bayraktar is supported in part by the National Science Foundation under grant DMS-1613170 and by the SusanM. Smith Professorship.
In the above S is the stopping region, and the economic interpretation for Definition 1.1 is clear,there is no incentive to deviate. That is, in (1.2) when x / ∈ S , it is better to continue and get E x [ δ ( τ S ) X τ S ], rather than to stop and get x ; on the surface a similar statement applies to (1.3).However, when the time of return for X is 0 (i.e., P ( τ { x } = 0 | X = x ) = 1), which is satisfied forcontinuous-time Markov chain and many one-dimension diffusion processes, τ S = τ + S and thus (1.3)trivially holds. In other words, when the time of return is 0, there is no actual deviation capturedby (1.3) from stopping to continuing, and Definition 1.1 is equivalent to the following. Definition 1.2.
A measurable set S ⊂ X is said to be a mild equilibrium, if x ≤ E x [ δ ( τ S ) X τ S ] =: J ( x, S ) , ∀ x / ∈ S. (1.5)Consequently, with the time of return being 0 the notion of mild equilibrium cannot fully capturethe economic meaning of equilibrium. It is easy to see that S = X is always a mild equilibrium,and it is not clear why always stopping immediately is a reasonable strategy.As can be seen from [4, 5, 7, 6], there is often a continuum of mild equilibria in many naturalmodels, which naturally leads to the question of equilibrium selection. In [7], optimal mild equilib-rium in the sense of point-wise dominance is considered. In particular, from [7, Definition 2.3] wehave the following definition. Definition 1.3.
A mild equilibrium S ∗ is said to be optimal, if for any other mild equilibrium S , x ∨ J ( x, S ∗ ) ≥ x ∨ J ( x, S )( ⇐⇒ J ( x, S ∗ ) ≥ J ( x, S )) , ∀ x ∈ X . Note that x ∨ J ( x, S ) represents the value associated with the stopping region/strategy S . In [7]the existence of optimal equilibrium is established. A discrete-time version is in [8].The second notion of equilibrium, which we call weak equilibrium in this paper, is proposed in[3] and further investigated in [2]. Following [3], we have the definition of weak equilibrium (weadapt the definition slightly for our setting). Definition 1.4.
A measurable set S ⊂ X is said to be a weak equilibrium, if x ≤ E x [ δ ( τ S ) X τ S ] , ∀ x / ∈ S, lim inf ε ց x − E x [ δ ( τ εS ) X τ εS ] ε ≥ , ∀ x ∈ S, (1.6) where τ εS = inf { t ≥ ε : X t ∈ S } . (1.7)Compared to (1.3), the first-order condition (1.6) does capture the deviation from stopping tocontinuing. However, similar to that for time-inconsistent control (see e.g., [1, Remark 3.5] and[9]), the first-order criterion does not correspond to the equilibrium concept perfectly: when thelimit in (1.6) equals zero, it is possible that for all ε > , x < E x [ δ ( τ εS ) X τ εS ], in which case there isan incentive to deviate.To sum up, the economic interpretation of being “equilibrium” for mild and weak ones is inad-equate. To resolve this issue, we introduce the following concept of strong equilibrium, which isinspired by [9]. Definition 1.5.
A measurable set S ⊂ X is said to be a strong equilibrium, if (cid:26) x ≤ E x [ δ ( τ S ) X τ S ] , ∀ x / ∈ S, ∃ h ( x ) > , s.t. ∀ ε ∈ (0 , h ( x )) , x ≥ E x [ δ ( τ εS ) X τ εS ] , ∀ x ∈ S. (1.8) Compared to (1.3) and (1.6), condition (1.8) not only captures the deviation from stopping tocontinuing, but also more precisely indicates the disincentive of such deviation. Consequently, astrong equilibrium delivers better economic meaning as being an “equilibrium”.In this paper, when X is a Markov chain we show that an optimal mild equilibrium is a strongequilibrium (see Theorem 2.1). (Obviously, a strong equilibrium is also weak, and a weak equilib-rium is also mild.) We also provide examples showing that a strong equilibrium may not be anoptimal mild equilibrium, and a weak equilibrium may not be strong. Therefore, we thoroughlyobtain the relation between mild, weak, strong, and optimal mild (and thus optimal weak, optimalstrong) equilibria. Moreover, we provide a new iteration method which directly constructs an opti-mal mild equilibrium and thus also establish its existence (see Thoerem 2.2). In [8, 7], an optimalequilibrium is constructed by the intersection of all (mild) equilibriums. In principle, this requiresus to first find all (mild) equilibria in order to get the optimal one, which may not be implementablein many cases. The new iteration method proposed in this paper is much easier and more efficientto implement. It would be interesting to see whether such results can be extended to diffusionmodels, which we will leave for future research.The rest of the paper is organized as follows. Section 2 collects the main results of the paper.An optimal mild equilibrium is proved to be a strong equilibrium, and can be directly constructedvia a new iteration method. Section 3 focuses on a concrete two-state model, which demonstratesthe differences between these equilibria.2. The Main Results
In this section, we apply the concepts in Section 1 to a continuous-time Markov chain and presentour main results under this setting. Let X = ( X t ) t ≥ be a time-homogeneous continuous-timeMarkov chain. It has a finite or countably infinite state space X ⊂ [0 , ∞ ). Let λ x be the transitionrate out of the state x ∈ X , and q xy be the transition rate from state x to y for y = x . Then wehave that λ x = P y = x q xy . The discount function t δ ( t ) is assumed to be non-exponential anddecreasing, with δ (0) = 1 and lim t →∞ δ ( t ) = 0. Let the filtration ( F t ) t ∈ [0 , ∞ ) be generated by X .Furthermore, we make the following assumptions on X and δ ( · ). Assumption 2.1. (i) C := sup X < ∞ and λ := sup x ∈ X λ x < ∞ .(ii) X is irreducible, i.e., for any x, y ∈ X , inf { t ≥ X t = y | X = x } < ∞ , a.s.. Assumption 2.2. (i) δ is log-subadditive, i.e., δ ( s ) δ ( t ) ≤ δ ( s + t ) , ∀ s, t > . (2.1) (ii) t δ ( t ) is differentiable at t = 0 , and δ ′ (0) < . Remark 2.1.
Assumption 2.2 (i) is closely related to decreasing impatience in Behavioral Eco-nomics and Finance and commonly used when studying non-exponential discounting problems; seee.g., [8, 7, 4] . The following is the first main result of this paper, which shows that an optimal mild equilibriumis a strong equilibrium. The proof is provided in Section 2.1.
Theorem 2.1.
Let Assumptions 2.1 and 2.2 hold. If S is an optimal mild equilibrium, then it isa strong equilibrium. ERHAN BAYRAKTAR, JINGJIE ZHANG, AND ZHOU ZHOU
Since all mild equilibria are strong equilibria, an optimal mild equilibrium will generate largervalues than any strong equilibrium as well. With Theorem 2.1, we can conclude that any optimalmild equilibrium is a strong equilibrium and in fact is an optimal strong equilibrium.The following is the second main result of this paper. It provides an iteration method whichdirectly constructs an optimal mild equilibrium, and thus also establishes the existence of weak,strong, and optimal mild equilibria. The proof of this result is presented in Section 2.2.
Theorem 2.2.
Let S := ∅ , and S n +1 := S n ∪ ( x ∈ X \ S n : x > sup S : S n ⊂ S ⊂ X \{ x } J ( x, S ) ) . (2.2) Let S ∞ := ∪ ∞ n =0 S n . (2.3) If Assumptions 2.1 (i) and 2.2 (i) hold, then S ∞ is an optimal mild equilibrium. If in additionAssumption 2.2 (ii) holds, then S ∞ is a strong equilibrium. Proof of Theorem 2.1.
Recall τ S , τ εS , J ( · , · ) defined in (1.4),(1.7),(1.5) respectively. We havethe following characterization of (1.6) in Definition 1.2. Proposition 2.1.
Let Assumptions 2.1 and 2.2 (ii) hold. Then S ⊂ X is a weak equilibrium if andonly if S is a mild equilibrium and for all x ∈ S , x ( λ x − δ ′ (0)) ≥ X y ∈ S \{ x } yq xy + X y ∈ S c J ( y, S ) q xy . Proof.
By definition, we only need to check condition (1.6) in Definition 1.2 is equivalent to theabove inequality.Denote T x := inf { t ≥ X t = x, X = x } as the holding time at state x , which has exponentialdistribution with parameter λ x . Then E x [ δ ( τ εS ) X τ εS ] = E x [ δ ( τ εS ) X τ εS { T x >ε } ] + X y ∈ X \{ x } E x [ δ ( τ εS ) X τ εS { T x ≤ ε,X Tx = y,T y + T x >ε } ] + O ( ε )= δ ( ε ) xe − λ x ε + X y ∈ S \{ x } δ ( ε ) y q xy λ x + X y ∈ S c E y [ δ ( ε + τ S ) X τ S ] q xy λ x ( λ x ε + O ( ε )) + O ( ε ) . Notice that δ ( ε ) = 1 + δ ′ (0) ε + O ( ε ). Therefore we have E x [ δ ( τ εS ) X τ εS ] = x + − x ( λ x − δ ′ (0)) + X y ∈ S \{ x } yq xy + X y ∈ S c q xy E y [ δ ( ε + τ S ) X τ S ] ε + o ( ε ) . Therefore, (1.6) is equivalent to x ( λ x − δ ′ (0)) ≥ X y ∈ S \{ x } yq xy + X y ∈ S c E y [ δ ( τ S ) X τ S ] q xy . (cid:3) Corollary 2.1.
Let Assumptions 2.1 and 2.2 (ii) hold. If S is a mild equilibrium and satisfies x ( λ x − δ ′ (0)) > X y ∈ S \{ x } yq xy + X y ∈ S c E y [ δ ( τ S ) X τ S ] q xy , then it is a strong equilibrium. For the rest of the paper, we will sometimes use the notation ρ ( x, S ) := inf { t ≥ X xt ∈ S } in the place of τ S to emphasize the initial state X = x ( X x here is the Markov chain starting at x ). Lemma 2.1.
Let Assumption 2.2 (i) hold. For x ∈ S , denote ˆ S = S \{ x } . If S is an optimal mildequilibrium, then for any y / ∈ S , J ( y, ˆ S ) − J ( y, S ) ≥ E y [ δ ( τ S ) { X τS = x } ]( J ( x, ˆ S ) − x ) . Proof.
Since ˆ S ⊂ S , we have ρ ( y, S ) ≤ ρ ( y, ˆ S ). Then J ( y, ˆ S ) − J ( y, S )= E y [ δ ( ρ ( y, ˆ S )) X ρ ( y, ˆ S ) { X ρ ( y,S ) = x } ] + E y [ δ ( ρ ( y, ˆ S )) X ρ ( y, ˆ S ) { X ρ ( y,S ) ∈ ˆ S } ] − E y [ δ ( ρ ( y, S )) X ρ ( y,S ) ]= E y [ δ ( ρ ( y, ˆ S )) X ρ ( y, ˆ S ) { X ρ ( y,S ) = x } ] + E y [ δ ( ρ ( y, S )) X ρ ( y,S ) { X ρ ( y,S ) ∈ ˆ S } ] − E y [ δ ( ρ ( y, S )) X ρ ( y,S ) ]= E y [ δ ( ρ ( y, ˆ S )) X ρ ( y, ˆ S ) { X ρ ( y,S ) = x } ] − x E y [ δ ( ρ ( y, S )) { X ρ ( y,S ) = x } ] ≥ E y [ δ ( ρ ( y, S )) { X ρ ( y,S ) = x } E [ δ ( ρ ( y, ˆ S ) − ρ ( y, S )) X ρ ( x, ˆ S ) |F ρ ( y,S ) ]] − x E y [ δ ( ρ ( y, S )) { X ρ ( y,S ) = x } ]= E y [ δ ( τ S ) { X τS = x } ]( E x [ δ ( τ ˆ S ) X τ ˆ S ] − x ) , where we use (2.1) for the inequality above. (cid:3) Lemma 2.2.
Let Assumption 2.2 (i) hold. If S is an optimal mild equilibrium, then for any x ∈ S we have that x ≥ J ( x, ˆ S ) , where ˆ S = S \{ x } . As a result, / ∈ S and J ( y, S ) > for all y ∈ X .Proof. If ˆ S is also a mild equilibrium, then x ≤ J ( x, ˆ S ) ≤ J ( x, S ) = x, and thus x = J ( x, ˆ S ).If ˆ S is not a mild equilibrium, then there exists y / ∈ ˆ S such that J ( y, ˆ S ) < y ≤ J ( y, S ). ByLemma 2.1, 0 > J ( y, ˆ S ) − J ( y, S ) ≥ E y [ δ ( τ S ) I { X τS = x } ]( J ( x, ˆ S ) − x ) , which implies that x > J ( x, ˆ S ) . (2.4)Now suppose 0 ∈ S . By the above result, we have 0 ≥ J (0 , S \{ } ). Since X τ S \{ } > J (0 , S \{ } ) >
0, which is a contraction. As a result, 0 / ∈ S and J ( y, S ) > y ∈ X . (cid:3) Proof of Theorem 2.1 . By Assumption 2.2, δ ( t ) ≥ e δ ′ (0) t for all t ≥
0. Moreover, there exist t > t > t , δ ( t ) > e δ ′ (0) t since δ is non-exponential. As a result, for any x ∈ X , E x [ δ ( T x )] = Z ∞ δ ( t ) e − λ x t dt > Z ∞ λ x e ( δ ′ (0) − λ x ) t dt = λ x λ x − δ ′ (0) . Denote c x := λ x λ x − δ ′ (0) . ERHAN BAYRAKTAR, JINGJIE ZHANG, AND ZHOU ZHOU If S = { x } , then X y = x J ( y, S ) q xy ≤ x X x = y E y [ δ ( T y )] q xy < xλ x < x ( λ x − δ ′ (0)) , which implies that S is a strong equilibrium.For the rest of the proof, we assume S contains at least two points. Fix any x ∈ S , we have J ( x, ˆ S ) = X y ∈ S \{ x } q xy λ x E x [ δ ( τ ˆ S ) X τ ˆ S | X T x = y ] + X y / ∈ S q xy λ x E x [ δ ( τ ˆ S ) X τ ˆ S | X T x = y ] . Since for y ∈ S \ { x } , E x [ δ ( τ ˆ S ) X τ ˆ S | X T x = y ] = y E x [ δ ( τ ˆ S ) | X T x = y ] = y E x [ δ ( T x ) | X T x = y ] = y E x [ δ ( T x )] , and for y ∈ S c , E x [ δ ( τ ˆ S ) X τ ˆ S | X T x = y ] ≥ E x [ δ ( T x ) δ ( τ ˆ S − T x ) X τ ˆ S | X T x = y ]= E x [ δ ( T x ) | X T x = y ] · E x [ δ ( τ ˆ S − T x ) X τ ˆ S | X T x = y ] = E x [ δ ( T x )] · J ( y, ˆ S ) , we have that J ( x, ˆ S ) ≥ X y ∈ S \{ x } q xy λ x y + X y / ∈ S q xy λ x J ( y, ˆ S ) · E x [ δ ( T x )] . (2.5)Denote I := X y ∈ S \{ x } q xy λ x y, II := X y / ∈ S q xy λ x J ( y, S ) , ˆII := X y / ∈ S q xy λ x J ( y, ˆ S ) . By Lemma 2.2, y > y ∈ ˆ S and J ( y, ˆ S ) > y / ∈ ˆ S , thus I + ˆII >
0. This togetherwith E x [ δ ( T x )] > c x implies that J ( x, ˆ S ) > (I + ˆII) c x . Then x − J ( x, ˆ S ) < x − (I + ˆII) c x = x − (I + II) c x + (II − ˆII) c x = x − (I + II) c x + c x X y / ∈ S q xy λ x ( J ( y, S ) − J ( y, ˆ S )) ≤ x − (I + II) c x + c x X y / ∈ S q xy λ x ( E y [ δ ( τ S ) { X τS = x } ]( x − J ( x, ˆ S )) , where the last line follows from Lemma 2.1. Thus − c x X y / ∈ S q xy λ x ( E y [ δ ( τ S ) { X τS = x } ]) ( x − J ( x, ˆ S )) < x − (I + II) c x . (2.6)Notice that c x X y / ∈ S q xy λ x ( E y [ δ ( τ S ) { X τS = x } ] ≤ c x X y / ∈ S q xy λ x ≤ c x < . Then by Lemma 2.2, x − (I + II) c x > , ∀ x ∈ X , which implies S is a strong equilibrium. (cid:3) Proof of Theorem 2.2.
We start with the following lemma, which in particular indicatesthat a smaller mild equilibrium generates larger values.
Lemma 2.3.
Let Assumption 2.2 (i) hold. If S is a mild equilibrium, then for any subset R ⊂ X with S ⊂ R , we have J ( x, S ) ≥ J ( x, R ) , ∀ x ∈ X . Proof.
Since S ⊂ R , ρ ( x, S ) ≥ ρ ( x, R ) for all x ∈ X . J ( x, S ) = E x [ δ ( ρ ( x, S )) X ρ ( x,S ) ]= E x [ E x [ δ ( ρ ( x, S )) X ρ ( x,S ) |F ρ ( x,R ) ]] ≥ E x [ δ ( ρ ( x, R )) E x [ δ ( ρ ( x, S ) − ρ ( x, R )) X ρ ( x,S ) |F ρ ( x,R ) ]]= E x [ δ ( ρ ( x, R )) E X ρ ( x,R ) [ δ ( ρ ( X ρ ( x,R ) , S )) X ρ ( x,S ) ]] ≥ E x [ δ ( ρ ( x, R )) X ρ ( x,R ) ] = J ( x, R ) . The last inequality holds because S is a mild equilibrium and by definition, E X ρ ( x,R ) [ δ ( ρ ( X ρ ( x,R ) , S )) X ρ ( x,S ) ] ≥ X ρ ( x,R ) . (cid:3) Corollary 2.2.
Let Assumption 2.2 (i) hold. If S is the smallest mild equilibrium, i.e. S ⊂ e S forany mild equilibrium e S , then S is an optimal mild equilibrium. Thanks to this corollary, in order to show S ∞ defined in (2.3) is an optimal mild equilibrium, itsuffices to show that S ∞ is the smallest one.Recall S n defined in (2.2). We have the following lemma. Lemma 2.4.
For any mild equilibrium R , we have that S n ⊂ R for all n ∈ N .Proof. We prove this lemma by induction. First S ⊂ R . Suppose S n ⊂ R for n ≥
0. Since R is amild equilibrium, for any x / ∈ R , x ≤ J ( x, R ) ≤ sup S : S n ⊂ S ⊂ X \{ x } J ( x, S ) . Therefore x / ∈ S n +1 . As a result, S n +1 ⊂ R for all n ∈ N . (cid:3) Lemma 2.5.
Let Assumption 2.1 (i) hold. For y / ∈ S ∞ , denote V n := sup S : S n ⊂ S ⊂ X \{ y } J ( y, S ) , V ∞ := sup S : S ∞ ⊂ S ⊂ X \{ y } J ( y, S ) , then we have V n ց V ∞ , n → ∞ .Proof. Since S ∞ = S n ≥ S n , we have ρ ( y, S ∞ \ S n ) → ∞ , n → ∞ . Then for any ε >
0, there exists N = N ( ε, y ) such that for n > N , E y [ δ ( τ S ∞ \ S n )] < ε since lim t →∞ δ ( t ) = 0.For any R n such that S n ⊂ R n ⊂ X \{ y } , denote R n := R n S S ∞ , then we have, J ( y, R n ) − J ( y, R n ) = E y [( δ ( τ R n ) X τ Rn − δ ( τ R n ) X τ Rn ) { X τRn ∈ S ∞ \ R n } ] ≤ C E y [ δ ( τ R n ) { X τRn ∈ S ∞ \ R n } ] ≤ C E y [ δ ( τ S ∞ \ R n ) { X τRn ∈ S ∞ \ R n } ] ≤ Cε ERHAN BAYRAKTAR, JINGJIE ZHANG, AND ZHOU ZHOU
Since S ∞ ⊂ R n ⊂ X \{ y } , by definition, J ( y, R n ) ≤ V ∞ . Therefore we have that for any ε > N such that for any n ≥ N , V n = sup R n : S n ⊂ R n ⊂ X \{ y } J ( y, R n ) ≤ V ∞ + Cε.
Clearly S n ⊂ S n +1 implies that V n is non-increasing and V n ≥ V ∞ for all n . This completes theproof that V n ց V ∞ , n → ∞ . (cid:3) Proof of Theorem 2.2 . By Corollary 2.2 and Lemma 2.4, to show that S ∞ is an optimal mildequilibrium, it suffices to show S ∞ is a mild equilibrium.Suppose S ∞ is not a mild equilibrium. Then α := sup x ∈ X { x − J ( x, S ∞ ) } > . For any ε >
0, there exists y / ∈ S ∞ such that y − J ( y, S ∞ ) ≥ α − ε . Since y / ∈ S n for all n ≥
0, wehave y ≤ sup S : S n ⊂ S ⊂ X \{ y } J ( y, S ) , ∀ n ≥ . By Lemma 2.5, y ≤ sup S : S ∞ ⊂ S ⊂ X \{ y } J ( y, S ) . Thus, there exists subset R with S ∞ ⊂ R ⊂ X \{ y } such that y ≤ J ( y, R ) + ε. Then we have J ( y, R ) − J ( y, S ∞ ) ≥ y − ε + α − ε − y = α − ε . Since S ∞ ⊂ R , ρ ( y, S ∞ ) ≥ ρ ( y, R ).It follows that J ( y, R ) − J ( y, S ∞ ) = E y [ δ ( ρ ( y, R )) X ρ ( y,R ) ] − E y [ E y [ δ ( ρ ( y, S ∞ )) X ρ ( y,S ∞ ) |F ρ ( y,R ) ]] ≤ E y [ δ ( ρ ( y, R )) X ρ ( y,R ) ] − E y [ δ ( ρ ( y, R )) E y [ δ ( ρ ( y, S ∞ ) − ρ ( y, R )) X ρ ( y,S ∞ ) |F ρ ( y,R ) ]]= E y [ δ ( ρ ( y, R ))( X ρ ( y,R ) − E X ρ ( y,R ) [ δ ( ρ ( X ρ ( y,R ) , S ∞ )) X ρ ( X ρ ( y,R ) ,S ∞ ) ]) ≤ E y [ δ ( ρ ( y, R ))] α ≤ E y [ δ ( T y )] α. By Assumption 2.2 (i), λ = sup x ∈ X λ x < ∞ and since y / ∈ R , we have 0 < E y [ δ ( T y )] < c < c = R ∞ δ ( t ) λe − λt dt . By choosing 0 < ε ≤ α (1 − c )2 , we obtain a contradiction.Next let us prove S ∞ is a strong equilibrium. If X is irreducible, then S ∞ is a strong equilibriumby Theorem 2.1. In general, following the proof for Proposition 2.1, to show S ∞ is a strongequilibrium, it suffices to show that for any x ∈ S ∞ with λ x > x ( λ x − δ ′ (0)) > X y ∈ S ∞ \{ x } yq xy + X y ∈ S c ∞ E y [ δ ( τ S ) X τ S ] q xy . (2.7)Take x ∈ S ∞ with λ x >
0. Following the argument for (2.5), we have that J ( x, ˆ S ∞ ) ≥ X y ∈ S ∞ \{ x } q xy λ x y + X y / ∈ S ∞ q xy λ x J ( y, ˆ S ∞ ) · E x [ δ ( T x )] , where ˆ S ∞ = S ∞ \ { x } . Using an argument similar to that for (2.6), we have that − c x X y / ∈ S ∞ q xy λ x ( E y [ δ ( τ S ∞ ) { X τS ∞ = x } ]) ( x − J ( x, ˆ S ∞ )) ≤ x − (I ∞ + II ∞ ) c x , where I ∞ := X y ∈ S ∞ \{ x } q xy λ x y and II ∞ := X y / ∈ S ∞ q xy λ x J ( y, S ∞ ) . Since S ∞ is the smallest mild equilibrium, ˆ S ∞ is not a mild equilibrium. Then x > J ( x, ˆ S ∞ ) by(2.4). Therefore, x − (I ∞ + II ∞ ) c x > , which implies (2.7). (cid:3) A Two-state Example
In this section, we will use an example to illustrate that a mild equilibrium may not be a weakequilibrium, a weak equilibrium may not be a strong equilibrium and a strong equilibrium may notbe an optimal mild equilibrium.Consider a two-state continuous-time Markov chain X t ∈ { a, b } for t ≥
0. Assume a > , b > a > b . The generator is Q = (cid:20) − λ a λ a λ b − λ b (cid:21) , where λ a > λ b > { a, b } . Clearly S = ∅ and S = { b } cannot be mild equilibria and S = { a, b } is a mild equilibrium. Next, let’s check when S = { a } is a mild equilibrium.By definition S = { a } is a mild equilibrium if and only if b ≤ a E b [ δ ( T b )] = a Z ∞ δ ( t ) λ b e − λ b t dt. Consider the following cases.(i) If ba = R ∞ δ ( t ) λ b e − λ b t dt <
1, then both { a } and { a, b } are optimal mild equilibria and thusboth are strong equilibria.(ii) If ba < R ∞ δ ( t ) λ b e − λ b t dt <
1, then { a } is the only optimal mild equilibrium, which is also astrong equilibrium. But the mild equilibrium { a, b } may not be a weak equilibrium. For example,when ba < λ b λ b − δ ′ (0) <
1, the second condition for weak equilibrium is violated at state b , thus it isnot a weak equilibrium.(iii) λ a λ a − δ ′ (0) < < ab holds automatically since a > b and δ ′ (0) <
0. If λ b λ b − δ ′ (0) < ba < R ∞ δ ( t ) λ b e − λ b t dt <
1, then { a, b } is not an optimal mild equilibrium, but it is a weak equilibriumand also a strong equilibrium.(iv) If λ b λ b − δ ′ (0) = ba < { a, b } is a weak equilibrium, but it may not be a strong equilibrium, i.e.condition (1.8) on strong equilibrium may not hold at state b . This can be shown by computingthe related term of order ε .Since P ( X ε = a | X = b ) = λ b ε − λ b + λ a λ b ε + o ( ε ) , and P ( X ε = b | X = b ) = 1 − λ b ε + λ b + λ a λ b ε + o ( ε ) , we have b − E b [ δ ( ε ) X ε ]= b − δ ( ε )[ a P ( X ε = a | X = b ) + b P ( X ε = b | X = b )]= b − (1 + δ ′ (0) ε + δ ′′ (0)2 ε + o ( ε ))[ a ( λ b ε − λ b + λ a λ b ε + o ( ε )) + b (1 − λ b ε + λ b + λ a λ b ε + o ( ε ))]= ( bλ b − aλ b − bδ ′ (0)) ε + [ b ( λ b δ ′ (0) − λ b + λ a λ b − δ ′′ (0)2 ) − a ( δ ′ (0) λ b − λ b + λ a λ b ε + o ( ε )Therefore when the first order term and the second order term respectively satisfy b ( λ b − δ ′ (0)) − aλ b = 0 , (3.1)and b ( λ b δ ′ (0) − λ b + λ a λ b − δ ′′ (0)2 ) − a ( δ ′ (0) λ b − λ b + λ a λ b < , (3.2) { a, b } is a weak equilibrium but not a strong equilibrium. Using (3.1), (3.2) can be simplified to λ a + λ b < δ ′′ (0) − δ ′ (0) ) − δ ′ (0) . (3.3)An interesting case is when δ ( t ) = βt . Then (3.3) does not hold: δ ′ (0) = − β and δ ′′ (0) = 2 β .In this case δ ′′ (0) − δ ′ (0) ) − δ ′ (0) = 0, which contradicts λ a + λ b >
0. That means if we have hyperbolicdiscount function, a weak equilibrium is always a strong equilibrium in the two-state setting.But when δ ( t ) = (1 + βt ) − , then it can easily be seen that (3.3) holds: δ ′ (0) = − β , δ ′′ (0) = β implies that when 0 < λ a + λ b < β and ba = λ b λ b + β , { a, b } is a weak equilibrium but not a strongequilibrium. In this case, { a, b } is not an optimal mild equilibrium. References [1] Tomas Bj¨ork, Mariana Khapko, and Agatha Murgoci. On time-inconsistent stochastic control in continuous time.
Finance and Stochastics , 21(2):331–360, Apr 2017.[2] S¨oren Christensen and Kristoffer Lindensj¨o. On time-inconsistent stopping problems and mixed strategy stoppingtimes.
Stochastic Processes and their Applications , (Forthcoming).[3] S¨oren Christensen and Kristoffer Lindensj¨o. On finding equilibrium stopping times for time-inconsistent mar-kovian problems.
SIAM Journal on Control and Optimization , 56(6):4228–4255, 2018.[4] Yu-Jui Huang and Adrien Nguyen-Huu. Time-consistent stopping under decreasing impatience.
Finance andStochastics , 22(1):69–95, Jan 2018.[5] Yu-Jui Huang, Adrien Nguyen-Huu, and Xun Yu Zhou. General stopping behaviors of na¨ıve and non-committedsophisticated agents, with application to probability distortion.
Mathematical Finance , (Published online), Jul2019.[6] Yu-Jui Huang and Xiang Yu. Optimal Stopping under Model Ambiguity: a Time-Consistent Equilibrium Ap-proach. arXiv e-prints , page arXiv:1906.01232, Jun 2019.[7] Yu-Jui Huang and Zhou Zhou. Optimal Equilibria for Time-Inconsistent Stopping Problems in Continuous Time.
Mathematical Finance , (Forthcoming).[8] Yu-Jui Huang and Zhou Zhou. The optimal equilibrium for time-inconsistent stopping problems - the discrete-time case.
SIAM J. Control and Optimization , 57:590–609, 2017.[9] Yu-Jui Huang and Zhou Zhou. Strong and Weak Equilibria for Time-Inconsistent Stochastic Control in Contin-uous Time. arXiv e-prints , page arXiv:1809.09243, Sep 2018.[10] Robert H. Strotz. Myopia and inconsistency in dynamic utility maximization.
Review of Economic Studies ,23(3):165–180, 1955. (Erhan Bayraktar) Department of Mathematics, University of Michigan.
E-mail address : [email protected] (Jingjie Zhang) Department of Mathematics, University of Michigan.
E-mail address : [email protected] (Zhou Zhou) School of Mathematics and Statistics, University of Sydney.
E-mail address ::