Existence and uniqueness of recursive utilities without boundedness
aa r X i v : . [ ec on . T H ] J u l Existence and uniqueness of recursive utilitieswithout boundedness ∗ Timothy M. Christensen † July 30, 2020.
Abstract
This paper derives primitive, easily verifiable sufficient conditions for existence anduniqueness of recursive utilities for a number of important classes of preferences. Inorder to accommodate models commonly used in practice, we allow both the statespaceand per-period utilities to be unbounded. For many of the models we study, existenceand uniqueness is established under a single “thin tail” condition on the distribution ofgrowth in per-period utilities. We illustrate our approach with applications to robustpreferences, models of ambiguity aversion and learning about hidden states, dynamicdiscrete choice models, and Epstein–Zin preferences.
Keywords:
Stochastic recursive utility, ambiguity, model uncertainty, existence, unique-ness.
JEL codes:
C62, C65, D81, D84, E7, G10 ∗ This paper is a revised version of the preprint arXiv:1812.11246 which was posted to arXiv in December2018. I thank L. Hansen, T. Sargent, and participants of the “Blue Collar Working Group” at the Universityof Chicago, and several seminar audiences for helpful comments. This material is based upon work supportedby the National Science Foundation under Grant No. SES-1919034. † Department of Economics, New York University, 19 W. 4th Street, 6th floor, New York, NY 10012,USA. E-mail address: [email protected] Introduction
Recursive utilities play a central role in modern macroeconomics and finance. Under recur-sive preferences, the lifetime value (or continuation value) of a stream of per-period utilitiesis defined as the solution to a nonlinear, forward-looking equation. Despite their central role,existence and uniqueness of recursive utilities is an unresolved issue as the recursions aretypically not contraction mappings in the usual sense (Marinacci and Montrucchio, 2010).There are a large number of existence and uniqueness results for recursive utilities in modelswith compact statespace, and possibly also bounded per-period utilities. However, modelsused in practice typically feature unbounded (i.e., non-compact) statespaces and unboundedutilities. For instance, the extensive long-run risks literature typically models state vari-ables as vector autoregressive processes with unbounded shocks. A seemingly reasonableapproach for models with non-compact statespace is simply to truncate (i.e. compactifty)the statespace and apply results for models with compact statespace. After all, this trunca-tion occurs implicitly when computing solutions numerically, even though the model mayhave unbounded statespace. However, truncation of the statespace can materially affect ex-istence and uniqueness. This is highlighted by an empirically relevant example presented inAppendix A, which is based on Bidder and Smith (2018) and Wachter (2013). In this exam-ple, there is always a unique solution in the truncated model (irrespective of the truncationlevel) even when there is no solution or multiple solutions without truncation. This exampleclearly shows there are some important yet subtle differences between models with compactand non-compact statespace. Understanding when the original model without truncationhas a unique solution therefore remains a pressing issue, especially for reconciling numericalsolutions with solutions of the original model.In this paper, we derive primitive sufficient conditions for existence and uniqueness of recur-sive utilities in infinite-horizon Markovian environments. To handle models typically usedin practice, we allow both the support of the Markov state vector and per-period utilities tobe unbounded. For many of the models we study, the single primitive sufficient condition forboth existence and uniqueness is that the distribution of per-period utilities (in stationaryenvironments) or growth in per-period utilities (in nonstationary environments) has thin Epstein and Zin (1989), Alvarez and Jermann (2005), Marinacci and Montrucchio (2010), Balbus(2015), Guo and He (2017), Bloise and Vailakis (2018), and Boroviˇcka and Stachurski (2020). See, e.g., Bansal and Yaron (2004), Hansen, Heaton, and Li (2008), Barillas, Hansen, and Sargent(2009), Wachter (2013), Bansal, Kiku, Shaliastovich, and Yaron (2014), Croce, Lettau, and Ludvigson(2015), Bidder and Smith (2018), Collard, Mukerji, Sheppard, and Tallon (2018), andSchorfheide, Song, and Yaron (2018). The literature has therefore typically appealed to fixed-point results forpositive operators acting on a positive cone of functions. Such arguments often rely oncertain topological properties of the positive cone in the space of bounded functions on acompact set, or other operator-theoretic side conditions.Our point of departure is to dispense altogether with positivity-based arguments and em-bed a transformation of the value function, such as its logarithm, in a class of unboundedbut thin-tailed functions. The class is an exponential-Orlicz class used in empirical processtheory in statistics (van der Vaart and Wellner, 1996) and modern high-dimensional prob-ability (Vershynin, 2018). Exponential-Orlicz classes are naturally suited to the recursionswe study, which involve the composition of exponential and logarithmic transforms andexpected values. The key high-level condition we use to establish uniqueness is that a sub-gradient of the recursion is monotone and its spectral radius is strictly less than one. Formany of the models we study, the subgradient is a discounted conditional expectation undera distorted law of motion. Verifying the spectral radius condition in these models amountsto checking a primitive thin-tail condition on the change-of-measure distorting the law ofmotion. We then specialize this condition to particular models, deriving more primitivethin-tail conditions on the distribution of per-period utilities (in stationary environments)or growth in per-period utilities (in nonstationary environments) which are easy to verify:one simply has to know the tail behavior of the distribution.To illustrate the usefulness of our results, we present applications to four broad classes ofmodels. Section 3 studies a recursion arising under preferences for “robustness”, namely risk-sensitive preferences (Hansen and Sargent, 1995), multiplier preferences (Hansen and Sargent,2001), constraint preferences (Hansen, Sargent, Turmuhambetova, and Williams, 2006), aswell as under Epstein and Zin (1989) preferences with intertemporal elasticity of substitu- An alternative approach taken by Marinacci and Montrucchio (2010) is to establish contractivity inthe Thompson metric. This establishes uniqueness in a class of comparable functions, where the set of allfunctions w comparable to v is { w : a − w ( x ) ≤ v ( x ) ≤ aw ( x ) for some a > } . Such a class can betoo restrictive when the statespace is unbounded. For instance, the exponential-affine functions v ( x ) =exp( a + b x ) and v ( x ) = exp( a + b x ) are only comparable if b = b . This section first describes the classes of “thin-tailed” functions with which we work. Abasic existence and uniqueness result is presented. The key condition for uniqueness is ahigh-level spectral radius condition. Sufficient conditions for this are derived in terms of athin-tail condition on a change-of-measure. We use this intermediate result to derive moreprimitive sufficient conditions in the applications in Sections 3-6.4 .1 Orlicz classes
We begin by briefly reviewing some relevant properties of Orlicz classes. We refer the readerto section 10 of Krasnosel’skii and Rutickii (1961) for further details.Let ( X , X , µ ) be a σ -finite measure space. In most of what follows we will study Marko-vian environments in which X t is a state vector supported on X and µ is the stationarydistribution of X t . Let ψ : R + → R + be monotone, continuously differentiable, and strictlyconvex with ψ (0) = 0 and ψ ( x ) /x → + ∞ as x → + ∞ . The Luxemburg norm of f : X → R is defined as k f k ψ = inf (cid:26) c > Z ψ ( | f ( x ) | /c ) d µ ( x ) ≤ (cid:27) . Let L denote the (equivalence class of) all measurable f : X → R for which k f k ψ < ∞ .Also let E = (cid:26) f ∈ L : Z ψ ( | f ( x ) | /c ) d µ ( x ) < ∞ for each c > (cid:27) , which is the closure of L ∞ (the space of all µ -essentially bounded functions) in L . Both L and E are ordered Banach spaces when equipped with the norm k · k ψ and partial ordering f ≥ g denoting f ( x ) ≥ g ( x ) µ -almost everywhere. Iflim x → + ∞ xψ ′ ( x ) ψ ( x ) < + ∞ then L = E ; otherwise, E is a proper subset of L . Example 1: Lp classes.
Let ψ ( x ) = x p for p ∈ (1 , ∞ ). Then k · k ψ is equivalent to the L p norm k f k p := ( R | f ( x ) | p d µ ( x )) /p and L = E = L p . Example 2: Exponential-Orlicz classes.
These will play an important role in whatfollows. We shall use L φ r and E φ r to denote the spaces L and E corresponding to ψ ( x ) =exp( x r ) − r ≥
1, and denote the Luxemburg norm by k · k φ r . Here E φ r is a propersubset of L φ r . When µ is a probability measure, L ∞ ֒ → E φ r ֒ → L φ r ֒ → E φ s ֒ → L φ s ֒ → L p are continuous embeddings for 1 ≤ s < r < ∞ , with k f k p ≤ p !(log 2) /r − k f k φ r for each1 ≤ p < ∞ , and k f k φ s ≤ (log 2) /r − /s k f k φ r (van der Vaart and Wellner, 1996, p. 95). Let E be a closed linear subspace of E . A (possibly nonlinear) operator T : E → E is continuous if for any convergent sequence { f n } n ≥ ⊂ E with k f n − f k ψ → T f n − T f k ψ →
0. A linear operator D : E → E is bounded if it is continuous. Let k D k E := sup {k D f k ψ : f ∈ E , k f k ψ = 1 } ρ ( D ; E ) := lim n →∞ k D n k /n E (1)denote the operator norm and spectral radius of D , where D n f denotes D applied n timesin succession to f . The operator T is monotone if T f ≥ T g whenever f ≥ g . A boundedlinear operator D f : E → E is a subgradient of T at f ∈ E if the inequality T g − T f ≥ D f ( g − f )holds for each g ∈ E . We say that a decreasing sequence of functions { v n } n ≥ ⊂ E is boundedfrom below by v ∈ E if lim inf n →∞ v n ≥ v ; similarly, an increasing sequence of functions { v n } n ≥ ⊂ E is bounded from above by ¯ v ∈ E if lim sup n →∞ v n ≤ ¯ v . Let T n v denote T applied n times in succession to v . The following Proposition is a useful starting point fororganizing the discussion that follows. Proposition 2.1. (i) Existence: Let T be a continuous and monotone operator on E andlet there exist v, ¯ v ∈ E such that either (a) T ¯ v ≤ ¯ v and { T n ¯ v } n ≥ is bounded from below by v , or (b) T v ≥ v and { T n v } n ≥ is bounded from above by ¯ v . Then: T n ¯ v (if (a) holds) or T n v (if (b) holds) converges to a fixed point v ∈ E , with v ≤ v ≤ ¯ v .(ii) Uniqueness: Suppose that at each of its each fixed points v ∈ E , T has a subgradient D v which is monotone with ρ ( D v ; E ) < . Then: T has at most one fixed point in E . When uniqueness cannot be guaranteed, we use ordering and stability criteria to refine theset of fixed points. Let V denote the set of fixed points of T . Say v is the smallest fixedpoint of T if v ≤ v ′ for each v ′ ∈ V . Say v is stable if ρ ( D v ; E ) < v is a useful property. In many of the examples we consider below,the subgradient is of the form D v = β ˜ E with β ∈ (0 , E is a distorted probabilitymeasure. In these examples, stability ensures that discounted expected utilities under ˜ E are finite. Stability of v also ensures that fixed-point iteration on a neighborhood of v willconverge to v . Corollary 2.1.
Let v be a fixed point of T with ρ ( D v ; E ) < . Then: v is both the smallestfixed point and the unique stable fixed point of T in E . .3 Verifying the spectral radius condition In models featuring forward-looking agents, the subgradient is typically a discounted condi-tional expectation. However, nonlinearities of the recursion can introduce a wedge betweenthe probability measure describing the evolution of state variables and the probability mea-sure under which the expectation is taken.When there is no such wedge (e.g., time-separable preferences and rational expectations),the spectral radius condition is easily seen to hold. Let X = { X t } t ≥ be a strictly stationaryMarkov process with transition kernel Q and stationary distribution µ . Let D v = β E Q , where E Q denotes conditional expectation under Q . Then for any c > f ∈ E , Z ψ ( | E Q f ( x ) | /c ) d µ ( x ) ≤ Z E Q [ ψ ( | f ( X t +1 ) | /c ) | X t = x ] d µ ( x ) = Z ψ ( | f ( x ) | /c ) d µ ( x ) , by Jensen’s inequality and the fact that µ is the stationary distribution associated with X .Therefore, k D v k E = β and so ρ ( D v ; E ) = β .This argument breaks down in the settings we study, where D v = β ˜ E , where ˜ E denotesconditional expectation under a distribution different from Q . We verify the spectral radiuscondition under a thin-tail condition on the change-of-measure transforming E Q into ˜ E .Suppose ˜ E f ( x ) = E Q [ m ( X t , X t +1 ) f ( X t ) | X t = x ] , (2)where m is the (conditional) change-of-measure transforming E Q into ˜ E . Repeatedly apply-ing D v involves repeatedly multiplying by m , taking conditional expectations under Q , anddiscounting. Provided the moments of m don’t grow too quickly, repeatedly applying D v tothin-tailed functions ensures discounting eventually dominates and the spectral radius con-dition holds. We now formalize this reasoning. Let log m ∨ m and 0. Also let µ ⊗ Q denote the joint (stationary) distribution of ( X t , X t +1 ). Lemma 2.1.
Let D = β ˜ E where β ∈ (0 , and ˜ E is of the form (2) with E µ ⊗ Q [exp( | log m ( X t , X t +1 ) ∨ | r /c )] < ∞ (3) for some c > and r > . Then: D is a continuous linear operator on E φ s with ρ ( D ; E φ s ) < for each s ≥ . Note that we do not require stationarity (or any other property) of X under the law ofmotion corresponding to ˜ E . 7 Application 1: Robust (and related) preferences
Consider an infinite-horizon environment in which the continuation value V t of a stream ofper-period utilities { U t } t ≥ from date t forwards is defined recursively by V t = U t − βθ log E h e − θ − V t +1 (cid:12)(cid:12)(cid:12) F t i , (4)where F t is the date- t information set, β ∈ (0 ,
1) is a time preference parameter, and θ >
0. Recursion (4) arises in a number of settings. It is the risk-sensitive recursion ofHansen and Sargent (1995), where θ is interpreted as a risk-sensitivity parameter. The re-cursion also arises under “robust” preferences which express an aversion to model uncer-tainty, namely multiplier preferences (Hansen and Sargent, 2001) and constraint preferences(Hansen et al., 2006), in which θ encodes the agent’s aversion to model uncertainty. Finally,recursion (4) is equivalent to Epstein and Zin (1989) preferences with unit IES, in whichcase θ is a transformation of the risk aversion parameter.We follow much of the literature and consider environments characterized by a stationaryMarkov state process X = { X t : t ≥ } supported on a statespace X ⊆ R d . The set F t willdenote the information set generated by the realization of X up to date t . Let Q denotethe Markov transition kernel and E Q denote conditional expectation with respect to Q . Insuch environments it follows for certain commonly used specifications of U t that there exists v : X → R and u : X × X → R and such that v ( X t ) = − θ (cid:18) V t − − β U t (cid:19) , u ( X t , X t +1 ) = U t +1 − U t . For instance, this is true when U t = log( C t ) and consumption growth log( C t +1 /C t ) is afunction of ( X t , X t +1 ). Under these conditions, the recursion may be rewritten in terms ofthe scaled continuation value function v : v ( x ) = β log E Q h e v ( X t +1 )+ αu ( X t ,X t +1 ) (cid:12)(cid:12)(cid:12) X t = x i , (5)where α = − ( θ (1 − β )) − . Recursion (5) may be expressed in operator notation as v = T v , Our results trivially extend to allow log( C t +1 /C t ) = g ( X t , X t +1 , Y t +1 ) where the conditional distribu-tion of ( X t +1 , Y t +1 ) given ( X t , Y t ) depends only on X t by redefining the state as ( X t , Y t ). Any value functionwill only depend on X , however, as it is the only relevant conditioning variable. T f ( x ) = β log E Q h e f ( X t +1 )+ αu ( X t ,X t +1 ) (cid:12)(cid:12)(cid:12) X t = x i . Hansen and Scheinkman (2012) and Christensen (2017) studied this recursion in the contextof Epstein–Zin preferences with unit IES allowing unbounded X . Hansen and Scheinkman(2012) derived sufficient conditions for existence of a fixed point but did not study unique-ness. Their conditions restrict moments of a Perron–Frobenius eigenfunction of an operatorand require convergence of a sequence of iterates of a related recursion. Christensen (2017)established uniqueness on a neighborhood for the same recursion under a spectral radiuscondition but did not establish existence or global uniqueness. We establish both theseproperties under a primitive tail condition the stationary distribution of u ( X t , X t +1 ). Given the form of T , the functions f and u must have sufficiently thin tails in order that T f be well defined. We therefore work with Orlicz classes of the form E φ r with r ≥ µ of X . The single condition we require is that thestationary distribution of per-period utility growth has thin tails: for some r ≥
1, we have E µ ⊗ Q [exp( | u ( X t , X t +1 ) | r /c )] < ∞ for all c >
0. (6)We verify this condition below in three examples. In the second example, we show thatuniqueness can fail when condition (6) does not hold.We shall establish existence and uniqueness by applying Proposition 2.1. The operator T is continuous, monotone, and convex under condition (6); see Lemma B.5. The proofof existence constructs an upper value ¯ v and shows the sequence of iterates { T n ¯ v } n ≥ isbounded from below. For uniqueness, the operator T obeys a subgradient inequality withsubgradient D v f ( x ) = β E v f ( x ) , E v is a distorted conditional expectation: E v f ( x ) = E Q [ m v ( X t , X t +1 ) f ( X t +1 ) | X t = x ] ,m v ( X t , X t +1 ) = e v ( X t +1 )+ αu ( X t ,X t +1 ) E Q [ e v ( X t +1 )+ αu ( X t ,X t +1 ) | X t ] . For robust preferences, E v may be interpreted as expectation under the agent’s “worst-case”model. The spectral radius condition is verified by applying Lemma 2.1; see Lemma B.6. Theorem 3.1.
Let condition (6) hold. Then: T has a fixed point v ∈ E φ r . Moreover, if r > then: (i) v is the unique fixed point of T in E φ s for each s ∈ (1 , r ] , and (ii) v is boththe smallest fixed point and the unique stable fixed point of T in E φ . Example 1: Linear-Gaussian environments.
Condition (6) holds for all r ∈ [1 , u ( X t , X t +1 ) = λ ′ X t + λ ′ X t +1 and its stationary distribution is Gaussian.This specification arises, for instance, with U t = log( C t e λ ′ X t ) where log( C t +1 /C t ) is a func-tion of ( X t , X t +1 ) and the process X is a stationary Gaussian VAR(1): X t +1 = ν + AX t + u t +1 , u t +1 ∼ N (0 , Σ) , with all eigenvalues of A inside the unit circle. This setting was considered in Hansen et al.(2008), Barillas et al. (2009), and several other works. It is known that T has a fixed pointof the form v ( x ) = a + b ′ x where b = αβ ( I − βA ′ ) − ( λ + A ′ λ ) and a = β − β (cid:16) ( αλ + b ) ′ ν + 12 ( αλ + b ) ′ Σ ′ ( αλ + b ) (cid:17) . Theorem 3.1 shows that v ( x ) = a + b ′ x is the unique fixed point in E φ s for all s ∈ (1 , E φ . Example 2: Fat tails and rare disasters.
This example shows there can exist multiplefixed points when condition (6) is violated. The model features time-varying rare disastersfrom Bidder and Smith (2018). A similar model is studied in Wachter (2013) in the contextof Epstein–Zin preferences with IES = 1. Consumption growth g t +1 := log( C t +1 /C t ) ismodeled as g t +1 = ν g + w z,t +1 + σw g,t +1 , with w g,t +1 ∼ N (0 , w z,t +1 | j t +1 ∼ N ( ν j j t +1 , σ j j t +1 ) where ν j < j t +1 | h t is Poisson withmean h t which follows an autoregressive gamma (ARG) process. Defining X t = ( g t , h t ), we10ee that u ( X t , X t +1 ) = g t +1 . By iterated expectations we may deduce E µ ⊗ Q h e cu ( X t ,X t +1 ) i = e cν g + c σ E µ (cid:20) exp (cid:18) h t (cid:18) exp (cid:26) cν j + c σ j (cid:27) − (cid:19)(cid:19)(cid:21) . Condition (6) is violated for this model: the expectation on the right-hand side is onlyfinite c in a neighborhood of zero because the stationary distribution of h t is a Gammadistribution. Indeed, it is known that there may exist zero, one, or two fixed points of theform v ( x ) = a + b ′ x under this specification. The precise number of fixed points of this formis determined by the number of real solutions to a particular quadratic equation.One could modify the above specification so that w z,t +1 | j t +1 ∼ N ( µ j j ςt +1 , σ j ) for some ς ∈ [ , r ∈ [1 , /ς ). Therefore, there is a unique fixed point v ∈ E φ s for all s ∈ (1 , /ς ), and a unique stable fixed point in E φ . Example 3: Regime-switching.
Consider the same setup from Example 1 but sup-pose now that the parameters of the VAR are state-dependent (see, e.g., Hamilton (1989),Cecchetti, Lam, and Mark (1990, 2000), Hansen and Sargent (2010), and Ang and Timmermann(2012)): X t +1 = ν s t + A s t X t + u t +1 , u t +1 ∼ N (0 , Σ s t ) , where s t is stationary, exogenous Markov state taking values in { , . . . , N } , and all eigen-values of A s are inside the unit circle for each s = 1 , . . . , N . The full state vector is now( X t , s t ), which is jointly Markovian and stationary. The stationary distribution of growthin per-period utilities u ( X t , X t +1 ) is sub-Gaussian (see, e.g., Vershynin, 2018, Section 2.5),and so condition (6) holds for all r ∈ [1 , E φ s for all s ∈ (1 ,
2) (with E φ s defined with respect to the stationary distributionof ( X t , s t )), and a unique stable fixed point in E φ . This section extends the setting from Section 3 to a class of dynamic models where theagent learns about a hidden state, e.g. a regime, stochastic volatility, growth process, or time-varying parameter. This setting is relevant for several types of preferences, including: (i) theextension of multiplier preferences by Hansen and Sargent (2007, 2010) to include concernsabout misspecification of beliefs about the hidden state, (ii) generalized recursive smooth11mbiguity preferences of Ju and Miao (2012) with unit IES, (iii) special cases of recursivesmooth ambiguity preferences studied by Klibanoff et al. (2009), and (iv) Epstein and Zin(1989) recursive preferences with unit IES and learning.
We again consider environments characterized by a Markov state process X = { X t } t ≥ withtransition kernel Q . Partition the state as X t = ( ϕ t , ξ t ) where the agent observes ϕ t butdoes not observe ξ t . Let O t = σ ( ϕ t , ϕ t − , . . . , ϕ ) denote the history of the observed stateto date t . Beliefs about ξ t are summarized by a posterior distribution Π t conditional on O t . We consider environments in which the continuation value V t of a stream of per-periodutilities { U t } t ≥ from date t forward is defined recursively as V t = U t − βθ log E Π t (cid:20) E Q h e − ϑ − V t +1 (cid:12)(cid:12)(cid:12) O t , ξ t i ϑθ (cid:12)(cid:12)(cid:12)(cid:12) O t (cid:21) , (7)for β ∈ (0 , Q ) and beliefs about the hidden state (Π t ), where ϑ > θ > Q and Π t , respectively. When U t = log C t , recursion (7) also arisesunder generalized recursive smooth ambiguity preferences of Ju and Miao (2012) with unitIES, where θ and ϑ are one-to-one transformations of their ambiguity aversion and riskaversion parameters, respectively. When ϑ = θ , recursion (7) reduces to V t = U t − βϑ log E Π t h E Q h e − ϑ − V t +1 (cid:12)(cid:12)(cid:12) O t , ξ t i(cid:12)(cid:12)(cid:12) O t i . With U t = log C t , this recursion corresponds to Epstein–Zin recursive preferences with unitIES and learning about the hidden state. In the limit as ϑ → ∞ (thus, the agent is confidentin Q but has doubts about the hidden state) recursion (7) becomes V t = U t − βθ log E Π t h e − θ − E Q [ V t +1 |O t ,ξ t ] (cid:12)(cid:12)(cid:12) O t i . (8)This recursion is obtained under recursive smooth ambiguity preferences of Klibanoff et al.(2009), when their function φ is taken to be φ ( x ) = exp( − θ − x ).We impose several (standard) conditions to make the problem tractable. First, the state is12ssumed to have a conventional hidden Markov structure, in which Q ( X t +1 | X t ) = Q ϕ ( ϕ t +1 | ξ t ) Q ξ ( ξ t +1 | ξ t ) . This nests models with regime-switching studied by Ju and Miao (2012) as well as mod-els with learning about a hidden growth term as in Hansen and Sargent (2007, 2010),Croce et al. (2015) and Collard et al. (2018). Our analysis extends to allow ϕ t to influence ϕ t +1 , but we maintain this simpler presentation for convenience.Second, we assume Π t is summarized by a finite-dimensional sufficient statistic ˆ ξ t :Π t ( ξ t ) = Π ξ ( ξ t | ˆ ξ t )for some conditional distribution Π ξ , where ˆ ξ is updated according to a time-invariant rule:ˆ ξ t +1 = Ξ( ˆ ξ t , ϕ t +1 ) . These conditions are satisfied under Bayesian updating when the state ξ t takes finitelymany values (e.g. a hidden regime) and when X t evolves as a Gaussian state-space model;see below. The rule for ˆ ξ t could also represent belief updating in a boundedly-rational way.Let ˆ X t = ( ϕ ′ t , ˆ ξ ′ t ) ′ and let X ˆ X , X ˆ ξ , and X ϕ denote the support of ˆ X t , ˆ ξ t , and ϕ t .We assume learning is in a “steady state”, i.e., { ( ξ t , ˆ X t ) } t ≥ is stationary. In linear-Gaussianenvironments, learning corresponds to the Kalman filter. If the filter is not initialized inits steady-state then this process will typically be non-stationary. The stationary problemstudied here is a boundary problem representing convergence of the filter to its steady state.Solutions can be obtained by backwards iteration from the steady-state boundary solution. Uniqueness of the limiting steady state recursion is necessary for uniqueness of the sequenceof backward iterates.Finally, we require that there exists v : X ˆ ξ → R and u : X ϕ → R such that v ( ˆ ξ t ) = − θ (cid:18) V t − − β U t (cid:19) , u ( ϕ t +1 ) = U t +1 − U t . We give two examples of environments in which the preceding conditions hold. In bothexamples, U t = log( C t ) and log( C t +1 /C t ) is a function of ϕ t +1 . A similar approach is taken by Collin-Dufresne, Johannes, and Lochstoer (2016) in models featuringEpstein–Zin preferences and learning about parameters of the data-generating process. xample 1: Regime switching. Suppose that ξ t ∈ { , . . . , N } denotes a hidden Markovstate with transition matrix Λ. Let the conditional distribution of ϕ t +1 given ξ t = ξ havedensity q ( ·| ξ ). The posterior Π t is identified with a vector ˆ ξ t of regime probabilities given O t . Beliefs ˆ ξ t are updated as ˆ ξ t +1 = Λ q ( ϕ t +1 ) ⊙ ˆ ξ t ′ ( q ( ϕ t +1 ) ⊙ ˆ ξ t ) , where q ( ϕ t +1 ) is the N -vector whose entries are q ( ϕ t +1 | ξ ) for ξ ∈ { , . . . , N } , ⊙ denoteselement-wise product, and 1 is a N -vector of ones (see, e.g., Hamilton, 1994, Section 4.2).For example, Ju and Miao (2012) study an economy in which consumption and dividendgrowth is jointly dependent on a hidden regime ξ t :log( C t +1 /C t ) = κ ξ t + u Ct +1 , log( D t +1 /D t ) = ζ log( C t +1 /C t ) + g d + u Dt +1 , where u Ct and u Dt are i.i.d. N (0 , σ C ) and N (0 , σ D ). The observable state is ϕ t = log( C t /C t − ).The stationary distribution of u ( ϕ t +1 ) is a finite mixture of Gaussians. Our results also allowthe volatility of consumption and dividend growth to be state-dependent. Example 2: Gaussian state-space models.
Suppose X evolves under Q according to: ϕ t +1 = Aξ t + u ϕt +1 , ξ t +1 = Bξ t + u ξt +1 , where u ϕt and u ξt are i.i.d. N (0 , Σ u ) and N (0 , Σ w ), respectively, and all eigenvalues of B are inside the unit circle. This is the setting studied in Hansen and Sargent (2007, 2010),Croce et al. (2015), Collard et al. (2018), and several other works. If ξ ∼ N (ˆ µ , ˆΣ ) underΠ then ξ t ∼ N (ˆ µ t , ˆΣ t ) under Π t . The matrix ˆΣ t will converge to a fixed matrix ¯Σ as t → ∞ .In this steady state, the sufficient statistic for Π t is ˆ ξ t = ˆ µ t which is updated asˆ ξ t +1 = B ˆ ξ t + B ¯Σ A ′ ( A ¯Σ A ′ + Σ u ) − ( ϕ t +1 − A ˆ ξ t ) . The stationary distribution of u ( ϕ t ) is Gaussian. The only related existence and uniqueness result we are aware of in any of these setting isthat of Klibanoff et al. (2009) for recursive smooth ambiguity preferences (recursion (8)).Their result applies to bounded functions and requires bounded per-period utilities.14 .3 New results
Recursion (7) may be reformulated as the fixed-point equation v = T v where T f ( ˆ ξ t ) = β log E Π ξ (cid:20) E Q ϕ h e θϑ f (Ξ(ˆ ξ t ,ϕ t +1 ))+ αu ( ϕ t +1 ) (cid:12)(cid:12)(cid:12) ξ t , ˆ ξ t i ϑθ (cid:12)(cid:12)(cid:12)(cid:12) ˆ ξ t (cid:21) . Recursion (8) in the limiting case with ϑ = + ∞ may be reformulated as the fixed-pointequation v = T v where T f ( ˆ ξ t ) = β log E Π ξ h e E Qϕ [ f (Ξ(ˆ ξ t ,ϕ t +1 ))+ αu ( ϕ t +1 ) | ξ t , ˆ ξ t ] (cid:12)(cid:12)(cid:12) ˆ ξ t i . The existence and uniqueness results presented below apply to either case, though the proofsare presented only for the more involved setting in which ϑ < ∞ .Let E φ r ˆ X be defined relative to the stationary distribution µ of ˆ X t = ( ϕ ′ t , ˆ ξ ′ t ) ′ . Similarly, let E φ r ϕ ⊂ E φ r ˆ X and E φ r ˆ ξ ⊂ E φ r ˆ X denote functions in E φ r ˆ X depending only on ϕ or ˆ ξ , respectively.The key regularity condition is again that the stationary distribution of utility growth hasthin tails: u ∈ E φ r ϕ (9)for some r ≥
1. Note that this condition depends only on the marginal distribution of theobserved state and is therefore easy to verify.We establish existence and uniqueness of fixed points of T by applying Proposition 2.1.Further details on the form of the subgradient and verification of Lemma 2.1 are deferredto Appendix B.4. Theorem 4.1.
Let condition (9) hold. Then: T has a fixed point v ∈ E φ r ˆ ξ . Moreover, if r > , then: (i) v is the unique fixed point of T in E φ s ˆ ξ for all s ∈ (1 , r ] , and (ii) v is boththe smallest fixed point and the unique stable fixed point of T in E φ ˆ ξ . Example 1: Regime switching (continued).
In the example of Ju and Miao (2012),the stationary distribution of u ( ϕ t +1 ) is a finite mixture of Gaussians, so (9) holds forall r ∈ [1 , E φ s for all s ∈ (1 , E φ . 15 xample 2: Gaussian state-space models (continued). Here the stationary distri-bution of u ( ϕ t +1 ) is Gaussian, so (9) holds for all r ∈ [1 , E φ s for all s ∈ (1 , E φ .It is straightforward (albeit more cumbersome notationally) to extend the preceding analysisto allow for u to depend on ( ϕ t , ϕ t +1 ) and to allow the law of motion to be of the moregeneral form Q ( X t +1 | X t ) = Q ϕ ( ϕ t +1 | ξ t , ϕ t ) Q ξ ( ξ t +1 | ξ t ) . In this case, however, the effective state vector will be ˆ X t rather than ˆ ξ t . In this section we study infinite-horizon DDC models following Rust (1987). The valuefunction recursion in infinite-horizon DDC models has a similar structure to the recursionsstudied in the previous sections under the conventional assumption on the distribution oflatent utility shocks. We allow the statespace to be unbounded and continuous and per-period utilities to be unbounded. As noted by Norets (2010), the recursion might not be acontraction when the statespace and per-period utilities are unbounded.
We first briefly summarize the DDC framework following Rust (1987) to fix ideas andnotation. At each date t ∈ T , an agent chooses among D discrete alternatives indexed by d ∈ { , , . . . , D } to maximize the expected present discounted value of utility. The periodutility from choosing action d at date t is u ( d, X t , ε t ) = u d ( X t ) + ε dt where X t is a state vector that is observed by the econometrician and agent, supported onstatespace X ⊆ R d , and the vector ε t = ( ε t , . . . , ε Dt ) ′ is a vector of utility shocks. As inmuch of the literature, we assume the ε t are i.i.d. type-I extreme value (standard Gumbel)distributed, and that X has a conditional distribution which factorizes as F ( X t +1 , ε t +1 | X t = x, ε t = ε, D t = d ) = M ( X t +1 | x, d ) G ( ε t +1 )16or every ( x, ε, d ), where M is a time-invariant Markov transition kernel and G denotes theassumed distribution of ε t +1 . Let β ∈ (0 ,
1) denote the agent’s time preference parameter.The agent’s problem may be expressed recursively: v ( x ) = E G (cid:20) max d (cid:0) u d ( x ) + ε dt + β E M [ v ( X t +1 ) | X t = x, D t = d ] (cid:1)(cid:21) where v is the agent’s ex ante value function and E G [ · ] denotes expectation over ε under G .In view of the parametric assumption on G , the recursion becomes v ( X t ) = log D X d =1 e u d ( X t )+ β E M [ v ( X t +1 ) | X t ,D t = d ] ! + γ EM (10)where γ EM ≈ . v = T v , where T f ( x ) = log D X d =1 e u d ( X t )+ β E M [ f ( X t +1 ) | X t ,D t = d ] ! + γ EM . Motivated by computational considerations, the existing literature typically discretizes thestatespace to a finite grid, in which case T is a contraction mapping on the space B ( X )of bounded functions on X equipped with the sup norm. If X is continuous but compact,the operator T is a contraction on B ( X ) (Rust, Traub, and Wozniakowski, 2002). Blevins(2014) allows for continuous, unbounded state (and continuous choices) but requires the per-period utilities u d , d = 1 , . . . , D , to be uniformly bounded, in which case T is a contractionmapping on B ( X ). Norets (2010) allows for unbounded u d under a particular choice ofweighted sup norm where the weighting function is chosen to be compatible with utilitiesand the transition kernel M . He shows that a power of T is a contraction mapping on theclass of functions with finite weighted sup norm. Our results are specific to the conventionaltype-I extreme value assumption whereas the results in Norets (2010) apply more generally. We apply Proposition 2.1 to derive existence and uniqueness conditions allowing unbounded,continuous statespace X and unbounded per-period utilities u d , d = 1 , . . . D .17s the equilibrium law of motion of X depends on the solution to the dynamic decisionproblem, here we define the transition kernel Q ( X t +1 | X t ) = 1 D D X d =1 M ( X t +1 | X t , d ) . We assume that the process X has a unique stationary distribution µ under Q . This is triv-ially true when there is a renewal action , say d ∗ , for which Q ( X t +1 | X t , d ∗ ) does not dependon X t . That is, Q ( ·| X t , d ∗ ) = ν ( · ) for some distribution ν , so Q ( ·| X t ) ≥ D − ν ( · ) holds forevery X t . This inequality verifies Doeblin’s minorization condition and therefore guaranteesexistence of a unique stationary distribution (Meyn and Tweedie, 2009, Theorem 16.2.4).As emphasized in Arcidiacono and Miller (2011), many models in the DDC literature doindeed have renewal choices, including the bus engine replacement model of Rust (1987),so our results necessarily encompass, but are not limited to, such models. The key condition here is that each of the period utility functions have thin tails under µ : u , . . . , u D ∈ E φ r for some r ≥
1. (11)We establish existence and uniqueness by verifying applying Proposition 2.1. The subgra-dient of T is D v f ( x ) = β E Q [ m v ( X t , X t +1 ) f ( X t +1 ) | X t = x ] , (12)where m v ( X t , X t +1 ) = D X d =1 w d,v ( X t ) m d ( X t +1 | X t ) , with m d ( ·| X t ) denoting the Radon–Nikodym derivative of M ( ·| X t , d ) with respect to Q ( ·| X t ),and with weights w v,d given by w v,d ( x ) = e u d ( x )+ β E M [ v ( X t +1 ) | X t = x,D t = d ] P Dd ′ =1 e u d ′ ( x )+ β E M [ v ( X t +1 ) | X t = x,D t = d ′ ] . (13)Interestingly, here the weights w v,d ( x ) are precisely the conditional choice probabilities foraction d in state x arising from the solution of the agent’s dynamic decision problem. Thespectral radius condition is again verified by applying Lemma 2.1. The existence of renewal actions allows the expression for continuation values to be differenced out fromthe expression for conditional choice probabilities, simplifying estimation (see, e.g., Arcidiacono and Miller(2011)). Nevertheless, existence and uniqueness of continuation values remains relevant, inter alia, for quan-tifying the welfare effects of policy interventions. heorem 5.1. Let u , . . . , u D ∈ E φ r for some r ≥ . Then: T has a fixed point v ∈ E φ r .Moreover, v is the unique fixed point of T in E φ s for all s ∈ [1 , r ] . In this section we study Epstein and Zin (1989) preferences with IES = 1. Existence anduniqueness when state variables have unbounded support remains an open question. Allow-ing unbounded support is of particular importance, however, as prominent models, such asthose in the long-run risks literature, typically feature state variables that evolve as vectorautoregressive processes with unbounded shocks. This is a complicated issue and we donot seek to provide a complete treatment. Indeed, there are currently no uniqueness resultsfor the recursion we study with unbounded statespace. Rather, we show how our approachmay be used to derive primitive existence conditions in empirically relevant settings withunbounded statespace. The continuation value V t of the agent’s consumption plan from time t forward solves V t = n (1 − β )( C t ) − ρ + β E [( V t +1 ) − γ |F t ] − ρ − γ o − ρ , where C t is date- t consumption, F t is the date- t information set, γ > /ρ > ρ = 1 in this section; the case with ρ = 1 is subsumed inthe analysis of Section 3. We consider environments characterized by a stationary Markovstate process X = { X t : t ≥ } with supported on a statespace X ⊆ R d . Let Q denote theMarkov transition kernel and E Q denote conditional expectation with respect to Q . Also letlog( C t +1 /C t ) = g ( X t , X t +1 ) for some function g : X × X → R . Then (1 − ρ ) log( V t /C t ) = v ( X t ) where the function v : X → R solves v ( X t ) = log (cid:18) (1 − β ) + β E Q h e κv ( X t +1 )+(1 − γ ) g ( X t ,X t +1 ) (cid:12)(cid:12)(cid:12) X t i κ (cid:19) (14) Our results trivially extend to allow log( C t +1 /C t ) = g ( X t , X t +1 , Y t +1 ) where the conditional distribu-tion of ( X t +1 , Y t +1 ) given ( X t , Y t ) depends only on X t by redefining the state as ( X t , Y t ). κ = − γ − ρ (see, e.g., Hansen et al. (2008)). The properties of this recursion differ de-pending on whether κ < κ ∈ (0 , κ ∈ [1 , ∞ ). We focus on the former, as it is thepertinent case in the long-run risks literature where standard typically γ > /ρ > Epstein and Zin (1989) and Marinacci and Montrucchio (2010) derived sufficient conditionsfor existence and uniqueness when consumption growth is bounded. Alvarez and Jermann(2005) establish existence and uniqueness when consumption growth is i.i.d. with boundedinnovations. Guo and He (2017) establish sufficient conditions for existence and uniquenessfor with finite statespace.The two most closely related works are Hansen and Scheinkman (2012; HS hereafter) andBoroviˇcka and Stachurski (2020; BS hereafter). HS and BS present conditions for existencewhen κ < X is unbounded. We also only present sufficient conditions for existencebecause the operator does not have a subgradient of the form studied in Section 2.3 when ρ = 1. HS and BS work with a positive transformation of v , such as h := e v , and work inan L space under to the stationary distribution µ of X in BS or a distorted probabilitymeasure (denoted ˜ µ below) in HS. Our approach imposes stronger integrability conditionsbut it guarantees that all moments of h and 1 /h are finite and, therefore, that the SDF β ( C t +1 /C t ) − ρ " V − γt +1 E Q [ V − γt +1 |F t ] ρ − γ − γ ≡ β ( C t +1 /C t ) − ρ (cid:20) h ( X t +1 )( C t +1 /C t ) − γ E Q [ h ( X t +1 )( C t +1 /C t ) − γ | X t ] (cid:21) ρ − γ − γ (15)is well defined provided consumption growth has sufficiently thin tails.The existence conditions in HS restrict the size of a Perron–Frobenius eigenvalue (condition(19) below) and moments of its eigenfunction under ˜ µ . Our first two results impose astronger thin-tail condition on the eigenfunction, though these restrictions do not seem tobite for models commonly encountered. Our final result imposes a thin-tail condition underthe true stationary measure. BS showed the eigenvalue condition is necessary and sufficientfor existence in L under some weak-compactness and irreducibility side conditions on anoperator. We require no such operator-theoretic side conditions. Hansen and Scheinkman (2012) also establish sufficient conditions for uniqueness when κ ≥ .3 New results Under general conditions (see Hansen and Scheinkman (2009) and Christensen (2015, 2017)),there exists a strictly positive function ι and scalar λ > the equation λι ( x ) = E Q [ ι ( X t +1 )( C t +1 /C t ) − γ | X t = x ] . (16)Hansen and Scheinkman (2009) show that ι and λ may be used to induce a distorted con-ditional expectation˜ E f ( x ) = E Q (cid:20) ι ( X t +1 )( C t +1 /C t ) − γ λι ( X t ) f ( X t +1 ) (cid:12)(cid:12)(cid:12)(cid:12) X t = x (cid:21) . Hansen and Scheinkman (2012) show solving (14) is equivalent to finding a fixed point of T f ( x ) = log (cid:16) (1 − β ) ι ( x ) − κ + βλ κ ˜ E [ e κf ( X t +1 ) | X t = x ] κ (cid:17) , (17)with the v to recursion (14) and the fixed point of T differing additively by κ log ι .For the first two results, we assume X is stationary under the law of motion correspondingto the distorted conditional expectation ˜ E (Theorem 6.3 below does not require this). Let˜ µ denote the stationary distribution induced by ˜ E . The first result is for Orlicz spaces ˜ E φ r defined relative to ˜ µ (subsequent results pertain to the true stationary measure µ ).Our first regularity condition requires that log ι has thin tails, in the sense thatlog ι ∈ ˜ E φ r for some r ≥
1. (18)Under this condition, Lemma B.10 shows that T is a continuous, monotone operator on ˜ E φ s for each 1 ≤ s ≤ r . It is clear that T v ≥ log((1 − β ) ι ( x ) − κ ). Therefore, should there exist a¯ v ∈ ˜ E φ r for which T ¯ v ≤ ¯ v , the sequence of iterates T n ¯ v must be bounded from below. Theremainder of the proof shows that the inequality T ¯ v ≤ ¯ v holds for the function¯ v ( x ) = log (1 − β ) ∞ X n =0 ( βλ κ ) n ˜ E n ( ι − κ )( x ) ! . The sum is convergent under the eigenvalue condition from Hansen and Scheinkman (2012): βλ κ < . (19) Note the function ι is defined only up to scale normalization. emark 6.1. Although T is not contractive, it follows from Proposition 2.1(i) that thesequence of iterates ¯ v, T ¯ v, T ¯ v, . . . will converge to a fixed point of T under the conditionsof any of Theorems 6.1-6.3 below. Theorem 6.1.
Let κ < and conditions (18) and (19) hold. Then: T has a fixed point in ˜ E φ s and therefore the recursion (14) has a solution v ∈ ˜ E φ s for all s ∈ [1 , r ] . We now translate Theorem 6.1 back to existence results in spaces defined relative to thestationary distribution µ of X under a second thin-tail condition. Suppose that ˜ µ is ab-solutely continuous with respect to the true stationary distribution µ of X , and that µ isabsolutely continuous with respect to ˜ µ . If so, we let ∆ = d˜ µ d µ denote the change of measureof ˜ µ with respect to µ . The second thin-tail condition we require pertains to ∆: E µ [∆( X t ) ε ] < ∞ and E µ [∆( X t ) − ε ] < ∞ for some ε >
0. (20)A sufficient condition for (20) is that log ∆ ∈ L φ . The spaces E φ r and ˜ E φ r are equivalentunder condition (20); see Lemma B.3. We may therefore restate condition (18) aslog ι ∈ E φ r for some r ≥
1. (21)We have the following version of Theorem 6.1 restated for the space E φ r . Theorem 6.2.
Let κ < and conditions (19), (20), and (21) hold. Then: T has a fixedpoint in E φ s and therefore the recursion (14) has a solution v ∈ E φ s for all s ∈ [1 , r ] . Unlike the preceding two results and those in Hansen and Scheinkman (2012), the finalexistence result does not require stationarity of X under ˜ E . However, this result requires afurther eigenvalue condition: βλ κ − < . (22)As λ > Theorem 6.3.
Let κ < and conditions (19), (21), and (22) hold. Then: T has a fixedpoint in E φ s and therefore the recursion (14) has a solution v ∈ E φ s for all s ∈ [1 , r ] . Example: Linear-Gaussian environments.
Consider an environment studied in Sec-tion I.A of Bansal and Yaron (2004), Hansen et al. (2008), and Bansal et al. (2014), amongst22thers. Let X evolve as a stationary Gaussian VAR(1): X t +1 = ν + AX t + u t +1 , u t ∼ N (0 , Σ) , with all eigenvalues of A are inside the unit circle and log( C t +1 /C t ) = δ ′ X t +1 for some vector δ , which is trivially true if log consumption growth is itself a component of X t . Solving (16), ι ( x ) = e (1 − γ ) δ ′ A ( I − A ) − x , λ = e (1 − γ )22 δ ′ ( I − A ) − Σ( I − A ′ ) − δ +(1 − γ ) δ ′ ( I − A ) − ν . To apply Theorem 6.2 we must verify conditions (19), (20), and (21). To verify condition(20), first note ι ( X t +1 )( C t +1 /C t ) − γ λι ( X t ) = e (1 − γ ) δ ′ ( I − A ) − u t +1 − (1 − γ )22 δ ′ ( I − A ) − Σ( I − A ′ ) − δ so the u t are i.i.d. N ((1 − γ ) δ ′ ( I − A ) − Σ , Σ) under ˜ E . Equivalently, under ˜ E we have X t +1 = ν + (1 − γ ) δ ′ ( I − A ) − Σ + AX t + u t +1 , u t ∼ N (0 , Σ) . This implies the stationary distributions µ and ˜ µ are both Gaussian, with different meansbut the same covariance. In consequence, log ∆( x ) is affine in x and so condition (20) holdsfor any ε >
0. As log ι ( x ) is also affine in x , we have that log ι ∈ E φ r for all r ∈ [1 , βe (1 − ρ )(1 − γ )2 δ ′ ( I − A ) − Σ( I − A ′ ) − δ +(1 − ρ ) δ ′ ( I − A ) − ν < . Note also that here log( C t +1 /C t ) ∈ E φ r for all r ∈ [1 , A Truncation affects existence and uniqueness
This appendix provides an example to show truncating the statespace affects existence anduniqueness.Consider the model of Bidder and Smith (2018) and Wachter (2013) from Section 3. Inthat example, condition (6) fails and it is known that there can exist zero, one, or twofixed points of the form v ( x ) = a + b ′ x , where the multiplicity of fixed points depends23n the number of real solutions to a particular quadratic equation. We will now see thattruncating the statespace always results in a unique fixed point, irrespective of non-existenceor non-uniqueness in the original, un-truncated model.The state variable h t , representing the intensity at which disasters arrive, is supported on[0 , ∞ ). Suppose that its support it truncated to [0 , ¯ h ] for some ¯ h < ∞ . Note that the onlyrelevant conditioning variable is h , so it suffices to confine our attention to functions thatdepend on h only. A truncated transition kernel Q ¯ h ( x ′ | x ) ≡ Q ¯ h ( x ′ | h ) may be constructednaturally by restricting Q to the truncated space and renormalizing: Q ¯ h ( x ′ | x ) = Q ( x ′ | x )1l { ≤ h ′ ≤ ¯ h } E Q [1l { ≤ h t +1 ≤ ¯ h }| X t = x ]for each x = ( g, h ) ∈ R × [0 , ¯ h ]. Under this truncation, E Q ¯ h h e αu ( X t ,X t +1 ) (cid:12)(cid:12)(cid:12) X t = x i = exp (cid:18) cν g + c σ h (cid:18) exp (cid:26) cν j + c σ j (cid:27) − (cid:19)(cid:19) which is bounded between exp( cν g + c σ ) and exp( cν g + c σ + ¯ h (exp { cν j + c σ j } − T ¯ h denote the operator T with the true transition kernel Q replaced by the truncatedtransition kernel Q ¯ h . It is straightforward to verify that T ¯ h satisfies Blackwell’s conditionsfor a contraction mapping on the space B ([0 , ¯ h ]) of bounded functions on [0 , ¯ h ] equippedwith the sup-norm. Therefore, T ¯ h has a unique, globally attracting fixed point in B ([0 , ¯ h ])for all ¯ h < ∞ . Existence and uniqueness of a fixed point of T ¯ h in B ([0 , ¯ h ]) holds irrespectiveof the existence or uniqueness of fixed points of the original, un-truncated operator T . B Proofs
B.1 Ancillary results
This first Lemma appears in Chapter 2.3 of the manuscript Pollard (2015) and is usedfrequently to control the Orlicz norm k · k ψ . We include a proof for convenience. Lemma B.1 (Pollard (2015)) . Let E µ [ ψ ( | f ( X t ) | /C )] ≤ C ′ for finite constants C > and C ′ ≥ . Then: k f k ψ ≤ CC ′ . roof of Lemma B.1. Take τ ∈ [0 , ψ : E µ [ ψ ( τ | f ( X t ) | /C )] ≤ τ E µ [ ψ ( | f ( X t ) | /C )] + (1 − τ ) ψ (0) = τ E µ [ ψ ( | f ( X t ) | /C )] . The result follows by setting τ = 1 /C ′ . Lemma B.2 (Karakostas (2008); Chen, Jia, and Jiao (2016)) . Let < p i < ∞ for i ∈ N , and P ∞ i =1 1 p i = 1 . If Q ∞ i =1 k f i k p i < ∞ then Q ∞ i =1 f i is well defined and k Q ∞ i =1 f i k ≤ Q ∞ i =1 k f i k p i . Let µ and ν be two probability measures on a measurable space ( X , X ). We make explicitthe dependence of function classes and norms on the measures µ and ν . Let ∆ = d µ d ν , andlet k ∆ k L p ( ν ) denote its L p ( ν ) norm. Lemma B.3.
Let µ ≪ ν and R ∆ p d ν < ∞ for some p > . Then: E φ r ( ν ) ֒ → E φ r ( µ ) and L φ r ( ν ) ֒ → L φ r ( µ ) for each r ≥ .Proof of Lemma B.3. To see that E φ r ( ν ) ⊆ E φ r ( µ ), take any f ∈ E φ r ( ν ) and c >
0. Then: E µ h e | f ( X ) /c | r i = E ν h ∆( X ) e | f ( X ) /c | r i ≤ k ∆ k L p ( ν ) E ν h e | f ( X ) / ( c/q /r ) | r i q < ∞ , where q > p . Therefore, f ∈ E φ r ( µ ). Similarly, L φ r ( ν ) ⊆ L φ r ( µ ).For continuity of the embedding, take f ∈ L φ r ( ν ) and c = q r k f k φ r ( ν ) . Substituting into theabove display yields E µ [ e | f ( X ) /c | r ] ≤ q k ∆ k L p ( ν ) . Therefore, k f k L φr ( µ ) ≤ ((2 q k ∆ k L p ( ν ) − ∨ k f k L φr ( ν ) by Lemma B.1. B.2 Proofs for Section 2
Proof of Proposition 2.1.
Existence: we prove this for case (a); similar arguments apply for(b). The sequence { ¯ v n } n ≥ with ¯ v n = T n ¯ v is monotone: v ≤ . . . ≤ ¯ v n +1 ≤ ¯ v n ≤ . . . ≤ ¯ v with v, ¯ v ∈ E . The sequence is therefore bounded in E and hence in L because E ֒ → L . It followsby Beppo Levi’s monotone convergence theorem (Malliavin, 1995, Theorem I.7.1) that thereexists v ∈ L such that lim n →∞ ¯ v n = v (almost everywhere) and lim n →∞ k ¯ v n − v k .To strengthen convergence in k · k to convergence in k · k ψ , first observe that v ≤ v ≤ ¯ v v ∈ E . To establish a contradiction, suppose that lim sup n →∞ k ¯ v n − v k ψ ≥ ε forsome ε >
0. Then: lim sup n →∞ Z ψ ( | ¯ v n − v | /ε ) d µ ≥ . (23)Note that { f n } n ∈ N with f n = ψ ( | ¯ v n − v | /ε ) is a monotone sequence of non-negative functionswith lim sup n →∞ f n = 0 (almost everywhere). Moreover, each f n ≤ ψ (( | ¯ v | + | v | + | v | ) /ε )where R ψ (( | ¯ v | + | v | + | v | ) /ε ) d µ < ∞ for each ε > v , v and v all belong to E .Therefore, by reverse Fatou:lim sup n →∞ Z ψ ( | ¯ v n − v | /ε ) d µ ≤ Z lim sup n →∞ ψ ( | ¯ v n − v | /ε ) d µ = 0contradicting (23). It follows that k ¯ v n − v k ψ →
0. Finally, k T v − v k ψ ≤ k T v − T ¯ v n k ψ + k T ¯ v n − v k ψ = k T v − T ¯ v n k ψ + k ¯ v n +1 − v k ψ → T , hence T v = v .Uniqueness: Suppose that v, v ′ ∈ E are fixed points of T . By the subgradient inequality v ′ − v = T v ′ − T v ≥ D v ( v ′ − v )which implies that ( I − D v )( v ′ − v ) ≥ . (24)As ρ ( D v ; E ) <
1, we have ( I − D v ) − = P ∞ i =0 ( D v ) i where the series converges in operatornorm (Kress, 2014, Theorem 10.15). The operator D v is monotone and so ( I − D v ) − is alsomonotone. Applying ( I − D v ) − to both sides of equation (24) yields v ′ − v ≥
0. A parallelargument yields v − v ′ ≥
0. Therefore, v = v ′ . Proof of Corollary 2.1.
By the subgradient inequality, for v, v ′ ∈ V : v ′ − v = T v ′ − T v ≥ D v ( v ′ − v )hence ( I − D v )( v ′ − v ) ≥
0. When ρ ( D v ; E ) <
1, the operator ( I − D v ) is invertible on E with( I − D v ) − = P ∞ n =0 D nv . As D v is monotone, so too is ( I − D v ) − . Applying ( I − D v ) − toboth sides of the above display yields v ′ − v ≥
0, so v is the smallest fixed point of T .Suppose any other v ′ ∈ V distinct from v were also stable. Then we could apply an identicalargument to obtain the reverse inequality v − v ′ ≥
0, a contradiction.26efore proceeding, we present an intermediate result used to prove Lemma 2.1. Note thatcondition (3) implies that (log m ∨ ∈ L φ r ( µ ⊗ Q ), the Orlicz class of functions f : X ×X → R defined relative to the stationary distribution µ ⊗ Q of ( X t , X t +1 ). With slight abuse ofnotation, let k (log m ∨ k φ r denote the corresponding Orlicz norm of (log m ∨ Lemma B.4.
Let ˜ E be of the form (2) and let m satisfy condition (3). Then for any p ∈ (1 , ∞ ) : E µ ⊗ Q [ m ( X t , X t +1 ) np ] /p ≤ e (2 n k (log m ∨ k φr ) rr − (2 p ) r − + 2 p . Moreover, for any β ∈ (0 , there exists C ∈ (0 , ∞ ) and c ∈ (0 , − β ) depending only on β , r , k (log m ∨ k φ r , and p such that the inequality E µ ⊗ Q [ m ( X t , X t +1 ) np ] /p ≤ Ce ( β + c ) − n holds for each n ≥ .Proof of Lemma B.4. First note E µ ⊗ Q [ m ( X t , X t +1 ) np ] ≤ E µ ⊗ Q [ e np | log m ( X t ,X t +1 ) ∨ | ]. To sim-plify notation, let Y t = ( X t , X t +1 ), a = log m ∨
0, and k a k φ r = k (log m ∨ k φ r . All Prstatements are taken with respect to µ ⊗ Q . Let A be a positive constant (specified below)and set | a | = a + + a − with a + = | a | {| a | ≤ A } and a − = | a | {| a | > A } . For any z >
0, wehave Pr (cid:16) e np | a ( Y t ) | ≥ z (cid:17) = ≤ Pr (cid:18) a + ( Y t ) ≥ log z np (cid:19) + Pr (cid:18) a − ( Y t ) ≥ log z np (cid:19) . (25)By Markov’s inequality and definition of k · k φ r , we havePr (cid:18) a − ( Y t ) ≥ log z np (cid:19) ≤ Pr (cid:18) | a ( Y t ) | r ≥ A r − log z np (cid:19) = Pr exp | a ( Y t ) | r k a k rφ r ! ≥ exp k a k rφ r A r − log z np !! ≤ E µ ⊗ Q [exp ( | a ( Y t ) / k a k φ r | r )]exp (cid:16) k a k rφr A r − log z np (cid:17) ≤ − k a k rφ r A r − log z np ! . A = ( k a k rφ r np ) r − , we obtainPr (cid:18) a − ( Y t ) ≥ log z np (cid:19) ≤ z − . As 2 z − ≥ z ≤ √
2, we therefore have Z ∞ Pr (cid:18) a − ( Y t ) ≥ log z np (cid:19) d z ≤ √ Z ∞√ z − d z = 2 . (26)For the first term on the right-hand side of (25), as a + ≤ A we havePr (cid:18) a + ( Y t ) ≥ log z np (cid:19) = 0 if z > e npA . (27)Note 2 npA = (2 np k a k φ r ) rr − r − . Using the fact that E [ Z ] = R ∞ Pr( Z ≥ z ) d z for a non-negative random variable Z , we may deduce from (25), (26), and (27) that E µ ⊗ Q [ m ( X t , X t +1 ) np ] ≤ Z ∞ Pr( e np | a ( Y ) | ≥ z ) d z ≤ e (2 np k a k φr ) rr − r − + 2 . The first assertion follows because ( x + y ) /p ≤ x /p + y /p for x, y ≥ p ≥
1. Thesecond assertion follows as n rr − = o (( β + c ) − n ) for any β ∈ (0 ,
1) and c ∈ (0 , − β ). Proof of Lemma 2.1.
We first show D is a bounded linear operator on L φ s for any s ≥ s ≥ f ∈ L φ s with k f k φ s > q ∈ (0 , E from (2), andH¨older’s inequality with p − + q − = 1, we obtain E µ (cid:20) e | D f ( X t ) / ( q s β k f k φs ) | s (cid:21) = E µ h e q − | ˜ E f ( X t ) / k f k φs | s i ≤ E µ ⊗ Q h m ( X t , X t +1 ) e q − | f ( X t +1 ) / k f k φs | s i ≤ E µ ⊗ Q [ m ( X t , X t +1 ) p ] p E µ h e | f ( X t ) / k f k φs | s i q ≤ q E µ ⊗ Q [ m ( X t , X t +1 ) p ] p , where the final line uses definition of k·k φ s . Note all moments of m are finite under condition283). It follows by Lemma B.1 and definition of the operator norm k D k L φs that k D k L φs ≤ (cid:16)(cid:16) q E µ ⊗ Q [ m ( X t , X t +1 ) p ] p − (cid:17) ∨ (cid:17) q s β < ∞ . That D : E φ s → E φ s may be deduced similarly. Boundedness of D on E φ s now followsbecause E φ s is a closed linear subspace of L φ s .We use Lemma B.4 to establish the spectral radius condition. We prove the result for thespaces L φ s ; the results for E φ s follow because E φ s is a closed linear subspace of L φ s . Firstconsider the case with s >
1. Fix p, q ∈ (1 , ∞ ) with p − + q − = 1. For any f ∈ L φ s with k f k φ s >
0, by two applications of Jensen’s inequality we have E µ (cid:20) e | D n f ( X t ) / ( q s ( β s − s ) n k f k φs ) | s (cid:21) = E µ h e β n q − | ˜ E n f ( X t ) / k f k φs | s i ≤ E µ h e q − | ˜ E n f ( X t ) / k f k φs | s i β n ≤ E µ h ˜ E n g ( X t ) i β n , where g ( x ) = exp( q − | f ( x ) / k f k φ s | s ). By H¨older’s inequality, Lemma B.4, and definition of k · k φ s , we may deduce E µ h ˜ E n g ( X t ) i ≤ E µ ⊗ Q [ m ( X t , X t +1 ) np ] p E µ h e | f ( X t ) / k f k φs | s i q ≤ q Ce ( β + c ) − n for constants C ∈ (0 , ∞ ) and c ∈ (0 , − β ) not depending on f . Therefore, E µ (cid:20) e | D n f ( X t ) / ( q s ( β s − s ) n k f k φs ) | s (cid:21) ≤ (cid:16) q Ce ( β + c ) − n (cid:17) β n . It follows by Lemma B.1 and definition of the operator norm k D n k L φs that k D n k L φs ≤ (cid:18)(cid:18)(cid:16) q Ce ( β + c ) − n (cid:17) β n − (cid:19) ∨ (cid:19) q s ( β s − s ) n and therefore ρ ( D ; L φ s ) ≡ lim n →∞ k D n k /nL φs ≤ β s − s < s = 1. Let c be as in Lemma B.4. Fix any ε ∈ (0 ,
1) and note29hat β < β + εc < β + c <
1. For any f ∈ L φ with k f k φ >
0, we have: E µ h e | D n f ( X t ) / ( qβ n ( β + εc ) − n k f k φ ) | i = E µ h e ( β + εc ) n q − | ˜ E n f ( X t ) / k f k φ | i ≤ E µ h e q − | ˜ E n f ( X t ) / k f k φ | i ( β + εc ) n ≤ E µ h ˜ E n g ( X t ) i ( β + εc ) n , where g ( x ) = exp( q − | f ( x ) | / k f k φ ). By similar arguments to above, we obtain E µ h e | D n f ( X t ) / ( qβ n ( β + εc ) − n k f k φ ) | i ≤ (2 q Ce ( β + c ) − n ) ( β + εc ) n . By Lemma B.1 and definition of the operator norm k D n k L φ , we may deduce that k D n k φ ≤ (cid:16)(cid:16) (2 q Ce ( β + c ) − n ) ( β + εc ) n − (cid:17) ∨ (cid:17) q (cid:18) ββ + εc (cid:19) n , from which it follows similarly that ρ ( D ; L φ ) ≡ lim n →∞ k D n k /nL φ ≤ ββ + εc < B.3 Proofs for Section 3
Proof of Theorem 3.1.
We verify the conditions of Proposition 2.1. For existence, LemmaB.5 shows T is a continuous, monotone, and convex operator on E φ s for each 1 ≤ s ≤ r .Let ¯ v ( x ) = (1 − β ) ∞ X n =0 β n +1 log (cid:0) ( E Q ) n h ( x ) (cid:1) , where h ( x ) = E Q [ e α − β u ( X t ,X t +1 ) | X t = x ]. We first show that E µ [exp( | ¯ v ( X t ) / ( βc ) | r )] < ∞ holds for each c ∈ (0 , P ∞ n =1 (1 − β ) β n = 1and convexity of x e | x/c | r and x e | (log x ) /c | r for c ∈ (0 , E µ h e | ¯ v ( X t ) / ( βc ) | r i = E µ " exp (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) (1 − β ) ∞ X n =0 β n log (cid:0) ( E Q ) n h ( X t ) (cid:1) /c (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) r ! ≤ (1 − β ) ∞ X n =0 β n E µ h exp (cid:16)(cid:12)(cid:12) log (cid:0) ( E Q ) n h ( x ) (cid:1) /c (cid:12)(cid:12) r (cid:17)i ≤ (1 − β ) ∞ X n =0 β n E µ ⊗ Q h e | αc (1 − β ) u ( X t + n ,X t + n +1 ) | r i = E µ ⊗ Q h e | αc (1 − β ) u ( X t + n ,X t + n +1 ) | r i < ∞ . v ∈ E φ r .We now show that T ¯ v ≤ ¯ v . By Holder’s inequality we first have T ¯ v ( X t ) ≤ β log (cid:18) E Q h e ¯ v ( X t +1 ) /β (cid:12)(cid:12)(cid:12) X t i β E Q h e α − β u ( X t ,X t +1 ) (cid:12)(cid:12)(cid:12) X t i − β (cid:19) = β log E Q [ e ¯ v ( X t +1 ) /β | X t ] + (1 − β ) β log h ( X t ) . (28)By Lemma B.2, we may deducelog E Q h e ¯ v ( X t +1 ) /β (cid:12)(cid:12)(cid:12) X t i = log E Q " ∞ Y n =0 (cid:0) ( E Q ) n h ( X t +1 ) (cid:1) (1 − β ) β n (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) X t ≤ log ∞ Y n =0 E Q (cid:2) (cid:0) ( E Q ) n h ( X t +1 ) (cid:1)(cid:12)(cid:12) X t (cid:3) (1 − β ) β n ! = (1 − β ) ∞ X n =1 β n − log (cid:0) ( E Q ) n h ( X t ) (cid:1) . (29)Substituting (29) into (28) yields T ¯ v ≤ ¯ v .We now show { T n ¯ v } n ≥ is bounded from below, first observe that T f ( x ) = β log E Q [ e f ( X t +1 )+ αu ( X t ,X t +1 ) | X t = x ] ≥ β E Q [ f ( X t +1 ) + αu ( X t , X t +1 ) | X t = x ] . Therefore, T n ¯ v ≥ ( β E Q ) n ¯ v + n − X s =0 ( β E Q ) s ( h )for each n ≥
1, where h ( x ) = β E Q [ αu ( X t , X t +1 ) | X t = x ]. Note also that k β E Q k E φr = β and ρ ( β E Q ; E φ r ) = β (see Section 2.3), and so we obtain lim inf n →∞ T n ¯ v ≥ ( I − β E Q ) − h ∈ E φ r .Uniqueness: v is a fixed point of T : E φ s → E φ s for each s ∈ [1 , r ]. Moreover, T : E φ s → E φ s is convex by Lemma B.5 and D v is a bounded, monotone linear operator with ρ ( D v ; E φ s ) < s ∈ [1 , r ] by Lemma B.6. Uniqueness in E φ s with s ∈ (1 , r ] follows by Proposition 2.1(ii).That v is the smallest and unique stable fixed point in E φ follows by Corollary 2.1. Lemma B.5.
Let condition (6) hold. Then: T is a continuous, monotone and convex op-erator on E φ s for each ≤ s ≤ r .Proof of Lemma B.5. Fix any 1 ≤ s ≤ r . Take any f ∈ E φ s and c ∈ (0 , e | (log x ) /c | s for c ∈ (0 ,
1] and Jensen’s inequality: E µ [exp( | T f ( X t ) / ( βc ) | s )] = E µ (cid:20) exp (cid:18)(cid:12)(cid:12)(cid:12)(cid:12) c log E Q h e f ( X t +1 )+ αu ( X t ,X t +1 ) (cid:12)(cid:12)(cid:12) X t i(cid:12)(cid:12)(cid:12)(cid:12) s (cid:19)(cid:21) ≤ E µ (cid:20) E Q (cid:20) exp (cid:18)(cid:12)(cid:12)(cid:12)(cid:12) c log e f ( X t +1 )+ αu ( X t ,X t +1 ) (cid:12)(cid:12)(cid:12)(cid:12) s (cid:19)(cid:12)(cid:12)(cid:12)(cid:12) X t (cid:21)(cid:21) = E µ ⊗ Q (cid:20) exp (cid:18)(cid:12)(cid:12)(cid:12)(cid:12) f ( X t +1 ) + αu ( X t , X t +1 ) c (cid:12)(cid:12)(cid:12)(cid:12) s (cid:19)(cid:21) < ∞ which is finite for any f ∈ E φ s under condition (6). Therefore, T : E φ s → E φ s .Continuity: Fix any f ∈ E φ s . Take g ∈ E φ s with k g k φ s ∈ (0 , − /s ] and set c = 2 /s k g k φ s . Byconvexity of x e | (log x ) /c | s for c ∈ (0 ,
1] and the Jensen and Cauchy-Schwarz inequalities, E µ [ φ s ( | T ( f + g )( X t ) − T f ( X t ) | / ( βc ))] + 1 = E µ (cid:20) exp (cid:18)(cid:12)(cid:12)(cid:12)(cid:12) c log E f h e g ( X t +1 ) (cid:12)(cid:12)(cid:12) X t i(cid:12)(cid:12)(cid:12)(cid:12) s (cid:19)(cid:21) ≤ E µ (cid:20) E f (cid:20) exp (cid:18)(cid:12)(cid:12)(cid:12)(cid:12) c log e g ( X t +1 ) (cid:12)(cid:12)(cid:12)(cid:12) s (cid:19)(cid:12)(cid:12)(cid:12)(cid:12) X t (cid:21)(cid:21) = E µ ⊗ Q (cid:20) m f ( X t , X t +1 ) exp (cid:18)(cid:12)(cid:12)(cid:12)(cid:12) g ( X t +1 ) c (cid:12)(cid:12)(cid:12)(cid:12) s (cid:19)(cid:21) ≤ E µ (cid:2) e | g ( X t ) /c | s (cid:3) / E µ ⊗ Q [ m f ( X t , X t +1 ) ] / = q E µ ⊗ Q [ m f ( X t , X t +1 ) ]because c = 2 /s k g k φ s . Finiteness of E µ ⊗ Q [ m f ( X t , X t +1 ) ] holds for any f ∈ E φ s under (6).Continuity now follows by Lemma B.1. Monotonicity follows from monotonicity of exp( · ),log( · ), and conditional expectations. Convexity follows by applying H¨older’s inequality to E Q h e τ ( v ( X t +1 )+ αu ( X t ,X t +1 ))+(1 − τ )( v ( X t +1 )+ αu ( X t ,X t +1 )) (cid:12)(cid:12)(cid:12) X t = x i with p = τ − and q = (1 − τ ) − . Lemma B.6.
Let condition (6) hold and fix any v ∈ E φ r ′ with r ′ > . Then: each s ≥ , D v is a continuous linear operator on E φ s with ρ ( D v ; E φ s ) < .Proof of Lemma B.6. We verify condition (3) from Lemma 2.1. The log change-of-measureis log m v ( X t , X t +1 ) = v ( X t +1 ) + αu ( X t , X t +1 ) − log E Q [ e v ( X t +1 )+ αu ( X t ,X t +1 ) | X t ] . v ∈ E φ r ′ with r ′ >
1, setting r = ( r ∧ r ′ ) > c ∈ (0 , E µ h e | log E Q [ e v ( Xt +1)+ αu ( Xt,Xt +1) | X t ] /c | r i ≤ E µ ⊗ Q h e | ( v ( X t +1 )+ αu ( X t ,X t +1 )) /c | r i by Jensen’s inequality. The right-hand side is finite by condition (6). Therefore, E µ ⊗ Q h e | log m v ( X t ,X t +1 ) /c | r i < ∞ for any c ∈ (0 , B.4 Proofs for Section 4
Recall ˆ X t = ( ˆ ξ t , ϕ t ). The conditional distribution ˆ Q of ( ξ t , ˆ X t +1 ) given ˆ X t may be repre-sented by E ˆ Q [ h ( ξ t , ˆ X t +1 ) | ˆ X t ] = E ˆ Q [ h ( ξ t , ˆ X t +1 ) | ˆ ξ t ] = E Π ξ ⊗ Q ϕ [ h ( ξ t , ϕ t +1 , Ξ( ˆ ξ t , ϕ t +1 )) | ˆ ξ t ] . Recall that µ is the stationary distribution of ˆ X t under ˆ Q . For v ∈ E φ ˆ ξ , define m Π ξ v ( ξ t , ˆ ξ t ) = E Q ϕ h e θϑ v (Ξ(ˆ ξ t ,ϕ t +1 ))+ αu ( ϕ t +1 ) (cid:12)(cid:12)(cid:12) ξ t , ˆ ξ t i ϑθ E Π ξ (cid:20) E Q ϕ h e θϑ v (Ξ(ˆ ξ t ,ϕ t +1 ))+ αu ( ϕ t +1 ) (cid:12)(cid:12)(cid:12) ξ t , ˆ ξ t i ϑθ (cid:12)(cid:12)(cid:12)(cid:12) ˆ ξ t (cid:21) m Q ϕ v ( ξ t , ˆ ξ t , ϕ t +1 ) = e θϑ v (Ξ(ˆ ξ t ,ϕ t +1 ))+ αu ( ϕ t +1 ) E Q ϕ h e θϑ v (Ξ(ˆ ξ t ,ϕ t +1 ))+ αu ( ϕ t +1 ) (cid:12)(cid:12)(cid:12) ξ t , ˆ ξ t i . The quantity m Π ξ v distorts the posterior distribution for ξ t given ˆ X t whereas m Q ϕ v distortsthe conditional distribution Q ϕ . To simplify notation, define the distorted conditional ex-pectations E Π ξ v and E Q ϕ v by E Π ξ v f ( ˆ ξ ) = E Π ξ h m Π ξ v ( ξ t , ˆ ξ t ) f ( ξ t , ˆ ξ t ) (cid:12)(cid:12)(cid:12) ˆ ξ t = ˆ ξ i , E Q ϕ v f ( ξ, ˆ ξ ) = E Π ξ h m Q ϕ v ( ξ t , ˆ ξ t , ϕ t +1 ) f ( ξ t , ˆ ξ t , ϕ t +1 ) (cid:12)(cid:12)(cid:12) ξ t = ξ, ˆ ξ t = ˆ ξ i . The subgradient of T at v is the composition of these two distorted conditional expectations,discounted by β : D v f ( ˆ ξ ) = β E ˆ Q h m v ( ξ t , ˆ ξ t , ϕ t +1 ) f ( ˆ ξ t +1 ) (cid:12)(cid:12)(cid:12) ˆ ξ t = ˆ ξ i (30)where m v ( ξ t , ˆ ξ t , ϕ t +1 ) = m Π ξ v ( ξ t , ˆ ξ t ) m Q ϕ v ( ξ t , ˆ ξ t , ϕ t +1 ).33 roof of Theorem 4.1. We verify the conditions of Proposition 2.1. Lemma B.7 shows that T is a continuous, monotone, and convex operator on E φ s ˆ ξ for each 1 ≤ s ≤ r . If θ < ϑ , let¯ v ( ˆ ξ ) = (1 − β ) ∞ X n =0 β n +1 log (cid:18)(cid:16) E ˆ Q (cid:17) n +1 g ( ˆ ξ ) (cid:19) , where g ( ˆ X t ) = exp( αϑ (1 − β ) θ u ( ϕ t )). For any c >
0, by Jensen’s inequality we may deduce E µ [ e | ¯ v (ˆ ξ t ) / ( βc ) | r ] ≤ (1 − β ) ∞ X n =0 β n E µ (cid:20)(cid:18)(cid:16) E ˆ Q (cid:17) n +1 g r ( ˆ ξ t ) (cid:19)(cid:21) , where g r ( ˆ X t ) = exp( | αϑ (1 − β ) θc u ( ϕ t ) | r ). As u ∈ E φ r ϕ , the right-hand side of the precedingdisplay is finite and so ¯ v ∈ E φ r ˆ ξ .To show T ¯ v ≤ ¯ v , first by the Jensen and H¨older inequalities, T ¯ v ( ˆ ξ ) = β log E Π ξ (cid:20) E Q ϕ h e θϑ ¯ v (Ξ(ˆ ξ t ,ϕ t +1 ))+ αu ( ϕ t +1 ) (cid:12)(cid:12)(cid:12) ξ t , ˆ ξ t i ϑ/θ (cid:12)(cid:12)(cid:12)(cid:12) ˆ ξ t = ˆ ξ (cid:21) ≤ β log E ˆ Q h e ¯ v (ˆ ξ t +1 )+ α ϑθ u ( ϕ t +1 ) (cid:12)(cid:12)(cid:12) ˆ ξ t = ˆ ξ i ≤ β log E ˆ Q h e ¯ v (ˆ ξ t +1 ) /β (cid:12)(cid:12)(cid:12) ˆ ξ t = ˆ ξ i + β (1 − β ) log E ˆ Q h e αϑ (1 − β ) θ u ( ϕ t +1 ) (cid:12)(cid:12)(cid:12) ˆ ξ t = ˆ ξ i . By Lemma B.2, we may deducelog E ˆ Q h e ¯ v (ˆ ξ t +1 ) /β (cid:12)(cid:12)(cid:12) ˆ ξ t = ˆ ξ i ≤ (1 − β ) ∞ X n =1 β n − log (cid:18)(cid:16) E ˆ Q (cid:17) n +1 g ( ˆ ξ ) (cid:19) , hence T ¯ v ≤ ¯ v .On the other hand, if ϑ ≤ θ , let ¯ v ( ˆ ξ ) = ϑθ (1 − β ) P ∞ n =0 β n +1 log(( E ˆ Q ) n +1 g ( ˆ ξ )) where g ( ˆ X t ) = e α − β u ( ϕ t ) . By similar arguments to above, we may use the condition u ∈ E φ r ϕ todeduce ¯ v ∈ E φ r ˆ ξ . Again by the Jensen and H¨older inequalities, T ¯ v ( ˆ ξ ) = β log E Π ξ (cid:20) E Q ϕ h e θϑ ¯ v (Ξ(ˆ ξ t ,ϕ t +1 ))+ αu ( ϕ t +1 ) (cid:12)(cid:12)(cid:12) ˆ ξ t , ξ t i ϑθ (cid:12)(cid:12)(cid:12)(cid:12) ˆ ξ t = ˆ ξ (cid:21) ≤ ϑθ β log E ˆ Q h e θϑ ¯ v (ˆ ξ t +1 )+ αu ( ϕ t +1 ) (cid:12)(cid:12)(cid:12) ˆ ξ t = ˆ ξ i ≤ ϑθ β log E ˆ Q h e θϑ ¯ v (ˆ ξ t +1 ) /β (cid:12)(cid:12)(cid:12) ˆ ξ t = ˆ ξ i + ϑθ β (1 − β ) log E ˆ Q h e α − β u ( ϕ t +1 ) (cid:12)(cid:12)(cid:12) ˆ ξ t = ˆ ξ i . The inequality T ¯ v ≤ ¯ v now follows by similar arguments to the previous case.34o show that the sequence of iterates T n ¯ v is bounded from below, first note that for any f ∈ E φ r ˆ ξ , we have T f ( ˆ ξ ) ≥ β E ˆ Q (cid:20) f ( ˆ ξ t +1 ) + α ϑθ u ( ϕ t +1 ) (cid:12)(cid:12)(cid:12)(cid:12) ˆ ξ t = ˆ ξ (cid:21) which follows by several applications of Jensen’s inequality. It follows that T n ¯ v ( ˆ ξ ) ≥ (cid:16) β E ˆ Q (cid:17) n ¯ v ( ˆ ξ ) + n − X i =0 (cid:16) β E ˆ Q (cid:17) i g ( ˆ ξ )where g ( ˆ ξ ) = β E ˆ Q [ α ϑθ u ( ϕ t +1 ) | ˆ ξ t = ˆ ξ ] ∈ E φ r ˆ ξ . Note also that ρ ( β E ˆ Q ; E φ r ) = β (see Section2.3), hence lim inf n →∞ T n ¯ v ≥ ( I − β E ˆ Q ) − g ∈ E φ r . This completes the proof of existence.For uniqueness, v is necessarily a fixed point of T : E φ s ˆ ξ → E φ s ˆ ξ for each 1 ≤ s ≤ r .The subgradient D v is monotone. Lemma B.8 shows D v : E φ s ˆ ξ → E φ s ˆ ξ is bounded and ρ ( D v ; E φ s ˆ ξ ) < s ∈ [1 , r ]. Uniqueness follows by Proposition 2.1(ii) and Corollary 2.1. Lemma B.7.
Let condition (9) hold. Then: T is a continuous, monotone, and convexoperator on E φ s ˆ ξ for each ≤ s ≤ r .Proof of Lemma B.7. Fix s ∈ [1 , r ]. We first show E µ [exp( | T f ( ˆ ξ t ) / ( βc ) | s )] < ∞ holds foreach f ∈ E φ s ˆ ξ and c ∈ (0 , ϑθ ∧ x e | (log x ) /c | s for c ∈ (0 ,
1] and Jensen’sinequality, E µ " exp (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) T f ( ˆ ξ t ) βc (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) s ! = E µ (cid:20) exp (cid:18) c s (cid:12)(cid:12)(cid:12)(cid:12) log E Π ξ (cid:20) E Q ϕ h e θϑ f (Ξ(ˆ ξ t ,ϕ t +1 ))+ αu ( ϕ t +1 ) (cid:12)(cid:12)(cid:12) ξ t , ˆ ξ t i ϑθ (cid:12)(cid:12)(cid:12)(cid:12) ˆ ξ t (cid:21)(cid:12)(cid:12)(cid:12)(cid:12) s (cid:19)(cid:21) ≤ E µ (cid:20) E Π ξ (cid:20) exp (cid:18) c s (cid:12)(cid:12)(cid:12)(cid:12) log E Q ϕ h e θϑ f (Ξ(ˆ ξ t ,ϕ t +1 ))+ αu ( ϕ t +1 ) (cid:12)(cid:12)(cid:12) ξ t , ˆ ξ t i ϑθ (cid:12)(cid:12)(cid:12)(cid:12) s (cid:19)(cid:12)(cid:12)(cid:12)(cid:12) ˆ ξ t (cid:21)(cid:21) ≤ E µ (cid:20) E Π ξ (cid:20) E Q ϕ (cid:20) exp (cid:18) c s (cid:12)(cid:12)(cid:12)(cid:12) ϑθ log e θϑ f (Ξ(ˆ ξ t ,ϕ t +1 ))+ αu ( ϕ t +1 ) (cid:12)(cid:12)(cid:12)(cid:12) s (cid:19)(cid:12)(cid:12)(cid:12)(cid:12) ξ t , ˆ ξ t (cid:21)(cid:12)(cid:12)(cid:12)(cid:12) ˆ ξ t (cid:21)(cid:21) = E µ ⊗ Π ξ ⊗ Q ϕ (cid:20) exp (cid:18) c s (cid:12)(cid:12)(cid:12)(cid:12) f (Ξ( ˆ ξ t , ϕ t +1 )) + ϑcθ αu ( ϕ t +1 ) (cid:12)(cid:12)(cid:12)(cid:12) s (cid:19)(cid:21) which is finite because f ∈ E φ s ˆ ξ and u ∈ E φ r ϕ . Therefore, T : E φ s ˆ ξ → E φ s ˆ ξ .For continuity, fix f ∈ E φ s ˆ ξ . Take g ∈ E φ s ˆ ξ with 0 < k g k φ s ≤ − /s (1 ∧ ϑθ ) and set c =35 /s k g k φ s . Note T ( f + g )( ˆ ξ ) − T f ( ˆ ξ ) = β log (cid:18) E Π ξ f (cid:20) E Q ϕ f h e θϑ g (Ξ(ˆ ξ t ,ϕ t +1 )) (cid:12)(cid:12)(cid:12) ξ t , ˆ ξ t i ϑθ (cid:12)(cid:12)(cid:12)(cid:12) ˆ ξ t = ˆ ξ (cid:21)(cid:19) . By similar arguments to the above, we may deduce E µ " exp (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) T ( f + g )( ˆ ξ t ) − T f ( ˆ ξ t ) βc (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) s ! ≤ E µ (cid:20) E Π ξ f (cid:20) E Q ϕ f (cid:20) exp (cid:18)(cid:12)(cid:12)(cid:12)(cid:12) c g (Ξ( ˆ ξ t , ϕ t +1 )) (cid:12)(cid:12)(cid:12)(cid:12) s (cid:19)(cid:12)(cid:12)(cid:12)(cid:12) ξ t , ˆ ξ t (cid:21)(cid:12)(cid:12)(cid:12)(cid:12) ˆ ξ t (cid:21)(cid:21) = E µ (cid:20) E ˆ Q (cid:20) m f ( ξ t , ˆ ξ t , ϕ t +1 ) exp (cid:18)(cid:12)(cid:12)(cid:12)(cid:12) c g (Ξ( ˆ ξ t , ϕ t +1 )) (cid:12)(cid:12)(cid:12)(cid:12) s (cid:19)(cid:12)(cid:12)(cid:12)(cid:12) ˆ ξ t (cid:21)(cid:21) ≤ E µ ⊗ ˆ Q h m f ( ξ t , ˆ ξ t , ϕ t +1 ) i / E µ h exp(2 | g ( ˆ ξ t +1 ) /c | s i / ≤ (cid:16) E µ ⊗ ˆ Q h m f ( ξ t , ˆ ξ t , ϕ t +1 ) i(cid:17) / , because c = 2 /s k g k φ s . The expectation on the right-hand side is finite because f ∈ E φ s ˆ ξ and u ∈ E φ r ϕ . It follows by Lemma B.1 that k T ( f + g ) − T f k φ s → k g k φ s → Lemma B.8.
Let condition (9) hold. Fix any v ∈ E φ r ′ ˆ ξ with r ′ > . Then: for each s ≥ , D v is a continuous linear operator on E φ s ˆ ξ with ρ ( D v ; E φ s ˆ ξ ) < .Proof of Lemma B.8. It suffices to verify the conditions of Lemma 2.1. Note that the processˆ ξ = { ˆ ξ t } t ∈ T is a stationary Markov process (this follows from our maintained assumptionsthat learning is in a steady state and the conventional hidden Markov structure on X ). Byiterated expectations, we may rewrite the subgradient from (30) as D v f ( ˆ ξ ) = β E ˆ Q h ¯ m v ( ˆ ξ t , ˆ ξ t +1 ) f ( ˆ ξ t +1 ) (cid:12)(cid:12)(cid:12) ˆ ξ t = ˆ ξ i where ¯ m v ( ˆ ξ t , ˆ ξ t +1 ) denotes the conditional expectation of m v ( ξ t , ˆ ξ t , ϕ t +1 ) given ˆ ξ t , ˆ ξ t +1 underˆ Q . The thin-tail condition on m v then follows by similar arguments to the proof of LemmaB.6 for any v ∈ E φ r ′ ˆ ξ with r ′ >
1. 36 .5 Proofs for Section 5
Define S f ( x ) = log D D X d =1 e u d ( x )+ β E M [ f ( X t +1 ) | X t = x,D t = d ] ! , i.e., S f = T f − log D − γ EM . It suffices to derive the existence and uniqueness results for S rather than T as their fixed points differ only by translation by a constant. The operator S satisfies a subgradient inequality with subgradient D v defined in Section 5. Proof of Theorem 5.1.
We prove the result by applying Proposition 2.1. Lemma B.9 estab-lishes that S is a continuous, monotone, and convex operator on E φ s for each 1 ≤ s ≤ r .Let U ( x ) = D P Dd =1 exp( u d ( x )1 − β ) and define¯ v ( x ) = (1 − β ) ∞ X n =0 β n log (cid:0) ( E Q ) n U ( x ) (cid:1) . We may deduce that ¯ v ∈ E φ r using the condition u , . . . , u D ∈ E φ r . To see that T ¯ v ≤ ¯ v ,first note that by H¨older’s inequality and Jensen’s inequality: S f ( x ) ≤ log D D X d =1 e ud ( x )1 − β ! − β D D X d =1 e E M [ f ( X t +1 ) | X t = x,D t = d ] ! β = (1 − β ) log U ( x ) + β log D D X d =1 e E M [ f ( X t +1 ) | X t = x,D t = d ] ! ≤ (1 − β ) log U ( x ) + β log (cid:16) E Q h e f ( X t +1 ) (cid:12)(cid:12)(cid:12) X t = x i(cid:17) . Substituting in the above expression for ¯ v and using Lemma B.2, we obtain: S ¯ v ( x ) ≤ (1 − β ) log U ( x ) + β log (cid:16) E Q h e (1 − β ) P ∞ n =0 β n log ( ( E Q ) n U ( X t +1 ) ) (cid:12)(cid:12)(cid:12) X t = x i(cid:17) = (1 − β ) log U ( x ) + β log E Q " ∞ Y n =0 (cid:0) ( E Q ) n U ( X t +1 ) (cid:1) (1 − β ) β n (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) X t = x ≤ (1 − β ) log U ( x ) + β log ∞ Y n =0 E Q (cid:2) ( E Q ) n U ( X t +1 ) (cid:12)(cid:12) X t = x (cid:3) (1 − β ) β n ! = (1 − β ) log U ( x ) + (1 − β ) ∞ X n =0 β n +1 log (cid:0) ( E Q ) n +1 U ( x ) (cid:1) = ¯ v ( x )37s required. To see that the sequence S n ¯ v is bounded from below, observe that: S ¯ v ( x ) ≥ D D X d =1 (cid:0) u d ( x ) + β E M [ ¯ v ( X t +1 ) | X t = x, D t = d ] (cid:1) =: u ( x ) + β E Q ¯ v ( x )where u = D P Dd =1 u d ∈ E φ r . It follows by induction that S n ¯ v ≥ P n − j =0 ( β E Q ) j u + ( β E Q ) n ¯ v .As ρ ( β E Q ; E φ r ) = β <
1, we may deduce: lim inf n →∞ S n ¯ v ≥ ( I − β E Q ) − u ∈ E φ r . Applyingpart (i) of Proposition 2.1 establishes existence of a fixed point v ∈ E φ r .For uniqueness, as 0 ≤ w d,v ≤
1, we have | m v ( X t , X t +1 ) | ≤ D at any v ∈ E φ s . It follows byLemma 2.1 that D v is a continuous, linear operator on E φ s with ρ ( D v ; E φ s ) <
1. Uniquenessnow follows by Proposition 2.1(ii).
Lemma B.9. S is a continuous, monotone, and convex operator on E φ s for each ≤ s ≤ r .Proof of Lemma B.9. First, take any f ∈ E φ s and any c ∈ (0 , E µ [exp( | S f ( X t ) /c | s )] = E µ " exp (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) c log D D X d =1 e u d ( X t )+ β E M [ f ( X t +1 ) | X t ,D t = d ] !(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) s ! ≤ E µ " D D X d =1 e | E M [ c ( u d ( X t )+ βf ( X t +1 )) | X t ,D t = d ] | s ≤ E µ " D D X d =1 E M h e | c ( u d ( X t )+ βf ( X t +1 )) | s (cid:12)(cid:12)(cid:12) X t , D t = d i ≤ E µ " D D X d =1 E M h e s − | c u d ( X t ) | s +2 s − | c βf ( X t +1 ) | s (cid:12)(cid:12)(cid:12) X t , D t = d i ≤ E µ ⊗ Q h e s − P Dd =1 | c u d ( X t ) | s +2 s − | c βf ( X t +1 ) | s i where the first and second inequalities are by Jensen’s inequality and convexity of x e | (log x ) /c | s for c ∈ (0 , c p inequality, and the fourth is by the triangleinequality. The right-hand side is finite because f ∈ E φ s and u d ∈ E φ r for each 1 ≤ d ≤ D .To verify continuity, take f ∈ E φ s and g ∈ E φ s with k g k φ s ≤ c = k g k φ s . Then: S ( f + g )( x ) − S f ( x ) = log (cid:18) D X d =1 w d,f ( x ) e β E M [ g ( X t +1 ) | X t = x,D t = d ] (cid:19) w d,f ( x ) is defined in equation (13). Therefore E µ h e | ( S ( f + g )( X t ) − S f ( X t )) / ( βc ) | s i = E µ " exp (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) βc log D X d =1 w d,f ( X t ) e β E M [ g ( X t +1 ) | X t ,D t = d ] !(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) s ! ≤ E µ " D X d =1 w d,f ( X t ) e | c E M [ g ( X t +1 ) | X t ,D t = d ] | s ≤ E µ " D X d =1 e | c E M [ g ( X t +1 ) | X t ,D t = d ] | s ≤ E µ " D X d =1 E M h e | c g ( X t +1 ) | s (cid:12)(cid:12)(cid:12) X t , D t = d i = D × E µ h e | c g ( X t ) | s i where the first and third inequalities are by Jensen’s inequality, the second inequality isbecause 0 ≤ w d,f ( x ) ≤
1, and the final line is by stationarity. It follows by taking c = k g k φ s and applying Lemma B.1 that k S ( f + g ) − S f k φ s ≤ (2 D − β k g k φ s , verifying continuity. B.6 Proof for Section 6
Proof of Theorem 6.1.
In view of the discussion preceding Theorem 6.1 and Lemma B.10,it suffices to show that ¯ v ∈ ˜ E φ r and that T ¯ v ≤ ¯ v . By (19), convexity of x e | (log x ) /c | r for c ∈ (0 , c ∈ (0 ,
1] we have E ˜ µ h e | ¯ v ( X t ) /c | r i = E ˜ µ (cid:20) e (cid:12)(cid:12)(cid:12) log (cid:16) (1 − β ) P ∞ n =0 ( βλ κ ) n ˜ E n ( ι − κ )( X t ) (cid:17) /c (cid:12)(cid:12)(cid:12) r (cid:21) ≤ (1 − βλ κ ) ∞ X n =0 ( βλ κ ) n E ˜ µ h ˜ E n exp (cid:16)(cid:12)(cid:12)(cid:12) log (cid:16) (1 − β )(1 − βλ κ ) − ( ι − κ )( X t ) (cid:17) /c (cid:12)(cid:12)(cid:12) r (cid:17)i = E ˜ µ h exp (cid:16)(cid:12)(cid:12)(cid:12) log ι ( X t ) / ( κc (1 − β ) − (1 − βλ κ )) (cid:12)(cid:12)(cid:12) r (cid:17)i < ∞ by condition (18), with the final equality because ˜ µ is the stationary distribution corre-sponding to ˜ E . Therefore, ¯ v ∈ E φ r . 39o see that T ¯ v ≤ ¯ v , first observe that by Jensen’s inequality (as κ < T ¯ v ( x ) = log (1 − β ) ι ( x ) − κ + βλ κ ˜ E " (1 − β ) ∞ X n =0 ( βλ κ ) n ˜ E n ( ι − κ )( X t +1 ) ! κ (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) X t = x κ ≤ log (1 − β ) ι ( x ) − κ + βλ κ ˜ E " (1 − β ) ∞ X n =0 ( βλ κ ) n ˜ E n ( ι − κ )( X t +1 ) (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) X t = x = log (1 − β ) ι ( x ) − κ + (1 − β ) ∞ X n =1 ( βλ κ ) n ˜ E n ( ι − κ )( x ) ! = ¯ v ( x ) . Existence now follows by Proposition 2.1(i).
Lemma B.10.
Let condition (18) hold. Then the operator T from (17) is a continuous,monotone operator on ˜ E φ s for each ≤ s ≤ r .Proof of Lemma B.10. Fix any s ∈ [1 , r ]. We first show that E µ [ e | T f ( X t ) /c | s ] < ∞ holds forany f ∈ ˜ E φ s and c sufficiently small. By convexity of x e | (log x ) /c | s for c ∈ (0 ,
1] and twoapplications of Jensen’s inequality and iterated expectations, for any c ∈ (0 , ∧ | κ | − ] weobtain E ˜ µ h e | T f ( X t ) /c | s i = E ˜ µ h exp (cid:16)(cid:12)(cid:12)(cid:12) log (cid:16) (1 − β ) ι ( X t ) − κ + βλ κ ˜ E [ e κf ( X t +1 ) | X t ] κ (cid:17) /c (cid:12)(cid:12)(cid:12) s (cid:17)i ≤ E ˜ µ h (1 − β ) e | log ι ( X t ) / ( κc ) | s + β exp (cid:16)(cid:12)(cid:12)(cid:12) log (cid:16) ˜ E [ λe κf ( X t +1 ) | X t ] (cid:17) / ( κc ) (cid:12)(cid:12)(cid:12) s (cid:17)i ≤ E ˜ µ h (1 − β ) e | log ι ( X t ) / ( κc ) | s + β ˜ E h exp (cid:16)(cid:12)(cid:12)(cid:12) log (cid:16) λe κf ( X t +1 ) (cid:17) / ( κc ) (cid:12)(cid:12)(cid:12) s (cid:17)(cid:12)(cid:12)(cid:12) X t ii = (1 − β ) E ˜ µ h e | log ι ( X t ) / ( κc ) | s i + β E ˜ µ h e | (log λ ) / ( κc )+ f ( X t ) /c | s i , where the right-hand side is finite under condition (18), and the final equality is because ˜ µ is the stationary distribution corresponding to ˜ E . Therefore, T : ˜ E φ s → ˜ E φ s .For continuity, fix f ∈ ˜ E φ s and take any h ∈ ˜ E φ s with k h k φ s (with the norm defined relativeto the measure ˜ µ ) sufficiently small in a sense we make precise below. Then T ( f + h )( x ) − T f ( x ) = log ( (1 − β ) ι ( x ) − κ + βλ κ w ( x )˜ E f [ e κh ( X t +1 ) | X t = x ] κ (1 − β ) ι ( x ) − κ + βλ κ w ( x ) ) where w ( x ) = ˜ E (cid:2) e κf ( X t +1 ) (cid:12)(cid:12) X t = x (cid:3) /κ and ˜ E f denotes the distorted conditional expectation40perator ˜ E f g ( x ) := ˜ E [ m f ( X t , X t +1 ) g ( X t +1 ) | X t = x ] where m f ( X t , X t +1 ) = e κf ( X t +1 ) ˜ E [ e κf ( X t +1 ) | X t ] . Take any c ∈ (0 , ∧ | κ | − ]. By convexity of x e | (log x ) /c | s for c ∈ (0 , E ˜ µ h e | ( T ( f + h )( X t ) − T f ( X t )) /c | s i = E ˜ µ exp (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) c log (1 − β ) ι ( X t ) − κ + βλ κ w ( X t )˜ E f (cid:2) e κh ( X t +1 ) (cid:12)(cid:12) X t (cid:3) κ (1 − β ) ι ( X t ) − κ + βλ κ w ( X t ) (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) s ≤ E ˜ µ " (1 − β ) ι ( X t ) − κ + βλ κ w ( X t ) e | cκ log ˜ E f [ e κh ( Xt +1) | X t ] | s (1 − β ) ι ( X t ) − κ + βλ κ w ( X t ) ≤ E ˜ µ h e | cκ log ˜ E f [ e κh ( Xt +1) | X t ] | s i ≤ E ˜ µ h ˜ E f h e | c h ( X t +1 ) | s (cid:12)(cid:12)(cid:12) X t ii ≤ E ˜ µ h e | κf ( X t ) | i E ˜ µ h e | c h ( X t ) | s i . For h ∈ E φ s with k h k φ s ≤ (1 ∧ | κ | − ), setting c = 2 k h k φ s we therefore have E µ h e | ( T ( f + h )( X t ) − T f ( X t )) / (2 k h k φs ) | s i ≤ (cid:16) E ˜ µ h e | κf ( X t ) | i(cid:17) . Continuity now follows by Lemma B.1. Monotonicity of T follows form monotonicity ofconditional expectations and monotonicity of the log and exp functions. Proof of Theorem 6.2.
Immediate from Theorem 6.1 and Lemma B.3.
Proof of Theorem 6.3.
The proof follows by similar arguments to Theorem 6.1, we list onlythe modifications here. First use (22) and the definition of ˜ E to rewrite ¯ v as¯ v ( x ) = log (1 − β ) ∞ X n =0 ( βλ κ − ) n E Q ⊗ n [ ι ( X t + n ) κ − κ | X t = x ] ι ( x ) − ! . Let b = (1 − β )(1 − βλ κ − ) − . By (22), convexity of x e | (log x ) /c | r for c ∈ (0 , c ∈ (0 ,
1] we have E µ h e | ¯ v ( X t ) /c | r i ≤ E µ " (1 − βλ κ − ) ∞ X n =0 ( βλ κ − ) n e (cid:12)(cid:12)(cid:12)(cid:12) log (cid:18) b E Q ⊗ n [ ι ( X t + n ) κ − κ | X t ] ι ( X t ) − (cid:19) /c (cid:12)(cid:12)(cid:12)(cid:12) r ≤ (1 − βλ κ − ) ∞ X n =0 ( βλ κ − ) n E µ h E Q ⊗ n h exp (cid:16)(cid:12)(cid:12)(cid:12) log( bι ( X t + n ) κ − κ ι ( X t ) − ) /c (cid:12)(cid:12)(cid:12) r (cid:17)(cid:12)(cid:12)(cid:12) X t ii . As | log( bι ( X t + n ) κ − κ ι ( X t ) − ) /c | r ≤ (cid:0) | (log b ) /c | r + | κ − κ log ι ( X t + n ) /c | r + | log ι ( X t )) /c | r (cid:1) ,we may use H¨older’s inequality, stationarity of X , and condition (21) to deduce that theright-hand side of the above display is finite. Therefore, ¯ v ∈ E φ r .The only other modification we require is to show that T is a continuous operator on E φ s for each 1 ≤ s ≤ r . By the first chain of inequalities in the proof of Lemma B.10, for any f ∈ E φ s and c ∈ (0 , ∧ | κ | − ], we obtain E µ h e | T f ( X t ) /c | s i ≤ (1 − β ) E µ h e | log ι ( X t ) / ( κc ) | s i + β E µ h ˜ E h exp (cid:16)(cid:12)(cid:12)(cid:12) log (cid:16) λe κf ( X t +1 ) (cid:17) / ( κc ) (cid:12)(cid:12)(cid:12) s (cid:17)(cid:12)(cid:12)(cid:12) X t ii = (1 − β ) E µ h e | log ι ( X t ) / ( κc ) | s i + βλ E µ ⊗ Q h e log( ι ( X t +1 ) /ι ( X t )) e | (log λ ) / ( κc )+ f ( X t +1 ) /c | s i . Finiteness of the right-hand side then follows by applying H¨older’s inequality and usingstationarity of X and condition (21). Therefore, T : E φ s → E φ s . The proof of continuityfollows by a similar modification. References
Alvarez, F. and U. J. Jermann (2005). Using asset prices to measure the persistence of themarginal utility of wealth.
Econometrica 73 (6), 1977–2016.Amann, H. (1976). Fixed point equations and nonlinear eigenvalue problems in orderedBanach spaces.
SIAM Review 18 (4), 620–709.Ang, A. and A. Timmermann (2012). Regime changes and financial markets.
Annual Reviewof Financial Economics 4 (1), 313–337.Arcidiacono, P. and R. A. Miller (2011). Conditional choice probability estimation of dy-namic discrete choice models with unobserved heterogeneity.
Econometrica 79 (6), 1823–1867.Balbus, L. (2015). On non-negative recursive utilities in dynamic programming with non-linear aggregator and CES. SSRN working paper no. 2703975.42ansal, R., D. Kiku, I. Shaliastovich, and A. Yaron (2014). Volatility, the macroeconomy,and asset prices.
The Journal of Finance 69 (6), 2471–2511.Bansal, R. and A. Yaron (2004). Risks for the long run: A potential resolution of assetpricing puzzles.
The Journal of Finance 59 (4), 1481–1509.Barillas, F., L. P. Hansen, and T. J. Sargent (2009). Doubts or variability?
Journal ofEconomic Theory 144 (6), 2388 – 2418.Bidder, R. and M. Smith (2018). Doubts and variability: A robust perspective on exoticconsumption series.
Journal of Economic Theory 175 , 689–712.Blevins, J. R. (2014). Nonparametric identification of dynamic decision processes withdiscrete and continuous choices.
Quantitative Economics 5 (3), 531–554.Bloise, G. and Y. Vailakis (2018). Convex dynamic programming with (bounded) recursiveutility.
Journal of Economic Theory 173 (Supplement C), 118–141.Boroviˇcka, J. and J. Stachurski (2020). Necessary and sufficient conditions for existenceand uniqueness of recursive utilities.
The Journal of Finance 75 (3), 1457–1493.Cecchetti, S. G., P.-S. Lam, and N. C. Mark (1990). Mean reversion in equilibrium assetprices.
The American Economic Review 80 (3), 398–418.Cecchetti, S. G., P.-S. Lam, and N. C. Mark (2000). Asset pricing with distorted beliefs:Are equity returns too good to be true?
American Economic Review 90 (4), 787–805.Chen, W., L. B. Jia, and Y. Jiao (2016). H¨older’s inequalities involving the infinite productand their applications in martingale spaces.
Analysis Mathematica 42 (2), 121–141.Christensen, T. M. (2015). Nonparametric identification of positive eigenfunctions.
Econo-metric Theory 31 (6), 1310–1330.Christensen, T. M. (2017). Nonparametric stochastic discount factor decomposition.
Econo-metrica 85 (5), 1501–1536.Collard, F., S. Mukerji, K. Sheppard, and J.-M. Tallon (2018). Ambiguity and the historicalequity premium.
Quantitative Economics 9 (2), 945–993.Collin-Dufresne, P., M. Johannes, and L. A. Lochstoer (2016). Parameter learning in generalequilibrium: The asset pricing implications.
American Economic Review 106 (3), 664–698.Croce, M. M., M. Lettau, and S. C. Ludvigson (2015). Investor information, long-run risk,and the term structure of equity.
The Review of Financial Studies 28 (3), 706–742.Epstein, L. G. and S. E. Zin (1989). Substitution, risk aversion, and the temporal behavior ofconsumption and asset returns: A theoretical framework.
Econometrica 57 (4), 937–969.Guo, J. and X. D. He (2017). Recursive utility with investment gains and losses: Existence,uniqueness, and convergence. SSRN working paper no. 2790768.43amilton, J. D. (1989). A new approach to the economic analysis of nonstationary timeseries and the business cycle.
Econometrica 57 (2), 357–384.Hamilton, J. D. (1994). State-space models. Volume 4 of
Handbook of Econometrics ,Chapter 50, pp. 3039 – 3080. Elsevier.Hansen, L. P., J. C. Heaton, and N. Li (2008). Consumption strikes back? Measuringlong-run risk.
Journal of Political Economy 116 , 260–302.Hansen, L. P. and T. J. Sargent (1995). Discounted linear exponential quadratic gaussiancontrol.
IEEE Transactions on Automatic Control 40 (5), 968–971.Hansen, L. P. and T. J. Sargent (2001). Robust control and model uncertainty.
TheAmerican Economic Review 91 (2), 60–66.Hansen, L. P. and T. J. Sargent (2007). Recursive robust estimation and control withoutcommitment.
Journal of Economic Theory 136 (1), 1–27.Hansen, L. P. and T. J. Sargent (2010). Fragile beliefs and the price of uncertainty.
Quan-titative Economics 1 (1), 129–162.Hansen, L. P., T. J. Sargent, G. Turmuhambetova, and N. Williams (2006). Robust controland model misspecification.
Journal of Economic Theory 128 (1), 45–90.Hansen, L. P. and J. A. Scheinkman (2009). Long-term risk: An operator approach.
Econo-metrica 77 (1), 177–234.Hansen, L. P. and J. A. Scheinkman (2012). Recursive utility in a Markov environment withstochastic growth.
Proceedings of the National Academy of Sciences 109 , 11967–11972.Ju, N. and J. Miao (2012). Ambiguity, learning, and asset returns.
Econometrica 80 (2),559–591.Karakostas, G. L. (2008). An extension of H¨older’s inequality and some results on infiniteproducts.
Indian Journal of Mathematics 50 (2), 303–307.Klibanoff, P., M. Marinacci, and S. Mukerji (2009). Recursive smooth ambiguity preferences.
Journal of Economic Theory 144 (3), 930–976.Krasnosel’skii, M. A. and Ya. B. Rutickii (1961).
Convex Functions and Orlicz Spaces . P.Noordhoff Ltd., Groningen.Kress, R. (2014).
Linear Integral Equations . Springer.Malliavin, P. (1995).
Integration and Probability . Springer.Marinacci, M. and L. Montrucchio (2010). Unique solutions for stochastic recursive utilities.
Journal of Economic Theory 145 (5), 1776–1804.Meyn, S. and R. L. Tweedie (2009).
Markov Chains and Stochastic Stability (2 ed.). Cam-bridge Mathematical Library. Cambridge University Press.44orets, A. (2010). Continuity and differentiability of expected value functions in dynamicdiscrete choice models.
Quantitative Economics 1 (2), 305–322.Pollard, D. (2015). Mini-empirical. .Rust, J. (1987). Optimal replacement of GMC bus engines: An empirical model of HaroldZurcher.
Econometrica 55 (5), 999–1033.Rust, J., J. F. Traub, and H. Wozniakowski (2002). Is there a curse of dimensionality forcontraction fixed points in the worst case?
Econometrica 70 (1), 285–329.Schorfheide, F., D. Song, and A. Yaron (2018). Identifying long-run risks: A bayesianmixed-frequency approach.
Econometrica 86 (2), 617–654.van der Vaart, A. W. and J. A. Wellner (1996).
Weak Convergence and Empirical Processes .Springer-Verlag.Vershynin, R. (2018).
High-Dimensional Probability: An Introduction with Applications inData Science . Cambridge University Press.Wachter, J. A. (2013). Can time-varying risk of rare disasters explain aggregate stockmarket volatility?