Existence and Uniqueness of Solutions to the Stochastic Bellman Equation with Unbounded Shock
aa r X i v : . [ ec on . T H ] J u l Existence and Uniqueness of Solutions to the StochasticBellman Equation with Unbounded Shock
Juan Pablo Rinc´on-Zapatero Universidad Carlos III de [email protected]
This Version: February 7, 2019
Abstract
In this paper we develop a general framework to analyze stochastic dynamic problemswith unbounded utility functions and correlated and unbounded shocks. We obtain newresults of the existence and uniqueness of solutions to the Bellman equation through ageneral fixed point theorem that generalizes known results for Banach contractions andlocal contractions. We study an endogenous growth model as well as the Lucas assetpricing model in an exchange economy, significantly expanding their range of applicability.
Keywords—
Stochastic dynamic programming, contraction mapping, Bellman equation, valuefunction, endogenous growth, asset pricing model
Stochastic dynamic programming incorporates uncertain events into a suitable framework to help thedecision-maker to design an optimal plan of action. A fruitful approach for showing the existence ofoptimal stationary plans is to prove that the dynamic programming equation admits a unique solution—the value function— in a suitable space of functions. See Blackwell (1965), Maitra (1968), Furukawa(1972), Bertsekas and Shreve (1978), Stokey and Lucas with Prescott (1989) or Hern´andez-Lerma and Lasserre(1999), where this problem is analyzed in detail. There is a huge literature that applies stochastic dy-namic programming to economics. Brock and Mirman (1972), Mirman and Zilcha (1975), Donaldson and Mehra(1983), Danthine and Donaldson (1981), Majumdar, Mitra and Nyarko (1989), Hopenhayn and Prescott(1992) or Mitra (1998) are only a few of the many relevant papers that have contributed to developingthis field of research. Olson and Roy (2006) makes a review of the contributions to the stochasticoptimal growth model. owever, only partial results have been developed so far about the existence and uniquenessof solutions to the dynamic programming equation with arbitrary utility functions and arbitraryshock spaces. A satisfactory theory of stochastic dynamic programming should include these cases.Rinc´on-Zapatero and Rodr´ıguez-Palmero (2003, 2009) developed a method to deal with unboundedutility functions in deterministic problems, the so–called local contraction approach, based on an ex-tension of the Banach contraction principle for function spaces whose topology is defined by a countablefamily of seminorms. The application of the method to the stochastic case is not straightforward ifone wants to dispense with artificial bounds on the exogenous shocks. Usually, the stochastic dynamicprogramming theory imposes those bounds in the form of a compact shock space. This is for analyticalconvenience only. There are at least two problems when dealing both with unbounded utility functions and withan unbounded shock space. One of them is technical, related to the integrability of the functionsinvolved and thus, to the correctness in the definition of the Bellman operator. A second prob-lem concerns the choice of a suitable family of seminorms (or pseudodistances) that preserves themonotonicity of the Bellman operator. The local contraction approach in the deterministic Bellmanequation constructs a family of compact sets in the endogenous state space to define the seminorms,see Rinc´on-Zapatero and Rodr´ıguez-Palmero (2003). When trying to imitate this in the stochasticBellman equation for the exogenous state space of shocks, one faces the difficulty that the Bellmanequation requires the computation of a conditional expectation. This is an averaging of a randomvariable on the whole exogenous space, breaking down the monotonicity properties of the Bellmanoperator. To overcome the difficulties, we work with an extended concept of contraction parameter(s)in the local contraction definition, to consider an operator that works on the whole family of semi-norms. Thus, in this framework, in addition to the selfmap for which we are interested in finding fixedpoints, there is a companion operator that, acting on the seminorms, plays the role of the contractionparameter of the former selfmap. See Definition 2.1 below. We state a fixed point theorem, Theorem2.5, that applies to this more general framework and we show how it covers previous fixed point results,including the classical Banach ContractionTheorem — and henceforth, the weighted norm approachin Boyd III (1990), or Becker and Boyd III (1997) — as well as those based on local contractions, seee.g., Rinc´on-Zapatero and Rodr´ıguez-Palmero (2003) and Martins da Rocha and Vailakis (2010). Thisidea is not new. Kozlov, Thim and Turesson (2010) developed a fixed point theorem in locally convex Had˘zi´c and Stankovi´c (1969) is one of the first papers dealing with this extension.Rinc´on-Zapatero and Rodr´ıguez-Palmero (2003), independently, introduced different hypotheses and ap-plied the results to the deterministic Bellman equation. Recent contributions of the local contraction conceptto dynamic programming are Martins da Rocha and Vailakis (2010), Matkowski and Nowak (2011) andBalbus, Reffett and Wozny (2018). In general, this assumption is incompatible with modelling the Markov chain as a first order stochasticdifference equation, even if the underlying i.i.d. shocks takes only finitely many values. Think, for instance,of the simple random walk. It takes every integer number with positive probability. Thus, its range can onlybe limited by imposing exogenous constraints, alien to the economic model, a limitation which will modify the“natural” solution and may distort its real implications. paces whose topology is given by a family of seminorms. However, the results obtained depend onthe companion contraction parameter operator being linear. The proof in Kozlov, Thim and Turesson(2010), of the existence and uniqueness of fixed points, heavily exploits the linearity of the companionoperator. This precludes the application of the results of this paper to the dynamic programmingequation, as it genuinely demands a nonlinear companion contraction parameter operator, due to thepresence of a maximization operation On the other hand, we simplify somewhat the hypotheses madeon the companion operator and extend the result to the consideration of arbitrary pseudodistances—and hence the topological space is not a locally convex space anymore—, which could be useful foranalyzing the unbounded from below case, and more importantly, we show how the dynamic program-ming equation fits well into this framework, attaining, to our knowledge, new existence and uniquenessresults.The present paper devotes a good deal of efforts to isolating a suitable space of functions and asuitable family of seminorms where the approach explained above is successful. We find a suitableframework, where the averaging needed to compute the conditional expectation does not break downthe monotonicity of the Bellman operator. The seminorms that we define combine the usual supremumnorm in the endogenous variables, with an L norm in the exogenous variables, and construct acomplete space of functions for dealing with the Bellman operator — a Carath´eodory function space—. The paper is organized as follows. Section 2 develops a fixed point theorem for operators actingon topological spaces whose topology is given by a family of pseudodistances that makes it Hausdorffand sequentially complete. The operators enjoy a contraction property, materialized in an associatedoperator acting on the family of pseudodistances, which plays the role of the contraction parameterin Banach’s Contraction Theorem. Thus, the result generalizes Banach’s Contraction Theorem andshows that the local contraction approach used in previous papers is a particular case of this moregeneral framework. Section 3 applies the theorem to the stochastic dynamic programming equationfor models with shocks driven by an exogenous Markov chain and with an unbounded shock space.We carefully choose the set of functions where the Bellman operator is defined and we provide a wayto methodically construct the objects needed to apply the fixed point theorem developed in Section2. We also show that the solution of the Bellman equation is the value function. In Section 4, westudy a model of endogenous growth —which encompasses the one sector optimal growth model—,and the Lucas asset pricing model in an exchange economy, in all cases allowing for correlated andunbounded shocks. Section 5 establishes the conclusions of the paper, and give some tips for furtherresearch. Appendix A contains the proofs that are not in the main text, with the exception of theproofs regarding the completeness of the function space and seminorms we consider to analyze thedynamic programming equation, which are developed in Appendix B. Appendix C discusses the issueof continuity of the Markov operator appearing in the dynamic programming equation, giving sufficientconditions to establish continuity, and providing an example of non–continuity. It is for this reason that we have to develop our own fixed point theorem, departing from the approachof Kozlov, Thim and Turesson (2010), as we cannot make use of an equality as its formula (4). Instead, weprovide an alternative condition, summarized in our assumption (VI) below. Local Contractions
Let ( E, D ) be a topological space, where E is a set whose topology is generated by a saturated familyof pseudometrics D = { d a } a ∈ A , with A an arbitrary index set. Since the family D is saturated, thetopology it generates is Hausdorff . We suppose that ( E, D ) is sequentially complete: if { x n } is asequence in E which is Cauchy with respect to all d a ∈ D , that is, if d a ( x n , x m ) → n, m → ∞ ,then there is x ∈ E such that d a ( x n , x ) → n → ∞ for all a ∈ A .Given a sequentially complete subset F ⊆ E , we study the existence and uniqueness of a fixedpoint of a mapping T : F → E .Let R A be the set of functions d : A → R + and let R A + be the non–negative cone of R A . On this setwe consider the order it generates, that is, for two elements d, d ′ ∈ R A + , we say that d ≤ d ′ if and only if d ( a ) ≤ d ′ ( a ) for all a ∈ A . The family D can be embedded into R A + , since that, for x, y ∈ E given, themapping a d a ( x, y ) defines a function in R A + , that we denote d x,y ( a ) := d a ( x, y ). In general, for agiven subset F ⊆ E , we let D ( F ) be the set of functions in R A + which are generated by pairs x, y ∈ F ,that is D ( F ) := { d : A → R + : d = d x,y for some x, y ∈ F } . Definition 2.1.
Let F ⊆ E . The mapping T : F → E is an L -local contraction on F with contractionoperator parameter L (COP, for short), if there are a set C ⊆ R A + such that D ( F ) ⊆ C , and anoperator L : C → R A + , such that d a ( T x, T y ) ≤ ( Ld x,y )( a ) , for all x, y ∈ F and for all a ∈ A . Note that the inequality above can be rewritten d T x,T y ≤ Ld x,y , that is, as an order relation inthe space R A + . The definition of L –contractions for mappings T : F −→ E , not imposing T : F −→ F ,will facilitate the definition of the COP parameter L of the Bellman operator in Section 3. Of course,the property T : F −→ F is fundamental for Theorem 3.5 below, and will checked carefully in Section3. The following two examples show that the operator L is a generalization of the concept of contrac-tion parameter of a (local) contraction mapping. Example 2.2 (Banach contractions) . In the classical Banach’s Theorem, E is dotted with a completemetric d , so the index set A is a singleton, D = { d } , and T is a contraction of constant parameter β ,with < β < : d ( T x, T y ) ≤ βd ( x, y ) , for any x, y ∈ E . The COP is L = βI , where I is the identitymap in R + . A pseudometric d : E × E → R + is a function satisfying d ( x, y ) ≥ d ( x, x ) = 0, d ( x, y ) = d ( y, x )and d ( x, z ) ≤ d ( x, y ) + d ( y, z ) for any x, y, z ∈ E , but d ( x, y ) = 0 does not imply x = y . The family D ofpseudometrics is saturated if d a ( x, y ) = 0 for all a ∈ A , implies x = y . Sometimes, the pseudometrics aredefined through seminorms p a , a ∈ A , by d a ( x, y ) = p a ( x − y ), where now E is a real vector space. A seminormis a function p : E → R + that satisfies all the axioms to be a norm, except that p ( x ) = 0 does not imply that x is the null vector of E . If the family of seminorms is saturated, then the topology defined by the family isHausdorff and the space and E is a locally convex space. See Willard (1970) for further details. generalization of the Banach contraction concept is provided in Wong (1968), where it is con-sidered T : E −→ E for which there is a function L : R + −→ R + satisfying d ( T x, T y ) ≤ L ( d ( x, y )) , (2.1) for all x, y ∈ E . Note that our definition is an extension of this concept to topological spaces whosetopology is given by a family of semidistances. Example 2.3 ( k –local contractions) . Suppose that A = N is countable. In Rinc´on-Zapatero and Rodr´ıguez-Palmero(2003, 2007), we introduced the concept of k –local contraction in the study of the deterministic Bell-man and Koopmans equations, respectively. A k –local contraction on F , k = 0 , , , . . . , is a mapping T : F ⊆ E −→ E satisfying d j ( T x, T y ) ≤ β j d j + k ( x, y ) for some fixed sequence of numbers { β j } j ∈ N with < β j < , and for all x, y ∈ F . If we let s = R N bethe set of real sequences and s + be the subset of s of nonnegative sequences, then the COP associatedwith T is the linear operator L : s + −→ s + acting on sequences given by L ( d , d , . . . , d j , . . . ) = ( β d k , β d k , . . . , β j d j + k , . . . ) , where k ≥ is fixed.Suppose that A = N is uncountable and let a mapping α : A −→ E such that for any a ∈ A , d a ≤ d α ( a ) . Martins da Rocha and Vailakis (2010) worked with the following generalization of thecountable class above: T : E −→ E is an α -local contraction if there exists a function β : A −→ [0 , such that d a ( T x, T y ) ≤ β ( a ) d α ( a ) ( x, y ) . The COP L acts as follows: given a function d : A −→ R + , the image function is ( Ld )( a ) = β ( a ) d ( α ( a )) , that is, a translation in the independent variable by α , and a multiplication by β . Itturns out that L is also a linear mapping, as in the countable case above. In what follows, we use the standard notation for successive iterations of the operators T and L .For instance, L is the identity operator on C , L = L , and for t ≥ L t = L ◦ L t − . We impose to C , L and T the assumptions (I) to (VI) listed below. The assumptions (I) to (V) concern the behavior of L on the set C . Assumption (VI) links directly the operators T and L .(I) D ( F ) ⊆ C . For all d, d ′ ∈ C , the sum d + d ′ ∈ C , and any bounded subset of C is countablychain complete Moreover, if d ′ ∈ C , d ∈ R A + and d ≤ d ′ , then d ∈ C . (II) L . (III) L is monotone: for all d, d ′ ∈ C with d ≤ d ′ , Ld ≤ Ld ′ . A subset S ⊆ C is bounded with respect to the order inherited from R A if there is d ′ ∈ C such that d ≤ d ′ forall d ∈ S . The bounded subset S is countably chain complete if for any countably chain d ≤ d ≤ · · · d t ≤ · · · in S , sup t ∈ N d t ∈ S . IV) L is subadditive: for any d, d ′ ∈ C L ( d + d ′ ) ≤ Ld + Ld ′ . (V) L is upper semicontinuous sup-preserving : for any bounded countably chain in C , d ≤ d ≤· · · ≤ d t ≤ · · · , L sup t d t ≤ sup t Ld t . (VI) There are x ∈ F and r : 0 ∈ C with d a ( x , T x ) ≤ r ( a ) and R ( a ) := ∞ X t =0 L t r ( a ) < ∞ , for all a ∈ A . Since L t r ∈ C , for all t = 0 , , . . . , and the countable chain { r , r + Lr , . . . , r + Lr + · · · + L t r , · · · } is bounded in C by (VI), R is in C by assumption (I).For F ⊆ E , x ∈ F , and m ∈ R A + , let the set V F ( x , m ) = { x ∈ F : d a ( x , x ) ≤ m ( a ) , ∀ a ∈ A } . When E is a metric space, that is, when A is a singleton, the pseudometric is a metric, and V F ( x , m )is simply the intersection with F of the closed ball centered at x and radius m . Lemma 2.4.
Let T : F −→ F be an L -local contraction on F ⊆ E and let x ∈ F be such that(I)–(VI) hold true for a suitable r ∈ C . Let R be defined as in (VI). Then(a) T : V F ( x , R ) −→ V F ( x , R ) .(b) For any a ∈ A , lim t →∞ ( L t R )( a ) = 0 . The following result is a fixed point theorem for L -local contractions. Theorem 2.5.
Let ( E, D ) be a Hausdorff and sequentially complete topological space. Let T : F → F be an L -local contraction on the sequentially complete subset F ⊆ E and let x ∈ F be such that(I)–(VI) hold true. Then there is a unique fixed point x ∗ ∈ V F ( x , R ) of T , which is the limit of anyiterating sequence y t +1 = T y t , t = 0 , , , . . . , where y = x ∈ V F ( x , R ) is arbitrary.Proof. Consider first the iterating sequence x t +1 = T x t , t = 0 , , , . . . (that is, the initial seed is x ).By Lemma 2.4, x t is in V F ( x , R ) for any t = 0 , , , . . . . Since T is an L -local contraction d a ( x t , x t +1 ) = d a ( T x t − , T x t ) ≤ Ld x t − ,x t ( a )and by induction d a ( x t , x t +1 ) ≤ ( L t d x ,T x )( a ). Let r > s ≥
1. Then by the triangle inequalityextended to finite sums d a ( x s , x r +1 ) ≤ r X t = s d a ( x t , x t +1 ) ≤ r X t = s L t d x ,T x ( a ) ≤ R ( a ) < ∞ , (2.2) For instance, the sup-preserving property, L (sup t d t ) = sup t Ld t , plays a prominent role in the Fixed PointTheorem of Kantorovich-Tarski. In our context, it can be weakened to a kind of upper semicontinuity. or all a ∈ A . By the Cauchy criterion for series, d a ( x s , x r +1 ) tends to 0 as r, s → ∞ . Since a is arbi-trary, the sequence { x t } ∞ t =0 is Cauchy, hence it converges to some x ∗ ∈ F , as F is sequentially complete.In fact, x ∗ ∈ V F ( x , R ). To see this, note that for all a ∈ A , the mapping x d a ( x , x ) is triv-ially sequentially continuous in the topology generated by D , hence d a ( x , x ∗ ) = d a ( x , lim t →∞ x t ) =lim t →∞ d a ( x , x t ) ≤ R ( a ) for any a ∈ A . Next, we prove that x ∗ is a fixed point of T . By estimate(2.2), letting r → ∞ , d a ( x s , x ∗ ) ≤ P ∞ t = s L t d x ,T x ( a ) = sup N P Nt = s L t d x ,T x ( a ). Hence Ld x s ,x ∗ ( a ) ≤ L (cid:16) sup N N X t = s L t d x ,T x (cid:17) ( a ) ≤ sup N L (cid:16) N X t = s L t d x ,T x (cid:17) ( a ) ≤ sup N N X t = s L t +1 d x ,T x ( a ) , which tends to 0 as s → ∞ , since R is finite. Hence Ld x s ,x ∗ → s → ∞ , for all a ∈ A . The firstline above is due to the monotonicity of L , the second line since L is upper sup-preserving, and thethird one, since L is subadditive. Now, given that d a ( x ∗ , T x ∗ ) ≤ d a ( x s +1 , x ∗ ) + d a ( x s +1 , T x ∗ ) ≤ d a ( x s +1 , x ∗ ) + Ld x s ,x ∗ ( a )and that both summands tend to 0 as s → ∞ , we conclude that x ∗ = T x ∗ . To prove uniqueness,we argue by contradiction, supposing the existence of another fixed point x ∗∗ ∈ V F ( x , R ). Then d a ( x ∗ , x ∗∗ ) ≤ R ( a ) for all a ∈ A , and hence d a ( x ∗ , x ∗∗ ) = d a ( T t x ∗ , T t x ∗∗ ) ≤ L t d x ∗ ,x ∗∗ ( a ) ≤ L t R ( a ) , since L is both monotone and subadditive . Taking the limit as t → ∞ and using Lemma 2.4, x ∗ = x ∗∗ is proven. Finally, let x ∈ V F ( x , R ) and let the iterating sequence y t +1 = T y t , for n = 0 , , , . . . , ,with y = x . Observe that d a ( x t , y t ) ≤ L t d x ,x ( a ) ≤ L t R ( a ) → n → ∞ ,by Lemma 2.4, so d a ( y t , x ∗ ) ≤ d a ( x t , y t ) + d a ( x t , x ∗ ) tends to 0 as t → ∞ .The next corollary provides conditions for the uniqueness of the fixed point in F and not onlyin V F ( x , R ). When T is indeed an L -local contraction on the whole E , this result provides globaluniqueness of the fixed point on E . Theorem 2.6.
Let ( E, D ) be a Hausdorff, sequentially complete space. Let T : F → F be an L -localcontraction on the sequentially complete subset F ⊆ E and let x ∈ F be such that (I)–(VI) hold true.Suppose that, for any x ∈ F , it is possible to choose r ∈ C satisfying (VI), such that x ∈ V F ( x , R ) .Then there is a unique fixed point of T in F and convergence to the fixed point of successive iterationsof T is attained from any x ∈ F . It is easy to prove that L t is monotone and subadditive for any t . roof. By Theorem 2.5, T admits a unique fixed point x ∗ in V F ( x , R ), where R = P ∞ t =0 L t r , for any r ∈ C for which R is a convergent series. Suppose, by contradiction, that T admits another fixed point x ∗∗ = x ∗ in F . By assumption, there is r ′ ∈ C such that x ∗∗ ∈ V F ( x , R ′ ), where R ′ = P ∞ t =0 L t r ′ isfinite. Hence, x ∗ = x ∗∗ . The convergence of iterating sequences is also an immediate consequence ofTheorem 2.5.Next we establish a useful sufficient condition for (VI). Note that the Bellman operator satisfiesthe extra condition imposed on L . Proposition 2.7.
Let ( E, D ) be a Hausdorff and sequentially complete topological space. Let T : F −→ F be an L –local contraction on F ⊆ E , with COP L satisfying (I) to (V) and L ( αd ) ≤ αLd ,for all d ∈ C , for all α ∈ [0 , . Let x ∈ F , for which there is t ∈ { , , , . . . } , s ∈ C , and θ ∈ [0 , such that L t d ≤ s and Ls ≤ θs, (2.3) where d ( a ) = d a ( x , T x ) . Then (VI) holds with r = d .Proof. Note that P t = t L t d ≤ s + Ls + L s + · · · ≤ (1 + θ + θ + · · · ) s = − θ s . Hence, P ∞ t =0 L t d = P t − t =0 L t d + P ∞ t = t L t d ≤ P t − t =0 L t d + − θ s is finite for all a ∈ A . Consider a dynamic programming model (
X, Z, Γ , Q, U, β ), where X × Z is the set of possible statesof the system, Γ is a correspondence that assigns a nonempty set Γ( x, z ) of feasible actions to eachstate ( x, z ) and Q is the transition function, which associates a conditional probability distribution Q ( z, · ) on Z to each z ∈ Z . Hence, the law of motion is assumed to be a first-order Markov process,which could be degenerated, giving rise to a deterministic model. We will use indistinctly the notation Q z ( · ) = Q ( z, · ); the function U is the one–period return function, defined on the graph of Γ, Ω = { ( x, y, z ) : ( x, y ) ∈ X × Z, y ∈ Γ( x, z ) } , and β is a discount factor.Starting at some state ( x , z ), the agent chooses an action x ∈ Γ( x , z ), obtaining a returnof U ( x , x , z ) and the system moves to the next state ( x , z ), which is drawn according to theprobability distribution Q ( ·| z ). Iteration of this process yields a random sequence ( x , z , x , z , . . . )and a total discounted return P ∞ t =0 β t U ( x t , x t +1 , z t ). A history of length t is z t = ( z , z , . . . , z t ). Let Z t be the set of all histories of length t . A (feasible) plan π is a constant value π ∈ X and a sequenceof measurable functions π t : Z t −→ X , such that π t ( z t ) ∈ Γ( π t − ( z t − ) , z t ), for all t = 1 , , . . . . Denoteby Π( x , z ) the set of all feasible plans starting at the state ( x , z ). Any feasible plan π ∈ Π( x , z ),along with the transition function Q , defines a distribution P π, ( x ,z ) on all possible futures of thesystem { ( x t , z t ) } ∞ t =1 , as well as the expected total discounted utility u ( π, x , z ) = E π, ( x ,z ) ∞ X t =0 β t U ( x t , x t +1 , z t ) ! . he expectation E π, ( x ,z ) is taken with respect to the distribution P π, ( x ,z ) . The problem is thento find a plan π ∈ Π( x , z ) such that u ( π, ( x , z )) ≥ u ( b π, ( x , z )) for all b π ∈ Π( x , z ), for all( x , z ) ∈ X × Z . The value function of the problem is v ( x , z ) = sup π ∈ Π( x ,z ) u ( π, ( x , z )).Consider the functional equation corresponding to the above dynamic programming problem asstated in Stokey and Lucas with Prescott (1989). For x ∈ X , z ∈ Zv ( x, z ) = max y ∈ Γ( x,z ) (cid:26) U ( x, y, z ) + β Z Z v ( y, z ′ ) Q ( z, dz ′ ) (cid:27) . (3.1)A solution of the Bellman equation satisfying additional assumptions is the value function of theinfinite programming problem. This is the content of Theorem 3.5 below, whose proof needs the notionof the probability measure µ t defined on the sequence space of shocks ( Z t , Z t ) for finite t = 1 , , . . . ,where ( Z t , Z t ) = ( Z × · · · × Z, Z × · · · × Z ) ( t times) . For any rectangle B = A × · · · × A t ∈ Z t , µ t is defined by µ t ( z , B ) = Z A . . . Z A t − Z A t Q z t − ( dz t ) Q z t − ( dz t − ) · · · Q z ( dz ) , and by the Hahn Extension Theorems, µ t ( z , · ) has a unique extension to a probability measure on allof Z t . We omit the details, that can be found in Stokey and Lucas with Prescott (1989), Section 8.2,whose presentation we follow closely.Defining the Bellman operator in a suitable function space E , such that for f ∈ E ( T f )( x, z ) = max y ∈ Γ( x,z ) (cid:26) U ( x, y, z ) + β Z Z f ( y, z ′ ) Q ( z, dz ′ ) (cid:27) , the Bellman functional equation (3.1) is a fixed point problem for T . This fixed point problem is com-pletely understood for the case where U is bounded. There are now also different approaches for somespecial cases for unbounded U . It is worth mentioning the constant returns to scale model and the log-arithmic and the quadratic parametric examples analyzed in Stokey and Lucas with Prescott (1989),pp. 270–280, and the weighted norm approach in Boyd III (1990) and Hern´andez-Lerma and Lasserre(1999). One feature of all these approaches is that they consider a bounded (or compact) space ofshocks, an assumption that we want to dispense with. Allow for a non–compact shock space is impor-tant for a qualitative analysis of models, see for instance Binder and Pesaran (1999) and Stachurski(2002), and more recently, Ma and Stachurski (2017).We now impose the standing hypotheses. Most are taken from Stokey and Lucas with Prescott(1989), but there are essential differences, as we admit an unbounded utility U and an unboundedshock space Z . The weighted norm approach presents some limitations, which are explained, for instance, in Remark 9 ofMatkowski and Nowak (2011). This paper constitutes a first attempt to translate the approach initiated byRinc´on-Zapatero and Rodr´ıguez-Palmero (2003) for deterministic programs to the stochastic case. However,the results obtained do not cover a general model where shocks are driven by an exogenous transition probabil-ity. In fact, in the class of dynamic programming models described here and in Stokey and Lucas with Prescott(1989), assumption (A4) on the generation of shocks { z t } ∞ t =0 imposed in Matkowski and Nowak (2011), basi-cally implies that the space of shocks Z is compact, or that the underlying probability has compact support. B1) X ⊆ R l , Z ⊆ R k are Borel sets, with Borel σ -algebra X and Z , respectively. The set X isendowed with the Euclidean topology.(B2) 0 < β < Q : Z × Z → [0 ,
1] satisfies(a) for each z ∈ Z , Q ( z, · ) is a probability measure on ( Z, Z ); and(b) for each B ∈ Z , Q ( · , B ) is a Borel measurable function.(B4) The correspondence Γ : X × Z −→ X is nonempty, compact-valued and continuous.(B5) U : Ω −→ R is a Carath´eodory function, that is, it satisfies(a) for each ( x, y ) ∈ D := { ( x, y ) ∈ X × Y : ∃ z ∈ Z, y ∈ Γ( x, z ) } , the function of z , U ( x, y, · ) : Z −→ R is Borel measurable;(b) for each z ∈ Z , the function of ( x, y ), U ( · , · , z ) : D −→ R is continuous.The reason for working with Carath´eodory functions instead of continuous functions in the threevariables ( x, y, z ) is twofold. On the one hand, the Markov operator( M f )( x, z ) := Z Z f ( x, z ′ ) Q ( z, dz ′ ) , (3.2)does not preserve continuity of f , if f is continuous but not bounded, as the simple example inAppendix C shows.On the other hand, the Bellman operator is well defined for the class of Carath´eodory functionsin the unbounded case, while working with the supremum norm is not possible. A direct attackof the Bellman equation in the space of ( x, z )–continuous functions does not work for unboundedfunctions and/or unbounded shock space: known theorems on local contractions—with a countable oruncountable index set —are not suitable, due to the averaging operation involved in the computationof conditional expectations. For this reason we are going to use L -type seminorms, whose precisedefinition is given below.We now describe the function space, which details are given in Appendix B. For each z ∈ Z , let L ( Z, Z , Q z ) be the space of Borel measurable functions g : Z −→ R such that R Z | g ( z ′ ) | Q z ( dz ′ ) < ∞ .In what follows, we let K be the family of all compact subsets of X .Consider the space E := L ( Z ; C ( X )), formed by Carath´eodory functions f : X × Z −→ R suchthat the function z ′ max x ∈ K | f x ( z ′ ) | is in L ( Z, Z , Q z ), for all compact sets K ∈ K , and all z ∈ Z .See Appendix B for the definitions and the notation, where it is also proved the following fundamentalresult. Lemma 3.1. E = L ( Z ; C ( X )) is a complete locally convex space with the topology generated by thefamily of seminorms P := { p K,z } K ∈K ,z ∈ Z , given by p K,z ( f ) := Z Z max x ∈ K | f ( x, z ′ ) | Q z ( dz ′ ) . (3.3) It is well known that L ( Z, Z , Q z ) consists of equivalence classes rather than functions, identifying functionsthat are equal Q z –almost everywhere. n particular, the lemma states that E is sequentially complete. In the notation of Section 2, theindex set of the family of seminorms is A = K × Z .Given a solution f ∈ L ( Z ; C ( X )) of (3.1), define the policy correspondence G f : X × Z → X by G f ( x, z ) = { y ∈ Γ( x, z ) : f ( x, z ) = U ( x, y, z ) + βM f ( y, z ) } . (3.4)This is the optimal policy correspondence, denoted simply by Γ ∗ , when f is the value function, v .Remember from Section 2, that for a subset F ⊆ E , the set D ( F ) is in this context D ( F ) = { p : K × Z → R + : p ( K, z ) = p K,z ( f ) for some f ∈ F } . Notation 3.2.
Along the paper, we will use the notation ψ ( x, z ) = max y ∈ Γ( x,z ) U ( x, y, z ) = T x, z ) while, for p : K × Z R + , the function p [Γ] : X × Z R + is defined by p [Γ]( x, z ) = p (Γ( x, z ) , z ) ,that is, it is the function of ( x, z ) obtained through p , when the compact sets K equal Γ( x, z ) , for x ∈ X , z ∈ Z . The next result shows that T is an L -local contraction, and gives the expression of L : Given p : K × Z R + for which p [Γ] ∈ L ( Z ; C ( X )), the operator L computes the seminorm of the function p [Γ], that is, ( Lp )( K, z ) = βp K,z ( p [Γ]). Note that L is nonlinear . The expanded definition of theoperator L is the expression (3.5) below. Proposition 3.3.
Let the Bellman operator T : F −→ E , where F ⊆ L ( Z ; C ( X )) , such that for all p ∈ D ( F ) , p [Γ] ∈ L ( Z ; C ( X )) . Then, T is an L -local contraction on F with COP L : D ( F ) −→ R K× Z + given by ( Lp )( K, z ) = β Z Z max x ∈ K p (Γ( x, z ′ ) , z ′ ) Q z ( dz ′ ) , (3.5) for all K ∈ K and z ∈ Z .Proof. Following Blackwell (1965), we exploit the fact that T is monotone, in conjunction with theproperties of the seminorms p K,z . Let f, g ∈ E and let x ∈ X , K ∈ K and z ∈ Z . Let y ∈ Γ( x, z )and z ′ ∈ Z arbitrary. Then f ( y, z ′ ) ≤ g ( y, z ′ ) + | f ( y, z ′ ) − g ( y, z ′ ) | implies f ( y, z ′ ) ≤ g ( y, z ′ ) +max y ∈ Γ( x,z ) | f ( y, z ′ ) − g ( y, z ′ ) | and then, by monotonicity and linearity of the integral, Z Z f ( y, z ′ ) Q z ( dz ′ ) ≤ Z Z g ( y, z ′ ) Q z ( dz ′ )+ Z Z max y ∈ Γ( x,z ) | f ( y, z ′ ) − g ( y, z ′ ) | Q z ( dz ′ ) . We are allowed to take the integral by Lemma A.1. The inequality is maintained after multiplying by β and adding U ( x, y, z ) to both sides. Then, by taking the maximum in y ∈ Γ( x, z ) to both sides, wehave ( T f )( x, z ) ≤ ( T g )( x, z ) + β max y ∈ Γ( x,z ) Z Z max y ∈ Γ( x,z ) | f ( y, z ′ ) − g ( y, z ′ ) | Q z ( dz ′ )= ( T g )( x, z ) + β Z Z max y ∈ Γ( x,z ) | f ( y, z ′ ) − g ( y, z ′ ) | Q z ( dz ′ )= ( T g )( x, z ) + βp Γ( x,z ) ,z ( f − g ) . xchanging the roles of f and g , we have | ( T f )( x, z ) − ( T g )( x, z ) | ≤ βp Γ( x,z ) ,z ( f − g ) . It is convenient to write this inequality with the dummy variable z ′ instead of z . Now, taking themaximum in x ∈ K and averaging with respect to the measure Q z , we obtain Z Z max x ∈ K | ( T f )( x, z ′ ) − ( T g )( x, z ′ ) | Q z ( dz ′ ) ≤ β Z Z max x ∈ K p f − g (Γ( x, z ′ ) , z ′ ) Q z ( dz ′ ) . This inequality can be rewritten p K,z ( T f − T g ) ≤ ( Lp f − g )( K, z ), for all K ∈ K , z ∈ Z , where L is theoperator defined in (3.5).One of the difficulties in applying contraction techniques to the dynamic programming equation,when the return function and/or the space of shocks is unbounded, is the selection of a suitable spaceof functions where the Bellman operator is a selfmap. Assumption (B6) below provides a scheme toconstruct such a space along the lines of assumption (VI) in Section 2. This is in the same spirit ofAssumption 9.3 in Stokey and Lucas with Prescott (1989), pp. 248-249. Our assumption is not aboutbounding the one-shot utility function U along any policy path by a function that depends only ontime and the initial state, but about bounding its expected value with respect to the initial state. Thisis an important difference, as it allows us to deal with an unbounded space of shocks.(B6) There is a collection of nonnegative functions { l t } ∞ t =0 ∈ L ( Z ; C ( X )), such that for all x ∈ X ,for all z ∈ Z l ( x, z ) ≥ | ψ ( x, z ) | ; l t +1 ( x, z ) ≥ β Z Z max y ∈ Γ( x,z ) l t ( y, z ′ ) Q z ( dz ′ ) , for all t = 0 , , . . . ,and the series w := P ∞ t =0 l t is unconditionally convergent, that is, R ( K, z ) := ∞ X t =0 p K,z ( l t ) < ∞ , for all K ∈ K , for all z ∈ Z .Now we consider a suitable set C where L is defined. C = n p : K × Z R + : p ( K, z ) ≤ cR ( K, z ) for some c > p [Γ] ∈ L ( Z, C ( X )) o . (3.6)As it is proved in Lemma A.4, C is not trivial, as it contains the images of V (0 , R ) by the family ofseminorms P .Theorem 3.5 below is a fixed point theorem for the Bellman operator with unbounded utility andunbounded space of shocks. We state a previous lemma. Lemma 3.4.
Let assumptions (B1) to (B6) hold. Then T and L with C defined in (3.6) , satisfy (I)to (VI). heorem 3.5. Let assumptions (B1) to (B6) hold. The following is true.(a) The Bellman equation admits a unique solution v ∗ in V (0 , R ) .(b) If the correspondence G v ∗ defined in (3.4) admits a measurable selection, then v ∗ coincides with thevalue function, v = v ∗ , and for all v ∈ V (0 , R ) , T n v → v as n → ∞ , that is, p K,z ( T n v − v ) → ,for all K ∈ K and z ∈ Z . Moreover, for all z ∈ Z , the optimal policy correspondence Γ ∗ ( · , z ) : X → X is non-empty, compact valued and upper hemicontinuous.Proof. (a) T is an L –contraction by Proposition 3.3 and all the assumptions of Theorem 2.5 hold trueby Lemma 3.4. Hence T admits a unique fixed point v ∗ is V (0 , R ) and the rest of conclusions ofTheorem 3.5 hold true.(b) To see that v ∗ is the value function of the problem, we invoke Theorem 9.2 in Stokey and Lucas with Prescott(1989). Recall that, for any function F that is µ t ( z , · )-integrable, its conditional expectation can beexpressed as E z ( F ) := Z Z t F ( z t ) µ t ( z , dz t )= Z Z t − (cid:20)Z Z F ( z t − , z t ) Q z t − ( dz t ) (cid:21) µ t − ( z , dz t − )= Z Z (cid:20)Z Z t − F ( z , z t ) µ t − ( z , dz t ) (cid:21) Q z ( dz ) . The assumptions of Theorem 9.2 in Stokey and Lucas with Prescott (1989) are: (i) Γ is non-emptyvalued, with a measurable graph and admits a measurable selection; (ii) for each ( x , z ) and eachfeasible plan π from ( x , z ), U ( π t − ( z t − ) , π t ( z t ) , z t ) is µ t ( z , · )-integrable, t = 1 , , . . . , and the limit U ( x , π , z ) + lim n →∞ n X t =1 Z Z t β t U ( π t − ( z t − ) , π t ( z t ) , z t ) µ t ( z , dz t ) (3.7)exists; and (iii) lim t →∞ R Z t β t v ∗ ( π t − ( z t − ) , z t ) µ t ( z , dz t ) = 0.(i) is implied by (B5) and (ii) is implied by (B6), since | U ( π t − ( z t − ) , π t ( z t ) , z t ) | is clearly mea-surable, given that U is a Carath´eodory function. Moreover, since l in (B6) is in Ca( X × Z ), we canapply Fubini’s Theorem so that l ( π ( z ) , z ) is µ ( z , · )-integrable and Z Z l ( π ( z ) , z ) µ ( z , dz ) = Z Z (cid:18)Z Z l ( π ( z ) , z ) Q z ( dz ) (cid:19) µ ( z , dz ) ≤ Z Z β l ( π ( z ) , z ) µ ( z , dz ) ≤ β l ( x , z ) . Both inequalities are due to assumption (B6). By induction, we get that l ( π t − ( z t − ) , z t ) is µ t ( z , · )–integrable and Z Z t l ( π t − ( z t − ) , z t ) µ t ( z , dz t ) ≤ β t l t ( x , z ) . ince | U ( π t − ( z t − ) , π t ( z t ) , z t ) | ≤ l ( π t − ( z t − ) , z t ), the first part of (ii) is proved. Indeed, this esti-mate provides the bound | U ( x , π ( z ) , z ) | + n X t =1 Z Z t β t | U ( π t − ( z t − ) , π t ( z t ) , z t ) | µ t ( z , dz t ) ≤ | U ( x , π ( z ) , z ) | + n X t =1 l t ( x , z ) ≤ w ( x , z ) , hence the second part of (ii) also holds, that is, the limit (3.7) is finite. Moreover, since the aboveinequality holds for any π ∈ Π( x , z ), it shows that the n –th iteration of T on the null function asthe initial seed satisfies | T n x , z ) | ≤ w ( x , z ). Hence, since R Z | T n x , z ) − v ∗ ( x , z ) | Q z ( dz )tends to 0 as n → ∞ , by part (a) above, we obtain the bound Z Z | v ∗ ( x , z ) | Q z ( dz ) ≤ Z Z w ( x , z ) Q z ( dz ) . (3.8)This inequality will be used to show (iii). First, we claim that for any t , for any π ∈ Π( x , z ), Z Z t β t w ( π t − ( z t − ) , z t ) µ t ( z , dz t ) ≤ ∞ X s = t l s ( x , z ) . To prove it, we employ mathematical induction. Let t = 1. Then, by assumption (B6) Z Z βw ( π ( z ) , z ) µ ( z , dz ) = Z Z β ∞ X t =0 l t ( π ( z ) , z ) Q z ( dz )= ∞ X t =0 β Z Z l t ( π ( z ) , z ) Q z ( dz ) ≤ ∞ X t =0 l t +1 ( x , z ) . The exchange of the integral and infinite sum is possible by the Monotone Convergence Theorem.Suppose that the property is true for t and let us prove it for t + 1. Then it will hold for any t . Note Z Z t +1 β t +1 w ( π t ( z t ) , z t +1 ) µ t +1 ( z , dz t +1 )= Z Z (cid:18) β Z Z t β t w ( π t − ( z t − ) , z t ) µ t ( z , dz t ) (cid:19) Q z ( dz ) ≤ Z Z β ∞ X s = t l s ( π ( z ) , z ) Q z ( dz ) ≤ ∞ X s = t +1 l s ( x , z ) , again by the Monotone Convergence Theorem, and where we have used Fubini’s Theorem and theinduction hypothesis. This and (3.8) imply (iii), since the series w converges. Thus, v ∗ is the valuefunction. The claims about Γ ∗ are immediate from the Theorem of the Maximum of Berg´e.The following result provides a sufficient condition for (B6). roposition 3.6. Let assumptions (B1) to (B5) to hold. Suppose that there is l ∈ L ( Z ; C ( X )) with | ψ | ≤ l , α ≥ such that αβ < , and Z Z max y ∈ Γ( x,z ) l ( y, z ′ ) Q z ( dz ′ ) ≤ αl ( x, z ) , for all x ∈ X , z ∈ Z . Then (B6) holds, with R ( K, z ) = − αβ p K,z ( l ) .Proof. Choose l t = ( αβ ) t l , for t = 0 , , . . . . Then β Z Z max y ∈ Γ( x,z ) l t ( y, z ′ ) Q z ( dz ′ ) = β ( αβ ) t Z Z max y ∈ Γ( x,z ) l ( y, z ′ ) Q z ( dz ′ ) ≤ ( αβ ) t +1 l ( x, z ) = l t +1 ( x , z ) . Hence, w ( x , z ) = − αβ l ( x , z ) and R ( K, z ) = − αβ p K,z ( l ), for K ∈ K and z ∈ Z . Endogenous growth models have become fundamental to understand economic growth. From the hugeliterature studying this field, few contributions consider an unbounded shock space. Some exceptionsare Stachurski (2002) and Kamihigashi (2007), but with uncorrelated shocks. I consider here thestochastic endogenous growth model studied in Jones, Manuelli, Siu and Stacchetti (2005), which isdescribed as follows. The preferences of the agent over random consumptions sequences are given bymax E ∞ X t =0 β t c − σt υ ( ℓ t )1 − σ , (4.1)subject to c t + k t +1 + h t +1 ≤ z t Ak αt ( n t h t ) − α + (1 − δ k ) k t + (1 − δ h ) h t , (4.2) ℓ t + n t ≤ , (4.3) c t , k t , h t , ℓ t , n t ≥ t = 0 , , . . . , with k and h given. Here, { z t } is a Markov stochastic process with transitionprobability Q z ( · ) and Z = [1 , ∞ ); c t is consumption; ℓ t is leisure; n t is hours spent working; k t and h t are the stock of physical and human capital, respectively; δ k and δ h are the depreciation rates onphysical and human capital, respectively; and υ is a continuous function on (0 , k, h, z ) = n ( k ′ , h ′ , c, n, ℓ ) : (4.2)–(4.4) hold with x ′ = x t +1 , x = x t for x = k, h, c, n, ℓ, z o , nd the utility function is U ( c, ℓ ) = c − σ υ ( ℓ )1 − σ . Regarding the function υ , we consider υ ( ℓ ) = ℓ ψ (1 − σ ) .The endogenous state space is X = R + × R + and the family of compact sets K is formed by compactsets in the product space R + × R + . The Markov chain is given by the log–log processln z t +1 = ρ ln z t + ln w t +1 , (4.5)with ρ ≥ w ’s are i.i.d., with support in W ⊆ [1 , ∞ ). Let µ be the distribution measure of the w ’s. Note that ρ = 0 corresponds to shocks z t that are i.i.d.. Jones, Manuelli, Siu and Stacchetti(2005) suppose that z t = exp (cid:16) ζ t − σ ǫ − ρ ) (cid:17) , where ζ t +1 = ρζ t + ǫ t +1 and the ǫ ’s are i.i.d., normalwith mean 0 and variance σ ǫ . This corresponds to (4.5) with w t +1 = exp (cid:16) ǫ t +1 − σ ǫ ρ ) (cid:17) . We donot need to restrict ǫ to be normally distributed. To shorten notation, let us define δ = min { δ k , δ h } , γ = Aα α (1 − α ) − α + (1 − δ ), and g ( k, h ) = Ak α ( nh ) − α + (1 − δ )( k + h ). Also, let Θ = E( w − σ − ρ ) = R W w − σ − ρ µ ( dw ). Theorem 4.1.
Consider the endogenous growth model described in (4.1) – (4.5) with ≤ σ < and ≤ ρ < . If βγ − σ Θ < , (4.6) then the associated Bellman equation admits a unique solution, v ∗ , in the set V (0 , R ) , where, for K ∈ K and z ∈ Z R ( K, z ) = (cid:18) Θ1 − βγ − σ Θ (cid:19) z ρ (1 − σ )1 − ρ max ( k,h ) ∈ K g ( k, h ) − σ . Moreover, v ∗ is the value function v and p K,z ( T n v − v ) converges to 0 as n → ∞ , for all K ∈ K , z ∈ Z and all initial guess v ∈ V (0 , R ) .Proof. We check all the hypotheses of Theorem 3.5. It is clear that (B1)–(B5) are fulfilled. Regarding(B6), we will use Proposition 3.6 for a suitable function l . Since 0 ≤ σ <
1, both U and υ are boundedfrom below by zero, and υ is bounded above by 1. Since z ≥ δ , we have zAk α ( nh ) − α + (1 − δ k ) k + (1 − δ h ) h ≤ zg ( h, k ). Then ψ ( k, h, z ) ≤ − σ z − σ g ( k, h ) − σ ≤ − σ z − σ − ρ g ( k, h ) − σ = l ( k, h, z ) . Let us prove that β R Z b l ( k, h, z, z ′ ) Q z ( dz ′ ) ≤ αl ( k, h, z ), for all ( k, h ) ∈ K , for all z ∈ Z , and for all K ∈ K , where α = γ − σ Θ. Here, to simplify notation in what follows, we have defined b l ( k, h, z, z ′ ) = max ( k ′ ,h ′ ,c,n,ℓ ) ∈ Γ( k,h,z ) l ( k, h, z ) . With correlated shocks, the method developed in Matkowski and Nowak (2011) would require µ ( z ′ ∈ Z j +1 | Z j = z ) = 1 for a suitable increasing family { Z j } ∞ j =1 of compact sets that fills Z . We do not impose thisstrong constraint on µ . In fact, (4.5) do nos satisfy it if W is unbounded and µ has not compact support. Since, from (4.5), z t +1 = z ρt w t +1 , it is clear that, to keep z ≥
1, it is necessary (and sufficient) to have w ≥
1. Thus, the assumption that the random variable ǫ is normally distributed with mean 0 should bemodified to fulfill the requirement that the random variable w has support W in [1 , ∞ ). The assumption z ≥ irst, we determine a bound for b l . To this end, consider the Lagrange problemmax g ( k ′ , h ′ ) − σ , s. t.: k ′ + h ′ ≤ zg ( k, h ) ,k ′ , h ′ ≥ , (4.7)and notice that its feasible set is larger than Γ( k, h, z ). The constraint is binding at the optimal solution,which is k ′ = αzg ( k, h ), h ′ = (1 − α ) zg ( k, h ). Substituting this into the objective function of (4.7),we find its optimal value, γ − σ z − σ g ( k, h ) − σ . Thus, b l ( k, h, z, z ′ ) ≤ − σ ( z ′ ) − σ − ρ γ − σ z − σ g ( k, h ) − σ .Second, we use the conditional expectation R Z ( z ′ ) − σ − ρ Q z ( dz ′ ) = z ρ − σ − ρ Θ to estimate Z Z b l ( k, h, z ′ ) Q z ( dz ′ ) ≤ γ − σ z − σ z ρ − σ − ρ Θ g ( k, h ) − σ = γ − σ Θ l ( k, h, z ) . Since βγ − σ Θ <
1, Proposition 3.6 applies. The expression for R requires a simple computation. Lucas (1978) studied the determination of equilibrium asset prices in a pure exchange economy ina framework that has become classical in the economics and financial literature. Boundedness ofthe utility function, as well as compactness of the space of shocks, are important hypotheses in thedevelopment of this model. In this section, we show that these hypotheses can be dispensed with byusing the results of Theorem 3.5. In this way, we significantly extend the model’s range of applicability.We closely follow Stokey and Lucas with Prescott (1989) in the exposition of the problem. Thepreferences of the representative consumer over random consumption sequences areE ∞ X t =0 β t u ( c t ) , (4.8)where u : R + −→ R + is continuous, not necessarily bounded, with u (0) = 0 and where β ∈ (0 , i = 1 , . . . , k productive assets taking values on a set Z ⊆ R k + , not necessarily compact,with Borel sets Z . The components z i of the vector z = ( z , . . . , z k ) ⊤ in Z represents the dividendpaid by one unit of asset i . In the description of the model, all vectors are considered column vectors,and the symbol ⊤ denotes transposition. We assume that the dividends follow a Markov process,with stationary transition function Q on ( Z, Z ). Assets are traded on a competitive stock marketat an equilibrium price given by a stationary continuous price function p : Z −→ R k + , where p ( z ) =( p ( z ) , . . . , p k ( z )) ⊤ is the vector of asset prices if the current state of the economy is z (the notationfor prices should not be confused with the notation for seminorms, which always carry a subindex).The goal is to characterize equilibrium asset prices. Let x = ( x , . . . , x k ) ⊤ ∈ R k + be the vector of theconsumer’s asset holdings. Given the price function p , the initial state of the economy z and initial If u does not satisfy u ≥ u (0) = 0, but is bounded from below, it may be modified to u ( c ) − u (0) tofulfill our hypotheses. sset holdings x , the consumer chooses a sequence of plans for consumption and end-of-period assetholdings that maximizes discounted expected utility (4.8) subject to c t + x ⊤ t +1 p ( z t ) ≤ x ⊤ t ( z t + p ( z t )) for all z t , for all t , (4.9) c t , x t +1 ≥ z t , for all t . (4.10)The consumer holds exactly one unit of each asset in equilibrium, hence we can restrict the state spaceto X = [0 , x ] k , with x >
1, with its Borel subsets X . The correspondence Γ : X × Z −→ X isΓ( x, z ) = { y ∈ X : y ⊤ p ( z ) ≤ x ⊤ ( z + p ( z )) } . Assuming that p is continuous, Γ is nonempty, compact valued and continuous. Given the price p , thedynamic programming equation is v ( x, z ) = max y ∈ Γ( x,z ) n u ( x ⊤ z + ( x − y ) ⊤ p ( z )) + β Z Z v ( y, z ′ ) Q z ( dz ′ ) o . (4.11)We will look for solutions to this functional equation in the class L ( Z ; C ( X )). Since the state spaceis compact, and the utility function u is bounded from below, we take the trivial family of compactsets K = { X } in this model, and not the whole family of compact subsets of X .We impose the following assumptions. u is nondecreasing and concave; (4.12) p i ( z ) ≤ a ⊤ i z + b i for some vectors a i ≥ b i > , (4.13) i = 1 , . . . , k. Hence, we look for equilibrium prices in the class of functions that are bounded by an affine function.Other possibilities could obviously be explored. We state two results about the existence of equilibriumin a Lucas asset pricing model satisfying (4.12) and (4.13) under two different regimes for the Markovchain.(M1) The Markov chain is given by z t +1 = Bz t + w t , where B a matrix of order k with non–negativeentries and norm k B k <
1, and where { w t } ∞ t =1 are i.i.d. random vectors with support in aBorel subset W ⊆ R k + with finite expectation, 0 ≤ E w < ∞ .(M2) The Markov chain is given by z i,t +1 = z ρ i i,t w i,t +1 , for all t = 0 , , , . . . , where 0 ≤ ρ i ≤ i = 1 , . . . , k , and where { w t } ∞ t =1 are i.i.d. random vectors with support in a Borel subset W ⊆ [1 , ∞ ) k such that ρ i ≤ i = 1 , . . . , k and, if ρ i = 1, then E w i < /β. (4.14) The norm of a matrix B is defined by k B k = sup n k Bx kk x k : x ∈ R l with x = 0 o . The condition k B k < B —the maximum of the module of the eigenvalues of B —isless than one. Hence, we are now considering a linear log–log system of uncoupled equations for the evolution of dividends. n the proof that follows, as well as in the rest of the paper, we use the same notation for inequalitiesbetween scalars and inequalities between vectors, which have to be understood in a pointwise manner. Theorem 4.2.
Consider the Lucas Asset Pricing model described above in (4.8) - (4.10) , for which (4.12) and (4.13) hold and the Markov chain satisfies either (M1) or (M2). Then there is a uniquesolution of (4.11) in V (0 , R ) for a suitable R , which is the value function v of the problem, and theconclusions of Theorem 3.5 hold.Proof. Let A be the matrix whose columns are the vectors a , . . . , a k in (4.13), and let b = ( b , . . . , b k ) ⊤ ,hence we can write 0 ≤ p ( z ) ≤ Az + b . Let us construct a family of functions { l t } ∞ t =0 satisfyingassumption (B6). Obviously x ⊤ ( z + p ( z )) ≤ x ⊤ z ( I k + A ) + x ⊤ b, where I k is the indentity matrix. Since u is increasing and concave, for a supergradient u of u at x ⊤ b >
0, we have ψ ( x, z ) = u ( x ⊤ ( z + p ( z )) ≤ u ( x ⊤ b ) + u x ⊤ ( I k + A ) z. We define l ( z ) = u ( x ⊤ b )+ u x ⊤ ( I k + A ) z and, recursively, l t +1 ( z ) = β R Z l t ( z ′ ) Q ( z, dz ′ ), for t = 0 , , . . . .Suppose first that Q satisfies (M1). In this case, E z z ′ = Bz + E w , hence l ( z ) = β Z Z l ( z ′ ) Q ( z, dz ′ )= βu ( x ⊤ b ) + βu x ⊤ ( I k + A )E z z ′ = β ( u ( x ⊤ b ) + u x ⊤ ( I k + A )( Bz + E w ) . We will prove by induction that l t ( z ) = β t u ( x ⊤ b ) + β t u x ⊤ ( I k + A ) (cid:0) B t z + ( B t − + · · · + I k )E w (cid:1) , for all t = 1 , , . . . . For t = 1 it has been just proved. Suppose it is true for t . Then l t +1 = β t +1 u ( x ⊤ b ) + β t +1 u x ⊤ ( I k + A ) (cid:0) B t E z z ′ + ( B t − + · · · + I k )E w (cid:1) = β t +1 u ( x ⊤ b )+ β t +1 u x ⊤ ( I k + A ) (cid:0) B t ( Bz + E w ) + ( B t − + · · · + I k )E w (cid:1) , and we are done. On the other hand, ( B t − + · · · + I k )E w ≤ ( I k − B ) − E w , since B has nonnegativeentries, k B k < w >
0. Hence, the series w ( z ) = P ∞ t =0 l t ( z ) is unconditionally convergent,since it is bounded by the function of L ( Z ) defined by w ( z ) := 11 − β ( u ( x ⊤ b ) + βu x ⊤ ( I k + A ) (cid:0) ( I k − βB ) − z + ( I k − B ) − E w (cid:1) , where we have used P ∞ t =0 ( βB ) t = ( I k − βB ) − . Hence (B6) holds with R ( z ) = P ∞ t =0 p z ( l t ).If Q satisfies (M2), then E z z ′ = ( z ρ E w , . . . , z ρ k k E w k ) ⊤ . Define, as above, l ( z ) = u ( x · b ) + u x · z ( I k + A ) and let l t +1 ( z ) = β R Z l t ( z ′ ) Q ( z, dz ′ ), for t = 0 , , . . . . Then it is easy to prove by inductionthat l t ( z ) = β t u ( x ⊤ b ) + β t u x ⊤ ( I k + A ) (cid:16) z ρ t Π t − s =0 E (cid:16) w ρ s (cid:17) , . . . , z ρ tk k Π t − s =0 E (cid:16) w ρ sk k (cid:17)(cid:17) ⊤ . y Jensen’s inequality, E (cid:16) w ρ si i (cid:17) ≤ (E w i ) ρ si and thus l t ( z ) ≤ β t u ( x ⊤ b ) + β t u x ⊤ ( I k + A ) (cid:16) z ρ t (E w ) / (1 − ρ ) , . . . , z ρ tk k (E w k ) / (1 − ρ k ) (cid:17) ⊤ . In the case that ρ i < i = 1 , . . . , k , the series w ( z ) = P ∞ t =0 l t ( z ) is clearly (unconditionally)convergent since β < ρ i < i = 1 , . . . , k . The ratio test can be used to prove this claim.In the case in which ρ j = 1 for some j , then the bound above no longer applies, as a term z j (E w j ) t appears in position j of the vector (cid:16) z ρ t Π t − s =0 E w ρ s , . . . , z ρ tk k Π t − s =0 E w ρ sk k (cid:17) ⊤ . However, the assumption β E w j < w ( z ) = P ∞ t =0 l t ( z ).Hence, in both cases considered, (M1) and (M2), the condition (4.14) guarantees that (B6) holdswith R ( z ) = P ∞ t =0 p z ( l t ). Thus, Theorem 3.5 applies.To complete the circle, we have to prove that our conjecture (4.13) about the equilibrium priceholds. Following Lucas (1978) or Stokey and Lucas with Prescott (1989), we now assume the furtherconditions: u (0) = 0, u is continuously differentiable, with u ′ ( c ) > c ≥
0, (4.15)and strictly concave;we also imposethere are constants γ, δ ≥ cu ′ ( c ) ≤ γc + δ, for all c ≥
0; (4.16)there exists a > u ′ ( ⊤ z ) ≥ a for all z ∈ Z , where = (1 , . . . , ⊤ . (4.17)A function like u ( c ) = c − σ / (1 − σ ) + c , with 0 ≤ σ <
1, satisfies (4.15)-(4.17). Also, if Z isbounded, then (4.15) implies (4.17).Finding an equilibrium price function p ( z ) = ( p ( z ) , . . . , p k ( z )) ⊤ is equivalent to finding functions φ ( z ) , . . . , φ k ( z ) that satisfy the k independent functional equations φ i ( z ) = h i ( z ) + β Z Z φ i ( z ′ ) Q ( z, dz ′ ) , i = 1 , . . . , k, (4.18)where h i ( z ) = β R Z z ′ i u ′ ( ⊤ z ′ ) Q ( z, dz ′ ), for all i = 1 , . . . , k . Lucas (1978) shows that a solution to(4.18) provides an equilibrium price p given by p i ( z ) = φ i ( z ) u ′ ( ⊤ z ) , for i = 1 , . . . , k. (4.19)Let, as in Lucas (1978), the operator T i be T i f ( z ) = h i ( z ) + β Z Z f ( z ′ ) Q ( z, dz ′ ) , for all f ∈ L ( Z, Z , Q z ) , i = 1 , . . . , k. Note that, in this context, the seminorms are simply defined by p z ( f ) = R Z | f ( z ′ ) | Q z ( dz ′ ) | . It is prettyclear that the COP associated to T i is given by Lp ( z ) = β Z Z p ( z ′ ) Q z ( dz ′ ) , where p belongs to a suitable set C as defined in (3.6). heorem 4.3. Consider the Lucas Asset Pricing model described above in (4.8) - (4.10) , for which (4.15) – (4.17) hold and the Markov chain satisfies either (M1) or (M2). Then there is an equilibriumprice p satisfying (4.13) .Proof. Note that z ≥ z i u ′ ( ⊤ z ) ≤ ( ⊤ z ) u ′ ( ⊤ z ) ≤ γ ( ⊤ z ) + δ. (4.20)Suppose that Q satisfies (M1). Then h i ( z ) ≤ β (cid:0) γ (cid:0) ⊤ E z z ′ (cid:1) + δ (cid:1) ≤ β (cid:0) γ (cid:0) ⊤ ( Bz + E w ) (cid:1) + δ (cid:1) . This implies that the operator T i is a self–map in L ( Z ), for all i = 1 , . . . , k . We want to applyTheorem 3.5 to each of the operators T i , where the COP associated to T i is given just above thetheorem. Let l ( z ) = h i ( z ) and define l t ( z ) = β t (cid:0) γ (cid:0) ⊤ ( B t z + E w ) (cid:1) + δ (cid:1) , for t = 1 , , . . . . It is immediate to check that l t +1 ≥ β R Z l t ( z ′ ) Q ( z, dz ′ ) and that the series w ( z ) = P ∞ t =0 l t ( z ) is (unconditionally) convergent, since β k B k <
1. The sum of this series is w ( z ) = γ ⊤ ( I k − βB ) − z + − β ( γ ⊤ E w + δ ). Hence, (B6) holds. Moreover, following analogous reasonings as in theproof of part (b) of Theorem 3.5, the fixed point of T i , φ i , satisfies φ i ≤ w , and thus, by (4.17) p i ( z ) = φ i ( z ) u ′ ( ⊤ z ) ≤ a γ ⊤ ( I k − βB ) − z + 1 a (1 − β ) ( γ ⊤ E w + δ ) , for all i = 1 , . . . , k , where the right hand side is an affine function of z . Then, p = ( p , . . . , p k ) ⊤ satisfies(4.13) with a ⊤ i = 1 a γ ⊤ ( I k − βB ) − , b i = 1 a (1 − β ) ( γ ⊤ E w + δ ) , for all i = 1 , . . . , k. Suppose that Q satisfies (M2). Now E z z ′ = ( z ρ E w , . . . , z ρ k k E w k ) ⊤ . From (4.20), we have h i ( z ) ≤ β ( γ ( ⊤ ( z ρ E w , . . . , z ρ k k E w k )) + δ ) . Let l ( z ) = h i ( z ) and l t +1 ( z ) = β R Z l t ( z ′ ) Q ( z, dz ′ ), for t = 0 , , . . . . When 0 ≤ ρ i < i = 1 , . . . , k , using similar arguments as in the proof of Theorem 4.2, we have l t ( z ) ≤ β t γ ⊤ (cid:16) z ρ t (E w ) / (1 − ρ ) , . . . , z ρ tk k (E w k ) / (1 − ρ k ) (cid:17) + β t δ ≤ β t ( γµ ⊤ z + δ ) , where µ := max (cid:8) z (E w ) / (1 − ρ ) , . . . , z k (E w k ) / (1 − ρ k ) (cid:9) . Hence, the infinite series w ( z ) = P ∞ t =0 l t ( z ) = γµ ⊤ z + δ − β is (unconditionally) convergent, (B6) holds and the fixed point of T i , φ i , satisfies φ i ≤ w .It is clear then that the price p = ( p , . . . , p k ) ⊤ defined in (4.19) satisfies (4.13) with a i = γµa (1 − β ) and b i = δa (1 − β ) , for all i = 1 , . . . , k . In the case in which some ρ j = 1, the coordinate j on the vector (cid:16) z ρ t Π t − s =0 E( w ρ s ) , . . . , z ρ tk k Π t − s =0 E( w ρ sk k ) (cid:17) is equal to z j E w j , and then β E w j < P ∞ t =0 l t ( z ), which is then bounded by an affine expression in z ; hence, as in theprevious case, (4.13) holds. Conclusions
In this paper, we develop a general framework to analyze stochastic dynamic problems with unboundedutility functions and unbounded shock space. We obtain new results concerning the existence anduniqueness of solutions to the Bellman equation through a fixed point theorem that generalizes theresults known for Banach contractions and local contractions. This generalization is possible by con-sidering seminorms that give a different treatment to the endogenous state variable and the exogenousone. While a supremum norm on arbitrary compact sets is considered in the former variable, an L type norm is in the latter variable. Putting together this definition with the aforementioned general-ization of the local contraction concept, we are able to maintain the monotonicity (in a mild sense) ofthe Bellman operator, thus proving that it is essentially a contractive operator. The usefulness of theapproach and the applicability of the results are clearly revealed in the analysis of two fundamentalmodels of economic analysis: an endogenous growth model with a multiplicative structure in the shocksand the Lucas model of an exchange economy. The combination of unbounded rewards and unboundedshocks makes it hard to prove the existence of a unique fixed point of the Bellman equation. In thissense, another benefit of the paper is to provide a secure method to check the hypotheses needed toapply the approach, based on assumption (B6), and one that can be used straightforwardly to analyzeother models. A challenging problem is to extend the theorems to deal with the unbounded frombelow case in a more satisfactory way, as done in Rinc´on-Zapatero and Rodr´ıguez-Palmero (2003) orMartins da Rocha and Vailakis (2010) for the deterministic case, by introducing a suitable family ofpseudodistances. Proofs of auxiliary results
Proof of Lemma 2.4 . Due to the subhomogeneity of L for finite sums, L ( r + Lr + · · · + L T r ) ≤ Lr + · · · + L T +1 r ≤ R , for all finite T . Letting T → ∞ , we obtain r + LR ≤ R . Let x ∈ V F ( x , R ),so d a ( x , x ) ≤ R ( a ) for all a ∈ A . By the triangle inequality and since T is an L –local contraction d a ( x , T x ) ≤ d a ( x , T x ) + d a ( T x , T x ) ≤ d ( a ) + ( Ld a )( x , x ) ≤ d ( a ) + ( LR )( a ) ≤ R ( a ) . This proves (a). To show (b), note that, by the same arguments used to prove (a), for L t R ≤ L t r + L t +1 r + · · · , for all t = 0 , , . . . . Then L t R ( a ) is bounded by the remainder of the convergentseries R ( a ), thus it converges to 0 as t → ∞ , for all a ∈ A . Q.E.D.
Given f ∈ Ca( X × Z ), we denote b f ( x, z ) := max y ∈ Γ( x,z ) f ( y, z ) , c | f | ( x, z ) := max y ∈ Γ( x,z ) | f ( y, z ) | . We will make use of the following lemma in the main text and along this appendix.
Lemma A.1. (1) For all f ∈ Ca( X × Z ) , both b f , c | f | ∈ Ca( X × Z ) .(2) For all f ∈ L ( Z ; C ( X )) , both b f , c | f | ∈ L ( Z, Z , Q z ) , for all z ∈ Z .Proof. (1) Given the assumption made about the continuity of Γ, by the Berg´e Theorem of the Maxi-mum, the map x b f ( x, z ) is continuous, for any z ∈ Z fixed, and by the Measurable Theorem of theMaximum, z b f ( x, z ) is Borel measurable; thus, b f is a Carath´eodory function on X × Z . Obviously,the same is true for c | f | .(2) Since p K,z ( f ) < ∞ for all K ∈ K and all z ∈ Z , and Γ( x, z ) is a compact set for any x ∈ X , z ∈ Z , then R Z c | f | ( y, z ′ ) Q z ( dz ′ ) = p Γ( x,z ) ,z ( f ) < ∞ . Obviously, the same is true for b f . Proof of Lemma 3.4 . We organize the proof in several previous lemmas.
Lemma A.2.
Let assumptions (B1) to (B6) to hold. Then1. P ∞ t =0 L t p l < ∞ ;2. R [Γ] ∈ L ( Z ; C ( X )) and p l + LR ≤ R .Proof. Given x ∈ X and z ∈ Z , βp l t [Γ]( x, z ) = β R Z max y ∈ Γ( x,z ) l t ( y, z ′ ) Q z ( dz ′ ) ≤ l t +1 ( x, z ), hence p l t [Γ] ∈ L ( Z, C ( X )) and then Lp l t ( K, z ) = βp K,z ( p l t [Γ]) ≤ p K,z ( l t +1 ), for all t = 0 , , . . . . Thus, L t p l ≤ L t − p l ≤ · · · ≤ p l t . By (B6), the series P ∞ t =0 p l t ( K, z ) converges for all K ∈ K and z ∈ Z ,thus P ∞ t =0 L t p l converges. To conclude the proof, by the triangle inequality p K,z ( p l [Γ] + · · · + p l t [Γ]) ≤ p K,z ( p l [Γ]) + · · · + p K,z ( p l [Γ]) ≤ ( p K,z ( l ) + · · · + p K,z ( l t +1 ) . Letting t → ∞ and adding p K,z ( ψ ) to both sides of the above inequality, we have p K,z ( ψ )+ p K,z ( R [Γ]) ≤ R ( K, z ), showing at the same time that R [Γ] ∈ L ( Z ; C ( X )). emma A.3. Let assumptions (B1) to (B6) to hold. Then f ∈ V (0 , R ) implies T f ∈ L ( Z ; C ( X )) .Proof. Let f ∈ L ( Z ; C ( X )). We use the notation f x and f z , whose meaning is explained in AppendixB. The function f x is Borel measurable for all x ∈ X and Q z –integrable for any z ∈ Z . Thus, f x can bewritten as the difference of two positive, Q z –integrable functions, f x = f + x − f − x , where f + x = max( f x , f − x = max( − f x , M f + x and M f − x are Borel measurable. Since ( M f ) x = M ( f x ) = M ( f + x ) − M ( f − x ), ( M f ) x is measurable forany x ∈ X . To see that ( M f ) z is continuous, consider a sequence { x n } in X that converges to x ∈ X .Then the sequence and its limit form the compact set K = { x n }∪{ x } . Let f n := f x n , for n ≥
1. For all z ′ ∈ Z , f n ( z ′ ) → f x ( z ′ ) as n → ∞ , since f is continuous in x . Moreover, | f z ′ | ≤ sup x ∈ K | f z ′ ( x ) | , and z ′ sup x ∈ K | f z ′ ( x ) | is Q z –integrable by definition of L ( Z ; C ( X )), thus by the Lebesgue dominatedconvergence theorem( M f )( x n , z ) = Z Z f n ( z ′ ) Q z ( dz ′ ) → Z Z f x ( z ′ ) Q z ( dz ′ ) = ( M f )( x, z ) , thus ( M f ) z is continuous. Hence, M f is a Carath´eodory function and thus U ( x, y, z ) + βM f ( y, z ) iscontinuous in ( x, y ) for all z , and it is Borel measurable in z for all ( x, y ). By the Berg´e MaximumTheorem, the function T f is thus continuous in x for all z , and by the Measurable Maximum Theorem,it is Borel measurable for any x . In short, the function( x, z ) T f ( x, z ) = max y ∈ Γ( x,z ) ( U ( x, y, z ) + βM f ( y, z )) , is a Carath´eodory function. Moreover, if f ∈ F and x ∈ X , z ∈ Z | T f ( x, z ) | ≤ | max y ∈ Γ( x,z ) U ( x, y, z ) | + β max y ∈ Γ( x,z ) Z Z max y ∈ Γ( x,z ) | f ( y, z ′ ) | Q z ( dz ′ ) ≤ l ( x, z ) + β Z Z max y ∈ Γ( x,z ) w ( y, z ′ ) Q z ( dz ′ ) ≤ l ( x, z ) + βp Γ( x,z ) ,z ( f ) . Since Γ( x, z ) ∈ K , for f ∈ V (0 , R ), we have p Γ( x,z ) ,z ( f ) ≤ R [Γ]( x, z ). By Lemma A.2, p K,z ( l ) + βp K,z ( R [Γ]) ≤ R ( K, z ). Hence p K,z ( T f ) ≤ R ( K, z ). This proves that
T f ∈ V (0 , R ), and hencethat T f ∈ L ( Z, C ( X )). Lemma A.4.
Let assumptions (B1) to (B6) to hold. Then D ( V (0 , R )) ⊆ C .Proof. Since f ∈ V (0 , R ), p f ≤ R , hence we can take c = 1. Also, p f ∈ Ca( X × Z ), since p f [Γ]( x, z ) = R Z max y ∈ Γ( x,z ) | f ( y, z ′ ) | Q z ( dz ′ ) is continuous in x and Borel measurable in z , by LemmaA.1. Moreover, p f [Γ] ≤ R [Γ] implies p K,z ( p f [Γ]) ≤ p K,z ( R [Γ]) ≤ β R ( K, z ), by Lemma A.2. Hence, p f [Γ] ∈ L ( Z, C ( X )).Now, we are in position to prove Lemma 3.4. First, let us see that L : C −→ C . Let p ∈ C ; by thedefinition of the operator L and Lemma A.2 Lp ( K, z ) = βp K,z ( p [Γ]) ≤ βp K,z ( cR [Γ]) ≤ cR ( K, z ) ≤ cR ( K, z ) , nd so, Lp [Γ] ≤ cR [Γ] and Lp [Γ] ∈ L ( Z, C ( X )). Second, we prove that the assumptions (I) to (VI)are fulfilled. Regarding (I), note that p + q ∈ C if p, q ∈ C , trivially, as well it is also immediatethat if p ′ ∈ C and p ≤ p ′ , then p ∈ C . On the other hand, if a countable chain of partial sums p , p + p , p + p + p , . . . , is bounded by an element P in C , then the infinite sum, p := P ∞ n =0 p n , is welldefined and p ≤ P ≤ cR for some constant c . Moreover, since p [Γ] ≤ cR [Γ] and R [Γ] ∈ L ( Z ; C ( X ))by Lemma A.2, the Monotone Convergence Theorem implies that p [Γ]( x, · ) is Q z –integrable for all z ∈ Z and all x ∈ X . On the other hand, each function p i [Γ]( · , z ) is continuous in x , for all i = 1 , , . . . .By the Wierstrass M test, the function p [Γ]( · , z ) is also continuous in x for all z ∈ Z . These twoobservations imply that p [Γ] ∈ L ( Z ; C ( X )). (II) is trivial; (III) holds, since the integral is monotone,and regarding (IV), it holds true, since for all p, q ∈ C , p K,z ( p [Γ] + q [Γ]) ≤ p K,z ( p [Γ]) + p K,z ( q [Γ]) bydefinition of the seminorms p K,z , hence L ( p + q )( K, z ) = p K,z ( p [Γ] + q [Γ]) ≤ p K,z ( p [Γ]) + p K,z ( q [Γ])= Lp ( K, z ) + Lq ( K, z ) .L is clearly sup-preserving in C by the Monotone Convergence Theorem, hence (V) also holds. Finally,(VI) is implied by Lemma A.2 and Lemma A.3. B Function space
We describe in this section the function space used in Section 3 and we prove Lemma 3.1.Let the measurable space ( Z, Z ), where Z is the space of shocks and Z is the σ –algebra of Borelof Z . Remember that Q is a transition function Q : Z × Z → [0 ,
1] satisfying1. for each z ∈ Z , Q ( z, · ) is a probability measure on ( Z, Z ); and2. for each A ∈ Z , Q ( · , A ) is a Borel measurable function.To simplify notation, let Q z = Q ( z, · ). For each z ∈ Z , let L ( Z, Z , Q z ) be the space of Borelmeasurable and Q z –integrable functions g : Z −→ R . Let the L norm with respect to the fixedprobability measure Q z k g k z = Z Z | g ( z ′ ) | Q z ( dz ′ ) , where the notation k · k z means that integration is with respect to the probability measure Q z . Inwhat follows we will omit the σ –algebra Z from the notation.A function f : X × Z −→ R is a Carath´eodory function on X × Z if it satisfies1. for each x ∈ X , the function f x := f ( x, · ) : Z −→ R is Borel measurable;2. for each z ∈ Z , the function f z := f ( · , z ) : X −→ R is continuous.Under our assumptions, a Carath´eodory function is jointly measurable in X × Z , see Aliprantis, and Burkinshaw(1990), Lemma 4.50. Also, a function that is Carath´eodory on X × Z is obviously Carath´eodory on A × Z for all A ⊆ X . Let us denote by Ca( A × Z ) the set of all Carath´eodory functions on A × Z . et K denote the family of compact subsets K ⊆ X . Given z ∈ Z , consider the probability measure Q z ( · ). For a Carath´eodory function f on X × Z , let p K,z ( f ) = Z Z max x ∈ K | f ( x, z ′ ) | Q z ( dz ′ ) . Note that this integral is well defined, as for a Carath´eodory function f and compact set K ⊆ X , z ′ max x ∈ K | f x ( z ′ ) | is Borel measurable by the Measurable Maximum Theorem, see Aliprantis, and Burkinshaw(1990). Consider the space of Carath´eodory functions f on X × Z for which p K,z ( f ) is finite E := { f ∈ Ca( X × Z ) : p K,z ( f ) < ∞ for all K ∈ K , z ∈ Z } . It is clear that p K,z is a seminorm on E (but not a norm, obviously). Given K ∈ K and z ∈ Z , let E K,z := { f ∈ Ca( K × Z ) : p K,z ( f ) < ∞} . Lemma B.1.
For each K ∈ K and z ∈ Z , ( E K,z , p
K,z ) is a Banach space.Proof. It is clear that E K,z is a vector space and that p K,z is a norm Note that p K,z ( f ) = 0 implies R Z | f ( x, z ′ ) | Q z ( dz ′ ) = 0 for all x ∈ K , hence f ( x, z ′ ) = 0 for all x ∈ K , Q z –a.s.. Hence f = 0.Let { f n } be a Cauchy sequence. Then p K,z ( f n − f m ) → n, m → ∞ . There is n (1) such that p K,z ( f n (1) − f n ) < − for all n ≥ n (1). Now choose n (2) > n (1) such that p K,z ( f n (2) − f n (1) ) < − for all n ≥ n (2) and so on. Thus, we obtain a subsequence n ( j ) such that p K,z ( f n ( j +1) − f n ( j ) ) < − j for j = 1 , , . . . . Let, to simplify notation, f j := f n ( j ) and let g j ( z ′ ) := sup x ∈ K | f j +1 ( x, z ′ ) − f j ( x, z ′ ) | .Note that, for all N = 1 , , . . . N X j =1 Z Z g j ( z ′ ) µ z ( dz ′ ) < − + · · · + 2 − N < , hence G N = P Nj =1 g j is a monotone sequence of positive and integrable functions. By the MonotoneConvergence Theorem, the function G ( z ′ ) = lim N →∞ G N ( z ′ ) is integrable, and thus finite Q z –a.e., seeDieudonn´e (1974), (13.6.4). From this it follows that the series P ∞ j =1 g j ( z ′ ) converges Q z –a.e. Since ∞ X j =1 | f j +1 ( x, z ′ ) − f j ( x, z ′ ) | ≤ N X j =1 g j ( z ′ ) , (B.1)the series on the left hand side also converges for any x ∈ K , Q z –a.e. Consider the series f ( x, z ′ ) + ∞ X j =1 (cid:0) f j +1 ( x, z ′ ) − f j ( x, z ′ ) (cid:1) . (B.2)It converges Q z –a.e., and if f ( x, z ′ ) is its sum, note that by (B.1) and the Weierstrass M–test, theconvergence is uniform in the compact set K . Since every f j is a Carath´eodory function, the limit f ( x, z ′ ) is continuous in x . Moreover, the limit is measurable in z ′ and R Z max x ∈ K | f ( x, z ′ ) | Q z ( dz ′ ) ≤ R Z max x ∈ K (cid:0) | f ( x, z ′ ) | + G ( x, z ′ ) (cid:1) Q z ( dz ′ ) < ∞ , hence f ∈ E K,z . Let us see that the convergence of { f n } to f is in the norm p K,z . To show this, note that the convergence of the series (B.2) is uniformin x ∈ K , hence max x ∈ K | f j ( x, z ′ ) − f ( x, z ′ ) | ends to 0 as j → ∞ and thus, p K,z ( f n ( j ) − f ) → j → ∞ . However, p K,z ( f n − f m ) → n, m → ∞ , hencelim n,j →∞ p K,z ( f n − f ) ≤ lim n,j →∞ p K,z ( f n − f n ( j ) ) + lim n,j →∞ p K,z ( f n ( j ) − f ) = 0 . Thus, lim n →∞ p K,z ( f n − f ) = 0.Let H be the family of all finite subsets H of Z and let, for H ∈ H , p K,H ( f ) = sup z ∈ H p K,z . Note that p K,H is a norm on E K,H := { f ∈ Ca( K × Z ) : p K,H ( f ) < ∞} , and that, by Lemma B.1, ( E K,H , p
K,H ) is a Banach space, since the norms p K,H and p K,z are equivalent,for any z ∈ H , and generate the same topology, see Dieudonn´e (1974), (12.14.7). The importance ofchoosing this family of seminorms instead of the original one is that P H is a directed family by inclusion.That is, if we define H ≤ H ′ if H ⊆ H ′ , then for any H, H ′ ∈ H , there exists H ′′ ∈ H such that p K,H ′′ ≥ max { p K,H , p
K,H ′ } .An element in E K,H can be written f + M K,H in equivalence class notation where M K,H = { f ∈ E K,H : f x = 0 Q z –a.e., for all x ∈ K , for all z ∈ H } . Consider the sets E K,H := { f ∈ Ca( K × Z ) : p K,H ( f ) < ∞} , and the set of Carath´eodory functions that are integrable with respect to all z ∈ Z , E K := { f ∈ Ca( K × Z ) : p K,H ( f ) < ∞ for all H ∈ H} . Note that E K,H is the quotient space E K,H /M K,H . Let us define M K = T H ∈H M K,H and considerthe quotient space E K := E K /M K , formed by equivalence classes of functions in E K with respect tothe relation M K . That is, two functions of E K are in the same equivalence class if and only if, for any x ∈ K , f x ( z ′ ) = g x ( z ′ ) Q z –a.e., for all z ∈ Z . Lemma B.2. E K = \ H ∈H E K,H . Proof.
Let f + M K ∈ E K . Then f + M K ⊆ f + M K,H and p K,H ( f + M K ) = p K,H ( f + M K,H ) < ∞ for all H ∈ H , hence f + M K ∈ E K,H for all H ∈ H . Reciprocally, if g is a representative element ofan equivalence class in T H ∈H E K,H , then there is f ∈ E K,H such that g = f + M K,H for all H . Hence g − f ∈ M K,H for all H , and thus f − g ∈ M K , or g = f + M K , and hence g ∈ E K .We consider on E K the topology τ H generated by the family of seminorms P H = ( p K,H ) H ∈H . Weshow in the next result that ( E K , P H ) is the projective limit of the family of Banach spaces ( E K,H ) H ∈H ,lim ← E K,H , and thus it is a complete locally convex topological space. emma B.3. For each K ∈ K , ( E K , P H ) is a Hausdorff complete locally convex space.Proof. Let τ H be the projective topology on E K with respect to the family of Banach spaces ( E K,H , p
K,H ) H ∈H .The family H is directed by inclusion. Given H, H ′ ∈ H with H ≤ H ′ , let the linear mapping q HH ′ : E K,H ′ → E K,H be given by q HH ′ ( f + M H ′ ) = f + M H ′ . This is well defined, since E K,H ′ ⊆ E K,H and M H ′ ⊆ M H , for H ≤ H ′ . Clearly, each q HH ′ is continuous. Then, by Example 2.2.7 inBogachev and Smolyanov (2017), the projective limit lim ← E K,H coincides with F := T H ∈H E K,H .But E K = F by Lemma B.2. To see that τ H is Hausdorff (or separated), we have to prove that forall nonzero element g ∈ lim ← E K,H , there is H ∈ H and a neighborhood of the zero equivalence class, U K,H ⊆ E K,H , such that g + M K,H / ∈ U K,H , see Schaefer (1971) (II.5.1). Since g is nonzero, we havethat g + M K,H is nonzero for all H ∈ H , hence there is H ∈ H such that p K,H ( g + M K,H ) = p K,H ( g ) = δ >
0. Then, letting U K,H = { f + M K,H ∈ E K,H : p K,H ( f + M K,H ) < δ/ } , we are done.To conclude the proof, note that the projective limit of Banach spaces is a complete locally convexspace, see Schaefer (1971) (II.5.3). Proof of Lemma 3.1
As in the the previous lemma, we work with the directed family of semi-norms P H . As discussed above, it generates the same topology as P . Let P = Q K ∈K E K , endowedwith the Tychonoff product topology. By Lemma B.3, each E K is a complete locally convex space,thus P is also a complete locally convex space. Moreover, P is obviously Hausdorff, since the familyof seminorms is separating. Let us see that there is a linear homomorphism between E and P , so thelemma follows. Let φ : E −→ P be defined by φ ( f ) = ( f K ) K ∈K , where f K ∈ E K is the restrictionof f to K × Z (we dismiss now the equivalence class notation used in the proof of Lemma B.3, asthere is no possible confusion here, as the equivalence relation is M , defined prior to Lemma B.3). Itis clear that φ is linear and one-to-one, since f = 0 implies that there is K ∈ K such that f K = 0.It is also suprajective, since, under our hypotheses on X , Z and Q , every function f K in E K can beextended to a Carath´eodory function f K : X × Z −→ R , see Kucia (1998), Corollary 3. Consider thefunction f ( x, z ) = f K ( x, z ) if x ∈ K . This definition is consistent, since for another compact set K ′ ,if x ∈ K ∩ K ′ , then f K ( x, z ) = f K ∩ K ′ ( x, z ) = f K ′ ( x, z ). Moreover, f is a Carath´eodory function: foreach z ∈ Z , the restriction of f to a compact set K is continuous; hence, since X is locally compact, f is continuous; see Willard (1970), Lemma 43.10. Also, it is trivial that f is Borel measurable withrespect to z . Hence, for any ( f K ) K ∈K ∈ P , we have proved the existence of a function f in E forwhich φ ( f ) = ( f K ) K ∈K , and hence φ is suprajective. It remains to show that φ is continuous and thatit is open. Let π K : P → E K be the projection of E onto E K , defined as follows: if f ∈ E , then π K ( f ) = f K . The mappings π K are continuous by definition of the Tychonoff topology. Note that π K ◦ φ = π K , hence by Schaefer (1971) (II.5.2), φ is continuous. Moreover, from the previous identity, π K ◦ φ − = π K , hence by the same argument as above, φ − is continuous. C Continuity of the Markov operator
In this section we investigate the continuity of the fixed point of the Bellman operator in the variables( x, z ). Our exploration is not the most general possible. We restrict ourselves to a case which is common n many models in economics. General results about the continuity of the Markov operator M can beconsulted in Serfozo (1982) and Hern´andez-Lerma and Lasserre (2000). We state the following simpleresult. Lemma C.1.
Let f ∈ L ( Z ; C ( X )) . Suppose that there is a σ -finite measure λ on Z such that Q z is absolutely continuous with respect to λ , for all z ∈ Z , with density (or Radon-Nicodym derivative) ϕ ( z, z ′ ) , continuous with respect to z and such that, for all compact set K in X and K in Z , thereexists a function h ∈ L ( Z ) such that | f ( x, z ′ ) ϕ ( z, z ′ ) | ≤ h ( z ′ ) for almost all z ′ ∈ Z and all x ∈ K , z ∈ K . Then M f is continuous in ( x, z ) .Proof. The assumptions on Q imply M f ( x, z ) = Z Z f ( x, z ′ ) Q z ( dz ′ ) = Z Z f ( x, z ′ ) ϕ ( z, z ′ ) λ ( dz ′ ) . Theorem 20.3 in Aliprantis, and Burkinshaw (1990) applies to f ( x, z ′ ) ϕ ( z, z ′ ), hence M f is continuous.The issue of continuity of the value function in the unbounded case (and unbounded space ofshocks) is not an easy one. The translation of Lemma 12.14 in Stokey and Lucas with Prescott (1989)to this case is not straightforward, even if the Markov chain is strong Feller continuous. Recall that Q has the weak (strong) Feller property if M maps bounded continuous functions ( resp. boundedmeasurable functions) on Z into bounded continuous functions.To see the kind of problems that may emerge for unbounded functions, consider the followingexample. Let Z = [0 , ∞ ) and let the transition function Q : Z × Z −→ R be defined as follows: Q (0 , B ) = δ ( B ), where δ is the Dirac measure at the point 0, that is, δ ( B ) = 1 if 0 ∈ B and δ ( B ) = 0 otherwise. For 0 < z < Q ( z, B ) = R B dF z ( z ′ ), where F z ( z ′ ) = , if z ′ = 0; z ′ z + 1 − z, if 0 < z ′ ≤ z ;1 , if z ′ > z . , Finally, for z ≥ Q ( z, B ) = λ ( B ∩ [0 , λ denotes the Lebesgue measure of R .Note that, for 0 < z < F z is a distribution function: it is nondecreasing, continuous except at0, where the right sided limit exists, 0 ≤ F z ≤
1, and Z dF ( z ′ ) = ( z ′ z + 1 − z − | z ′ =0 + Z z z dz ′ = 1 − z + z = 1 . Moreover, it is clear that Q ( · , B ) is Borel measurable. Thus, Q is a transition function. Let f ( y, z ) = f ( z ) be independent of y and continuous in z . Then M f is well defined in this particular example and epends only on z , with ( M f )(0) = R f ( z ′ ) Q (0 , dz ′ ) = f (0) . For 0 < z <
M f )( z ) = Z f ( z ′ ) Q ( z, dz ′ )= Z f ( z ′ ) dF z ( z ′ )= f (0)( z ′ z + 1 − z − | z ′ =0 + Z z f ( z ′ ) z dz ′ = f (0)(1 − z ) + z Z z f ( z ′ ) dz ′ . Note that as f is continuous, the integral above exists, for any 0 < z < M f is continuous for0 < z <
1. For z ≥ M f is constant and given by(
M f )( z ) = Z f ( z ′ ) Q ( z, dz ′ ) = Z [0 , f ( z ′ ) dz ′ . Now, if f is measurable and bounded, there is k > | f | ≤ k , hence − kz z − ≤ R z f ( z ′ ) dz ′ ≤ kz z − , so R z f ( z ′ ) dz ′ tends to 0 as z → + , and then ( M f )( z ) tends to f (0) = ( M f )(0). Also,(
M f )( z ) tends to R [0 , f ( z ′ ) dz ′ = ( M f )(1) as z → − . Thus M f is continuous. Thereby, Q is strongFeller continuous. However, considering the unbounded function g ( z ′ ) = z ′ , we obtain M g (0) = g (0) =0 and M g ( z ) = for z >
0, thus
M g is discontinuous at 0.It is not difficult to find non–trivial continuous functions U for which the dynamic programmingequation with transition probability Q admits discontinuous solutions. For instance, let u ( z, c ) =(1 + z ) c be an utility function that depends on consumption c and shock z , and let Γ( m ) = [0 , m + y ],where m ≥ y > X = R + , Z = [0 , ∞ ], and let a discount factor β such that β < v ( m, z ) = max m ′ ∈ [0 ,m + y ] n (1 + z )( m + y − m ′ ) + β Z [1 , ∞ ) v ( m ′ , z ′ ) Q z ( dz ′ ) o , Notice that this specification corresponds to a pure currency economy model with linear utility, whereagents’ preferences are subject to random shocks. These random shocks are assumed to be governedby the Markov chain Q described above. See Stokey and Lucas with Prescott (1989) for further detailsabout this model. We are simply interested in showing that the value function is not jointly continuousin ( m, z ). It is easily checked that v ( m, z ) = ( m + y + y β − β , if z = 0;(1 + z )( m + y ) + y β − β , if z > , is a solution in the class Ca( R + × R + ), which is not continuous in z . Acknowledgements
Support from the Ministerio de Ciencia, Innovaci´on y Universidades – Agencia Estatal de Investigaci´on,grants ECO2017-86261-P, MDM 2014-0431 and Comunidad de Madrid (Spain), grant MadEco-CMS2015/HUM-3444 is gratefully acknowledged. eferences Aliprantis, C. D. and
Burkinshaw, O. (1990).
Principles of Real Analysis . Second edition. Aca-demic Press, London. ´Alvarez, F. and
Stokey, N. L. (1998). Dynamic programming with homogeneous functions.
J.Econ. Theory , 167–189. Balbus, L. , Reffett, K. and
Wozny, L. P. (2018). On uniqueness of time–consistent Markovpolicies for quasi–hyperbolic consumers under uncertainty.
J. Econ. Theory , forthcoming.
Becker, R. A. and
Boyd III, J. H. (1997).
Capital Theory, Equilibrium Analysis and RecursiveUtility . Blackwell, London.
Bertsekas, D. P. and
Shreve, S.E. (1978).
Stochastic Optimal Control: The Discrete–Time Case .Academic Press Inc., New York.
Binder, M. and
Pesaran, M. H. (1999). Stochastic growth models and their econometric implica-tions.
J. Econ. Growth Blackwell, D. (1965). Discounted dynamic programming.
Ann. Appl. Stat. Bogachev, V.I. and
Smolyanov, O. G. (2017).
Topological Vector Spaces and Their Applications .Springer Verlag.
Boyd III, J. H. (1990). Recursive utility and the Ramsey problem.
J. Econ. Theory Brock, W. A. and
Mirman, L. J. (1972). Optimal economic growth and uncertainty: the discountedcase.
J. Econ. Theory Danthine, J. P. and
Donaldson, J. B. (1981). Stochastic properties of fast vs. slow growingeconomies.
Econometrica Dieudonn´e. J. (1974). ´El´ements d’analyse. Tome II . Gauthier–Villars, Paris.
Donaldson, J. B. and
Mehra, R. (1983). Stochastic growth with correlated production shocks.
J.Econ. Theory Furukawa, N. (1972). Markovian decision processes with compact action spaces.
Ann. Math. Statist. Had˘zi´c, O. and
Stankovi´c, B. (1969). Some theorems on the fixed points in locally convex spaces.
Publications de L’institut Math´ematique Hern´andez-Lerma, O. and
Lasserre, J. B. (1999).
Further Topics on Discrete-Time MarkovControl Processes . Springer, New York. ern´andez-Lerma, O. and Lasserre, J. B. (2000). Fatou’s and Lebesgue’s convergence theoremsfor measures.
J. Appl. Math. Stoch. Anal. Hopenhayn, H. A. and
Prescott, E. C. (1992). Stochastic monotonicity and stationary distribu-tions for dynamic economies.
Econometrica
Jones, L. E. , Manuelli, R. E. , Siu, H. E. and
Stacchetti, E. (2005). Fluctuations in convexmodels of endogenous growth, I: Growth effects.
Rev. Econ. Dynam. Kamihigashi, T. (2007). Stochastic optimal growth with bounded or unbounded utility and withbounded or unbounded shocks.
J. Math. Econ. Kozlov, V. , Thim, J. and
Turesson, B. O. (2010). A fixed point theorem in locally convex spaces.
Collect. Math. Kucia, A. (1998). Some results on Carath´eodory selections and extensions.
J. Math. Anal. Appl.
Lucas Jr, R. E. (1978). Asset prices in an exchange economy.
Econometrica Ma, Q. and
Stachurski, J. (2017). Optimal timing of decisions: A general theory based on contin-uation values.
ArXiv Preprint arXiv:1703.09832v1 . Maitra, A. (1968). Discounted dynamic programming on compact metric spaces.
Sankhya Ser. A Majumdar, M. , Mitra T. and
Nyarko, Y. (1989). Dynamic optimization under uncertainty: non-convex feasible set. In: Feiwel, G. R. (ed.)
Joan Robinson and modern economic theory
Martins da Rocha, V. F. and
Vailakis, Y. (2010). Existence and uniqueness of a fixed point forlocal contractions.
Econometrica Matkowski, I. J. and
Nowak, I. (2011). On discounted dynamic programming with unboundedreturns.
Econ. Theory Mirman, l. J. and
Zilcha, I. (1975). On optimal growth under uncertainty.
J. Econ. Theory Mitra, K. (1998). On capital accumulation paths in a neoclassical stochastic growth model.
Econ.Theory Olson, L. J. and
Roy, S. (2006). Theory of stochastic optimal economic growth. In
Handbook onOptimal Growth 1 , 297–335. Springer-Verlag, Berlin, Heidelberg.
Rinc´on-Zapatero, J. P. and
Rodr´ıguez-Palmero, C. (2003). Existence and uniqueness of solu-tions to the Bellman equation in the unbounded case.
Econometrica inc´on-Zapatero, J. P. and Rodr´ıguez-Palmero, C. (2007). Recursive utility with unboundedaggregators.
Econ. Theory Rinc´on-Zapatero, J. P. and
Rodr´ıguez-Palmero, C. (2009). Corrigendum to “Existence anduniqueness of solutions to the Bellman equation in the unbounded case”.
Econometrica Schaefer, H. H. (1971).
Topological Vector Spaces . Springer, New York, NY.
Serfozo, R. (1982). Convergence of Lebesgue integrals with varying measures.
Sankhya: The IndianJ. of Stats. Stachurski, J. (2002). Stochastic optimal growth with unbounded shock.
J. Econ. Theory
Stokey, N. and
Lucas, R. E. with
Prescott, E. C. (1989).
Recursive Methods in EconomicDynamics . Harvard University Press, Cambridge, MA.
Willard, S. (1970).
General Topology . Addison Wesley, Reading, Massachusetts.
Wong, J. S. W. (1968). Two extensions of the Banach contraction mapping principle.
J. Math. Anal.Appl.438–443.