DDissipativity and optimal control
Lars Gr¨uneChair of Applied Mathematics, University of Bayreuth, [email protected] 1, 2021
The close link between dissipativity and optimal control is already apparent in Jan C.Willems’ first papers on the subject. Particularly, the paper [46], which appeared one yearbefore his famous dissipativity papers [47, 48], already contains a lot of insight on thisconnection for linear quadratic optimal control problems. In recent years, research on thislink has been revived with a particular focus on nonlinear problems and applications inmodel predictive control (MPC).This development was initiated about ten years ago in the paper [9] by Diehl, Amrit, andRawlings, in which strict dissipativity with linear storage function was used in order toconstruct a Lyapunov function for the closed-loop solution resulting from an economicMPC scheme. Soon after it was realized in [1] that linearity of the storage function is notreally needed for this result, meaning that the same Lyapunov function construction canbe carried out for general nonlinear strict dissipativity. In both cases, the link betweendissipativity and optimal control lies in the fact that the running cost (or the stage cost indiscrete time) serves as the supply rate in the dissipativity formulation.While these results use strict dissipativity of the optimal control problem within the MPCscheme in a formal way in order to obtain stability results via an appropriate Lyapunovfunction, the effect of strict dissipativity on optimal trajectories can also be analyzed in amore geometrical fashion. More precisely, generalizing techniques that were already aroundin the 1990s (see, e.g., [6, Theorem 4.2]), it was observed in [17, Theorems 5.3 and 5.6] thatstrict dissipativity plus a suitable controllability property is sufficient for the occurrence ofthe so-called turnpike property. This property, first observed in mathematical economy inthe works of Ramsey and von Neumann in the 1920s and 1930s [39, 45], and first called thisway by Dorfman, Samuelson and Solow in the 1950s [10], describes the fact that optimaltrajectories most of the time stay close to an optimal equilibrium.Based on this observation, in [29, 11, 22, 18] (see also [23, Chapter 7] and [13]) the stabilityresults from [1] could be extended to larger classes of MPC schemes and, in addition, non-averaged or transient approximate optimality of the MPC closed-loop could be established.Motivated by these new applications, the relation of strict dissipativity to classical notions1 a r X i v : . [ m a t h . O C ] J a n LARS GR ¨UNE of detectability for linear and nonlinear systems was also clarified [19, 20, 31]. This papersurveys these recent developments and some of Willems’ early results.
We consider optimal control problems either in continuous timeminimize J T ( x , u ) = (cid:90) T (cid:96) ( x ( t ) , u ( t )) dt (2.1)with respect to u ∈ U , u ( t ) ∈ U and x ( t ) ∈ X for all t ∈ [0 , T ], T ∈ R > , where U is anappropriate space of functions, X ⊂ R n and U ⊂ R m are the sets of admissible states and admissible control inputs , respectively, and˙ x ( t ) = f ( x ( t ) , u ( t )) , x (0) = x , (2.2)or in discrete time minimize J T ( x , u ) = T − (cid:88) k =0 (cid:96) ( x ( t ) , u ( t )) (2.3)with respect to u ∈ U , u ( t ) ∈ U and x ( t ) ∈ X for all t = 0 , . . . , T , T ∈ N , where U is anappropriate space of sequences, X and U are as above, and x ( t + 1) = f ( x ( t ) , u ( t )) , x (0) = x , (2.4)which we briefly write as x + = f ( x, u ). Here f : R n × R m → R n is either the vector fieldin continuous time or the iteration map in discrete time, while (cid:96) : R n × R m → R n is calledthe running cost in continuous time and the stage cost in discrete time. In order to unifythe notation, we use the symbol [ a, b ] both in continuous and discrete time. In continuoustime it denotes the usual closed interval { t ∈ R | a ≤ t ≤ b } , while in discrete time itdenotes { t ∈ Z | a ≤ t ≤ b } , where Z is the set of integers. For simplicity of exposition welimit ourselved to finite dimensional state space but we mention that some of the resultsdiscussed in this paper are also available in infinite dimensional settings. Given an initialvalue x ∈ X , we denote the set of control functions u ∈ U for which x ( t ) ∈ X holds for all t ∈ [0 , T ] by U ( x , T ). The optimal value function is then defined as V T ( x ) := inf u ∈U ( x ,T ) J T ( x , u )and a control u ∗ ∈ U ( x , T ) with corresponding trajectory x ∗ ( · ) is called optimal control for initial condition x and time horizon T if J T ( x , u ∗ ) = V T ( x ) . The corresponding trajectory x ∗ ( · ) is then called an optimal trajectory . ISSIPATIVITY AND OPTIMAL CONTROL Dissipativity in the sense of Willems as we use it in this paper involves an abstract notionof energy that is stored in the system. For each admissible state x ∈ R n , we denote theenergy in the system by λ ( x ). The function λ : R n → R is called storage function and it isusually assumed that λ is bounded from below, in order to avoid that an infinite amount ofenergy can be extracted from the system. In continuous time, dissipativity then demandsthat there exists another function, the so called supply rate s : R n × R m → R , such thatthe inequality λ ( x ( τ )) ≤ λ ( x ) + (cid:90) τ s ( x ( t ) , u ( t )) dt (2.5)holds for all τ ≥
0, all control functions u ∈ U ( x , τ ) and all initial conditions x ∈ X . Asin the optimal control problem (2.1), x ( t ) is supposed to satisfy (2.2). In discrete time, thedemanded inequality completely analogously reads λ ( x ( τ )) ≤ λ ( x ) + τ − (cid:88) t =0 s ( x ( t ) , u ( t )) , (2.6)with x ( t ) satisfying (2.4). It appears that this discrete-time variant of (2.5) was first usedby Byrnes and Lin in [4].The interpretation of these inequalities is as follows: They demand that the energy λ ( x ( τ ))in the system after a certain time τ is not larger than the initial energy λ ( x ) plus theintegral or sum over the supplied energy, expressed at each time instant by s ( x ( t )). Notethat s ( x ( t )) can be negative, which means that negative energy is supplied, i.e. that energyis extracted from the system.In continuous time, if λ is continuously differentiable, inequality (2.5) can equivalently berewritten in infinitesimal form Dλ ( x ) f ( x, u ) ≤ s ( x, u ) , (2.7)while in discrete time equivalently to (2.6) one can use the one-step form λ ( x + ) ≤ λ ( x ) + s ( x, u ) , (2.8)where we use the common brief notation x + = f ( x, u ).This dissipativity concept can be related to the optimal control problem from the previoussection by setting s ( x, u ) := (cid:96) ( x, u ) for the running or stage cost (cid:96) from (2.1) and (2.3),respectively. It appears that this connection was first made in Willems’ paper [46], whichremarkably appeared in the year before his seminal papers [47, 48], which introduced anddiscussed dissipativity in a comprehensive way. More precisely, in [46] continuous-time,linear, controllable dynamics f ( x, u ) = Ax + Bu and quadratic running cost (cid:96) ( x, u ) = x T Qx + 2 u T Cx + u T Ru without any definiteness assumption on Q and R are studied. Thepaper provides necessary and sufficient conditions for the existence of a storage function λ in terms of the finiteness of optimal value functions of certain related optimal controlproblems. These characterizations led to the concepts of available storage and requiredsupply , that are described in Section 6 in the appendix. What is important here that these LARS GR ¨UNE characterizations involve constraints on the asymptotic behavior of the optimal controlproblems under consideration. This already indicates the fundamental connection betweendissipativity and the long-term behavior of optimal trajectories, which was the key reasonfor the recent renaissance of this theory. Before we turn to the description of this recentdevelopment, we introduce a stricter variant of the dissipativity property.
Strict dissipativity is nothing but dissipativity with storage function s ( x, u ) − α ( (cid:107) x (cid:107) )in place of s ( x, u ), where α ∈ K ∞ with K ∞ := { α : [0 , ∞ ) → [0 , ∞ ) | α continuous, strictly decreasing, unbounded, and α (0) = 0 } and x e is an equilibrium of the control system, i.e. there is a control value u e with f ( x e , u e ) = 0 in continuous time and f ( x e , u e ) = x e in discrete time. Written explicitly,the corresponding dissipation inequalities read λ ( x ( t )) ≤ λ ( x ) + (cid:90) t s ( x ( τ ) , u ( τ )) − α ( (cid:107) x ( τ ) − x e (cid:107) ) dτ (2.9)and λ ( x ( k )) ≤ λ ( x ) + k − (cid:88) j =0 s ( x ( j ) , u ( j )) − α ( (cid:107) x ( j ) − x e (cid:107) ) , (2.10)The short versions (2.7) and (2.8) of these inequalities then become Dλ ( x ) f ( x, u ) ≤ s ( x, u ) − α ( (cid:107) x − x e (cid:107) ) , (2.11)and λ ( x + ) ≤ λ ( x ) + s ( x, u ) − α ( (cid:107) x − x e (cid:107) ) , (2.12)respectively.For some applications, it is necessary to use α ( (cid:107) ( x, u ) − ( x e , u e ) (cid:107) ) in place of α ( (cid:107) x − x e (cid:107) ).In this case, one speaks about strict ( x, u ) -dissipativity .One easily sees that (2.9) and (2.10) are more demanding than (2.5) and (2.6), respectively,since they do not only demand that the energy difference λ ( x ( t )) − λ ( x ) is bounded bythe integral or sum over the supplied energy s , but that actually some of the energy isdissipated if the system is not in the equilibrium x e , and the amount of dissipated energyincreases the further away the state x ( τ ) is from x e .At a first glance it seems that the difference between strict and non-strict dissipativityis only quantitative. Indeed, it follows readily from the definition that if the system isdissipative with supply s ( x, u ) then it is strictly dissipative with supply s ( x, u ) + α ( (cid:107) x (cid:107) ).However, if the supply function s is not merely a parameter we can play with but a functionthat results from some modelling procedure, then there is not only a quantitative but also aqualitative difference between strict and non-strict dissipativity. This in particular applies ISSIPATIVITY AND OPTIMAL CONTROL s is derived from the running or stage cost (cid:96) of an optimal controlproblem, i.e., when s = (cid:96) .When using strict dissipativity for the analysis of the control system behavior, it is oftennecessary to demand s ( x e , u e ) = 0, as we will see in the next section. This is in generala restrictive condition, but it is actually not restrictive in case that s is derived from theoptimal control problem. This is because the optimal trajectories for the cost (cid:96) ( x, u ) arethe same as for the cost (cid:96) ( x, u ) − (cid:96) ( x e , u e ). It is thus convenient and not restrictive to usethe supply rate s ( x, u ) = (cid:96) ( x, u ) − (cid:96) ( x e , u e ) . (2.13) The turnpike property demands that optimal trajectories (and in some variants also near-optimal trajectories) stay in the vicinity of an equilibrium x e most of the time. Of course,this statement needs to be made mathematically precise in order to be able to analyzeit mathematically. The meaning of “most of the time” is that the amount of time thetrajectory spends outside a given neighborhood of the equilibrium x e is bounded, and thebound is independent of the optimization horizon and of the initial condition, at least forinitial conditions which itself are contained in a bounded set.Formally, we say that an optimal control problem has the turnpike property , if there is anequilibrium x e , such that for any ε > K >
C >
T > x ∗ ( · ) with x ∗ (0) ∈ B K ( x e ) ∩ X the set of times Q := { t ∈ [0 , T ] | x ∗ ( t ) (cid:54)∈ B ε ( x e ) } satisfies |Q| ≤ C . Here |Q| is the Lebesgue measure of Q in continuous time or the numberof elements contained in Q in discrete time, and B K ( x e ) and B ε ( x e ) denote the balls around x e with radii K and ε , respectively.Figure 3.1 shows a sketch of a trajectory exhibiting the turnpike phenomenon. Here theset Q contains 7 time instants, which are marked with short red vertical lines on the t -axis.The equilibrium x e is represented by the dashed blue line.Figure 3.2 shows optimal trajectories for varying time horizons T = 5 , , . . . ,
19 for twosimple one-dimensional discrete-time optimal control problems. The first problem is givenby x + = u, (cid:96) ( x, u ) = − log(5 x . − u ) , X = [0 , , U = [0 . ,
10] (3.1)and the second by x + = 2 x + u, (cid:96) ( x, u ) = u , X = [ − , , U = [ − , . (3.2)The model (3.1) is an optimal investment problem from [3], in which x denotes the invest-ment in a company and 5 x . is the return from this investment (including the investmentitself) after one time period. As x + = u is the investment in the next time period, 5 x . − u is the amount of money that can be used for consumption in the current time period andthe optimal control problem thus models the maximization of the sum of the logarithmic LARS GR ¨UNE B ε ( x e ) tx ∗ ( t ) Figure 3.1: Illustration of the turnpike property in discrete time. The set Q contains 7time instants, which are marked with short red vertical lines on the t -axis. The equilibrium x e is represented by the dashed blue line.utility function log(5 x . − u ) over the time periods. Clearly, it is optimal to spend all theavailable money until the end of the time horizon T , which is why all optimal trajectoriesend at x = 0. However, in between the trajectories spend most of the time near the equi-librium x e ≈ . X withas little quadratic control effort (cid:96) ( x, u ) = u as possible. In the long run it is beneficialto stay near x e = 0, because the control effort for staying in X is very small near thisequilibrium. Thus, it makes sense for the optimal trajectories to stay near x e = 0 for mostof the time. At the end of the horizon, however, it is beneficial to turn off the controlcompletely, because this entirely reduces the control effort to 0 and does not violate thestate constraint provided the control is turned off sufficiently late.In both examples in Figure 3.2 it is clearly visible that the number of states outside a neigh-borhood of the respective equilibrium (indicated by the dashed blue line) remains constantwith increasing horizon length, which is exactly what the turnpike property demands.These examples show that the turnpike property occurs already in very simple problems. Itis, however, known that it also occurs in many much more complicated problems, includingoptimal control problems governed by partial differential equations; [37, 30, 51, 36, 42, 27, ISSIPATIVITY AND OPTIMAL CONTROL t x ∗ ( t ) t x ∗ ( t ) Figure 3.2: Optimal trajectories for the examples (3.1) and (3.2) for time horizon length T = 5 , , . . . , Q , lie only at thebeginning and at the end of the time interval. One speaks of the approaching arc at thebeginning and the leaving arc at the end of the time horizon. While the general definitionof the turnpike property allows for excursions from x e also in the middle of the time interval(as indicated in the sketch in Figure 3.1), these do not appear in the two examples. Thereason for this will be explained in the next section.The optimal trajectories in Figure 3.2 nicely illustrate the source of the name “turnpikeproperty”, which was coined in [10]: the behavior of the optimal trajectory is similar to acar driving from the initial to the end point, where the equilibrium—i.e., the dashed blueline—plays the role of a highway, or turnpike, as highways are called in parts of the USA. Ifthe time is sufficiently large, it pays off to first go to the highway (even if this is associatedwith some additional cost), stay there for most of the time and then leave the highwayat the end. There may be different reasons for the occurence of the leaving arc. We mayimpose a constraint x ( N ) = ˆ x for the terminal state that forces the trajectory to leave theturnpike, or it may simply be beneficial to leave the turnpike because this reduces the costof the overall trajectory. As we did not impose any terminal constraints, the latter mustbe the case in our examples, and it is actually easy to convince yourself that this is thecase.Besides the definition given above, there are a number of alternative definitions of the turn-pike property. A widely used variant is the exponential turnpike property , which demandsan inequality of the type (cid:107) x ∗ ( t ) − x e (cid:107) ≤ C ( e − σt + e − σ ( T − t ) )for all t ∈ [0 , T ] with constants C, σ >
0. On easily sees that this property implies theturnpike property above, since for each ε > τ > C ( e − σ t + e − σ ( T − t ) ) < ε forall t ∈ [ τ, T − τ ] holds, regardless of how large T is. Another variant demands the turnpikebehavior not only for the optimal trajectories x ∗ ( · ), but for all trajectories correspondingto control functions satisfying J ( x , u ) ≤ V ( x ) + δ , i.e. for all near optimal trajectories . LARS GR ¨UNE
For even more variants and a historic discussion of the turnpike property we refer to therecent survey [12].
Under a boundedness assumption on the optimal value functions it is fairly easy to provethat strict dissipativity implies the turnpike property. The boundedness assumption de-mands that there are a K ∞ -function γ and a constant C >
0, such that the optimal valueand the storage functions satisfy | V T ( x ) − V T ( x e ) | ≤ γ ( (cid:107) x − x e (cid:107) ) + C and | λ ( x ) | ≤ γ ( (cid:107) x − x e (cid:107) ) + C (3.3)for all x ∈ X and all T ≥
0. The first condition can be ensured by a reachability condition:if the equilibrium x e can be reached from every x ∈ X with costs that are bounded inbounded subsets of X , then the inequality for | V T ( x ) − V T ( x e ) | can be established for all T >
0; for details see [17, Section 6].We illustrate the derivation of the turnpike property under these boundedness assumptionsin the discrete time setting. Using (2.10) and (2.13) and choosing
D > λ ( x ) ≥− D for all x ∈ X we obtain J ( x , u ) = T − (cid:88) k =0 (cid:96) ( x ( t ) , u ( t )) ≥ T − (cid:88) k =0 α ( (cid:107) x ( t ) − x e (cid:107) ) − λ ( x ) + λ ( x ( T )) ≥ T − (cid:88) k =0 α ( (cid:107) x ( t ) − x e (cid:107) ) + T (cid:96) ( x e , u e ) − λ ( x ) − D Moreover, using the constant control u ≡ u e we obtain V T ( x e ) ≤ J T ( x e , u ) = T − (cid:88) t =0 (cid:96) ( x e , u e ) = T (cid:96) ( x e , u e ) . Together this implies for x = x ∗ with x ∗ (0) = x γ ( (cid:107) x − x e (cid:107) ) + C ≥ | V T ( x ) − V T ( x e ) | ≥ T − (cid:88) k =0 α ( (cid:107) x ∗ ( t ) − x e (cid:107) ) − λ ( x ) − D. Given an arbitrary ε >
0, if x ∗ spends too much time outside B ε ( x e ), then this inequalityis violated. From this fact the existence of the constant C in the turnpike property can beconcluded. Here the inequalities in (3.3) are needed in order to make sure that the constant C in the turnpike property only depends on the size K of the ball B K ( x e ) containing x and not on the individual initial value x .One easily checks that both examples (3.1) and (3.2) are strictly dissipative, the formerwith the storage function λ ( x ) = α ( x − x e ) /x e and the latter with the storage function ISSIPATIVITY AND OPTIMAL CONTROL λ ( x ) = − x /
2. This explains why we see the turnpike behavior in these examples. Notethat in both examples boundedness of X is important to ensure that λ is bounded frombelow on X .With a more involved proof, strict dissipativity with suitable bounds on the problem datacan also be used in order to obtain an exponential turnpike property, see [8]. Alternatively,the exponential turnpike property can be established using necessary optimality conditions(we refer to [37, 43, 42, 28] for a selection of papers in this direction), but the strictdissipativity based approach has the advantage that it works for arbitrary nonlinearities,while the optimality condition-based approach is typically limited to linear systems or tosmall nonlinearities via linearization arguments.The fact that strict dissipativity implies the turnpike property immediately raises thequestion how much stronger strict dissipativity is than the turnpike property. The answerlies in the observation that the inequality chain, above, can also be used for establishingthe turnpike property for trajectories and controls that are not strictly optimal but onlysatisfy the inequality J T ( x , u ) ≤ V T ( x ) + δ , i.e. they are near-optimal. In other words,we obtain a near-optimal turnpike property . For systems that are locally controllable in aneighborhood of x e and controllable to this neighborhood from all x ∈ X , it was shown in[21] that strict dissipativity is equivalent to the near-optimal turnpike property.Interestingly, the proof of this equivalence relies on a similar equivalence result for plain(i.e., nonstrict) dissipativity: In [34] it was shown that, again under a local controllabilityassumption, dissipativity is equivalent to the fact that the average cost for all admissible u satisfies the inequality lim sup T →∞ T J T ( x , u ) ≥ (cid:96) ( x e , u e ) , (3.4)a property that is known under the name of optimal operation at steady state .Strict dissipativity also explains why optimal trajectories exhibiting the turnpike propertytypically look as in Figure 3.2 and hardly ever as in Figure 3.1. The reason is that asimilar inequality as above shows that an excursion from the equilibrium x e followed bya return to x e causes costs that are significantly higher than staying in x e . Under theassumption that the optimal values are continuous near x e (in the sense that the optimalvalue function near x e has about the same values as in x e — a property that can again berigorously established for locally controllable systems), this implies that such excursionswill not happen in (near-)optimal trajectories.In contrast to that, an excursion from the equilibrium x e can be cheaper then staying in x e if the solution does not return to x e after the excursion. This is precisely the effect thatcreates the leaving arc of the optimal trajectories. As the gain that can be obtained fromthe leaving arc is bounded by the value of the storage function λ ( x ( T )) at the terminalstate of the trajectory, it is important that λ is bounded from below. Indeed, the example(3.2) would cease to be strictly dissipative if we changed X = [ − ,
2] to X = R . In thiscase, it is easily seen that the optimal trajectories would tend to ±∞ (with optimal control u ∗ ≡
0) instead of staying near x e = 0 most of the time, i.e. the turnpike property alsoceases to exist.0 LARS GR ¨UNE
Strict dissipativity generalizes a lot of well-known properties for optimal control problemsensuring a certain asymptotic behavior for the optimal trajectories. To begin with, it iseasily seen that strict dissipativity holds with storage function λ ≡ (cid:96) satisfying (cid:96) ( x e , u e ) = 0 and (cid:96) ( x, u ) ≥ α ( (cid:107) x − x e (cid:107) ) for all x ∈ X and u ∈ U . In case of aquadratic cost (cid:96) ( x, u ) = ( x − x e ) T Q ( x − x e ) + ( u − u e ) T R ( u − u e )this amounts to requiring that Q is positive definite. For linear quadratic problems withdynamics f ( x, u ) = Ax + Bu , generalized quadratic cost (cid:96) ( x, u ) = x T Qx + u T Ru + q T x + r T u with Q = C T C , and no state constraints, i.e., X = R n , strict dissipativity is equivalentto detectability of the pair ( A, C ), i.e. to the fact that all unobservable eigenvalues λ of A satisfy Re λ < | λ | < X is bounded with x e in its interior, then strict dissipativity is equivalent to the fact that all unobservableeigenvalues λ of A satisfy Re λ (cid:54) = 0 in continuous time or | λ | (cid:54) = 1 in discrete time (see [19]and [20] for proofs in discrete and continuous time, respectively). We note that the lastcriterion applies to example (3.2). Moreover, strict dissipativity holds if f is affine and (cid:96) is strictly convex (see [8] for a proof in discrete time that can also be carried over tocontinuous time); this criterion applies to example (3.1). Finally, as shown in [31], strictdissipativity also follows from nonlinear detectability notions, such as the one introducedin [15] or input-output-to-state stability (IOSS) [5]. Model Predictive Control (MPC), as described in Section 7 in the appendix, is a highlypopular control method, in which the computationally challenging solution of an infinitehorizon optimal control problem is replaced by the successive solutions of optimal controlproblems on finite time horizons. As described in the sidebar, this induces an error thatcan be analyzed employing strict dissipativity of the optimal control problem.
Strict dissipativity turns out to be useful for two important aspects of this analysis. Thefirst aspect is the stability analysis. In general, it is not clear, at all, that the MPC closed-loop solutions will exhibit stability-like behaviour. However, under the assumption thatstrict dissipativity holds, we can define an auxiliary optimal control problem by using themodified or rotated running or stage cost˜ (cid:96) ( x, u ) := (cid:96) ( x, u ) − (cid:96) ( x e , u e ) − Dλ ( x ) f ( x, u )in continuous time or˜ (cid:96) ( x, u ) := (cid:96) ( x, u ) − (cid:96) ( x e , u e ) + λ ( x ) − λ ( f ( x, u )) ISSIPATIVITY AND OPTIMAL CONTROL x e , i.e., it satisfies ˜ (cid:96) ( x e , u e ) = 0 and˜ (cid:96) ( x, u ) ≥ α ( (cid:107) x − x e (cid:107) ). Thus, if we consider the optimal control problems (2.1) or (2.3) with˜ (cid:96) in place of (cid:96) , it forces the optimal solutions to approach x e . Denote the correspondingfunctional and optimal value function by (cid:101) J T and (cid:101) V T , respectively, and the optimal solutionby ˜ x ∗ ( · ). Then by either adding a terminal cost (cid:101) F with suitable properties to (cid:101) J T (cf., e.g.,[7, 33], [40, Chapter 2], or [23, Chapter 5]) or by imposing conditions that ensure that (cid:101) V T isclose to (cid:101) V ∞ for sufficiently large T (cf., e.g., [15] or [23, Chapter 6]), one can prove that (cid:101) V T is a Lyapunov function for the MPC closed loop. The trouble is, though, that this is onlytrue if ˜ (cid:96) is used as running cost or stage cost in the MPC scheme. This, however, is oftennot practical because the storage function may not be known and difficult to compute. Inthis case, one would like to keep the original cost (cid:96) in the optimal control problem.In case the running or stage cost ˜ (cid:96) together with the terminal cost (cid:101) F is used, the trick isto define F = (cid:101) F − λ in order to arrive at an equivalent optimal control problem that uses (cid:96) and F and produces the same optimal solutions as the one using ˜ (cid:96) and (cid:101) F . The conditionson the terminal cost needed to make (cid:101) V T a Lyapunov functions can be formulated directlyfor F without having to use (or even to know) λ . The resulting required inequality for F is of the form DF ( x ) f ( x, u ) ≤ − (cid:96) ( x, u ) + (cid:96) ( x e , u e ) (4.1)in continuous time and F ( f ( x, u )) ≤ F ( x ) − (cid:96) ( x, u ) + (cid:96) ( x e , u e ) (4.2)in discrete time, which must hold for all x in the terminal constraint set X and a controlvalue u (depending on x ), which is such that the solution does not leave X . Since theoptimal control problems with ˜ (cid:96) and (cid:101) F on the one side and with (cid:96) and F on the other sideproduce the same optimal solutions, the MPC closed-loops resulting from the two problemsalso coincide and (cid:101) V T can again be used as a Lyapunov function to conclude asymptoticstability. This trick was first proposed in [9] and then refined in [1].Figure 4.1 show the MPC closed-loop trajectories and the corresponding predictions forexample (3.2), using the terminal constraint set X = { } and the terminal cost F ≡ F that meets the required condition(4.1) or (4.2). While the trivial choice X = { x e } and F ≡ X may cause problems in the numerical optimization and result in a smallset of feasible states for (2.1) or (2.3) when the terminal condition x ( T ) ∈ X is added.It may thus be attractive to drop the terminal constraint X and cost F . In this case,however, the trick with passing from (cid:101) F to F is not applicable. In fact, without terminalcosts the optimal trajectories and controls of the problem with (cid:96) and with ˜ (cid:96) do not coincideanymore. However, they may still coincide approximately in a suitable sense.In order to establish this property, we need to assume that the storage function λ and theoptimal value functions V T and (cid:101) V T are continuous in x e , uniformly in T (the former isnot very restrictive since often λ is a polynomial and the latter can again be ensured bylocal controllability, see [17, Section 6]). Next we use that both the problem with cost (cid:96) and the problem with cost ˜ (cid:96) are strictly dissipative and thus exhibit the turnpike property.From this we can conclude that for any two optimal trajectories x ∗ ( · ) and ˜ x ∗ ( · ) starting2 LARS GR ¨UNE t x M P C ( t ) t x M P C ( t ) Figure 4.1: MPC closed-loop trajectories (red solid) and predictions (black dashed) forexample (3.2), using the terminal constraint set X = { } and the terminal cost F ≡ T = 3 (left) and T = 5 (right). Note that the terminal constraint x ( T ) = 0 forcesall predictions to end in x e = 0.in the same initial value x , there will be a time τ such that they satisfy ˜ x ∗ ( τ ) ≈ x e and x ∗ ( τ ) ≈ x e , where the error hidden in “ ≈ ” tends to 0 as the horizon T increases. Togetherwith the continuity assumption on the optimal value function this implies that the cost ofthe two trajectories on the time interval [0 , τ ] is almost identical. This, finally, can be usedto conclude that (cid:101) V T is an approximate Lyapunov function, from which practical asymptoticstability, i.e., asymptotic stability of a neighborhood of x e , which shrinks down to { x e } when T increases, can finally be concluded. The details of this reasoning were originallyderived in the papers [17, 29, 11]. A concise presentation can be found in [23, Section 8.6]or in [13, Section 4].Figure 4.2 show the MPC closed-loop trajectories and the corresponding predictions forexample (3.2) without any terminal conditions. In comparison to Figure 4.1 the merelypractical asymptotic stability of the equilibrium x e = 0 is clearly visible, since the redclosed-loop solution does not converge to the equilibrium x e = 0 but only to a neighborhoodof this equilibrium, which becomes smaller as T increases.We note that under stronger assumptions than strict dissipativity stronger stability state-ments can be made for MPC without terminal conditions. For instance, under a nonlineardetectability condition and suitable bounds on the optimal value function, it was shownin [15] that true (as opposed to merely practical) asymptotic stability of x e for the MPCclosed loop can be expected for sufficiently large T . For positive definite cost, differentways of estimating the length of the horizon T needed for obtaining asymptotic stabilitywere proposed in [44, 16, 24, 41] (see also [23, Chapter 6]). Since MPC relies on the solution of optimal control problems, it seems reasonable toexpect that the MPC closed loop also enjoys certain optimality properties. To this end,we denote the MPC closed-loop trajectory by x MP C ( t ) and the corresponding control by u MP C ( t ). Then we can define different measures for the closed loop cost (we only give the ISSIPATIVITY AND OPTIMAL CONTROL t x M P C ( t ) t x M P C ( t ) Figure 4.2: MPC closed-loop trajectories (red solid) and predictions (black dashed) forexample (3.2) without terminal conditions with horizon T = 5 (left) and T = 10 (right).Without any terminal constraints all predictions end in x = 2, cf. also Figure 3.2(right).discrete-time formulations here as so far the corresponding results have only be derived fordiscrete-time systems): The infinite horizon performance J MP C ∞ ( x ) := ∞ (cid:88) t =0 (cid:96) ( x MP C ( t ) , u MP C ( t ))would be the “natural” measure if we consider MPC as an approximation to an infinitehorizon problem. However, as the infinite sum may not converge, we also look at othermeasures. We also consider the finite horizon closed-loop performance J MP CS ( x ) := S − (cid:88) t =0 (cid:96) ( x MP C ( t ) , u MP C ( t )) (4.3)and the averaged infinite horizon performance J MP C ∞ ( x ) := lim sup S →∞ S J
MP CS ( x ) . These last two performance measures complement each other, as the first measures theperformance on finite intervals [0 , S ] while the second measures the performance in thelimit for S → ∞ .The derivation of estimates for these quantities heavily relies on strict dissipativity andthe turnpike property, which are exploited for MPC both with and without terminal con-ditions, and on the stability properties described in the previous section. The key ideais to use the similarity of the initial pieces of optimal trajectories until they reach theoptimal equilibrium in order to derive approximate versions of the dynamic programmingprinciple. From these, estimates on the above quantities can be derived by induction over t . The following estimates were originally developed in the papers [17, 29, 22]. Concisepresentations can be found in [18] or [23, Chapter 8].For MPC with terminal conditions satisfying (4.2), the identity J MP C ∞ ( x ) = (cid:96) ( x e , u e ) (4.4)4 LARS GR ¨UNE holds for all N ∈ N , and because of (3.4) this is the best possible value the averageperformance functional can attain. In case V ∞ assumes finite values, there exists a function δ : N → [0 , ∞ ) with δ ( T ) → T → ∞ such that the inequality J MP C ∞ ( x ) ≤ V ∞ ( x ) + δ ( T ) (4.5)holds. In case V ∞ does not assume finite values, for each S ∈ N we can obtain the estimate J MP CS ( x ) ≤ inf u ∈ (cid:101) U S J S ( x , u ) + δ ( T ) + δ ( S ) (4.6)where δ is the same type of function as δ . Here (cid:101) U S denotes the set of admisible controlsfor which (cid:107) x u ( S, x ) − x e (cid:107) ≤ (cid:107) x MP C ( S, x ) − x e (cid:107) holds. As x MP C ( S, x ) → x e holds for S → ∞ , for large S the quantity inf u ∈ (cid:101) U S J S ( x , u ) measures the optimal transient cost fortrajectory going from x to a small neighborhood of x e . Thus, the estimates show thatMPC produces trajectories that produce optimal averaged cost and approximately optimaltransient cost.Without terminal conditions the results become somewhat weaker. Particularly, we can ingeneral no longer ensure that J MP C ∞ ( x ) is finite, even if V ∞ ( x ) is finite. However, we canstill establish counterparts of (4.4) and (4.6), namely J MP C ∞ ( x ) ≤ (cid:96) ( x e , u e ) + δ ( T ) (4.7)and J MP CS ( x ) ≤ inf u ∈ (cid:101) U S J S ( x , u ) + Sδ ( T ) + δ ( S ) , (4.8)with δ and δ of the same type as above. We note that the fact that the error term δ ( T )in (4.6) increases to Sδ ( T ) in (4.8) is not an effect of an insufficiently precise analysis butactually a natural consequence of the mere practical asymptotic stability of x e withoutterminal conditions: If the closed-loop solution does not converge to x e , the stage cost willtypically not converge to 0 but to a nonzero residual value, which keeps accumulating overthe time S . In order to illustrate this effect, Figure 4.3 shows the values of J MP CS ( x )for example (3.2) with terminal conditions ( ◦ ) and without terminal conditions ( × ) fordifferent S . In the left figure with horizon T = 5, the effect of the additional factor S in front of δ ( T ) in (4.8) is clearly visible. In the right figure with horizon T = 10, theerror term δ ( T ) in (4.8) is already so small that the effect of the factor S is not visibleanymore. For this horizon, MPC with and without terminal conditions yield almost thesame performance.Just as for stability, under stronger assumptions stronger statements can be made. Forinstance, for positive definite cost (cid:96) finiteness of J MP C ∞ ( x ) and an estimate of the form(4.5) can be obtained also for MPC without terminal conditions, see [24, 41]. Dissipativity and strict dissipativity are important systems theoretic properties with mul-tiple applications. In this paper we have shown that they naturally link to optimal control,
ISSIPATIVITY AND OPTIMAL CONTROL S J M P C S ( x ) S J M P C S ( x ) Figure 4.3: MPC closed-loop cost J MP CS ( x ) for example (3.2) with x = 2 for varying S .The solid line with circles shows the values with terminal conditions X = { } and F ≡ T = 5(left) and T = 10 (right).a fact that is already prominently present in Jan C. Willems’ earliest publications on thesubject. Here we provided a survey on recent results in this direction, which establish aclose link between strict dissipativity and the turnpike property and showed how theseconcepts can be used for analyzing stability and performance of MPC schemes.The present results have already been extended into various directions. In particular,dissipativity has been extended to optimal control problems that do not exhibit an optimalequilibrium but an optimal periodic orbit [49, 35, 50, 32] or general time-varying optimaltrajectories [26, 25]. In this context, the concept of overtaking optimality can be used inorder to define a meaning for optimality also in the case that the infinite horizon optimalvalue function V ∞ is not finite.A major open question is the relation between strict dissipativity and detectability-like no-tions for infinite dimensional systems. Even for linear-quadratic optimal control problemsthis relation is not yet fully understood. References [1]
D. Angeli, R. Amrit, and J. B. Rawlings , On average performance and sta-bility of economic model predictive control , IEEE Trans. Autom. Control, 57 (2012),pp. 1615–1626.[2]
T. Breiten and L. Pfeiffer , On the turnpike property and the receding-horizonmethod for linear-quadratic optimal control problems , SIAM J. Control Optim., 58(2020), pp. 1077–1102.[3]
W. A. Brock and L. Mirman , Optimal economic growth and uncertainty: thediscounted case , J. Econ. Theory, 4 (1972), pp. 479–513.6
LARS GR ¨UNE [4]
C. I. Byrnes and W. Lin , Losslessness, feedback equivalence, and the global stabi-lization of discrete-time nonlinear systems , IEEE Trans. Automat. Control, 39 (1994),pp. 83–98.[5]
C. Cai and A. R. Teel , Input-output-to-state stability for discrete-time systems ,Automatica, 44 (2008), pp. 326–336.[6]
D. A. Carlson, A. B. Haurie, and A. Leizarowitz , Infinite horizon optimalcontrol — Deterministic and Stochastic Systems , Springer-Verlag, Berlin, second ed.,1991.[7]
H. Chen and F. Allg¨ower , A quasi-infinite horizon nonlinear model predictivecontrol scheme with guaranteed stability , Automatica, 34 (1998), pp. 1205–1217.[8]
T. Damm, L. Gr¨une, M. Stieler, and K. Worthmann , An exponential turn-pike theorem for dissipative discrete time optimal control problems , SIAM J. ControlOptim., 52 (2014), pp. 1935–1957.[9]
M. Diehl, R. Amrit, and J. B. Rawlings , A Lyapunov function for economicoptimizing model predictive control , IEEE Trans. Autom. Control, 56 (2011), pp. 703–707.[10]
R. Dorfman, P. A. Samuelson, and R. M. Solow , Linear programming andeconomic analysis , A Rand Corporation Research Study, McGraw-Hill, New York-Toronto-London, 1958.[11]
T. Faulwasser and D. Bonvin , On the design of economic NMPC based on approx-imate turnpike properties , in Proceedings of the 54th IEEE Conference on Decisionand Control — CDC 2015, 2015, pp. 4964–4970.[12]
T. Faulwasser and L. Gr¨une , Turnpike properties in optimal control: an overviewof discrete-time and continuous-time results . To appear in the Handbook of NumericalAnalysis, 2021. Preprint available from https://arxiv.org/pdf/2011.13670.pdf.[13]
T. Faulwasser, L. Gr¨une, and M. A. M¨uller , Economic nonlinear model pre-dictive control , Foundations and Trends ® in Systems and Control, 5 (2018), pp. 1–98.[14] M. G. Forbes, R. S. Patwardhan, H. Hamadah, and R. B. Gopaluni , Modelpredictive control in industry: Challenges and opportunities , in Proceedings of the 9thIFAC Symposium on Advanced Control of Chemical Processes — ADCHEM 2015,vol. 48 of IFAC-PapersOnLine, Whistler, Canada, 2015, pp. 531–538.[15]
G. Grimm, M. J. Messina, S. E. Tuna, and A. R. Teel , Model predictive control:for want of a local control Lyapunov function, all is not lost , IEEE Trans. Automat.Control, 50 (2005), pp. 546–558.[16]
L. Gr¨une , Analysis and design of unconstrained nonlinear MPC schemes for finiteand infinite dimensional systems , SIAM J. Control Optim., 48 (2009), pp. 1206–1228.[17]
L. Gr¨une , Economic receding horizon control without terminal constraints , Automat-ica, 49 (2013), pp. 725–734.
ISSIPATIVITY AND OPTIMAL CONTROL
L. Gr¨une , Approximation properties of receding horizon optimal control , Jahresber.DMV, 118 (2016), pp. 3–37.[19]
L. Gr¨une and R. Guglielmi , Turnpike properties and strict dissipativity for discretetime linear quadratic optimal control problems , SIAM J. Cont. Optim., 56 (2018),pp. 1282–1302.[20]
L. Gr¨une and R. Guglielmi , On the relation between turnpike properties and dissi-pativity for continuous time linear quadratic optimal control problems , Math. ControlRel. Fields, 11 (2021), pp. 169–188.[21]
L. Gr¨une and M. A. M¨uller , On the relation between strict dissipativity and theturnpike property , Syst. Contr. Lett., 90 (2016), pp. 45–53.[22]
L. Gr¨une and A. Panin , On non-averaged performance of economic MPC withterminal conditions , in Proceedings of the 54th IEEE Conference on Decision andControl — CDC 2015, Osaka, Japan, 2015, pp. 4332–4337.[23]
L. Gr¨une and J. Pannek , Nonlinear Model Predictive Control. Theory and Algo-rithms , Springer-Verlag, London, 2nd ed., 2017.[24]
L. Gr¨une, J. Pannek, M. Seehafer, and K. Worthmann , Analysis of uncon-strained nonlinear MPC schemes with time varying control horizon , SIAM J. ControlOptim., 48 (2010), pp. 4938–4962.[25]
L. Gr¨une and S. Pirkelmann , Economic model predictive control for time-varyingsystem: Performance and stability results , Opt. Control Appl. Meth., 41 (2019),pp. 42–64. Special Issue: MPC for Energy Systems: Economic and Distributed Ap-proaches.[26]
L. Gr¨une, S. Pirkelmann, and M. Stieler , Strict dissipativity implies turnpikebehavior for time-varying discrete time optimal control problems , in Control systemsand mathematical methods in economics: Essays in Honor of Vladimir M. Veliov,G. Feichtinger, R. M. Kovacevic, and G. Tragler, eds., vol. 687 of Lecture Notes inEconom. and Math. Systems, Springer, Cham, 2018, pp. 195–218.[27]
L. Gr¨une, M. Schaller, and A. Schiela , Sensitivity analysis of optimal controlfor a class of parabolic PDEs motivated by model predictive control , SIAM J. ControlOptim., 57 (2019), pp. 2753–2774.[28]
L. Gr¨une, M. Schaller, and A. Schiela , Exponential sensitivity and turnpikeanalysis for linear quadratic optimal control of general evolution equations , J. Differ.Equ., 268 (2020), pp. 7311–7341.[29]
L. Gr¨une and M. Stieler , Asymptotic stability and transient optimality of economicMPC without terminal conditions , J. Proc. Control, 24 (2014), pp. 1187–1196.[30]
M. Gugat, E. Tr´elat, and E. Zuazua , Optimal Neumann control for the 1D waveequation: finite horizon, infinite horizon, boundary tracking terms and the turnpikeproperty , Systems Control Lett., 90 (2016), pp. 61–70.8
LARS GR ¨UNE [31]
M. H¨oger and L. Gr¨une , On the relation between detectability and strict dissipativ-ity for nonlinear discrete time systems , IEEE Control Syst. Lett., 3 (2019), pp. 458–462.[32]
J. K¨ohler, M. A. M¨uller, and F. Allg¨ower , On periodic dissipativity notionsin economic model predictive control , IEEE Control Syst. Lett., 2 (2018), pp. 501–506.[33]
D. Q. Mayne, J. B. Rawlings, C. V. Rao, and P. O. M. Scokaert , Constrainedmodel predictive control: stability and optimality , Automatica, 36 (2000), pp. 789–814.[34]
M. A. M¨uller, D. Angeli, and F. Allg¨ower , On necessity and robustness ofdissipativity in economic model predictive control , IEEE Trans. Autom. Control, 60(2015), pp. 1671–1676.[35]
M. A. M¨uller and L. Gr¨une , Economic model predictive control without terminalconstraints for optimal periodic behavior , Automatica, 70 (2016), pp. 128–139.[36]
A. Porretta , On the turnpike property for mean field games , Minimax Theory Appl.,3 (2018), pp. 285–312.[37]
A. Porretta and E. Zuazua , Long time versus steady state optimal control , SIAMJ. Control Optim., 51 (2013), pp. 4242–4273.[38]
S. Qin and T. Badgwell , A survey of industrial model predictive control technology ,Control Engineering Practice, 11 (2003), pp. 733–764.[39]
F. P. Ramsey , A mathematical theory of saving , The Economic Journal, 38 (1928),pp. 543–559.[40]
J. B. Rawlings, D. Q. Mayne, and M. M. Diehl , Model Predictive Control:Theory, Computation and Design , Nob Hill Publishing, Madison, Wisconsin, 2017.[41]
M. Reble and F. Allg¨ower , Unconstrained model predictive control and sub-optimality estimates for nonlinear continuous-time systems , Automatica, 48 (2011),pp. 1812–1817.[42]
E. Tr´elat, C. Zhang, and E. Zuazua , Steady-state and periodic exponential turn-pike property for optimal control problems in Hilbert spaces , SIAM J. Control Optim.,56 (2018), pp. 1222–1252.[43]
E. Tr´elat and E. Zuazua , The turnpike property in finite-dimensional nonlinearoptimal control , J. Differ. Equ., 258 (2015), pp. 81–114.[44]
S. E. Tuna, M. J. Messina, and A. R. Teel , Shorter horizons for model predic-tive control , in Proceedings of the 2006 American Control Conference, Minneapolis,Minnesota, USA, 2006, pp. 863–868.[45]
J. von Neumann , ¨Uber ein ¨okonomisches Gleichungssystem und eine Verallge-meinerung des Brouwerschen Fixpunktsatzes , in Ergebnisse eines MathematischenSeminars, K. Menger, ed., 1938. ISSIPATIVITY AND OPTIMAL CONTROL
J. C. Willems , Least squares stationary optimal control and the algebraic Riccatiequation , IEEE Trans. Autom. Control, 16 (1971), pp. 621–634.[47]
J. C. Willems , Dissipative dynamical systems. I. General theory , Arch. RationalMech. Anal., 45 (1972), pp. 321–351.[48]
J. C. Willems , Dissipative dynamical systems. II. Linear systems with quadraticsupply rates , Arch. Rational Mech. Anal., 45 (1972), pp. 352–393.[49]
M. Zanon, S. Gros, and M. Diehl , A Lyapunov function for periodic economicoptimizing model predictive control , in Proceedings of the 52nd IEEE Conference onDecision and Control — CDC2013, Florence, Italy, 2013, pp. 5107–5112.[50]
M. Zanon, L. Gr¨une, and M. Diehl , Periodic optimal control, dissipativity andMPC , IEEE Trans. Auto. Cont., 62 (2017), pp. 2943–2949.[51]
E. Zuazua , Large time control and turnpike properties for wave equations , Ann. Rev.Control, 44 (2017), pp. 199–210.0
LARS GR ¨UNE
Optimal control can be used in order to compute storage functions for dissipative systems,provided the supply rate s and—in case of strict dissipativity—the K ∞ -function α areknown. More precisely, the system is strictly dissipative if and only if the optimal valuefunction V ( x ) := sup T ≥ ,u ∈U (cid:90) T − s ( x ( τ ) , u ( τ )) + α ( (cid:107) x ( τ ) − x e (cid:107) ) dτ is finite for all initial values x . In this case, λ = V is a storage function called the availablestorage . An analogous construction works without α in case of non-strict dissipativity andwith a sum instead of the integral in case of discrete-time systems.That V is indeed a storage function follows for any t > u ∈ U from the inequalities V ( x ) = sup T ≥ ,u ∈U (cid:90) T − s ( x ( τ ) , u ( τ )) + α ( (cid:107) x ( τ ) − x e (cid:107) ) dτ ≥ sup T ≥ t,u ∈U (cid:90) T − s ( x ( τ ) , u ( τ )) + α ( (cid:107) x ( τ ) − x e (cid:107) ) dτ ≥ (cid:90) t − s ( x ( τ ) , ˆ u ( τ )) + α ( (cid:107) x ( τ ) − x e (cid:107) ) dτ + sup T ≥ ,u ∈U (cid:90) T − tt − s ( x ( τ ) , u ( τ )) + α ( (cid:107) x ( τ ) − x e (cid:107) ) dτ = (cid:90) t − s ( x ( τ ) , ˆ u ( τ )) + α ( (cid:107) x ( τ ) − x e (cid:107) ) dτ + V ( x ( t )) , which implies (2.9) for λ = V .If each x ∈ R n can be reached from the equilibrium x e , then another optimal controlcharacterization of a storage function is given by the required supply . In this case onedefines V ( x ) := inf T ≥ ,u ∈U : x (0)= xe,x ( T )= x (cid:90) T s ( x ( τ ) , u ( τ )) − α ( (cid:107) x ( τ ) − x e (cid:107) ) dτ. Then strict dissipativity holds if and only if V is bounded from below and then, again, λ = V is a storage function. Here, the storage function property follows from the factthat steering from x e to x ( t ) via x cannot be cheaper than steering from x e to x ( t ) in theoptimal way. For details on both constructions we refer to [47]. ISSIPATIVITY AND OPTIMAL CONTROL Model Predictive Control (MPC) is one of the most successful optimization based controltechniques with ample applications in industry [38, 14]. We describe it here in discrete time,noting that the adaptation to continuous time is relatively straightforward. For furtherreading we recommend, e.g., the monographs [40, 23].In many regulation problems, one would like to solve the infinite horizon optimal controlproblem minimize J ∞ ( x , u ) = ∞ (cid:88) k =0 (cid:96) ( x ( t ) , u ( t )) (7.1)with respect to u ∈ U , u ( t ) ∈ U and x ( t ) ∈ X for all t = 0 , , , . . . , where U is an appropriatespace of sequences, X and U are as above, and x ( t ) satisfies x (0) = 0 and (2.4). Unless theproblem has a very particular structure (as, e.g., linear dynamics, quadratic costs and noconstraints), a closed-form solution of (7.1) is usually not available and due to the infinitetime horizon a numerical solution is very costly to obtain, particularly if one wants toobtain the optimal control in feedback form, i.e., in the form u ∗ ( t ) = F ( x ( t )) for a suitablemap F .The key idea of MPC now consists in truncating the optimization horizon and insteadof (7.1) solve (2.3) or a variant thereof. This variant may include additional terminalconstraints of the form x ( T ) ∈ X and/or a terminal cost of the form F ( x ( T )) as anadditional summand in J T . The MPC loop then proceeds as follows:1) Pick an initial value x MP C (0) and a time horizon T . Set k := 0.2) Set x := x MP C ( k ) and solve (2.3) with this initial value. Denote the resultingoptimal control sequence by u ∗ ( · ) and set F MP C ( x MP C ( k )) := u ∗ (0).3) Apply F MP C ( x MP C ( k )), measure x MP C ( k + 1), set k := k + 1 and go to 2).The resulting solution x MP C ( t ) is called the MPC closed-loop solution while the individualopen-loop finite horizon optimal solutions computed by solving (2.3) in Step 2 are calledthe predictions .Figure 7.1 shows a schematic sketch of the resulting solutions loop. Here the red solid linedepicts the MPC closed-loop solution while the black dashed lines indicate the predictions.The figure shows the ideal case in which the closed-loop dynamics exactly coincides withthe dynamics used to compute the predictions.We note that by means of the dynamic programming principle (see, e.g., [23, Theorem4.6]) the solution strategy “solve the optimal control problem and use the first element ofthe resulting optimal control sequence as feedback control value” would yield an optimalfeedback law if we solved the infinite horizon problem (7.1) in Step 2. However, by resortingto the (numerically much more easy to obtain) solution of the finite horizon problem (2.3)in Step 2, we make an error that needs to be analyzed. MPC for stage costs considered inthis paper, which do not merely penalize the distance of the state from a desired steadystate, is often denoted as economic MPC , although the term general MPC would probablybe more fitting. In any case, strict dissipativity of the optimal control problem plays animportant role in the analysis of such MPC schemes.2
LARS GR ¨UNE ......... x MP C ( t ) t0 1 2 3 4 5 6