Value Function in Maximum Hands-off Control
aa r X i v : . [ c s . S Y ] D ec Value Function in Maximum Hands-off Control ⋆ Takuya Ikeda a , Masaaki Nagahara a a Graduate School of Informatics, Kyoto University, Kyoto 606-8501, Japan
Abstract
In this brief paper, we study the value function in maximum hands-off control. Maximum hands-off control, also known assparse control, is the L -optimal control among the admissible controls. Although the L measure is discontinuous and non-convex, we prove that the value function, or the minimum L norm of the control, is a continuous and strictly convex functionof the initial state in the reachable set, under an assumption on the controlled plant model. This property is important, inparticular, for discussing the sensitivity of the optimality against uncertainties in the initial state, and also for investigatingthe stability by using the value function as a Lyapunov function in model predictive control. Key words:
Optimal control, continuity, bang-bang control, discontinuous control, linear systems, minimum-time control.
Optimal control is widely used in recent industrial prod-ucts not just for achieving the best performance butfor reducing the control effort. For example, the classi-cal LQR (Linear Quadratic Regulator) control gives away to consider the tradeoff between performance andcontrol-effort reduction by using weighting functions onthe states and the control inputs with the L norm (i.e.the energy) [1].Recently, a novel control method, called maximumhands-off control , that maximizes the time duration inwhich the control is exactly zero among the admissi-ble controls [10,12]. An example of hands-off control isa stop-start system in automobiles, in which an auto-mobile automatically shuts down the engine (i.e. zerocontrol) to avoid it idling for long periods of time, andalso to reduce CO or CO2 emissions as well as fuelconsumption. Therefore, the hands-off control is alsocalled as green control [11]. Also, the hands-off controlis effective in hybrid/electric vehicles, railway vehicles,networked/embedded systems, to name a few [12].Maximum hands-off control is related to sparsity , whichis widely studied in compressed sensing [3]. Sparsity isalso applied to control problems such as networked con- ⋆ This paper was not presented at any IFAC meeting. Cor-responding author M. Nagahara.
Email addresses: [email protected] (Takuya Ikeda), [email protected] (Masaaki Nagahara). trol [13,8], security of control systems [4], state estima-tion [15], to name a few.A mathematical difficulty in the maximum hands-offcontrol is that the cost function, which is defined by the L measure (the support length of a function), is highlynonlinear; it is discontinuous and non-convex. To solvethis problem, a recent work [10,12] has proposed to re-duce the problem to an L optimal control problem, andshown the equivalence between the maximum hands-off(or L optimal) control and the L optimal control un-der the assumption of normality.Motivated by this work, we investigate the value func-tion in the maximum hands-off control. The value func-tion is defined as the optimal value of the cost functionof the optimal control problem. It is important to showthe continuity of the value function with respect to theinitial state; if the value function is continuous, then theoptimality property is less sensitive against uncertain-ties in the initial state. Also, the value function may beused as a Lyapunov function when the optimal control isadapted to model predictive control, and the continuityis necessary for the function to be a Lyapunov function[9]. Although the L measure in the maximum hands-offcontrol is discontinuous and non-convex, we prove thatthe value function is a continuous and strictly convexfunction of the initial state in the reachable set, underan assumption on the controlled plant model.The present paper expands on our recent conference con-tribution [7] by rearranging the contents and incorpo-rating analysis of convexity of the value function. Preprint submitted to Automatica 27 August 2018 he remainder of this paper is organized as follows: InSection 2, we give mathematical preliminaries for oursubsequent discussion. In Section 3, we review the prob-lem of maximum hands-off control. Section 4 investi-gates the continuity of the value function in maximumhands-off control, and Section 5 discusses its convexity.Section 6 presents an example of maximum hands-offcontrol to illustrate the properties of continuity and con-vexity. In Section 7, we offer concluding remarks.
This section reviews basic definitions, facts, and notationthat will be used throughout the paper.Let n be a positive integer. For a vector x ∈ R n anda scalar ε >
0, the ε -neighborhood of x is defined by B ( x, ε ) , { y ∈ R n : k y − x k < ε } , where k · k denotesthe Euclidean norm in R n . Let X be a subset of R n . Apoint x ∈ X is called an interior point of X if there exists ε > B ( x, ε ) ⊂ X . The interior of X is the setof all interior points of X , and we denote the interior of X by int X . A set X is said to be open if X = int X . Forexample, int X is open for every subset X ⊂ R n . A point x ∈ R n is called an adherent point of X if B ( x, ε ) ∩X 6 = ∅ for every ε >
0, and the closure of X is the set of alladherent points of X . A set X ⊂ R n is said to be closed if X = X , where X is the closure of X . The boundary of X is the set of all points in the closure of X , not belongingto the interior of X , and we denote the boundary of X by ∂ X , i.e., ∂ X = X − int X , where X − X is the set of allpoints which belong to the set X but not to the set X .In particular, if X is closed, then ∂ X = X − int X , since X = X . A set X ⊂ R n is said to be convex if, for any x, y ∈ X and any λ ∈ [0 , − λ ) x + λy belongs to X .A real-valued function f defined on R n is said to be upper semi-continuous on R n if for every α ∈ R theset { x ∈ R n : f ( x ) < α } is open, and f is said to be lower semi-continuous on R n if for every α ∈ R the set { x ∈ R n : f ( x ) > α } is open. It is known that a function f is continuous on R n if and only if it is upper and lowersemi-continuous on R n ; see e.g., [14, pp. 37].A real-valued function f defined on a convex set C ⊂ R n is said to be convex if f (cid:0) (1 − λ ) x + λy (cid:1) ≤ (1 − λ ) f ( x ) + λf ( y ) , (1)for all x , y ∈ C and all λ ∈ (0 , f is said to be strictly convex if the inequality (1) holds strictly when-ever x and y are distinct points and λ ∈ (0 , T >
0. For a continuous-time signal u ( t ) over a timeinterval [0 , T ], we define its L and L ∞ norms respec- tively by k u k , Z T | u ( t ) | dt, k u k ∞ , sup t ∈ [0 ,T ] | u ( t ) | . We define the support set of u , denoted by supp( u ), bythe closure of the set { t ∈ [0 , T ] : u ( t ) = 0 } . The L norm of a measurable function u as the length of its support,that is, k u k , m (cid:0) supp( u ) (cid:1) , where m is the Lebesguemeasure on R . In this paper, we consider a linear time-invariant systemrepresented by˙ x ( t ) = Ax ( t ) + Bu ( t ) , t ≥ , (2)where x ( t ) ∈ R n , u ( t ) ∈ R , A ∈ R n × n , and B ∈ R n × .Throughout this paper, we assume the following: Assumption 1
The pair ( A, B ) is controllable and thematrix A is nonsingular. Let
T > u = { u ( t ) : t ∈ [0 , T ] } ∈ L admissible if it steers x ( t ) from a given initial state x (0) = ξ ∈ R n to the origin at time T (i.e., x ( T ) = 0), and satisfies themagnitude constraint k u k ∞ ≤
1. We denote by U ( ξ ) theset of all admissible controls for an initial state ξ ∈ R n ,that is, U ( ξ ) , (cid:26) u ∈ L : Z T e − As Bu ( s ) ds = − ξ, k u k ∞ ≤ (cid:27) . (3)The maximum hands-off control is the minimum L -norm (or the sparsest) control among the admissible con-trol inputs. This control problem is formulated as fol-lows. Problem 2 (Maximum hands-off control)
For agiven initial state ξ ∈ R n , find an admissible control u ∈ U ( ξ ) that minimizes J ( u ) = k u k . The value function for this optimal control problem isdefined as V ( ξ ) , min u ∈U ( ξ ) J ( u ) = min u ∈U ( ξ ) k u k . (4)Note that the cost function J ( u ) can be rewritten as J ( u ) = Z T φ ( u ) dt, φ ( u ) u | u | Fig. 1. The L kernel φ ( u ) and its convex approximation | u | for the L norm. where φ is the L kernel function defined by φ ( u ) , (cid:26) , if u = 0 , , if u = 0 . Fig. 1 shows the graph of φ ( u ). As shown in this figure,the kernel function φ ( u ) is discontinuous at u = 0 andnon-convex. However, in the following sections, we willshow that the value function V ( ξ ) in (4) is continuousand strictly convex. In this section, we investigate the continuity of the valuefunction V ( ξ ) in (4).First, we define the reachable set for the control problem(Problem 2) by R , (cid:26) Z T e − As Bu ( s ) ds : k u k ∞ ≤ (cid:27) ⊂ R n . The following is a fundamental lemma of the paper:
Lemma 3
Suppose Assumption 1 is satisfied. Let usconsider L optimal control with J ( u ) := k u k = Z T | u ( t ) | dt,V ( ξ ) := min u ∈U ( ξ ) k u k . (5) Then, for every ξ ∈ R , we have V ( ξ ) = V ( ξ ) . PROOF.
By Assumption 1, the L -optimal controlproblem associate with (5) is normal [2, Theorem 6-13].Also, for ξ ∈ R , an L -optimal control u ∗ ∈ U ( ξ ) min-imizing J exists (see Lemma 10 in Appendix A), and u ∗ ( t ) ∈ {− , , } for almost all t ∈ [0 , T ] (this is called the “bang-off-bang” property ) [2, Section 6-14]. Thenby [10, Theorem 5], u ∗ is also the optimal control ofProblem 2, and we have V ( ξ ) = min u ∈U ( ξ ) k u k = k u ∗ k = k u ∗ k = V ( ξ ) , where we used the “bang-off-bang” property of u ∗ forthe third equality. ✷ Note that the absolute value | u | in (5) is a convex ap-proximation of φ ( u ) as shown in Fig. 1. Associated with V ( ξ ), we define the following subset of R with α ≥ R α , (cid:26) Z T e − As Bu ( s ) ds : k u k ∞ ≤ , k u k ≤ α (cid:27) . (6)For the set R α , we have another fundamental lemma. Lemma 4
Suppose Assumption 1 is satisfied. Then, forevery α ∈ [0 , T ] , R α = { ξ ∈ R : V ( ξ ) ≤ α } , (7) ∂ R α = { ξ ∈ R : V ( ξ ) = α } , (8)int R α = { ξ ∈ R : V ( ξ ) < α } . (9) PROOF.
See Appendix A. ✷ From these lemmas, we show the continuity of the valuefunction V ( ξ ). Theorem 5
If Assumption 1 is satisfied, then V ( ξ ) iscontinuous on R . PROOF.
Define V ( ξ ) , (cid:26) V ( ξ ) , if ξ ∈ R ,T, if ξ ∈ R n − R . It is enough to show that V ( ξ ) is continuous on R n .First, we show that the set { ξ ∈ R n : V ( ξ ) < α } (10)is open for every α ∈ R . If α ≤
0, then the set (10) isempty since for any ξ ∈ R n , V ( ξ ) ≥
0. If α > T , thenthe set (10) is R n , since for any ξ ∈ R , V ( ξ ) ≤ T . If0 < α ≤ T , then the set (10) is a subset of R , andcoincides with int R α by Lemma 4. Therefore, the set(10) is open for every α ∈ R . It follows that V ( ξ ) is uppersemi-continuous on R n .3ext, we show that the set { ξ ∈ R n : V ( ξ ) > α } (11)is open for every α ∈ R . If α < α ≥ T , then theset (11) is R n or empty, respectively. If 0 ≤ α < T , fromLemma 4, we have { ξ ∈ R n : V ( ξ ) > α } = R n − { ξ ∈ R : V ( ξ ) ≤ α } = R n − R α . Since R α is closed (see Lemma 8 in Appendix A), theset (11) is open for every α ∈ R . It follows that V ( ξ ) islower semi-continuous on R n .Since V ( ξ ) is upper and lower semi-continuous on R n , itis continuous on R n , and the conclusion follows. ✷ Theorem 5 leads to an important result of L optimalcontrol as follows. Corollary 6
If Assumption 1 is satisfied, then V ( ξ ) iscontinuous on R . PROOF.
This is a direct consequence of Lemma 3 andTheorem 5. ✷ Here we show the convexity of the value function V ( ξ ).Although the kernel function φ ( u ) in the cost functionis not convex as shown in Fig. 1, the value function V ( ξ )is a convex function on R . Theorem 7
If Assumption 1 is satisfied, then V ( ξ ) isstrictly convex on R . PROOF.
From Lemma 3, it is enough to prove thatthe L value function V ( ξ ) is strictly convex on R First, we prove that V ( ξ ) is convex on R . Take any ξ , η ∈ R , and λ ∈ (0 , L -optimalcontrols u ξ and u η for initial states ξ and η , respectively(see Lemma 10 in Appendix A). Obviously, the followingcontrol u , (1 − λ ) u ξ + λu η (12)steers the state from the initial state (1 − λ ) ξ + λη tothe origin at time T , and it satisfies k u k ∞ ≤
1. That is,we have u ∈ U (cid:0) (1 − λ ) ξ + λη (cid:1) . Therefore V (cid:0) (1 − λ ) ξ + λη (cid:1) ≤ k u k ≤ (1 − λ ) k u ξ k + λ k u η k = (1 − λ ) V ( ξ ) + λV ( η ) , (13) and hence V ( ξ ) is convex on R .Next, we will show the strict convexity of V ( ξ ). To provethis, we will show that a contradiction is implied byassuming that there exist ξ , η ∈ R with ξ = η and λ ∈ (0 ,
1) such that V (cid:0) (1 − λ ) ξ + λη (cid:1) = (1 − λ ) V ( ξ ) + λV ( η ) . (14)Let u ξ and u η be L -optimal controls for initial states ξ and η , respectively. Let u , (1 − λ ) u ξ + λu η as in (12).From (13) and (14), it follows that V (cid:0) (1 − λ ) ξ + λη (cid:1) = k u k = (1 − λ ) k u ξ k + λ k u η k , so the control u = (1 − λ ) u ξ + λu η is an L -optimalcontrol for the initial state (1 − λ ) ξ + λη .Now, by Assumption 1, u ξ ( t ) and u η ( t ) take the val-ues 1, 0, and − t ∈ [0 , T ]. So, the pair( u ξ ( t ) , u η ( t )) takes the following values on [0 , T ] exceptfor sets of measure zero:(1 , , (1 , , (1 , − , (0 , , (0 , , (0 , − , ( − , , ( − , , ( − , − . (15)For the pairs in (15) of ( u ξ ( t ) , u η ( t )), the control u =(1 − λ ) u ξ + λu η respectively takes the following values:1 , − λ, − λ, λ, , − λ, − λ, − λ, − . On the other hand, the control u is also L optimal andtakes the values 1, 0, and − t ∈ [0 , T ].Since λ ∈ (0 , m ( I , ∪ I , ∪ I , − ∪ I − , ) = 0 , (16)where I i,j , { t ∈ [0 , T ] : ( u ξ ( t ) , u η ( t )) = ( i, j ) } , for i, j ∈ {− , , } . If λ = 1 /
2, then we also have m ( I , − ∪ I − , ) = 0 , and it follows that m ( I , ∪ I , ∪ I − , − ) = T, that is, u ξ ( t ) = u η ( t ) for almost all t ∈ [0 , T ]. This im-plies ξ = η , but this contradicts the assumption, so wehave λ = 1 /
2. Then the pair ( u ξ ( t ) , u η ( t )) on [0 , T ] ex-cept for sets of measure zero takes values (1 , , − , − , − , − ξ = η , we have T , m ( I , − ∪ I − , ) > . (17)4et T , m ( I , ) and T , m ( I − , − ). From (16) andthe fact that u ξ + u η = 0 on I , − ∪ I − , ∪ I , , we have V (cid:18) ξ + 12 η (cid:19) = (cid:13)(cid:13)(cid:13)(cid:13) u ξ + 12 u η (cid:13)(cid:13)(cid:13)(cid:13) = 12 Z I , ∪I − , − | u ξ ( t ) + u η ( t ) | dt = T + T , (18)On the other hand,12 V ( ξ ) + 12 V ( η ) = 12 k u ξ k + 12 k u η k = T + T + T . (19)Equations (14), (18) and (19) imply that T = 0, whichcontradicts (17). ✷ In this section, we consider a simple example with a 1-dimensional linear control system˙ x ( t ) = ax ( t ) + bu ( t ) , where a < b = 0. This system obviously satisfiesAssumption 1, and let us verify the continuity and con-vexity of the value function V ( ξ ) on the reachable set R .The reachable set R and the maximum hands-off con-trol u ξ for an initial state ξ ∈ R are computed via thebang-bang principle [6, Theorem 12.1] and the minimumprinciple for L -optimal control [2, Section 6.14] as R = [ − x , x ] , x = −| b | a − (cid:0) e − aT − (cid:1) , and u ξ ( t ) = (cid:26) , t ∈ [0 , τ ξ ) , − sgn( b )sgn( ξ ) , t ∈ [ τ ξ , T ] , where sgn( x ) = x/ | x | for x = 0 and sgn(0) = 0, and τ ξ , − a − log (cid:0) e − aT + a | b − ξ | (cid:1) . Note that if ξ = 0, then u ( t ) = 0 for all t ∈ [0 , T ]. Thenwe have V ( ξ ) = T − τ ξ = T + a − log( e − aT + a | b − ξ | ) . For example, let a = − b = 1, and T = 5. Fig. 2 showsthe value function V ( ξ ) on R , where R = [ − e +1 , e − V ( ξ ) is continuous and strictlyconvex on R . -200 -150 -100 -50 0 50 100 150 2000123456 initiate pointvalue function Fig. 2. Value function V ( ξ ) for ξ ∈ R = [ − e + 1 , e − In this brief paper, we have proved the continuity and thestrict convexity of the value function of the maximumhands-off control problem under an assumption of thecontrolled system. Also, as a corollary we have shownthat those properties are also satisfied for L optimalcontrol under the same assumption. These properties ofthe vale function plays an important role to investigatethe stability when we extend the control to the modelpredictive control. Acknowledgements
This research is supported in part by JSPS Grant-in-Aid for Scientific Research (C) No. 24560543, Grant-in-Aid for Scientific Research on Innovative AreasNo. 26120521, and an Okawa Foundation ResearchGrant.
References [1] B. D. O. Anderson and J. B. Moore.
Optimal Control: LinearQuadratic Methods . Dover Publications, 2007.[2] M. Athans and P. L. Falb.
Optimal Control . DoverPublications, 1966.[3] Y. C. Eldar and G. Kutyniok.
Compressed Sensing: Theoryand Applications . Cambridge University Press, 2012.[4] H. Fawzi, P. Tabuada, and S. Diggavi. Secure estimation andcontrol for cyber-physical systems under adversarial attacks.
IEEE Trans. Autom. Control , 59(6):1454–1467, June 2014.[5] O. Hajek. L -optimization in linear systems with boundedcontrols. Journal of Optimization Theory and Applications ,29(3):409–436, November 1979.[6] H. Hermes and J. P. Lasalle.
Function Analysis and TimeOptimal Control . Academic Press, 1969.[7] T. Ikeda and M. Nagahara. Continuity of the valuefunction in sparse optimal control. submitted to the10th Asian Control Conference 2015 (ASCC2015) , 2015. http://arxiv.org/abs/1412.5707 .
8] H. Kong, G. C. Goodwin, and M. M. Seron. A cost-effectivesparse communication strategy for networked linear controlsystems: an SVD-based approach.
International Journal ofRobust and Nonlinear Control , 2014. (to appear).[9] M. Morari and J. H. Lee. Model predictive control: past,present and future.
Computers and Chemical Engineering ,23(4-5):667–682, 1999.[10] M. Nagahara, D. E. Quevedo, and D. Neˇsi´c. Maximum-hands-off control and L optimality. In , pages 3825–3830, December 2013.[11] M. Nagahara, D. E. Quevedo, and D. Neˇsi´c. Hands-off controlas green control. In SICE Control Division Multi Symposium2014 , March 2014. http://arxiv.org/abs/1407.2377 .[12] M. Nagahara, D. E. Quevedo, and D. Neˇsi´c. Maximum hands-off control: a paradigm of control effort minimization. 2014. http://arxiv.org/abs/1408.3025 .[13] M. Nagahara, D.E. Quevedo, and J. Østergaard. Sparsepacketized predictive control for networked control overerasure channels.
IEEE Trans. Autom. Control , 59(7):1899–1905, July 2014.[14] W. Rudin.
Real and Complex Analysis . McGraw-Hill, 3rdedition edition, 1987.[15] B. M. Sanandaji, M. B. Wakin, and T. L. Vincent.Observability with random observations.
IEEE Trans.Autom. Control , 59(11):3002–3007, November 2014.[16] T. B. Singh.
Elements of Topology . CRC Press, 2013.
A Proof of Lemma 4
A.1 Lemmas
To prove Lemma 4, we need some lemmas.
Lemma 8
The set R α in (6) satisfies the following:(1) For every α ∈ R , R α is compact.(2) For every α ∈ R , R α ⊂ R , with equality for α ≥ T .(3) R = { } .(4) R α ⊂ R β for ≤ α ≤ β . PROOF.
See [5, Lemma 2.1]. ✷ Lemma 9
For every α ∈ [0 , T ] , we have R α = { ξ ∈ R : ∃ u ∈ U ( ξ ) s . t . k u k ≤ α } . PROOF.
First, fix α ∈ [0 , T ] and take any ξ ∈ R α .Then, by the definition of R α , there exists u ∈ U ( ξ ) suchthat k u k ≤ α and ξ = Z T e − As Bu ( s ) ds. From (3), it follows that the control v := − u is anadmissible control, that is, v ∈ U ( ξ ), and also satis-fies k v k = k u k ≤ α . By definition, R α ⊂ R and hence ξ ∈ R . Therefore, we have ξ ∈ { ξ ∈ R : ∃ u ∈U ( ξ ) s.t. k u k ≤ α } .Conversely, fix α ∈ [0 , T ] and take any ξ ∈ { ξ ∈ R : ∃ u ∈ U ( ξ ) s.t. k u k ≤ α } . That is, ξ ∈ R is an initialstate for the system (2), and there exists an admissiblecontrol u ∈ U ( ξ ) such that k u k ≤ α . Then from (3), wehave ξ = Z T e − As B (cid:0) − u ( s ) (cid:1) ds. The control v = − u satisfies k v k = k u k ≤ α , k v k ∞ = k u k ∞ ≤
1, and hence we have ξ ∈ R α . ✷ Lemma 10
For each initial value ξ ∈ R , there exists anadmissible control u ∈ U ( ξ ) with minimal L -cost k u k .Furthermore, then, ξ ∈ ∂ R α with α = k u k . PROOF.
See [5, Lemma 3.1]. ✷ A.2 Proof of (7)First, fix α ∈ [0 , T ] and take any ξ ∈ R α . Then, fromLemma 8, we have ξ ∈ R , and from Lemma 10, thereexists an L -optimal control u ∗ ∈ U ( ξ ). Also, we have V ( ξ ) = k u ∗ k ≤ α by Lemma 9. Then, from Lemma 3,we have V ( ξ ) ≤ α . That is, we have ξ ∈ { ξ ∈ R : V ( ξ ) ≤ α } .Conversely, fix α ∈ [0 , T ] and take any ξ ∈ { ξ ∈ R : V ( ξ ) ≤ α } . From Lemma 3, we have V ( ξ ) ≤ α . Let β , V ( ξ ). From Lemma 10, we have ξ ∈ ∂ R β , and itfollows from Lemma 8 that ξ ∈ ∂ R β ⊂ R β ⊂ R α . A.3 Proof of (8) and (9)We prove the equation (8); then the equation (9) followsimmediately from (7) and (8), since R α is closed forevery α ≥ α = 0, then ∂ R = { } ,since R = { } . It follows from (7) that { ξ ∈ R : V ( ξ ) = 0 } = R = { } = ∂ R . Fix α ∈ (0 , T ]. We can take ξ ∈ ∂ R α , since ∂ R α is notempty. Since ξ ∈ R α , we have V ( ξ ) ≤ α . If V ( ξ ) < α ,then ξ ∈ ∂ R V ( ξ ) ⊂ R V ( ξ ) ⊂ int R α (see [5, Lemma4.2]), and hence a contradiction occurs. Therefore wehave V ( ξ ) = α , and hence ∂ R α ⊂ { ξ ∈ R : V ( ξ ) = α } and { ξ ∈ R : V ( ξ ) = α } is not empty for every α ∈ (0 , T ]. Then it follows from Lemma 10 that { ξ ∈ R : V ( ξ ) = α } ⊂ ∂ R α for every α ∈ (0 , T ], and the conclusion follows. R n and the empty set are the only subsets whose bound-aries are empty, since R n is connected [16, Chapter 3].is connected [16, Chapter 3].