Beyond Persistent Excitation: Online Experiment Design for Data-Driven Modeling and Control
BBeyond Persistency of Excitation:Online Experiment Design for Data-Driven Modeling and Control
Henk J. van Waarde
Abstract — This paper presents a new experiment designmethod for data-driven modeling and control. The idea isto select inputs online (using past input/output data), leadingto desirable rank properties of data Hankel matrices. Incomparison to the classical persistency of excitation condition,this online approach requires less data samples and is evenshown to be completely sample efficient.
I. I
NTRODUCTION
Recently, there has been an increasing interest in the directdesign of controllers using data [1]–[12]. Several contribu-tions study how controllers can be obtained from a givenbatch of (informative) data, even in the presence of noise[4], [6], [7] and for classes of nonlinear systems [11], [12].The question of how to obtain such informative data sets,however, is largely open. For data-driven control to becomean end-to-end solution, there is a need for new experimentdesign methods to empower the data-based design. This istrue especially for settings including noise and nonlineardynamics. However, even for linear systems with exact data,current experiment design methods are not sample efficient.In this paper we will explore a new idea for designing ex-periments for data-driven modeling and control. Experimentdesign is a classical problem that has been mostly studied inthe parametric identification literature. An established ideais to optimize a measure of the expected accuracy of theparameter estimates subject to input power constraints [13],[14]. This problem is usually tackled in the frequency domainand convex formulations have been provided in [15]. Thedual problem of finding the “least costly” input achieving afixed level of parameter accuracy has also been studied [16],in a closed-loop setting.Less results are known in the context of non-parametricmethods. In this area, a state-of-the-art result for linear time-invariant systems is
Willems et al.’s fundamental lemma [17].The idea is to select an input that is persistently exciting,which implies that a Hankel matrix of measured inputsand outputs satisfies a rank condition. This rank propertyis important, since it guarantees that all trajectories of thesystem can be parameterized in terms of the measuredtrajectory. Essentially, the Hankel matrix of measured inputsand outputs serves as a non-parametric model of the system.This idea is simple yet powerful, and has been successfullyemployed in a number of recent publications on data-drivensimulation and control, cf. [1]–[3].
The author is with the Control Group, Department of Engineering,University of Cambridge, Trumpington Street, Cambridge CB2 1PZ, UK.Email: [email protected] . Despite its elegance and clear impact on data-drivenmethods, the fundamental lemma has some limitations.1) It is not sample efficient in the sense that it uses moredata than strictly necessary. This is because the fun-damental lemma works with inputs that are persistentlyexciting of a certain order, which imposes a conservativelower bound on the number of samples. Large data setsare challenging from a computational point of view,especially for the real-time implementation of controllaws, see e.g. [18].2) It is not applicable to noisy data . A possible approachto deal with this is to perform approximate data-drivensimulation and control in a maximum likelihood frame-work [19], for which an experiment design method wasdeveloped in [20]. Another line of work establishesdata-driven control design methods with guaranteedstability and performance in the presence of boundedprocess noise [4], [6], [7]. However, for this, experimentdesign methods are missing and it is unclear whetherpersistency of excitation is helpful in this context.The purpose of this paper is to introduce another angleof attack to experiment design. In contrast to Willems’fundamental lemma, we do not use persistently excitinginputs, but instead design the inputs online . This means thatat each time step, the input is selected on the basis of inputsand outputs that have been collected at previous time steps.Such an online approach is natural because the first samplesof an experiment already contain valuable partial informationabout the system, which can be exploited in the design of theremaining samples. Although online approaches have beenstudied in various contexts [21], [22], such a method has notbeen explored in the setting of Willems’ lemma.As our main contributions, we provide online experimentdesign methods for both input/state and input/output systems.In both settings, we formally prove that the methods aresample efficient. This completely resolves the problem in 1),and thus shows an advantage of online experiment designover the classical persistency of excitation condition. Themain idea underlying our approach is to reduce the dimensionof the left kernel of a data Hankel matrix at each time step .Analogous to the fundamental lemma, our technical resultsare presented for noise-free data. Nonetheless, we believethat online experiment design will be crucial also in tacklingproblems involving noisy data as mentioned in 2). We willdiscuss this issue in more detail in Section IV.
Outline : In Section II we provide background material. InSection III we present our online experiment design method.Finally, Section IV contains our conclusions and discussion. a r X i v : . [ m a t h . O C ] F e b . Notation and terminology The left kernel of a real matrix M is denoted by leftker M .Consider a signal f : Z → R • and let i, j ∈ Z be integerssuch that i ≤ j . We denote by f [ i,j ] the restriction of f tothe interval [ i, j ] , that is, f [ i,j ] := (cid:2) f ( i ) (cid:62) f ( i + 1) (cid:62) · · · f ( j ) (cid:62) (cid:3) (cid:62) . With slight abuse of notation, we will also use the notation f [ i,j ] to refer to the sequence f ( i ) , f ( i + 1) , . . . , f ( j ) . Let k be a positive integer such that k ≤ j − i + 1 and define the Hankel matrix of depth k , associated with f [ i,j ] , as H k ( f [ i,j ] ) := f ( i ) f ( i + 1) · · · f ( j − k + 1) f ( i + 1) f ( i + 2) · · · f ( j − k + 2) ... ... ... f ( i + k − f ( i + k ) · · · f ( j ) . Note that the subscript k refers to the number of block rowsof H k . The sequence f [ i,j ] is called persistently exciting oforder k if H k ( f [ i,j ] ) has full row rank.II. R ANK CONDITIONS ON DATA MATRICES
Consider the linear time-invariant (LTI) system x ( t + 1) = A x ( t ) + B u ( t ) (1a) y ( t ) = C x ( t ) + D u ( t ) , (1b)where x ∈ R n denotes the state, u ∈ R m is the input and y ∈ R p is the output. Throughout the paper, we will assumethat (1) is minimal , i.e., the pair ( A, B ) is controllable and ( C, A ) is observable. The lag (cid:96) of (1) can be defined as thesmallest integer i for which the observability matrix O i := (cid:2) C (cid:62) ( CA ) (cid:62) ( CA ) (cid:62) · · · ( CA i − ) (cid:62) (cid:3) (cid:62) (2)has rank n . Now, let ( u [0 ,T − , y [0 ,T − ) be an input/outputtrajectory of (1) of length T ≥ (cid:96) . In addition, let L ≥ (cid:96) be an integer. The main goal of this article is to provide anew method to design the input sequence u [0 ,T − so thatthe resulting input/output Hankel matrix (cid:20) H L ( y [0 ,T − ) H L ( u [0 ,T − ) (cid:21) = y (0) y (1) · · · y ( T − L ) ... ... ... y ( L − y ( L ) · · · y ( T − u (0) u (1) · · · u ( T − L ) ... ... ... u ( L − u ( L ) · · · u ( T − (3)has rank n + mL . This rank condition plays a fundamentalrole in modeling and control using data. Indeed, it impliesthat any length- L input/output trajectory of (1) can beobtained from (3). More specifically, if (3) has rank n + mL then (¯ u [0 ,L − , ¯ y [0 ,L − ) is an input/output trajectory of (1)if and only if (cid:20) ¯ y [0 ,L − ¯ u [0 ,L − (cid:21) ∈ im (cid:20) H L ( y [0 ,T − ) H L ( u [0 ,T − ) (cid:21) . This parameterization of input/output trajectories has beenused extensively to simulate and control dynamical systems, see e.g., [1]–[3]. If
L > (cid:96) , the rank condition on (3) evenimplies that all input/output trajectories of (1) (not justthose of length L ) can be obtained from ( u [0 ,T − , y [0 ,T − ) ,see [23]. In this case, the input/output behavior of (1) is identifiable [24] and ( A, B, C, D ) can be computed up tosimilarity transformation, e.g., using subspace methods [25].A notable special case is that of full state measurement ,i.e., y ( t ) = x ( t ) and L = (cid:96) = 1 . In this case, (3) reduces to (cid:20) H ( x [0 ,T − ) H ( u [0 ,T − ) (cid:21) = (cid:20) x (0) x (1) · · · x ( T − u (0) u (1) · · · u ( T − (cid:21) . (4)If (4) has rank n + m , the input/state trajectory ( u [0 ,T − , x [0 ,T ] ) fully captures the behavior of (1a) whichallows the unique identification of A and B . A. Recap of Willems et al.’s fundamental lemma
Willems et al.’s fundamental lemma [17] is an important experiment design result. It reveals that the rank conditionon (3) is satisfied if the input sequence is chosen to bepersistently exciting. In a state-space setting, this result canbe stated as follows [17], [26].
Proposition 1:
Consider the minimal system (1). Let ( u [0 ,T − , y [0 ,T − ) be an input/output trajectory of (1) andlet L ≥ (cid:96) . If the input u [0 ,T − is persistently exciting oforder n + L then (3) has rank n + mL .We note that persistency of excitation of order n + L requires at least T ≥ ( m + 1)( n + L ) − data samples.III. O NLINE INPUT DESIGN
The main contribution of the paper will be to provide anew input design technique to guarantee the rank property of(3). In contrast to Willems’ fundamental lemma, this methodwill not rely on applying inputs that are persistently excitingof order n + L . Rather, each input u ( t ) is selected online , bymaking use of the previous samples y (0) , y (1) , . . . , y ( t − and u (0) , u (1) , . . . , u ( t − , see Figure 1. System u (0) y (0) System u (1) y (1) System u ( T − y ( T − ... ... Fig. 1. Illustration of the online input design approach. . Input design for input/state systems
To explain the idea, we will start with the simplestsetting of input/state data collected from system (1a). Inthis setting, the purpose is to design a sequence of inputs u (0) , u (1) , . . . , u ( T − so that the matrix (4) has full rowrank n + m . As mentioned, we opt for the online designof the inputs. Specifically, this means that for the design of u ( t ) we make use of the collected states x (0) , x (1) , . . . , x ( t ) and inputs u (0) , u (1) , . . . , u ( t − . The following theoremshows how such inputs, as well as a time horizon T , can bedesigned. Theorem 1:
Consider the controllable system (1a). Define T := n + m . Select a nonzero u (0) ∈ R m , and design theinput u ( t ) (for t = 1 , , . . . , T − ) as follows: • If x ( t ) (cid:54)∈ im H ( x [0 ,t − ) , select u ( t ) ∈ R m arbitrarily. • If x ( t ) ∈ im H ( x [0 ,t − ) there exists a ξ ∈ R n and anonzero η ∈ R m such that (cid:2) ξ (cid:62) η (cid:62) (cid:3) (cid:20) H ( x [0 ,t − ) H ( u [0 ,t − ) (cid:21) = 0 . (5)In this case, select u ( t ) such that ξ (cid:62) x ( t ) + η (cid:62) u ( t ) (cid:54) = 0 .Then, we have that rank (cid:20) H ( x [0 ,T − ) H ( u [0 ,T − ) (cid:21) = n + m. (6)We will not provide a proof of Theorem 1 at this stage,since the statement will follow from the more general The-orem 2 that will be proven in Section III-B. Instead, wewill explain the main ideas behind the result. The essenceof Theorem 1 is that the proposed input sequence increasesthe rank of the input/state Hankel matrix at every time step .To be specific, the input sequence guarantees that rank (cid:20) H ( x [0 ,t ] ) H ( u [0 ,t ] ) (cid:21) > rank (cid:20) H ( x [0 ,t − ) H ( u [0 ,t − ) (cid:21) (7)for all t = 1 , , . . . , T − . This increase in rank obviouslyoccurs for any input u ( t ) ∈ R m if x ( t ) (cid:54)∈ im H ( x [0 ,t − ) .Therefore, u ( t ) can be chosen arbitrarily in this case.However, it is not evident that (7) can be guaranteed if x ( t ) ∈ im H ( x [0 ,t − ) . Here, the inportant intermediatestep of Theorem 1 is to show that there exists a ξ anda nonzero η satisfying (5). The existence of these vectorsheavily relies on controllability of ( A, B ) , and can be provenusing a geometric argument that draws some inspiration fromHautus’ proof of Heymann’s lemma [27]. Once the existenceof ξ and η (cid:54) = 0 has been established, the construction of theinput u ( t ) should come without surprise. Indeed, choosing u ( t ) such that ξ (cid:62) x ( t ) + η (cid:62) u ( t ) (cid:54) = 0 ensures that (cid:2) ξ (cid:62) η (cid:62) (cid:3) is not in the left kernel of the input/state Hankel matrix upto time t . By (5), this implies that dim leftker (cid:20) H ( x [0 ,t ] ) H ( u [0 ,t ] ) (cid:21) < dim leftker (cid:20) H ( x [0 ,t − ) H ( u [0 ,t − ) (cid:21) , which is equivalent to (7). In this setting, we use x ( t ) in the design since it is independent of u ( t ) . Note that to guarantee the rank condition (6), we require atleast T ≥ n + m samples. A surprising fact is that Theorem 1always guarantees (6) with exactly n + m samples, despite thea priori lack of knowledge of the system matrices A and B .This makes our design method completely sample efficient in the sense that any experiment design method requires atleast as many samples as the one in Theorem 1. In particular,our method outperforms the usual condition of persistency ofexcitation of order n + 1 , which requires at least T ≥ nm + n + m samples to guarantee (6). Specifically, Theorem 1 savesat least nm samples compared to persistency of excitation.The number of samples in Theorem 1 is linear (instead ofquadratic) in the dimensions of the system variables.We emphasize that the success of Theorem 1 is due to its online nature: the input u ( t ) is computed by making use ofpast inputs and states, which contain valuable informationabout the unknown system. The method is thereby funda-mentally different from the classical persistency of excitationcondition, which is a purely offline condition: one can designa persistently exciting input before collecting any data. Inthe language of Sontag [28], persistently exciting inputs (oforder n + 1 ) are universal in the sense that they guarantee(6) for any controllable system (1a). In contrast, the inputsequence of Theorem 1 is tailored to the specific data-generating system (1a) that has produced the past inputs andstates. This allows for a reduction of the required numberof data samples, at the small cost of some simple onlinecomputations. B. Input design for input/output systems
Next, we turn our attention to input/output data generatedby system (1). In this setting, we want to find a sequence ofinputs u (0) , u (1) , . . . , u ( T − so that (3) has rank n + mL .As in the previous section, we opt for the online designof the inputs, meaning that for the design of u ( t ) we usethe collected outputs y (0) , y (1) , . . . , y ( t − and inputs u (0) , u (1) , . . . , u ( t − .The following theorem is a building block in our approach,and shows how full row rank of the input/state Hankel matrix (cid:20) H ( x [0 ,T − L ] ) H L ( u [0 ,T − ) (cid:21) = x (0) x (1) · · · x ( T − L ) u (0) u (1) · · · u ( T − L ) ... ... ... u ( L − u ( L ) · · · u ( T − (8)can be guaranteed if the state of (1a) is measured. Theorem 2:
Consider the controllable system (1a). De-fine T := n + ( m + 1) L − for L ≥ . Choose u (0) , u (1) , . . . , u ( L − ∈ R m arbitrarily, but not all zero.Furthermore, design u ( t ) (for t = L, L + 1 , . . . , T − ) asfollows: • If (cid:20) x ( t − L + 1) u [ t − L +1 ,t − (cid:21) (cid:54)∈ im (cid:20) H ( x [0 ,t − L ] ) H L − ( u [0 ,t − ) (cid:21) , (9)then choose u ( t ) ∈ R m arbitrarily. • If (cid:20) x ( t − L + 1) u [ t − L +1 ,t − (cid:21) ∈ im (cid:20) H ( x [0 ,t − L ] ) H L − ( u [0 ,t − ) (cid:21) , (10)hen there exist ξ ∈ R n and η , η , . . . , η L ∈ R m with η (cid:54) = 0 such that (cid:2) ξ (cid:62) η (cid:62) L · · · η (cid:62) η (cid:62) (cid:3) (cid:20) H ( x [0 ,t − L ] ) H L ( u [0 ,t − ) (cid:21) = 0 . (11)In this case, choose u ( t ) such that ξ (cid:62) x ( t − L + 1) + η (cid:62) L u ( t − L + 1) + · · · + η (cid:62) u ( t ) (cid:54) = 0 . Then, the input sequence u (0) , u (1) , . . . , u ( T − is suchthat rank (cid:20) H ( x [0 ,T − L ] ) H L ( u [0 ,T − ) (cid:21) = n + mL. (12) Proof:
Since u (0) , u (1) , . . . , u ( L − are not all zero,we clearly have rank (cid:20) x (0) u [0 ,L − (cid:21) = 1 . The idea of the proof is to show that in each time step, wecan increase the rank of the Hankel matrix. That is, for each t = L, L + 1 , . . . , T − , we want to prove that rank (cid:20) H ( x [0 ,t − L +1] ) H L ( u [0 ,t ] ) (cid:21) > rank (cid:20) H ( x [0 ,t − L ] ) H L ( u [0 ,t − ) (cid:21) . (13)Let t ∈ { L, L + 1 , . . . , T − } . First consider the case that(9) holds. Clearly, (13) is satisfied for any u ( t ) ∈ R m .Next, consider the case that (10) holds. We will prove bymeans of a contradiction argument that there exist vectors ξ and η , η , . . . , η L with η (cid:54) = 0 satisfying (11). If suchvectors do not exist then leftker (cid:20) H ( x [0 ,t − L ] ) H L ( u [0 ,t − ) (cid:21) = leftker (cid:20) H ( x [0 ,t − L ] ) H L − ( u [0 ,t − ) (cid:21) × { m } . (14)We claim that this implies that leftker (cid:20) H ( x [0 ,t − L ] ) H L ( u [0 ,t − ) (cid:21) = leftker (cid:20) H ( x [0 ,t − L ] ) H L − i ( u [0 ,t − i − ) (cid:21) ×{ im } (15)for all i = 1 , . . . , L . By hypothesis, (15) holds for i = 1 .Suppose that (15) holds for some ≤ i = k < L . Our goalis to show that (15) holds for i = k + 1 as well. Let (cid:2) ξ (cid:62) η (cid:62) L · · · η (cid:62) (cid:3) (cid:20) H ( x [0 ,t − L ] ) H L ( u [0 ,t − ) (cid:21) = 0 , (16)where ξ ∈ R n and η , . . . , η L ∈ R m . By the inductionhypothesis, we have η = · · · = η k = 0 so that (cid:2) ξ (cid:62) η (cid:62) L · · · η (cid:62) k +1 (cid:3) (cid:20) H ( x [0 ,t − L ] ) H L − k ( u [0 ,t − k − ) (cid:21) = 0 . (17)By (10), we obtain (cid:20) x ( t − L + 1) u [ t − L +1 ,t − k ] (cid:21) ∈ im (cid:20) H ( x [0 ,t − L ] ) H L − k ( u [0 ,t − k − ) (cid:21) . Combining the latter inclusion with (17) yields (cid:2) ξ (cid:62) η (cid:62) L · · · η (cid:62) k +1 (cid:3) (cid:20) H ( x [0 ,t − L +1] ) H L − k ( u [0 ,t − k ] ) (cid:21) = 0 . (18) Note that the laws of system (1a) imply that (cid:20) H ( x [1 ,t − L +1] ) H L − k ( u [1 ,t − k ] ) (cid:21) = (cid:20) A B
00 0 I (cid:21) (cid:20) H ( x [0 ,t − L ] ) H L − k +1 ( u [0 ,t − k ] ) (cid:21) . Therefore, (18) implies (cid:2) ξ (cid:62) A ξ (cid:62)
B η (cid:62) L · · · η (cid:62) k +1 (cid:3) (cid:20) H ( x [0 ,t − L ] ) H L − k +1 ( u [0 ,t − k ] ) (cid:21) = 0 . This yields (cid:104) ξ (cid:62) A ξ (cid:62)
B η (cid:62) L · · · η (cid:62) k +1 (cid:62) ( k − m (cid:105)(cid:20) H ( x [0 ,t − L ] ) H L ( u [0 ,t − ) (cid:21) = 0 . Finally, by the induction hypothesis we see that η k +1 = 0 , asdesired. Thus, (15) holds for all i = 1 , . . . , L . In particular,for i = L we obtain leftker (cid:20) H ( x [0 ,t − L ] ) H L ( u [0 ,t − ) (cid:21) = leftker H ( x [0 ,t − L ] ) × { mL } , equivalently (using the fact that (im X ) ⊥ = leftker X ), im (cid:20) H ( x [0 ,t − L ] ) H L ( u [0 ,t − ) (cid:21) = im H ( x [0 ,t − L ] ) × R mL . (19)Multiplication on both sides by the matrix (cid:2) A B (cid:3) yields A im H ( x [0 ,t − L ] ) + im B = im H ( x [1 ,t − L +1] ) . Using (10), we obtain A im H ( x [0 ,t − L ] ) + im B ⊆ im H ( x [0 ,t − L ] ) . In particular, we see that A im H ( x [0 ,t − L ] ) ⊆ im H ( x [0 ,t − L ] )im B ⊆ im H ( x [0 ,t − L ] ) . In other words, im H ( x [0 ,t − L ] ) is an A -invariant subspacecontaining im B . Since the reachable subspace (cid:104) A | im B (cid:105) of the pair ( A, B ) is the smallest A -invariant subspacecontaining im B (cf. [29, Ch. 3]), we see that R n = (cid:104) A | im B (cid:105) ⊆ im H ( x [0 ,t − L ] ) , where we made use of the fact that ( A, B ) is controllable.As such, im H ( x [0 ,t − L ] ) = R n and by (19) we concludethat im (cid:20) H ( x [0 ,t − L ] ) H L ( u [0 ,t − ) (cid:21) = R n + mL . This implies that t ≥ n + ( m + 1) L − which leads to acontradiction as t ∈ { L, L + 1 , . . . , T − } . As such, weconclude that (14) does not hold. Therefore, there exist ξ ∈ R n and η , η , . . . , η L ∈ R m with η (cid:54) = 0 such that (11)holds. Clearly, this means that there exists a u ( t ) such that ξ (cid:62) x ( t − L + 1) + η (cid:62) L u ( t − L + 1) + · · · + η (cid:62) u ( t ) (cid:54) = 0 . For such an input, we have that dim leftker (cid:20) H ( x [0 ,t − L ] ) H L ( u [0 ,t − ) (cid:21) > dim leftker (cid:20) H ( x [0 ,t − L +1] ) H L ( u [0 ,t ] ) (cid:21) , and consequently, (13) holds. This proves the theorem.Note that in the special case L = 1 , Theorem 2 reducesto Theorem 1. Next, we turn our attention to the situationn which we measure inputs and outputs and want to ensurethat (3) has rank n + mL . This is the topic of the followingtheorem. Theorem 3:
Consider the controllable and observable sys-tem (1). Let
L > (cid:96) and define T := n +( m +1) L − . Choose u (0) , u (1) , . . . , u ( L − ∈ R m arbitrarily, but not all zero.Furthermore, design u ( t ) (for t = L, L + 1 , . . . , T − ) asfollows: • If (cid:20) y [ t − L +1 ,t − u [ t − L +1 ,t − (cid:21) (cid:54)∈ im (cid:20) H L − ( y [0 ,t − ) H L − ( u [0 ,t − ) (cid:21) , (20)then choose u ( t ) ∈ R m arbitrarily. • If (cid:20) y [ t − L +1 ,t − u [ t − L +1 ,t − (cid:21) ∈ im (cid:20) H L − ( y [0 ,t − ) H L − ( u [0 ,t − ) (cid:21) , (21)there exist ξ , . . . , ξ L − ∈ R p and η , . . . , η L ∈ R m with η (cid:54) = 0 such that (cid:2) ξ (cid:62) L − · · · ξ (cid:62) η (cid:62) L · · · η (cid:62) (cid:3) (cid:20) H L − ( y [0 ,t − ) H L ( u [0 ,t − ) (cid:21) = 0 . (22)In this case, choose u ( t ) such that ξ (cid:62) L − y ( t − L + 1) + · · · + ξ (cid:62) y ( t − η (cid:62) L u ( t − L + 1) + · · · + η (cid:62) u ( t ) (cid:54) = 0 . (23)Then we have that rank (cid:20) H L ( y [0 ,T − ) H L ( u [0 ,T − ) (cid:21) = n + mL. (24) Proof:
Since u (0) , u (1) , . . . , u ( L − are not all zero,we clearly have rank (cid:20) y [0 ,L − u [0 ,L − (cid:21) = 1 . The idea of the proof is to show that for each time step t = L, L + 1 , . . . , T − , we have rank (cid:20) H L − ( y [0 ,t − ) H L ( u [0 ,t ] ) (cid:21) > rank (cid:20) H L − ( y [0 ,t − ) H L ( u [0 ,t − ) (cid:21) . (25)This would prove that rank (cid:20) H L − ( y [0 ,T − ) H L ( u [0 ,T − ) (cid:21) = n + mL, and consequently, (24) holds.Let t ∈ { L, L + 1 , . . . , T − } . First consider the case that(20) holds. Clearly, (25) is satisfied for any u ( t ) ∈ R m .Next, consider the case that (21) holds. We claim that leftker (cid:20) H L − ( y [0 ,t − ) H L ( u [0 ,t − ) (cid:21) (cid:54) = leftker (cid:20) H L − ( y [0 ,t − ) H L − ( u [0 ,t − ) (cid:21) ×{ m } . (26)We will prove this claim by contradiction. Thus, suppose that(26) does hold with equality, equivalently, im (cid:20) H L − ( y [0 ,t − ) H L ( u [0 ,t − ) (cid:21) = im (cid:20) H L − ( y [0 ,t − ) H L − ( u [0 ,t − ) (cid:21) × R m . (27) This implies that M im (cid:20) H ( x [0 ,t − L ] ) H L ( u [0 ,t − ) (cid:21) = M (cid:18) im (cid:20) H ( x [0 ,t − L ] ) H L − ( u [0 ,t − ) (cid:21) × R m (cid:19) , where M is defined as M := (cid:20) N I m (cid:21) , with N := (cid:20) O L − T L − I (cid:21) , O i defined in (2) and T i given by T i := D · · · CB D · · · CAB CB D · · · ... ... ... . . . ... CA i − B CA i − B CA i − B · · · D . Since ( C, A ) is observable and L > (cid:96) , the matrix M has fullcolumn rank. As such, we conclude that im (cid:20) H ( x [0 ,t − L ] ) H L ( u [0 ,t − ) (cid:21) = im (cid:20) H ( x [0 ,t − L ] ) H L − ( u [0 ,t − ) (cid:21) × R m , i.e., (14) holds. Similarly, we note that (21) implies N (cid:20) x ( t − L + 1) u [ t − L +1 ,t − (cid:21) ∈ N im (cid:20) H ( x [0 ,t − L ] H L − ( u [0 ,t − ) (cid:21) . Since N has full column rank, this implies that (10) holds.It now follows from the proof of Theorem 2 that t satisfies t ≥ n + ( m + 1) L − . This yields a contradiction since t ∈ { L, L + 1 , . . . , T − } . As such, (26) holds. Therefore,there exist ξ , ξ , . . . , ξ L − ∈ R p and η , η , . . . , η L ∈ R m with η (cid:54) = 0 such that (22) is satisfied. This means that wecan choose an input u ( t ) such that (23) holds. For this choiceof u ( t ) , we obtain dim leftker (cid:20) H L − ( y [0 ,t − ) H L ( u [0 ,t − ) (cid:21) > dim leftker (cid:20) H L − ( y [0 ,t − ) H L ( u [0 ,t ] ) (cid:21) . We conclude that (25) holds, which proves the theorem.We note that (24) can only hold if T ≥ n +( m +1) L − . ByTheorem 3, we can always design an informative experimentof length exactly n + ( m + 1) L − . In other words, thedesign procedure of Theorem 3 is again sample efficient. Incomparison, the usual persistency of excitation condition oforder n + L (Proposition 1) requires T ≥ n + ( m + 1) L + mn − samples. Theorem 3 thus saves at least mn samplesin comparison to persistency of excitation.Finally, is it noteworthy that exact knowledge of the state-space dimension is not required for Theorem 3, even though T is defined in terms of n . The only requirement is an upperbound L > (cid:96) on the lag of the system. The reason is that allthe steps of Theorem 3 can be executed without knowing n .One should only take some care with the stopping criterionsince T is a priori unknown. One simple approach to dealwith this is the following: apply the steps of Theorem 3 for t = 0 , , . . . until t is such that (27) is satisfied. By theproof of Theorem 3, the smallest t for which (27) holds is t = n +( m +1) L − . From this relation, n can be recovered,and for T := t , the rank condition (24) holds.V. C ONCLUSIONS AND DISCUSSION
The purpose of this paper has been to provide a newangle of attack to the problem of input design for data-driven modeling and control of linear systems. Instead ofthe classical persistency of excitation condition, we haveproposed an online method to select the inputs. We havestudied both input/state and input/output systems. For bothtypes of systems, we have provided an input design methodthat guarantees an important rank condition on a data Hankelmatrix. A notable feature of our approach is that it iscompletely sample efficient in the sense that it guaranteesthese rank conditions with the minimum number of samples.Compared to the usual persistency of excitation conditions,the online approach saves at least nm samples, where n and m are the dimensions of the state and input, respectively.The results presented in this paper are applicable to lineartime-invariant systems and exact data. In the case of noisymeasurements, rank conditions on Hankel matrices are ingeneral not sufficient to perform succesful data-driven mod-eling and control. Nonetheless, under suitable conditions,controllers with guaranteed stability and performance canbe obtained from data. In fact, data-based linear matrixinequality (LMI) conditions were studied in [4] and [6], and anecessary and sufficient condition was provided in [7] usinga generalization of Yakubovich’s S-lemma [30]. However,the problem of designing inputs that guarantee feasibility ofthe LMI’s in these papers is still open.It is important to note that such an input design problemis impossible to solve using offline conditions on the input.The reason is that feasibility of the data-based LMI’s isdependent on a signal-to-noise ratio, which is determined bythe true -a priori unknown- system . Therefore, making useof past data, and designing inputs online will be necessary. Infact, the experiment design method proposed in this paper ispromising also for control using noisy data. Indeed, recallthat the main idea behind our approach is to reduce thedimension of the left kernel of the Hankel matrix at each timestep. In a similar spirit, we envision the possibility to reducethe dimension of the convex solution set to the alternativeLMI [32] of the data-based LMI in [7]. This will be studiedin detail in our future work.R EFERENCES[1] I. Markovsky and P. Rapisarda, “Data-driven simulation and control,”
International Journal of Control , vol. 81, no. 12, pp. 1946–1959, 2008.[2] T. M. Maupong and P. Rapisarda, “Data-driven control: A behavioralapproach,”
Systems & Control Letters , vol. 101, pp. 37–43, 2017.[3] J. Coulson, J. Lygeros, and F. D¨orfler, “Data-enabled predictivecontrol: In the shallows of the DeePC,” in
Proceedings of the EuropeanControl Conference , June 2019, pp. 307–312.[4] C. De Persis and P. Tesi, “Formulas for data-driven control: Stabi-lization, optimality, and robustness,”
IEEE Transactions on AutomaticControl , vol. 65, no. 3, pp. 909–924, March 2020.[5] H. J. van Waarde, J. Eising, H. L. Trentelman, and M. K. Camlibel,“Data informativity: a new perspective on data-driven analysis andcontrol,”
IEEE Transactions on Automatic Control , vol. 65, no. 11,pp. 4753–4768, 2020. It is actually common that optimal excitation inputs depend on the data-generating system. In the identification literature, this has been referred toas the chicken-and-egg problem , see e.g. [31]. [6] J. Berberich, A. Koch, C. W. Scherer, and F. Allg¨ower, “Robust data-driven state-feedback design,” in
Proceedings of the American ControlConference , 2020, pp. 1532–1538.[7] H. J. van Waarde, M. K. Camlibel, and M. Mesbahi, “From noisydata to feedback controllers: non-conservative design via a matrix S-lemma,”
IEEE Transactions on Automatic Control , pp. 1–1, 2020.[8] J. Coulson, J. Lygeros, and F. D¨orfler, “Distributionallyrobust chance constrained data-enabled predictive control,” arxiv.org/abs/2006.01702 , 2020.[9] A. Xue and N. Matni, “Data-driven system level synthesis,” arxiv.org/abs/2011.10674 , 2020.[10] H. J. van Waarde and M. Mesbahi, “Data-driven parameterizations ofsuboptimal LQR and H2 controllers,” arxiv.org/abs/1912.07671 , 2020.[11] M. Guo, C. De Persis, and P. Tesi, “Learning control for polynomialsystems using sum of squares relaxations,” in
Proceedings of the IEEEConference on Decision and Control , 2020, pp. 2436–2441.[12] T. Dai and M. Sznaier, “A semi-algebraic optimization approachto data-driven control of continuous-time nonlinear systems,”
IEEEControl Systems Letters , vol. 5, no. 2, pp. 487–492, 2021.[13] G. C. Goodwin and R. L. Payne,
Dynamic System Identification:Experiment Design and Data Analysis . Academic Press, 1977.[14] M. Gevers and L. Ljung, “Optimal experiment designs with respectto the intended model application,”
Automatica , vol. 22, no. 5, pp.543–554, 1986.[15] H. Jansson and H. Hjalmarsson, “Input design via LMIs admittingfrequency-wise model specifications in confidence regions,”
IEEETransactions on Automatic Control , vol. 50, no. 10, pp. 1534–1549,2005.[16] X. Bombois, G. Scorletti, M. Gevers, P. M. J. Van den Hof, andR. Hildebrand, “Least costly identification experiment for control,”
Automatica , vol. 42, no. 10, pp. 1651–1662, 2006.[17] J. C. Willems, P. Rapisarda, I. Markovsky, and B. L. M. De Moor, “Anote on persistency of excitation,”
Systems & Control Letters , vol. 54,no. 4, pp. 325–329, 2005.[18] F. Fabiani and P. J. Goulart, “The optimal transport paradigmenables data compression in data-driven robust control,” arxiv.org/abs/2005.09393 , 2020.[19] M. Yin, A. Iannelli, and R. S. Smith, “Maximum likelihood estimationin data-driven modeling and control,” arxiv.org/abs/2011.00925 , 2020.[20] A. Iannelli, M. Yin, and R. S. Smith, “Experiment designfor impulse response identification with signal matrix models,” arxiv.org/abs/2012.08126 , 2020.[21] H. Hjalmarsson, M. Gevers, S. Gunnarsson, and O. Lequin, “Iterativefeedback tuning: theory and applications,”
IEEE Control SystemsMagazine , vol. 18, no. 4, pp. 26–41, Aug 1998.[22] K. Lindqvist and H. Hjalmarsson, “Identification for control: adaptiveinput design using convex optimization,” in
Proceedings of the IEEEConference on Decision and Control , vol. 5, 2001, pp. 4326–4331.[23] I. Markovsky, J. C. Willems, P. Rapisarda, and B. L. M. De Moor,“Algorithms for deterministic balanced subspace identification,”
Auto-matica , vol. 41, no. 5, pp. 755–766, 2005.[24] I. Markovsky and F. D¨orfler, “Identifiability in the behavioral setting,” homepages.vub.ac.be/imarkovs/publications/identifiability.pdf , 2020.[25] P. van Overschee and B. de Moor,
Subspace Identification for LinearSystems: Theory, Implementation, Applications . Kluwer AcademicPublishers, 1996.[26] H. J. van Waarde, C. De Persis, M. K. Camlibel, and P. Tesi, “Willems’fundamental lemma for state-space systems and its extension tomultiple datasets,”
IEEE Control Systems Letters , vol. 4, no. 3, pp.602–607, 2020.[27] M. Hautus, “A simple proof of Heymann’s lemma,”
IEEE Transactionson Automatic Control , vol. 22, no. 5, pp. 885–886, 1977.[28] E. Sontag, “On the length of inputs necessary in order to identify adeterministic linear system,”
IEEE Transactions on Automatic Control ,vol. 25, no. 1, pp. 120–121, 1980.[29] H. L. Trentelman, A. A. Stoorvogel, and M. Hautus,
Control Theoryfor Linear Systems . London, UK: Springer Verlag, 2001.[30] V. A. Yakubovich, “S-procedure in nonlinear control theory,”
VestnikLeningrad University Mathematics , vol. 4, pp. 73–93, 1977.[31] M. Forgione, X. Bombois, and P. M. J. Van den Hof, “Data-drivenmodel improvement for model-based control,”
Automatica , vol. 52,pp. 118–124, 2015.[32] V. Balakrishnan and L. Vandenberghe, “Semidefinite programmingduality and linear time-invariant systems,”