Model Reduction for Complex Hyperbolic Networks
PP r e p r i n t Model Reduction for Complex Hyperbolic Networks
Christian Himpe ∗ Mario Ohlberger ∗ Abstract
We recently introduced the joint gramianfor combined state and parameter reduc-tion [C. Himpe and M. Ohlberger. Cross-Gramian Based Combined State and Param-eter Reduction for Large-Scale Control Sys-tems. arXiv:1302.0634, 2013], which is ap-plied in this work to reduce a parametrizedlinear time-varying control system modelinga hyperbolic network. The reduction en-compasses the dimension of nodes and pa-rameters of the underlying control system.Networks with a hyperbolic structure havemany applications as models for large-scalesystems. A prominent example is the brain,for which a network structure of the vari-ous regions is often assumed to model prop-agation of information. Networks with manynodes, and parametrized, uncertain or evenunknown connectivity require many and in-dividually computationally costly simulations.The presented model order reduction enablesvast simulations of surrogate networks exhibit-ing almost the same dynamics with a smallerror compared to full order model.Keywords:
Hyperbolic Network, Model Reduc-tion, Combined Reduction, Cross Gramian, JointGramian, Empirical Gramian ∗ Contact: [email protected] , [email protected] , Institute for Computational andApplied Mathematics at the University of Münster, Einstein-strasse 62, D-48149 Münster, Germany Complex Networks are often employed as models forlarge-scale systems like connectivity inside the brain,linking structure of the Internet or trust relations insocial networks. Even in cosmology, causality canbe modeled based on a network as demonstrated in[1]. Such networks are of hyperbolic structure, inwhich older nodes are favorably connected comparedto younger ones. In many settings the interconnec-tions of a network contain uncertainties or representthe (possibly unknown) parametrized quantities of in-terest. Parametrized models with high-dimensionalstate and parameter spaces are often infeasible toevaluate many times for different locations of the pa-rameter space. This is due to two effects. First, thehigh-dimensional state space makes each integrationof the dynamic system computationally costly. Sec-ond, the high-dimensional parameter space may makemany simulations necessary. In this situation modelreduction will accelerate these otherwise costly ex-periments. Particularly, the combined reduction ofstate and parameter space will be illustrated.This setting for model reduction was inspired by [1].The gramian-based (state) reduction approach origi-nates in (approximate) balanced truncation compre-hensively described in [4]. An alternative computa-tional method for these gramians, based on properorthogonal decomposition, was introduced in [2] un-der the name empirical gramians . For the param-eter identification and combined state and parameterreduction, the empirical joint gramian from [6] is uti-lized.In the next section the construction of a hyperbolicnetwork is described. In section 3 the state reductionprocedure, then in section 4 the parameter identifica-tion and combined state and parameter reduction is a r X i v : . [ m a t h . O C ] O c t r e p r i n t TimeTime
Figure 1: Hyperboloid representing space-time forthe hyperbolic network. The contour line representthe projection of the node space at points in time.explained. For an efficient assembly of the requiredgramians, the empirical cross gramian is presented insection 5. In section 6 the usage of empirical grami-ans in the context of uncertainty quantification isoutlined. Finally, in section 7 a sample network isreduced.
Generating a hyperbolic network is a dynamic processwith a discrete time space. The following descriptionis taken from [1] and [1, Supplementary Notes II. C].At each time step t i a new node is born by drawingfrom a uniform random distribution on the circle S yielding a new node at α i ∈ [0 , π ] and a radius r i = 2 ln iv with network degree v . The new node x i connects to all existing nodes x ...i − that satisfy: r j + 2 ln( π − | π − | α i − α j || ) < , ∀ j < i. This leads to a "space-time" representation in whichthe network is created in the shape of a hyperboloidvisualized in Figure 1.In this work a maximum number of N nodes will beset. Such a hyperbolic network can be modeled asa matrix A ∈ R N × N , by treating a matrix element A i,j as connection from node i to j , which leads to a For unit curvature. dynamic system setting of a basic linear autonomoussystem: ˙ x = Ax, with state x ∈ R N and a system matrix A ∈ R N × N embodying the network structure. Usually some ex-ternal input or control is applied to the system andthe quantities of interest are some subset or linearcombination of the systems states. This leads to alinear control system ˙ x = Ax + Bu,y = Cx, with input u ∈ R J , input matrix B ∈ R N × J , outputs y ∈ R O and output matrix C ∈ R O × N . The matrices B, C can for example be used to excite only a certainnodes via B and observe the dynamics in others via C . In this setting it is assumed, the connections,being the components of A , between the networksnodes, are parametrized in each component by θ ∈ R N : ˙ x = A ( θ ) x + Bu,y = Cx.
As described above, in each time step a new nodeis born and connects to existing nodes. Hence, thecomponents of A , which are components of θ , too,change over time and yield a parametrized lineartime-varying control system : ˙ x = A ( θ ( t )) x + Bu,y = Cx.
Now, the hyperbolic network can be treated with themodel reduction methods of control theory.
A well known method for model order reduction ofthe state space in a control system setting is bal-anced truncation . In this approach a systemscontrollability and observability is balanced and the r e p r i n t least controllable and observable states are trun-cated. A systems controllability is encoded in agramian matrix W C which is a solution of the Lya-punov equation AW C + W C A T = BB T . A sys-tems observability is also encoded in a gramian ma-trix W O which is a solution of the Lyapunov equa-tion A T W O + W O A = C T C . Then, by a balanc-ing transformation, computed from the controllabil-ity and observability gramian, the control system istransformed in a manner such that the states areordered from the most to the least important, forthe systems dynamics. This ordering is based on theHankel singular values σ i = (cid:112) λ i ( W O W C ) . After thesorting, the least controllable and observable can betruncated. The cross gramian encodes controllabil-ity and observability into one gramian matrix. A so-lution of the Sylvester equation AW X + W X A = − BC yields the cross gramian matrix W X as a solution, ifthe system is square . In case the system is symmet-ric, meaning the systems gain g = − CA − B is sym-metric, then the absolute value of the cross gramianseigenvalues equal the Hankel singular values: | λ i ( W X ) | = (cid:112) λ i ( W O W C ) . Given an asymptotically stable system, the crossgramian can also be computed as the time inte-gral over the product of input-to-state and state-to-output map: W X = (cid:90) ∞ e At BCe At dt, which will be the basis for computing the empiricalgramian variant.The state reduction is based on the singular valuesof the cross gramian W X . A singular value decom-position of W X = U DV , provides a projection of thestates
V, U in which the states are sorted by their im-portance. Without loss of generality, the singular val-ues, composing the diagonal matrix D , are assumedto be sorted in descending order. Based on this pro-jection, the matrices A, B, C and the initial value x The system has the same number of inputs and outputs can be partitioned and reduced, V = (cid:18) V V (cid:19) ,U = (cid:0) U U (cid:1) , ⇒ ˜ A = V AU ˜ B = V B ˜ C = CU ˜ x (0) = V x (0) . This direct truncation approximates closely the bal-anced truncation of controllability and observabilitygramians, but does not require an additional balanc-ing transformation.In case the system is not square or not symmetric,following the approach from [4], the system
Σ = { A, B, C } can be embedded into a symmetric sys-tem ˆΣ = { ˆ A, ˆ B, ˆ C } . Since for each square matrix A ∈ R N × N there exists a symmetrizer J = J T suchthat AJ = JA T and thus the embedding system isgiven by: ˆ A = A, ˆ B = (cid:0) JC T B (cid:1) , ˆ C = (cid:18) CB T J − (cid:19) . If the system matrix A is symmetric, and thus J = , the embedding of the system simplifies to: ˆ A = A, ˆ B = (cid:0) C T B (cid:1) , ˆ C = (cid:18) CB T (cid:19) . Even though the number of inputs and outputs isincreased the number of states remains the same asin the original system.
The concept of controllability and especially observ-ability extends to parametrized systems by treating r e p r i n t the parameters as additional states. These parame-ter states are constant over time and are assigned theparameters value as initial states: ˙˘ x = (cid:18) ˙ x ˙ θ (cid:19) = (cid:18) f ( x ( t ) , u ( t ) , θ )0 (cid:19) ,y = g ( x ( t ) , u ( t ) , θ ) , ˘ x = (cid:18) x θ (cid:19) . This augmented system, used in [6], can now be sub-ject to a similar method to the direct truncation ofthe cross gramian for state reduction.The cross gramian of the augmented system yieldsthe joint gramian introduced in [6]: W J = (cid:18) W X W M (cid:19) ∈ R ( n + p ) × ( n + p ) , with its upper left block ( W X ) being the usual crossgramian of the system. The identifiability infor-mation of the parameters is encoded in W M . Theparameter related information is then extracted bythe Schur-complement of the symmetric part of thejoint gramian, resulting in the cross-identifiabilitygramian W ¨ I : W ¨ I := W ∗ M ( 12 ( W X + W TX )) − W M . A singular value decomposition of W ¨ I , provides a pro-jection of the parameters that are sorted by their im-portance: W ¨ I = U DV, ⇒ ˜ θ = V θ.
Based on this projection the parameters can be par-titioned and reduced, ˜ θ = (cid:18) ˜ θ ˜ θ (cid:19) , ⇒ (cid:107) ˜ θ (cid:107) ≈ (cid:107) ˜ θ (cid:107) . As described in [6], by employing a truncation ofstates based on the singular values of W X and pa-rameters based on the singular values of W ¨ I enablesthe combined reduction. Empirical gramians were introduced in [2] and aresolely based on simulations of the underlying controlsystem. These simulations use perturbations in input u and initial states x which are averaged. The re-quired perturbations are organized into sets allowinga systematic perturbation of input E u × R u × Q u andinitial states E x × R x × Q x : E u = { e i ∈ R j ; (cid:107) e i (cid:107) = 1; e i e j (cid:54) = i = 0; i = 1 , . . . , m } ,E x = { f i ∈ R n ; (cid:107) f i (cid:107) = 1; f i f j (cid:54) = i = 0; i = 1 , . . . , n } ,R u = { S i ∈ R j × j ; S ∗ i S i = ; i = 1 , . . . , s } ,R x = { T i ∈ R n × n ; T ∗ i T i = ; i = 1 , . . . , t } ,Q u = { c i ∈ R ; c i > i = 1 , . . . , q } ,Q x = { d i ∈ R ; d i > i = 1 , . . . , r } . Now the empirical cross gramian can be defined asfollows (taken from [6]):For sets E u , E x , R u , R x , Q u , Q x , input ¯ u dur-ing steady state ¯ x with output ¯ y , the empiricalcross gramian ˆ W X relating the states x hij of in-put u hij ( t ) = c h S i e j + ¯ u to output y kla of x kla = d k T l f a + ¯ x , is given by: ˆ W X = 1 | Q u || R u | m | Q x || R x | | Q u | (cid:88) h =1 | R u | (cid:88) i =1 m (cid:88) j =1 | Q x | (cid:88) k =1 | R x | (cid:88) l =1 · c h d k T l (cid:90) ∞ Ψ hijkl ( t ) dt T ∗ l , Ψ hijklab ( t ) = f ∗ b T ∗ k ∆ x hij ( t ) e ∗ i S ∗ h ∆ y kla ( t ) , ∆ x hij ( t ) = ( x hij ( t ) − ¯ x ) , ∆ y kla ( t ) = ( y kla ( t ) − ¯ y ) . The joint gramian [6] encapsulates the cross gramian,hence the empirical joint gramian is computed inthe same manner as the empirical cross gramian, yetof the augmented system. As shown in [3], the em-pirical gramians extend to time-varying systems, andthus can be applied in this setting for the hyperbolicnetworks. r e p r i n t The connections between the network nodes, whichare modeled by the components of the system matrix A , might contain uncertainties. Due to the computa-tion of empirical gramians based on simulations, po-tential uncertainties in initial state and external inputcan be incorporated by enlarging the correspondingset of perturbations respectively. Hence, for an aug-mented system uncertainties in the parameters canalso be included. This allows robust model reduction.Additionally, the parameter reducing projection canalso be used to reduce, for example in a Gaussiansetting, mean and covariance of a parameter distri-bution. To demonstrate the capabilities of this approach asynthetic hyperbolic network is utilized. As describedin section 2, the time varying system is growing witheach time step. This network with a maximum of nodes, thus a state dimension of x ∈ R , and inputsand outputs is selected. Furthermore, it is assumedthat each connection is reciprocal, hence A = A T and θ ⊂ R . All possible connections of all nodes aretreated as (time-varying) parameters in this setting, ˙ x = A ( θ ( t )) x + Bu,y = Cx.
Yet input matrix B ∈ R × and output matrix C ∈ R × are random and notably C (cid:54) = B T , whichrequires an embedding into a symmetric system (seeSection 3).First, in an offline phase, that has to be performedonly once, a reduced order model is created. Thereduction procedure uses the empirical joint gramianof Section 5 that computes the cross gramian of theembedded augmented system: ˙ x = (cid:18) A ( θ ( t ))0 (cid:19) x + (cid:18) C T B (cid:19) u,y = (cid:18) CB T (cid:19) x. Then, a reduction of states, based on the singular val-ues of W X , and of parameters, based on the singularvalues of W ¨ I , is performed. Second, in the onlinephase, the reduced model can be evaluated.The computations are performed using the empiricalgramian framework - emgr described in [5]. Sourcecode for the following experiments can be found at http://j.mp/ecc14_code .Since the network evolves during its evaluation thereduction has to be performed for unknown con-nectivity. A distribution of the singular values ofthe empirical cross gramian W X and the empiri-cal cross-identifiability gramian W ¨ I is given in Fig-ure 2 and Figure 3. The singular values of thesetwo empirical gramians describe the energy containedin the state and parameter respectively. A reduc-tion of the parameter space from dimension to is suggested by the singular values of the cross-identifiability gramian. For the state space a reduc-tion from dimension to is performed, based onthe singular values of the cross gramian. The reducedmodel can be evaluated and compared to the full or-der model. Figures 4 and 5 show the impulse responseof the full order and reduced order network, whilethe relative error between them is shown in Figure 6.Between the two time series of the full and reducedorder model impulse response, the relative L -erroris . . The sharp drop in the singular values ofthe cross-identifiability gramian determines the re-duced order model dimension; a truncation of moreparameter space dimensions will introduce a signifi-cant higher error. The descent of singular values ofthe cross gramian also allows a graduated increase oferror when truncating states. To scan various loca-tion of the parameter space, only the low-dimensionalreduced parameter space has to be scanned.Comparing the reduced order model with the full or-der model, the combined reduction decreased the in-tegration time by 21% and memory requirements by70%. See http://gramian.de r e p r i n t
10 20 30 40 50 6010 −16 −14 −12 −10 −8 −6 −4 −2 Singular Value S t a t e Figure 2: Distribution of the singular values of thecross gramian
200 400 600 800 1000 1200 1400 1600 1800 200010 −20 −18 −16 −14 −12 −10 −8 −6 −4 −2 Singular Value P a r a m e t e r Figure 3: Distribution of the singular values of thecross identifiability gramianOriginal Offline Online RelativeTime (s) Time (s) Time (s) L2-Error0.0203 277.1988 0.0160 0.00036Table 1: Original Time, Offline Time, Online Timeand Relative L2-Error; averaged over 100 simula-tions.
The numerical experiments suggest that the empiri-cal joint gramian [6], which is based on the empirical cross gramian, can be applied to reduce this type ofcontrol system with hyperbolic network structure. Asshown, a linear time-varying control system in whichalso the parameter values vary over time, can be han-dled by the empirical gramians. This concurrent re-duction of state and parameter spaces enables, forexample, scenarios in which the reduced model canbe used to scan the parameter space.Using the (empirical) cross gramian of the embeddedsystem is efficient, since the number of inputs andoutputs is small compared to the number of statesand no symmetrizer needs to be computed and in-verted, A has been chosen to be symmetric. Yet, fornetworks with a non-symmetric system matrix A , apossibly costly computation of the symmetrizer andits inverse is required. Thus, a generalization of thecross gramian to non-symmetric systems should beexplored. References [1] D. Krioukov, M. Kitsak, R. Sinkovits, D. Ride-out, D. Meyer and M. Boguñá.6 Network Cos-mology.
Nature Publishing Group, Scientific re-ports , 2:2012.[2] S. Lall, J.E. Marsden, and S. Glavaski. Empiricalmodel reduction of controlled nonlinear systems.
Proceedings of the IFAC World Congress , F:473–478, 1999.[3] M. Condon and R. Ivanov. Empirical balancedtruncation of nonlinear systems.
Journal of Non-linear Science , 14(5):405–414, 2004.[4] A.C. Antoulas. Approximation of large-scale dy-namical systems.
Society for Industrial Mathe-matics , volume 6, 2005.[5] C. Himpe and M. Ohberger. A Unified Fame-work for Empirical Gramians.
Hindawi Journalof Mathematics , 2013.[6] C. Himpe and M. Ohlberger. Cross-GramianBased Combined State and Parameter Reduc-tion for Large-Scale Control Systems.
Submitted
Preprint at arXiv(math.OC): 1302.0634, 2013. r e p r i n t Time S t a t e s
10 20 30 40 50 60 70 80 90 10012345678 00.20.40.60.811.2
Figure 4: Impulse response of the full order model.
Time S t a t e s
10 20 30 40 50 60 70 80 90 10012345678 00.20.40.60.811.2
Figure 5: Impulse response of the reduced ordermodel.
Time R e l a t i v e E rr o r
10 20 30 40 50 60 70 80 90 10012345678 051015x 10 −4−4