A machine learning based control of complex systems
AA machine learning based control of complex systems
P. Garc´ıa.Laboratorio de Sistemas Complejos,Departamento de F´ısica Aplicada,Facultad de Ingenier´ıa, Universidad Central de Venezuela.
Abstract
In this work, inspired in the symbolic dynamic of chaotic systems and using machine learningtechniques, a control strategy for complex systems is designed. Unlike the usual methodologiesbased on modeling, where the control signal is obtained from an approximation of the dynamicrule, here the strategy rest upon an approach of a function, that from the current state of thesystem, give the necessary perturbation to bring the system closer to a homoclinic orbit thatnaturally goes to the target.The proposed methodology is data-driven or can be developed in a based-model context andis illustrated with computer simulations of chaotic systems given by discrete maps, ordinarydifferential equations and coupled maps networks.Results shows the usefulness of the design of control techniques based on machine learningand numerical approach of homoclinic orbits.
There are experimental and/or numerical evidence of the chaotic behavior in areas such as: pop-ulation dynamics[1], epidemiology[2], ecosystems[3], cardiac rhythm[4], neurological systems[5],optoelectronic systems[6], chemical reactions[7], economic systems[8], communication systems[9]and a lot of more situations in physics, chemistry, biology and engineering[10]. Added to this,that behavior can be useful in areas such as secure communication [11], where chaotic systemsoffer an alternative for the development of new technologies.As can be seen in most of the before mentioned cases, it is desirable to have strategies thatallow the evolution of the system to be regulated: the population of animal communities canbe controlled to prevent extinctions or the spread of diseases, for obvious reasons the epidemicdiseases must be controlled, the heart rate must be controlled in the case of arrhythmia, brainactivity must be controlled in case of epilepsy or finally some chemical, physical or computa-tional systems can be controlled for the benefit of all. This represents sufficient motivation todevote effort to the design of models and/or control schemes for such systems.From a general perspective these systems can be modeled as s t = f tr ( s ), where r is a set ofparameters, t ∈ (cid:60) or t ∈ N represents the time, s ∈ S the state of the system and f , a functionthat regulates the evolution of the system; so that, the control over them can only be achievedapplying disturbances of some of these components.When systems like the previously mentioned have chaotic behavior: on the one hand they havesensitivity to the initial conditions, which makes them difficult to predict in the long term, buton the other it offers the opportunity to change their behavior radically using small perturba-tions, this is, with a low cost and without producing important alterations in the system. a r X i v : . [ n li n . C D ] M a r he seminal article on this topic presents a method for the control of chaotic systems, knownas OGY method[12], this is the archetype of the methods that belong to the class of those thatdisturb the parameter. As evidence of it importance, from the point of view of basic researchor technology of the finding of Ott, Grebogi and York[12], where the term controlling chaos wascoined, there are many other methods for stabilizing chaotic systems that have been suggestedafterwards, see for instance [13] and references therein.Another method almost as celebrated, as the previous one, is a feedback method proposed byPyragas et. al. [14] belonging to the methods that disturb f . In this method, the stabilizationof unstable periodic orbits of a chaotic system is achieved by combined feedback with the useof a specially designed external oscillator. Neither of the two methods require an a priory ana-lytical knowledge of the dynamics system and may be applicable in experimental situations.Among the strategies mentioned above, and many others, there are a significant amount of con-trol strategies based on linear approximations of the dynamics around the target. When is it so,it is necessary that the linear approach is a good representation of the system or equivalentlythat the trajectory of the system is close enough to the target.In this work, unlike those that model the dynamics to design the control function, here thecontrol function is directly obtained by using Gaussian Processes Regression method[15] whichis training using as examples a set of states, that are close to a points of a homoclinic orbit.The article is organized as follow: in Section II a symbolic dynamic inspired but learning machinebased approach to control of chaotic dynamic is presented. Here, model-based and data-drivenversions of the method are presented. Section III is devoted to presented some examples of theperformance of the methodology and finally in Section IV we give some concluding remarks. Colloquially, a control problem can be paraphrased as a sequence of three steps: (i) the identi-fication of the control’s object, (ii) the selection of the control’s target and (iii) the design of acontrol strategy.
We will consider chaotic systems in one or more dimensions, continuous or discrete and repre-sented by discrete maps, ordinary differential equations or coupled maps networks. However, inorder to show the ideas behind our control problem in a simple form, we will start consideringas control object a chaotic, one-dimensional and uni-modal map s t +1 = f r ( s t ), for later in theresults section use as control objects the rest of previously listed systems.The targets, in all examples, are given by a unstable fixed-point solutions ( s ∗ ) of the dynamicalsystem, although the control strategy can be adapted to the case of other types of periodicorbits. Our approach to the control strategy propose to the estimate a small perturbation u t , to theactual state s t , in such a way that in the ( t + 1)-th iteration of f the orbit come close to somepoint s , whose evolution has the form s , s , · · · , s k − , s ∗ + δ , where δ is a small value. This is,we should to estimate a perturbation u t , such that, if s t +1 = f r ( s t ) + u t , (1)then | s t +1 − s | < | s t − s | , with s ∗ + δ = f kr ( s ). .2.1 Target improve In some sense, the idea behind of the methodology is like to the OGY method, it consists inbringing the system closer, not immediately, to the control’s target but to points of orbits thatnaturally takes it as close as possible to target. Thus, if we are able to estimate these new pointsand include it in the target, we expect that the control scheme to be more effective. The orbitsthat fulfill that condition, in the case of the fixed points of chaotic systems, are the homoclinicorbits . This orbits joins a saddle equilibrium point to itself[16], and offer a number of potentialtargets equal to the points in this orbit. However, we must to note that here, unlike the OGYmethod, we do not disturb the parameters of the system or use explicitly a linear approximationof the system.Our control strategy start by determining the set of estates { s i } qi =1 . Clearly, these statescan be estimated if we iterate the inverse function of f , from s ∗ + δ , this is, s = f − kr ( s ∗ + δ ),but this can no be done simply because the inverse of a nonlinear function is multi-valued.On the other hand, it is well known[17] that the phase space evolution of an uni-modaldiscrete map can be translated, in biunivocal way, into a binary representation by using apartition of the state space in two regions, labeled each of them by one symbol ( σ ), 0 or 1. Thesymbolic representation is obtained replacing s t by the label associated to the element of thepartition which belongs s t → σ t . With this new information f − r , can be written as as: f − r ( s t , Σ) = (cid:26) f − r, ( s t ) if σ t = 0 f − r, ( s t ) if σ t = 1 , (2)where f r, and f r, are the monotone branch at the left or right from the maximum of f ,respectively. In this form, every state s t , can be represented by a sequence of symbols Σ = { σ t + i } li =0 , given by the symbolic chain generated from the initial condition s t . Here l give theaccuracy with which s t is represented. In particular, the states from which the system reachesa neighborhood of the fixed point are given by: s = f − r,σ ◦ · · · ◦ f − r,σ l − ◦ f − r,σ l ( s ∗ + δ ) (3)Due to the contraction of f − r,σ k and for l large enough, the iteration of (3) converges approx-imately to the real number s independently of the value of δ , if the sequence of symbols σ t isappropriate, i. e. if these sequences are of the formΣ i = σ i , σ i , · · · , σ ik , σ ∗ , · · · , σ ∗ (cid:124) (cid:123)(cid:122) (cid:125) ( l − k ) times (4)and are large enough.It form a set of 2 k possible binary sequences. In general not all of them are admissible (seefor example [17], Sec. 2.5.5), but the existence of some sequences of states associated withthe admissible chains is guaranteed by the presence of homoclinic orbits in the attractor of f .Hence this sequences approximating segments of homoclinic orbits, will be used to calculate the2 k initial conditions, s i = f − lr ( s R , Σ i ) with s R a random real number in the adequate domain,which places the system in a stable orbit towards the target s ∗ . Finally, the addition of the set s ’s to the target, clearly will contributes to decrease the convergence time from the system’scurrent state to the control’s target. In order to estimate the sequence u k we start generating, from q strings like (4), q initial con-ditions S = { s , s , · · · , s q } , from which the system reach a neighborhood of s ∗ in l iterations. ith the before estimate states, M pairs ( s t , u t ) are generated using, u t = s t +1 − N ( S , s t +1 ) , (5)from the elements of an typical orbit { s t } Nt =1 of the system. Here, N ( S , s t +1 ) is a function thatpick out, the element of S nearest to s t +1 is defined as: N ( S , s t ) = arg min s i {| s i − s t +1 |} . (6)These pairs will later serve as training data for a machine learning scheme that estimates u t = u ( s t ), for any s t in the domain of f . In this way the control problem is turned into aregression problem, that we propose can be to solved using artificial intelligence techniques.As one, among many alternatives to approximate u , in this work we use a powerful strategyfrom machine learning known as Gaussian Process Regression[15]. Here we will understandthese processes according to their usual definition: a set of indexed random variables, suchthat every finite subset of those random variables has a multivariate normal distribution. Thedistribution of a GP is the joint distribution of all those random variables, and as such, it is adistribution over functions with a continuous domain.These process are specified by its mean function m ( s ) and covariance function k ( s, s i ) and area natural generalization of the Gaussian distribution whose mean and covariance are representedby a vector and a matrix, respectively. We will represent this process, with the usual notation,as: u ∼ GP ( m ( s ) , k ( s, s (cid:48) )) , (7)and we will read this, as: the function u is distributed as a GP with prior mean function m ( x ) and covariance function k ( s, s (cid:48) ) . If we generate a random vector from this distribution, u ∼ N ( m ( s ), k ( s, s (cid:48) )), this vector will have the function values u ( s ) as coordinates indexed by s . In this work, the strategy of control depend on the methodology to approximate u but webelieve that any methodology used, to interpolate the response of the system to states that arenot present in the their training set, can solve the control problem. Since it is not always possible to have the symbolic dynamics of the system, this section showshow it is possible to implement our idea using data from a typical orbit of the system. Thusa data-driven version of the strategy is easily designed if we replace the symbolic dynamicsknowledge with an algorithm that identifies, in the time series, a set of sequences of statesconverging to s ∗ .At this point, we could use a numerical method of detection of homoclinic orbits as inreferences [19, 20, 21] and find the points which we will include in the target. However, we willopt for a simpler strategy: we start identifying, in the time series, the set of the nearest statesto s ∗ , { s m , s m , . . . , s m q } , so that s i = s m i − l , with i = 1 , · · · , q and l is a integer identifyingthe l -th predecessor of s m i . Thus the set S = { s i } qi =1 is constructed and given that set thecontrol strategy is the same that in previous section.In the following section the performance of the methodology is shown. In the case of thelogistic map we will present both approximations (mode-based and model-free versions), in therest it is not so simple to obtain the symbolic representation so we will only show results of themodel-free approach. Results
In order to show the performance of the proposed scheme, we use as examples: the Logistic map,the Henon map, the Lorenz system and a network of coupled discrete maps. These representone, two, three and n -dimensional systems with discrete time and continuous time. In this case, f ( s t ) = 4 s t (1 − s t ) has an unstable fixed point at s ∗ = 3 /
4. The inverse functionof f can be written using the symbolic dynamic: f − ( s t , Σ) = (cid:40) + √ − s t if σ t = 0 − √ − s t if σ t = 1 , (8)The controlled system can be written as s t +1 = 4 s t (1 − s t ) + u t , with u t = u ( x t ). In thiscase, since the symbolic dynamic is known, the model-based and data-driven control schemesare showed. Figures 1 and 2, shows the results of the control scheme application using 500 datapoints with l = 6, k = 4 and q = 16 in the case of model based control and l = 6 and q = 16 inthe case of data-driven control. - - s t u t ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● t s t t u t Figure 1:
Model based control approach of Logistic map. Left: Black points represents the training setof the Gaussian process. Gray dashed line represents the predicted values of the function u ( s t ) and grayshadow represents the confidence interval of the predictions, given by two times the standard deviation ofthe errors. Right: Evolution of the controlled system. The inset shows the applied perturbations. In bothcases, the results shown were averaged for 100 initial random conditions.It is worth to note that in general, if all possible sequences of symbolic states are consideredas training data generation, the quality of the performance of the strategy could be affected,because this number of pairs does not necessarily coincide with the admissible ones. Now, ifthe dynamics are available, it is possible to determine the admissible symbolic sequences usingthe strategy proposed in [17]. The study of this aspect of the problem is not the subject of thisarticle, but will be addressed in future works. The Henon map is used as canonical example of bi-dimensional chaotic system and given by adifference equation system. It has an unstable fixed point at ( x ∗ , y ∗ ) = (0 . , . x t +1 = 1 − a ( x t + y t ) + u t ,y t +1 = bx t . (9) .0 0.2 0.4 0.6 0.8 1.0 - - s n u n ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● t s t t u t Figure 2:
Data-driven control approach of Logistic map. Left: Black points represents the training set of theGaussian process. Gray dashed line represents the predicted values of the function u ( s t ) and gray shadowrepresents the confidence interval of the predictions, given as in the model-based case. Right: Evolution ofthe controlled system. The inset shows the applied perturbations. In both cases, the results shown wereaveraged for 100 initial random conditions.Here u t = u ( x t , y t ) and 10 data points are used to to train the Gaussian Process withparameters, l = 6, k = 3 and q = 64. t x t , y t - - u t Figure 3:
Henon map. Evolution of the controlled system. The inset shows the applied perturbations. Inboth cases, the results shown were averaged for 100 initial random conditions.
The Lorenz system is the typical example of the continuous chaotic dynamical system repre-sented by the set of ordinary differential equations. It has an unstable fixed point at (8 . , . , x ( t ) = σ ( y ( t ) − x ( t )) , ˙ y ( t ) = − x ( t ) z ( t ) + rx ( t ) − y ( t ) + u t , ˙ z ( t ) = x ( t ) y ( t ) − bz ( t ) . Here u t = u ( x ( t ) , y ( t ) , z ( t )) and 8 × data points are used to to train the Gaussian Processwith parameters, l = 8, k = 4 and q = 128. The systems composed of parts that have some degree of autonomy and interact, are frequent ininnumerable situations and in areas from basic to applied science, so they must be an obligatoryexample of control object. Here, without loss of generality we use the logistic map, f ( s it ) =
50 100 150 200 250 300051015202530 t x t , y t , z t u t Figure 4:
Lorenz system. Evolution of the controlled system. The inset shows the applied perturbations.In both cases, the results shown were averaged for 100 initial random conditions. rs it (1 − s it ), as the nonlinear local dynamic to construct a generic complex dynamical networkconsisting of M identical nodes, s it +1 = f ( s it ) − (cid:15)k i M (cid:88) j =1 L ij f ( s jt ) , (10)where L = ( L ij ) Mij =1 is the usual (symmetric) Laplacian matrix with diagonal entries L ii = k i and k i the out degree of the node i . Thus, the network (10) can be represented as s t +1 = F ( s t ) = f ( s t ) − (cid:15) C f ( s t ) , (11)where the bolt font are vectors, C = ( L ij /k i )) Nij =1 is an asymmetric matrix with real eigenvalues[18]and (cid:15) a real parameter. Thus, the network can be represented as s t +1 = Af ( s t ), with A = I − (cid:15) C .This system have a unstable and homogeneous fixed point given by s ∗ = ( s ∗ , s ∗ , . . . , s ∗ N ). In ourcase M = 4, (cid:15) = 0 .
05 and A = .
95 0 .
025 0 .
025 0 . .
025 0 .
95 0 . . .
025 0 . .
95 0 . . .
025 0 .
025 0 . . (12)Although the symbolic dynamics of this network can be constructed in a way similar to theone-dimensional case, here we will only present results for the data-dependent control strategy.The following figure shows evolution of the controlled system a a function of the time.In this example, the controlled system is written as: s it +1 = f ( s it ) − (cid:15)k i M (cid:88) j =1 L ij f ( s jt ) + u it , (13)where u it = u i ( s t ).Figures 5 and 6 shows the structure of the network used as example in this article andthe evolution of the controlled network, respectively. The control functions u it was generated bytraining the Gaussian Process with 10 data points with parameters l = 4, k = 3 and q = 4 × .In spite of the results shown in Figure 6, refer to a network with the topology of the networkin Figure 5, the performance of the control scheme is similar for many networks with the samenumber of units. Numerical experiments suggest that, as this number increases the amountof data needed to obtain good approximations of u quickly grows so it would be necessary, inthe case of large networks, to combine the control strategy with a numerical methodology to
23 4
Figure 5:
Coupled chaotic maps. t s i t u t Figure 6: Coupled maps network. Evolution of the controlled system. Continuous black, dashedblack, continuous gray and dashed gray lines, shows the evolution of the four nodes. The inset showsthe applied perturbations. approximate homoclinic orbits. This is out of the scope of this work and it is currently underinvestigation.
A methodology for control of complex systems based on the estimation of the control function,from the observation of the system and using a machine learning technique, was presented.Although in this particular case we have used the regression method based on Gaussian pro-cesses, the methodology allows us to use the regression technique that best suits the particularproblem.Despite the existence of other control methods that use artificial intelligence strategies, theproposed method is novel in the sense that it directly approximates the control signal and doesso with very little computational effort.Finally, results shows that the use machine learning techniques and numerical approximationof homoclinic orbits can be useful in the design of control techniques of complex systems. eferences [1] Q. Din, Complexity and chaos control in a discrete-time prey-predator model, Communica-tions in Nonlinear Science and Numerical Simulation, 44 (2017) 113–134.[2] R. K. Upadhyay, N. Bairagi, K Kundu and J. Chattopadhyay, Chaos in eco-epidemiologicalproblem of the Salton Sea and its possible control, Applied Mathematics and Computation,196, 1 (2008), 392–401.[3] A. Singh, S. Gakkhar, Controlling chaos in a food chain model, Mathematics and Computersin Simulation, 115 (2015) 24–36.[4] Y. Zhao, J. Sun and M. Small, Evidence consistent with deterministic chaos in human cardiacdata: surrogate and nonlinear dynamical modeling, International Journal of Bifurcation andChaos, 18, 1 (2008) 141–160.[5] B. Hu, Q. Wang, Controlling absence seizures by deep brain stimulus applied on substantianigra pars reticulata and cortex, Chaos, Solitons and Fractals 80 (2015) 13–23.[6] D. Ghosh, A. Mukherjee, N. Ranjan Das, B. Nath Biswas, Generation and control of chaosin a single loop optoelectronic oscillator, Optik, 165 (2018) 275–287.[7] C. Xu, Y. Wu, Bifurcation and control of chaos in a chemical system, Applied MathematicalModelling, 39, 8 (2015) 2295–2310.[8] P. R. L. Alves, L. G. S. Duarte and L. A. C. P. da Mota, Detecting chaos and predicting inDow Jones Index, Chaos, Solitons and Fractals 110 (2018) 232238.[9] S. Mukherjee, R. Ray, R. Samanta, M. H. Khondekar and G. Sanyal, Nonlinearity and chaosin wireless network traffic, Chaos, Solitons and Fractals 96 (2017) 2329.[10] Z. Elhadj, Models and applications of chaos theory in modern sciences, CRC Press, Taylorand Francis (2011).[11] L. Rosier, G. Millrioux, G. Bloch, Chaos synchronization on the N-torus and cryptography,Comptes Rendus Mcanique, 332, 12 (2004) 969–972.[12] E. Ott, C. Grebogi, and J. A. Yorke, Controlling chaos, Phys. Rev. Lett. 64 (1990) 1196–1199.[13] A. Souza de Paula and M. Amorim Savi, Comparative analysis of chaos control methods:A mechanical system case study, International Journal of Non-Linear Mechanics, 46 (2011)10761089.[14] K. Pyragas, Continuous control of chaos by self-controlling feedback, Physics Letters A,170, 6 (1992) 421-428.[15] C. E. Rasmussen and C. K. I., Williams, Gaussian Processes for Machine Learning. TheMIT Press, Cambridge, Massachusetts 2006.[16] L. Block, Homoclinic points of mappings of the interval, Proceedings of the AmericanMathematical Society, 72, 3 (1978) 576–580.[17] Hao Bai-Lin, Elementary Symbolic Dynamics and Chaos in Dissipative Systems, WorldScientific (1989).[18] L. Y. Xiang, Z. X. Liu, Z. Q. Chen, F. Chen and Z. Z.Yuan, Pinning control of complexdynamical networks with general topology, Physica A, 379 (2007) 298–306.[19] V. Avrutin, B. Schenke and L. Gardini, Calculation of homoclinic and heteroclinic orbitsin 1D maps, Commun Nonlinear Sci Numer Simulat, 22, 13, (2015) 1201–1214.
20] W.-J. Beyn and J.-M. Kleinkauf, The numerical compuation of homoclinic orbits for maps,Siam J. Numer. Anal. 34, 3 (1997) 1207–1236.[21] V. R. Korostyshevskiy, T. Wanner, A Hermite spectral method for the computation ofhomoclinic orbits and associated functionals, Journal of Computational and Applied Math-ematics 206, 2 (2007) , 986–1006.20] W.-J. Beyn and J.-M. Kleinkauf, The numerical compuation of homoclinic orbits for maps,Siam J. Numer. Anal. 34, 3 (1997) 1207–1236.[21] V. R. Korostyshevskiy, T. Wanner, A Hermite spectral method for the computation ofhomoclinic orbits and associated functionals, Journal of Computational and Applied Math-ematics 206, 2 (2007) , 986–1006.