Cross and joint ordinal partition transition networks for multivariate time series analysis
CCross and joint ordinal partition transition networks for multivariate time seriesanalysis
Heng Guo a , Jiayang Zhang a , Yong Zou a, ∗∗ , Shuguang Guan a, ∗∗ a Department of Physics, East China Normal University, Shanghai, 200062, China
Abstract
We propose to construct cross and joint ordinal pattern transition networks from multivariate time series for two cou-pled systems, where synchronizations are often present. In particular, we focus on phase synchronization, which isone of prototypical scenarios in dynamical systems. We systematically show that cross and joint ordinal patternstransition networks are sensitive to phase synchronization. Furthermore, we find that some particular missing ordinalpatterns play crucial roles in forming the detailed structures in the parameter space whereas the calculations of permu-tation entropy measures often do not. We conclude that cross and joint ordinal partition transition network approachesprovide complementary insights to the traditional symbolic analysis of synchronization transitions.
Keywords: nonlinear time series analysis, complex networks, ordinal pattern partition, transition network, phasesynchronization
1. Introduction
Complex network theory provides an importantparadigm for understanding the structural properties ofcomplex systems composed of different interacting en-tities. During the last decade, various complex networkapproaches have been proposed to extract useful in-sights from time series data [1]. Depending on slightlydifferent definitions of nodes and links of network rep-resentations for time series, there are methods based onrecurrences [2, 3], visibility conditions [4], cycles detec-tions [5] and correlation networks [6]. Some successfulapplications of these methods include time series fromclimate, sunspots, oil-water flows, and financial mar-ket [1, 7–16]. The consideration of a time series as acomplex network allows a reinterpretation of many net-work theoretic measures in terms of characteristic phasespace properties of a dynamical system [17].Recently, a growing number of works are focused ontransforming time series into networks by ordinal parti-tions of time series [18–22]. The basic idea of ordinalpartition network method can be traced back to identi-fying ordinal patterns of time series [23–25]. Given a ∗ Corresponding author ∗∗ Corresponding author
Email addresses: [email protected] (Yong Zou), [email protected] (Shuguang Guan) one-dimensional time series { x ( t ) } t =1 , ··· ,L comprisingof L points from a dynamical system, we first recon-struct its phase space by time delay embedding tech-nique which yields (cid:126)x ( t ) = [ x ( t ) , x ( t + τ ) , · · · , x ( t +( D x − τ )] , where D x and τ are embedding dimen-sion and delay respectively [26, 27]. The next stepis to compute the rank order of each embedded vector [ x ( t ) , x ( t + τ ) , · · · , x ( t + ( D x − τ )] , which is conve-niently denoted by a symbol π x ( t ) . When sliding win-dows from t = 1 to N = L − ( D x − τ in the embed-ded space, a symbolic representation of the trajectory π x ( t ) is produced. Following the symbolic representa-tion, one traditional approach is to compute permuta-tion entropy H O based on the frequency plot of orderpatterns. Generally speaking, for a time series gener-ated by a stochastic process for N → ∞ , it is knownthat all D x ! patterns almost occur with equal probabili-ties, which yields the maximal value of H O . However,for a time series produced by deterministic dynamics,the frequency plot of the D x ! patterns is not uniform,which leads to reduced value of H O . In some cases, aset of patterns may never occur and these missing pat-terns are often called forbidden patterns, which provideimportant information for quantifying determinism intime series data [28–32]. Then, the level of determin-ism of a time series from a particular dynamical systemmay be suggested by permutation entropy H O , which Preprint submitted to Frontiers of Physics June 6, 2018 a r X i v : . [ n li n . C D ] J un onsists in very well established statistical measures innonlinear time series analysis [23, 25, 33]. Some appli-cations include characterizations of the difference be-tween healthy and patients from EEG data [18, 19]. It isworth noting that complications may arise in real timeanalysis because missing ordinal patterns might be re-lated to finite time length during the period of observa-tion and correlated stochastic processes, which requiresome revised methods for the detection of determinismin relatively short noisy data [28–32].The primary idea of ordinal partition transition net-work takes into account the inhomogeneous evolution-ary behavior among the ordinal patterns [18, 19], whichprovides complementary information on the standardordinal symbolic analysis of time series. Most of theseworks have focused on univariate time series { x ( t ) } .Embedding parameters D x and τ have crucial impactson the resulting ordinal partition transition networks,especially for forbidden patterns [20–22]. In addition,multivariate time series are ubiquitous in nature, rang-ing from stock markets and climate sciences. In a recentwork, we proposed to construct ordinal partition transi-tion networks from multivariate data [34]. The resultingnetwork is a directed and weighted network characteriz-ing the pattern transition properties of time series in itsassociated velocity space. This novel approach has beensuccessfully applied to capture phase coherence to non-phase coherence transitions and to characterize paths tophase synchronization, showing complementary insightto the traditional symbolic analysis of nonlinear time se-ries analysis.In this work, we further extend these ideas [34] toconstruct cross and joint ordinal partition transition net-works for two coupled systems. We show that bothcross and joint ordinal pattern transition networks areable to capture synchronization transitions, in particu-lar, focusing on the transitions to phase synchronization(PS) which is one of paradigmatic types in synchroniza-tion phenomena [35–40]. The outline of this paper isas follows: first, we illustrate cross and joint networkconstruction approaches and then introduce two entropymeasures to quantify the inhomogeneous frequencies ofordinal patterns and their transitions in Sec. 2. We applythese two entropy measures to characterize the synchro-nization transitions with both unidirectional and bidi-rectional coupling schemes in Sec. 3, and then someconclusions are drawn in Sec. 4.
2. Cross and joint ordinal partition transition net-works
The ordinal partition transition networks are illus-trated by the following coupled R¨ossler systems [41,42], which shows transition scenarios to phase synchro-nization for various coupling schemes. In particular, PShas been observed for both unidirectional and bidirec-tional couplings. Note that the network constructionmethod is not restricted by the following model. Theordinary differential equations (ODEs) of the coupledsystem read ˙ x = − (1 . − ∆) y − z + κ ( x − x ) , ˙ y = (1 . − ∆) x + 0 . y , ˙ z = 0 . z ( x − . , (1) ˙ x = − (1 . y − z + κ ( x − x ) , ˙ y = (1 . x + 0 . y , ˙ z = 0 . z ( x − . , (2)where ω = 1 . − ∆ , ω = 1 . are frequenciesfor the systems, and κ , are coupling strength. In thismodel, we distinguish a unidirectional coupling casefrom a bidirectional coupling one because the thresh-old values to PS have been reported in the literature[41, 42]. More specifically the unidirectional couplingcan be achieved by κ = 0 , namely, a drive-responsescheme [41] and bidirectional coupling is often cho-sen as κ = κ = κ [42]. We numerically integratethe ODEs using the fourth-order Runge-Kutta methodwith random initial conditions, and the integration step h = 0 . . The first transient data points are dis-carded and time series consisting of N = 800000 datapoints are analyzed. Ordinal Pattern Transition network.
We start theidea by constructing the ordinal pattern transition net-work for a single system, using the first system (Eqs.(1), ω = 1 . , ∆ = 0 , κ , = 0 ) as an example [34].Given time series ( x ( t ) , y ( t ) , z ( t )) in the correspond-ing three dimensional phase space ( n = 3 ), the or-dinal pattern transition network is reconstructed basedon the signs of the increments of each each variablee (∆ x ( t ) , ∆ y ( t ) , ∆ z ( t )) , where ∆ x ( t ) = x ( t +1) − x ( t ) , ∆ y ( t ) = y ( t + 1) − y ( t ) , and ∆ z ( t ) = z ( t + 1) − z ( t ) .In particular, the definitions of the ordinal patterns basedon the increment series capture the variations of the tra-jectory in its associated velocity space. The definition ofpatterns Π( t ) ∈ ( π , · · · , π i ) , i = 1 , · · · , are enumer-ated in Tab. 1. As time evolves, the transition behavior2 .167 π π (cid:652) (cid:21) π π π π π ( (cid:68) ) ( (cid:71) )( (cid:70) ) ( (cid:69) ) π π (cid:22) π (cid:21) π (cid:23) π (cid:27) π (cid:24) π (cid:26) π (cid:22) π (cid:26) π (cid:25) π (cid:24) π (cid:27) π (cid:23) π (cid:21) π (cid:20) π (cid:21) π (cid:24) π (cid:25) π (cid:26) π (cid:20) π (cid:22) π (cid:25) π (cid:27) π (cid:25) Figure 1:
Ordinal pattern transition network, where the thick-ness (and values) on network links represent the correspond-ing transition frequency between two patterns. (a) A sin-gle chaotic R¨ossler system. (b) Cross ordinal pattern transi-tion network (COPT) for two coupled R¨ossler systems, (c)an alternative version of COPT, and (d) joint ordinal patterntransition network (JOPT). For (b)-(d), the directions of linkshave been suppressed for better visualizations. In addition,the coupling strength κ , is in the non-sync regime, namely, κ = 0 , κ = 0 . . between patterns are illustrated in Fig. 1(a). The deter-ministic transitions are explained by the ordinal parti-tions of phase space by null-clines. A transition betweentwo patterns means the trajectory crosses a null-cline,which leads to a local maximum or minimum. Becauseof the continuity of the system in phase space, we onlyobserve the transition route π → π → π → π → π → π → π , yielding two missing patterns π and π as shown in Fig. 1(a). These details have been wellillustrated in [34]. Π π π π π π π π π ∆ x + + + + − − − − ∆ y + + − − + + − − ∆ z + − + − + − + − Table 1: Definitions of ordinal patterns of the three dimensional timeseries ( x ( t ) , y ( t ) , z ( t )) , where ∆ x = x ( t + 1) − x ( t ) , ∆ y = y ( t +1) − y ( t ) , and ∆ z = z ( t + 1) − z ( t ) [34]. Generalizing the above ideas from a single sys- tem to two coupled systems ( κ , (cid:54) = 0) , wepropose two different ways to construct ordi-nal pattern transition networks. Suppose wehave time series ( x ( t ) , y ( t ) , z ( t )) of one sys-tem and ( x ( t ) , y ( t ) , z ( t )) of the other, wefirst compute the respective increments of thetwo systems as (∆ x ( t ) , ∆ y ( t ) , ∆ z ( t )) and (∆ x ( t ) , ∆ y ( t ) , ∆ z ( t )) . Note that the incrementseries capture the dynamic properties of time series inthe difference space and the signs of the each variablereflects either the increasing ( + ) or the decreasing( − ) trend. Next, we compare the two systems in thefollowing: Cross ordinal pattern transition network (COPT).
ACOPT compares the relative speeds between two sys-tems by the signs of (∆ x ( t ) − ∆ x ( t )) , (∆ y ( t ) − ∆ y ( t )) and (∆ z ( t ) − ∆ z ( t )) . The pattern definitionsof a COPT are shown in Tab. 2. An example of COPTis shown in Fig. 1(b). In this work, we only considerthe signs of the same variable from two coupled sys-tems. A further generalization of the pattern definitionsof a COPT is to consider the signs of cross-variables, forinstance, (∆ x ( t ) − ∆ y ( t )) and (∆ x ( t ) − ∆ z ( t )) ,respectively. Considering the effects of the different Π π π π π π π π π ∆ x − ∆ x + + + + − − − − ∆ y − ∆ y + + − − + + − − ∆ z − ∆ z + − + − + − + − Table 2: Pattern definitions of a COPT. Note that “ + ” means a posi-tive value while “ − ” is for a negative value. magnitudes of the three variables, we also compute an alternative COPT by replacing ∆ x ( t ) − ∆ x ( t ) by ∆ x ( t ) /x ( t ) − ∆ x ( t ) /x ( t ) , respectively, ∆ y ( t ) − ∆ y ( t ) by ∆ y ( t ) /y ( t ) − ∆ y ( t ) /y ( t ) , and ∆ z ( t ) − ∆ z ( t ) by ∆ z ( t ) /z ( t ) − ∆ z ( t ) /z ( t ) . An exam-ple of the alternative COPT is shown in Fig. 1(c).Comparing Fig. 1(b) to 1(c), the alternative COPT re-flects better the non-coherent transitions between ordi-nal patterns since the coupling strength is in the non-synchronization regime ( κ = 0 and κ = 0 . ). In thefollowing, we compute the alternative COPTs withoutdistinguishing these two slightly different versions. Joint ordinal pattern transition networks (JOPT).
AJOPT compares the relative speeds between two sys-tems by the signs of ∆ x ( t ) · ∆ x ( t ) , ∆ y ( t ) · ∆ y ( t ) and ∆ z ( t ) · ∆ z ( t ) and the pattern definitions of aJOPT are summarized in Tab. 3. An example of JOPTis shown in Fig. 1(d). In contrast to cross ordinal pat-terns, we notice that the joint ordinal patterns represent3 π π π π π π π π ∆ x · ∆ x + + + + − − − − ∆ y · ∆ y + + − − + + − − ∆ z · ∆ z + − + − + − + − Table 3: Pattern definitions of a JOPT. Note that “+” means a positivevalue while “ − ” is for a negative value. whether the respective variables of two systems showthe same trend of changes or not, regardless of the mag-nitudes of the respective variables.A direct quantitative comparison between a COPTand a JOPT seems not possible since we have differentdefinitions for patterns. Anyway, we present qualitativesimilarities when we show a COPT and a JOPT for twointeracting stochastic processes in Fig. 2. In particular,we consider two stationary processes X t and Y t , eachof which admits an autoregressive representation X t = p (cid:88) j =1 a j X t − j + p (cid:88) j =1 b j Y t − j + η t , (3) Y t = p (cid:88) j =1 c j X t − j + p (cid:88) j =1 d j Y t − j + η t , (4)where the noise terms η t , η t are uncorrelated, p is theorder, a j , b j , c j and d j are model coefficients. Theterms of b j and c j reflect the interacting strength be-tween X t and Y t and therefore if X t and Y t are indepen-dent, b j and c j are uniformly zero. Here we considera simple case of p = 2 , a j = 0 . , b j = 0 . , c j = 0 . ,and d j = 0 , when there is only unidirectional interac-tions from Y t to X t .In order to keep the same number of ordinal patternsas defined in Tabs. 2 and 3, we reconstruct a three di-mensional phase space for X t (respectively Y t ) usingtime delay embedding techniques ( τ = 1 ). Figure 2ashows the ordinal pattern transition network for a singlestochastic process where we find rather random transi-tion patterns in the resulting network. These randomtransitions between patterns have been observed in theCOPT and JOPT (Figs. 2b, c and d). When increas-ing the interacting strength term b j , one would expectsome reduced level of random pattern transitions in the X t process because of the unidirectional interactionsfrom the Y t process. The dependence on the interactionstrength requires further investigations, which is beyondthe topic of the current work. In the next step, we quantify the heterogeneous prop-erties of the resulting networks. For a deterministic sys- (cid:652) (cid:23) (cid:652) (cid:27) (cid:652) (cid:22) (cid:652) (cid:24) (cid:652) (cid:26) (cid:652) (cid:25) (cid:652) (cid:20) (cid:652) (cid:21) ( (cid:68) ) (cid:652) (cid:21) (cid:652) (cid:27) (cid:652) (cid:22) (cid:652) (cid:20) (cid:652) (cid:26) (cid:652) (cid:23) (cid:652) (cid:25) (cid:652) (cid:24) ( (cid:69) ) (cid:652) (cid:21) (cid:652) (cid:22) (cid:652) (cid:20) (cid:652) (cid:27) (cid:652) (cid:26) (cid:652) (cid:24) (cid:652) (cid:25) (cid:652) (cid:23) ( (cid:70) ) (cid:652) (cid:25) (cid:652) (cid:23) (cid:652) (cid:26) (cid:652) (cid:22) (cid:652) (cid:27) (cid:652) (cid:24) (cid:652) (cid:20) (cid:652) (cid:21) ( (cid:71) ) Figure 2:
Ordinal pattern transition network for (a) a singlestochastic process. (b) a COPT for two coupled stochasticprocesses, (c) an alternative version of COPT, and (d) a JOPT.For better visualization, the arrows of links are suppressed. tem, the frequencies of ordinal patterns are generallydifferent from each other and the existence of forbiddenpatterns is simply a special case in this regard. Tradi-tionally, permutation entropy H is introduced to char-acterize the inhomogeneous appearance of ordinal pat-terns as following H O = − n (cid:88) i =1 p ( π i ) log p ( π i ) , (5)where the sum runs over all D = 2 n permutations and n is the dimension of one system and p ( π i ) is the proba-bility of order pattern π i . We use the observation fre-quency F ( π i ) to estimate p ( π i ) and furthermore weuse log and hence the units of H O are bits. For a n -dimensional independent identical distributed stochasticprocess, one obtains the largest entropy H O = n sinceeach of D = 2 n ordinal patterns is expected to have thesame frequency.The computation of H O characterizes the differ-ent frequencies of order patterns and it has been welldemonstrated that the transition behavior between or-dinal patterns is not fully captured by H O [34]. Tothis end, we first indicate each directed link represent-ing the order pattern transitions in the resulting networkby its transition frequency w ij = p ( π i → π j ) , follow-ing the time iterations of the series. In order to em-4hasize the importance of non-self transitions betweenordinal patterns, self-loops have been removed as sug-gested [34]. Finally, we obtain a weighted directednetwork characterized by a weighted adjacency matrix W = { w ij } , i, j ∈ [1 , n ] . The matrix W fulfils thenormalization (cid:80) n i,j w ij = 1 . Here, based on W , theregularity of the order pattern transition properties isquantified by the Shannon entropy H T , which is H T = − n (cid:88) i,j =1 w ij log w ij , (6)where the sum runs over all possible n transitions. Ina full analogy to H O , for a n -dimensional independentidentical distributed stochastic process, one obtains thelargest entropy H T = 2 n .Both measures have been demonstrated to show thecapabilities to capture the different bifurcation transi-tion scenarios. In the examples of this work, we showthat H O and H T are sensitive to PS.
3. Detecting transitions to PS
In this section, traditional measures characterizing PSare briefly reviewed and more historical details can befound in [35, 43, 44]. PS is characterized by the phaselocking | mφ − nφ | < C , where m , n and C are con-stants and φ ( t ) and φ ( t ) are phases of the two oscilla-tors. In the case of two coupled R¨ossler oscillators, weeasily compute phases by φ ( t ) = arctan y ( t ) /x ( t ) and φ ( t ) = arctan y ( t ) /x ( t ) since the systems arein phase coherent regimes. In addition, m and n are of-ten chosen as because phase synchronization ismore often observed. Equivalently, the phase locking ischaracterized as the average frequency mismatch dropsto zero, namely, ∆Ω = 0 , where ∆Ω = Ω − Ω and Ω , = π (cid:68) dφ , ( t ) dt (cid:69) , where (cid:104)·(cid:105) is an average over time T . PS has been related to the spectrum of Lyapunovexponents [45] and it is characterized by the transitionof the Lyapunov exponent from zero to negative values[35]. In the coupled R¨ossler systems (Eqs. 1), PS canbe achieved by applying the unidirectional couplingscheme κ = 0 and relative small frequency mismatch ∆ between ω and ω [41]. More specifically, wechoose ∆ = 0 . and ( ω , ω ) = (0 . , . . Notethat in this model, generalized synchronization is ob-tained for relative large values of ∆ = 0 . which leads Figure 3:
Network measures of COPTs versus the couplingstrength κ for two unidirectionally coupled R¨ossler systems( κ = 0 , ∆ = 0 . , namely, ω = 0 . and ω = 1 . inEqs. 1. (a) frequencies of ordinal patterns F ( π i ) while in-creasing the coupling strength κ . (b) H O and H T . (c) Theaverage rotation frequencies Ω and Ω . In all cases, verticaldashed highlight the critical synchronization transition thresh-olds κ = 0 . [41]. Superscripts C denote COPTs. to ( ω , ω ) = (0 . , . . The interrelationship betweenPS and generalized synchronization has been systemat-ically investigated in [41].We first shown in Fig. 3 that the variations of networkmeasures of COPTs depending on the coupling param-eters κ ( κ = 0 ). When increasing the coupling κ ,frequencies of ordinal patterns F ( π i ) experience rela-tive large variations (Figs. 3(a)). In consequence, both H T and H O show fast decays to small values when thecoupling threshold κ passing the transition values (ashighlighted by vertical dashed lines in Fig. 3(b)). Thetransition point to PS has been validated by the averagerotation frequencies of Ω and Ω since they are lockedto the same value at κ = 0 . . We choose three rep-resentative coupling values κ = 0 . , . , and . to show the structural variations of the correspondingCOPTs in Fig. 5(a-c). In the non-synchrony regime( κ = 0 . ), the transitions between ordinal patternsand their frequencies are rather random (Fig. 5(a)). Asthe coupling increases to just above the critical value κ = 0 . , a dominant (more deterministic) transitionroute emerges (Fig. 5(b)). When PS is achieved forlarge κ = 0 . , more missing patterns have been ob-served (Fig. 5(c)). We note that the different pattern def-initions are used between the ordinal transition networkfor a single chaotic R¨ossler system as shown in Fig. 1(a)5 igure 4: Network measures of JOPTs versus the couplingstrength κ for two unidirectionally coupled R¨ossler systems( κ = 0 ). (a) Frequencies of ordinal patterns while increas-ing the coupling strength. (b) H O and H T . Superscripts J denoted JOPTs. and the COPT for two coupled systems (Fig. 5(c)). Theunidirectional coupling leads to the entrainment of theresponse to the drive system, which yields the same or-dinal partitions for phase space as the drive. Due to thecontinuity of phase space trajectory, we observe a sim-ilar deterministic transition route between phase spacepartitions as for a single system.Concerning network measures from JOPTs, we haveobtained rather similar results as shown in Fig. 4. Threerepresentative JOPTs on the route to PS are illustratedin Fig. 5(d-f). As the coupling strength increases, therandom transitions between patterns become more de-terministic yielding more missing patterns.Based on the results of Figs. 3, 4 and 5, we con-clude that H O and H T can effectively capture the tran-sitions to PS. However, the detailed variations of par-ticular patterns remain unclear during the transitions toPS. In other words, we have to look into the details ofthe appearance or disappearance of each ordinal pat-terns (as shown in Figs. 3(a) and 4(a)) because some ofthe patterns become suppressed, which provide furtherevidence in characterizing the synchronization dynam-ics.It is easy to understand the occurrences of missingpatterns on the transition routes to synchronization. Asthe coupling strength is increased during the synchro-nization transitions, the high dimensional coupled sys-tems are constrained to a lower dimensional synchro-nization manifold which means that the determinism ofthe system is increased. In consequence, both the fre-quencies of order patterns and the transitions betweendifferent patterns become more inhomogeneous. There-fore, we compute the missing probability (frequency) of each pattern while varying the coupling strengths. In this section, we implement bidirectional couplingscheme when choosing κ = κ = κ . In addi-tion, we show the transition to synchronization in atwo dimensional parameter space of coupling strength κ ∈ [0 . , . and natural frequency mismatch ∆ ∈ [ − . , . . We choose this particular range of pa-rameters because there are different routes of chaos-chaos, chaos-period-chaos transitions to synchroniza-tions [42]. This space (∆ , κ ) is further divided into × grid points by equal step size. For eachparameter combination, we integrate the ODEs with thesame strategy as described in Sec. 2.1 to obtain timeseries for the coupled system. Then, we construct bothCOPTs and JOPTs.First, we show the average frequency mismatches ∆Ω in the parameter space (∆ , κ ) and PS is character-ized by the well-known Arnold tongue [35], as shownin Fig. 6(a). PS are observed inside this Arnold tongue,while no PS outside this region. We further validatethese results by computing the Lyapunov spectrum ofthe whole system based on Eqs. 1 (Fig. 6(b)). Wenote that it may not be possible to distinguish the tip ofthe Arnold tongue only by considering the sum of posi-tive Lyapunov exponents because of the non-hyperbolicproperty of the coupled -dimensional system. One in-teresting structure inside the Arnold tongue is the peri-odic region close to . < κ < . (like two eyes),which has been reported in [42]. It is often claimed thatall transitions between different types of synchroniza-tion are related to the changes in the Lyapunov spec-trum [45]. In particular, for low values of the couplingstrength, one has the following configuration for thecoupled -dimension system: { λ > , λ > , λ ∼ , λ ∼ , λ < , λ < } . Increasing the couplingstrength, PS is achieved when λ becomes negative. Ifthe coupling strength is further increased, generalizedsynchronization is obtained as λ ∼ and λ < .However, for intermediate coupling strengths and rela-tive large frequency mismatches, the spectrum of Lya-punov exponents has limited power in explaining theweak correlations outside the Arnold tongue, althoughthe phases are not locked [42].Figures 6(c-f) show the parameter space color codedby the entropy values H O and H T , which are calcu-lated from the respective COPTs and JOPTs. The outerboarders of the Arnold tongue have been successfullycaptured by H O and H T , comparing to the average fre-quency mismatch plot (Fig. 6(a)). More importantly,all these entropy values are able to capture the tip of6 igure 5: Network illustrations for COPTs (a-c) and JOPTs (e-f). (a, d) κ = 0 . , (b, e) κ = 0 . , and (c, f) κ = 0 . .Directions of pattern transitions and the corresponding frequencies are indicated by arrows and the values on links. . < κ < . except for the case of H T computed from COPTs.This is because H O and H T are averaged over all pat-terns.In order to illustrate more clearly the gradient struc-tures inside and on the edge of the Arnold tongue, weplot the missing (failure) probabilities of individual or-dinal patterns in the space (∆ , κ ) . For one combinationof parameter pair (∆ , κ ) , we run n = 200 simulationsof random initial conditions. The COPT and JOPT havebeen reconstructed for each realization of N = 800000 time points and the frequency plots of ordinal patternshave been obtained. Furthermore, this ensemble of in-dependent realizations results in a failure with probabil-ity p π i = 1 if pattern π i is missing and a success withprobability p π i = 0 if π i is observed, respectively.In the case of COPTs, Fig. 7 shows that π , π , π , π , π and π are observed patterns for theentire range of parameters we considered. In addition,we find that pattern π is completely not observed in theArnold tongue and the disappearance of π determinessharply the outer boarder. In addition, pattern π playsa key role in forming the periodic eye structures insidethe Arnold tongue. In the case of JOPTs, Fig. 8 showsthat patterns π , π , π and π do appear in the majorityarea of the parameter space. Again, the disappearanceof patterns π and π determine sharply the outer board-ers of the Arnold tongue, while the inside structures aredetermined by π and π .
4. Conclusions
In summary, we propose to construct cross and jointordinal pattern transition network from multivariatetime series, which has been particularly applied to ana-lyze synchronization transitions. Note that a COPT andJOPT are two slightly different ways to construct net-works from time series, providing complementary in-formation. The ordinal patterns of a COPT are definedby considering the signs of the difference of ∆ (cid:126)x − ∆ (cid:126)x between two subsystems. In contrast, the ordinal pat-terns of a JOPT are defined by the signs of the productof ∆ (cid:126)x · ∆ (cid:126)x . It is certain that the amplitudes of oscil-lations of different variables influence directly the defi-nition of a COPT. However amplitudes become not im- portant for a JOPT because only the signs of the productare considered. In addition, it is straightforward to gen-eralize the ideas of JOPTs from two to three (or even n )coupled subsystems with an extended number of patterndefinitions. We plan to further generalize the ideas ofJOPTs to multilayer or multiplex networks for time se-ries analysis. However, it remains to be a big challengefor constructing a COPT for three coupled subsystems.Based on the cross and joint ordinal pattern tran-sition networks, we propose two entropy measures tocharacterize the resulting networks. Namely, H O iscalculated from the frequencies of each patterns and H T is obtained from the transition frequencies betweenany pair of patterns. Our results show that both H O and H T track successfully the critical coupling thresh-old to phase synchronization. The applications of ourmethod to generalization synchronization analysis is amore challenging task and will be a subject for futurework [43].In the two parameter space of (∆ , κ ) , both entropymeasures capture the tip of the Arnold tongue success-fully, providing complementary information to the tra-ditional measure of Lyapunov exponents. In order toshow the intricate structures inside the Arnold tongue,our results suggest that we should study the missingprobability of each pattern separately, instead of relyingon the global measures of H O and H T . This is becauseindividual pattern shows different sensitivity to dynamictransitions. In particular, we have observed particularmissing patterns which correspond to the outer board-ers and the inner structures in the parameter space.In addition to synchronization analysis based on realtime series, it remains to be an interesting topic to iden-tify the driver-response relationship, especially to iden-tify indirect from direct coupling directions [46–49].From the viewpoint of ordinal pattern perspective, it ispossible to combine ordinal recurrence plots [50] andcross and joint ordinal partition transition network ap-proaches to tackle this problem. More importantly, weplan to address the statistical significance of the cou-pling directions for real time series data.
5. Acknowledgement
This work is in part financially sponsored by Nat-ural Science Foundation of Shanghai (Grants No.17ZR1444800 and 18ZR1411800).
References [1] R. V. Donner, M. Small, J. F. Donges, N. Marwan, Y. Zou,R. Xiang, J. Kurths, Recurrence-based time series analy- igure 6: (color online) Two parameter space (frequency mismatches ∆ and coupling strength κ = κ = κ ), which is color codedby the network statistics obtained from COPTs and JOPTs. (a) the difference of the mean frequencies ∆Ω = Ω − Ω , (b) thesum of non-negative Lyapunov exponents of the coupled system. COPTs: (c) H O , (d) H T , and JOPTs: (e) H O , (f) H T . Figure 7: (color online) Two parameter space (∆ , κ ) highlighted by the missing (failure) probability of individual pattern obtainedfrom COPTs. Note that π , π , π , π , π and π are observed for the entire range of parameters (white areas). Red regionscorrespond to the missing probabilities of patterns, which show that π captures mainly the outer boarders of the Arnold tongueand π captures the inside structures including two eye-like periodic regions. igure 8: (color online) Two parameter space (∆ , κ ) highlighted by the missing (failure) probability of individual pattern obtainedfrom JOPTs. Note that π , π , π and π are observed for the majority area of the parameter space (white areas). Red regionscorrespond to the missing probabilities of patterns showing that π and π capture the outer boarders, while π and π affect theinside structures of the Arnold tongue. sis by means of complex network methods, Int. J. Bifur-cation Chaos 21 (4) (2011) 1019–1046. doi:10.1142/S0218127411029021 .[2] R. V. Donner, Y. Zou, J. F. Donges, N. Marwan, J. Kurths,Recurrence networks – A novel paradigm for nonlinear timeseries analysis, New J. Phys. 12 (3) (2010) 033025. doi:10.1088/1367-2630/12/3/033025 .[3] N. Marwan, J. F. Donges, Y. Zou, R. V. Donner, J. Kurths, Com-plex network approach for recurrence analysis of time series,Phys. Lett. A 373 (46) (2009) 4246–4254. doi:10.1016/j.physleta.2009.09.042 .[4] L. Lacasa, B. Luque, F. Ballesteros, J. Luque, J. C. Nuno,From time series to complex networks: The visibility graph,Proc. Natl. Acad. Sci. USA 105 (13) (2008) 4972–4975. doi:10.1073/pnas.0709247105 .[5] J. Zhang, M. Small, Complex network from pseudoperiodictime series: Topology versus dynamics, Phys. Rev. Lett.96 (2006) 238701. doi:10.1103/PhysRevLett.96.238701 .[6] Y. Yang, H. Yang, Complex network-based time series analysis,Physica A 387 (5) (2008) 1381 – 1386. doi:10.1016/j.physa.2007.10.055 .[7] J. F. Donges, R. V. Donner, M. H. Trauth, N. Marwan, H. J.Schellnhuber, J. Kurths, Nonlinear detection of paleoclimate-variability transitions possibly related to human evolution, Proc.Natl. Acad. Sci. USA 108 (2011) 20422–20427. doi:10.1073/pnas.1117052108 .[8] Y. Zou, R. V. Donner, M. Wickramasinghe, I. Z. Kiss, M. Small,J. Kurths, Phase coherence and attractor geometry of chaoticelectrochemical oscillators, Chaos 22 (3) (2012) 033130. doi:10.1063/1.4747707 .[9] Z.-K. Gao, W.-D. Dang, Y.-X. Yang, Q. Cai, Multiplex multi-variate recurrence network from multi-channel signals for re-vealing oil-water spatial flow behavior, Chaos 27 (3) (2017) 035809. doi:10.1063/1.4977950 .[10] J. B. Elsner, T. H. Jagger, E. A. Fogarty, Visibility networkof united states hurricanes, Geophys. Res. Lett. 36 (16) (2009)L16702. doi:10.1029/2009GL039129 .[11] Y. Zou, M. Small, Z. Liu, J. Kurths, Complex network approachto characterize the statistical features of the sunspot series, NewJ. Phys. 16 (1) (2014) 013051. doi:10.1088/1367-2630/16/1/013051 .[12] Y. Zou, R. Donner, N. Marwan, M. Small, J. Kurths, Long-termchanges in the north-south asymmetry of solar activity: a non-linear dynamics characterization using visibility graphs, Nonlin.Proc. Geophys. 21 (6) (2014) 1113–1126. doi:10.5194/npg-21-1113-2014 .[13] R. Zhang, Y. Zou, J. Zhou, Z.-K. Gao, S. Guan, Visibilitygraph analysis for re-sampled time series from auto-regressivestochastic processes, Commun. Nonlinear Sci. Numer. Simulat.42 (2017) 396 – 403. doi:10.1016/j.cnsns.2016.04.031 .[14] Z. Czechowski, M. Lovallo, L. Telesca, Multifractal analysis ofvisibility graph-based ito-related connectivity time series, Chaos26 (2) (2016) 023118. doi:10.1063/1.4942582 .[15] C. Zhang, Y. Chen, G. Hu, Network reconstructions with par-tially available data, Frontiers of Physics 12 (3) (2017) 128906. doi:10.1007/s11467-017-0664-z .[16] Z.-Q. Jiang, Y.-H. Yang, G.-J. Wang, W.-X. Zhou, Jointmultifractal analysis based on wavelet leaders, Frontiersof Physics 12 (6) (2017) 128907. doi:10.1007/s11467-017-0674-x .[17] R. V. Donner, J. Heitzig, J. F. Donges, Y. Zou, N. Marwan,J. Kurths, The geometry of chaotic dynamics — a complexnetwork perspective, Eur. Phys. J. B 84 (4) (2011) 653–672. doi:10.1140/epjb/e2011-10899-1 .[18] M. McCullough, M. Small, T. Stemler, H. H.-C. Iu, Time laggedordinal partition networks for capturing dynamics of continuous ynamical systems, Chaos 25 (5) (2015) 053101. doi:10.1063/1.4919075 .[19] C. W. Kulp, J. M. Chobot, H. R. Freitas, G. D. Sprechini, Usingordinal partition transition networks to analyze ecg data, Chaos26 (7) (2016) 073114. doi:10.1063/1.4959537 .[20] C. W. Kulp, J. M. Chobot, B. J. Niskala, C. J. Needhammer,Using forbidden ordinal patterns to detect determinism in ir-regularly sampled time series, Chaos 26 (2) (2016) 023107. doi:10.1063/1.4941674 .[21] M. McCullough, K. Sakellariou, T. Stemler, M. Small, Count-ing forbidden patterns in irregularly sampled time series. i. theeffects of under-sampling, random depletion, and timing jitter,Chaos 26 (12) (2016) 123103. doi:10.1063/1.4968551 .[22] K. Sakellariou, M. McCullough, T. Stemler, M. Small, Countingforbidden patterns in irregularly sampled time series. ii. reliabil-ity in the presence of highly irregular sampling, Chaos 26 (12)(2016) 123104. doi:10.1063/1.4970483 .[23] C. Bandt, B. Pompe, Permutation entropy: A natural complexitymeasure for time series, Phys. Rev. Lett. 88 (17) (2002) 174102. doi:10.1103/PhysRevLett.88.174102 .[24] U. Parlitz, H. Suetani, S. Luther, Identification of equiva-lent dynamics using ordinal pattern distributions, Eur. Phys.J. S.T. 222 (2) (2013) 553–568. doi:10.1140/epjst/e2013-01859-2 .[25] J. M. Amig´o, K. Keller, V. A. Unakafova, Ordinal symbolicanalysis and its application to biomedical recordings, Phil.Trans. R. Soc. A 373 (2034) (2014) 20140091. doi:10.1098/rsta.2014.0091 .[26] F. Takens, Detecting strange attractors in turbulence, in:D. Rand, L.-S. Young (Eds.), Dynamical Systems and Turbu-lence, Warwick 1980, Vol. 898 of Lecture Notes in Mathemat-ics, Springer, New York, 1981, pp. 366–381. doi:10.1007/BFb0091924 .[27] H. Kantz, T. Schreiber, Nonlinear Time Series Analysis, 2ndEdition, Cambridge University Press, Cambridge, 2004.[28] J. M. Amig´o, S. Zambrano, M. A. F. Sanju´an, True andfalse forbidden patterns in deterministic and random dynam-ics, Europhys. Lett. 79 (5) (2007) 50001. doi:10.1209/0295-5075/79/50001 .[29] J. M. Amig´o, S. Zambrano, M. A. F. Sanju´an, Combinato-rial detection of determinism in noisy time series, Europhys.Lett. 83 (6) (2008) 60005. doi:10.1209/0295-5075/83/60005 .[30] O. A. Rosso, L. C. Carpi, P. M. Saco, M. G. Ravetti, H. A. Lar-rondo, A. Plastino, The Amig´o paradigm of forbidden/missingpatterns: a detailed analysis, Eur. Phys. J. B 85 (12) (2012) 419. doi:10.1140/epjb/e2012-30307-8 .[31] O. A. Rosso, L. C. Carpi, P. M. Saco, M. G. Ravetti, A. Plastino,H. A. Larrondo, Causality and the entropy-complexity plane:Robustness and missing ordinal patterns, Physica A 391 (12)(2012) 42 – 55. doi:10.1016/j.physa.2011.07.030 .[32] C. W. Kulp, L. Zunino, Discriminating chaotic and stochasticdynamics through the permutation spectrum test, Chaos 24 (3)(2014) 033116. doi:10.1063/1.4891179 .[33] A. Politi, Quantifying the dynamical complexity of chaotic timeseries, Phys. Rev. Lett. 118 (2017) 144101. doi:10.1103/PhysRevLett.118.144101 .[34] J. Zhang, J. Zhou, M. Tang, H. Guo, M. Small, Y. Zou, Con-structing ordinal partition transition networks from multivariatetime series, Scientific Reports 7 (2017) 7795. doi:10.1038/s41598-017-08245-x .[35] A. Pikovsky, M. Rosenblum, J. Kurths, Synchronization – AUniversal Concept in Nonlinear Sciences, Cambridge Univer-sity Press, 2001.[36] G. V. Osipov, B. Hu, C. Zhou, M. V. Ivanchenko, J. Kurths, Three types of transitions to phase synchronization in coupledchaotic oscillators, Phys. Rev. Lett. 91 (2) (2003) 024101. doi:10.1103/PhysRevLett.91.024101 .[37] J. Zhang, Y.-Z. Yu, X.-G. Wang, Synchronization of coupledmetronomes on two layers, Front. Phys. 12 (6) (2017) 120508. doi:10.1007/s11467-017-0675-9 .[38] H.-B. Chen, Y.-T. Sun, J. Gao, C. Xu, Z.-G. Zheng, Orderparameter analysis of synchronization transitions on star net-works, Front. Phys. 12 (6) (2017) 120504. doi:10.1007/s11467-017-0651-4 .[39] X. Huang, J. Gao, Y.-T. Sun, Z.-G. Zheng, C. Xu, Effects of frus-tration on explosive synchronization, Front. Phys. 11 (6) (2016)110504. doi:10.1007/s11467-016-0597-y .[40] L.-M. Ying, J. Zhou, M. Tang, S.-G. Guan, Y. Zou, Mean-fieldapproximations of fixation time distributions of evolutionarygame dynamics on graphs, Frontiers of Physics 13 (1) (2017)130201. doi:10.1007/s11467-017-0698-2 .[41] Z. Zheng, G. Hu, Generalized synchronization versus phasesynchronization, Phys. Rev. E 62 (2000) 7882–7885. doi:10.1103/PhysRevE.62.7882 .[42] M. C. Romano, M. Thiel, J. Kurths, W. von Bloh, Multivariaterecurrence plots, Phys. Lett. A 330 (3) (2004) 214 – 223. doi:10.1016/j.physleta.2004.07.066 .[43] L. M. Pecora, T. L. Carroll, Synchronization of chaotic systems,Chaos 25 (9) (2015) 097611. doi:10.1063/1.4917383 .[44] S. Boccaletti, J. Kurths, G. Osipov, D. Valladares, C. Zhou,The synchronization of chaotic systems, Physics Reports366 (1) (2002) 1 – 101. doi:10.1016/S0370-1573(02)00137-0 .[45] M. G. Rosenblum, A. S. Pikovsky, J. Kurths, From phase to lagsynchronization in coupled chaotic oscillators, Phys. Rev. Lett.78 (1997) 4193–4196. doi:10.1103/PhysRevLett.78.4193 .[46] M. G. Rosenblum, A. S. Pikovsky, Detecting direction of cou-pling in interacting oscillators, Phys. Rev. E 64 (4) (2001)045202. doi:10.1103/PhysRevE.64.045202 .[47] M. C. Romano, M. Thiel, J. Kurths, C. Grebogi, Estimation ofthe direction of the coupling by conditional probabilities of re-currence, Phys. Rev. E 76 (2007) 036211. doi:10.1103/PhysRevE.76.036211 .[48] J. Nawrath, M. C. Romano, M. Thiel, I. Z. Kiss, M. Wick-ramasinghe, J. Timmer, J. Kurths, B. Schelter, Distinguishingdirect from indirect interactions in oscillatory networks withmultiple time scales, Phys. Rev. Lett. 104 (3) (2010) 038701. doi:10.1103/PhysRevLett.104.038701 .[49] Y. Zou, M. C. Romano, M. Thiel, N. Marwan, J. Kurths, In-ferring indirect coupling by means of recurrences, Int. J. Bi-furcation Chaos 21 (04) (2011) 1099–1111. doi:10.1142/S0218127411029033 .[50] A. Groth, Visualization of coupling in time series by order re-currence plots, Phys. Rev. E 72 (4) (2005) 046220. doi:10.1103/PhysRevE.72.046220 ..