Adjoint-based variational method for constructing periodic orbits of high-dimensional chaotic systems
AADJOINT-BASED VARIATIONAL METHOD FOR CONSTRUCTINGPERIODIC ORBITS OF HIGH-DIMENSIONAL CHAOTIC SYSTEMS ∗ SAJJAD AZIMI † , OMID ASHTARI † , AND
TOBIAS M. SCHNEIDER †‡ Abstract.
Chaotic dynamics in systems ranging from low-dimensional nonlinear differentialequations to high-dimensional spatio-temporal systems including fluid turbulence is supported bynon-chaotic, exactly recurring time-periodic solutions of the governing equations. These unstableperiodic orbits capture key features of the turbulent dynamics and sufficiently large sets of orbitspromise a framework to predict the statistics of the chaotic flow. Computing periodic orbits for high-dimensional spatio-temporally chaotic systems remains challenging as known methods either showpoor convergence properties because they are based on time-marching of a chaotic system causingexponential error amplification; or they require constructing Jacobian matrices which is prohibitivelyexpensive. We propose a new matrix-free method that is unaffected by exponential error amplifi-cation, is globally convergent and can be applied to high-dimensional systems. The adjoint-basedvariational method constructs an initial value problem in the space of closed loops such that peri-odic orbits are attracting fixed points for the loop-dynamics. We introduce the method for generalautonomous systems. An implementation for the one-dimensional Kuramoto-Sivashinsky equationdemonstrates the robust convergence of periodic orbits underlying spatio-temporal chaos. Conver-gence does not require accurate initial guesses and is independent of the period of the respectiveorbit.
Key words. spatio-temporal chaos, unstable periodic orbits, adjoint methods, variational meth-ods, matrix-free numerical methods, Kuramoto-Sivashinsky, dynamical systems approach to turbu-lence
AMS subject classifications.
1. Introduction.
Ideas from low-dimensional chaotic dynamical systems haverecently led to new insights into high-dimensional spatio-temporally chaotic systemsincluding fluid turbulence. The idea for a dynamical description of turbulence has along history [42, 30, 19] and stems from the observation that turbulent flows often showrecognizable transient coherent patterns that recur over time and space [20]. Onlyin the last 15 years, however, has concrete progress allowed dynamical systems to betruly established as a new paradigm to study turbulence [24, 13, 22]. This progress isbased on the discovery of unstable non-chaotic steady and time-periodic solutions ofthe fully nonlinear Navier-Stokes equations which leads to a description of turbulenceas a walk through a connected forest of these dynamically connected invariant (‘exact’)solutions in the infinite-dimensional state space of the flow equations [16, 10, 41, 34].Of special importance are time-periodic exactly recurring flows. These so-calledunstable periodic orbits capture the evolving dynamics of the flow [21] and formthe elementary building blocks of the chaotic dynamics. Periodic orbits have beenrecognized as being key for understanding chaos since the 1880s [33, 35, 18]. Pro-vided results from low-dimensional hyperbolic dissipative systems carry over to high-dimensional spatio-temporally chaotic systems, periodic orbits lie dense in the chaoticset supporting turbulence. The turbulent trajectory thus almost always shadows aperiodic orbit. As a consequence, periodic orbit theory allows to express ergodic en-semble averages of the turbulent flow as weighted sums over periodic orbits. In these ∗ Submitted to the editors DATE.
Funding:
This work was supported by the Swiss National Science Foundation (SNSF) undergrant no. 200021-160088 † Emergent Complexity in Physical Systems Laboratory (ECPS), ´Ecole Polythechnique F´ed´eralede Lausanne, CH-1015 Lausanne, Switzerland (https://ecps.epfl.ch). ‡ tobias.schneider@epfl.ch 1 a r X i v : . [ n li n . C D ] J u l S. AZIMI, O. ASHTARI, AND T. M. SCHNEIDER ‘cycle expansions’, the statistical weight of an individual orbit is controlled by itsstability features [3, 7, 1, 2, 26, 8]. Sufficiently complete sets of periodic orbits forthree-dimensional fluid flows may thus eventually allow to quantitatively describe sta-tistical properties of turbulence in terms of exact invariant solutions of the underlyingflow equations [6]. Even if a full description of turbulence in terms of periodic orbitsremains beyond our reach, individual periodic orbits are of significant importance asthey capture key physical processes underlying the turbulent dynamics and may in-form control strategies [29]. Consequently, robust tools for computing periodic orbitsof high-dimensional spatio-temporally chaotic systems including three-dimensionalfluid flows are needed.High-dimensional spatio-temporal systems, including spectrally discretized three-dimensional fluid flow problems, are often characterized by more than N = 10 highlycoupled degrees of freedom. Computing periodic orbits of such high-dimensionalstrongly coupled systems remains computationally challenging. The commonly usedshooting method considers an initial value problem yielding trajectories satisfying theevolution equations and varies the initial condition until the solution closes on itself.To find the initial condition u and the period T , Newton iteration is used to numer-ically solve the nonlinear equation g ( u , T ) = f T ( u ) − u , where f T is the evolutionof the state u over time T . To solve this system of nonlinear coupled equations, astandard Newton method would require constructing the full Jacobian matrix with O ( N ) elements. This is practically impossible for high-dimensional strongly coupledsystems with large N . Key for computing periodic orbits of high-dimensional systemsare thus matrix-free Newton methods that do not construct the Jacobian matrix butonly require successive evaluations of the function g , implying time-stepping of theevolution equations. Commonly used algorithms are Krylov subspace methods [23, 38]including the Newton-GMRES-hook-step method by Viswanath [44, 45, 10] as well asslight variations with alternative trust-region optimizations [11, 12].The matrix-free Newton approach is well suited for computing fixed points, wherethe ‘period’ T can be chosen arbitrarily, but the Newton approach poses fundamen-tal challenges for periodic orbits. The defining property of a chaotic system is anexponential-in-time separation of trajectories which leads to a sensitive dependenceon initial conditions. Very small changes in the initial condition u are thus expo-nentially amplified by the required time-integration. Finding zeros of g thus becomesan ill-conditioned problem. Consequently, an extremely good initial guess is requiredfor the Newton method to converge. Generating sufficiently accurate initial guessesis very challenging and often impossible. Owing to the finite numerical precision ofdouble-precision arithmetic long and unstable orbits are even entirely impossible toconverge. Examples demonstrating the difficulty in finding periodic orbits of high-dimensional systems using shooting methods include the seminal work by Chandlerand Kerswell [6], who computed approximately 100 orbits for a two-dimensional modelflow and describe the time-consuming and tedious manual work to find initial guessesand trying to converge them. Likewise van Veen et al. [43] recently computed a singleperiodic orbit for box turbulence with only moderate resolution of 64 grid points.The authors reach a moderately small residual of 1 . · − and thus many orders ofmagnitude larger than machine precision only after “several months of computing onmodern GPU cards, due to the poor conditioning of the linear problems associatedwith Newtons method”. Consequently, more robust methods with larger radii of con-vergence than those of shooting methods are needed to compute periodic orbits ofhigh-dimensional spatio-temporally chaotic systems.For low-dimensional systems more robust methods for finding periodic orbits have DJOINT-BASED VARIATIONAL METHOD N = 64 Fourier modes to discretize the problem.Unfortunately, the robust variational method of Lan and Cvitanovi´c cannot bescaled to high-dimensional problems such as fluid turbulence. The method is notmatrix-free but requires the explicit construction of Jacobian matrices and their in-version. Moreover, accurate computations of tangents to the loop by finite differencesrequire the loop to be represented by a sufficiently large number of closely-spacedinstantaneous fields. The size of the Jacobian matrix to be inverted scales with thenumber of instantaneous fields M and the spatial degrees of freedom N as O ( M N ).This scaling reflects the prohibitively large memory requirements for high-dimensionalsystems. The only attempt to apply the method to a higher-dimensional system weare aware of is Fazendairo et al. [15, 4] who study forced box-turbulence in a triple-periodic box using Lattice-Boltzmann computations. They provide evidence for theconvergence of two periodic orbits but reaching a modestly small residual of O (10 − )on a relatively small 64 spatial lattice requires tens of thousands of CPU cores. Asstated by Fazendeiro et. al., even finding the shortest orbits of 3D flows using themethod by Lan and Cvitanovi´c requires petascale computing resources. Despite itsrobustness, the variational method by Lan and Cvitanovi´c is thus too computation-ally expensive to be realistically used for high-dimensional spatio-temporally chaoticsystems.Here we propose a novel matrix-free method that provides the same favorable con-vergence properties of the variational method by Lan and Cvitanovi´c [27, 28] but canbe applied to high-dimensional systems. The method combines a variational approachsimilar to Lan and Cvitanovi´c with an adjoint-based minimization technique inspiredby recent work of Farazmand [14] on computing steady state solutions. Combiningthe variational approach with adjoints allows us to construct an initial value prob-lem in the space of closed loops such that unstable periodic orbits become attractingfixed points of the dynamics in loop-space. Converging to a periodic orbit thus onlyrequires evolving an initial guess under the dynamics in loop-space. We develop thematrix-free adjoint-based variational method for general autonomous dynamical sys-tems. As a proof-of-concept, the introduced method is applied to the one-dimensionalKuramoto-Sivashinsky equation (KSE) [25, 39]. The KSE is a model system showingspatio-temporal chaos that has commonly been used as a sandbox model to developalgorithms that are eventually applied to three-dimensional fluid flows. We demon-strate the robust convergence of multiple periodic orbits of varying complexity andperiods. The implementation utilizes a spectral Fourier discretization in the temporaldirection to significantly reduce the prohibitively large memory requirements of themethod by Lan and Cvitanovi´c. S. AZIMI, O. ASHTARI, AND T. M. SCHNEIDER
The structure of the paper is as follows: First, the proposed method for comput-ing periodic orbits is introduced for a general autonomous system. Section 2 describesthe setup of the variational problem and section 3 discusses the adjoint-based mini-mization technique. In section 4, we apply the adjoint-based variational method tothe KSE and demonstrate the convergence of periodic orbits in this spatio-temporallychaotic system. Section 5 summarizes the manuscript and discusses future applica-tions to three-dimensional fluid turbulence.
2. Variational method for finding periodic orbits.
We consider a generaldynamical system for an n -dimensional real field (cid:126)u defined over a spatial domainΩ ⊂ R d and varying in time t , (cid:126)u : Ω × R → R n , ( (cid:126)x, t ) (cid:55)→ (cid:126)u ( (cid:126)x, t ) . The evolution of the field (cid:126)u is first-order in time and governed by an autonomouspartial differential equation (PDE) of the form(2.1) ∂(cid:126)u∂t = N ( (cid:126)u ) . The nonlinear differential operator N enforces boundary conditions at ∂ Ω, the bound-aries of the spatial domain Ω. A periodic orbit is a temporally periodic solution ofthe governing equation,(2.2) f T ( (cid:126)u ) − (cid:126)u = (cid:126) , where f T = (cid:82) t + Tt N dt (cid:48) indicates the nonlinear evolution over the period T .The shooting method considers solutions of the initial value problem and variesthe initial condition (cid:126)u ( (cid:126)x ) until the solution closes on itself and becomes periodic.Equation (2.2) is thus treated as an algebraic equation for the initial condition andthe period. An alternative approach is to consider already time-periodic fields andvary those until they satisfy the governing equations. Instead of identifying an initialcondition as in a shooting method, we consider the entire orbit as a solution of aboundary value problem in the ( d + 1)-dimensional space-time domain. To ensureperiodicity of the solution in time, the boundary conditions in space are augmentedby periodic boundary conditions in time. The field (cid:126)u ( (cid:126)x, t ) is thus defined on Ω × [0 , T ) periodic .The length of the domain in time T is unknown and needs to be determinedas part of the solution. To convert the problem to a boundary value problem on afixed domain, we rescale time t (cid:55)→ s := t/T , where s denotes the normalized timecoordinate. The rescaled field (cid:126) ˜ u ( (cid:126)x, s ) := (cid:126)u ( (cid:126)x, s · T ) , is defined on a fixed domain (cid:126) ˜ u : Ω × [0 , periodic → R n , ( (cid:126)x, s ) (cid:55)→ (cid:126) ˜ u ( (cid:126)x, s ) . A periodic orbit is characterized by the space-time field (cid:126) ˜ u ( (cid:126)x, s ) and the period T satisfying(2.3) − T ∂(cid:126) ˜ u∂s + N ( (cid:126) ˜ u ) = (cid:126) . DJOINT-BASED VARIATIONAL METHOD s . To simplify the notation, the overhead tilde is omitted in the remainder of thearticle.A periodic orbit is defined by the combination of a field (cid:126)u ( (cid:126)x, s ) and a period T that together satisfy the boundary value problem (2.3). Geometrically the periodicorbit is a closed trajectory in state space. To characterize general closed curves instate space, we define a loop l ( (cid:126)x, s ) as a tuple of a field (cid:126)u ( (cid:126)x, s ) and a period T . A loopdoes not necessarily satisfy the PDE of the boundary value problem (2.3) but sharesall boundary conditions in space and time with periodic orbits. We denote the spaceof all loops by P = (cid:40) l ( (cid:126)x, s ) = (cid:20) (cid:126)u ( (cid:126)x, s ) T (cid:21) (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) (cid:126)u : Ω × [0 , periodic → R n , T ∈ R + (cid:126)u satisfies BC at ∂ Ω and is periodic in s (cid:41) . (2.4)Periodic orbits are specific elements of the loop-space P that satisfy the PDE (2.3).A general loop only satisfies the boundary conditions but not the PDE.The idea of the variational method is to consider an initial loop l ( (cid:126)x, s ) ∈ P and to evolve the loop until it satisfies the boundary value problem (2.3). The loopthereby converges to a periodic orbit. To evolve a loop towards a periodic orbit weminimize the cost function J measuring the deviation of a loop from a solution of theboundary value problem, J : P → R + , l (cid:55)→ J ( l ) := (cid:90) (cid:90) Ω (cid:126)r · (cid:126)rd(cid:126)xds. (2.5)where (cid:126)r is the residual of Equation (2.3):(2.6) (cid:126)r = − T ∂(cid:126)u∂s + N ( (cid:126)u ) . The cost function J penalizes a nonzero residual (cid:126)r . For a periodic orbit J is zerootherwise it takes positive values. Thus, absolute minima of J correspond to periodicorbits. The problem of finding periodic orbits has thereby been converted into anoptimization over loop-space P .Geometrically, minimizing the cost function corresponds to deforming a closedcurve, a loop, in the system’s state space, the space spanned by all instantaneousfields (cid:126)u ( (cid:126)x ) satisfying the boundary conditions, until the loop becomes an integralcurve of the vector field N ( (cid:126)u ) induced by the dynamical system. The loop therebybecomes a solution of the PDE and represents a periodic orbit. This is schematicallyshown in Figure 1. At each point (cid:126)u along the loop, the vector field defines the flowdirection N ( (cid:126)u ) while ∂(cid:126)u/∂t = T − ∂(cid:126)u/∂s is the tangent vector along the loop (seepanel a ). The cost function J measures the misalignment between the vector field andthe loop’s tangent vectors integrated along the entire loop. Consequently, minimizing J towards its absolute minimum J = 0 deforms the loop until the tangent vectorseverywhere match the flow and the loop becomes an integral curve of the vector field,as exemplified in panel b . The loop is locally deformed to align with the vector fieldand no time-marching causing exponential instabilities is required.
3. Adjoint-based method for minimizing the cost function J . We recastthe problem of finding periodic orbits as a minimization problem in the space of
S. AZIMI, O. ASHTARI, AND T. M. SCHNEIDER ( a ) T ∂~u∂s N ( ~u ) s ( b ) J Fig. 1 . Schematic of the variational method for finding periodic orbits. (a) An arbitrary closedloop (blue line) parametrized by s ∈ [0 , does not satisfy the governing equations as its loop tangent ∂(cid:126)u/∂t = T − ∂(cid:126)u/∂s is misaligned relative to the vector field N ( (cid:126)u ) induced by the dynamical system.(b) Minimizing a cost function J measuring the misalignment between the vector field and the looptangent deforms the loop. When the global minimum of the cost function with J = 0 is reached thetangent vectors everywhere match the flow, ∂(cid:126)u/∂t = N ( (cid:126)u ) . The loop becomes an integral curve ofthe vector field and a periodic orbit is identified. all loops. Absolute minima of the cost function J with value J = 0 correspond toperiodic orbits. To minimize J without constructing Jacobians we develop an adjoint-based approach inspired by the recently introduced method by Farazmand [14] whocomputes equilibria of a two-dimensional flow. We construct an initial value problemin loop-space P whose dynamics monotonically decreases the cost function J until aminimum of J is reached.To derive an appropriate variational dynamics in loop-space, we define the spaceof generalized loops: P g = (cid:40) q ( (cid:126)x, s ) = (cid:20) (cid:126)q ( (cid:126)x, s ) q (cid:21) (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) (cid:126)q : Ω × [0 , periodic → R n , q ∈ R (cid:126)q is periodic in s (cid:41) . (3.1)Elements q ∈ P g do not necessarily satisfy the spatial boundary condition of periodicorbits at ∂ Ω and are thus termed generalized loops. Obviously, the space of loops P is a subset of the space of generalized loops P ⊂ P g . For a loop, the componentsof the generalized loop have specific meaning, (cid:126)q = (cid:126)u and q = T . Throughout thispaper, generalized loops are denoted by boldface letters. The space of generalized DJOINT-BASED VARIATIONAL METHOD P g carries a real-valued inner product (cid:104) , (cid:105) : P g × P g → R , (cid:104) q , q (cid:48) (cid:105) = (cid:28) (cid:20) (cid:126)q q (cid:21) , (cid:20) (cid:126)q (cid:48) q (cid:48) (cid:21) (cid:29) = (cid:90) (cid:90) Ω (cid:126)q · (cid:126)q (cid:48) d(cid:126)xds + q q (cid:48) , (3.2)and an L -norm(3.3) || q || = (cid:112) (cid:104) q , q (cid:105) = (cid:115)(cid:90) (cid:90) Ω (cid:126)q · (cid:126)q d(cid:126)xds + q . The objective is to construct a dynamical system in the space of loops P suchthat along its solutions the cost function J monotonically decreases and periodicorbits become attracting fixed points of the dynamical system. We parametrize theevolution of loops in P by a fictitious time τ : l ( τ ) = [ (cid:126)u ( (cid:126)x, s ; τ ); T ( τ )] and define anevolution equation,(3.4) ∂ l ∂τ = G ( l )with operator G chosen such that(3.5) ∂J∂τ ≤ ∀ τ. The rate of change of J along solutions of Equation (3.4) is (see Appendix A fordetails)(3.6) ∂J∂τ = 2 (cid:104) LLL ( l ; G ) , R (cid:105) . where R ∈ P g is a generalized loop(3.7) R ( l ) = (cid:20) (cid:126)r (cid:21) , with (cid:126)r ( l ) the residual field (2.6). LLL ( l ; G ) is the directional derivative of the residual R in the direction G , evaluated for the current loop l :(3.8) LLL ( l ; G ) = lim (cid:15) → R ( l + (cid:15) G ) − R ( l ) (cid:15) . Using the adjoint of the directional derivative, we express Equation (3.6) as(3.9) ∂J∂τ = 2 (cid:10) G , LLL † ( l ; R ) (cid:11) where LLL † is the adjoint operator of LLL with(3.10) (cid:104)
LLL ( q ; q (cid:48) ) , q (cid:48)(cid:48) (cid:105) = (cid:10) q (cid:48) , LLL † ( q ; q (cid:48)(cid:48) ) (cid:11) , for all generalized loops q , q (cid:48) and q (cid:48)(cid:48) . This form allows to enforce the monotonicdecrease of the cost function J by explicitly choosing the operator G :(3.11) G = − LLL † ( l ; R ) . S. AZIMI, O. ASHTARI, AND T. M. SCHNEIDER
With this choice for G , the cost function evolves as(3.12) ∂J∂τ = 2 (cid:10) − LLL † ( l ; R ) , LLL † ( l ; R ) (cid:11) = − (cid:12)(cid:12)(cid:12)(cid:12) LLL † ( l ; R ) (cid:12)(cid:12)(cid:12)(cid:12) ≤ . Thus, along solutions of ∂ l /∂τ = G ( l ) = − LLL † ( l ; R ) the cost function J is guaranteedto monotonically decrease.To find a periodic orbit using the adjoint approach, an initial loop is advancedunder the dynamical system in loop-space, until a minimum of the cost function,corresponding to an attracting fixed point with ∂ τ l = , is reached. If an absoluteminimum, J = 0, is reached, the loop satisfies the boundary value problem (2.3) andrepresents a periodic orbit. The cost function J is invariant under a reparametrization s (cid:55)→ s (cid:48) = ( s + σ ) mod 1 corresponding to a phase shift by σ in the temporal periodicdirection. Consequently, the phase of the minimizing loop is not chosen by the adjoint-based variational method but depends on the initial condition.
4. Application to Kuramoto-Sivashinsky equation.
We demonstrate theadjoint-based variational method for the one-dimensional Kuramoto-Sivashinsky equa-tion (KSE) [25, 39]. This nonlinear partial differential equation for a one-dimensionalfield u ( x, t ) on a 1D periodic interval x ∈ [0 , L ) = Ω reads(4.1) ∂u∂t = − u ∂u∂x − ∂ u∂x − ν ∂ u∂x ; x ∈ [0 , L ) periodic , t ∈ R with a constant ’superviscosity’ ν >
0. The KSE has the general form of Equa-tion (2.1) with n = d = 1. We denote the scalar spatial coordinate by x . Rescalingthe field u by the inverse of L indicates that the only control parameter is L = L/ √ ν the ratio of the domain length and the square-root of the constant ν . Consequently,fixing the domain length L and varying ν is equivalent to fixing ν and treating L asa control parameter. Both scalings are used in literature. Here, we fix ν = 1 andconsider L as the control parameter. The equivariance group of the KSE containscontinuous shifts in x and the discrete center symmetry,(4.2) x → − x ; u → − u. We discuss periodic orbits both in the full unconstrained space and in the subspaceof fields invariant under the discrete center symmetry.The trivial solution of the KSE, u = const, is linearly unstable for L > π √ ν [9].A series of bifurcations leads to increasingly complex dynamics when L is increased.We consider the parameter value L = 39 where the KSE shows spatio-temporallychaotic dynamics reminiscent of turbulence [40]. For the 1D-KSE a loop consists of a one-dimensional field u ( x, s ) defined over [0 , L ) × [0 ,
1) andthe period T . The residual of the boundary value problem for a periodic orbit (2.6),expressed as generalized loop R (see Equation (3.7)), is(4.3) R ( l ) = r ( l )0 = − T ∂u∂s − u ∂u∂x − ∂ u∂x − ∂ u∂x , where vector notation has been suppressed because the dimension of the field is n = 1. DJOINT-BASED VARIATIONAL METHOD R . Partial integration directly yields theadjoint operator for the KSE problem (see Appendix B),(4.4) LLL † ( l ; R ) = T ∂r∂s + u ∂r∂x − ∂ r∂x − ∂ r∂x (cid:90) (cid:90) L T ∂u∂s rdxds . Consequently, the dynamical system in loop-space ∂ l /∂τ = − LLL † ( l ; R ) (see (3.11))minimizing the cost function J is(4.5) ∂ l ∂τ = ∂u∂τ∂T∂τ = − T ∂r∂s − u ∂r∂x + ∂ r∂x + ∂ r∂x − (cid:90) (cid:90) L T ∂u∂s rdxds . The first component of Equation (4.5) prescribes the deformation of the field u ( x, s ),while the second component updates the period T .The dynamical system in loop-space formulated for the KSE, Equation (4.5), isequivariant with respect to the discrete symmetry:(4.6) Ξ : ( x, s ) → ( − x, s ) ; (cid:20) uT (cid:21) → (cid:20) − uT (cid:21) . If an initial loop is invariant under the action of Ξ, the evolution in τ will preservethe symmetry. Since the transformation of the instantaneous field x → − x ; u ( · , s ) →− u ( · , s ) for all s ∈ [0 ,
1) corresponds to the center-symmetry (4.2) of the KSE equa-tion, the dynamical system in loop-space also preserves the center symmetry of theKSE. An initial loop with field component within the center-symmetric subspace ofKSE is invariant under Ξ, which is preserved under τ -evolution. Consequently, theadjoint-based variational method preserves the discrete center-symmetry of the KSE. Expressing the field component of the dy-namical system (4.5) in terms of u using Equation (4.3) yields,(4.7) ∂u∂τ = G , L + G , NL , where the linear and nonlinear terms have the form, G , L = 1 T ∂ u∂s − ∂ u∂x − ∂ u∂x − ∂ u∂x G , NL = − ∂ u∂x ∂u∂x − ∂ u∂x ∂ u∂x − ∂ u∂x ∂u∂x + u ∂ u∂x + u (cid:18) ∂u∂x (cid:19) + 2 uT ∂ u∂x∂s + 1 T ∂u∂x ∂u∂s .
The field u ( x, s ) is defined on a doubly-periodic space-time domain. We thus numer-ically solve the evolution equation with a pseudospectral method [5] using a Fourier0 S. AZIMI, O. ASHTARI, AND T. M. SCHNEIDER discretization in both space and time. The spectral representation with M modes inspace and N modes along the temporal direction is, u ( x m , s n ) = M − (cid:88) j = − M N − (cid:88) k = − N ˆ u j,k exp (cid:26) πi (cid:18) mjM + nkN (cid:19)(cid:27) . (4.8)In physical space, the field is represented by grid values at the Gauss-Lobatto collo-cation points { u ( x m , s n ) } with ( x m , s n ) = ( mL/M, n/N ) and index ranges 0 ≤ m ≤ M − ≤ n ≤ N −
1. In spectral space, the set of discrete Fourier coefficients { ˆ u j,k } with − M/ ≤ j ≤ M/ − − N/ ≤ k ≤ N/ − ∂ ˆ u j,k ∂τ = (cid:34) − (cid:18) πkT (cid:19) − (cid:18) πjL (cid:19) + 2 (cid:18) πjL (cid:19) − (cid:18) πjL (cid:19) (cid:35) ˆ u j,k + ( ˆ G , NL ) j,k , where the discrete Fourier transform is indicated by a hat. To evaluate the nonlinearterm ˆ G , NL derivatives are calculated in spectral space and transformed to physicalspace, where products are pointwise operations. Transforming the result back tospectral space yields the required terms. In both the spatial and temporal directiondealiasing following the 2/3 rule [5] is applied. To advance the evolution equation(4.7) in the fictitious time τ we implement a semi-implicit time-stepping method. Animplicit-explicit Euler method treats the linear terms implicitly and the nonlinearterms ˆ G , NL are discretized explicitly.The second component of the evolution equation (4.5) evolves the period of theloop T . We use an explicit Euler method for time-stepping. The integral definingthe right-hand-side is evaluated analogous to the pseudo-spectral treatment of thenonlinear terms in the evolution equation of the field. The integrand is evaluatedin physical space followed by transformation to spectral space, where the integral isgiven by the (0 ,
0) Fourier mode multiplied by L .Since the purpose of defining the initial value problem in loop-space is to identifyattractors corresponding to solutions of the boundary value problem for periodic or-bits, stability and simplicity of the implementation are more important than accuracywhen choosing a time-stepping scheme. The simple Euler method is only first orderaccurate in τ but remains stable for the chosen fixed time step ∆ τ = 0 . The adjoint-basedvariational method advances some initial loop under the dynamical system that min-imizes the cost function J . If a minimum with J = 0 is reached the loop satisfiesthe boundary value problem for a periodic orbit. Initial guesses for the procedure areextracted from chaotic solutions of the KSE (4.1) u ( x, t ). The common approach forgenerating guesses used in conjunction with Newton-GMRES-based shooting meth-ods extracts close recurrences measured in terms of the L -distance from minima ofthe recurrence map c ( t, T ) = || u ( · , t + T ) − u ( · , t ) || [3]. Here, the L -norm is given by || u || ( t ) = (cid:115)(cid:90) L u ( x, t ) dx. (4.10)Exploiting the large radius of convergence of the variational method, we here choosea much simpler and computationally significantly cheaper method. Initial guesses DJOINT-BASED VARIATIONAL METHOD || u || ( t ) where || u || ( t + T ) ≈ || u || ( t ). The segment of the solution between those subsequent maxima yieldsthe field component of the initial loop. To ensure a smooth closed loop with fieldcomponent satisfying periodic boundary conditions in the temporal direction, thesolution segment is Fourier-transformed in time and high-frequency components arefiltered out [27]. The double-periodic field u ( x, s ) complemented by the period definesan initial guess l = [ u ( x, s ); T ].The initial guess l is evolved under the dynamical system in loop-space (3.4).Along the evolution the cost function J is guaranteed to monotonically decrease andreach a minimum. Consequently, the adjoint-based variational method is globallyconvergent. However, it is not guaranteed that an absolute minimum with J = 0 isreached but the dynamics may asymptote towards a local minimum with J >
0. Ifa global minimum is reached, a periodic orbit satisfying the boundary value problem(2.3) is found. We consider a periodic orbit converged, when √ J < − is achieved.The periodic orbit corresponds to an attracting fixed point of the dynamical systemin loop-space so that we expect exponential convergence at a rate controlled by theleading eigenvalue of the loop dynamics linearized around the attracting fixed point. We demonstrate the adjoint-based variationalmethod to construct periodic orbits of the KSE for the parameter value L = 39. Atthis value, the dynamics is chaotic and a large number of unstable periodic orbits areknown to exists [29]. Periodic orbits of the KSE are found by evolving initial loopsunder the dynamical system in loop-space (4.5). The pseudo-spectral method uses64 ×
64 Fourier modes in spatial and temporal directions to discretize the field u ( x, s ).A fixed time step of ∆ τ = 0 .
15 leads to stable time-stepping.Periodic orbits of the KSE are attracting solutions of an initial value problem inthe space of loops P that monotonically decreases the cost function J , as shown inFigure 2. In the top panel, the square root of the cost function, √ J , as a function ofthe fictitious time τ is shown. After τ ≈ . · the convergence criterion √ J ≤ − is reached. Since the cost function J is the average of (cid:82) Ω r dx over s , the square rootof J scales with the L -norm (4.10) of the residual field r and should be used asthe convergence criterion. Along the evolution of the loop with τ the cost function J monotonically decreases. After an initial fast decrease, √ J decays exponentiallywith τ . This suggests the convergence towards a periodic orbit along the leadingeigendirection of the dynamical system in loop-space linearized about the attractingfixed point. Geometrically, the dynamical system in loop-space (4.5) continuouslydeforms the initial loop until the loop satisfies the KSE and thereby becomes a periodicorbit. The deformation is visualized in the bottom panel, where the evolution of theloop shown in a two dimensional projection of the state space. A very substantialdeformation of the loop is associated with the fast decrease of J within the initial10% of the integration time.In addition to the two-dimensional field defined over the fixed space-time domain[0 , L ) × [0 , T is required to define a loop. Evolvinga loop towards a periodic orbit implies finding the period T , which re-scales thetemporal length of the space-time domain s → t = T · s and thereby determinesthe length of extension of the domain in the direction of time t . Figure 3 shows theconvergence of T to the period of the periodic orbit together with the space-timecontours of the corresponding initial loop u ( x, t = T · s ) and the converged periodicorbit u ( x, t = T · s ). As for the geometry of the loop (Figure 2) substantial changes2 S. AZIMI, O. ASHTARI, AND T. M. SCHNEIDER τ ( × ) − − − − − − √ J (i)(ii)(iii)(iv)(v) (vi) − P (i) (ii) (iii) − − P − P (iv) − − P (v) − − P (vi) Fig. 2 . Convergence of the adjoint-based variational method for finding periodic orbits of theKSE: The initial value problem in loop-space evolves loops such that the cost function J decreasesmonotonically along the fictitious time τ (top). The exponential decay of J towards zero indicatesconvergence towards a periodic orbit satisfying J = 0 . Geometrically, the variational dynamicsdeforms a closed loop until it becomes an integral curve of the flow and thus a periodic orbit ofthe KSE. This is shown in the bottom panel, where the evolution of the loop is visualized in a two-dimensional projection of state space. Blue solid lines indicate the evolving loop at times indicatedin the top panel. The dashed gray line is the converged periodic orbit. The state space projections P ( s ) and P ( s ) are defined by the imaginary parts of the first and second spatial Fourier coefficientsof the field u ( x ) . in the period T under the adjoint-based variational dynamics are mostly observedwithin the initial 10% of the integration of the dynamical system in loop-space (4.5).Already at τ = 2 · , T is very close to the period of the periodic orbit T = 59 . τ = 4 · from Figure 3 since changes would not be visible.The fast initial decrease of the cost function J followed by a slow exponential decaytowards zero suggests that the loop approaches the periodic orbit along the leadingeigendirection of the loop dynamics linearized around the attracting fixed point. Mostof the computational efforts are spent on following the exponential decay until thecost function has reached sufficiently low values, although this part of the dynamicsis, at least approximately, linear. Consequently, the convergence of the method canbe accelerated by explicitly exploiting the linearized dynamics in the vicinity of theattracting fixed point. A straightforward method reducing the computational costsby approximately 50% is discussed in Appendix C. More sophisticated optimizationscan be implemented and will be helpful when applying the adjoint-based variationalmethod to three-dimensional fluid flows. DJOINT-BASED VARIATIONAL METHOD τ ( × ) T x
400 59 . t x ( a )( b )( c ) − . − . . . . Fig. 3 . A periodic orbit is characterized by the combination of the field u ( x, s ) on a fixed double-periodic space-time domain and the time period T that rescales the temporal direction s → t = T · s .The variational dynamics adapts T until the period of the periodic orbit is determined (top). Findingthe period T corresponds to determining the length of the domain in time t . This is evidenced byspace-time contours of the solution u ( x, t = T · s ) for the initial condition (b) and the convergedperiodic orbit (c). The period of the initial loop and the periodic orbit are T = 40 and T = 59 . ,respectively. − − P ( a ) T = 25 . ( b ) T = 53 . ( c ) T = 76 . − −
10 0 10 20 P − − P ( d ) T = 106 . − −
10 0 10 20 P ( e ) T = 123 . − −
10 0 10 20 P ( f ) T = 147 . Fig. 4 . Periodic orbits of increasing length and complexity converged by the adjoint-basedvariational method. The two-dimensional projection of state space as in Figure indicates, theinitial loops (dashed orange lines) as well as the converged periodic orbits (solid blue lines). Theperiod of the converged orbits are given in each panel. The gray line in the background of eachpanel is the trajectory of a long chaotic solution in the center symmetry subspace of the KSE (4.2) .All initial loops are chosen from the center-symmetric subspace. The dynamical system in loop-space preserves the discrete symmetry of the initial loops Ξ so that all converged periodic orbits arealso center-symmetric although the symmetry has not been imposed by the method. Note the largedifferences between initial loops and converged periodic orbits highlighting the global convergence ofthe adjoint-based variational method. S. AZIMI, O. ASHTARI, AND T. M. SCHNEIDER x . ( a ) x . ( b ) x . ( c ) x . ( d ) x . ( e ) . t x ( f ) − . − . − . − . . . . . . Fig. 5 . Space-time contours of the converged periodic orbits from Figure with time periods of(a) T = 25 . , (b) T = 53 . , (c) T = 76 . , (d) T = 106 . , (e) T = 123 . , and (f) T = 147 . .Unlike shooting methods, where exponential error amplification during time-integration along theorbit renders long orbits inaccessible, the adjoint-based variational method deforms orbits locallyand thus converges independent of the orbit period. One major advantage of the adjoint-based variational method is that the success-ful convergence towards a periodic orbit is independent of the period of the respectiveorbit. This is in contrast to shooting methods, where the exponential amplificationof errors during time-marching along the orbit can hinder computing long orbits. Wedemonstrate the convergence of orbits of increasing period and complexity in Figure 4.Six converged periodic orbits with periods ranging from T = 25 .
37 to T = 147 .
42 areshown in terms of state-space projections, together with initial loops extracted from achaotic time-series of the KSE. The apparent large difference between initial loop andconverged orbit demonstrates that the adjoint-based variational method offers a verylarge radius of convergence and convergence therefore does not depend on an initialcondition in the close vicinity of the converged orbit. The evolution of loops under thedynamical system in loop-space converges to minima of the cost function J for anyinitial condition. While globally convergent, the variational method is not guaranteedto converge to absolute minima of J with J = 0, corresponding to periodic orbits, butthe dynamics may approach a local minimum with J >
0. For initial loops extractedfrom recurrences in a one-dimensional projection of state space, as discussed in 4.3,we observe approximately 70% of all initial conditions to converge to periodic orbitswith J = 0. An example of a loop approaching a local minimum of J is shown inAppendix D.Following Lasagna [29], initial loops for the six orbits discussed in Figure 4 areextracted from a chaotic trajectory of the KSE in the subspace of center-symmetricfields. All initial conditions for the initial value problem in loop-space are thereforecenter symmetric. The dynamical system in loop-space (4.5) preserves the symmetryΞ of loops (4.6) that corresponds to the center symmetry of instantaneous fields inthe KSE system (4.2). Consequently, all converged periodic orbits also lie in thecenter symmetry subspace, as confirmed by Figure 5, where space-time contours of DJOINT-BASED VARIATIONAL METHOD
5. Summary and conclusion.
Unstable periodic orbits have been recognizedas building blocks of the dynamics in driven dissipative spatio-temporally chaotic sys-tems including fluid turbulence. Periodic orbits capture key features of the dynamicsand reveal physical processes sustaining the turbulent flow. Constructing a sufficientlylarge set of periodic orbits moreover carries the hope to eventually yield a predictiverational theory of turbulence, where ‘properties of the turbulent flow can be mathe-matically deduced from the fundamental equations of hydrodynamics’, as expressedby Hopf in 1948 [19]. Despite the importance of unstable periodic orbits, computingthese exact solutions for high-dimensional spatio-temporally chaotic systems remainschallenging. Known methods either show poor convergence properties because theyare based on time-marching a chaotic system causing exponential error amplifica-tion; or they require constructing Jacobian matrices which is prohibitively expensivefor high-dimensional problems. We therefore introduce a new matrix-free methodfor computing periodic orbits that is unaffected by exponential error amplification,shows robust convergence properties and can be applied to high-dimensional spatio-temporally chaotic systems. As a proof-of-concept we implement the method for theone-dimensional KSE and demonstrate the convergence of periodic orbits underlyingspatio-temporal chaos.The adjoint-based variational method constructs a dynamical system that evolvesentire loops such that the value of a cost function measuring deviations of the loopfrom a solution of the governing equations monotonically decreases. Periodic orbitscorrespond to attracting fixed points of the variational dynamics. Due to the varia-tional approach, the method provides a large radius of convergence so that periodicorbits can be found from inaccurate initial guesses. For the KSE we demonstratethe robust convergence properties by successfully computing periodic orbits from in-accurate initial guesses. These guesses are extracted from the projection of the freechaotic dynamics on a single scalar quantity, instead from close recurrences based onthe L -distance between spatial fields [3]. Reliable convergence to machine precisionis observed independent of the period of the orbit.The large convergence radius of the adjoint-based variational method relaxes ac-curacy requirements for initial guesses when those are extracted from the chaoticdynamics. Since initial guesses are characterized by an entire loop, one may usefast-to-compute models approximating the full dynamics to construct initial guessesfor periodic orbits of the full dynamics. Such an approach would not be reasonablefor classical shooting methods where initial guesses are characterized by an instanta-neous initial condition and the difference between model and full dynamics would beamplified exponentially by the time-marching. Suitable models that may help pro-vide initial guesses for constructing large sets of periodic orbits for a given chaoticsystem include under-resolved simulations, spatially filtered equations such as LESin fluids applications [36] and classical POD / DMD based models [31]. In addition,recent breakthroughs in machine learning allow to create data-driven low-dimensionalmodels of the chaotic dynamics that replicate spatio-temporal chaos in one- and two-dimensional systems with remarkable accuracy [32, 47, 46].The feasibility of the proposed method has been demonstrated for a one-dimensionalchaotic PDE but the method applies to general autonomous systems and we plan toimplement it for the full three-dimensional Navier-Stokes equations. Specifically, weaim for an implementation within our own open-source software Channelflow (chan-6 S. AZIMI, O. ASHTARI, AND T. M. SCHNEIDER nelflow.ch) [17]. In the context of this software not only the identification of periodicorbits but also their numerical continuation will benefit from the adjoint-based varia-tional approach. When transferring the adjoint-based variational approach to three-dimensional fluid turbulence, we envision further optimizations of the method. First,we will exploit that during its approach to the attracting fixed point representing theperiodic orbit, the evolution is well approximated by the linearization of the dynam-ics around the attracting fixed point. This allows to accelerate the time-marchingin loop-space and thereby the exponential convergence, as exemplified for the KSE.Second, one may complement the adjoint dynamics with Newton descent to identifythe attracting fixed point in loop-space, following the analogous hybrid approach foridentifying equilibrium solutions [14]. Alternatively, we will combine the adjoint-basedvariational method with a Newton-GMRES-based shooting method. Such a hybridmethod offers the large radius of convergence of the adjoint-based variational methodin combination with the fast quadratic convergence of Newton’s method. To allow forconverging long and unstable periodic orbits, a multi-shooting variant of the standardNewton-GMRES-hook-step method [37] will be used.
Appendix A. Rate of change of the cost function J . The rate of changeof the cost function J with respect to the fictitious time τ is given in Equation (3.6).Here we derive this expression including the specific form of R . With the definitionof the cost function J (2.5) J ( l ) = (cid:90) (cid:90) Ω (cid:126)r ( l ) .(cid:126)r ( l ) d(cid:126)xds, the rate of change of J with respect to the fictitious time τ is ∂J∂τ = 2 (cid:90) (cid:90) Ω ( ∇ l (cid:126)r · G ) · (cid:126)rd(cid:126)xds. where ∂ l /∂τ = G from definition (3.4) has been used. Using the definition of the innerproduct in the space of generalized loops (3.2), we can express the rate of change as ∂J∂τ = 2 (cid:42) ∇ l (cid:126)r · G , (cid:126)r (cid:43) . Here we choose the second component of both generalized loops to be zero. With thischoice, the rate of change of J is given by ∂J∂τ = 2 (cid:104) LLL ( l ; G ) , R (cid:105) , where LLL ( l ; G ) indicates the directional derivative of R = [ (cid:126)r ; 0] along G , defined in(3.8). Appendix B. Adjoint operator for KSE.
We explicitly derive the form ofthe adjoint operator for the KSE problem given in Equation (4.4). In this appendix,subscripts 1 and 2 denote the field component and the scalar component of generalizedloops, respectively. The directional derivative of KSE along G is LLL ( l ; G ) = G T ∂u∂s − T ∂G ∂s − ∂ ( uG ) ∂x − ∂ G ∂x − ∂ G ∂x DJOINT-BASED VARIATIONAL METHOD (cid:104)
LLL ( l ; G ) , R (cid:105) = (cid:90) (cid:90) L L R dxds + L R = (cid:90) (cid:90) L L R dxds + 0= (cid:90) (cid:90) L (cid:18) G T ∂u∂s − T ∂G ∂s − ∂ ( uG ) ∂x − ∂ G ∂x − ∂ G ∂x (cid:19) R dxds = (cid:90) (cid:90) L G T ∂u∂s R dxds (B.1) + (cid:90) (cid:90) L (cid:18) − T ∂G ∂s − ∂ ( uG ) ∂x − ∂ G ∂x − ∂ G ∂x (cid:19) R dxds. This inner product must be equal to(B.2) (cid:10) G , LLL † ( l ; R ) (cid:11) = (cid:90) (cid:90) L L † G dxds + L † G , where the adjoint operator is indicated by a dagger. Direct comparison of equations(B.1) and (B.2) results in (cid:90) (cid:90) L L † G dxds = (cid:90) (cid:90) L (cid:18) − T ∂G ∂s − ∂ ( uG ) ∂x − ∂ G ∂x − ∂ G ∂x (cid:19) R dxds (B.3a) L † G = (cid:32)(cid:90) (cid:90) L T ∂u∂s R dxds (cid:33) G . (B.3b)The form of L † is directly given by (B.3b): L † ( q ; R ) = (cid:90) (cid:90) L T ∂u∂s R dxds. Using integration by parts and the periodicity of the domain in space and time,Equation (B.3a) becomes (cid:90) (cid:90) L L † G dxds = (cid:90) (cid:90) L (cid:18) T ∂R ∂s + u ∂R ∂x − ∂ R ∂x − ∂ R ∂x (cid:19) G dxds. Consequently, L † ( l ; R ) = 1 T ∂R ∂s + u ∂R ∂x − ∂ R ∂x − ∂ R ∂x where R = r . The adjoint operator acting on loops therefore has the form LLL † ( l ; R ) = T ∂r∂s + u ∂r∂x − ∂ r∂x − ∂ r∂x (cid:90) (cid:90) L T ∂u∂s rdxds . S. AZIMI, O. ASHTARI, AND T. M. SCHNEIDER n ( × ) − − − − − − √ J with extrapolationwithout extrapolation Fig. 6 . Accelerated convergence of the adjoint-based variational method. Convergence historyfor the periodic orbit discussed in figures 2 and 3, for the standard method (orange dashed line)and the modified method involving linear extrapolations along the solution trajectory in the loop-space. The linear extrapolations are based on a linear approximation of the loop dynamics aroundthe attracting fixed point in loop-space corresponding to the periodic orbit. The square root of thecost function is shown as a function of the number of fictitious time steps n . The first extrapolationis performed when √ J = 10 − . Between two consecutive extrapolations, the dynamical system inloop-space is integrated until the value √ J is halved. In this example case, extrapolations reduce thetotal number of fictitious time steps by more than . Appendix C. Acceleration of the convergence by linearized approxima-tion.
We demonstrate a straightforward method for accelerating the convergence of theadjoint-based variational method. We iterate between time-stepping of the dynamicalsystem in loop-space (4.5) and a linear extrapolation along the evolution trajectoryof the loops. This extrapolation is based on the assumption that the evolution followsthe leading eigendirection of the linearization about the attracting loop. Extrapola-tions yield the initial conditions of the subsequent advancing of the loop in τ . Thisprocedure is repeated until the periodic orbit is converged. Figure 6 compares theconvergence of the periodic orbit shown in figures 2 and 3 by continuous integrationof the dynamical system in loop-space (4.5) and the accelerated method iteratingbetween time-stepping of the full dynamics and extrapolations, both from the sameinitial condition. Vertical drops of the cost function shown in the graph correspondto the extrapolations. In this example the accelerated method reduces the requiredtotal number of numerical steps of integration by more than 50%. Appendix D. Convergence to local and global minima of J . Here weshow an example of time-stepping of the dynamical system in loop-space where thefinal loop corresponds to local minimum of J with a nonzero value. Consequently, noperiodic orbit is found. Acknowledgments.
We thank Florian Reetz for insightful discussions on theimplementation of the proposed method both for the KSE but also for future imple-mentations within Channelflow. SA acknowledges support by the State Secretariatfor Education, Research and Innovation SERI via the Swiss Government ExcellenceScholarship.
REFERENCESDJOINT-BASED VARIATIONAL METHOD τ ( × ) − − − − − − √ J ( a ) − . . . . . . − − P ( b ) − − P − − − P ( c ) Fig. 7 . Minimizing J by the adjoint-based variational method. ( a ) Evolution of √ J with τ for two different initial loops. The blue line shows the convergence for the loop that approaches aperiodic orbit with J = 0 while the red line shows the convergence for a loop that approaches a localminimum of J with a nonzero value J > . The corresponding initial (dashed lines) and convergedloops (solid lines) in the two-dimensional projection of the state space as in Figure are visualizedfor the converged loop with J > (panel b) and the periodic orbit with J → (panel c). [1] R. Artuso, E. Aurell, and P. Cvitanovi´c , Recycling of strange sets: I. Cycle expansions ,Nonlinearity, 3 (1990), pp. 325–359, https://doi.org/10.1088/0951-7715/3/2/005.[2]
R. Artuso, E. Aurell, and P. Cvitanovi´c , Recycling of strange sets: II. Applications , Non-linearity, 3 (1990), pp. 361–386, https://doi.org/10.1088/0951-7715/3/2/006.[3]
D. Auerbach, P. Cvitanovi´c, J.-P. Eckmann, G. Gunaratne, and I. Procaccia , Exploringchaotic motion through periodic orbits , Physical Review Letters, 58 (1987), pp. 2387–2389,https://doi.org/10.1103/PhysRevLett.58.2387.[4]
B. M. Boghosian, A. Brown, J. L¨att, H. Tang, L. M. Fazendeiro, and P. V. Coveney , Un-stable periodic orbits in the Lorenz attractor. , Philosophical transactions. Series A, Math-ematical, physical, and engineering sciences, 369 (2011), pp. 2345–53, https://doi.org/10.1098/rsta.2011.0067.[5]
C. Canuto, M. Y. Hussaini, A. Quarteroni, and T. A. Zang , Spectral Methods: Funda-mentals in Single Domains , Springer, 2006, https://doi.org/10.1007/978-3-540-30726-6.[6]
G. J. Chandler and R. R. Kerswell , Invariant recurrent solutions embedded in a turbulenttwo-dimensional Kolmogorov flow , Journal of Fluid Mechanics, 722 (2013), pp. 554–595,https://doi.org/10.1017/jfm.2013.122.[7]
P. Cvitanovi´c , Invariant measurement of strange sets in terms of cycles , Physical ReviewLetters, 61 (1988), pp. 2729–2732, https://doi.org/10.1103/PhysRevLett.61.2729.[8]
P. Cvitanovi´c, R. Artuso, G. Mainieri, G. Tanner, and G. Vattay , Chaos: Classical andquantum , chaosbook.org, Niels Bohr Institute, Copenhagen, 2016.[9]
P. Cvitanovi´c, R. L. Davidchack, and E. Siminos , On the state space geometry of theKuramoto-Sivashinsky flow in a periodic domain , SIAM Journal on Applied DynamicalSystems, 9 (2010), pp. 1–33, https://doi.org/10.1137/070705623.[10]
P. Cvitanovi´c and J. F. Gibson , Geometry of the turbulence in wall-bounded shear flows: pe-riodic orbits , Physica Scripta, T142 (2010), p. 014007, https://doi.org/10.1088/0031-8949/2010/T142/014007.[11]
J. E. Dennis and R. B. Schnabel , Numerical methods for unconstrained optimization andnonlinear equations , SIAM, 1996.[12]
Y. Duguet, C. C. T. Pringle, and R. R. Kerswell , Relative periodic orbits in transitionalpipe flow , Physics of Fluids, 20 (2008), p. 114102, https://doi.org/10.1063/1.3009874.[13]
B. Eckhardt, T. M. Schneider, B. Hof, and J. Westerweel , Turbulence Transition inPipe Flow , Annual Review of Fluid Mechanics, 39 (2007), pp. 447–468, https://doi.org/ S. AZIMI, O. ASHTARI, AND T. M. SCHNEIDER10.1146/annurev.fluid.39.050905.110308.[14]
M. Farazmand , An adjoint-based approach for finding invariant solutions of Navier-Stokesequations , Journal of Fluid Mechanics, 795 (2016), pp. 278–312, https://doi.org/10.1017/jfm.2016.203.[15]
L. M. Fazendeiro, B. M. Boghosian, P. V. Coveney, and J. L¨att , Unstable periodic orbitsin weak turbulence , Journal of Computational Science, 1 (2010), pp. 13–23, https://doi.org/10.1016/j.jocs.2010.03.004.[16]
J. F. Gibson, J. Halcrow, and P. Cvitanovi´c , Visualizing the geometry of state space inplane Couette flow , Journal of Fluid Mechanics, 611 (2008), pp. 107–130, https://doi.org/10.1017/S002211200800267X.[17]
J. F. Gibson, F. Reetz, S. Azimi, A. Ferraro, T. Kreilos, H. Schrobsdorff, M. Farano,A. F. Yesil, S. S. Sch¨utz, M. Culpo, and T. M. Schneider , Channelflow 2.0 , in prepa-ration, (2019).[18]
M. C. Gutzwiller , Chaos in classical and quantum mechanics , Journal of Physics A: Mathe-matical and Theoretical, 43 (1990), p. 285302, https://doi.org/10.1088/1751-8113/43/28/285302.[19]
E. Hopf , A mathematical example displaying features of turbulence , Communications on Pureand Applied Mathematics, 1 (1948), pp. 303–322, https://doi.org/10.1002/cpa.3160010401.[20]
J. Jim´enez , Coherent structures in wall-bounded turbulence , Journal of Fluid Mechanics, 842(2018), p. P1, https://doi.org/10.1017/jfm.2018.144.[21]
G. Kawahara and S. Kida , Periodic motion embedded in plane Couette turbulence: re-generation cycle and burst , Journal of Fluid Mechanics, 449 (2001), pp. 291–300, https://doi.org/10.1017/S0022112001006243.[22]
G. Kawahara, M. Uhlmann, and L. van Veen , The Significance of Simple Invariant Solutionsin Turbulent Flows , Annual Review of Fluid Mechanics, 44 (2012), pp. 203–225, https://doi.org/10.1146/annurev-fluid-120710-101228.[23]
C. T. Kelley , Solving Nonlinear Equations with Newton’s Method , Society for Industrial andApplied Mathematics, 1 2003, https://doi.org/10.1137/1.9780898718898.[24]
R. R. Kerswell , Recent progress in understanding the transition to turbulence in a pipe ,Nonlinearity, 18 (2005), pp. R17–R44, https://doi.org/10.1088/0951-7715/18/6/R01.[25]
Y. Kuramoto and T. Tsuzuki , Persistent Propagation of Concentration Waves in DissipativeMedia Far from Thermal Equilibrium , Progress of Theoretical Physics, 55 (1976), pp. 356–369, https://doi.org/10.1143/ptp.55.356.[26]
Y. Lan , Cycle expansions: From maps to turbulence , Communications in Nonlinear Scienceand Numerical Simulation, 15 (2010), pp. 502–526, https://doi.org/10.1016/j.cnsns.2009.04.022.[27]
Y. Lan and P. Cvitanovi´c , Variational method for finding periodic orbits in a general flow ,Physical Review E, 69 (2004), p. 016217, https://doi.org/10.1103/PhysRevE.69.016217.[28]
Y. Lan and P. Cvitanovi´c , Unstable recurrent patterns in Kuramoto-Sivashinsky dynamics ,Physical Review E, 78 (2008), p. 026208, https://doi.org/10.1103/PhysRevE.78.026208.[29]
D. Lasagna , Sensitivity Analysis of Chaotic Systems Using Unstable Periodic Orbits , SIAMJournal on Applied Dynamical Systems, 17 (2018), pp. 547–580, https://doi.org/10.1137/17M114354X.[30]
C. C. Lin , On the Stability of Two-Dimensional Parallel Flows , Proceedings of the NationalAcademy of Sciences, 30 (1944), pp. 316–324, https://doi.org/10.1073/pnas.30.10.316.[31]
B. J. McKeon , The engine behind (wall) turbulence: perspectives on scale interactions , Journalof Fluid Mechanics, 817 (2017), p. P1, https://doi.org/10.1017/jfm.2017.115.[32]
J. Pathak, B. Hunt, M. Girvan, Z. Lu, and E. Ott , Model-Free Prediction of Large Spa-tiotemporally Chaotic Systems from Data: A Reservoir Computing Approach , PhysicalReview Letters, 120 (2018), p. 024102, https://doi.org/10.1103/PhysRevLett.120.024102.[33]
H. Poincar´e , Les m´ethodes nouvelles de la m´ecanique c´eleste , vol. 10, Gauthier-Villars, Paris,1892.[34]
F. Reetz, T. Kreilos, and T. M. Schneider , Exact invariant solution reveals the originof self-organized oblique turbulent-laminar stripes , Nature Communications, 10 (2019),p. 2277, https://doi.org/10.1038/s41467-019-10208-x.[35]
D. Ruelle , Thermodynamic formalism: The mathematical structures of classical equilibriumstatistical mechanics , Addison-Wesley, 1 ed., 1978.[36]
P. Sagaut , Large eddy simulation for incompressible flows: an introduction , Springer Science& Business Media, 2006.[37]
J. S´anchez and M. Net , On the Multiple Shooting Continuation of Periodic Orbits By New-tonKrylov Methods , International Journal of Bifurcation and Chaos, 20 (2010), pp. 43–61,https://doi.org/10.1142/S0218127410025399.DJOINT-BASED VARIATIONAL METHOD [38] J. Sanchez, M. Net, B. Garcia-Archilla, and C. Simo , NewtonKrylov continuation of peri-odic orbits for NavierStokes flows , Journal of Computational Physics, 201 (2004), pp. 13–33, https://doi.org/10.1016/j.jcp.2004.04.018.[39]
G. I. Sivashinsky , Nonlinear analysis of hydrodynamic instability in laminar flamesI. Deriva-tion of basic equations , Acta Astronautica, 4 (1977), pp. 1177–1206, https://doi.org/10.1016/0094-5765(77)90096-0.[40]
Y. S. Smyrlis and D. T. Papageorgiou , Computational study of chaotic and ordered solutionsof the Kuramoto-Sivashinsky equation , tech. report, Institute for Computer Applicationsin Science and Engineering, NASA Langley Research Center, 1996.[41]
B. Suri, J. Tithof, R. O. Grigoriev, and M. F. Schatz , Forecasting Fluid Flows Using theGeometry of Turbulence , Physical Review Letters, 118 (2017), p. 114501, https://doi.org/10.1103/PhysRevLett.118.114501.[42]
T. Y. Thomas , Qualitative Analysis of the Flow of Fluids in Pipes , American Journal ofMathematics, 64 (1942), pp. 754–767.[43]
L. van Veen, A. Vela-Martin, and G. Kawahara , Time-periodic inertial range dynamics ,arXiv preprint, (2018).[44]
D. Viswanath , Recurrent motions within plane Couette turbulence , Journal of Fluid Mechan-ics, 580 (2007), pp. 339–358, https://doi.org/10.1017/S0022112007005459.[45]
D. Viswanath , The critical layer in pipe flow at high Reynolds number , Philosophical Transac-tions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences,367 (2009), pp. 561–576.[46]
P. R. Vlachas, W. Byeon, Z. Y. Wan, T. P. Sapsis, and P. Koumoutsakos , Data-drivenforecasting of high-dimensional chaotic systems with long short-term memory networks ,Proceedings of the Royal Society A: Mathematical, Physical and Engineering Science, 474(2018), p. 20170844, https://doi.org/10.1098/rspa.2017.0844.[47]