On extremals of the entropy production by "Langevin-Kramers" dynamics
aa r X i v : . [ c ond - m a t . s t a t - m ec h ] J a n On extremals of the entropy production by“Langevin–Kramers” dynamics
Paolo Muratore-Ginanneschi
Department of Mathematics and Statistics PL 68 FIN-00014 University of Helsinki FinlandE-mail: [email protected]
PACS numbers: 05.40.-a Fluctuation phenomena statistical physics, 05.70.Ln Nonequilib-rium and irreversible thermodynamics, 02.50.Ey Stochastic processes, 02.30.Yy Control The-ory
Abstract.
We refer as “Langevin–Kramers” dynamics to a class of stochastic differentialsystems exhibiting a degenerate “metriplectic” structure. This means that the drift field can bedecomposed into a symplectic and a gradient-like component with respect to a pseudo-metrictensor associated to random fluctuations affecting increments of only a sub-set of the degreesof freedom. Systems in this class are often encountered in applications as elementary modelsof Hamiltonian dynamics in an heat bath eventually relaxing to a Boltzmann steady state.Entropy production control in Langevin–Kramers models differs from the now well-understood case of Langevin–Smoluchowski dynamics for two reasons. First, the definition ofentropy production stemming from fluctuation theorems specifies a cost functional which doesnot act coercively on all degrees of freedom of control protocols. Second, the presence of asymplectic structure imposes a non-local constraint on the class of admissible controls. UsingPontryagin control theory and restricting the attention to additive noise, we show that smooth protocols attaining extremal values of the entropy production appear generically in continuousparametric families as a consequence of a trade-off between smoothness of the admissibleprotocols and non-coercivity of the cost functional. Uniqueness is, however, always recoveredin the over-damped limit as extremal equations reduce at leading order to the Monge–Amp`ere–Kantorovich optimal mass transport equations. n extremals of the entropy production by “Langevin–Kramers” dynamics
1. Introduction
The contrivance and development of techniques that can be used to investigate the physics ofvery small system is currently attracting great interest [1]. Examples of very small systemare bio-molecular machines consisting of few or, in some cases, even one molecule. Thesesystems are able to “efficiently”, in some sense, operate in non-equilibrium environmentsstrongly affected by thermal fluctuations [2]. For example, protein biosynthesis relies on thequality and efficiency with which the ribosome, a complex molecular machine, is able topair mRNA codons with matching tRNA. During such process, known as “decoding”, theribosome and tRNA undergo large conformation changes which appear to correspond to anoptimal in energy landscape recognition strategy [3].One relevant motivation for attaining a precise characterizations of thermodynamicefficiency of single-biomolecule systems such as molecular switches is the hope that theirproperties can be exploited in molecular-scale information processing [4]. A long-standingconjecture by Landauer [5] surmises that the erasure of information is a dissipative process.Brownian computers [4] incorporate this conjecture in the form of models amenable toexperimental [6] and theoretical [7] investigation. In particular, the erasure of one bit ofinformation can be modeled by steering the evolution of the probability density of a diffusionprocess in a bistable potential by manipulating the height and the depth of the potentialwells [4]. Using the optimal control techniques introduced in the context of stochasticthermodynamics by [8, 9], the authors of [7] (see also [10]) showed that the minimal averageheat release during the erasure of one bit of information generically exceeds Landauer’s k B β − ln 2 bound where β is the inverse temperature and k B Boltzmann’s constant. Thistheoretical result is consistent with the experimental findings of [6] where one bit erasurewas modeled by a system of a single colloidal particle trapped in a modulated double-wellpotential.The generality of Landauer’s argument upholding an ultimate physical limit ofirreversible computation and the existence of the recent experimental supporting evidencesindicate that the conclusions of [7] should remain true for Markov models more generalthan Langevin–Smoluchowski’s dynamics. This intuition is corroborated by the fact thatthe entropy produced by smoothly evolving the probability distribution of Markov jumpprocesses between two assigned states is also subject to an analogous lower bound [11].The scope of the present contribution is to explore minimal entropy production transitionsof continuous Markov processes governed by kinematic laws comprising a dissipative and asymplectic, conservative, component. The consideration of such systems, commonly referredto as Langevin–Kramers or under-damped, is important as they provide models of Newtonianmechanics in a heat bath.The structure of the paper is as follows. In section 2 we describe the kinematicproperties of a Langevin–Kramers diffusion. We also define the class A of admissibleHamiltonians governing the Langevin–Kramers dynamics to which we restrict our attentionwhile considering the optimal control problem for the entropy production. From themathematical slant, it is obvious that an optimal control problem is well-posed if we assign n extremals of the entropy production by “Langevin–Kramers” dynamics E t f ,t o over a finite control horizon [ t o , t f ] in terms of the current velocity of the Langevin–Kramers diffusion process. As in the Langevin–Smoluchowski case [15, 9, 16], the currentvelocity parametrization plays a substantive role in unveiling the properties of the entropyproduction from both the thermodynamic and the control point of view. In the Langevin–Kramers case, the entropy production turns out to be a non-coercive [17] cost functional.Namely, the decomposition of the current velocity into dissipative and symplectic componentsevinces that the entropy production is in fact a quadratic functional of the dissipativecomponent alone. The origin of the phenomenon is better understood by revisiting theprobabilistic interpretation of the entropy production which we do in section 4. There werecall that E t f ,t o coincides with the Kullback–Leibler divergence K(P χ || P ˜ χ ) [18] between theprobability measure P χ of the primary Langevin–Kramers process χ and that P ˜ χ of a process ˜ χ obtained from the former by inverting the sign of the dissipative component of the drift andevolving in the opposite time direction. The Kullback–Leibler divergence is a relative entropymeasuring the information loss occasioned when P ˜ χ is used to approximate P χ . These factssubstantiate, on the one hand, the identification of the entropy production as a natural indicatorof the irreversibility of a physical process and, as such, as the embodiment of the second lawof thermodynamics. On the other hand, they pinpoint that the interpretation of the entropyproduction as a Kullback–Leibler divergence is possible in the Kramers–Langevin case onlyby applying in the construction of the auxiliary process ˜ χ a different time reversal operationthan in the Langevin–Smoluchowski case. Non-coercivity is the result of such time reversaloperation.The consequences of non-coercivity of the cost functional on the entropy productionoptimal control are, however, tempered by the regularity requirements imposed on the class A of admissible Hamiltonians. Simple considerations (section 5) based on smoothness of theevolution show that the Langevin–Kramers entropy production must be bounded from belowby the entropy production generated by an optimally controlled Langevin–Smoluchowskidiffusion connecting in the same horizon the configuration space marginals of the initialand final phase space probability densities. This result motivates the analysis of section 6where we avail us of Pontryagin’s maximum principle to directly investigate extremals of theentropy production in a finite time transition between assigned states. Pontryagin’s maximumprinciple is formulated in terms of Lagrange multipliers acting as conjugate “momentum”variables (see e.g. [19, 20, 21]). We can therefore construe it as an “Hamiltonian formulation”of Bellman’s optimal control theory which is based upon dynamic programming equations(see e.g. [17, 22, 23]). Relying on Pontryagin’s maximum principle we conveniently arriveat the first main finding of the present paper encapsulated in the extremal equations (54). Onn extremals of the entropy production by “Langevin–Kramers” dynamics the space A of admissible control Hamiltonians the entropy production generically attains anhighly degenerate minimum value . A distinctive feature of the extremal equations (54) is thatthe coupling between the dynamic programming equation and the Fokker–Planck equationtakes the non-local form of a third auxiliary equation. We attribute the occurrence of a non-local coupling to the divergenceless component of the Langevin–Kramers drift.We illustrate these results in section 7 where we consider the explicitly solvable caseof Gaussian statistics. While considering this example, we also inquire the recovery inthe over-damped limit of the expression of the minimal entropy production by Langevin–Smoluchowski diffusions, a problem to which we systematically turn in section 8. Therewe derive our second main result: upon applying a multi-scale (homogenization) asymptoticanalysis [24, 25] we show that the cell problem associated to the extremal equations (54) takesthe form of a Monge–Amp`ere–Kantorovich mass transport problem [26] for configurationspace marginals of the phase space probability densities. The noteworthy aspect of thisresult is that the degeneracy of the extremals (54) does not appear in the cell problemso that consistence with the results obtained for Langevin–Smoluchowski diffusions [7] isguaranteed.Finally, the last section 9 is devoted to a discussion and some conjectures concerning theexistence singular control strategies which we explicitly ruled out while deriving the extremalequations (54).
2. From kinematics to dynamics
We consider a phase-space dynamics governed by d χ t = ( J − G ) · ∂ χ t H d tτ + r β τ G / · d ω t (1a) P( x ≤ χ t o < x + d x ) = m o ( x )d d x (1b)In (1a) ω = { ω t , t ≥ t o } denotes an R d -valued Wiener-process while J and G are d -dimensional contravariant tensors of rank 2 with constant entries. In particular, J is the “co-symplectic” form J = " d − d ⇒ J † J = d (2)where d stands for the identity in d -dimensions. Furthermore, we associate to thermal noisefluctuations the constant pseudo-metric tensor G = P v (3)with P v the vertical projector in phase space P h ≡ d ⊕ & P v ≡ ⊕ d (4)so that d = P h ⊕ P v and, for any x = q ⊕ p ∈ R d with p , q ∈ R d , " q ≡ P h · x & " p ≡ P v · x (5) n extremals of the entropy production by “Langevin–Kramers” dynamics β and τ are positive definite constants. We attribute to β the physical interpretationof the inverse of the temperature and to τ that of the characteristic time scale of the system.We will measure any other quantity encountered throughout the paper in units of β and τ .The kinematics in (1a) satisfies the conditions required by H¨ormander theorem toprove that for any sufficiently regular, bounded from below and, growing sufficiently fastat infinity Hamilton function H , the process χ = { χ t , t ∈ [ t o , t f ] } admits a smoothtransition probability density notwithstanding the degenerate form of the noise (see e.g. [27]).Furthermore, if H is time independent, it is straightforward to verify that the measure relaxesto a steady state such that P( x ≤ χ ∞ < x + d x ) = β d e β [ F − H ( x )] d d x (6a) F ≡ − β ln Z R d d d x β d e − β H ( x ) (6b)The expression of the normalization constant in (6a) befits the interpretation of F as equilibrium free energy . For any finite time, we find expedient to write the probability densityof the system in the form P( x ≤ χ t < x + d x ) = m ( x , t ) d d x ≡ β d e − S ( x ,t ) d d x (7)We will refer to the non-dimensional function S as the microscopic entropy of the systeminasmuch it specifies the amount of information required to describe the state of the systemgiven that the state occurs with probability (7) [28]. The average variation of S with respectto the measure of χ , which we denote by E ( χ ) , ( S t − S t ) = E ( χ ) [ S ( χ t , t ) − S ( χ t , t )] ∀ t ≥ t ≥ (8)specifies the variation of the Shannon-Gibbs entropy of the system. The representation (7)of the probability density establishes an elementary link between the kinematics and thethermodynamics of the system. In order to describe the dynamics, we need to specify theHamiltonian H . Our aim here is to determine H by solving an optimal control problemassociated to the minimization of a certain thermodynamic functional, the entropy production,during a transition evolving the initial state (1b) into a final state P( x ≤ χ < x + d x ) = m f ( x ) d d x (9)in a finite time horizon [ t o , t f ] . From this slant, we need to regard H as the element of a class A of admissible controls comprising time -dependent phase space functions H : R d × R + R (10)satisfying the following requirements. Any H ∈ A must be at least twice differentiable withrespect to its phase space dependence and once differentiable for any t ∈ [ t o , t f ] . Furthermore,we require that for any Hamiltonian in A the evolution of the probability density of χ obeysa Fokker–Planck equation throughout the control horizon [ t o , t f ] . As we adopt the workinghypothesis that the initial and final state are described by probability densities integrable over R d , admissible Hamiltonians must then preserve this property for any t ∈ [ t o , t f ] . Thesehypotheses entail A ⊂ C (2 , ( R d , [ t o , t f ]) ∩ L (2) ( R d , m d d x ) (11) n extremals of the entropy production by “Langevin–Kramers” dynamics m d d x portends square integrability requirement with respect to the density of (2). Wewill reserve the simpler notation L (2) ( R d ) to the space of functions square integrable withrespect to the Lebesgue measure. The scalar generator L of (1) acts on any differentiable phase space function f as ( L f )( x , t ) ≡ (cid:26) [( J − G ) · ∂ x H ( x , t )] · ∂ x + 1 β G : ∂ x ⊗ ∂ x (cid:27) f ( x , t ) (12)In (12) and in what follows we use the notation A : B ≡ Tr A † B (13)for the scalar product between matrices. It is worth noticing that it is possible to write thegenerator in terms of a symplectic and a “metric” bracket operation. Namely, the transpose J † of J defines a symplectic form between differentiable phase space functions f i , i = 1 , ( f , f ) J † ≡ ( ∂ x f ) · ( J † · ∂ x f ) ≡ ∂ p f d · ∂ q f − ∂ q f d · ∂ p f (14)coinciding with Poisson brackets in a Darboux chart. In (12) and (14) the symbol “ · ” standsfor the dot-product in R d and d · for the analogous operation in R d . Similarly, it is possible toassociate to G the degenerate metric brackets ( f , f ) G ≡ ( ∂ x f ) · ( G · ∂ x f ) (15)acting on scalars in R d and the pseudo-norm k f k G ≡ f · G · f ∀ f ∈ R d (16)The generator becomes L f = ( H , f ) J † + G f (17)where G f = − ( H , f ) G + 1 β G : ∂ x ⊗ ∂ x f (18)The Poisson brackets embody the energy conserving component of the kinematics. Thedifferential operation G describes dissipation which occurs via a deterministic frictionmechanism associated to the metric brackets and via thermal interactions encapsulated in thesecond order differential terms. In the analytic mechanics literature it is customary to refer tosystems whose generator comprises a symplectic and a metric structure as “ metriplectic ” seee.g. [29, 30].The L (2) ( R d ) adjoint of L with respect to the Lebesgue measure governs the evolutionof the probability density m of the system. The anti-symmetry of the Poisson brackets yields L † f = − ( H , f ) J † + G † f (19)with G † the L (2) ( R d ) -adjoint of (18). n extremals of the entropy production by “Langevin–Kramers” dynamics
3. Thermodynamic functionals
Following [13], we identify the heat released during individual realizations of χ with theStratonovich stochastic integral Q t f ,t o = − Z t f t o d χ t / · ∂ χ t H (20)The / betokens Stratonovich’s mid-point convention. If in conjunction with (20) we definethe work as W t f ,t o = Z t f t o d t ∂ t H (21)we recover the first law of thermodynamics in the form ( W − Q ) t f ,t o = Z t f t o (cid:16) d t ∂ t H + d χ t / · ∂ χ t H (cid:17) = H t f − H t o (22)As our working hypotheses allow us to perform integrations by parts without generatingboundary terms, the definition of the Stratonovich integral begets (see e.g. [31] pag. 33)the equality E ( χ ) Z t f t o d χ t / · ∂ χ t H = E ( χ ) Z t f t o d tτ v t · ∂ χ t H (23)for v = J · ∂ x H − G · ∂ x (cid:18) H − β S (cid:19) (24)the current velocity [32] (see also Appendix A) and S the microscopic entropy (7). Uponinserting (24) into (23) after straightforward algebra we arrive at E t f ,t o β ≡ E ( χ ) (cid:26) Q t f ,t o + S t f − S t o β (cid:27) = E ( χ ) Z t f t o d tτ k ∂ χ t A k G ≥ (25)We interpret the phase space function A = H − β S (26)as the “ non-equilibrium Helmholtz energy density” of the system and, the non-dimensionalquantity E as the entropy production during the transition (see e.g. [33, 15, 34]). Theinterpretation is upheld by observing that the entropy production rate, E ( χ ) k ∂ χ t A k G , isa positive definite quantity generically vanishing only at equilibrium. On this basis, we regardthe inequality (25) as the embodiment of the second law of thermodynamics. Furthermore,the positive definiteness of the entropy production yields a Jarzynski type [35] bound for themean work E ( χ ) W t f ,t o = E ( χ ) ( A t f − A t o ) + 1 β E t f ,t o ≥ E ( χ ) ( A t f − A t o ) (27) n extremals of the entropy production by “Langevin–Kramers” dynamics
4. Probabilistic interpretation of the thermodynamic functionals
The entropy production (25) admits an intrinsic information theoretic interpretation as aquantifier of the irreversibility of a transition. Namely it coincides with the Kullback–Leiblerdivergence between the measure of the process (1) and that of the backward-in-time diffusionprocess ˜ χ = { ˜ χ t ; t ∈ [ t o , t f ] } obtained by reverting the sign of the dissipative component ofthe drift: d ˜ χ t = ( J + G ) · ∂ ˜ χ t H d tτ + r β τ G / · d ω t (28a) P( x ≤ ˜ χ t f < x + d x ) = m ( x , t f )d d x (28b)In (28b) m ( x , t f ) is the probability density generated by (1) evaluated at t f whilst H in (28a)is the very same Hamiltonian entering (1). The drift in (28a) must be interpreted as the meanbackward derivative D − ˜ χ t ˜ χ t of the process ˜ χ (see Appendix A for details). In order to compare χ with ˜ χ we suppose that the corresponding probability measures P χ and P ˜ χ have supportover the same Borel sigma algebra F [ t o ,t f ] and are absolutely continuous with respect to theLebesgue measure. The difference between χ and ˜ χ consists then in the fact that for any t ∈ [ t o , t f ] , χ t is adapted (i.e. measurable with respect) to the sub-sigma algebra F [ t o ,t ] of F [ t o ,t f ] comprising all “past” events at time t . The realization ˜ χ t of ˜ χ is instead adapted to thesub-sigma algebra F [ t,t f ] of F [ t o ,t f ] comprising all “future” events at time t (see e.g. [36] fordetails). A tangible consequence of this difference is that for any integrable test vector field V : R d R d and any t ∈ [ t o , t f ] the Ito pre-point stochastic integral satisfies E ( χ ) Z tt o d χ t · V ( χ t ) = E ( χ ) Z tt o d t [( J − G ) · ∂ χ t H ]( χ t ) · V ( χ t ) (29)while instead the post-point prescription yields E (˜ χ ) Z tt o d ˜ χ t · V ( ˜ χ t ) = E (˜ χ ) Z tt o d t [( J + G ) · ∂ ˜ χ t H ]( ˜ χ t ) · V ( ˜ χ t ) (30)We will now avail us of these observations to prove that Proposition 4.1. if we decompose the current velocity (24) into a “dissipative” component v + ( x , t ) = − G · ∂ x A ( x , t ) (31) and a divergenceless “conservative” component v − ( x , t ) = J · ∂ x H ( x , t ) (32) then Kullback–Leibler divergence between P ˜ χ and P χ depends only on v + and is equal to K(P χ || P ˜ χ ) ≡ E ( χ ) ln dP χ dP ˜ χ = β E ( χ ) Z t f t o d tτ k ∂ χ t A k G (33) Proof.
The proof proceeds in two steps: first we introduce two auxiliary diffusion processes oneforward and the other backward in time for which we know the expression of the Radon-Nikodym derivative of the corresponding probability measures; then we apply Cameron–Martin–Girsanov’s formula (see e.g. [37]) to relate the auxiliary processes to χ and ˜ χ . n extremals of the entropy production by “Langevin–Kramers” dynamics First step
We call η = { η t ; t ∈ [ t o , t f ] } the diffusion with F [ t o ,t ] -adapted realizations solution of the forward stochastic dynamics d η t = J · ∂ η t H d tτ + r β τ G / · d ω t (34a) P( x ≤ η t o < x + d x ) = m o ( x )d d x (34b)Similarly, let ˜ η = { ˜ η t ; t ∈ [ t o , t f ] } diffusion governed by the backward dynamics d ˜ η t = J · ∂ ˜ η t H d tτ + r β τ G / · d ω t (35a) P( x ≤ ˜ χ t f < x + d x ) = m ( x , t f )d d x (35b)with ˜ η t F [ t,t f ] -adapted. The simultaneous occurrence of additive noise and divergence-less drift in (34a), (35a) occasions the identity p η ( x , t | x , t ) = p ˜ η ( x , t | x , t ) (36)satisfied by the transition probability densities of η and ˜ η for all x , x ∈ R d and for all t , t ∈ [ t o , t f ] , t ≤ t . We therefore conclude that dP ˜ η dP η = m ( η t f , t f ) m o ( η t o ) (37) Second step
We apply the composition property of the Radon–Nikodym derivative in order to couch(33) into the form
K(P χ || P ˜ χ ) = − E ( η ) dP χ dP η ln (cid:18) dP ˜ χ dP η (cid:30) dP χ dP η (cid:19) (38)Cameron–Martin–Girsanov’s formula yields immediately dP χ dP η = exp (cid:26) β Z t f t o (cid:20) − ( G · ∂ η t H ) · (cid:18) d η t − d tτ J · ∂ η t H (cid:19) − d tτ k ∂ η t H k G (cid:21)(cid:27) (39)which by is a martingale at time t f by (29). In order to compute dP ˜ χ / d P η we first useCameron–Martin–Girsanov’s formula adapted to backward processes [38] dP χ dP η = exp (cid:26) β Z t f t o (cid:20) ( G · ∂ ˜ η t H ) · (cid:18) d ˜ η t − d tτ J · ∂ ˜ η t H (cid:19) − d tτ k ∂ ˜ η t H k G (cid:21)(cid:27) (40)Then we apply again the composition property to write dP ˜ χ dP η = dP ˜ χ dP ˜ η dP ˜ η dP η = m ( η t f , t f ) m o ( η t o ) exp (cid:26) β Z t f t o (cid:20) ( G · ∂ η t H ) · (cid:18) d η t − d tτ J · ∂ η t H (cid:19) − d tτ k ∂ η t H k G (cid:21)(cid:27) (41)since η t on the right hand side plays the role of a mute integration variable. Uponinserting (40) and (41) in (38) and expressing the stochastic integrals into the time-reversal invariant Stratonovich mid-point discretization we arrive at ln dP ˜ χ dP χ = Z t f t o d tτ (cid:26) [( J · ∂ χ t H ) · ∂ χ t + τ ∂ t ] ln m β d (cid:27) + β Z t f t o (cid:20) d χ t − d tτ ( J · ∂ χ t H ) (cid:21) / · (cid:18) G · ∂ χ t H + 1 β ∂ χ t ln m β d (cid:19) (42) n extremals of the entropy production by “Langevin–Kramers” dynamics E ( χ ) (cid:26) [( J · ∂ χ t H ) · ∂ χ t + τ ∂ t ] ln m β d (cid:27) = Z R d d d x ( ∂ x · v + τ ∂ t ) m = 0 (43)In virtue of the properties of the Stratonovich integral (see e.g. [31] pag. 33), theexpectation value of the second integral in (42) yields E ( χ ) Z t f t o (cid:20) d χ t − d tτ ( J · ∂ χ t H ) (cid:21) / · (cid:18) G · ∂ χ t H + 1 β ∂ χ t ln m β d (cid:19) = E ( χ ) Z t f t o d tτ v + · (cid:18) G · ∂ χ t H + 1 β ∂ χ t ln m β d (cid:19) = − E ( χ ) Z t f t o d tτ k ∂ χ t A k G (44)where the last equality holds because G is a projector.Some remarks are in order.(i) The information theoretic interpretation of the entropy production is a consequence ofthe fluctuation relation type [39, 40, 35, 41, 42, 43, 33, 34] equality (42). Reference [44]discusses in details the relation between fluctuation relations for Markov processes andexponential martingales. Finally, a recent nice overview of fluctuation theorems can befound in the lectures [10].(ii) The proof of the identity (33) is based on the comparison between a forward anda backward dynamics in the sense of Nelson [31, 32] and admits a straightforwardgeneralization to all the cases discussed in [34]. In particular, choosing the auxiliaryprocess η to be the stochastic development map (see e.g. [45]) yields readily covariantexpressions for the entropy production by diffusion on Riemann manifolds [15, 16].(iii) The stochastic development map in the Euclidean case with flat metric reduces to thestandard Wiener process. An alternative proof of (33) can be then obtained by taking thelimit of vanishing noise acting on the position coordinate process.(iv) The dissipative (31) and conservative (32) components of the current velocity are not L (2) ( R d , m d d x ) -orthogonal. Therefore, it is not natural to regard the dissipativecomponent as an independent control of the entropy production.
5. A general bound for the entropy production from moments equation
The main consequence of the last remark the foregoing section is that the Hamiltonian H is the natural control functional for the entropy production. The entropy production is,however, independent of derivatives of H with respect to position coordinates. This factposes the question whether the uncoerced degrees of freedom can be used to steer a smoothLangevin–Kramers dynamics to accomplish a finite-time transition between assigned statesfor arbitrarily low values of the entropy production. A simple lower bound provided by the“macroscopic”, in kinetic theory sense (see e.g. [46]), dynamics shows that this cannot be thecase. Let ˜ m ( q , t ) ≡ Z R d d d p m ( p , q , t ) (45) n extremals of the entropy production by “Langevin–Kramers” dynamics m over momenta it is readily seen that ˜ m obeys τ ∂ t ˜ m + ∂ q · ˜ m ˜ v = 0 (46)We define the “macroscopic drift” ˜ v as the average (˜ m ˜ v )( q , t ) ≡ Z R d d d p ( m ∂ p A )( p , q , t ) (47)over the momentum gradient of the non-equilibrium Helmholtz energy density (26). Let ˜ V ≡ ⊕ ˜ v the phase space lift of ˜ v . Since G is the vertical projector in R d , an immediateconsequence of (47) is the inequality E t f ,t o = Z t f t o d t Z R d d d x m (cid:16) k ∂ x A − ˜ V k G + k ˜ V k G (cid:17) ≥ Z t f t o d t Z R d d d q ˜ m k ˜ v k d = ˜ E t f ,t o (48)for kk d the Euclidean norm in R d . Taking into account that (46) must also hold true,we interpret ˜ v as the current velocity of an effective Langevin–Smoluchowski dynamics.Furthermore, ˜ E t f ,t o attains a minimum if the pair (˜ m , ˜ v ) is determined from the solutionof Monge–Amp`ere–Kantorovich problem [9, 7]. We will see in section 8 that the boundbecomes tight in the presence of a strong separation of scales between position and momentumdynamics.
6. Entropy production extremals via Pontryagin theory
The existence of the general bound (48) indicates that the question of existence of entropyproduction extremals in the admissible class A (11) is well posed. In order to directly pursuethe quest, we introduce the Pontryagin functional [19] A ( m , V, A ) = Z t f t o d tτ Z R d d d x (cid:8) m k ∂ x A k G − V (cid:0) τ ∂ t − L † (cid:1) m (cid:9) (49)complemented by the boundary conditions m ( x , t o ) = m o ( x ) & m ( x , t f ) = m f ( x ) (50)The functional (49) specifies a generalized entropy production in which the dynamicalconstraint on the probability density appears explicitly. The “costate” field V : R d × [ t o , t f ] R is a Lagrange multiplier imposing the probability density m to evolve according the Fokker–Planck of (1). The sign convention of V suits the identification of the extremal value of thecostate with the “value” or “cost-to-go” function of Bellman’s formulation of optimal controltheory [17, 22]. If we exploit the anti-symmetry of the Poisson brackets ( S , m ) J † = − ( ln m , m ) J † = 0 (51)and the definition of the non-equilibrium Helmholtz energy density (26), we can always couch L † m into a first order differential operation over the probability density L † m = − ∂ x · [ m ( J − G ) · ∂ x A ] (52) n extremals of the entropy production by “Langevin–Kramers” dynamics A and the probability density m . The right hand side of (52) coincides with the L (2) ( R d ) -dual of the generator of deterministic transport by the vector field a ≡ ( J − G ) · ∂ x A (53)effectively describing a “coarse graining” of the underlying stochastic dynamics.Deterministic transport by (53) arises from the fact that the entropy production is a functionalof the individual probability density specified by the boundary conditions (50). This is atvariance with the stochastic optimal control problems considered in [19, 17] where the costor pay-off functional is a linear functional of the transition probability density of the process.The entropy production optimal control problem belongs instead to the class encompassed bythe “weak-sense” (stochastic) control theory of [22]. We determine extremals of (49) by considering independent variations of m , V and A inthe admissible class (11). The admissible class hypothesis allows us to perform freely allthe integrations by parts needed to extricate space-time local stationary conditions. Afterstraightforward algebra (Appendix B), the variations of m , V and A respectively yield τ ∂ t V + ( A , V ) J † − ( A , V ) G + k ∂ x A k G = 0 (54a) τ ∂ t S + ( A , S ) J † + 1 β S A = 0 (54b) ( S , V ) J † + 1 β S V = 2 β S A (54c)By S we denote in (54a), (54b) the operator S f = − ( S, f ) G + G : ∂ x ⊗ ∂ x f (55) negative definite for any f ∈ L (2) ( R d , m d d x ) (Appendix B). The extremal equations (54)are complemented by the boundary conditions: S ( x , t o ) = − ln m o ( x ) β d & S ( x , t f ) = − ln m f ( x ) β d (56)The value function (54a) and entropy (54b) equations describe deterministic transport by the“coarse-grained” current velocity (53). This latter vector field vanishes at equilibrium, so that(54) in this case admit the physically natural solution ∂ t V = ∂ t S = A = 0 (57)with H = 1 β S (58)The condition (54c) plays for (54) a role analogous to that of pressure in hydrodynamics [47].It enforces a non-local coupling between the microscopic entropy S , and the non-equilibriumHelmholtz energy density A . As in the case of hydrodynamics non-locality arises from the n extremals of the entropy production by “Langevin–Kramers” dynamics V = 2 A analogousto the one minimizing the entropy production by a Langevin–Smoluchowski dynamics [9, 7].Beside non-locality, a second major difference with Langevin–Smoluchowski is that theextremal equations (54) are highly degenerate . Namely, (54c) does not impose any constraintbetween the configuration space projection ∂ q A of the gradient of A and the value function.This is a immediate consequence of the independence of the entropy production from ∂ q A .The generic consequence of degeneration is that (54) describe a continuous family of controls for which the entropy production attains a local, at least, minimum in A . In the comingsection 7 we will illustrate the situation with an explicit example.
7. An analytically solvable case
We can explore more explicitly (54) if we assume a Gaussian statistics for the initial and finalstates of the system. In particular, we restrict the attention to a two-dimensional phase spaceand suppose that the microscopic entropy of the initial i = o and final i = f states be at mostquadratic in x = q ⊕ p : S i ( p, q ) = β ( p − µ p ;i ) σ p ;i cos θ i + β ( q − µ q ;i ) σ q ;i cos θ i − β tan θ i ( p − µ p ;i )( q − µ q ;i ) σ p ;i σ q ;i cos θ i − ln (cid:18) π σ p ;i σ q ;i cos θ i (cid:19) (59)corresponding to E χ t i = " µ q ;i µ p ;i (60)and E ( χ t i − E χ t i ) ⊗ ( χ t i − E χ t i ) = 1 β " σ q ;i σ q ; t i σ p ; t i sin θ t i σ q ;f σ p ;i sin θ t i σ p ; t i (61)In particular, we choose µ p ;i = µ q ;i = θ o = 0 whilst ≤ θ f < π/ parametrizes thedegree of correlation between position and momentum variables of the final state. Underthese assumptions, we look for the solution of the extremal equations by means of quadraticAns¨atze for the microscopic entropy S ( p, q, t ) = β ( p − µ p ; t ) σ p ; t cos θ t + β ( q − µ q ; t ) σ q ; t cos θ t − β tan θ t ( p − µ p ; t )( q − µ q ; t ) σ p ; t σ q ; t cos θ t − ln 12 π σ p ; t σ q ; t cos θ t (62)and the non-equilibrium Helmholtz energy A ( p, q, t ) = A t q + 2 A t p q + A t p a t q + a t p (63) n extremals of the entropy production by “Langevin–Kramers” dynamics t ∈ [ t o , t f ] . The Ans¨atze imply that the entropy production E t f ,t o = β Z t f t o d tτ (cid:8) a t ( A t µ p ; t + A t µ q ; t ) + a t (cid:9) + β Z t f t o d tτ (cid:8) A t (cid:0) µ p ; t + σ p ; t cos θ t (cid:1) + A t (cid:0) µ q ; t + σ q ; t cos θ t (cid:1)(cid:9) + 2 β Z t f t o d tτ A t A t ( µ p ; t µ q ; t + σ p ; t σ q ; t sin θ t ) (64)does not depend explicitly upon A t and a t .Using the quadratic Ans¨atze (62), (63) in (54c) we obtain V ( p, q, t ) = (cid:0) q ∂ p ∂ q − q ∂ p (cid:1) (cid:20) A ( p, q, t ) − y t β S ( p, q, t ) (cid:21) + 2 y t β S ( p, q, t ) + ¯ V ( t ) (65)where y t ≡ β ∂ p A∂ p S (66)is a function of the time variable alone well-defined as long as the probability density of thestate is non-degenerate. The explicit value of ¯ V ( t ) does not play any role in the considerationswhich follow. If we now insert (65) into (54a) and (54b), these equations foliate into a closedsystem of ordinary differential equations for the coefficients of the Ans¨atze (62) and (63). Thecalculation is laborious but straightforward. Upon setting A t = − τ ˙ y t y t (67)we find for the coefficients of second order monomials in (62) and (63) the set of relations A t = y t β (cid:18) ∂ q S − ∂ p ∂ q S − y t ˙ y t ∂ t ∂ p ∂ q S (cid:19) − A t (68a) A t = y t β ∂ p ∂ q S + τ ˙ y t y t (68b) ∂ p S = − τ β ˙ y t y t (68c) ∂ q S = ( ∂ p ∂ q S ) ∂ p S − τ β ¨ y t − y t ... y t y t (68d)The cross correlation coefficient ∂ p ∂ q S of the microscopic entropy enters these equations as afree parameter only subject to the boundary conditions. It turns out that the function y t mustsatisfy the fourth order non-linear differential equation.... y t ˙ y t − y t ¨ y t ... y t + ¨ y t = 0 (69)with solution y t = τ Ω { c + c Ω t + c [sin (Ω t + ϕ ) − sin ϕ ] } (70) n extremals of the entropy production by “Langevin–Kramers” dynamics c , c , Ω , ϕ are fixed by the boundary conditions. Upon imposing theboundary conditions for t o = 0 (71)and requiring continuity of solutions for S f t f ↓ → S o , we get into c = − σ p ;o σ q ;o (72)and c = − σ q ;o ϕ (73)while Ω and ϕ satisfy the transcendental equations: σ p ;f = (cid:8) σ p ;o cos ϕ + σ q ;o [Ω t f + sin(Ω t f + ϕ ) − sin ϕ ] (cid:9)
16 cos θ f cos ϕ cos t f + ϕ (74a) σ q ;f σ q ;o = cos t f + ϕ cos ϕ (74b)We verify that the coefficients of the microscopic entropy are positive definite: ∂ p S = 16 β cos ϕ cos t + ϕ (cid:8) σ p ;o cos ϕ + σ q ;o [Ω t + sin(Ω t + ϕ ) − sin ϕ ] (cid:9) ≥ (75)and ∂ q S = β cos ϕ σ q ;o cos t + ϕ + ( ∂ p ∂ q S ) ∂ p S ≥ (76)The equations for the coefficients of the first degree monomials yield µ q : t = µ q ;f tt f (77)and a t = µ q ;f τ (cid:0) t tan Ω t + ϕ (cid:1) t f − µ p ; t A t − y t µ q ; t β ∂ p ∂ q S (78)The remaining independent equations determine a t as a functional of ∂ p ∂ q S and µ p ; t and theirtime derivatives. We do not need, however, the explicit expression to compute the entropyproduction for which we find E t f ,t o β = µ q ;f τt f + σ q ;o Ω τ t f β cos ϕ (79)Four properties of the extremal value of the entropy production (79) are worth emphasizing.First (79) is fully specified by the boundary conditions and by the degrees of freedom fixedby the extremal equations (54). This fact is an a-posteriori evidence of the degeneration of theextremal protocols. Second, (79) does not depend upon the expected value of the momentumvariable but only upon its variance. The third property is that, (79) corresponds to a constant entropy production rate over the transition horizon. This phenomenon is reminiscent of theLangevin–Smoluchowski case where the entropy production coincides with the kinetic energy n extremals of the entropy production by “Langevin–Kramers” dynamics t f ↑ ∞ . The position variablevariance remains finite in such a limit if Ω t f is finite. This lead us to infer generically a /t f decay of the entropy production in such limit.The explicit dependence of (79) on the boundary conditions can be obtained in severalspecial cases. σ q ;o = σ q ;f The condition is satisfied for
Ω = y t = 0 (80)in the control horizon. Correspondingly, (74a) yields σ p ;f = σ p ;o cos θ f (81)If θ f = 0 , (81) states that, while enforcing (80) we can use ∂ p ∂ q S to steer the system to afinal state with larger momentum variance and non-vanishing correlation between positionand momenta. For vanishing Ω the entropy production is determined by the variation of theposition average: E t f , β = µ q ;f τt f (82)Correspondingly, the non-equilibrium Helmholtz energy and the stochastic entropy can becouched into the form A = µ q ;f τ pt f + σ p ;o q σ q ;o τ dd t tanh θ t − τ qt f (cid:18) µ q ;f + t f ˙ µ p ; t + µ q ;f σ p ;o σ q ;o t dd t tanh θ t (cid:19) (83a) S = β ( p − µ p ; t ) σ p ;o + β σ q ;o cos θ t (cid:18) q − µ q ;f tt f (cid:19) − β tanh θ t σ p ;o σ q ;o ( p − µ p ; t ) (cid:18) q − µ q ;f tt f (cid:19) − ln 12 π σ p ;o σ q ;o (83b)with tanh θ t , σ q ; t and ˙ µ p ; t arbitrary differentiable functions matching the boundary conditions.If we add the requirement θ f = θ t = 0 (83) shows that a transition changing only the meanvalue of the position variable requires a quadratic additive Hamiltonian i.e. of the form H ( p, q, t ) = H p ( p, t ) + H q ( q, t ) where H p ( p, t ) must, however, include a linear momentumdependence . Let us suppose that there exists an non-dimensional parameter ε such that the elements of thecorrelation matrix of the final state admit an expansion of the form σ p ;f = σ p ;o (cid:20) p ε + p ε p ε O ( ε ) (cid:21) (84a) n extremals of the entropy production by “Langevin–Kramers” dynamics σ q ;f = σ q ;o (cid:20) q ε + q ε q ε O ( ε ) (cid:21) (84b)with cos θ f = 1 − w ε O ( ε ) (85)Under the foregoing hypothesis we find Ω = 2 ( q + p ) σ p ;o εt f σ q ;o + 2 ( p − w + q − q ) σ p ;o ε t f σ q ;o + O ( ε ) (86a) ϕ = − q + p ) σ p ;o q σ q ;o − ( q + p ) σ p ;o + [ q ( w + 2 q ) + p q − p q ] σ q ;o (cid:2) ( q + p ) σ q ;o + q σ q ;o (cid:3) σ q ;o σ p ;o ε + O ( ε ) (86b)which give for the entropy production E t f , β = µ q ;f τt f + (cid:2) ( p + q ) σ p ;o + q σ q ;o (cid:3) τ ε β t f + (cid:2) ( p + q ) ( p + q + ( p − q ) q − w ) σ p ;o + q q σ q ;o (cid:3) τ ε β t f + O ( ε ) (87)It is interesting to explore the consequence of this formula in three sub-cases. For this purposewe introduce the non-dimensional parameter λ = σ p ;o σ q ;o (88)measuring the scale separation between momentum and position fluctuations. q n = p n = 0 ∀ n > Both the position and the momentum variances are linear in ε . We can therefore recastthe expansion of the entropy production directly in terms of the change of the variances acrossthe control horizon. We obtain E t f , β = µ q ;f τt f + ( σ q ;f − σ q ;o ) τβ t f + [ σ p ;f − σ p ;o + λ ( σ q ;f − σ q ;o )] τβ t f + [( σ p ;f − σ p ;o ) − λ ( σ q ;f − σ q ;o ) ]( σ q ;f − σ q ;o ) τβ σ q ;o t f + 2 λ [ σ p ;f − σ p ;o + λ ( σ q ;f − σ q ;o )] (1 − cos θ f ) τβ σ q ;o t f + h.o.t (89) θ f = σ p ;f − σ p ;o = 0 and ε = ( σ q ;f − σ q ;o ) /σ q ;o Under these hypotheses, the marginal momentum distribution in the final state coincideswith that of the initial state. As a result, the expansion of the phase ϕ starts from theneighborhood of π/ . The entropy production reduces to E t f , β = µ q ;f τt f + τ (1 + λ ) ( σ q ;f − σ q ;f ) β t f − τ λ ( σ q ;f − σ q ;o ) β σ q ;o t f + O ( σ q ;f − σ q ;o ) (90) n extremals of the entropy production by “Langevin–Kramers” dynamics (a) Momentum variance σ p ; t (b) Position variance σ q ; t for ∂ q ∂ p S = 0 (c) Contribution to the entropyproduction by a change ofposition variance δ = − . δ = − . δ = − . δ =0 . δ =0 . δ =0 . δ = − . δ = − . δ = − . δ =0 . δ =0 . δ =0 . Figure 1.
Evolution of the fluctuation variances for t f = 1 and τ = 0 . . As in subsection 7.2.3we assume σ p ;o = σ p ;f = σ q ;o = 1 and θ f = 0 . Correspondingly, we plot fig. 1(b) afterimposing ∂ q ∂ p S = 0 in the control horizon. The plots correspond to different values of δ = σ q ;f − . In fig 1(a) the constant value of σ p ; t corresponds to σ q ;f = 1 . . The larger theabsolute value of the deviation from unity of σ q ;f , the larger the departure of σ p ; t from the unity inside the control horizon. In fig 1(c) we plot the corresponding values of E t f , − βµ q ;f τ /t f In fig. 1 we report the behavior of the momentum and position variance for vanishingcross-correlation. It is worth emphasizing that the momentum variance does not remainconstant during the control horizon unless σ q ;f = σ q ;o = Ω = 0 (91) θ f = σ p ;f − σ p ;o = 0 and ε = ( σ q ;f − σ q ;o ) /σ q ;o for λ ≪ At variance with the foregoing we now assume a wide scale separation between theposition and momentum variance. We readily see from (86) that ( ϕ , Ω) = ( − π + O ( λ ) , O ( λ )) . We can solve the boundary condition equations (74) in the limit of vanishing λ up to all order accuracy in ε : Ω = 2 λ εt f (cid:26) − ε + 2 ε O ( ε ) (cid:27) λ ↓ → λ εt f [3 + ε (3 + ε )] + o ( λ ) (92a) ϕ = − π + 2 λ (cid:26) − ε + 2 ε O ( ε ) (cid:27) λ ↓ → − π + Ω t f ε + o ( λ ) (92b)The corresponding value of the entropy production is E t f , β = µ q ;f τt f + ( σ q ;f − σ q ;o ) β t f + o ( λ ) (93)We notice that this is exactly the entropy production by a transition governed by a Langevin–Smoluchowski dynamics between Gaussian states [9, 7, 16]. Indeed, the regime we areconsidering here corresponds to the “ over-damped ” asymptotics of the Langevin–Kramersdynamics. Namely, upon inserting (92) into the quadratic Ans¨atze for the non-equilibriumHelmholtz energy and microscopic entropy densities we get into ( ∂ q A )(0 , q, t ) | µ p ; t =0 = − µ q ;f + q ( σ q ;f − σ q ;o ) σ q ;o t ( σ q ;f − σ q ;o ) t f σ q ;o τt f + o ( λ ) (94a) n extremals of the entropy production by “Langevin–Kramers” dynamics ( ∂ p A )(0 , q, t ) | µ p ; t =0 = − ( ∂ p A )(0 , q, t ) | µ p ; t =0 + o ( λ ) (94b) ( ∂ q S )(0 , q, t ) = β (cid:16) q − µ q ;f tt f (cid:17) σ q ;o h t ( σ q ;f − σ q ;o ) t f σ q ;o i + o ( λ ) (94c)We see that the λ -independent parts of (94a) and (94c) coincide with the values obtainedfor the same quantities in the Langevin–Smoluchowski case [9, 7, 16]. In the forthcomingsection, we will show that (54) encapsulate also in general the results obtained forLangevin–Smoluchowski dynamics. In particular, the equality (94b) guarantees that anhomogenization theory “centering condition” holds for the Gaussian model so that theLangevin–Smoluchowski dynamics is recovered as the solution of a suitable “cell problem”[25]. (a) Relative variation of the momen-tum variance for σ p ;o = λ = 1 (b) Relative variation of the momen-tum variance for σ p ;o = λ = 0 . δ = − . δ = − . δ = − . δ =0 . δ =0 . δ =0 . δ = − . δ = − . δ = − . δ =0 . δ =0 . δ =0 . Figure 2.
Relative variation | σ p ; t − σ p ;o | /σ p ;o for σ p ;o = 1 fig. 2(a) and σ p ;o = 0 . fig. 2(a).In both cases σ q ;o = 1 so that σ p ;o = λ . The other parameters as in fig. 2. As λ decreases,the approximation of the marginal momentum distribution by an “equilibrium” distributionimproves its accuracy while remaining not uniform inasmuch the deviation increases with ( σ q ;f − σ q ;o ) and reaches a maximum for t ∼ t f / .
8. “Over-damped” asymptotics
In the presence of a wide separation of between the characteristic scales of the momentum andposition variables, the Langevin–Smoluchowski or “over-damped” dynamics often providesa good approximation to the Langevin–Kramers dynamics. The entropy production by asmooth Langevin–Smoluchowski dynamics attains a minimum value if the control potentialobeys a Monge–Amp`ere–Kantorovich dynamics [8, 9, 7, 16]. In this last section our aim isto investigate in which sense we can recover from (54) the results previously established forLangevin-Smoluchowski dynamics. To address this question, we suppose that the probabilitydensities of the initial and final states take the additive form m o ( p , q ) = (cid:18) β π λ (cid:19) d exp (cid:26) − β k p k d λ − β U o ( q ) (cid:27) (95)and m f ( p , q ) = (cid:18) β π λ (cid:19) d exp (cid:26) − β k p k d λ − β U f ( q ) (cid:27) (96) n extremals of the entropy production by “Langevin–Kramers” dynamics λ ≪ a non-dimensional parameter generalizing (88) in order to describe the scaleseparation between momentum and position dynamics.Multi-scale perturbation theory (often also referred to as homogenization theory see e.g.[24, 25]) in powers of λ equips us with the tools to extricate the asymptotic expression ofsolutions of (54) for β k p k d ≪ λ ≪ in the form A ( x , t ) = X i =0 λ i A ( i ) (cid:16) p λ , q , t . . . (cid:17) + o ( λ ) := ˜ A ( ˜ p , q , t . . . ) (97)and similarly for S and V . The . . . in (97) portend the scales which we eventually neglect inthe asymptotics. Once we availed us of (97), the extremal equations (54) become λ (cid:16) ˜ S , ˜ V (cid:17) ∼ J † + 1 λ β ˜ S ( ˜ V − A ) = 0 (98a) τ ∂ t ˜ V + 1 λ (cid:16) ˜ A , ˜ V (cid:17) ∼ J † − λ (cid:16) ∂ ˜ p ˜ A (cid:17) d · ∂ ˜ p (cid:16) ˜ V − ˜ A (cid:17) = 0 (98b) τ ∂ t ˜ S + 1 λ (cid:16) ˜ A , ˜ S (cid:17) ∼ J † + 1 λ β ˜ S ˜ A = 0 (98c)where we used the notation ˜ x = q ⊕ ˜ p (cid:16) ˜ S , ˜ V (cid:17) ∼ J † ≡ ( ∂ ˜ x ˜ S ) · (cid:16) J † · ∂ ˜ x ˜ V (cid:17) = ( ∂ ˜ p ˜ S ) d · ∂ q ˜ V − ( ∂ q ˜ S ) d · ∂ ˜ p ˜ V (99)and ˜ S := − ( ∂ ˜ p ˜ S ) d · ∂ ˜ p + d : ∂ ˜ p ⊗ ∂ ˜ p (100)In what follows, we will also write ˜ S (0) to denote the replacement in (100) of ˜ S with its zerothorder approximation ˜ S (0) .As often occurs for homogenization of parabolic equation [25], we need to analyze thefirst three orders of the regular expansion in powers of λ in order to fully determine the leadingorder contributions to S and A . This is because the first order is needed to assess the centeringcondition coupling the widely separated scales which we wish to resolve in the asymptotics.The second order approximation uses the information conveyed by the centering conditionto determine the cell problem , a closed equation for the effective dynamics in the limit ofvanishing λ . From (98a) we get the condition ˜ S (0) (cid:0) A (0) − V (0) (cid:1) = 0 (101)stating that at leading order the value function V (0) may differ from the non equilibriumHelmholtz energy at most by a function independent of momentum variables: V (0) = 2 A (0) + V (0:0) (102)where ∂ ˜ p V (0:0) = 0 (103) n extremals of the entropy production by “Langevin–Kramers” dynamics ∂ ˜ p A (0) d · ∂ ˜ p A (0) = 0 (104a) β ˜ S (0) A (0) = 0 (104b)The boundary conditions (95), (96) translate into S o = S f + o ( λ ) (105)Hence, we see from (104) that (105) is satisfied upon setting S (0) ( ˜ p ) = k ˜ p k d S (0:0) ( q , t, . . . ) (106)and ∂ ˜ p A (0) = ∂ ˜ p V (0) = 0 (107) Maxwell momentum distribution is the unique element of the kernel of ˜ S (0) † in L ( R d ) . ByFredholm alternative (see e.g. [25]) (cid:0) S (0) , V (0) (cid:1) ∼ J † − β ˜ S (0) (2 A (1) − V (1) ) = 0 (108)admits a unique solution if and only if the solvability condition Z R d d d p e − S (0) (cid:0) S (0) , V (0) (cid:1) ∼ J † = − Z R d d d p (cid:0) e − S (0) , V (0) (cid:1) ∼ J † = ∂ q Z R d d d p e − S (0) ∂ ˜ p V (0) (109)holds true which is always the case if (107) is verified. Hence we conclude V (1) = 2 (cid:0) A (1) + ˜ p · ∂ q A (0) (cid:1) + ˜ p · ∂ q V (0:0) + V (1:0) (110)with ∂ ˜ p V (1:0) = 0 (111)Turning to the value function equation (98b), we see that − X i =0 ∂ ˜ p V (1 − i ) d · ∂ ˜ p A ( i ) + (cid:0) A (0) , V (0) (cid:1) ∼ J † + 2 ∂ ˜ p A (1) d · ∂ ˜ p A (0) = 0 (112)is also satisfied by (107). New information comes from the expansion of the microscopicentropy equation − ∂ q A (0) · ∂ ˜ p S (0) + ˜ S (0) A (1) = 0 (113)which yields the centering condition of the expansion: A (1) = − ˜ p · ∂ q A (0) (114) n extremals of the entropy production by “Langevin–Kramers” dynamics p with a non-trivial cellproblem in ( q , t ) which we will determine by requiring solvability in the sense of Fredholm’salternative at order O ( λ ) . Contrasting (114) with (110) we infer that ∂ ˜ p V (1) = ∂ q V (0:0) (115)It is worth here to emphasize the relevant simplification induced by the over-damped limit.The over-damped limit entitled us to neglect the Poisson bracket also in the sub-leading order(108) of the expansion of (54c). The crucial consequence is that the relation between V and A remains local within accuracy. Intermediate asymptotics around (104) do not enjoy thisproperty. This is not surprising in light of example of section 7.2.3 showing that, even in theGaussian case, the coincidence of the initial and final marginal momentum distribution doesnot imply in general thermalization. The extremal equation (98a) yields now the condition ∂ ˜ p S (0) d · ∂ q V (1) − ∂ q S (0) d · ∂ q V (0 , − β ˜ S (0) (2 A (2) − V (2) ) = 0 (116)The solvability condition imposes ∂ q V (0 , = 0 (117)whence V (2) = 2 A (2) + ˜ p · ∂ q V (1:0) + V (2:0) (118)with ∂ ˜ p V (2:0) = 0 . Hence, combining (102) with (117), (114) and (105), the value functionequation reduces to τ ∂ t A (0) − ∂ q A (0) d · ∂ q A (0) = 0 (119)Finally, the equation for the microscopic entropy is ∂ t S (0) − ∂ q A (0) d · ∂ q S (0) + ˜ p · ∂ q ⊗ ∂ q A (0) · ∂ ˜ p S (0) + 1 β ˜ S (0) A (2) = 0 (120)Regarding this latter as an equation for A (2) and invoking again Fredholm’s alternative, wesee that it admits a unique solution if and only if τ ∂ t S (0:0) − ∂ q A (0) d · ∂ q S (0:0) + 1 β d : ∂ q ⊗ ∂ q A (0) = 0 (121)holds true. The system formed by the equalities (106), (114) and the cell problem equations(119), (121) fully specifies the homogenization asymptotics we set out to derive. The cell problem equations (119), (121) specify a Monge–Amp`ere–Kantorovich evolution[26] between a initial configuration space state with density ˜ m o ( q ) = (cid:18) β π (cid:19) d/ e − β U o ( q ) (122) n extremals of the entropy production by “Langevin–Kramers” dynamics ˜ m f ( q ) = (cid:18) β π (cid:19) d/ e − β U f ( q ) (123)The recovery of the Monge–Amp`ere–Kantorovich equations unveils the link between theminimum entropy production by the phase space process (1) and the optimal control of thecorresponding thermodynamic quantity which can be directly defined in the over-dampedlimit. As a matter of fact, the expansion of A starts with the O ( λ ) term specified by thecentering condition (114) which is linear in ˜ p = p /λ . The upshot is that the over-dampedexpansion of the minimum over A of the Langevin–Kramers entropy production starts with E t f , = β Z t f d tτ Z R d d d q β d/ e − S (0 , (cid:0) ∂ q A (0) (cid:1) d · ∂ q A (0) + O ( λ ) (124)We therefore proved that the leading order of the expansion coincides with the minimalentropy production by the Langevin–Smoluchowski dynamics.
9. Discussion
Many physical systems are modeled by kinetic-plus-potential Hamiltonians H ( p , q ) = k p k d U ( q , t ) (125)The example of section 7.1 evinces that requiring (125) adds an optimization constraintwhich is not generically satisfied by the extremal equations (54) over A . Furthermore, thekinetic-plus-potential hypothesis deeply affects the control problem by introducing two newdifficulties. First, it restricts to the gradient ∂ q U of the potential energy the available d control degrees of freedom. In this regard, it is worth emphasizing that it is a non-trivialconsequence of H¨ormander theorem (see e.g. [27] and references therein) that a sufficientlyregular (125) is enough to generate a Fokker–Planck evolution of a smooth initial densityfor a Langevin–Kramers dynamics with degenerate noise acting only on d out of d degreesof freedom. Physical intuition suggests, however, that the surmise (125) should not createan insurmountable difficulty for controllability by which we mean the existence of a non-empty set of potentials U ( q , t ) able to steer a transition between two probability densitiesverifying physically plausible assumptions. The second and more substantial difficulty isthat inserting (125) into (25) yields an entropy production expression which depends uponthe control only implicitly through the probability measure. Controls are in such a caseonly subject to the constraint imposed by the requirement of steering a finite-time transitionbetween smooth probability densities. General considerations [17] lead us to envisage thatentropy production may only attain an infimum when evaluated according to a singularcontrol strategy. Such a strategy may take the form of a potential U confining the momentumprocess within a “inactivity region” where U vanishes. We expect the boundary of suchinactivity region to be marked by the the vanishing of the momentum gradient ∂ p V of thevalue function of the corresponding dynamic programming equation. Proving the realizabilityand optimality of such a control strategy are challenges lying beyond the scopes of the n extremals of the entropy production by “Langevin–Kramers” dynamics A , we focused instead on control strategies which we interpret as “macroscopic” in viewthe regularity assumptions on the control Hamiltonian. These assumptions are analogousto those adopted in previous studies of the entropy production by Langevin–Smoluchowskidynamics [7] or by Markov jump processes [11]. We therefore gather that the existence of theentropy production minimum (54), degenerate because of non-coercivity, and which recoversin the over-damped limit the Monge–Amp`ere–Kantorovich evolution, yields a robust generalpicture of the “optimal” thermodynamics for a large class of physical processes described byMarkovian evolution equations. Acknowledgements
It is a pleasure to thank Carlos Mej´ıa–Monasterio for discussions and useful comments onthis manuscript. The work of PMG is supported by by the Center of Excellence “Analysis andDynamics”of the Academy of Finland. The results of this paper were first presented during theconference “ ”Rome, September 23-25, 2013. The author wishes to warmly thank the organizers to give himthe opportunity to partake the event as invited speaker.
AppendicesAppendix A. Mean derivatives and current velocity of a diffusion process
We recall that the drift of an R d -valued diffusion processes ζ ≡ { ζ t , t ∈ [ t o , t f ] } withgenerator L = b · ∂ x + 12 K : ∂ x ⊗ ∂ x (A.1)can be regarded as the mean forward derivative of the process: D x ζ t ≡ lim d t ↓ E ζ t = x ζ t +d t − ζ t d t = L x (A.2)Under standard regularity hypotheses [32], it is possible to define the mean backwardderivative of the very same process as D − x ζ t ≡ lim d t ↓ E ζ t = x ζ t − ζ t − dt d t (A.3) Proposition Appendix A.1.
Let ζ ≡ { ζ t , t ∈ [ t o , t f ] } be a smooth diffusion with generator(A.1) and density m . The mean forward derivative is D − x ζ t = − m ( x , t ) (cid:0) L † x − x L † (cid:1) τ m ( x , t ) (A.4) Proof.
By hypothesis ζ is Markovian with density m in the time interval [ t o , t f ] . Given itsforward transition probability density p the backward transition probability p ∗ of the same n extremals of the entropy production by “Langevin–Kramers” dynamics p ∗ ( x , t | x , t ) = 1 m ( x , t ) p ( x , t | x , t ) m ( x , t ) (A.5)for any x , x ∈ R d , t , t ∈ [ t o , t f ] such that t ≥ t . By (A.5) it follows immediately E ζ t = x ζ t − d t = Z d d x x p ( x , t | x , t − d t ) m ( x , t − d t ) m ( x , t ) (A.6)If we integrate the Fokker-Planck and its adjoint equation over a time horizon of order O (d t ) we arrive at E ζ t = x ζ t − d t = x +d tτ Z d d x x m ( x , t ) L δ (2 d ) ( x − x ) − δ (2 d ) ( x − x ) L † m ( x , t ) m ( x , t ) + O (cid:18) d tτ (cid:19) (A.7)which inserted in the definition (A.3) yields the claim.The mean backward drift governs the Fokker-Planck evolution of the density of theprocess from t f to t o [32]. By H¨ormander theorem [27], the proposition above encompassesthe degenerate noise case described by (1). We are therefore entitled to write τ D − x χ t = J · ∂ x H − G · ∂ x (cid:18) H + 2 β ln m β d (cid:19) (A.8)The current velocity of a smooth diffusion is defined as v ( x , t ) ≡ τ D x + D − x ζ t (A.9)whence (24) follows immediately. The advantage of the current velocity representation is thatthe Fokker-Planck equation for the probability density m in [ t o , t f ] is mapped by (A.9) into thedeterministic mass conservation equation τ ∂ t m + ∂ x · v m = 0 (A.10) Appendix B. Variations of the Pontryagin functional
We avail us of the identity (52) to treat (49) as a functional of the independent fields A and m . The variation of (49) with respect to the costate function being trivial, we restrict herethe attention only to those with respect to the probability density m and the non-equilibriumHelmholtz energy density A . The boundary terms generated by the variation of m vanishbecause of the boundary conditions (50): A ′ m ( m , V, A ) = Z t f t o d tτ Z R d d d x m ′ (cid:8) k ∂ x A k G +[ τ ∂ t + ( ∂ x A ) · ( J † − G ) · ∂ x ] V (cid:9) (B.1)Upon applying the definition of the brackets (14), (15) we arrive at (54a). The variation of A can be couched into the form A ′ A ( m , V, A ) = − Z t f t o d tτ Z R d d d x A ′ ∂ x · m (cid:8) G · ∂ x A − ( J † − G ) · ∂ x V (cid:9) (B.2) n extremals of the entropy production by “Langevin–Kramers” dynamics L (2) ( R d , m d d x ) [48] equipped with the exterior derivative d S = e − S d e S (B.3)Namely it states that the dual d ∗ S to (B.3) must annihilate the -form α = [2 ∂ x A + ( J + G ) · ∂ x V ] · d x (B.4)In terms of the operator (55) the condition translates into (54c). We also notice that that (55) isa degenerate “Witten” Laplacian [48] on the same complex in consequence of the inequality Z R d d d x m f S f = − Z R d d d x m k ∂ x f k G ≤ (B.5)holding for any f ∈ L ( R d , m d d x ) .We end this this appendix with a remark. If the nullspace in L (2) ( R d , m d d x ) of theWitten Laplacian ¯ S = − ( ∂ x S ) · ∂ x + d : ∂ x ⊗ ∂ x (B.6)consists only of constant functions then on the De Rahm–Witten complex (B.3) then currentvelocity (24) admits the Hodge decomposition v = − ∂ x H + h (B.7)where H is a differentiable phase-space function specified by the solution of ¯ S H = − ( S , A ) J † + S A (B.8)and h ≡ e S ∂ x · e − S H (B.9) H being differentiable anti-symmetric rank-two tensor. By construction the elements of thedecomposition in (B.7) are orthogonal in L (2) ( R d , m d d x ) .There are two interesting consequences of (B.7). The first is that mass-transport equationfor m depends only upon H owing to ∂ x · (cid:0) m e S ∂ x · e − S H (cid:1) = β d ∂ x ⊗ ∂ x : e − S H = 0 (B.10)The second is that identifying the the gradient in (B.7) as the dissipative component of thedynamics allows us to define the “entropy production” ˜ E t f ,t o = β Z t f t o d tτ Z R d d d x m k ∂ x H k (B.11)At variance with (25), is a coercive functional of H the optimal control whereof reduces by(B.10) to that of the Langevin–Smoluchowski case in R d . It must be stressed, however, thatcarries different physical information than (25) since this latter depends also on h . n extremals of the entropy production by “Langevin–Kramers” dynamics References [1] Ritort F 2008 Nonequilibrium Fluctuations in Small Systems: From Physics to Biology
Advances inChemical Physics vol 137 ed Rice S A (John Wiley & Sons, Inc., Hoboken, NJ, USA.) (
Preprint )[2] Haynie D T 2008
Biological thermodynamics
Cell
471 – 479 ISSN 0092-8674 URL [4] Bennett C H 1982
International Journal of Theoretical Physics IBM Journal of Research and Development http://researchweb.watson.ibm.com/journal/rd/053/ibmrd0503C.pdf [6] B´erut A, Arakelyan A, Petrosyan A, Ciliberto S, Dillenschneider R and Lutz E 2012 Nature
Journal ofStatistical Physics
Preprint )[8] Aurell E, Mej´ıa-Monasterio C and Muratore-Ginanneschi P 2011
Physical Review Letters
Preprint )[9] Aurell E, Mej´ıa-Monasterio C and Muratore-Ginanneschi P 2012
Phys-ical Review E (2) 020103(R) ( Preprint ) URL http://link.aps.org/doi/10.1103/PhysRevE.85.020103 [10] Gawe¸dzki K 2013 Fluctuation Relations in Stochastic Thermodynamics lecture notes (
Preprint )[11] Muratore-Ginanneschi P, Mej´ıa-Monasterio C and Peliti L 2013
Journal of Statistical Physics
Preprint )[12] Alemany A, Ribezzi M and Ritort F 2011
AIP Conference Proceedings
Preprint ) URL http://link.aip.org/link/?APC/1332/96/1 [13] Sekimoto K 1998
Progress of Theoretical Physics Supplement
Stochastic Energetics ( Lecture Notes in Physics vol 799) (Springer) ISBN-13: 978-3-642-05410-5[15] Jiang D Q, Qian M and Qian M P 2004
Mathematical Theory of Nonequilibrium Steady States ( LectureNotes in Mathematics vol 1833) (Springer) ISBN-13: 978-3-540-20611-8[16] Muratore-Ginanneschi P 2013
Journal of Physics A: Mathematical and General Preprint )[17] Fleming W H and Soner M H 2006
Controlled Markov processes and viscosity solutions
Stochasticmodelling and applied probability vol 25) (Springer) ISBN-13: 978-0-387-26045-7[18] Kullback S and Leibler R 1951
Annals of Mathematical Statistics http://projecteuclid.org/euclid.aoms/1177729694 [19] Fleming W and Rishel R 1975 Deterministic and Stochastic Optimal Control
Applications of Mathematics(Springer) ISBN-13: 978-1-461-26382-1; e-ISBN-13: 978-1-461-26380-7[20] Malliaris A G and Brock W A 1999
Stochastic methods in economics and finance
Advancedtextbooks in economics vol 17) (Elsevier Science) ISBN-13: 978-0-444-86201-3[21] Evans L C An Introduction to Mathematical Optimal Control Theory lecture notes, University of Californiaat Berkeley URL http://math.berkeley.edu/˜evans/control.course.pdf [22] Guerra F and Morato L M 1983
Physical Review D [24] Bensoussan A, Lions J L and Papanicolaou G 1978 Asymptotic Analysis for Periodic Structures ( Studies inmathematics and its applications vol 5) (North-Holland, Amsterdam) ISBN-13: 978-0-444-85172-7[25] Pavliotis G A and Stuart A M 2008
Multiscale methods: averaging and homogenization ( Texts in appliedmathematics vol 53) (Springer) ISBN-13: 978-0-387-73828-4[26] Villani C 2009
Optimal transport: old and new ( Grundlehren der mathematischen Wissenschaften vol 338) n extremals of the entropy production by “Langevin–Kramers” dynamics (Springer) URL [27] Rey-Bellet L 2006 Ergodic Properties of Markov Processes Quantum Open Systems II. TheMarkovian approach ( Lecture Notes in Mathematics vol 1881) (Springer) pp 1–39 URL [28] Shannon C E 1948
The Bell System Technical Journal cm.bell-labs.com/cm/ms/what/shannonday/shannon1948.pdf [29] Morrison P J 1986 Physica D Journal of Physics: Conference Series http://stacks.iop.org/1742-6596/169/i=1/a=012006 [31] Nelson E 1985
Quantum fluctuations
Princeton series in Physics (Princeton University Press) ISBN-13:978-0-691-08379-7 URL https://web.math.princeton.edu/˜nelson/books.html [32] Nelson E 2001
Dynamical Theories of Brownian Motion https://web.math.princeton.edu/˜nelson/books.html [33] Maes C, Redig F and Moffaert A V 2000
Journal of Mathematical Physics Communications in Mathematical Physics
Preprint )[35] Jarzynski C 1997
Physical Review Letters (14) 2690–2693 ( Preprint cond-mat/9610209 )[36] Zambrini J C 1986
Journal of Mathematical Physics http://dx.doi.org/doi/10.1063/1.527002 [37] Klebaner F C 2005 Introduction to stochastic calculus with applications
S´eminaire de probabilit´es de Strasbourg [39] Evans D J and Searles D J 1994 Physical Review E (2) 1645–1648 URL http://link.aps.org/doi/10.1103/PhysRevE.50.1645 [40] Gallavotti G and Cohen E G D 1995 Physical Review Let-ters (14) 2694–2697 ( Preprint chao-dyn/9410007 ) URL http://link.aps.org/doi/10.1103/PhysRevLett.74.2694 [41] Kurchan J 1998
Journal of Physics A: Mathematical and General Preprint cond-mat/9709304 ) URL http://stacks.iop.org/0305-4470/31/i=16/a=003 [42] Lebowitz J L and Spohn H 1999
Journal of Statistical Physics Preprint cond-mat/9811220 ) URL http://arxiv.org/abs/cond-mat/9811220 [43] Crooks G E 1999
Physical Review E (3) 2721–2726 ( Preprint cond-mat/9901352 ) URL http://link.aps.org/doi/10.1103/PhysRevE.60.2721 [44] Ch´etrite R and Gupta S 2011
Journal of Statistical Physics
Preprint )[45] Coulibaly-Pasquier K A 2011
Annales de l’Institut Henri Poincar´e, Prob-abilit´es et Statistiques Preprint hal-00352805 ) URL http://projecteuclid.org/euclid.aihp/1300887280 [46] Struchtrup H 2005
Macroscopic Transport Equations for Rarefied Gas Flows: Approximation Methods inKinetic Theory ( Interaction of Mechanics and Mathematics vol 258) (Springer) ISBN-13: 978-3-540-24542-1[47] Bloch A M, Crouch P E, Holm D D and Marsden J E 2000 An Optimal Control Formulation for InviscidIncompressible Ideal Fluid Flow
Proceedings of the 39th IEEE Conference on Decision and Control (IEEE) (
Preprint nlin/0103042 )[48] Helffer B 2002
Semiclassical Analysis, Witten Laplacians and Statistical Mechanics ( Series on PartialDifferential Equations and Applications vol 1) (World Scientific Publishing Company) ISBN-13: 978-981-238-098-2vol 1) (World Scientific Publishing Company) ISBN-13: 978-981-238-098-2