Dan Goreac

Asymptotic Control for a Class of Piecewise Deterministic Markov Processes Associated to Temperate Viruses

We aim at characterizing the asymptotic behavior of value functions in the control of piece-wise deterministic Markov processes (PDMP) of switch type under nonexpansive assumptions. For a particular class of processes inspired by temperate viruses, we show that uniform limits of discounted problems as the discount decreases to zero and time-averaged problems as the time horizon increases to infinity exist and coincide. The arguments allow the limit value to depend on initial configuration of the system and do not require dissipative properties on the dynamics. The approach strongly relies on viscosity techniques, linear programming arguments and coupling via random measures associated to PDMP. As an intermediate step in our approach, we present the approximation of discounted value functions when using piecewise constant (in time) open-loop policies.

Algebraic invariance conditions in the study of approximate (null-)controllability of Markov switch processes

We aim at studying approximate null-controllability properties of a particular class of piecewise linear Markov processes (Markovian switch systems). The criteria are given in terms of algebraic invariance and are easily computable. We propose several necessary conditions and a sufficient one. The hierarchy between these conditions is studied via suitable counterexamples. Equivalence criteria are given in abstract form for general dynamics and algebraic form for systems with constant coefficients or continuous switching. The problem is motivated by the study of lysis phenomena in biological organisms and price prediction on spike-driven commodities.

A note on the controllability of jump diffusions with linear coefficients

We investigate the approximate controllability property for a class of linear stochastic equations driven by independent Brownian motion and Poisson random measure. The paper generalizes recent results of Buckdahn et al. (2006, A characterization of approximately controllable linear stochastic differential equations. Stochastic Partial Differential Equations and Applications (G. Da Prato & L. Tubaro eds). Series of Lecture Notes in Pure and Applied Mathematics, vol. 245. London: Chapman & Hall, pp. 253–260) and Goreac (2008, A Kalman-type condition for stochastic approximate controllability. C. R. Math. Acad. Sci. Paris, 346, 183–188; 2009, Approximate controllability for linear stochastic differential equations in infinite dimensions. Appl. Math. Optim., 60, 105–132). An equivalent conditional invariance criterion is given. In general, this (explicit) criterion involves an L2-space, but, for particular cases, an iterative finite scheme is provided.

Optimality Issues for a Class of Controlled Singularly Perturbed Stochastic Systems

The present paper aims at studying stochastic singularly perturbed control systems. We begin by recalling the linear (primal and dual) formulations for classical control problems. In this framework, we give necessary and sufficient support criteria for optimality of the measures intervening in these formulations. Motivated by these remarks, in a first step, we provide linearized formulations associated with the value function in the averaged dynamics setting. Second, these formulations are used to infer criteria allowing to identify the optimal trajectory of the averaged stochastic system.

Controllability Properties of Linear Mean-Field Stochastic Systems

We study some controllability properties for linear stochastic systems of mean-field type. First, we give necessary and sufficient criteria for exact terminal-controllability. Second, we characterize the approximate and approximate null-controllability via duality techniques. Using Riccati equations associated to linear quadratic problems in the control of mean-field systems, we provide a (conditional) viability criterion for approximate null-controllability. In the classical diffusion framework, approximate and approximate null-controllability are equivalent. This is no longer the case for mean-field systems. We provide sufficient (algebraic) invariance conditions implying approximate null-controllability. We also present a general class of systems for which our criterion is equivalent to approximate null-controllability property. We also introduce some rank conditions under which approximate and approximate null-controllability are equivalent. Several examples and counter-examples as well as a partial algorithm are provided.

Some Applications of Linear Programming Formulations in Stochastic Control

We present two applications of the linearization techniques in stochastic optimal control. In the first part, we show how the assumption of stability under concatenation for control processes can be dropped in the study of asymptotic stability domains. Generalizing Zubov’s method, the stability domain is then characterized as some level set of a semicontinuous generalized viscosity solution of the associated Hamilton–Jacobi–Bellman equation. In the second part, we extend our study to unbounded coefficients and apply the method to obtain a linear formulation for control problems whenever the state equation is a stochastic variational inequality.

Controllability Issues for Randomly Switching Piecewise Linear Markov Processes * *The author wishes to acknowledge support from ANR project PIECE, ANR-12-JS01-0006

We investigate some stability properties of approximate null-controllability (A0C). We show that A0C is inherited when increasing the number of observed jumps. However, in all generality, there is no direct connection between A0C with an initial mode and the A0C starting from directly accessible modes. We have also given a sufficient condition under which checking A0C implies AC and discuss some examples. Unfortunately, such conditions are too strong and do not cover all AC systems (see examples). In a second part, we give a Riccati technique-based viability kernel characterisation of A0C. Some conditions are inferred for AC.

A Piecewise Deterministic Markov Toy Model for Traffic/Maintenance and Associated Hamilton---Jacobi Integrodifferential Systems on Networks

We study optimal control problems in infinite horizon whxen the dynamics belong to a specific class of piecewise deterministic Markov processes constrained to star-shaped networks (corresponding to a toy traffic model). We adapt the results in Soner (SIAM J Control Optim 24(6):1110–1122, 1986) to prove the regularity of the value function and the dynamic programming principle. Extending the networks and Krylov’s “shaking the coefficients” method, we prove that the value function can be seen as the solution to a linearized optimization problem set on a convenient set of probability measures. The approach relies entirely on viscosity arguments. As a by-product, the dual formulation guarantees that the value function is the pointwise supremum over regular subsolutions of the associated Hamilton–Jacobi integrodifferential system. This ensures that the value function satisfies Perron’s preconization for the (unique) candidate to viscosity solution.

Some support considerations in the asymptotic optimality of two-scale controlled PDMP

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés. Some Support Considerations in the Asymptotic Optimality of Two-Scale Controlled PDMP D Goreac, Oana Silvia Serea

Viability of stochastic semi-linear control systems via the quasi-tangency condition

In this paper, we study a criterion for the viability of stochastic semi-linear control systems on a real, separable Hilbert space. The necessary and sufficient conditions are given using the notion of stochastic quasi-tangency. As a consequence, we prove that approximate viability and the viability property coincide for stochastic linear control systems. We obtain Nagumos stochastic theorem and we present a method allowing to provide explicit criteria for the viability of smooth sets. We analyse the conditions characterizing the viability of the unit ball. The paper generalizes recent results from the deterministic framework.