Sean Summers

In silico feedback for in vivo regulation of a gene expression circuit

We show that difficulties in regulating cellular behavior with synthetic biological circuits may be circumvented using in silico feedback control. By tracking a circuits output in Saccharomyces cerevisiae in real time, we precisely control its behavior using an in silico feedback algorithm to compute regulatory inputs implemented through a genetically encoded light-responsive module. Moving control functions outside the cell should enable more sophisticated manipulation of cellular processes whenever real-time measurements of cellular variables are possible.

Verification of discrete time stochastic hybrid systems: A stochastic reach-avoid decision problem

We present a dynamic programming based solution to a probabilistic reach-avoid problem for a controlled discrete time stochastic hybrid system. We address two distinct interpretations of the reach-avoid problem via stochastic optimal control. In the first case, a sum-multiplicative cost function is introduced along with a corresponding dynamic recursion which quantifies the probability of hitting a target set at some point during a finite time horizon, while avoiding an unsafe set during each time step preceding the target hitting time. In the second case, we introduce a multiplicative cost function and a dynamic recursion which quantifies the probability of hitting a target set at the terminal time, while avoiding an unsafe set during the preceding time steps. In each case, optimal reach while avoid control policies are derived as the solution to an optimal control problem via dynamic programming. Computational examples motivated by two practical problems in the management of fisheries and finance are provided.

A Multiresolution Approximation Method for Fast Explicit Model Predictive Control

A model predictive control law is given by the solution to a parametric optimization problem that can be pre-computed offline and provides an explicit map from state to control input. In this paper, an algorithm is introduced based on wavelet multiresolution analysis that returns a low complexity explicit model predictive control law built on a hierarchy of second-order interpolets. The resulting interpolation is shown to be everywhere feasible and continuous. Further, tests to confirm stability and to compute a bound on the performance loss are introduced. Since the controller approximation is built on a grid hierarchy, convergence to a stabilizing control law is guaranteed and the evaluation of the control law in real-time systems is naturally fast and runs in a bounded logarithmic time. Two examples are provided; A two-dimensional example with an evaluation speed of 31 ns and a four-dimensional example with an evaluation speed of 119 ns.

On the connections between PCTL and dynamic programming

Probabilistic Computation Tree Logic (PCTL) is a well-known modal logic which has become a standard for expressing temporal properties of finite-state Markov chains in the context of automated model checking. In this paper, we consider PCTL for noncountable-space Markov chains, and we show that there is a substantial affinity between certain of its operators and problems of Dynamic Programming. We prove some basic properties of the solutions to the latter. We also provide two examples and demonstrate how recovery strategies in practical applications, which are naturally stated as reach-avoid problems, can be viewed as particular cases of PCTL formulas.

A stochastic reach-avoid problem with random obstacles

We present a dynamic programming based solution to a stochastic reachability problem for a controlled discrete-time stochastic hybrid system. A sum-multiplicative cost function is introduced along with a corresponding dynamic recursion which quantifies the probability of hitting a target set at some point during a finite time horizon, while avoiding an obstacle set during each time step preceding the target hitting time. In contrast with earlier works which consider the reach and avoid sets as both deterministic and time invariant, we consider the avoid set to be both time-varying and probabilistic. Optimal reach-avoid control policies are derived as the solution to an optimal control problem via dynamic programming. A computational example motivated by aircraft motion planning is provided.

Stochastic dynamics of genetic networks

MOTIVATION Identification of regulatory networks is typically based on deterministic models of gene expression. Increasing experimental evidence suggests that the gene regulation process is intrinsically random. To ensure accurate and thorough processing of the experimental data, stochasticity must be explicitly accounted for both at the modelling stage and in the design of the identification algorithms. RESULTS We propose a model of gene expression in prokaryotes where transcription is described as a probabilistic event, whereas protein synthesis and degradation are captured by first-order deterministic kinetics. Based on this model and assuming that the network of interactions is known, a method for estimating unknown parameters, such as synthesis and binding rates, from the outcomes of multiple time-course experiments is introduced. The method accounts naturally for sparse, irregularly sampled and noisy data and is applicable to gene networks of arbitrary size. The performance of the method is evaluated on a model of nutrient stress response in Escherichia coli.

Moment estimation for chemically reacting systems by extended Kalman filtering.

In stochastic models of chemically reacting systems that contain bimolecular reactions, the dynamics of the moments of order up to n of the species populations do not form a closed system, in the sense that their time-derivatives depend on moments of order n + 1. To close the dynamics, the moments of order n + 1 are generally approximated by nonlinear functions of the lower order moments. If the molecule counts of some of the species have a high probability of becoming zero, such approximations may lead to imprecise results and stochastic simulation is the only viable alternative for system analysis. Stochastic simulation can produce exact realizations of chemically reacting systems, but tends to become computationally expensive, especially for stiff systems that involve reactions at different time scales. Further, in some systems, important stochastic events can be very rare and many simulations are necessary to obtain accurate estimates. The computational cost of stochastic simulation can then be prohibitively large. In this paper, we propose a novel method for estimating the moments of chemically reacting systems. The method is based on closing the moment dynamics by replacing the moments of order n + 1 by estimates calculated from a small number of stochastic simulation runs. The resulting stochastic system is then used in an extended Kalman filter, where estimates of the moments of order up to n, obtained from the same simulation, serve as outputs of the system. While the initial motivation for the method was improving over the performance of stochastic simulation and moment closure methods, we also demonstrate that it can be used in an experimental setting to estimate moments of species that cannot be measured directly from time course measurements of the moments of other species.

Fast explicit nonlinear model predictive control via multiresolution function approximation with guaranteed stability

In this paper an algorithm for nonlinear explicit model predictive control is introduced based on multiresolution function approximation that returns a low complexity approximate receding horizon control law built on a hierarchy of second order interpolets. Feasibility and stability guarantees for the approximate control law are given using reachability analysis, where interval methods are used to construct a capture basin (feasible region). A constructive algorithm is provided that combines adaptive function approximation with interval methods to build a receding horizon control law that is suboptimal, yet with a region of guaranteed feasibility and stability. The resulting control law is built on a grid hierarchy that is fast to evaluate in real-time systems.

Approximate dynamic programming via sum of squares programming

We describe an approximate dynamic programming method for stochastic control problems on infinite state and input spaces. The optimal value function is approximated by a linear combination of basis functions with coefficients as decision variables. By relaxing the Bellman equation to an inequality, one obtains a linear program in the basis coefficients with an infinite set of constraints. We show that a recently introduced method, which obtains convex quadratic value function approximations, can be extended to higher order polynomial approximations via sum of squares programming techniques. An approximate value function can then be computed offline by solving a semidefinite program, without having to sample the infinite constraint. The policy is evaluated online by solving a polynomial optimization problem, which also turns out to be convex in some cases. We experimentally validate the method on an autonomous helicopter testbed using a 10-dimensional helicopter model.

A stochastic games framework for verification and control of discrete time stochastic hybrid systems

We describe a framework for analyzing probabilistic reachability and safety problems for discrete time stochastic hybrid systems within a dynamic games setting. In particular, we consider finite horizon zero-sum stochastic games in which a control has the objective of reaching a target set while avoiding an unsafe set in the hybrid state space, and a rational adversary has the opposing objective. We derive an algorithm for computing the maximal probability of achieving the control objective, subject to the worst-case adversary behavior. From this algorithm, sufficient conditions of optimality are also derived for the synthesis of optimal control policies and worst-case disturbance strategies. These results are then specialized to the safety problem, in which the control objective is to remain within a safe set. We illustrate our modeling framework and computational approach using both a tutorial example with jump Markov dynamics and a practical application in the domain of air traffic management.