Jakob Ruess

Moment-based inference predicts bimodality in transient gene expression

Recent computational studies indicate that the molecular noise of a cellular process may be a rich source of information about process dynamics and parameters. However, accessing this source requires stochastic models that are usually difficult to analyze. Therefore, parameter estimation for stochastic systems using distribution measurements, as provided for instance by flow cytometry, currently remains limited to very small and simple systems. Here we propose a new method that makes use of low-order moments of the measured distribution and thereby keeps the essential parts of the provided information, while still staying applicable to systems of realistic size. We demonstrate how cell-to-cell variability can be incorporated into the analysis obviating the need for the ubiquitous assumption that the measurements stem from a homogeneous cell population. We demonstrate the method for a simple example of gene expression using synthetic data generated by stochastic simulation. Subsequently, we use time-lapsed flow cytometry data for the osmo-stress induced transcriptional response in budding yeast to calibrate a stochastic model, which is then used as a basis for predictions. Our results show that measurements of the mean and the variance can be enough to determine the model parameters, even if the measured distributions are not well-characterized by low-order moments only—e.g., if they are bimodal.

Designing experiments to understand the variability in biochemical reaction networks

Exploiting the information provided by the molecular noise of a biological process has proved to be valuable in extracting knowledge about the underlying kinetic parameters and sources of variability from single-cell measurements. However, quantifying this additional information a priori, to decide whether a single-cell experiment might be beneficial, is currently only possible in systems where either the chemical master equation is computationally tractable or a Gaussian approximation is appropriate. Here, we provide formulae for computing the information provided by measured means and variances from the first four moments and the parameter derivatives of the first two moments of the underlying process. For stochastic kinetic models for which these moments can be either computed exactly or approximated efficiently, the derived formulae can be used to approximate the information provided by single-cell distribution experiments. Based on this result, we propose an optimal experimental design framework which we employ to compare the utility of dual-reporter and perturbation experiments for quantifying the different noise sources in a simple model of gene expression. Subsequently, we compare the information content of a set of experiments which have been performed in an engineered light-switch gene expression system in yeast and show that well-chosen gene induction patterns may allow one to identify features of the system which remain hidden in unplanned experiments.

Iterative experiment design guides the characterization of a light-inducible gene expression circuit

Significance System identification addresses the problem of identifying unknown model parameters from measured data of a real system. In the case of biochemical reaction networks, the available measurements are typically sparse because of technical and/or economic reasons. Therefore, it is of paramount importance to maximize the information that can be gained by each experiment. Here, we apply a systematic design scheme for single-cell experiments based on information theoretic criteria. For the considered light-inducible gene expression circuit, we show that this scheme allows one to precisely identify model parameters that were practically unidentifiable from data measured in random experiments. This result provides evidence that optimal experiment design is a key requirement for the successful identification of biochemical reaction networks. Systems biology rests on the idea that biological complexity can be better unraveled through the interplay of modeling and experimentation. However, the success of this approach depends critically on the informativeness of the chosen experiments, which is usually unknown a priori. Here, we propose a systematic scheme based on iterations of optimal experiment design, flow cytometry experiments, and Bayesian parameter inference to guide the discovery process in the case of stochastic biochemical reaction networks. To illustrate the benefit of our methodology, we apply it to the characterization of an engineered light-inducible gene expression circuit in yeast and compare the performance of the resulting model with models identified from nonoptimal experiments. In particular, we compare the parameter posterior distributions and the precision to which the outcome of future experiments can be predicted. Moreover, we illustrate how the identified stochastic model can be used to determine light induction patterns that make either the average amount of protein or the variability in a population of cells follow a desired profile. Our results show that optimal experiment design allows one to derive models that are accurate enough to precisely predict and regulate the protein expression in heterogeneous cell populations over extended periods of time.

Moment estimation for chemically reacting systems by extended Kalman filtering.

In stochastic models of chemically reacting systems that contain bimolecular reactions, the dynamics of the moments of order up to n of the species populations do not form a closed system, in the sense that their time-derivatives depend on moments of order n + 1. To close the dynamics, the moments of order n + 1 are generally approximated by nonlinear functions of the lower order moments. If the molecule counts of some of the species have a high probability of becoming zero, such approximations may lead to imprecise results and stochastic simulation is the only viable alternative for system analysis. Stochastic simulation can produce exact realizations of chemically reacting systems, but tends to become computationally expensive, especially for stiff systems that involve reactions at different time scales. Further, in some systems, important stochastic events can be very rare and many simulations are necessary to obtain accurate estimates. The computational cost of stochastic simulation can then be prohibitively large. In this paper, we propose a novel method for estimating the moments of chemically reacting systems. The method is based on closing the moment dynamics by replacing the moments of order n + 1 by estimates calculated from a small number of stochastic simulation runs. The resulting stochastic system is then used in an extended Kalman filter, where estimates of the moments of order up to n, obtained from the same simulation, serve as outputs of the system. While the initial motivation for the method was improving over the performance of stochastic simulation and moment closure methods, we also demonstrate that it can be used in an experimental setting to estimate moments of species that cannot be measured directly from time course measurements of the moments of other species.

Moment-Based Methods for Parameter Inference and Experiment Design for Stochastic Biochemical Reaction Networks

Continuous-time Markov chains are commonly used in practice for modeling biochemical reaction networks in which the inherent randomness of the molecular interactions cannot be ignored. This has motivated recent research effort into methods for parameter inference and experiment design for such models. The major difficulty is that such methods usually require one to iteratively solve the chemical master equation that governs the time evolution of the probability distribution of the system. This, however, is rarely possible, and even approximation techniques remain limited to relatively small and simple systems. An alternative explored in this article is to base methods on only some low-order moments of the entire probability distribution. We summarize the theory behind such moment-based methods for parameter inference and experiment design and provide new case studies where we investigate their performance.

Identifying stochastic biochemical networks from single-cell population experiments: A comparison of approaches based on the Fisher information

In biochemical reaction networks stochasticity arising from molecular fluctuations often plays an important role. Recent years have seen an increasing number of studies which employed single-cell population experiments and used the measured stochasticity to estimate the parameters of stochastic kinetic models. Currently, there exist two approaches for extracting information about the model parameters from the observed variability. One uses the complete distribution of the measured population and is usually computationally very expensive, whereas the other uses only low-order moments and thereby sacrifices a part of the information in order to reduce the computational cost. Here, using three qualitatively different benchmark models, we investigate on the one hand how much information can be gained by exploiting the single-cell resolution of population experiments and on the other hand how much information is lost if only low-order moments of the measured distributions are considered. Our study allows one to better understand the advantage of single-cell population experiments and to gain insights into when and where moment-based parameter inference methods for their analysis are appropriate.

Adaptive Moment Closure for Parameter Inference of Biochemical Reaction Networks

Continuous-time Markov chain (CTMC) models have become a central tool for understanding the dynamics of complex reaction networks and the importance of stochasticity in the underlying biochemical processes. When such models are employed to answer questions in applications, in order to ensure that the model provides a sufficiently accurate representation of the real system, it is of vital importance that the model parameters are inferred from real measured data. This, however, is often a formidable task and all of the existing methods fail in one case or the other, usually because the underlying CTMC model is high-dimensional and computationally difficult to analyze. The parameter inference methods that tend to scale best in the dimension of the CTMC are based on so-called moment closure approximations. However, there exists a large number of different moment closure approximations and it is typically hard to say a priori which of the approximations is the most suitable for the inference procedure. Here, we propose a moment-based parameter inference method that automatically chooses the most appropriate moment closure method. Accordingly, contrary to existing methods, the user is not required to be experienced in moment closure techniques. In addition to that, our method adaptively changes the approximation during the parameter inference to ensure that always the best approximation is used, even in cases where different approximations are best in different regions of the parameter space.

Shaping bacterial population behavior through computer-interfaced control of individual cells

Bacteria in groups vary individually, and interact with other bacteria and the environment to produce population-level patterns of gene expression. Investigating such behavior in detail requires measuring and controlling populations at the single-cell level alongside precisely specified interactions and environmental characteristics. Here we present an automated, programmable platform that combines image-based gene expression and growth measurements with on-line optogenetic expression control for hundreds of individual Escherichia coli cells over days, in a dynamically adjustable environment. This integrated platform broadly enables experiments that bridge individual and population behaviors. We demonstrate: (i) population structuring by independent closed-loop control of gene expression in many individual cells, (ii) cell–cell variation control during antibiotic perturbation, (iii) hybrid bio-digital circuits in single cells, and freely specifiable digital communication between individual bacteria. These examples showcase the potential for real-time integration of theoretical models with measurement and control of many individual cells to investigate and engineer microbial population behavior.Individual bacteria interact with each other and their environment to produce population-level patterns of gene expression. Here the authors use an automated platform combined with optogenetic feedback to manipulate population behaviors through dynamic control of individual cells.

Minimal moment equations for stochastic models of biochemical reaction networks with partially finite state space

Many stochastic models of biochemical reaction networks contain some chemical species for which the number of molecules that are present in the system can only be finite (for instance due to conservation laws), but also other species that can be present in arbitrarily large amounts. The prime example of such networks are models of gene expression, which typically contain a small and finite number of possible states for the promoter but an infinite number of possible states for the amount of mRNA and protein. One of the main approaches to analyze such models is through the use of equations for the time evolution of moments of the chemical species. Recently, a new approach based on conditional moments of the species with infinite state space given all the different possible states of the finite species has been proposed. It was argued that this approach allows one to capture more details about the full underlying probability distribution with a smaller number of equations. Here, I show that the result that less moments provide more information can only stem from an unnecessarily complicated description of the system in the classical formulation. The foundation of this argument will be the derivation of moment equations that describe the complete probability distribution over the finite state space but only low-order moments over the infinite state space. I will show that the number of equations that is needed is always less than what was previously claimed and always less than the number of conditional moment equations up to the same order. To support these arguments, a symbolic algorithm is provided that can be used to derive minimal systems of unconditional moment equations for models with partially finite state space.

Grey-box techniques for the identification of a controlled gene expression model

The aim of this paper is to propose a computationally efficient technique for the identification of stochastic biochemical networks, involving only zero and first order reactions, from distribution measurements of the cell population. The moments of the species in such networks evolve according to an affine system, hence the use of grey-box identification methods is suggested. The performance of existing methods and of a new method, based on the transfer function computation, is compared using as benchmark a standard gene expression model. The developed discussion is of interest for the general grey-box identification problem.