Mikael Sunnåker

Approximate Bayesian Computation

Approximate Bayesian computation (ABC) constitutes a class of computational methods rooted in Bayesian statistics. In all model-based statistical inference, the likelihood function is of central importance, since it expresses the probability of the observed data under a particular statistical model, and thus quantifies the support data lend to particular values of parameters and to choices among different models. For simple models, an analytical formula for the likelihood function can typically be derived. However, for more complex models, an analytical formula might be elusive or the likelihood function might be computationally very costly to evaluate. ABC methods bypass the evaluation of the likelihood function. In this way, ABC methods widen the realm of models for which statistical inference can be considered. ABC methods are mathematically well-founded, but they inevitably make assumptions and approximations whose impact needs to be carefully assessed. Furthermore, the wider application domain of ABC exacerbates the challenges of parameter estimation and model selection. ABC has rapidly gained popularity over the last years and in particular for the analysis of complex problems arising in biological sciences (e.g., in population genetics, ecology, epidemiology, and systems biology).

Automatic generation of predictive dynamic models reveals nuclear phosphorylation as the key Msn2 control mechanism.

Topological filtering identifies biological networks compatible with known data and enables quantitative analysis of regulatory mechanisms. Reducing the Options Quantitative analysis of signaling systems is challenging because limited quantitative data are available and the data can be represented by many network models. Sunnåker et al. developed a computational approach called topological filtering to systematically and automatically integrate modeling and data acquisition to infer the set of mechanistically plausible models, thus vastly reducing the number of potential models. The approach iteratively eliminates reactions from the model to identify only those topological networks that fit the data. Application of their method to an extracellular signal–regulated kinase (ERK) pathway that could be represented by 512 possible network topologies reduced the possibilities to 16 and showed that a set of feedback reactions were necessary to quantitatively represent the results. Topological filtering applied to the regulation of the localization of Msn2, a yeast transcription factor controlled by phosphorylation by PKA (protein kinase A) in response to changes in glucose abundance, identified a single model that fit the data. Comparison of model predictions with experimental data showed that the nuclear phosphorylation rate was key to controlling Msn2 nuclear abundance in response to cAMP (cyclic adenosine monophosphate), a signal produced as cells recover from glucose starvation. Predictive dynamical models are critical for the analysis of complex biological systems. However, methods to systematically develop and discriminate among systems biology models are still lacking. We describe a computational method that incorporates all hypothetical mechanisms about the architecture of a biological system into a single model and automatically generates a set of simpler models compatible with observational data. As a proof of principle, we analyzed the dynamic control of the transcription factor Msn2 in Saccharomyces cerevisiae, specifically the short-term mechanisms mediating the cells’ recovery after release from starvation stress. Our method determined that 12 of 192 possible models were compatible with available Msn2 localization data. Iterations between model predictions and rationally designed phosphoproteomics and imaging experiments identified a single-circuit topology with a relative probability of 99% among the 192 models. Model analysis revealed that the coupling of dynamic phenomena in Msn2 phosphorylation and transport could lead to efficient stress response signaling by establishing a rate-of-change sensor. Similar principles could apply to mammalian stress response pathways. Systematic construction of dynamic models may yield detailed insight into nonobvious molecular mechanisms.

A method for zooming of nonlinear models of biochemical systems

BackgroundModels of biochemical systems are typically complex, which may complicate the discovery of cardinal biochemical principles. It is therefore important to single out the parts of a model that are essential for the function of the system, so that the remaining non-essential parts can be eliminated. However, each component of a mechanistic model has a clear biochemical interpretation, and it is desirable to conserve as much of this interpretability as possible in the reduction process. Furthermore, it is of great advantage if we can translate predictions from the reduced model to the original model.ResultsIn this paper we present a novel method for model reduction that generates reduced models with a clear biochemical interpretation. Unlike conventional methods for model reduction our method enables the mapping of predictions by the reduced model to the corresponding detailed predictions by the original model. The method is based on proper lumping of state variables interacting on short time scales and on the computation of fraction parameters, which serve as the link between the reduced model and the original model. We illustrate the advantages of the proposed method by applying it to two biochemical models. The first model is of modest size and is commonly occurring as a part of larger models. The second model describes glucose transport across the cell membrane in bakers yeast. Both models can be significantly reduced with the proposed method, at the same time as the interpretability is conserved.ConclusionsWe introduce a novel method for reduction of biochemical models that is compatible with the concept of zooming. Zooming allows the modeler to work on different levels of model granularity, and enables a direct interpretation of how modifications to the model on one level affect the model on other levels in the hierarchy. The method extends the applicability of the method that was previously developed for zooming of linear biochemical models to nonlinear models.

Near-optimal experimental design for model selection in systems biology

Motivation: Biological systems are understood through iterations of modeling and experimentation. Not all experiments, however, are equally valuable for predictive modeling. This study introduces an efficient method for experimental design aimed at selecting dynamical models from data. Motivated by biological applications, the method enables the design of crucial experiments: it determines a highly informative selection of measurement readouts and time points. Results: We demonstrate formal guarantees of design efficiency on the basis of previous results. By reducing our task to the setting of graphical models, we prove that the method finds a near-optimal design selection with a polynomial number of evaluations. Moreover, the method exhibits the best polynomial-complexity constant approximation factor, unless P = NP. We measure the performance of the method in comparison with established alternatives, such as ensemble non-centrality, on example models of different complexity. Efficient design accelerates the loop between modeling and experimentation: it enables the inference of complex mechanisms, such as those controlling central metabolic operation. Availability: Toolbox ‘NearOED’ available with source code under GPL on the Machine Learning Open Source Software Web site (mloss.org). Contact: busettoa@inf.ethz.ch Supplementary information: Supplementary data are available at Bioinformatics online.

Investigations of a compartmental model for leucine kinetics using non-linear mixed effects models with ordinary and stochastic differential equations.

Non-linear mixed effects (NLME) models represent a powerful tool to simultaneously analyse data from several individuals. In this study, a compartmental model of leucine kinetics is examined and extended with a stochastic differential equation to model non-steady-state concentrations of free leucine in the plasma. Data obtained from tracer/tracee experiments for a group of healthy control individuals and a group of individuals suffering from diabetes mellitus type 2 are analysed. We find that the interindividual variation of the model parameters is much smaller for the NLME models, compared to traditional estimates obtained from each individual separately. Using the mixed effects approach, the population parameters are estimated well also when only half of the data are used for each individual. For a typical individual, the amount of free leucine is predicted to vary with a standard deviation of 8.9% around a mean value during the experiment. Moreover, leucine degradation and protein uptake of leucine is smaller, proteolysis larger and the amount of free leucine in the body is much larger for the diabetic individuals than the control individuals. In conclusion, NLME models offers improved estimates for model parameters in complex models based on tracer/tracee data and may be a suitable tool to reduce data sampling in clinical studies.

Zooming of states and parameters using a lumping approach including back-translation

BackgroundSystems biology models tend to become large since biological systems often consist of complex networks of interacting components, and since the models usually are developed to reflect various mechanistic assumptions of those networks. Nevertheless, not all aspects of the model are equally interesting in a given setting, and normally there are parts that can be reduced without affecting the relevant model performance. There are many methods for model reduction, but few or none of them allow for a restoration of the details of the original model after the simplified model has been simulated.ResultsWe present a reduction method that allows for such a back-translation from the reduced to the original model. The method is based on lumping of states, and includes a general and formal algorithm for both determining appropriate lumps, and for calculating the analytical back-translation formulas. The lumping makes use of efficient methods from graph-theory and ϵ-decomposition and is derived and exemplified on two published models for fluorescence emission in photosynthesis. The bigger of these models is reduced from 26 to 6 states, with a negligible deviation from the reduced model simulations, both when comparing simulations in the states of the reduced model and when comparing back-translated simulations in the states of the original model. The method is developed in a linear setting, but we exemplify how the same concepts and approaches can be applied to non-linear problems. Importantly, the method automatically provides a reduced model with back-translations. Also, the method is implemented as a part of the systems biology toolbox for matlab, and the matlab scripts for the examples in this paper are available in the supplementary material.ConclusionsOur novel lumping methodology allows for both automatic reduction of states using lumping, and for analytical retrieval of the original states and parameters without performing a new simulation. The two models can thus be considered as two degrees of zooming of the same model. This is a conceptually new development of model reduction approaches, which we think will stimulate much further research and will prove to be very useful in future modelling projects.

Topological augmentation to infer hidden processes in biological systems

Motivation: A common problem in understanding a biochemical system is to infer its correct structure or topology. This topology consists of all relevant state variables—usually molecules and their interactions. Here we present a method called topological augmentation to infer this structure in a statistically rigorous and systematic way from prior knowledge and experimental data. Results: Topological augmentation starts from a simple model that is unable to explain the experimental data and augments its topology by adding new terms that capture the experimental behavior. This process is guided by representing the uncertainty in the model topology through stochastic differential equations whose trajectories contain information about missing model parts. We first apply this semiautomatic procedure to a pharmacokinetic model. This example illustrates that a global sampling of the parameter space is critical for inferring a correct model structure. We also use our method to improve our understanding of glutamine transport in yeast. This analysis shows that transport dynamics is determined by glutamine permeases with two different kinds of kinetics. Topological augmentation can not only be applied to biochemical systems, but also to any system that can be described by ordinary differential equations. Availability and implementation: Matlab code and examples are available at: http://www.csb.ethz.ch/tools/index. Contact: mikael.sunnaker@bsse.ethz.ch; andreas.wagner@ieu.uzh.ch Supplementary information: Supplementary data are available at Bioinformatics online.

Model Extension and Model Selection

In this chapter we are concerned with the topic of construction , assessment, and selection of models in general, and of biochemical models in particular. Standard approaches to model construction and (automated) generation of candidate models are first discussed. We then present the most commonly used methods for model assessment, as well as the underlying concepts and ideas. In particular we focus on the information theoretic and Bayesian approaches to model selection. Information theoretic methods for model selection include the Akaike information criterion and the more recent deviance information criterion. Bayesian approaches include the computation of posterior ratios for relative model probabilities from Bayes factors as well as the approximate Bayesian information criterion. We also briefly discuss other methods such as cross-validation and bootstrapping techniques, and the theoretically appealing approach of minimum description length. We sketch how the most important results can be derived, emphasize distinctions between the methods, and discuss how model inference methods are employed in practice. We conclude that there is no generally applicable method for model assessment: a suitable choice depends on the specific inference problem, and to some extent also on the subjective preferences of the modeler.

Efficient Characterization of Parametric Uncertainty of Complex (Bio)chemical Networks

Parametric uncertainty is a particularly challenging and relevant aspect of systems analysis in domains such as systems biology where, both for inference and for assessing prediction uncertainties, it is essential to characterize the system behavior globally in the parameter space. However, current methods based on local approximations or on Monte-Carlo sampling cope only insufficiently with high-dimensional parameter spaces associated with complex network models. Here, we propose an alternative deterministic methodology that relies on sparse polynomial approximations. We propose a deterministic computational interpolation scheme which identifies most significant expansion coefficients adaptively. We present its performance in kinetic model equations from computational systems biology with several hundred parameters and state variables, leading to numerical approximations of the parametric solution on the entire parameter space. The scheme is based on adaptive Smolyak interpolation of the parametric solution at judiciously and adaptively chosen points in parameter space. As Monte-Carlo sampling, it is “non-intrusive” and well-suited for massively parallel implementation, but affords higher convergence rates. This opens up new avenues for large-scale dynamic network analysis by enabling scaling for many applications, including parameter estimation, uncertainty quantification, and systems design.

Computational Design of Informative Experiments in Systems Biology

Accurate predictions of the behavior of biological systems can be achieved through multiple iterations of modeling and experimentation. In this chapter, we present the central ideas for the design of informative experiments in systems biology. We start by formalizing the task, and proceed by introducing the required tools to process data subject to uncertainty. We analyze design approaches which are Bayesian and information-theoretic in nature. A particular emphasis is placed on implicit and explicit assumptions of the available techniques. Two main design goals are here compared: reducing uncertainty and challenging existing belief. Finally, we discuss the limitations of the presented approaches to provide general guidelines for predictive modeling.