On structural and practical identifiability
Franz-Georg Wieland, Adrian L. Hauber, Marcus Rosenblatt, Christian Tönsing, Jens Timmer
OOn structural and practical identifiability
Franz-Georg Wieland a,b,c , Adrian L. Hauber a,b , Marcus Rosenblatt a,b , Christian T¨onsing a,b,c , Jens Timmer a,b,c, ∗ a Institute of Physics, University of Freiburg, Hermann-Herder-Str. 3, 79104 Freiburg, Germany b Freiburg Center for Data Analysis and Modelling (FDM), University of Freiburg, Ernst-Zermelo-Str. 1, 79104 Freiburg, Germany c Centre for Integrative Biological Signalling Studies (CIBSS), University of Freiburg, Sch¨anzlestr. 18, 79104 Freiburg, Germany
Abstract
We discuss issues of structural and practical identifiability of partially observed di ff erential equations which are oftenapplied in systems biology. The development of mathematical methods to investigate structural non-identifiabilityhas a long tradition. Computationally e ffi cient methods to detect and cure it have been developed recently. Practicalnon-identifiability on the other hand has not been investigated at the same conceptually clear level. We argue thatpractical identifiability is more challenging than structural identifiability when it comes to modelling experimentaldata. We discuss that the classical approach based on the Fisher information matrix has severe shortcomings. Asan alternative, we propose using the profile likelihood, which is a powerful approach to detect and resolve practicalnon-identifiability. Highlights • With recent advances structural identifiability is no longer a major issue • Practical identifiability is still challenging • Fisher information matrix is misleading • Profile likelihood can solve practical identifiability challenge
Keywords:
Identifiability, Structural identifiability, Practical identifiability, Profile likelihood, Fisher informationmatrix, Non-linear dynamics, ODE models, Experimental design, Model reduction, Observability ∗ Corresponding author
Email address: [email protected] (Jens Timmer)
Preprint submitted to Current Opinion in Systems Biology February 11, 2021 a r X i v : . [ s t a t . M E ] F e b raphical abstractIntroduction Biological modelling and Box’s statement
Traditional biological reasoning often is rather qualitative, descriptive, and static, which results e.g. in cell biologyin so-called “pathway cartoons”. Mathematical models based on di ff erential equations can help to turn these into aquantitative, predictive, and dynamic understanding of the underlying system. Discussing modelling in general, in1979, George E.P. Box coined his famous statement: “All models are wrong, but some are useful” [1]. While theformer part of the quote is intuitively clear, since every model necessarily poses a simplification of reality, the latterhighlights the importance of assessing what constitutes a useful model. Bad, good and useful models
Three properties comprise a useful model. Firstly, it has to capture the main e ff ects of the question of interest, i.e.describe the data with reasonable accuracy, and neglect the rest. Secondly, a useful model has to make experimentallyfalsifiable predictions in order to be testable. Models that exhibit these two properties are good models. Thirdly, themodel should enable to gather insights about the biological system. In a typical modelling process, one starts o ff withan initial model based on current biological knowledge. Usually, this model cannot explain the data and therefore isa bad model. Based on biological intuition and trial-and-error, one increases the model complexity until the data can2e fitted. Often, this leads to an over-parameterised model that over-fits the data. The parameters of such a model andin turn its predictions are not well-determined and it thus remains a bad model.The path from such a bad model towards a good model is laborious: additional data needs to be measured andintegrated, the model complexity needs to be reduced and balanced to the available data, or a combination of both.This process needs to be iterated until a good model is found, which has well determined parameters and predictions.However, such a good model also needs to deliver biological insights in order to be useful . Only this third propertyturns a good model into a useful model. In this sense, the final goal of mathematical modelling in systems biologyis not the model itself but to use the model to understand biology. One example of how a model can be used to gainbiological insight, which would be unattainable by merely assessing the data by itself, was given by Becker et al. [2]. Parameter identifiability
The concept of identifiability is strongly linked to the transition from bad models to good models. Identifiabilityanalysis is necessary to create good models that can describe the data and have well-determined parameters andpredictions. It is especially important when modelling biological systems because the limited amount and qualityof the experimental data with large measurement noise in only partially observed systems often leads to bad modelsduring the modelling process. Concerning identifiability, one distinguishes between structural identifiability dealingwith inherently indeterminable parameters due to the model structure itself, and practical identifiability, dealing withinsu ffi ciently informative measurements to determine the parameters with adequate precision. Partially observed dynamical systems
A biological system is translated into ordinary di ff erential equations (ODEs)˙ x = f ( x , p , u ) , (1)comprising n model states x ( t ), unknown parameters p to be estimated from time-resolved experimental data, andexternal stimuli u ( t ). Since data is often recorded on a relative scale, scaling and o ff set parameters for background cor-rections need to be estimated in parallel. Furthermore, in typical applications not all components of a cell-biologicalsystem can be measured, e.g. because of the limited availability or restricted capability of antibodies to discriminatebetween un-phosphorylated, i.e. inactive, and phosphorylated, i.e. active, proteins. Thus, an observation function g ( · )is required that maps the internal states x to the observations: y = g ( x , p , t ) . (2)Typically, the dimension m of y is smaller than the dimension n of x . We are therefore dealing with parameterestimation in partially observed systems. Moreover, in systems biology, these ODE models are typically sti ff , non-linear, sparse and non-autonomous, and the discrete time observations are noisy.3arameter estimation is usually performed based on the weighted residual sum of squares, the negative log-likelihood assuming Gaussian errors χ ( p ) = m (cid:88) k = d k (cid:88) l = y Dkl − g k ( p , t l ) σ Dkl , (3)to determine the agreement of experimental data with the model trajectories, where y Dkl and σ Dkl represent d k data pointsand measurement errors at time points t l for each observable. A common point estimate for the best parameter vectoris the maximum likelihood estimatorˆ p = arg min (cid:104) χ ( p ) (cid:105) . (4) Structural identifiability
Definition of structural identifiability and connection to observability
Partially observed dynamical systems often exhibit structural non-identifiability. Structural identifiability is theability to uniquely estimate parameters from any given model output. A parameter p i is globally structurally identifi-able [3], if for all parameter vectors p , it holds y ( p ) = y ( p (cid:48) ) ⇒ p i = p (cid:48) i . (5)An individual parameter p i is structurally non-identifiable, if changing the parameter does not alter the model trajec-tory y , because the changes can be fully compensated by altering other parameters. Local structural identifiability ofa parameter is defined by reducing the definition to a neighbourhood v ( p ) instead of the entire parameter space. Amodel is structurally identifiable, if all of its parameters are structurally identifiable. Multiple related definitions forstructural identifiability exist, for a comprehensive discussion see a recent overview [4].A structurally non-identifiable parameter implies the existence of a manifold in parameter space upon which thetrajectory y is unchanged. However, on this manifold the dynamic variables x of the model can change, e.g. by ascaling factor, and are thus not uniquely determinable. This is denoted as non-observability, a concept closely relatedto parameter non-identifiability [5, 6, 7, 8, 9]. A priori analysis of structural identifiability
Two basic approaches exist to assess structural identifiability of non-linear dynamic models.
A priori methodsonly use the model definition, while a posteriori methods use the available data to find non-identifiable parameters.Many a priori algorithms have been developed based on a variety of approaches. Powerful methods use Lie grouptheory, since non-identifiabilities are closely related to symmetries in the system [10, 11, 12, 13]. Furthermore, avariety of notable methods exist, which are based on power series expansion [14], generating series [15, 16], semi-numerical approaches [17, 18], di ff erential algebra [19, 20, 21, 22, 23, 24, 25, 26, 27], di ff erential geometry [28],4nd numerical algebraic geometry [29]. For reviews of some of these approaches, see [28, 30, 31]. Many of theseapproaches, especially the early developed methods, can only be applied to rather low-dimensional systems becauseof their computational complexity. Thus, recent developments have mainly focused on improving the computationale ffi ciency of the algorithms, e.g. by local sensitivity calculations.As a promising example, Joubert et al. [32] proposed a comprehensive and computationally fast pipeline to curestructural non-identifiabilities by re-parameterisation of the model in a five-step procedure: (i) a numerical identi-fiability analysis based on sensitivities, (ii) symbolic identifiability calculations for the low-dimensional candidatesfrom (i), this renders the procedure fast, (iii) computation of new model parameters, this step is not unique, but re-quires decisions of the modeler, (iv) simplify the original model leading to a lower dimensional parameter vector, andfinally (v) check the identifiability of the re-parameterised model. In an application to a model with 21 states and75 parameters, two groups of non-identifiable parameters were detected and the model was re-parameterised withinminutes. Analysis of structural identifiability using experimental data
In contrast to the aforementioned methods, a posteriori methods use the available data to perform identifiabilityanalysis. They infer structural non-identifiability based on model fits to experimental data. Similar to some sensitivity-based a priori approaches, these approaches only assess local structural identifiability.One approach by Hengl et al. [33] suggested to perform numerous fits and investigate non-parametrically whetherthe final parameter estimates form a low-dimensional manifold in parameter space. This approach also allows todisentangle di ff erent sets of coupled non-identifiable parameters.An informative and successful method is based on the profile likelihood [34]. The idea of the profile likelihood isto vary one parameter p i after the other around the maximum likelihood estimate (Equation (4)) and re-optimise theremaining onesPL ( p i ) = min p j (cid:44) i (cid:104) χ ( p ) (cid:105) . (6)For the two-parameter examples in Figure 1, the blue dashed lines show the path in the parameter space determined byEquation (6). Figure 1A shows the profile likelihood of an identifiable parameter. For a structurally non-identifiableparameter the profile likelihood yields a flat line as shown in Figure 1B. Plotting the remaining parameters along theprofiled parameter reveals which parameters are coupled to the non-identifiable one [35]. The profile likelihood wasrecently extended to include two-dimensional profiles to allow for the identification of parameter interdependence[36].Profile likelihood calculation can be computationally demanding for larger systems due to the numerical re-optimisation. Addressing this issue, a fast a posteriori method to test identifiability without the need to calculatecomplete profiles using radial penalisation was recently developed [37].5 igure 1: Likelihood contour plots and profile likelihood for an identifiable parameter and structurally and practically non-identifiableparameters.
Subfigures (A), (B), and (C) show contour plots of χ above as well as the profile likelihood versus the parameter below. Lightercolours in the contour plots signify a lower value of χ . Thresholds for confidence intervals corresponding to a confidence level of 95 % are shownin red and plotted both in the contour plots and the profile likelihood plots. The lowest value of χ is denoted by a grey asterisk in both the contourplot and the profile likelihood plot. For the identifiable parameter (A) the profile likelihood reaches both an upper and lower threshold thus leadingto a finite confidence interval. For the structurally non-identifiable parameter (B) the profile likelihood is completely flat, thus yielding infiniteconfidence intervals. In the contour plot this translates to a flat path, along which χ does not change. The practically non-identifiable parameter(C) shows an infinite extension of the low χ region for lower values of the parameter, never reaching the 95 % confidence interval threshold. Incontrast, a finite upper confidence bound can be derived. Structural non-identifiability can also be investigated a posteriori by a Bayesian Markov chain Monte Carlo(MCMC) sampling approach. However, for non-identifiable systems e ffi cient mixing and thus convergence of theMarkov chains is di ffi cult [38]. This problem can be cured by informative priors but these would mask the problemand should only be implemented if they are based on actual biological insights and prior information. One recentapplication in the field identified a minimal subset of reactions in a signalling network with a combination of paralleltempering and LASSO regression methods [39]. Re-parameterising structurally non-identifiable models
Given the recent advances in the computational e ffi ciency of methods, we essentially consider determining struc-tural identifiability no longer a bottleneck in the modelling of non-linear dynamic systems with ODEs. When thestructurally non-identifiable parameters are determined, the problem is usually fixed by a re-parameterisation of themodel. In the simplest case this is accomplished by fixing some of the involved parameters to a certain value. Theprice to be paid is typically that the information about the scale of some components is lost. Nevertheless, biolog-6cally meaningful re-parameterisation of the models after finding non-identifiabilites remains a challenging task (G.Massonis et al. , arXiv:2012.09826v2). Practical identifiability
From structural to practical identifiability
Structural identifiability implies practical identifiability only for an infinite amount of data with zero noise. Prac-tical identifiability is important for obtaining precise parameter estimates. Moreover, it is especially crucial to en-sure that model predictions are well-determined. It is analyzed increasingly often to judge a model’s predictivity[40, 41, 42, 43, 44]. The notion of practical identifiability has been rather vague in the literature, mainly referring to large confidence intervals [45, 46, 47]. Some approaches exist that define practical identifiability as a combinationof model structure and experimental protocol without actual data [48, 49]. In contrast, we consider a combination ofmodel and data as practically identifiable if the confidence intervals of all estimated parameters are of finite size [35].
Parameter confidence intervals and identifiability
The profile likelihood (Equation (6)) provides a proper assessment of confidence intervals of estimated parametersin ODE models (Figure 1) by CI PL ( p i ) = (cid:8) p i | PL ( p i ) ≤ χ ( ˆ p ) + ∆ α (cid:9) , (7)where ∆ α denotes the α quantile of the χ distribution with df = igure 2: Parameter confidence intervals based on Fisher information matrix and profile likelihood.
Confidence intervals for five parametersbased on profile likelihood (blue) and on quadratic approximation using the Fisher information matrix (FIM) (orange). FIM-based confidenceintervals have two major problems. Firstly, due to the non-linearity of the underlying systems, the Cram´er-Rao bound on the error is invalid andthus the FIM-based confidence intervals become uncontrollable for a finite amount of measurements. While in (A) the FIM-based interval is largerthan the profile likelihood-based interval, in (B) it is smaller. Secondly, FIM based intervals are insensitive to practical non-identifiabilities. In (C),the FIM-based confidence interval is finite and thus the practically non-identifiable parameter is not detected. In (D), the practically non-identifiableparameter leads to a flat FIM-based interval, wrongly suggesting structural non-identifiability. While the structurally non-identifiable parameterin (E) is correctly detected, similarly to (D) the calculation of the FIM is challenging due to its singularity in flat likelihood landscapes. Theparameters (A)-(D) are adapted from two applications in synthetic biology ((A),(B),(C) from Schneider et al. [52], (D) from Ochoa-Fernandezet al. [51]). Parameter (E) is from a minimal non-identifiable toy model. Grey asterisks signify the maximum likelihood estimate of the parameter.
Bayesian methods for identifiability analysis
Bayesian sampling approaches, e.g. MCMC, can be used to assess practical identifiability [53, 54, 55]. This,however is only feasible if the model is structurally identifiable, since structural non-identifiabilities will lead to badmixing of the sampling algorithms. Given a structurally identifiable model, MCMC sampling yields similar resultsas the profile likelihood analysis [38]. However, a recent application in a spatio-temporal reaction–di ff usion model8howed, that it is one order of magnitude slower than the profile likelihood [56]. Experimental design and model reduction
Model predictions
To test the predictive power of a model, confidence intervals for the predictions can be computed. For this purpose,forward evaluations of the model are utilized, e.g. bootstrap approaches [57] or sensitivity analysis [58]. They typicallyrequire large numerical e ff orts in the context of non-linear biological models with a high-dimensional parameter space.A more powerful approach is the prediction profile likelihood PPL ( z ) = min p ∈{ p | g pred ( p ) = z } (cid:104) χ ( p ) (cid:105) , (8)which is obtained by minimising χ ( p ) (Equation (3)) under the constraint that the model response g pred ( p ) is equalto the prediction z . The prediction profile likelihood propagates the uncertainty from the experimental data to theprediction by exploring the prediction space instead of the parameter space [59].If the model predictions are not of su ffi cient precision, one has two principal options to tailor the model complexityto the information content of the data: (i) measure additional data, corresponding to an increase of the dimensionof the observation function g in Equation (2), or (ii) reduce the model complexity according to the available data,corresponding to a decrease of the dimension of the parameter space and / or of the ODE system f in Equation (1).Both options increase the practical identifiability of the model. Achieving practical identifiability by new measurements with optimal experimental design
Practical identifiability can be achieved by adding new data [44, 60]. The process of determining the most infor-mative targets and time points for the new measurements is known as optimal experimental design and is frequentlyapplied in di ff erent modelling fields, e.g. metabolic models [61], animal science [62], linear perturbation networks[63] or synthetic biology [64]. The task is related to the search of an additional measurement that contains the max-imal information about the system or parts of it. For improving the identifiability of a specific parameter, the modeltrajectories along the corresponding parameter profile can be investigated [65, 66]. Thereby, measurement points withmaximal information content for the parameter of interest can be determined, which corresponds to trajectories withhigh spread. Similarly, the prediction profile likelihood (Equation (8)) determines the prediction uncertainty of themodel at a potential new measurement time point [59] thus promoting the identifiability of the whole model. Mea-surement points with high prediction uncertainty are e ff ective to constrain the model further, whereas measurementswith a low prediction uncertainty are better suited for model selection purposes.9 chieving practical identifiability by reducing model complexity If measuring additional data is not feasible, the complexity of the model has to be reduced. One way is to fixparameter values or ratios of parameters by means of prior knowledge [67], sensitivity analysis [68, 69] or profilelikelihood [70]. However, fixing parameters can decrease the interpretative relevance of the model’s predictions.Taking this into account, a systematic model reduction strategy that tailors model complexity to the available datawas suggested by Maiwald et al. [71]. Based on likelihood profiles, they discuss four basic scenarios that are dis-criminated based on the profile likelihood by the combinations of: either (i) the profile flattens out for a logarithmisedparameter going to infinity or (ii) to minus infinity, and either other parameters are (a) coupled to the investigatedone or (b) not. For all four possible combinations, there is a cure. For case (i / a), one di ff erential equation is replacedby an algebraic equation, for (i / b), states can be lumped, for (ii / a), a variable is fixed, leading to a structural non-identifiability that can be cured by the methods discussed above, and for (ii / b), a reaction can be removed from themodel. This model reduction strategy has been applied e.g. in [51, 52, 72]. Independent of the applied method, modelreduction steps, and in particular the conclusions thereof, should always be documented together with the modelaccording to good scientific practice to facilitate reproducibility. Conclusions
Given the multitude of recently developed methods [13, 16, 27, 32], we consider the file of identifying structurallynon-identifiable parameters as closed. Future research in this field could focus on identifying biologically plausiblere-parameterisations of the model, for which no comprehensive method yet exists. Furthermore, the extension of theconcept of identifiability to di ff erent model types, e.g. mixed e ff ects models [73, 74], is of interest.Achieving practical identifiability for model and data is more laborious in practice. Practical non-identifiabilitiescan be detected reliably, e.g. by the profile likelihood method [31]. In order to achieve identifiability, the modelcomplexity has to be reduced or additional data must be added. Profile likelihood-based model reduction [71] andoptimal experimental design [66] provide valuable methods for these purposes. A flowchart locating structural andpractical identifiability analysis as discussed in this review within the entire modelling process is given in Figure 3.Although the availability of advanced methods for the detection and cure of structural and practical non-identifiabilitiesis promising, two related challenges remain. In many applications identifiability analysis is not performed with state-of-the-art methods. Particularly, identifiability analysis based on the Fisher information matrix can be misleading intypical applications in systems biology. We propose a more consequent use of the discussed methods for structuralidentifiability and especially profile likelihood for practical identifiability analysis in order to check the limitationsand predictive power of mathematical models. In summary, we believe the focal point of research in systems biologyshould always remain on the biological insights that can be gained from mathematical models which are structurallyand practically identifiable. 10 igure 3: Flowchart of the entire modelling process from inital to final model including identifiability analysis.
The modelling process beginswith the inception of an initial model based on prior knowledge and the underlying biological research question. It ends with the final validatedmodel and the biological insights it provides. The flowchart shows, how identifiability analysis is embedded into the overall modelling workflow.The topics discussed in this review related to structural identifiability (blue) and practical identifiability (red) are highlighted with colours in theflowchart. The remaining tiles in gray represent aspects that are beyond the scope of this review. The intricacy of the flowchart shows, that thepath to biological insights requires multiple iterations of di ff erent methods. Identifiability analysis is an integral part of this workflow and shouldbe performed to gain insights from predictive models with well-determined parameters. Furthermore, methods dealing with structural and practicalidentifiability should always be focused on ultimately progressing along the path towards biological insights. Conflict of interest statement
The authors declare that they have no known competing financial interests or personal relationships that couldhave appeared to influence the work reported in this paper.11 cknowledgements
This work was supported by the German Research Foundation (DFG) under Germany’s Excellence Strategy(CIBSS – EXC-2189 – Project ID 390939984), the German Research Foundation (DFG) through grant 272983813 / TRR179, the Deutsche Krebshilfe (grant 70112355), the German Federal Ministry for Education and Research within theresearch network Systems Medicine of the Liver (LiSyM; grant 031L0048), and by the state of Baden-W¨urttembergthrough bwHPC and the German Research Foundation (DFG) through grant INST 35 / References [1] G. E. P. Box, Robustness in the strategy of scientific model building, in: R. L. Launer, G. N. Wilkinson (Eds.), Robustness in Statistics,Academic Press, New York, 1979, pp. 201–236. doi: .[2] V. Becker, M. Schilling, J. Bachmann, U. Baumann, A. Raue, T. Maiwald, J. Timmer, U. Klingm¨uller, Covering a broad dynamic range:Information processing at the erythropoietin receptor, Science 328 (2010) 1404–1408. doi: .[3] E. Walter, L. Pronzato, Identification of Parametric Models from Experimental Data, Springer, London, U.K., 1997.[4] F. Anstett-Collin, L. Denis-Vidal, G. Mill´erioux, A priori identifiability: An overview on definitions and approaches, Annual Reviews inControl 50 (2020) 139–149. doi: .*Rigorously assessing 21 definitions of a priori identifiability, the authors provide an overview over analytical and algebraicdefinitions as well as local and global definitions and their dependencies.[5] R. Kalman, On the general theory of control systems, IRE Transactions on Automatic Control 4 (1959) 110–110. doi: .[6] R. Bellman, K. Åstr¨om, On structural identifiability, Mathematical Biosciences 7 (1970) 329–339. doi: .[7] M. J. Chappell, K. R. Godfrey, S. Vajda, Global identifiability of the parameters of nonlinear systems with spezified inputs: a comparison ofmethods, Math. Biosciences 102 (1990) 41–73. doi: .[8] A. F. Villaverde, N. Tsiantis, J. R. Banga, Full observability and estimation of unknown inputs, states and parameters of nonlinear biologicalmodels, Journal of The Royal Society Interface 16 (2019) 20190043. doi: .*A comprehensive approach to observability analysis is provided in this paper. This includes assessing inputs, states and pa-rameters. Structural parameter identifiability is assessed with an approach based on di ff erential geometry.[9] T. Schmitt, B. Ritter, Data-based identifiability and observability assessment for nonlinear control systems using the profile likelihood method,in: 21st IFAC World Congress, Germany, 2020. doi: .[10] J. W. Yates, N. D. Evans, M. J. Chappell, Structural identifiability analysis via symmetries of di ff erential equations, Automatica 45 (2009)2585–2591. doi: .[11] B. Merkt, D. Kaschek, J. Timmer, Higher-order Lie symmetries in identifiability and predictability analysis of dynamic models, PhysicalReview E 92 (2015) 012920. doi: .[12] A. F. Villaverde, N. D. Evans, M. J. Chappell, J. R. Banga, Input-dependent structural identifiability of nonlinear systems, IEEE ControlSystems Letters 3 (2019) 272–277. doi: .[13] G. Massonis, A. F. Villaverde, Finding and breaking Lie symmetries: Implications for structural identifiability and observability in biologicalmodelling, Symmetry 12 (2020) 469. doi: . *The authors provide an improved algorithm to find model symmetries using Lie algebra that is computationally more e ffi cientand thus applicable to higher dimensional models.[14] H. Pohjanpalo, System identifiability based on the power series expansion of the solution, Mathematical Biosciences 41 (1978) 21–33.doi: .[15] E. Walter, Y. Lecourtier, Global approaches to identifiability testing for linear and nonlinear state space models, Mathematics and Computersin Simulation 24 (1982) 472–482. doi: .[16] T. S. Ligon, F. Fr¨ohlich, O. T. Chis¸, J. R. Banga, E. Balsa-Canto, J. Hasenauer, GenSSI 2.0: Multi-experiment structural identifiabilityanalysis of SBML models, Bioinformatics 34 (2018) 1421–1423. doi: .*The authors provide an open-source Matlab toolbox for structural identifiability analysis and the reparametrization of struc-turally non-identifiable models. They use the generating series approach and support Systems Biology Markup Language im-port.[17] A. Sedoglavic, A probabilistic algorithm to test local algebraic observability in polynomial time, Journal of Symbolic Computation 33 (2002)735–755. doi: .[18] J. Karlsson, M. Anguelova, M. Jirstrand, An e ffi cient method for structural identifiability analysis of large dynamic systems, IFAC Proceed-ings Volumes 45 (2012) 941–946. doi: .[19] L. Ljung, T. Glad, On global identifiability for arbitrary model parametrizations, Automatica 30 (1994) 265–276. doi: .[20] M. P. Saccomani, S. Audoly, L. D’Angi`o, Parameter identifiability of nonlinear systems: The role of initial conditions, Automatica 39 (2003)619–632. doi: .[21] G. Bellu, M. P. Saccomani, S. Audoly, L. D’Angi`o, DAISY: A new software tool to test global identifiability of biological and physiologicalsystems, Computer Methods and Programs in Biomedicine 88 (2007) 52–61. doi: .[22] N. Meshkat, C. E.-Z. Kuo, J. DiStefano, On finding and using identifiable parameter combinations in nonlinear dynamic systems biologymodels and COMBOS: A novel web implementation, PLoS ONE 9 (2014) e110261. doi: .[23] K. Thomaseth, M. P. Saccomani, Local identifiability analysis of nonlinear ODE models: How to determine all candidate solutions, IFAC-PapersOnLine 51 (2018) 529–534. doi: .[24] A. Varghese, S. Narasimhan, N. Bhatt, A priori parameter identifiability in complex reaction networks, IFAC-PapersOnLine 51 (2018)760–765. doi: .[25] M. P. Saccomani, G. Bellu, S. Audoly, L. D’Angi´o, A new version of DAISY to test structural identifiability of biological models, in:Computational Methods in Systems Biology, 2019, pp. 329–334. doi: .[26] H. Hong, A. Ovchinnikov, G. Pogudin, C. Yap, SIAN: Software for structural identifiability analysis of ODE models, Bioinformatics 35(2019) 2873–2874. doi: .[27] H. Hong, A. Ovchinnikov, G. Pogudin, C. Yap, Global identifiability of di ff erential models, Communications on Pure and Applied Mathe-matics 73 (2020) 1831–1879. doi: .**Combining a probabilistic and analytical criterion in one algorithm, the authors perform a priori structural identifiabilityanalysis e ffi ciently and quickly. The algorithm is available in the SIAN Maple software package.[28] A. F. Villaverde, Observability and structural identifiability of nonlinear biological systems, Complexity 2019 (2019) 1–12. doi: .[29] D. J. Bates, J. D. Hauenstein, N. Meshkat, Identifiability and numerical algebraic geometry, PLOS ONE 14 (2019) e0226299. doi: .[30] O. T. Chis, J. R. Banga, E. Balsa-Canto, Structural identifiability of systems biology models: A critical comparison of methods, PLoS ONE6 (2011) e27755. doi: .
31] A. Raue, J. Karlsson, M. P. Saccomani, M. Jirstrand, J. Timmer, Comparison of approaches for parameter identifiability analysis of biologicalsystems, Bioinformatics 30 (2014) 1440–1448. doi: .[32] D. Joubert, J. D. Stigter, J. Molenaar, An e ffi cient procedure to assist in the re-parametrization of structurally unidentifiable models, Mathe-matical Biosciences 323 (2020) 108328. doi: .**Using a combination of numerical and algebraic methods, the authors provide an incredibly fast algorithm for assessingstructural identifiability. Their approach uses sensitivity analysis to provide candidates for further symbolic analysis, beforecomputing new algebraic relations for the non-identifiable parameters.[33] S. Hengl, C. Kreutz, J. Timmer, T. Maiwald, Data-based identifiability analysis of non-linear dynamical models, Bioinformatics 23 (2007)2612–2618. doi: .[34] S. A. Murphy, A. W. van der Vaart, On profile likelihood, Journal of the American Statistical Association 95 (2000) 449. doi: .[35] A. Raue, C. Kreutz, T. Maiwald, J. Bachmann, M. Schilling, U. Klingm¨uller, J. Timmer, Structural and practical identifiability anal-ysis of partially observed dynamical models by exploiting the profile likelihood, Bioinformatics 25 (2009) 1923–1929. doi: .[36] O. M. Brastein, B. Lie, R. Sharma, N.-O. Skeie, Parameter estimation for externally simulated thermal network models, Energy and Buildings191 (2019) 200–210. doi: .*Extension of the profile likelihood method for 2D likelihood profiles that allow for analysis of pairwise parameter interdepen-dence.[37] C. Kreutz, An easy and e ffi cient approach for testing identifiability, Bioinformatics 34 (2018) 1913–1921. doi: .**Fast alternative method to test structural and practical identifiability without the need to calculate the profile likelhood foreach model parameter.[38] A. Raue, C. Kreutz, F. J. Theis, J. Timmer, Joining forces of Bayesian and frequentist methodology: A study for inference in the presence ofnon-identifiability, Phil. Trans. Roy. Soc. A 371 (2013) 20110544. doi: .[39] S. Gupta, R. E. C. Lee, J. R. Faeder, Parallel tempering with LASSO for model reduction in systems biology, PLOS Computational Biology16 (2020) e1007669. doi: .[40] K. Busch, K. Klapproth, M. Barile, M. Flossdorf, T. Holland-Letz, S. M. Schlenner, M. Reth, T. H¨ofer, H.-R. Rodewald, Fundamentalproperties of unperturbed haematopoiesis from stem cells in vivo, Nature 518 (2015) 542–546. doi: .[41] I. David, A. Ricard, A unified model for inclusive inheritance in livestock species, Genetics 212 (2019) 1075–1099. doi: .[42] R. Duchesne, A. Guillemin, F. Crauste, O. Gandrillon, Calibration, selection and identifiability analysis of a mathematical model of the invitro erythropoiesis in normal and perturbed contexts, In Silico Biology 13 (2019) 55–69. doi: .[43] W. Zhou, R. Huang, K. Liu, W. Zhang, A novel interval-based approach for quantifying practical parameter identifiability of a lithium-ionbattery model, International Journal of Energy Research 44 (2020) 3558–3573. doi: .[44] K. E. Johnson, G. R. Howard, D. Morgan, E. A. Brenner, A. L. Gardner, R. E. Durrett, W. Mo, A. Al’Khafaji, E. D. Sontag, A. M. Jarrett,T. E. Yankeelov, A. Brock, Integrating transcriptomics and bulk time course data into a mathematical framework to describe and predicttherapeutic resistance in cancer, Physical Biology 18 (2020) 016001. doi: .*The authors emphasise practical identifiability as a central requirement for the validity of their model predictions. It is achievedby inclusion of single-cell RNA-Seq data into the ODE model via a machine learning algorithm.[45] M. Nihtil¨a, J. Virkkunen, Practical identifiability of growth and substrate consumption models, Biotechnology and Bioengineering 19 (1977)1831–1850. doi: .
46] A. Holmberg, On the practical identifiability of microbial growth models incorporating Michaelis-Menten type nonlinearities, MathematicalBiosciences 62 (1982) 23–43. doi: .[47] H. Miao, X. Xia, A. S. Perelson, H. Wu, On identifiability of nonlinear ODE models and Aapplications in viral dynamics, SIAM Review 53(2011) 3–39. doi: .[48] C. Gontier, J.-P. Pfister, Identifiability of a binomial synapse, Frontiers in Computational Neuroscience 14 (2020). doi: .*An alternative definition of practical identifiability is provided by Gontier and Pfister which is independent of the specific datarealization. This method allows to study how practical identifiability is a ff ected by di ff erent experimental protocols. However,it requires a nested hierarchy of competing models which is not clear a priori for complex models as used frequently in systemsbiology.[49] M. P. Saccomani, K. Thomaseth, The union between structural and practical identifiability makes strength in reducing oncological modelcomplexity: A case study, Complexity 2018 (2018) 1–10. doi: .[50] M. C. Neale, M. B. Miller, The use of likelihood-based confidence intervals in genetic models, Behavior Genetics 27 (1997) 113–120.doi: .[51] R. Ochoa-Fernandez, N. B. Abel, F.-G. Wieland, J. Schlegel, L.-A. Koch, J. B. Miller, R. Engesser, G. Giuriani, S. M. Brandl, J. Timmer,W. Weber, T. Ott, R. Simon, M. D. Zurbriggen, Optogenetic control of gene expression in plants in the presence of ambient white light,Nature Methods 17 (2020) 717–725. doi: .[52] N. Schneider, F.-G. Wieland, D. Kong, A. A. M. Fischer, M. H¨orner, J. Timmer, H. Ye, W. Weber, Liquid-liquid phase separation oflight-inducible transcription factors increases transcription activation in mammalian cells and mice, Science Advances 7 (2021) eabd3568.doi: .[53] I. Siekmann, J. Sneyd, E. J. Crampin, MCMC can detect nonidentifiable models, Biophysical Journal 103 (2012) 2275–2286. doi: .[54] K. E. Hines, T. R. Middendorf, R. W. Aldrich, Determination of parameter identifiability in nonlinear biophysical models: A Bayesianapproach, Journal of General Physiology 143 (2014) 401–416. doi: .[55] Y. Zuo, S. Sarkar, M. T. Corwin, K. Olson, R. D. Badawi, G. Wang, Structural and practical identifiability of dual-input kinetic modeling indynamic PET of liver inflammation, Physics in Medicine & Biology 64 (2019) 175023. doi: .[56] M. J. Simpson, R. E. Baker, S. T. Vittadello, O. J. Maclaren, Practical parameter identifiability for spatio-temporal models of cell invasion,Journal of The Royal Society Interface 17 (2020) 20200055. doi: .*By means of spatio-temporal models, the authors show that profile likelihood leads to the same results as the establishedMCMC approach while being one order of magnitude faster to compute.[57] M. Joshi, A. Seidel-Morgenstern, A. Kremling, Exploiting the bootstrap method for quantifying parameter confidence intervals in dynamicalsystems, Metabolic Engineering 8 (2006) 447–455. doi: .[58] L. Sachs, Applied Statistics, Springer Series in Statistics, Springer New York, New York, NY, 1984. doi: .[59] C. Kreutz, A. Raue, J. Timmer, Likelihood based observability analysis and confidence intervals for predictions of dynamic models, BMCSystems Biology 6 (2012) 120. doi: .[60] S. S. Bhonsale, B. Stokbroekx, J. Van Impe, Assessment of the parameter identifiability of population balance models for air jet mills,Computers & Chemical Engineering 143 (2020) 107056. doi: .[61] H. G. Frøysa, H. J. Skaug, G. Alendal, Experimental design for parameter estimation in steady-state linear models of metabolic networks,Mathematical Biosciences 319 (2020) 108291. doi: .[62] R. Mu˜noz-Tamayo, L. Puillet, J. B. Daniel, D. Sauvant, O. Martin, M. Taghipoor, P. Blavy, Review: To be or not to be an identifiable model.Is this a relevant question in animal science modelling?, Animal 12 (2018) 701–712. doi: .[63] T. Gross, N. Bl¨uthgen, Identifiability and experimental design in perturbation studies, Bioinformatics 36 (2020) i482–i489. doi: .
64] L. Bandiera, D. Gomez-Cabeza, J. Gilman, E. Balsa-Canto, F. Menolascina, Optimally designed model selection for synthetic biology, ACSSynthetic Biology 9 (2020) 3134–3144. doi: .[65] A. Raue, V. Becker, U. Klingm¨uller, J. Timmer, Identifiability and observability analysis for experimental design in nonlinear dynamicalmodels, Chaos 20 (2010) 45105. doi: .[66] B. Steiert, A. Raue, J. Timmer, C. Kreutz, Experimental design for parameter estimation of gene regulatory networks, PLoS ONE 7 (2012)e40052. doi: .[67] A. Tivay, X. Jin, A. K.-Y. Lo, C. G. Scully, J.-O. Hahn, Practical use of regularization in individualizing a mathematical model of cardiovas-cular hemodynamics using scarce data, Frontiers in Physiology 11 (2020). doi: .[68] Q. P. He, J. Wang, Application of systems engineering principles and techniques in biological big data analytics: A review, Processes 8(2020) 951. doi: .[69] Z. Zi, Sensitivity analysis approaches applied to systems biology models, IET Systems Biology 5 (2011) 336–346. doi: .[70] A. Pironet, P. D. Docherty, P. C. Dauby, J. G. Chase, T. Desaive, Practical identifiability analysis of a minimal cardiovascular system model,Computer Methods and Programs in Biomedicine 171 (2019) 53–65. doi: .[71] T. Maiwald, H. Hass, B. Steiert, J. Vanlier, R. Engesser, A. Raue, F. Kipkeew, H. H. Bock, D. Kaschek, C. Kreutz, J. Timmer, Driving themodel to its limit: Profile likelihood based model reduction, PLoS ONE 11 (2016) e0162366. doi: .[72] C. T¨onsing, J. Timmer, C. Kreutz, Profile likelihood-based analyses of infectious disease models, Statistical Methods in Medical Research27 (2018) 1979–1998. doi: .*This study applies profile likelihood-based identifiability analysis to infectious diseases models, which are based on ODEmodels similarly to systems biology context. Using profile likelihood, a stepwise data-based model reduction procedure isshown for a Zika virus model.[73] L. Cucurull-Sanchez, M. J. Chappell, V. Chelliah, S. Y. Amy Cheung, G. Derks, M. Penney, A. Phipps, R. S. Malik-Sheri ff , J. Timmis,M. J. Tindall, P. H. Graaf, P. Vicini, J. W. T. Yates, Best practices to maximize the use and reuse of quantitative and systems pharmacologymodels: Recommendations from the United Kingdom quantitative and systems pharmacology network, CPT: Pharmacometrics & SystemsPharmacology 8 (2019) 259–272. doi: .[74] D. L. Janz´en, M. Jirstrand, M. J. Chappell, N. D. Evans, Three novel approaches to structural identifiability analysis in mixed-e ff ects models,Computer Methods and Programs in Biomedicine 171 (2019) 141–152. doi: ..