# On structural and practical identifiability

Franz-Georg Wieland, Adrian L. Hauber, Marcus Rosenblatt, Christian Tönsing, Jens Timmer

OOn structural and practical identiﬁability

Franz-Georg Wieland a,b,c , Adrian L. Hauber a,b , Marcus Rosenblatt a,b , Christian T¨onsing a,b,c , Jens Timmer a,b,c, ∗ a Institute of Physics, University of Freiburg, Hermann-Herder-Str. 3, 79104 Freiburg, Germany b Freiburg Center for Data Analysis and Modelling (FDM), University of Freiburg, Ernst-Zermelo-Str. 1, 79104 Freiburg, Germany c Centre for Integrative Biological Signalling Studies (CIBSS), University of Freiburg, Sch¨anzlestr. 18, 79104 Freiburg, Germany

Abstract

We discuss issues of structural and practical identiﬁability of partially observed di ﬀ erential equations which are oftenapplied in systems biology. The development of mathematical methods to investigate structural non-identiﬁabilityhas a long tradition. Computationally e ﬃ cient methods to detect and cure it have been developed recently. Practicalnon-identiﬁability on the other hand has not been investigated at the same conceptually clear level. We argue thatpractical identiﬁability is more challenging than structural identiﬁability when it comes to modelling experimentaldata. We discuss that the classical approach based on the Fisher information matrix has severe shortcomings. Asan alternative, we propose using the proﬁle likelihood, which is a powerful approach to detect and resolve practicalnon-identiﬁability. Highlights • With recent advances structural identiﬁability is no longer a major issue • Practical identiﬁability is still challenging • Fisher information matrix is misleading • Proﬁle likelihood can solve practical identiﬁability challenge

Keywords:

Identiﬁability, Structural identiﬁability, Practical identiﬁability, Proﬁle likelihood, Fisher informationmatrix, Non-linear dynamics, ODE models, Experimental design, Model reduction, Observability ∗ Corresponding author

Email address: [email protected] (Jens Timmer)

Preprint submitted to Current Opinion in Systems Biology February 11, 2021 a r X i v : . [ s t a t . M E ] F e b raphical abstractIntroduction Biological modelling and Box’s statement

Traditional biological reasoning often is rather qualitative, descriptive, and static, which results e.g. in cell biologyin so-called “pathway cartoons”. Mathematical models based on di ﬀ erential equations can help to turn these into aquantitative, predictive, and dynamic understanding of the underlying system. Discussing modelling in general, in1979, George E.P. Box coined his famous statement: “All models are wrong, but some are useful” [1]. While theformer part of the quote is intuitively clear, since every model necessarily poses a simpliﬁcation of reality, the latterhighlights the importance of assessing what constitutes a useful model. Bad, good and useful models

Three properties comprise a useful model. Firstly, it has to capture the main e ﬀ ects of the question of interest, i.e.describe the data with reasonable accuracy, and neglect the rest. Secondly, a useful model has to make experimentallyfalsiﬁable predictions in order to be testable. Models that exhibit these two properties are good models. Thirdly, themodel should enable to gather insights about the biological system. In a typical modelling process, one starts o ﬀ withan initial model based on current biological knowledge. Usually, this model cannot explain the data and therefore isa bad model. Based on biological intuition and trial-and-error, one increases the model complexity until the data can2e ﬁtted. Often, this leads to an over-parameterised model that over-ﬁts the data. The parameters of such a model andin turn its predictions are not well-determined and it thus remains a bad model.The path from such a bad model towards a good model is laborious: additional data needs to be measured andintegrated, the model complexity needs to be reduced and balanced to the available data, or a combination of both.This process needs to be iterated until a good model is found, which has well determined parameters and predictions.However, such a good model also needs to deliver biological insights in order to be useful . Only this third propertyturns a good model into a useful model. In this sense, the ﬁnal goal of mathematical modelling in systems biologyis not the model itself but to use the model to understand biology. One example of how a model can be used to gainbiological insight, which would be unattainable by merely assessing the data by itself, was given by Becker et al. [2]. Parameter identiﬁability

The concept of identiﬁability is strongly linked to the transition from bad models to good models. Identiﬁabilityanalysis is necessary to create good models that can describe the data and have well-determined parameters andpredictions. It is especially important when modelling biological systems because the limited amount and qualityof the experimental data with large measurement noise in only partially observed systems often leads to bad modelsduring the modelling process. Concerning identiﬁability, one distinguishes between structural identiﬁability dealingwith inherently indeterminable parameters due to the model structure itself, and practical identiﬁability, dealing withinsu ﬃ ciently informative measurements to determine the parameters with adequate precision. Partially observed dynamical systems

A biological system is translated into ordinary di ﬀ erential equations (ODEs)˙ x = f ( x , p , u ) , (1)comprising n model states x ( t ), unknown parameters p to be estimated from time-resolved experimental data, andexternal stimuli u ( t ). Since data is often recorded on a relative scale, scaling and o ﬀ set parameters for background cor-rections need to be estimated in parallel. Furthermore, in typical applications not all components of a cell-biologicalsystem can be measured, e.g. because of the limited availability or restricted capability of antibodies to discriminatebetween un-phosphorylated, i.e. inactive, and phosphorylated, i.e. active, proteins. Thus, an observation function g ( · )is required that maps the internal states x to the observations: y = g ( x , p , t ) . (2)Typically, the dimension m of y is smaller than the dimension n of x . We are therefore dealing with parameterestimation in partially observed systems. Moreover, in systems biology, these ODE models are typically sti ﬀ , non-linear, sparse and non-autonomous, and the discrete time observations are noisy.3arameter estimation is usually performed based on the weighted residual sum of squares, the negative log-likelihood assuming Gaussian errors χ ( p ) = m (cid:88) k = d k (cid:88) l = y Dkl − g k ( p , t l ) σ Dkl , (3)to determine the agreement of experimental data with the model trajectories, where y Dkl and σ Dkl represent d k data pointsand measurement errors at time points t l for each observable. A common point estimate for the best parameter vectoris the maximum likelihood estimatorˆ p = arg min (cid:104) χ ( p ) (cid:105) . (4) Structural identiﬁability

Deﬁnition of structural identiﬁability and connection to observability

Partially observed dynamical systems often exhibit structural non-identiﬁability. Structural identiﬁability is theability to uniquely estimate parameters from any given model output. A parameter p i is globally structurally identiﬁ-able [3], if for all parameter vectors p , it holds y ( p ) = y ( p (cid:48) ) ⇒ p i = p (cid:48) i . (5)An individual parameter p i is structurally non-identiﬁable, if changing the parameter does not alter the model trajec-tory y , because the changes can be fully compensated by altering other parameters. Local structural identiﬁability ofa parameter is deﬁned by reducing the deﬁnition to a neighbourhood v ( p ) instead of the entire parameter space. Amodel is structurally identiﬁable, if all of its parameters are structurally identiﬁable. Multiple related deﬁnitions forstructural identiﬁability exist, for a comprehensive discussion see a recent overview [4].A structurally non-identiﬁable parameter implies the existence of a manifold in parameter space upon which thetrajectory y is unchanged. However, on this manifold the dynamic variables x of the model can change, e.g. by ascaling factor, and are thus not uniquely determinable. This is denoted as non-observability, a concept closely relatedto parameter non-identiﬁability [5, 6, 7, 8, 9]. A priori analysis of structural identiﬁability

Two basic approaches exist to assess structural identiﬁability of non-linear dynamic models.

A priori methodsonly use the model deﬁnition, while a posteriori methods use the available data to ﬁnd non-identiﬁable parameters.Many a priori algorithms have been developed based on a variety of approaches. Powerful methods use Lie grouptheory, since non-identiﬁabilities are closely related to symmetries in the system [10, 11, 12, 13]. Furthermore, avariety of notable methods exist, which are based on power series expansion [14], generating series [15, 16], semi-numerical approaches [17, 18], di ﬀ erential algebra [19, 20, 21, 22, 23, 24, 25, 26, 27], di ﬀ erential geometry [28],4nd numerical algebraic geometry [29]. For reviews of some of these approaches, see [28, 30, 31]. Many of theseapproaches, especially the early developed methods, can only be applied to rather low-dimensional systems becauseof their computational complexity. Thus, recent developments have mainly focused on improving the computationale ﬃ ciency of the algorithms, e.g. by local sensitivity calculations.As a promising example, Joubert et al. [32] proposed a comprehensive and computationally fast pipeline to curestructural non-identiﬁabilities by re-parameterisation of the model in a ﬁve-step procedure: (i) a numerical identi-ﬁability analysis based on sensitivities, (ii) symbolic identiﬁability calculations for the low-dimensional candidatesfrom (i), this renders the procedure fast, (iii) computation of new model parameters, this step is not unique, but re-quires decisions of the modeler, (iv) simplify the original model leading to a lower dimensional parameter vector, andﬁnally (v) check the identiﬁability of the re-parameterised model. In an application to a model with 21 states and75 parameters, two groups of non-identiﬁable parameters were detected and the model was re-parameterised withinminutes. Analysis of structural identiﬁability using experimental data

In contrast to the aforementioned methods, a posteriori methods use the available data to perform identiﬁabilityanalysis. They infer structural non-identiﬁability based on model ﬁts to experimental data. Similar to some sensitivity-based a priori approaches, these approaches only assess local structural identiﬁability.One approach by Hengl et al. [33] suggested to perform numerous ﬁts and investigate non-parametrically whetherthe ﬁnal parameter estimates form a low-dimensional manifold in parameter space. This approach also allows todisentangle di ﬀ erent sets of coupled non-identiﬁable parameters.An informative and successful method is based on the proﬁle likelihood [34]. The idea of the proﬁle likelihood isto vary one parameter p i after the other around the maximum likelihood estimate (Equation (4)) and re-optimise theremaining onesPL ( p i ) = min p j (cid:44) i (cid:104) χ ( p ) (cid:105) . (6)For the two-parameter examples in Figure 1, the blue dashed lines show the path in the parameter space determined byEquation (6). Figure 1A shows the proﬁle likelihood of an identiﬁable parameter. For a structurally non-identiﬁableparameter the proﬁle likelihood yields a ﬂat line as shown in Figure 1B. Plotting the remaining parameters along theproﬁled parameter reveals which parameters are coupled to the non-identiﬁable one [35]. The proﬁle likelihood wasrecently extended to include two-dimensional proﬁles to allow for the identiﬁcation of parameter interdependence[36].Proﬁle likelihood calculation can be computationally demanding for larger systems due to the numerical re-optimisation. Addressing this issue, a fast a posteriori method to test identiﬁability without the need to calculatecomplete proﬁles using radial penalisation was recently developed [37].5 igure 1: Likelihood contour plots and proﬁle likelihood for an identiﬁable parameter and structurally and practically non-identiﬁableparameters.

Subﬁgures (A), (B), and (C) show contour plots of χ above as well as the proﬁle likelihood versus the parameter below. Lightercolours in the contour plots signify a lower value of χ . Thresholds for conﬁdence intervals corresponding to a conﬁdence level of 95 % are shownin red and plotted both in the contour plots and the proﬁle likelihood plots. The lowest value of χ is denoted by a grey asterisk in both the contourplot and the proﬁle likelihood plot. For the identiﬁable parameter (A) the proﬁle likelihood reaches both an upper and lower threshold thus leadingto a ﬁnite conﬁdence interval. For the structurally non-identiﬁable parameter (B) the proﬁle likelihood is completely ﬂat, thus yielding inﬁniteconﬁdence intervals. In the contour plot this translates to a ﬂat path, along which χ does not change. The practically non-identiﬁable parameter(C) shows an inﬁnite extension of the low χ region for lower values of the parameter, never reaching the 95 % conﬁdence interval threshold. Incontrast, a ﬁnite upper conﬁdence bound can be derived. Structural non-identiﬁability can also be investigated a posteriori by a Bayesian Markov chain Monte Carlo(MCMC) sampling approach. However, for non-identiﬁable systems e ﬃ cient mixing and thus convergence of theMarkov chains is di ﬃ cult [38]. This problem can be cured by informative priors but these would mask the problemand should only be implemented if they are based on actual biological insights and prior information. One recentapplication in the ﬁeld identiﬁed a minimal subset of reactions in a signalling network with a combination of paralleltempering and LASSO regression methods [39]. Re-parameterising structurally non-identiﬁable models

Given the recent advances in the computational e ﬃ ciency of methods, we essentially consider determining struc-tural identiﬁability no longer a bottleneck in the modelling of non-linear dynamic systems with ODEs. When thestructurally non-identiﬁable parameters are determined, the problem is usually ﬁxed by a re-parameterisation of themodel. In the simplest case this is accomplished by ﬁxing some of the involved parameters to a certain value. Theprice to be paid is typically that the information about the scale of some components is lost. Nevertheless, biolog-6cally meaningful re-parameterisation of the models after ﬁnding non-identiﬁabilites remains a challenging task (G.Massonis et al. , arXiv:2012.09826v2). Practical identiﬁability

From structural to practical identiﬁability

Structural identiﬁability implies practical identiﬁability only for an inﬁnite amount of data with zero noise. Prac-tical identiﬁability is important for obtaining precise parameter estimates. Moreover, it is especially crucial to en-sure that model predictions are well-determined. It is analyzed increasingly often to judge a model’s predictivity[40, 41, 42, 43, 44]. The notion of practical identiﬁability has been rather vague in the literature, mainly referring to large conﬁdence intervals [45, 46, 47]. Some approaches exist that deﬁne practical identiﬁability as a combinationof model structure and experimental protocol without actual data [48, 49]. In contrast, we consider a combination ofmodel and data as practically identiﬁable if the conﬁdence intervals of all estimated parameters are of ﬁnite size [35].

Parameter conﬁdence intervals and identiﬁability

The proﬁle likelihood (Equation (6)) provides a proper assessment of conﬁdence intervals of estimated parametersin ODE models (Figure 1) by CI PL ( p i ) = (cid:8) p i | PL ( p i ) ≤ χ ( ˆ p ) + ∆ α (cid:9) , (7)where ∆ α denotes the α quantile of the χ distribution with df = igure 2: Parameter conﬁdence intervals based on Fisher information matrix and proﬁle likelihood.

Conﬁdence intervals for ﬁve parametersbased on proﬁle likelihood (blue) and on quadratic approximation using the Fisher information matrix (FIM) (orange). FIM-based conﬁdenceintervals have two major problems. Firstly, due to the non-linearity of the underlying systems, the Cram´er-Rao bound on the error is invalid andthus the FIM-based conﬁdence intervals become uncontrollable for a ﬁnite amount of measurements. While in (A) the FIM-based interval is largerthan the proﬁle likelihood-based interval, in (B) it is smaller. Secondly, FIM based intervals are insensitive to practical non-identiﬁabilities. In (C),the FIM-based conﬁdence interval is ﬁnite and thus the practically non-identiﬁable parameter is not detected. In (D), the practically non-identiﬁableparameter leads to a ﬂat FIM-based interval, wrongly suggesting structural non-identiﬁability. While the structurally non-identiﬁable parameterin (E) is correctly detected, similarly to (D) the calculation of the FIM is challenging due to its singularity in ﬂat likelihood landscapes. Theparameters (A)-(D) are adapted from two applications in synthetic biology ((A),(B),(C) from Schneider et al. [52], (D) from Ochoa-Fernandezet al. [51]). Parameter (E) is from a minimal non-identiﬁable toy model. Grey asterisks signify the maximum likelihood estimate of the parameter.

Bayesian methods for identiﬁability analysis

Bayesian sampling approaches, e.g. MCMC, can be used to assess practical identiﬁability [53, 54, 55]. This,however is only feasible if the model is structurally identiﬁable, since structural non-identiﬁabilities will lead to badmixing of the sampling algorithms. Given a structurally identiﬁable model, MCMC sampling yields similar resultsas the proﬁle likelihood analysis [38]. However, a recent application in a spatio-temporal reaction–di ﬀ usion model8howed, that it is one order of magnitude slower than the proﬁle likelihood [56]. Experimental design and model reduction

Model predictions

To test the predictive power of a model, conﬁdence intervals for the predictions can be computed. For this purpose,forward evaluations of the model are utilized, e.g. bootstrap approaches [57] or sensitivity analysis [58]. They typicallyrequire large numerical e ﬀ orts in the context of non-linear biological models with a high-dimensional parameter space.A more powerful approach is the prediction proﬁle likelihood PPL ( z ) = min p ∈{ p | g pred ( p ) = z } (cid:104) χ ( p ) (cid:105) , (8)which is obtained by minimising χ ( p ) (Equation (3)) under the constraint that the model response g pred ( p ) is equalto the prediction z . The prediction proﬁle likelihood propagates the uncertainty from the experimental data to theprediction by exploring the prediction space instead of the parameter space [59].If the model predictions are not of su ﬃ cient precision, one has two principal options to tailor the model complexityto the information content of the data: (i) measure additional data, corresponding to an increase of the dimensionof the observation function g in Equation (2), or (ii) reduce the model complexity according to the available data,corresponding to a decrease of the dimension of the parameter space and / or of the ODE system f in Equation (1).Both options increase the practical identiﬁability of the model. Achieving practical identiﬁability by new measurements with optimal experimental design

Practical identiﬁability can be achieved by adding new data [44, 60]. The process of determining the most infor-mative targets and time points for the new measurements is known as optimal experimental design and is frequentlyapplied in di ﬀ erent modelling ﬁelds, e.g. metabolic models [61], animal science [62], linear perturbation networks[63] or synthetic biology [64]. The task is related to the search of an additional measurement that contains the max-imal information about the system or parts of it. For improving the identiﬁability of a speciﬁc parameter, the modeltrajectories along the corresponding parameter proﬁle can be investigated [65, 66]. Thereby, measurement points withmaximal information content for the parameter of interest can be determined, which corresponds to trajectories withhigh spread. Similarly, the prediction proﬁle likelihood (Equation (8)) determines the prediction uncertainty of themodel at a potential new measurement time point [59] thus promoting the identiﬁability of the whole model. Mea-surement points with high prediction uncertainty are e ﬀ ective to constrain the model further, whereas measurementswith a low prediction uncertainty are better suited for model selection purposes.9 chieving practical identiﬁability by reducing model complexity If measuring additional data is not feasible, the complexity of the model has to be reduced. One way is to ﬁxparameter values or ratios of parameters by means of prior knowledge [67], sensitivity analysis [68, 69] or proﬁlelikelihood [70]. However, ﬁxing parameters can decrease the interpretative relevance of the model’s predictions.Taking this into account, a systematic model reduction strategy that tailors model complexity to the available datawas suggested by Maiwald et al. [71]. Based on likelihood proﬁles, they discuss four basic scenarios that are dis-criminated based on the proﬁle likelihood by the combinations of: either (i) the proﬁle ﬂattens out for a logarithmisedparameter going to inﬁnity or (ii) to minus inﬁnity, and either other parameters are (a) coupled to the investigatedone or (b) not. For all four possible combinations, there is a cure. For case (i / a), one di ﬀ erential equation is replacedby an algebraic equation, for (i / b), states can be lumped, for (ii / a), a variable is ﬁxed, leading to a structural non-identiﬁability that can be cured by the methods discussed above, and for (ii / b), a reaction can be removed from themodel. This model reduction strategy has been applied e.g. in [51, 52, 72]. Independent of the applied method, modelreduction steps, and in particular the conclusions thereof, should always be documented together with the modelaccording to good scientiﬁc practice to facilitate reproducibility. Conclusions

Given the multitude of recently developed methods [13, 16, 27, 32], we consider the ﬁle of identifying structurallynon-identiﬁable parameters as closed. Future research in this ﬁeld could focus on identifying biologically plausiblere-parameterisations of the model, for which no comprehensive method yet exists. Furthermore, the extension of theconcept of identiﬁability to di ﬀ erent model types, e.g. mixed e ﬀ ects models [73, 74], is of interest.Achieving practical identiﬁability for model and data is more laborious in practice. Practical non-identiﬁabilitiescan be detected reliably, e.g. by the proﬁle likelihood method [31]. In order to achieve identiﬁability, the modelcomplexity has to be reduced or additional data must be added. Proﬁle likelihood-based model reduction [71] andoptimal experimental design [66] provide valuable methods for these purposes. A ﬂowchart locating structural andpractical identiﬁability analysis as discussed in this review within the entire modelling process is given in Figure 3.Although the availability of advanced methods for the detection and cure of structural and practical non-identiﬁabilitiesis promising, two related challenges remain. In many applications identiﬁability analysis is not performed with state-of-the-art methods. Particularly, identiﬁability analysis based on the Fisher information matrix can be misleading intypical applications in systems biology. We propose a more consequent use of the discussed methods for structuralidentiﬁability and especially proﬁle likelihood for practical identiﬁability analysis in order to check the limitationsand predictive power of mathematical models. In summary, we believe the focal point of research in systems biologyshould always remain on the biological insights that can be gained from mathematical models which are structurallyand practically identiﬁable. 10 igure 3: Flowchart of the entire modelling process from inital to ﬁnal model including identiﬁability analysis.

The modelling process beginswith the inception of an initial model based on prior knowledge and the underlying biological research question. It ends with the ﬁnal validatedmodel and the biological insights it provides. The ﬂowchart shows, how identiﬁability analysis is embedded into the overall modelling workﬂow.The topics discussed in this review related to structural identiﬁability (blue) and practical identiﬁability (red) are highlighted with colours in theﬂowchart. The remaining tiles in gray represent aspects that are beyond the scope of this review. The intricacy of the ﬂowchart shows, that thepath to biological insights requires multiple iterations of di ﬀ erent methods. Identiﬁability analysis is an integral part of this workﬂow and shouldbe performed to gain insights from predictive models with well-determined parameters. Furthermore, methods dealing with structural and practicalidentiﬁability should always be focused on ultimately progressing along the path towards biological insights. Conﬂict of interest statement

The authors declare that they have no known competing ﬁnancial interests or personal relationships that couldhave appeared to inﬂuence the work reported in this paper.11 cknowledgements

This work was supported by the German Research Foundation (DFG) under Germany’s Excellence Strategy(CIBSS – EXC-2189 – Project ID 390939984), the German Research Foundation (DFG) through grant 272983813 / TRR179, the Deutsche Krebshilfe (grant 70112355), the German Federal Ministry for Education and Research within theresearch network Systems Medicine of the Liver (LiSyM; grant 031L0048), and by the state of Baden-W¨urttembergthrough bwHPC and the German Research Foundation (DFG) through grant INST 35 / References [1] G. E. P. Box, Robustness in the strategy of scientiﬁc model building, in: R. L. Launer, G. N. Wilkinson (Eds.), Robustness in Statistics,Academic Press, New York, 1979, pp. 201–236. doi: .[2] V. Becker, M. Schilling, J. Bachmann, U. Baumann, A. Raue, T. Maiwald, J. Timmer, U. Klingm¨uller, Covering a broad dynamic range:Information processing at the erythropoietin receptor, Science 328 (2010) 1404–1408. doi: .[3] E. Walter, L. Pronzato, Identiﬁcation of Parametric Models from Experimental Data, Springer, London, U.K., 1997.[4] F. Anstett-Collin, L. Denis-Vidal, G. Mill´erioux, A priori identiﬁability: An overview on deﬁnitions and approaches, Annual Reviews inControl 50 (2020) 139–149. doi: .*Rigorously assessing 21 deﬁnitions of a priori identiﬁability, the authors provide an overview over analytical and algebraicdeﬁnitions as well as local and global deﬁnitions and their dependencies.[5] R. Kalman, On the general theory of control systems, IRE Transactions on Automatic Control 4 (1959) 110–110. doi: .[6] R. Bellman, K. Åstr¨om, On structural identiﬁability, Mathematical Biosciences 7 (1970) 329–339. doi: .[7] M. J. Chappell, K. R. Godfrey, S. Vajda, Global identiﬁability of the parameters of nonlinear systems with speziﬁed inputs: a comparison ofmethods, Math. Biosciences 102 (1990) 41–73. doi: .[8] A. F. Villaverde, N. Tsiantis, J. R. Banga, Full observability and estimation of unknown inputs, states and parameters of nonlinear biologicalmodels, Journal of The Royal Society Interface 16 (2019) 20190043. doi: .*A comprehensive approach to observability analysis is provided in this paper. This includes assessing inputs, states and pa-rameters. Structural parameter identiﬁability is assessed with an approach based on di ﬀ erential geometry.[9] T. Schmitt, B. Ritter, Data-based identiﬁability and observability assessment for nonlinear control systems using the proﬁle likelihood method,in: 21st IFAC World Congress, Germany, 2020. doi: .[10] J. W. Yates, N. D. Evans, M. J. Chappell, Structural identiﬁability analysis via symmetries of di ﬀ erential equations, Automatica 45 (2009)2585–2591. doi: .[11] B. Merkt, D. Kaschek, J. Timmer, Higher-order Lie symmetries in identiﬁability and predictability analysis of dynamic models, PhysicalReview E 92 (2015) 012920. doi: .[12] A. F. Villaverde, N. D. Evans, M. J. Chappell, J. R. Banga, Input-dependent structural identiﬁability of nonlinear systems, IEEE ControlSystems Letters 3 (2019) 272–277. doi: .[13] G. Massonis, A. F. Villaverde, Finding and breaking Lie symmetries: Implications for structural identiﬁability and observability in biologicalmodelling, Symmetry 12 (2020) 469. doi: . *The authors provide an improved algorithm to ﬁnd model symmetries using Lie algebra that is computationally more e ﬃ cientand thus applicable to higher dimensional models.[14] H. Pohjanpalo, System identiﬁability based on the power series expansion of the solution, Mathematical Biosciences 41 (1978) 21–33.doi: .[15] E. Walter, Y. Lecourtier, Global approaches to identiﬁability testing for linear and nonlinear state space models, Mathematics and Computersin Simulation 24 (1982) 472–482. doi: .[16] T. S. Ligon, F. Fr¨ohlich, O. T. Chis¸, J. R. Banga, E. Balsa-Canto, J. Hasenauer, GenSSI 2.0: Multi-experiment structural identiﬁabilityanalysis of SBML models, Bioinformatics 34 (2018) 1421–1423. doi: .*The authors provide an open-source Matlab toolbox for structural identiﬁability analysis and the reparametrization of struc-turally non-identiﬁable models. They use the generating series approach and support Systems Biology Markup Language im-port.[17] A. Sedoglavic, A probabilistic algorithm to test local algebraic observability in polynomial time, Journal of Symbolic Computation 33 (2002)735–755. doi: .[18] J. Karlsson, M. Anguelova, M. Jirstrand, An e ﬃ cient method for structural identiﬁability analysis of large dynamic systems, IFAC Proceed-ings Volumes 45 (2012) 941–946. doi: .[19] L. Ljung, T. Glad, On global identiﬁability for arbitrary model parametrizations, Automatica 30 (1994) 265–276. doi: .[20] M. P. Saccomani, S. Audoly, L. D’Angi`o, Parameter identiﬁability of nonlinear systems: The role of initial conditions, Automatica 39 (2003)619–632. doi: .[21] G. Bellu, M. P. Saccomani, S. Audoly, L. D’Angi`o, DAISY: A new software tool to test global identiﬁability of biological and physiologicalsystems, Computer Methods and Programs in Biomedicine 88 (2007) 52–61. doi: .[22] N. Meshkat, C. E.-Z. Kuo, J. DiStefano, On ﬁnding and using identiﬁable parameter combinations in nonlinear dynamic systems biologymodels and COMBOS: A novel web implementation, PLoS ONE 9 (2014) e110261. doi: .[23] K. Thomaseth, M. P. Saccomani, Local identiﬁability analysis of nonlinear ODE models: How to determine all candidate solutions, IFAC-PapersOnLine 51 (2018) 529–534. doi: .[24] A. Varghese, S. Narasimhan, N. Bhatt, A priori parameter identiﬁability in complex reaction networks, IFAC-PapersOnLine 51 (2018)760–765. doi: .[25] M. P. Saccomani, G. Bellu, S. Audoly, L. D’Angi´o, A new version of DAISY to test structural identiﬁability of biological models, in:Computational Methods in Systems Biology, 2019, pp. 329–334. doi: .[26] H. Hong, A. Ovchinnikov, G. Pogudin, C. Yap, SIAN: Software for structural identiﬁability analysis of ODE models, Bioinformatics 35(2019) 2873–2874. doi: .[27] H. Hong, A. Ovchinnikov, G. Pogudin, C. Yap, Global identiﬁability of di ﬀ erential models, Communications on Pure and Applied Mathe-matics 73 (2020) 1831–1879. doi: .**Combining a probabilistic and analytical criterion in one algorithm, the authors perform a priori structural identiﬁabilityanalysis e ﬃ ciently and quickly. The algorithm is available in the SIAN Maple software package.[28] A. F. Villaverde, Observability and structural identiﬁability of nonlinear biological systems, Complexity 2019 (2019) 1–12. doi: .[29] D. J. Bates, J. D. Hauenstein, N. Meshkat, Identiﬁability and numerical algebraic geometry, PLOS ONE 14 (2019) e0226299. doi: .[30] O. T. Chis, J. R. Banga, E. Balsa-Canto, Structural identiﬁability of systems biology models: A critical comparison of methods, PLoS ONE6 (2011) e27755. doi: .

31] A. Raue, J. Karlsson, M. P. Saccomani, M. Jirstrand, J. Timmer, Comparison of approaches for parameter identiﬁability analysis of biologicalsystems, Bioinformatics 30 (2014) 1440–1448. doi: .[32] D. Joubert, J. D. Stigter, J. Molenaar, An e ﬃ cient procedure to assist in the re-parametrization of structurally unidentiﬁable models, Mathe-matical Biosciences 323 (2020) 108328. doi: .**Using a combination of numerical and algebraic methods, the authors provide an incredibly fast algorithm for assessingstructural identiﬁability. Their approach uses sensitivity analysis to provide candidates for further symbolic analysis, beforecomputing new algebraic relations for the non-identiﬁable parameters.[33] S. Hengl, C. Kreutz, J. Timmer, T. Maiwald, Data-based identiﬁability analysis of non-linear dynamical models, Bioinformatics 23 (2007)2612–2618. doi: .[34] S. A. Murphy, A. W. van der Vaart, On proﬁle likelihood, Journal of the American Statistical Association 95 (2000) 449. doi: .[35] A. Raue, C. Kreutz, T. Maiwald, J. Bachmann, M. Schilling, U. Klingm¨uller, J. Timmer, Structural and practical identiﬁability anal-ysis of partially observed dynamical models by exploiting the proﬁle likelihood, Bioinformatics 25 (2009) 1923–1929. doi: .[36] O. M. Brastein, B. Lie, R. Sharma, N.-O. Skeie, Parameter estimation for externally simulated thermal network models, Energy and Buildings191 (2019) 200–210. doi: .*Extension of the proﬁle likelihood method for 2D likelihood proﬁles that allow for analysis of pairwise parameter interdepen-dence.[37] C. Kreutz, An easy and e ﬃ cient approach for testing identiﬁability, Bioinformatics 34 (2018) 1913–1921. doi: .**Fast alternative method to test structural and practical identiﬁability without the need to calculate the proﬁle likelhood foreach model parameter.[38] A. Raue, C. Kreutz, F. J. Theis, J. Timmer, Joining forces of Bayesian and frequentist methodology: A study for inference in the presence ofnon-identiﬁability, Phil. Trans. Roy. Soc. A 371 (2013) 20110544. doi: .[39] S. Gupta, R. E. C. Lee, J. R. Faeder, Parallel tempering with LASSO for model reduction in systems biology, PLOS Computational Biology16 (2020) e1007669. doi: .[40] K. Busch, K. Klapproth, M. Barile, M. Flossdorf, T. Holland-Letz, S. M. Schlenner, M. Reth, T. H¨ofer, H.-R. Rodewald, Fundamentalproperties of unperturbed haematopoiesis from stem cells in vivo, Nature 518 (2015) 542–546. doi: .[41] I. David, A. Ricard, A uniﬁed model for inclusive inheritance in livestock species, Genetics 212 (2019) 1075–1099. doi: .[42] R. Duchesne, A. Guillemin, F. Crauste, O. Gandrillon, Calibration, selection and identiﬁability analysis of a mathematical model of the invitro erythropoiesis in normal and perturbed contexts, In Silico Biology 13 (2019) 55–69. doi: .[43] W. Zhou, R. Huang, K. Liu, W. Zhang, A novel interval-based approach for quantifying practical parameter identiﬁability of a lithium-ionbattery model, International Journal of Energy Research 44 (2020) 3558–3573. doi: .[44] K. E. Johnson, G. R. Howard, D. Morgan, E. A. Brenner, A. L. Gardner, R. E. Durrett, W. Mo, A. Al’Khafaji, E. D. Sontag, A. M. Jarrett,T. E. Yankeelov, A. Brock, Integrating transcriptomics and bulk time course data into a mathematical framework to describe and predicttherapeutic resistance in cancer, Physical Biology 18 (2020) 016001. doi: .*The authors emphasise practical identiﬁability as a central requirement for the validity of their model predictions. It is achievedby inclusion of single-cell RNA-Seq data into the ODE model via a machine learning algorithm.[45] M. Nihtil¨a, J. Virkkunen, Practical identiﬁability of growth and substrate consumption models, Biotechnology and Bioengineering 19 (1977)1831–1850. doi: .

46] A. Holmberg, On the practical identiﬁability of microbial growth models incorporating Michaelis-Menten type nonlinearities, MathematicalBiosciences 62 (1982) 23–43. doi: .[47] H. Miao, X. Xia, A. S. Perelson, H. Wu, On identiﬁability of nonlinear ODE models and Aapplications in viral dynamics, SIAM Review 53(2011) 3–39. doi: .[48] C. Gontier, J.-P. Pﬁster, Identiﬁability of a binomial synapse, Frontiers in Computational Neuroscience 14 (2020). doi: .*An alternative deﬁnition of practical identiﬁability is provided by Gontier and Pﬁster which is independent of the speciﬁc datarealization. This method allows to study how practical identiﬁability is a ﬀ ected by di ﬀ erent experimental protocols. However,it requires a nested hierarchy of competing models which is not clear a priori for complex models as used frequently in systemsbiology.[49] M. P. Saccomani, K. Thomaseth, The union between structural and practical identiﬁability makes strength in reducing oncological modelcomplexity: A case study, Complexity 2018 (2018) 1–10. doi: .[50] M. C. Neale, M. B. Miller, The use of likelihood-based conﬁdence intervals in genetic models, Behavior Genetics 27 (1997) 113–120.doi: .[51] R. Ochoa-Fernandez, N. B. Abel, F.-G. Wieland, J. Schlegel, L.-A. Koch, J. B. Miller, R. Engesser, G. Giuriani, S. M. Brandl, J. Timmer,W. Weber, T. Ott, R. Simon, M. D. Zurbriggen, Optogenetic control of gene expression in plants in the presence of ambient white light,Nature Methods 17 (2020) 717–725. doi: .[52] N. Schneider, F.-G. Wieland, D. Kong, A. A. M. Fischer, M. H¨orner, J. Timmer, H. Ye, W. Weber, Liquid-liquid phase separation oflight-inducible transcription factors increases transcription activation in mammalian cells and mice, Science Advances 7 (2021) eabd3568.doi: .[53] I. Siekmann, J. Sneyd, E. J. Crampin, MCMC can detect nonidentiﬁable models, Biophysical Journal 103 (2012) 2275–2286. doi: .[54] K. E. Hines, T. R. Middendorf, R. W. Aldrich, Determination of parameter identiﬁability in nonlinear biophysical models: A Bayesianapproach, Journal of General Physiology 143 (2014) 401–416. doi: .[55] Y. Zuo, S. Sarkar, M. T. Corwin, K. Olson, R. D. Badawi, G. Wang, Structural and practical identiﬁability of dual-input kinetic modeling indynamic PET of liver inﬂammation, Physics in Medicine & Biology 64 (2019) 175023. doi: .[56] M. J. Simpson, R. E. Baker, S. T. Vittadello, O. J. Maclaren, Practical parameter identiﬁability for spatio-temporal models of cell invasion,Journal of The Royal Society Interface 17 (2020) 20200055. doi: .*By means of spatio-temporal models, the authors show that proﬁle likelihood leads to the same results as the establishedMCMC approach while being one order of magnitude faster to compute.[57] M. Joshi, A. Seidel-Morgenstern, A. Kremling, Exploiting the bootstrap method for quantifying parameter conﬁdence intervals in dynamicalsystems, Metabolic Engineering 8 (2006) 447–455. doi: .[58] L. Sachs, Applied Statistics, Springer Series in Statistics, Springer New York, New York, NY, 1984. doi: .[59] C. Kreutz, A. Raue, J. Timmer, Likelihood based observability analysis and conﬁdence intervals for predictions of dynamic models, BMCSystems Biology 6 (2012) 120. doi: .[60] S. S. Bhonsale, B. Stokbroekx, J. Van Impe, Assessment of the parameter identiﬁability of population balance models for air jet mills,Computers & Chemical Engineering 143 (2020) 107056. doi: .[61] H. G. Frøysa, H. J. Skaug, G. Alendal, Experimental design for parameter estimation in steady-state linear models of metabolic networks,Mathematical Biosciences 319 (2020) 108291. doi: .[62] R. Mu˜noz-Tamayo, L. Puillet, J. B. Daniel, D. Sauvant, O. Martin, M. Taghipoor, P. Blavy, Review: To be or not to be an identiﬁable model.Is this a relevant question in animal science modelling?, Animal 12 (2018) 701–712. doi: .[63] T. Gross, N. Bl¨uthgen, Identiﬁability and experimental design in perturbation studies, Bioinformatics 36 (2020) i482–i489. doi: .

64] L. Bandiera, D. Gomez-Cabeza, J. Gilman, E. Balsa-Canto, F. Menolascina, Optimally designed model selection for synthetic biology, ACSSynthetic Biology 9 (2020) 3134–3144. doi: .[65] A. Raue, V. Becker, U. Klingm¨uller, J. Timmer, Identiﬁability and observability analysis for experimental design in nonlinear dynamicalmodels, Chaos 20 (2010) 45105. doi: .[66] B. Steiert, A. Raue, J. Timmer, C. Kreutz, Experimental design for parameter estimation of gene regulatory networks, PLoS ONE 7 (2012)e40052. doi: .[67] A. Tivay, X. Jin, A. K.-Y. Lo, C. G. Scully, J.-O. Hahn, Practical use of regularization in individualizing a mathematical model of cardiovas-cular hemodynamics using scarce data, Frontiers in Physiology 11 (2020). doi: .[68] Q. P. He, J. Wang, Application of systems engineering principles and techniques in biological big data analytics: A review, Processes 8(2020) 951. doi: .[69] Z. Zi, Sensitivity analysis approaches applied to systems biology models, IET Systems Biology 5 (2011) 336–346. doi: .[70] A. Pironet, P. D. Docherty, P. C. Dauby, J. G. Chase, T. Desaive, Practical identiﬁability analysis of a minimal cardiovascular system model,Computer Methods and Programs in Biomedicine 171 (2019) 53–65. doi: .[71] T. Maiwald, H. Hass, B. Steiert, J. Vanlier, R. Engesser, A. Raue, F. Kipkeew, H. H. Bock, D. Kaschek, C. Kreutz, J. Timmer, Driving themodel to its limit: Proﬁle likelihood based model reduction, PLoS ONE 11 (2016) e0162366. doi: .[72] C. T¨onsing, J. Timmer, C. Kreutz, Proﬁle likelihood-based analyses of infectious disease models, Statistical Methods in Medical Research27 (2018) 1979–1998. doi: .*This study applies proﬁle likelihood-based identiﬁability analysis to infectious diseases models, which are based on ODEmodels similarly to systems biology context. Using proﬁle likelihood, a stepwise data-based model reduction procedure isshown for a Zika virus model.[73] L. Cucurull-Sanchez, M. J. Chappell, V. Chelliah, S. Y. Amy Cheung, G. Derks, M. Penney, A. Phipps, R. S. Malik-Sheri ﬀ , J. Timmis,M. J. Tindall, P. H. Graaf, P. Vicini, J. W. T. Yates, Best practices to maximize the use and reuse of quantitative and systems pharmacologymodels: Recommendations from the United Kingdom quantitative and systems pharmacology network, CPT: Pharmacometrics & SystemsPharmacology 8 (2019) 259–272. doi: .[74] D. L. Janz´en, M. Jirstrand, M. J. Chappell, N. D. Evans, Three novel approaches to structural identiﬁability analysis in mixed-e ﬀ ects models,Computer Methods and Programs in Biomedicine 171 (2019) 141–152. doi: ..