Joshua F. Apgar

Sloppy models, parameter uncertainty, and the role of experimental design

Computational models are increasingly used to understand and predict complex biological phenomena. These models contain many unknown parameters, at least some of which are difficult to measure directly, and instead are estimated by fitting to time-course data. Previous work has suggested that even with precise data sets, many parameters are unknowable by trajectory measurements. We examined this question in the context of a pathway model of epidermal growth factor (EGF) and neuronal growth factor (NGF) signaling. Computationally, we examined a palette of experimental perturbations that included different doses of EGF and NGF as well as single and multiple gene knockdowns and overexpressions. While no single experiment could accurately estimate all of the parameters, experimental design methodology identified a set of five complementary experiments that could. These results suggest optimism for the prospects for calibrating even large models, that the success of parameter estimation is intimately linked to the experimental perturbations used, and that experimental design methodology is important for parameter fitting of biological models and likely for the accuracy that can be expected from them.

Stimulus Design for Model Selection and Validation in Cell Signaling

Mechanism-based chemical kinetic models are increasingly being used to describe biological signaling. Such models serve to encapsulate current understanding of pathways and to enable insight into complex biological processes. One challenge in model development is that, with limited experimental data, multiple models can be consistent with known mechanisms and existing data. Here, we address the problem of model ambiguity by providing a method for designing dynamic stimuli that, in stimulus–response experiments, distinguish among parameterized models with different topologies, i.e., reaction mechanisms, in which only some of the species can be measured. We develop the approach by presenting two formulations of a model-based controller that is used to design the dynamic stimulus. In both formulations, an input signal is designed for each candidate model and parameterization so as to drive the model outputs through a target trajectory. The quality of a model is then assessed by the ability of the corresponding controller, informed by that model, to drive the experimental system. We evaluated our method on models of antibody–ligand binding, mitogen-activated protein kinase (MAPK) phosphorylation and de-phosphorylation, and larger models of the epidermal growth factor receptor (EGFR) pathway. For each of these systems, the controller informed by the correct model is the most successful at designing a stimulus to produce the desired behavior. Using these stimuli we were able to distinguish between models with subtle mechanistic differences or where input and outputs were multiple reactions removed from the model differences. An advantage of this method of model discrimination is that it does not require novel reagents, or altered measurement techniques; the only change to the experiment is the time course of stimulation. Taken together, these results provide a strong basis for using designed input stimuli as a tool for the development of cell signaling models.

BI 885578, a Novel IGF1R/INSR Tyrosine Kinase Inhibitor with Pharmacokinetic Properties That Dissociate Antitumor Efficacy and Perturbation of Glucose Homeostasis.

Inhibition of the IGF1R, INSRA, and INSRB receptor tyrosine kinases represents an attractive approach of pharmacologic intervention in cancer, owing to the roles of the IGF1R and INSRA in promoting cell proliferation and survival. However, the central role of the INSRB isoform in glucose homeostasis suggests that prolonged inhibition of this kinase could result in metabolic toxicity. We describe here the profile of the novel compound BI 885578, a potent and selective ATP-competitive IGF1R/INSR tyrosine kinase inhibitor distinguished by rapid intestinal absorption and a short in vivo half-life as a result of rapid metabolic clearance. BI 885578, administered daily per os, displayed an acceptable tolerability profile in mice at doses that significantly reduced the growth of xenografted human GEO and CL-14 colon carcinoma tumors. We found that treatment with BI 885578 is accompanied by increases in circulating glucose and insulin levels, which in turn leads to compensatory hyperphosphorylation of muscle INSRs and subsequent normalization of blood glucose within a few hours. In contrast, the normalization of IGF1R and INSR phosphorylation in GEO tumors occurs at a much slower rate. In accordance with this, BI 885578 led to a prolonged inhibition of cell proliferation and induction of apoptosis in GEO tumors. We propose that the remarkable therapeutic window observed for BI 885578 is achieved by virtue of the distinctive pharmacokinetic properties of the compound, capitalizing on the physiologic mechanisms of glucose homeostasis and differential levels of IGF1R and INSR expression in tumors and normal tissues. Mol Cancer Ther; 14(12); 2762–72. ©2015 AACR.

Reply to Comment on “Sloppy models, parameter uncertainty, and the role of experimental design”

We welcome the commentary from Chachra, Transtrum, and Sethna1 regarding our paper “Sloppy models, parameter uncertainty, and the role of experimental design,”2 as their intriguing work shaped our thinking in this area.3 Sethna and colleagues introduced the notion of sloppy models, in which the uncertainty in the values of some combinations of parameters is many orders of magnitude greater than others.4 In our work we explored the extent to which large parameter uncertainties are an intrinsic characteristic of systems biology network models, or whether uncertainties are instead closely related to the collection of experiments used for model estimation. We were gratified to find the latter result –– that parameters are in principle knowable, which is important for the field of systems biology. The work also showed that small parameter uncertainties can be achieved and that the process can be greatly accelerated by using computational experimental design approaches5–9 deployed to select sets of experiments that effectively exercise the system in complementary directions.2 The comment by Chachra et al. does not disagree with any of these points, but rather emphasizes two quantitative issues.1 Firstly, even when all parameter combinations have small uncertainties, the fit model can still be sloppy in that some parameter combinations are known orders of magnitude better than others (in our paper this ratio of uncertainties was around 300).1, 2 This is certainly correct, although to truly ask whether sloppiness is inherent in the model or is due to the experiments used for fitting, one should apply optimal experimental design to the objective of minimizing sloppiness. We have done an initial trial and were able to establish all parameter directions to near 10% or less uncertainty while reducing the ratio to 55, and we expect that with more effort further reductions could be achieved. Secondly, Chachra et al. commented that the quantity of data required to achieve small parameter uncertainties could be large.1 We certainly agree. In our paper we effectively used 3,000 individual measurements spread across five experimental perturbations (600 data points per experiment), each measurement with the relatively high precision of 10%, to fit just 48 parameters. A greater number of less precise experimental measurements would be required, but the number could be decreased if less precision in the fit parameters were required. As but one example of how this tradeoff plays out in the example used in our paper,2 if the total number of measurements were reduced from 600 data points per experiment to just 68, then 13 experimental perturbations are required. If the experimental uncertainty were then doubled from 10% to 20%, then the required number of perturbations would increase further to 33, but if the desired parameter uncertainty then were to similarly double from 10% to 20%, the number of experimental perturbations would return to 13 (and is, in fact, a mathematically equivalent problem with an identical set of solutions). It should be noted that we did not optimize the selection of species or time points to measure, although it is known that not all contribute equally,7–9 and our techniques applied to species and time point selection could presumably lead to significant data reductions. This consideration, coupled with dramatic increases in capacities of new technologies for making large-scale measurements in systems biology, makes it is less likely that data limitations will be determining. Moreover, the application of optimal experimental design computations to plan experimental campaigns should then be increasingly useful to strategically plan experiments. This example emphasizes the tradeoff between the number of measurements per experimental perturbation and the number of experimental perturbations. Depending on the relative effort of producing one or the other, an appropriately customized campaign could be developed. Finally, it remains an unanswered question just how accurately parameters need to be known to achieve accurate predictions. One of the notions arising from the concept of model sloppiness is that some predictions can be made quite accurately with very inaccurate parameters,3 but this is of little use without a method for knowing when one is in this situation. Propagation of parameter uncertainty is one approach to estimating prediction accuracy. By clarifying the link between parameter uncertainty and experimental conditions, our work points to another approach.2 Because the link between parameter uncertainty and experimental conditions extends to experiments that have yet to be done (namely, predictions), combinations of experimental perturbations and measurements that would not further reduce parameter uncertainty significantly are expected to be well represented by the current model and should be relatively high confidence predictions. We are investigating this relationship in more detail.

Recycling Circuit Simulation Techniques for Mass-Action Biochemical Kinetics

Many numerical techniques developed for analyzing circuits can be “recycled”—that is, they can be used to analyze mass-action kinetics (MAK) models of biological processes. But the recycling must be judicious, as the differences in behavior between typical circuits and typical MAK models can impact a numerical technique’s accuracy and efficiency. In this chapter, we compare circuits and MAK models from this numerical perspective, using illustrative examples, theoretical comparisons of properties such as conservation and invariance of the non-negative orthant, as well as computational results from biological system models.

Quantitative Systems Pharmacology Model of hUGT1A1‐modRNA Encoding for the UGT1A1 Enzyme to Treat Crigler‐Najjar Syndrome Type 1

Crigler‐Najjar syndrome type 1 (CN1) is an autosomal recessive disease caused by a marked decrease in uridine‐diphosphate‐glucuronosyltransferase (UGT1A1) enzyme activity. Delivery of hUGT1A1‐modRNA (a modified messenger RNA encoding for UGT1A1) as a lipid nanoparticle is anticipated to restore hepatic expression of UGT1A1, allowing normal glucuronidation and clearance of bilirubin in patients. To support translation from preclinical to clinical studies, and first‐in‐human studies, a quantitative systems pharmacology (QSP) model was developed. The QSP model was calibrated to plasma and liver mRNA, and total serum bilirubin in Gunn rats, an animal model of CN1. This QSP model adequately captured the observed plasma and liver biomarker behavior across a range of doses and dose regimens in Gunn rats. First‐in‐human dose projections made using the translated model indicated that 0.5 mg/kg Q4W dose should provide a clinically meaningful and sustained reduction of >5 mg/dL in total bilirubin levels.

Abstract 573: Preclinical evaluation of JTX-2011, an anti-ICOS agonist antibody

ICOS (inducible co-stimulator molecule) is a co-stimulatory molecule and a member of the CD28 superfamily expressed primarily on T lymphocytes. Analysis of cancer patient samples as well as rodent preclinical data have implicated a role for the ICOS pathway in cancer immunotherapy. We have generated a panel of anti-ICOS monoclonal antibodies with in vitro agonistic properties. The anti-ICOS antibodies are efficacious as monotherapies and in combination with anti-PD1 in multiple syngeneic tumor models. Mechanistic studies demonstrate that tumor regression is associated with enhanced ratios of cytotoxic CD8:T regulatory (Treg) cells as well as preferential reduction in ICOS-high Tregs in the tumor microenvironment. JTX-2011, a species cross-reactive high affinity humanized agonist monoclonal antibody, has been selected for development. Evaluation of JTX-2011 in nonhuman primate models will be presented, including data informing safety and PK parameters. Our preclinical data provides rational for clinical development of JTX-2011 as a cancer immunotherapeutic to be tested as both a monotherapy as well as in combination with immunotherapies in solid tumor indications. Citation Format: Jennifer S. Michaelson, Christopher J. Harvey, Kutlu G. Elpek, Ellen Duong, Matthew Wallace, Chengyi J. Shu, Sriram Sathyanarayanan, Robert Mabry, Lindsey Shallberg, Tong Zi, Amit Deshpande, Stephen L. Sazinsky, Joshua Apgar, Deborah Law. Preclinical evaluation of JTX-2011, an anti-ICOS agonist antibody. [abstract]. In: Proceedings of the 107th Annual Meeting of the American Association for Cancer Research; 2016 Apr 16-20; New Orleans, LA. Philadelphia (PA): AACR; Cancer Res 2016;76(14 Suppl):Abstract nr 573.

Abstract A151: Quantitative systems pharmacology approaches accelerate lead generation and optimization of a PD-1 x TIM-3 therapeutic in immuno-oncology

The goal of this collaboration was to provide early quantitative decision making guidance for the project team by developing and interrogating a quantitative systems pharmacology (QSP) model of the co-modulation inhibitory receptors PD-1 and TIM-3 in immuno-oncology. The QSP model was to: (1) provide predictions of the best-in-class profile for a PD-1 and TIM-3 dual antagonist, (2) accelerate project timelines, (3) provide biological insights, and (4) reduce experimental costs. The QSP model was based on first principles as a system of elementary mass-action, mechanistic PKPD, ordinary differential equations. The model parameters and reactions were based on biophysics, and are interpretable. The model reactions include protein synthesis and elimination, ligand-receptor and drug-target formation and turnover, and drug administration and first order clearance. There were four versions of the model: PD-1 monospecific, TIM-3 monospecific, PD-1 x TIM-3 bispecific and fixed dose combination (FDC) targeting PD-1 and TIM-3. The monospecific models were then benchmarked against published data such that model parameter values were set to known values and unknown parameters were estimated. Once benchmarked, the FDC and bispecific models were analyzed by systematically investigating how tuning the model parameters (e.g., affinity, avidity, dose, half-life, target expression, etc.) impacted target inhibition, and to simulate patient variability. The model was in good agreement with published clinical data from nivolumab and pembrolizumab, and data from RMT3-23 in the TIM-3 driven mouse model. QSP model analysis predicted: (1) there would be diminishing returns on very tight binding biologics due to Target Mediated Drug Disposition (TMDD) that offsets potency, and (2) there is no advantage between FDC, 2-2 bispecific, and 2-1 bispecific formats, which are predicted to be roughly equivalent. As a result of these analyses, there was a significant reduction in the number of experiments, and acceleration of project timelines by (1) eliminating rounds of affinity maturation, as drug leads were in predicted optimal drug parameter ranges, and (2) eliminating the need to construct and evaluate bi-specific constructs and proceed with FDCs. Citation Format: Joshua F. Apgar, Jamie Wong, Ryan Ryan Phennicie, Mike Briskin, John M. Burke. Quantitative systems pharmacology approaches accelerate lead generation and optimization of a PD-1 x TIM-3 therapeutic in immuno-oncology. [abstract]. In: Proceedings of the AACR-NCI-EORTC International Conference: Molecular Targets and Cancer Therapeutics; 2015 Nov 5-9; Boston, MA. Philadelphia (PA): AACR; Mol Cancer Ther 2015;14(12 Suppl 2):Abstract nr A151.

Abstract 4061: Computational exploration of mechanistic determinants of antibody drug-conjugate pharmacokinetics using quantitative systems pharmacology modeling strategies

The pharmacokinetics of antibody drug conjugate (ADC) therapeutics typically show a discrepancy between the PK of total antibody (conjugated and unconjugated antibody) and that of conjugated antibody, carrying one or more payload molecules. This discrepancy is often attributed to deconjugation (Kamath, 2014), however recent evidence suggests that the underlying mechanisms may be more complex. This work employs a computational quantitative systems pharmacology (QSP), or mechabistic PK/PD approach to understand the impact of drug antibody ratio (DAR) and the resulting changes in molecular properties on overall PK and relative payload disposition as observed in preclinical and clinical studies. Our work establishes the benefit of using computational models to design novel ADCs and to optimize the discovery and development of existing ADCs. Citation Format: A. Katharina Wilkins, Andrew Matteson, Lore Gruenbaum, Jennifer Park, John M. Burke, Joshua Apgar. Computational exploration of mechanistic determinants of antibody drug-conjugate pharmacokinetics using quantitative systems pharmacology modeling strategies [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2017; 2017 Apr 1-5; Washington, DC. Philadelphia (PA): AACR; Cancer Res 2017;77(13 Suppl):Abstract nr 4061. doi:10.1158/1538-7445.AM2017-4061

Abstract SY03-02: Preclinical assessment of JTX-2011, an agonist antibody targeting ICOS, supports evaluation in ICONIC clinical trial

ICOS (the inducible T-cell co-stimulator) is a co-stimulatory molecule expressed on the surface of T cells and a member of the CD28 family, which includes clinically validated targets of cancer immunotherapies, such as PD-1 and CTLA-4. Clinical data identified ICOS as a potentially key molecule in providing optimal antitumor benefit following anti-CTLA-4 therapy. We have developed a species cross-reactive humanized IgG1 agonist antibody, JTX-2011, that binds ICOS and is designed to induce an antitumor immune response. Our preclinical data suggest that JTX-2011 functions through a dual mechanism of action, by stimulating T effector cells (Teff) and depleting intratumoral T regulatory cells (Tregs). The ICOS antibody is efficacious as a single agent in mouse syngeneic tumor models and demonstrates enhanced activity when administered in combination with anti-PD-1. Single-agent activity in the preclinical models appears to correlate with ICOS expression, with greater efficacy observed in tumor models that exhibit a higher percentage of ICOS-expressing immune cell infiltrate. An integrated expression analysis of human tumors identified non-small cell lung cancer (NSCLC) and head and neck squamous cell carcinoma (HNSCC) as indications with higher percentages of ICOS-expressing cell infiltrate. Preclinical studies performed in rodent and monkeys evaluated safety, pharmacokinetics, and pharmacodynamics of JTX-2011 to inform the first in-human study. The ICONIC phase I/II clinical trial is currently ongoing for evaluation of JTX-2011 alone or in combination with the anti-PD-1 antibody Nivolumab in patients with advanced solid tumors and incorporates a patient enrichment strategy design based on the preclinical and translational findings. Citation Format: Jennifer S. Michaelson, Christopher Harvey, Kutlu Elpek, Ellen Duong, Lindsey Shallberg, Matthew Wallace, Robert Mabry, Jenny Shu, Amit Deshpande, Tong Zi, Stephen Sazinsky, Joshua Apgar, Barbara Mounho-Zamora, Michael Briskin, Elizabeth Trehu, Jason Reeves, Heather Hirsch, Sriram Sathyanarayanan, Deborah Law. Preclinical assessment of JTX-2011, an agonist antibody targeting ICOS, supports evaluation in ICONIC clinical trial [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2017; 2017 Apr 1-5; Washington, DC. Philadelphia (PA): AACR; Cancer Res 2017;77(13 Suppl):Abstract nr SY03-02. doi:10.1158/1538-7445.AM2017-SY03-02