Elodie L. Plan

Model Evaluation of Continuous Data Pharmacometric Models: Metrics and Graphics

This article represents the first in a series of tutorials on model evaluation in nonlinear mixed effect models (NLMEMs), from the International Society of Pharmacometrics (ISoP) Model Evaluation Group. Numerous tools are available for evaluation of NLMEM, with a particular emphasis on visual assessment. This first basic tutorial focuses on presenting graphical evaluation tools of NLMEM for continuous data. It illustrates graphs for correct or misspecified models, discusses their pros and cons, and recalls the definition of metrics used.

Performance Comparison of Various Maximum Likelihood Nonlinear Mixed-Effects Estimation Methods for Dose–Response Models

Estimation methods for nonlinear mixed-effects modelling have considerably improved over the last decades. Nowadays, several algorithms implemented in different software are used. The present study aimed at comparing their performance for dose–response models. Eight scenarios were considered using a sigmoid Emax model, with varying sigmoidicity and residual error models. One hundred simulated datasets for each scenario were generated. One hundred individuals with observations at four doses constituted the rich design and at two doses, the sparse design. Nine parametric approaches for maximum likelihood estimation were studied: first-order conditional estimation (FOCE) in NONMEM and R, LAPLACE in NONMEM and SAS, adaptive Gaussian quadrature (AGQ) in SAS, and stochastic approximation expectation maximization (SAEM) in NONMEM and MONOLIX (both SAEM approaches with default and modified settings). All approaches started first from initial estimates set to the true values and second, using altered values. Results were examined through relative root mean squared error (RRMSE) of the estimates. With true initial conditions, full completion rate was obtained with all approaches except FOCE in R. Runtimes were shortest with FOCE and LAPLACE and longest with AGQ. Under the rich design, all approaches performed well except FOCE in R. When starting from altered initial conditions, AGQ, and then FOCE in NONMEM, LAPLACE in SAS, and SAEM in NONMEM and MONOLIX with tuned settings, consistently displayed lower RRMSE than the other approaches. For standard dose–response models analyzed through mixed-effects models, differences were identified in the performance of estimation methods available in current software, giving material to modellers to identify suitable approaches based on an accuracy-versus-runtime trade-off.

Pharmacometrics Markup Language (PharmML): Opening New Perspectives for Model Exchange in Drug Development

The lack of a common exchange format for mathematical models in pharmacometrics has been a long‐standing problem. Such a format has the potential to increase productivity and analysis quality, simplify the handling of complex workflows, ensure reproducibility of research, and facilitate the reuse of existing model resources. Pharmacometrics Markup Language (PharmML), currently under development by the Drug Disease Model Resources (DDMoRe) consortium, is intended to become an exchange standard in pharmacometrics by providing means to encode models, trial designs, and modeling steps.

Improved utilization of ADAS-cog assessment data through item response theory based pharmacometric modeling.

PurposeThis work investigates improved utilization of ADAS-cog data (the primary outcome in Alzheimer’s disease (AD) trials of mild and moderate AD) by combining pharmacometric modeling and item response theory (IRT).MethodsA baseline IRT model characterizing the ADAS-cog was built based on data from 2,744 individuals. Pharmacometric methods were used to extend the baseline IRT model to describe longitudinal ADAS-cog scores from an 18-month clinical study with 322 patients. Sensitivity of the ADAS-cog items in different patient populations as well as the power to detect a drug effect in relation to total score based methods were assessed with the IRT based model.ResultsIRT analysis was able to describe both total and item level baseline ADAS-cog data. Longitudinal data were also well described. Differences in the information content of the item level components could be quantitatively characterized and ranked for mild cognitively impairment and mild AD populations. Based on clinical trial simulations with a theoretical drug effect, the IRT method demonstrated a significantly higher power to detect drug effect compared to the traditional method of analysis.ConclusionA combined framework of IRT and pharmacometric modeling permits a more effective and precise analysis than total score based methods and therefore increases the value of ADAS-cog data.

Likert Pain Score Modeling: A Markov Integer Model and an Autoregressive Continuous Model

Pain intensity is principally assessed using rating scales such as the 11‐point Likert scale. In general, frequent pain assessments are serially correlated and underdispersed. The aim of this investigation was to develop population models adapted to fit the 11‐point pain scale. Daily Likert scores were recorded over 18 weeks by 231 patients with neuropathic pain from a clinical trial placebo group. An integer model consisting of a truncated generalized Poisson (GP) distribution with Markovian transition probability inflation was implemented in NONMEM 7.1.0. It was compared to a logit‐transformed autoregressive continuous model with correlated residual errors. In both models, the score baseline was estimated to be 6.2 and the placebo effect to be 19%. Developed models similarly retrieved consistent underlying features of the data and therefore correspond to platform models for drug effect detection. The integer model was complex but flexible, whereas the continuous model can more easily be developed, although requires longer runtimes.

Performance of Three Estimation Methods in Repeated Time-to-Event Modeling

It is not uncommon that the outcome measurements, symptoms or side effects, of a clinical trial belong to the family of event type data, e.g., bleeding episodes or emesis events. Event data is often low in information content and the mixed-effects modeling software NONMEM has previously been shown to perform poorly with low information ordered categorical data. The aim of this investigation was to assess the performance of the Laplace method, the stochastic approximation expectation–maximization (SAEM) method, and the importance sampling method when modeling repeated time-to-event data. The Laplace method already existed, whereas the two latter methods have recently become available in NONMEM 7. A stochastic simulation and estimation study was performed to assess the performance of the three estimation methods when applied to a repeated time-to-event model with a constant hazard associated with an exponential interindividual variability. Various conditions were investigated, ranging from rare to frequent events and from low to high interindividual variability. The method performance was assessed by parameter bias and precision. Due to the lack of information content under conditions where very few events were observed, all three methods exhibit parameter bias and imprecision, however most pronounced by the Laplace method. The performance of the SAEM and importance sampling were generally higher than Laplace when the frequency of individuals with events was less than 43%, while at frequencies above that all methods were equal in performance.

Modeling and simulation of count data.

Count data, or number of events per time interval, are discrete data arising from repeated time to event observations. Their mean count, or piecewise constant event rate, can be evaluated by discrete probability distributions from the Poisson model family. Clinical trial data characterization often involves population count analysis. This tutorial presents the basics and diagnostics of count modeling and simulation in the context of pharmacometrics. Consideration is given to overdispersion, underdispersion, autocorrelation, and inhomogeneity.

Modeling longitudinal daily seizure frequency data from pregabalin add-on treatment.

The purpose of this study was to describe longitudinal daily seizure count data with respect to the effects of time and pregabalin add‐on therapy. Models were developed in a stepwise manner: base model, time effect model, and time and drug effect (final) model, using a negative binomial distribution with Markovian features. Mean daily seizure count (λ) was estimated to be 0.385 (relative standard error [RSE] 3.09%) and was further increased depending on the seizure count on the previous day. An overdispersion parameter (OVDP), representing extra‐Poisson variation, was estimated to be 0.330 (RSE 11.7%). Interindividual variances on λ and OVDP were 84.7% and 210%, respectively. Over time, λ tended to increase exponentially with a rate constant of 0.272 year−1 (RSE 26.8%). A mixture model was applied to classify responders/nonresponders to pregabalin treatment. Within the responders, λ decreased exponentially with respect to dose with a constant of 0.00108 mg−1 (RSE 11.9%). The estimated responder rate was 66% (RSE 27.6%). Simulation‐based diagnostics showed the model reasonably reproduced the characteristics of observed data. Highly variable daily seizure frequency was successfully characterized incorporating baseline characteristics, time effect, and the effect of pregabalin with classification of responders/nonresponders, all of which are necessary to adequately assess the efficacy of antiepileptic drugs.

Approaches to Simultaneous Analysis of Frequency and Severity of Symptoms

Mechanistic models that synthesize pharmacological and (patho)physiological process information provide a rich basis for the characterization of drug action. However, the underlying clinical data are often simplified in a manner that does not allow models to fully elucidate the structure of the drug effect. In this article, we describe data‐simplification strategies that are in routine use to describe disease symptoms and compare them with a model developed for handling the true complexities of the data.

Transient Lower Esophageal Sphincter Relaxation Pharmacokinetic-Pharmacodynamic Modeling: Count Model and Repeated Time-To-Event Model

Transient lower esophageal sphincter relaxation (TLESR) is the major mechanism for gastroesophageal reflux. Characterizations of candidate compounds for reduction of TLESRs are traditionally done through summary exposure and response measures and would benefit from model-based analyses of exposure-TLESR events relationships. Pharmacokinetic (PK)-pharmacodynamic (PD) modeling approaches treating TLESRs either as count data or repeated time-to-event (RTTE) data were developed and compared in terms of their ability to characterize system and drug characteristics. Vehicle data comprising 294 TLESR events were collected from nine dogs. Compound [(R)-(+)-[2,3-dihydro-5-methyl-3-(4-morpholinylmethyl)pyrrolo[1,2,3-de]-1,4-benzoxazin-6-yl]-1-naphthalenylmethanone mesylate (WIN55212-2)] data containing 66 TLESR events, as well as plasma concentrations, were obtained from four dogs. Each experiment lasted for 45 min and was initiated with a meal. Counts in equispaced 5- and 1-min intervals were modeled based on a Poisson probability distribution model. TLESR events were analyzed with the RTTE model. The PK was connected to the PD with a one-compartment model. Vehicle data were described by a baseline and a surge function; the surge peak was determined to be approximately 9.69 min by all approaches, and its width in time at half-maximal intensity was 5 min (1-min count and RTTE) or 10 min (5-min count). TLESR inhibition by WIN55212-2 was described by an Imax model, with an IC50 of on average 2.39 nmol · l−1. Modeling approaches using count or RTTE data linked to a dynamic PK-PD representation of exposure are superior to using summary PK and PD measures and are associated with a higher power for detecting a statistically significant drug effect.