Stuart L. Beal

SOME SUGGESTIONS FOR MEASURING PREDICTIVE PERFORMANCE

The performance of a prediction or measurement method is often evaluated by computing the correlation coefficient and/or the regression of predictions on true (reference) values. These provide, however, only a poor description of predictive performance. The mean squared prediction error (precision) and the mean prediction error (bias) provide better descriptions of predictive performance. These quantities are easily computed, and can be used to compare prediction methods to absolute standards or to one another. The measures, however, are unreliable when the reference method is imprecise. The use of these measures is discussed and illustrated.

Evaluation of methods for estimating population pharmacokinetic parameters. I. Michaelis-menten model: Routine clinical pharmacokinetic data

Individual pharmacokinetic parameters quantify the pharmacokinetics of an individual, while population pharmacokinetic parameters quantify population mean kinetics, interindividual variability, and residual intraindividual variability plus measurement error. Individual pharmacokinetics are estimated by fitting individual data to a pharmacokinetic model. Population pharmacokinetic parameters are estimated either by fitting all individuals data together as though there were no individual kinetic differences (the naive pooled data approach), or by fitting each individuals data separately, and then combining the individual parameter estimates (the two-stage approach). A third approach, NONMEM, takes a middle course between these, and avoids shortcomings of each of them. A data set consisting of 124 steady-state phenytoin concentration-dosage pairs from 49 patients, obtained in the routine course of their therapy, was analyzed by each method. The resulting population parameter estimates differ considerably (population mean Km, for example, is estimated as 1.57, 5.36, and 4.44 μg/ml by the naive pooled data, two-stage, and NONMEM approaches, respectively). Simulations of the data were analyzed to investigate these differences. The simulations indicate that the pooled data approach fails to estimate variabilities and produces imprecise estimates of mean kinetics. The two-stage appproach produces good estimates of mean kinetics, but biased and imprecise estimates of interindividual variability. NONMEM produces accurate and precise estimates of all parameters, and also reasonable confidence intervals for them. This performance is exactly what is expected from theoretical considerations and provides empirical support for the use of NONMEM when estimating population pharmacokinetics from routine type patient data.

Evaluation of methods for estimating population pharmacokinetic parameters II. Biexponential model and experimental pharmacokinetic data

Individual pharmacokinetic parameters quantify the pharmacokinetics of an individual, while population pharmacokinetic parameters quantify population mean kinetics, interindividual variability, and residual variability, including intraindividual variability and measurement error. Individual pharmacokinetics are estimated by fitting individual data to a pharmacokinetic model. Population pharmacokinetic parameters have been estimated either by fitting all individuals data together as though there were no individual kinetic differences, the naive pooled data (NPD) approach, or by fitting each individuals data separately and then combining the individual parameter estimates, the two stage (TS) approach. A third approach, NONMEM, takes a middle course between these. This study provides further evidence of NONMEMs validity by comparing, using simulation, the three approaches on three types of data sets corresponding to three typical types of pharmacokinetic studies. The estimates of population parameters provided by the NPD method are poorer than those provided by either of the other methods. The estimates provided by the TS method are adequate for mean values and for residual variability, but not for interindividual kinetic variability. NONMEMs estimates are as good as those of the TS method for mean parameters and for residual variability, and considerably better for interindividual variability. The latter estimates are still not acceptable in an absolute sense. This is probably due, not to an intrinsic fault of the method (as it is in the case of the TS approach), but to an insufficient number of individuals being studied.

Evaluation of methods for estimating population pharmacokinetic parameters. III. Monoexponential model: routine clinical pharmacokinetic data.

Individual pharmacokinetic parameters quantify the pharmacokinetics of an individual, while population pharmacokinetic parameters quantify population mean kinetics, interindividual kinetic variability, and residual variability, including intraindividual variability and measurement error. Individual pharmacokinetics are estimated by fitting a pharmacokinetic model to individual data. Population pharmacokinetic parameters have traditionally been estimated by doing this separately for each individual, and then combining the individual parameter estimates, the Standard Two Stage (STS) approach. Another approach, NONMEM, appropriately pools data across individuals and is therefore less dependent on individual parameter estimates. This study provides further evidence of NONMEMs validity and usefulness by comparing both approaches on simulated routine-type pharmacokinetic data arising from a monoexponential model. The estimates of population parameters (notably those describing interindividual variability) provided by the STS method are poorer than those provided by NONMEM, especially when there is considerable residual error. Further, NONMEMs estimates of population parameters do not require that the data be restricted to special types of routine data such as those obtained only at steady state, or only at peak or trough, nor do the estimates improve with such data. NONMEMs estimates do improve, however, when a data set is enhanced by the addition of single-observation-per-individual type data. Thus, population parameters can be estimated efficiently from data that simulate real clinical pharmacokinetic conditions.

Extended least squares nonlinear regression: A possible solution to the “choice of weights” problem in analysis of individual pharmacokinetic data

It is often difficult to specify weights for weighted least squares nonlinear regression analysis of pharmacokinetic data. Improper choice of weights may lead to inaccurate and/or imprecise estimates of pharmacokinetic parameters. Extended least squares nonlinear regression provides a possible solution to this problem by allowing the incorporation of a general parametric variance model. Weighted least squares and extended least squares analyses of data from a simulated pharmacokinetic experiment were compared. Weighted least squares analysis of the simulated data, using commonly used weighting schemes, yielded estimates of pharmacokinetic parameters that were significantly biased, whereas extended least squares estimates were unbiased. Extended least squares estimates were often significantly more precise than were weighted least squares estimates. It is suggested that extended least squares regression should be further investigated for individual pharmacokinetic data analysis.

Three new residual error models for population PK/PD analyses

Residual error models, traditionally used in population pharmacokinetic analyses, have been developed as if all sources of error have properties similar to those of assay error. Since assay error often is only a minor part of the difference between predicted and observed concentrations, other sources, with potentially other properties, should be considered. We have simulated three complex error structures. The first model acknowledges two separate sources of residual error, replication error plus pure residual (assay) error. Simulation results for this case suggest that ignoring these separate sources of error does not adversely affect parameter estimates. The second model allows serially correlated errors, as may occur with structural model misspecification. Ignoring this error structure leads to biased random-effect parameter estimates. A simple autocorrelation model, where the correlation between two errors is assumed to decrease exponentially with the time between them, provides more accurate estimates of the variability parameters in this case. The third model allows time-dependent error magnitude. This may be caused, for example, by inaccurate sample timing. A time-constant error model fit to time-varying error data can lead to bias in all population parameter estimates. A simple two-step time-dependent error model is sufficient to improve parameter estimates, even when the true time dependence is more complex. Using a real data set, we also illustrate the use of the different error models to facilitate the model building process, to provide information about error sources, and to provide more accurate parameter estimates.

Interaction Between Structural, Statistical, and Covariate Models in Population Pharmacokinetic Analysis

The influence of the choice of pharmacokinetic model on subsequent determination of covariate relationships in population pharmacokinetic analysis was studied using both simulated and real data sets. Simulations and data analysis were both performed with the program NONMEM. Data were simulated using a two-compartment model, but at late sample times, so that preferential selection of the two-compartment model should have been impossible. A simple categorical covariate acting on clearance was included. Initially, on the basis of a difference in the objective function values, the two-compartment model was selected over the one-compartment model. Only when the complexity of the one-compartment model was increased in terms of the covariate and statistical models was the difference in objective function values of the two structural models negligible. For two real data sets, with which the two-compartment model was not selected preferentially, more complex covariate relationships were supported with the one-compartment model than with the two-compartment model. Thus, the choice of structural model can be affected as much by the covariate model as can the choice of covariate model be affected by the structural model; the two choices are interestingly intertwined. A suggestion on how to proceed when building population pharmacokinetic models is given.

Computation of the explicit solution to the Michaelis-Menten equation

An explicit solution to the Michaelis-Menten differential equation with bolus and zero-order input is presented. This solution involves certain simple functions whose values are not readily available. Efficient algorithms for computing values of these functions to any desired degree of accuracy and FORTRAN codes for implementing them are also given.

On the solution to the Michaelis-Menten equation

An “explicit” solution to the Michaelis-Menten differential equation is presented. Instead of involving the exponential function, the solution involves another simple function. A table for this function is presented.

Some clarifications regarding moments of residence times with pharmacokinetic models.

The stochastic formulation of linear kinetic models is elaborated in order to introduce some new concepts and help clarify the meaning and role of residence time moments. Certain conditional moments are introduced. Multicompartment and steady-state dosing within the stochastic context are considered. A general model-independent formula for steady state volume of distribution and a new concept of steady-state moments are presented. A technique for constructing a model of a given topology from its moments is also given.