Varun Ramamohan

A Monte Carlo approach to the estimation & analysis of uncertainty in clinical laboratory measurement processes

Clinical laboratory testing is a vital component of many stages of the medical decision making process, and therefore information about the quality of the measurement process is critical to the medical decision-making process. A statement of uncertainty of the result of a laboratory test provides this information. To obtain this information, the clinical laboratory measurement process is conceptualized as a self-contained system, the concept of process phases is introduced, and a broadly applicable algorithm describing the modeling and estimation of uncertainty of such processes is developed. The article discusses how performance specifications for individual components can be used to characterize their uncertainty, and uses Monte Carlo simulation to integrate these individual component uncertainties into a net system uncertainty. The proposed approach is illustrated by developing a mathematical model of the serum cholesterol assay analysis procedure. The uses of the model are to: 1) simulate, evaluate and optimize quality control policies without resorting to conducting controlled experiments, 2) obtain performance targets for the measurement process by using uncertainty estimates from the simulation, 3) estimate the contribution of each source of uncertainty to the net system uncertainty, and 4) study the effects of varying the parameters of the system on the net system uncertainty are illustrated with examples.

Application of mathematical models of system uncertainty to evaluate the utility of assay calibration protocols

Abstract Background: Laboratory protocols used to calibrate commercial clinical chemistry systems affect test result quality. Mathematical models of system uncertainty can be developed using performance parameters provided by the manufacturer for various subsystems. These models can be used to evaluate protocols for specific laboratory operations. Methods: A mathematical model was developed to estimate the uncertainty inherent in the Roche Diagnostics P-Modular system, and included uncertainties associated with the sample and reagent pipettes, spectrometer and the calibration process. The model was then used to evaluate various alternate calibration protocols: calibration based on mean of replicate measurements (n=1–6) and calibration based on conditional acceptance when the following quality control specimen was within one standard deviation of target. The effect of calibrator concentrations on assay measurement uncertainty was also studied, and calibrator concentrations that minimize uncertainty at a specific concentration were identified. Results: The simulation model produced uncertainty estimates of 3.5% for the serum cholesterol assay and identified sample pipette (40%) and spectrometer (21%) as the largest contributors to measurement uncertainty. Each additional replicate calibrator measurements result in diminishing reductions in measurement uncertainty, with maximum reductions (19%) achieved with five replicate measurements. The conditional acceptance of calibration only when the control was within 1s resulted in an 18% reduction. Conclusions: The model can be used to evaluate the utility of laboratory protocols and establish realistic assay performance targets. The model also can help instrument manufacturers and laboratorians identify major contributors to assay measurement uncertainty, which helps improve performance in future assay systems.

Category-specific uncertainty modeling in clinical laboratory measurement processes.

Abstract Background: A statement of measurement uncertainty describes the quality of a clinical assay analysis result, and uncertainty models of clinical assays can be used to evaluate and optimize laboratory protocols designed to minimize the measurement uncertainty associated with an assay. In this study, we propose a methodology to lend systematic structure to the uncertainty modeling process. Methods: Clinical laboratory assays are typically classified based on the chemical reaction involved, and therefore, based on the assay analysis methodology. We use this fact to demonstrate that uncertainty models for assays within the same category are structurally identical in all respects except for the values of certain model parameters. This is accomplished by building uncertainty models for assays belonging to two categories – substrate assays based on optical absorbance analysis of endpoint reactions, and ion selective electrode (ISE) assays based on potentiometric measurements of electromotive force. Results: Uncertainty models for the substrate assays and the ISE assays are built, and for each category, a general mathematical framework for the uncertainty model is developed. The parameters of the general framework that vary from assay to assay for each category are identified and listed. Conclusions: Estimates of measurement uncertainty from the models were compared with estimates of uncertainty from quality control data from the clinical laboratory. We demonstrate that building a general modeling framework for each assay category and plugging in parameter values for each assay is sufficient to generate uncertainty models for an assay within a given category.

Modeling the effect of instrument drift in clinical laboratories: A serum bilirubin assay case study:

Clinical laboratory tests play a vital role in the medical decision making process, including diagnosis, prognostic assessment and drug dosage prescription. Drift or degradation in the performance of the analytic instrument over time can have a significant effect on the uncertainty of the clinical laboratory measurement test result. In this paper, we model the drift in the analytic instrumentation used to perform the laboratory tests, and estimate its effect on the uncertainty of the measurement result. This is accomplished developing a physics-based mathematical model of the total bilirubin laboratory test that describes the measurement result as a function of various sources of uncertainty operating within the total bilirubin measurement process. The Monte Carlo method is used to estimate the uncertainty associated with this model. Drift in the instrument is modeled as affecting both the mean (inaccuracy) and the standard deviation (imprecision) of each source of uncertainty. Further, recalibrating the instrument is postulated as a method to nullify the effect of instrument drift on inaccuracy of the measurement result, and the model is used to estimate the average time interval between successive calibrations such that the drift does not exceed clinically significant total error limits and prevents misdiagnosis.

Effect of uncertainty in calibration on the correlation structure of the rheumatoid factor immunoassay calibration function

Clinical laboratory measurements are vital to the medical decision-making process, and specifically, measurement of rheumatoid factor antibodies is part of the disease criteria for various autoimmune conditions. Uncertainty estimates describe the quality of the measurement process, and uncertainty in calibration of the instrument used in the measurement can be an important contributor to the net measurement uncertainty. In this paper, we develop a physics-based mathematical model of the rheumatoid factor measurement process, or assay, and then use the Monte Carlo method to investigate the effect of uncertainty in the calibration process on the correlation structure of the parameters of the calibration function. We demonstrate numerically that a change in uncertainty of the calibration process can be quantified by one of two metrics: (1) the 1-norm condition number of the correlation matrix, or (2) the sum of the absolute values of the correlation coefficients between the parameters of the calibration function.

A mathematical model of measurement uncertainty of single substrate enzyme assays

Clinical laboratory tests provide critical information at every stage of the medical decision‐making process, and measurement of the activity levels of enzymes such as alkaline phosphatase, lactate dehydrogenase, etc. provide information regarding various body functions such as the liver and gastrointestinal tract. The uncertainty associated with these enzyme measurement processes describes the quality of the measurement process, and therefore methods to improve the quality of the measurement process require minimizing the measurement uncertainty of the enzyme assay. In this study, we develop a mathematical model of the lactate dehydrogenase measurement process, with uncertainty introduced into its parameters that represent the sources of variation in the different components and stages of the measurement process. The Monte Carlo method is then utilized to estimate the uncertainty associated with the model, and therefore the measurement process. An empirical function used to generate estimates of uncertainty for patient samples with unknown activity levels is constructed using the model. The model is then used to quantify the contributions of the individual sources of uncertainty to the net measurement uncertainty and also quantify the effect of uncertainty within the calibration process on the distribution of the measurement result. Copyright

A simulation-based approach to modeling the uncertainty of two-substrate clinical enzyme measurement processes

Results of clinical laboratory tests inform every stage of the medical decision-making process, and measurement of enzymes such as alanine aminotransferase provide vital information regarding the function of organ systems such as the liver and gastrointestinal tract. Estimates of measurement uncertainty quantify the quality of the measurement process, and therefore, methods to improve the quality of the measurement process require minimizing assay uncertainty. To accomplish this, we develop a physics-based mathematical model of the alanine aminotransferase assay, with uncertainty introduced into its parameters that represent variation in the measurement process, and then use the Monte Carlo method to quantify the uncertainty associated with the model of the measurement process. Furthermore, the simulation model is used to estimate the contribution of individual sources of uncertainty as well as that of uncertainty in the calibration process to the net measurement uncertainty.