David A. Simmons

Accuracy Evaluation of Five Blood Glucose Monitoring Systems: The North American Comparator Trial

Background: This study evaluated differences in accuracy between the CONTOUR® NEXT EZ (EZ) blood glucose monitoring system (BGMS) and four other BGMSs [ACCU-CHEK® Aviva (ACAP), FreeStyle Freedom Lite® (FFL), ONE TOUCH® Ultra®2 (OTU2), and TRUEtrack® (TT)]. Methods: Up to three capillary blood samples (N = 393) were collected from 146 subjects with and without diabetes. One sample per subject was tested with fresh (natural) blood; the other samples were glycolyzed to lower blood glucose to <70 mg/dl. Meter results were compared with results from plasma from the same sample tested on a Yellow Springs Instruments (YSI) 2300 STAT Plus™ glucose analyzer. Blood glucose monitoring system accuracy was compared using mean absolute relative difference (MARD; from laboratory reference method results) and other analyses. Separate analyses on fresh (natural) samples only were conducted to determine potential effects of glycolysis on MARD values of systems utilizing glucose-oxidase-based test strip chemistry. Results: Across the tested glucose range, the EZ had the lowest MARD of 4.7%; the ACAP, FFL, OTU2, and TT had MARD values of 6.3%, 18.3%, 23.4%, and 26.2%, respectively. For samples with glucose concentrations <70 mg/dl, the EZ had the lowest MARD (0.65%), compared with the ACAP (2.5%), FFL (18.3%), OTU2 (22.4%), and TT (33.2%) systems. Conclusions: The EZ had the lowest MARD across the tested glucose ranges when compared with four other BGMSs when all samples were analyzed as well as when natural samples only were analyzed.

Evaluation of an Over-the-Counter Glycated Hemoglobin (A1C) Test Kit:

Background: Glycated hemoglobin (A1C) monitoring is an integral component of diabetes management. This study was conducted to evaluate the performance of the A1CNow® SELFCHECK device when used by lay users and health care professionals (HCPs) to measure A1C. Methods: Subjects performed two A1CNow SELFCHECK finger-stick self-tests followed by a finger-stick test of the subjects blood by a HCP. The primary endpoint assessed accuracy of the subject and HCP A1CNow SELFCHECK readings. Secondary endpoints included precision, comprehension of instructional material (written material ± DVD), and product satisfaction. For accuracy comparison, a venous blood sample was drawn from each subject and tested by laboratory (TOSOH) analysis. Subject comprehension of product instructional material was evaluated via first-time failure (FTF) rate as recorded by the HCP, and subject satisfaction was assessed through written survey. Results: A total of 110 subjects with (n = 93) and without (n = 17) diabetes participated. Of 177 subject A1C values, 165 (93.2%) were within the acceptable range of ±13.5% of the laboratory reference value and considered accurate. Regression analysis showed good correlation of subject values to laboratory and HCP results (R 2 = 0.93 for both). The average within-subject coefficient of variation was 4.57% (n = 74). The FTF rates with and without instructional DVD were 11.3% (n = 56) and 39.6% (n = 54), respectively. Subjects with diabetes/prediabetes overwhelmingly indicated that they were “very” to “extremely” likely (93.5%) to discuss their home A1C results with their HCP. Conclusions: Lay users found the A1CNow SELFCHECK easy to use, and both lay users and HCPs were able to measure A1C accurately.

How Should Blood Glucose Meter System Analytical Performance Be Assessed

Blood glucose meter system analytical performance is assessed by comparing pairs of meter system and reference instrument blood glucose measurements measured over time and across a broad array of glucose values. Consequently, no single, complete, and ideal parameter can fully describe the difference between meter system and reference results. Instead, a number of assessment tools, both graphical (eg, regression plots, modified Bland–Altman plots, and error grid analysis) and tabular (eg, International Organization for Standardization guidelines, mean absolute difference, and mean absolute relative difference) have been developed to evaluate meter system performance. The strengths and weaknesses of these methods of presenting meter system performance data, including a new method known as Radar Plots, are described here.

A new test strip technology platform for self-monitoring of blood glucose.

In the management of diabetes, accuracy of devices used for self-monitoring of blood glucose (SMBG) is critical because SMBG results can affect patient diabetes-related health outcomes. A new blood glucose monitoring system (BGMS) platform has been developed that is based on the new CONTOUR® NEXT (CN) test strip. This BGMS platform uses a proprietary electron mediator and algorithm to minimize errors at different steps in the testing process, thus minimizing outliers and significantly improving accuracy from prior-generation blood glucose meter systems. As demonstrated by questionnaire results from clinical studies with the new BGMS platform, accuracy and ease of use are important considerations for people with diabetes and their health care professionals when selecting an SMBG device. This article provides an overview of laboratory studies and clinical trials in the hands of lay users involving the performance of the portfolio of blood glucose meters that uses the new test strip. Each BGMS in the platform, which includes the CONTOUR XT (CONTOUR NEXT EZ in the United States), CONTOUR NEXT LINK, CONTOUR NEXT USB, and CN systems, demonstrated advanced accuracy both in the laboratory and in the hands of subjects (people with diabetes) and trained health care professionals. All systems met and exceeded International Organization for Standardization accuracy criteria (both ISO 15197:2003 and ISO 15197:2013). Each system in the new BGMS platform delivers advanced accuracy, which is essential to people who utilize SMBG for improved management.

Evaluation of a novel continuous glucose measurement device in patients with diabetes mellitus across the glycemic range.

Background: This glucose clamp study assessed the performance of an electrochemical continuous glucose monitoring (CGM) system for monitoring levels of interstitial glucose. This novel system does not require use of a trocar or needle for sensor insertion. Method: Continuous glucose monitoring sensors were inserted subcutaneously into the abdominal tissue of 14 adults with type 1 or type 2 diabetes. Subjects underwent an automated glucose clamp procedure with four consecutive post-steady-state glucose plateau periods (40 min each): (a) hypoglycemic (50 mg/dl) (b) hyperglycemic (250 mg/dl), (c) second hypoglycemic (50 mg/dl), and (d) euglycemic (90 mg/dl). Plasma glucose results obtained with YSI glucose analyzers were used for sensor calibration. Accuracy was assessed retrospectively for plateau periods and transition states, when glucose levels were changing rapidly (approximately 2 mg/dl/min). Results: Mean absolute percent difference (APD) was lowest during hypoglycemic plateaus (11.68%, 14.15%) and the euglycemic-to-hypoglycemic transition (14.21%). Mean APD during the hyperglycemic plateau was 17.11%; mean APDs were 18.12% and 19.25% during the hypoglycemic-to-hyperglycemic and hyperglycemic-to-hypoglycemic transitions, respectively. Parkes (consensus) error grid analysis (EGA) and rate EGA of the plateaus and transition periods, respectively, yielded 86.8% and 68.6% accurate results (zone A) and 12.1% and 20.0% benign errors (zone B). Continuous EGA yielded 88.5%, 75.4%, and 79.3% accurate results and 8.3%, 14.3%, and 2.4% benign errors for the euglycemic, hyperglycemic, and hypoglycemic transition periods, respectively. Adverse events were mild and unlikely to be device related. Conclusion: This novel CGM system was safe and accurate across the clinically relevant glucose range.

Performance of a New Blood Glucose Monitoring System in the Hands of Intended Users

BACKGROUND This study assessed the performance of a blood glucose monitoring system (BGMS) in development that uses a new generation of blood glucose test strips with capillary and venous blood in the hands of its intended users, people with diabetes and healthcare professionals (HCPs). SUBJECTS AND METHODS In total, 93 subjects ≥ 18 years old (median age, 33 years) with type 1 (78%) or type 2 (22%) diabetes participated. Untrained subjects performed self-test fingersticks using a Microlet(®)2 lancing device (Bayer HealthCare LLC, Diabetes Care, Tarrytown, NY) followed by testing of their own capillary blood on the BGMS. HCPs performed fingersticks (using a Tenderlett(®) lancing device [International Technidyne Corp., Edison, NJ]) and venipunctures on subjects and tested both capillary and venous samples from subjects on the BGMS. All BGMS results were compared with Yellow Springs Instruments (YSI) (YSI Life Sciences, Inc., Yellow Springs, OH) laboratory results. Analytical accuracy was assessed according to International Organization for Standardization (ISO) 15197:2003 guidelines (i.e., within ± 15 mg/dL or ± 20% of the YSI results for samples with glucose concentrations < 75 mg/dL and ≥ 75 mg/dL, respectively) and more stringent criteria (i.e., within ± 15 mg/dL or ± 15% of the YSI results for samples with glucose concentrations < 100 mg/dL and ≥ 100 mg/dL, respectively). RESULTS Overall, 98.9% (180/182) of subject Microlet2 capillary fingerstick results, 99.5% (182/183) of HCP Tenderlett capillary fingerstick results, and 100% (186/186) of venous results met current ISO criteria and more stringent criteria. The average hematocrit was 44%, with values ranging from 32% to 52%. CONCLUSIONS Test results from both capillary fingerstick and venous samples with a new BGMS in development met current accuracy guidelines as well as proposed tighter criteria.

The Quantitative Relationship Between ISO 15197 Accuracy Criteria and Mean Absolute Relative Difference (MARD) in the Evaluation of Analytical Performance of Self-Monitoring of Blood Glucose (SMBG) Systems.

The relationship between International Organization for Standardization (ISO) accuracy criteria and mean absolute relative difference (MARD), 2 methods for assessing the accuracy of blood glucose meters, is complex. While lower MARD values are generally better than higher MARD values, it is not possible to define a particular MARD value that ensures a blood glucose meter will satisfy the ISO accuracy criteria. The MARD value that ensures passing the ISO accuracy test can be described only as a probabilistic range. In this work, a Bayesian model is presented to represent the relationship between ISO accuracy criteria and MARD. Under the assumptions made in this work, there is nearly a 100% chance of satisfying ISO 15197:2013 accuracy requirements if the MARD value is between 3.25% and 5.25%.

Using Radar Plots to Demonstrate the Accuracy and Precision of 6 Blood Glucose Monitoring Systems

Background: Previously, fingertip capillary blood glucose measurements from the CONTOUR®NEXT (CN) blood glucose monitoring system (BGMS) and 5 other BGMSs were evaluated in comparison with measurements from a reference YSI glucose analyzer. Here, we use Radar Plots to graphically represent the accuracy and precision results from the previous study, including whether they met ISO 15197:2013 accuracy criteria. Method: A Radar Plot, a new method for capturing a distinct, single visualization of BGMS analytical performance, is a collection of concentric circles, each representing a particular magnitude of error. The center of the plot represents zero error (BGMS result is equivalent to reference result); as points are more distant from the center, the error increases, expressed in units of mg/dL or percentage for YSI values <100 and ≥100 mg/dL, respectively. The position of the data point above or below the horizontal line bisecting the plot indicates whether the BGMS measurement error was positive (BGMS result > YSI result) or negative (BGMS result < YSI result). Points within the “15-15 Zone,” representing ±15 mg/dL or ±15% error, satisfy ISO 15197:2013 accuracy criteria. Results: The percentage of results within the 15-15 Zone ranged from 83.6% to 99.8% for the 6 BGMSs (99.6% for CN). Conclusions: Radar Plots provide a different method for visually comparing the analytical performance of multiple BGMSs. The tight clustering of data points at the center of the CN Radar Plot illustrates the analytical performance of CN compared with 5 other BGMSs.

Fundamental Importance of Reference Glucose Analyzer Accuracy for Evaluating the Performance of Blood Glucose Monitoring Systems (BGMSs).

Background: As blood glucose monitoring system (BGMS) accuracy is based on comparison of BGMS and laboratory reference glucose analyzer results, reference instrument accuracy is important to discriminate small differences between BGMS and reference glucose analyzer results. Here, we demonstrate the important role of reference glucose analyzer accuracy in BGMS accuracy evaluations. Methods: Two clinical studies assessed the performance of a new BGMS, using different reference instrument procedures. BGMS and YSI analyzer results were compared for fingertip blood that was obtained by untrained subjects’ self-testing and study staff testing, respectively. YSI analyzer accuracy was monitored using traceable serum controls. Results: In study 1 (N = 136), 94.1% of BGMS results were within International Organization for Standardization (ISO) 15197:2013 accuracy criteria; YSI analyzer serum control results showed a negative bias (−0.64% to −2.48%) at the first site and a positive bias (3.36% to 6.91%) at the other site. In study 2 (N = 329), 97.8% of BGMS results were within accuracy criteria; serum controls showed minimal bias (<0.92%) at both sites. Conclusions: These findings suggest that the ability to demonstrate that a BGMS meets accuracy guidelines is influenced by reference instrument accuracy.

Re-evaluating a standard approach to assessing glucose monitor performance.

Mahoney and Ellison reviewed 52 articles published between 2002 and 2006 on handheld blood glucose monitor performance. They concluded that multiple guidelines for study design, methodology, and reporting exist but are often not adhered to, a fact that affected both the quality of some studies and the conclusions drawn from them. They proposed that the unavailability of guidelines from a single source contributed to this situation. In a second publication, these same authors proposed a 14point checklist based on a number of published guidelines with the intent that this list be a mandatory, single source for the design and reporting of handheld blood glucose monitor evaluations. While we appreciate these efforts and agree with the spirit of the attempt to standardize and raise the quality of blood glucose monitor evaluations in general, we have concerns regarding several issues. In general, our concerns include: