Lilly Q. Yue

Comparison between surrogate indexes of insulin sensitivity/resistance and hyperinsulinemic euglycemic clamp estimates in rats.

Assessing insulin resistance in rodent models gives insight into mechanisms that cause type 2 diabetes and the metabolic syndrome. The hyperinsulinemic euglycemic glucose clamp, the reference standard for measuring insulin sensitivity in humans and animals, is labor intensive and technically demanding. A number of simple surrogate indexes of insulin sensitivity/resistance have been developed and validated primarily for use in large human studies. These same surrogates are also frequently used in rodent studies. However, in general, these indexes have not been rigorously evaluated in animals. In a recent validation study in mice, we demonstrated that surrogates have a weaker correlation with glucose clamp estimates of insulin sensitivity/resistance than in humans. This may be due to increased technical difficulties in mice and/or intrinsic differences between human and rodent physiology. To help distinguish among these possibilities, in the present study, using data from rats substantially larger than mice, we compared the clamp glucose infusion rate (GIR) with surrogate indexes, including QUICKI, HOMA, 1/HOMA, log (HOMA), and 1/fasting insulin. All surrogates were modestly correlated with GIR (r = 0.34-0.40). Calibration analyses of surrogates adjusted for body weight demonstrated similar predictive accuracy for GIR among all surrogates. We conclude that linear correlations of surrogate indexes with clamp estimates and predictive accuracy of surrogate indexes in rats are similar to those in mice (but not as substantial as in humans). This additional rat study (taken with the previous mouse study) suggests that application of surrogate insulin sensitivity indexes developed for humans may not be appropriate for determining primary outcomes in rodent studies due to intrinsic differences in metabolic physiology. However, use of surrogates may be appropriate in rodents, where feasibility of clamps is an obstacle and measurement of insulin sensitivity is a secondary outcome.

Statistical and Regulatory Issues with the Application of Propensity Score Analysis to Nonrandomized Medical Device Clinical Studies

Propensity score analysis is a versatile statistical method used mainly in observational studies for improving treatment comparison by adjusting for up to a relatively large number of potentially confounding covariates. Recently, there has been an increased interest in applying this method to nonrandomized medical device clinical studies. In the application of the methodology, some statistical and regulatory issues arise in both study design and analysis of study results, such as the need for pre-specifying clinically relevant covariates to be measured, appropriate patient populations, and the essential elements of statistical analysis, planning sample size in the context of propensity score methodology, handling missing covariates in generating propensity scores, and assessing the success of the propensity score method by evaluating treatment group overlap in terms of the distributions of propensity scores. In this paper, the advantages and limitations of this methodology will be revisited, and the above issues will be discussed and illustrated with examples from a regulatory perspective.

Statistical and Regulatory Issues in Nonrandomized Medical Device Clinical Studies

While randomized, well-controlled, clinical trials have been viewed as the gold standard in the evaluation of medical products, it is not uncommon for medical device clinical studies to depart from the paradigm of randomized trials, due to ethical or practical reasons. In nonrandomized studies, the advantages of well-designed and conducted randomized clinical trials are no longer available, and consequently the statistical inference obtained from such studies may carry a lower level of scientific assurance, compared to randomized trials. This paper provides a brief overview of nonrandomized medical device clinical studies in terms of design and statistical analysis as well as regulatory issues, including some challenges that frequently arise in those endeavors.

Mixed Noninferiority Margin and Statistical Tests in Active Controlled Trials

In an active controlled noninferiority trial without a placebo arm, one of the major considerations is the selection of the noninferiority margin. Although the ICH E10 guideline provides general principles for the selection of appropriate noninferiority margins, there are no established rules or gold standards for the selection of noninferiority margins in active control trials. Hung et al. (2003) proposed a margin selection based on relative risk. However, with relative risk, it is difficult to adjust for covariates. On the other hand, Chow and Shao (2006) proposed a method for selecting noninferiority margins based on treatment difference. The determination of noninferiority margin based on either a test for treatment difference or a test for relative risk would be critical. In this paper, we propose a method for noninferiority testing with the use of a mixed null hypothesis. The mixed null hypothesis consists of a margin based on treatment difference and a margin based on relative risk. Both noninferiority margins will simultaneously satisfy the principles as described in the ICH E10 guideline. Statistical tests for mixed noninferiority margin are also derived. An example concerning the efficacy of a test therapy to an active control on a clinical adverse event in the target patient population with cardiovascular disease is presented to illustrate the proposed method. Simulation studies were also conducted to assess the type I error rate and the power.

Regulatory Issues of Propensity Score Methodology Application to Drug and Device Safety Studies

While randomized, well-controlled, clinical trials have been viewed as the gold standard in the evaluation of medical products, including drugs, biological products, and medical devices, it is not uncommon for safety assessment to be performed using observational studies, for ethical or practical reasons. In observational studies, various biases could be introduced in every stage and aspect of study, and consequently the resulting statistical inference may carry a lower level of scientific assurance, compared to randomized trials. To ensure the objectivity of study design and interpretability of the results, it is critical to address the challenges of such studies. In this paper, we share regulatory considerations on the prospective design of observational studies to address safety issues using propensity score methodology.

Special Issue on Medical Device Clinical Studies – Guest Editor's Note

A medical device is an item for treating or diagnosing a health condition whose intended use is not achieved primarily by chemical or biological action within the body (Section 201(h) of the Federal Food Drug & Cosmetic (FD&C) Act). Simply put, a medical device is any medical item that is not a drug or biological product, a category that encompasses an extremely broad range of products. Usually, the development of a device is an evolutionary process that begins with a creative idea of invention which is then continually improved over time, even during the pre-market study. This special issue on medical device features discussions on clinical study design and data analysis for therapeutic and diagnostic devices from Frequentist and Bayesian perspectives. In the paper titled “Statistics in the World of Medical Devices: The Contrast with Pharmaceuticals”, Campbell explores similarities and differences between medical devices and pharmaceutical drugs in terms of their nature, industrial culture, and how they are regulated in the U.S. and globally. The author discusses statistical issues concerning the evaluation of devices vs. those of drugs, and highlights new challenges for the statistical world in the development and evaluation of these new medical products. While randomized, well-controlled, clinical trials have been viewed as the gold standard in the evaluation of medical products, it is not uncommon for medical device clinical studies to depart from the paradigm of randomized trials, due to ethical or practical reasons. Li and Yue provide a brief overview of nonrandomized medical device clinical studies in terms of design and statistical analysis as well as regulatory issues, including some challenges that frequently arise in those endeavors. They hope that those issues and challenges will receive increased attention as nonrandomized studies continue to play an important role in the evaluation of the safety and effectiveness of medical devices. On the other hand, the issues and challenges should also serve to demonstrate why randomized trials are still preferred and strongly encouraged whenever possible, especially in the development of new technology in the medical device world.

Issues in the Use of Existing Data: As Controls in Pre-Market Comparative Clinical Studies

Randomized, well-controlled clinical trials have been viewed as the gold standard in the evaluation of medical products, and observational comparative clinical studies also play an important role in the evaluation in both premarket and postmarket settings. Such observational comparative studies could be concurrent or nonconcurrent depending on the timing when patients get treated. A nonconcurrent control group could be formed from patients with existing data, when indeed appropriate. For example, a control group could come from patients with historical data collected from earlier investigational device exemption (IDE) studies of previously approved medical products or selected from a well-designed and executed registry database. However, the construction of a control group from existing data presents extra challenges compared to the formation of a concurrent control group. In this chapter, some of the design challenges, such as validity of study design, historical control group selection and treatment group comparability, and identification of a control group from an applicable registry database, are discussed and illustrated with examples from regulatory perspectives.

Guest Editors’ Note: Special Issue Associated With the 2013 ASA Biopharmaceutical FDA/Industry Statistics Workshop

As co-chairs of the 2013 ASA Biopharmaceutical FDA/Industry Statistics Workshop, we have noted that unlike many workshops and conferences, there was no special issue in a journal celebrating the wide variety and high quality of the topics discussed at this popular annual workshop. During the initial meeting of our steering committee, a gracious invitation was put forth by Shein-Chung Chow to create a special issue in the Journal of Biopharmaceutical Statistics (JBS) in association with the 2013 workshop. As co-chairs, with the support of our steering committee, we thought this was an excellent idea, and this issue of JBS is the culmination of that idea. This issue contains 11 papers, some invited by us in our role as guest editors, and some contributed by presenters at the 2013 meeting. During the workshop, there were two sessions held with invited presentations dedicated to the special issue. The contributed papers were presented within their own sessions. The 11 papers contained in this special issue are contributed from industry, academia, and government statisticians, and well represent the cross section of presenters and attendees who attend the workshop on an annual basis. We are proud to have initiated what we hope will be a long tradition of special issues in journals associated with the workshop, as we both believe that the workshop is one of the most valuable and fruitful conferences held each year under the broad umbrella of medical product statistical issues. The papers presented in this special issue are of a wide variety. In the article titled “Enhancing Trial Integrity by Protecting the Independence of Data Monitoring Committees in Clinical Trials,” Fleming et al. reinforce the importance of protecting the independence of the data monitoring committee (DMC) in fulfilling its responsibility of safeguarding the interests of subjects while enhancing the credibility and integrity of the trial. The authors have identified some specific issues that could diminish the level of independence of these committees, provided insights into how these issues have emerged and their importance, and made recommendations for approaches to effectively address them. The article is very useful to those who implement the DMC process or who rely on or serve as DMC members. Gallo et al. explore alternative views on setting clinical trial futility criteria. The literature is ripe with descriptions of a variety of approaches to set criteria for futility stopping, including rules based on conditional power, predictive power, predictive probability, and beta spending functions, among other approaches. The authors consider an objective approach that sensibly balances the risks of making incorrect decisions (e.g.,