Richard C. Zink

Summarizing the incidence of adverse events using volcano plots and time intervals

Background Adverse event incidence analyses are a critical component for describing the safety profile of any new intervention. The results typically are presented in lengthy summary tables. For therapeutic areas where patients have frequent adverse events, analysis and interpretation are made more difficult by the sheer number and variety of events that occur. Understanding the risk in these instances becomes even more crucial. Purpose We describe a space-saving graphical summary that overcomes the limitations of traditional presentations of adverse events and improves interpretability of the safety profile. Methods We present incidence analyses of adverse events graphically using volcano plots to highlight treatment differences. Data from a clinical trial of patients experiencing an aneurysmal subarachnoid hemorrhage are used for illustration. Adjustments for multiplicity are illustrated. Results Color is used to indicate the treatment with higher incidence; bubble size represents the total number of events that occur in the treatment arms combined. Adjustments for multiple comparisons are displayed in a manner to indicate clearly those events for which the difference between treatment arms is statistically significant. Furthermore, adverse events can be displayed by time intervals, with multiple volcano plots or animation to appreciate changes in adverse event risk over time. Such presentations can emphasize early differences across treatments that may resolve later or highlight events for which treatment differences may become more substantial with longer follow-up. Limitations Treatment arms are compared in a pairwise fashion. Conclusions Volcano plots are space-saving tools that emphasize important differences between the adverse event profiles of two treatment arms. They can incorporate multiplicity adjustments in a manner that is straightforward to interpret and, by using time intervals, can illustrate how adverse event risk changes over the course of a clinical trial.

On the Importance of a Single Data Standard

The Clinical Data Interchange Standards Consortium (CDISC) has developed standards for data models, study design, and supporting clinical trial documents. CDISC standards have made such gains that the Center for Drug Evaluation and Research strongly encourages their use and implementation for submission of drug applications. However, despite the advances and improvements in these standards over the years and the ever-looming threat that they may someday be a requirement for drug applications, many are skeptical of a single standard and fail to understand the far-reaching advantages of adopting it. In this article we address the concerns and present numerous tangible benefits of CDISC standards.

Assessment of Methods to Identify Patient Subgroups with Enhanced Treatment Response in Randomized Clinical Trials

In contrast to the “one-size-fits-all” approach of traditional drug development, the need to identify subjects with an enhanced treatment effect is a critical component for tailored therapeutics or personalized medicine. Typically, the goal is to determine which patient receives additional benefit from the treatment in terms of an efficacy response. Alternatively, finding subgroups based on the important safety endpoints could be considered to identify those individuals experiencing a reduced risk of key adverse events, or to identify subjects for whom the new therapy may be inappropriate. A number of methods for identifying subgroups with enhanced treatment response have been developed recently, and it is natural to expect many more in the coming years. In order for the development programs for tailored therapeutics to be successful, it is imperative to identify the best method(s) for subgroup identification to be applied in practice. Further, it is likely that no single method will be optimal across all scenarios, so fully characterizing the properties of each methodology is of the utmost importance. To accomplish these goals, the researchers who develop every new and existing method should ideally make use of the same set of simulated data scenarios and report their findings using the same performance measures. We outline and describe the key attributes and scenarios for simulated data as well as the performance measures to enable consistent and rigorous assessment of subgroup identification methods.

Statistical and graphical approaches for disproportionality analysis of spontaneously-reported adverse events in pharmacovigilance.

AIM Combine disproportionality analysis with dynamically interactive graphics to understand spontaneously-reported adverse events in pharmacovigilance. METHODS Four statistical methods, including Reporting Odds Ratio, Proportional Reporting Ratio, Multi-Item Gamma Poisson Shrinker and Bayesian Confidence Propagation Neural Network that are used for computing disproportionality are described. Tree maps and other graphical techniques are used to display the disproportionality results. RESULTS Spontaneously-reported adverse events in pharmacovigilance are collected from physicians, patients, or the medical literature by regulatory agencies, pharmaceutical companies and device manufacturers to monitor the safety of a product once it reaches the market. In order to identify potential safety-signals, disproportionality analysis methods compare the rate at which a particular event of interest co-occurs with a given drug with the rate this event occurs without the drug in the event database. Tree maps are employed to interactively display the adverse events for particular drugs and compare the adverse events among the drugs. CONCLUSION Interactive graphical displays of disproportionality allow the analyst to quickly identify safety signals and perform additional follow-up analyses. Combining statistical methods with dynamically interactive graphics affords insights into the data inaccessible by traditional analysis methods.

Evaluation and Review of Strategies to Assess Cardiovascular Risk in Clinical Trials in Patients with Type 2 Diabetes Mellitus

This article is a result of the efforts of the American Statistical Association Biopharmaceutical Section Working Group on Safety. With representatives from different institutions, this group reviewed the drugs approved by the United States Food and Drug Administration (FDA) to treat Type 2 diabetes mellitus during 2002–2014 with a focus on the cardiovascular (CV) risk assessment. The main objective of this article is to understand the impact of FDA guidance of 2008 on assessment of CV risk in antidiabetes development programs, which are summarized and displayed in chronological order. Compared to New Drug Applications (NDAs) submitted prior to the FDA 2008 guidance, the number of patient-years significantly increased for NDAs approved in the post-guidance era. To meet guidance requirements on CV risk assessment, meta-analyses and large cardiovascular outcome trials (CVOTs) have been conducted. These CVOTs provide an opportunity to assess safety signals beyond CV risk and assess the benefit/risk ratio better in diabetic patients with a high risk for CV events, but they also present challenges. The advantages and disadvantages of different CV assessment strategies are summarized in this manuscript. Finally, we raise some emerging questions and discuss future opportunities for CV risk assessment research. Supplementary materials for this article are available online.

ORTH: R and SAS software for regression models of correlated binary data based on orthogonalized residuals and alternating logistic regressions

This article describes a new software for modeling correlated binary data based on orthogonalized residuals, a recently developed estimating equations approach that includes, as a special case, alternating logistic regressions. The software is flexible with respect to fitting in that the user can choose estimating equations for association models based on alternating logistic regressions or orthogonalized residuals, the latter choice providing a non-diagonal working covariance matrix for second moment parameters providing potentially greater efficiency. Regression diagnostics based on this method are also implemented in the software. The mathematical background is briefly reviewed and the software is applied to medical data sets.

Sources of Safety Data and Statistical Strategies for Design and Analysis: Postmarket Surveillance

Background: Safety data are continuously evaluated throughout the life cycle of a medical product to accurately assess and characterize the risks associated with the product. The knowledge about a medical product’s safety profile continually evolves as safety data accumulate. Methods: This paper discusses data sources and analysis considerations for safety signal detection after a medical product is approved for marketing. This manuscript is the second in a series of papers from the American Statistical Association Biopharmaceutical Section Safety Working Group. Results: We share our recommendations for the statistical and graphical methodologies necessary to appropriately analyze, report, and interpret safety outcomes, and we discuss the advantages and disadvantages of safety data obtained from passive postmarketing surveillance systems compared to other sources. Conclusions: Signal detection has traditionally relied on spontaneous reporting databases that have been available worldwide for decades. However, current regulatory guidelines and ease of reporting have increased the size of these databases exponentially over the last few years. With such large databases, data-mining tools using disproportionality analysis and helpful graphics are often used to detect potential signals. Although the data sources have many limitations, analyses of these data have been successful at identifying safety signals postmarketing. Experience analyzing these dynamic data is useful in understanding the potential and limitations of analyses with new data sources such as social media, claims, or electronic medical records data.

Introduction to the Special Section for Sources of Safety Data and Statistical Strategies for Design and Analysis

In mid-2016, having just completed a second manuscript examining several statistical issues related to cardiovascular outcome trials for patients with type 2 diabetes mellitus (T2DM), the Safety Working Group of the Biopharmaceutical Section of the American Statistical Association (ASA) found itself in need of a new research topic. After considering many possibilities, the group decided to focus on the strengths and limitations of the various data sources used to evaluate patient safety throughout the medical product life cycle. However, summarizing the data sources alone paints a somewhat incomplete picture. As any statistician will tell you, how these data are collected, the design (or lack thereof) of the experiment from which they are collected, and the quality of the measurements themselves play a key role in how the data are analyzed and, ultimately, how the analysis is interpreted and communicated. Therefore, detailed descriptions of the data, design, and appropriate analysis methods are required to fully characterize the challenges of patient safety in medical product development. Initially, the plan was to write a single manuscript, but it became clear very quickly that a single manuscript would be woefully inadequate to describe our research in sufficient detail. The 4 manuscripts contained in this Special Section are the result of more than a year of research, writing, and revision. The manuscripts, comprising several hundred references and nearly 70 journal pages, required restraint on the part of the authors, hard as that may be to believe. More had been written, but tough editorial decisions were made to balance the breadth and depth of the topics presented. While these articles are written from the perspective of statisticians from industry and the US Food and Drug Administration (FDA), the authors have taken great care to make the manuscripts accessible to audience from diverse backgrounds. This was a deliberate choice, as patient safety requires the expertise of many disciplines. The first paper, “Clinical Trials,” provides a thorough overview of safety for randomized studies, with a primary focus on Phase II and III clinical trials. This article highlights many of the challenges inherent in the analysis of safety data, challenges that are revisited in the other papers. The second paper, “Postmarket Surveillance,” describes the passive postmarket systems of the FDA, the European Medicines Agency, and the World Health Organization. It emphasizes the importance of data visualization for the effective summary and communication of important safety signals. The third paper, “Real World Insights,” discusses electronic medical and health records, administrative claims data, patient registries, and pragmatic trials. These data are of greater regulatory interest with the passing of the 21st Century Cures Act. Several case studies are described, including the use of the Sentinel System to identify excess bleeding events, a pregnancy registry to assess potential birth defects, and a pragmatic trial to evaluate the safety profile of a new therapy in patients with chronic obstructive pulmonary disease. The final manuscript, “Transforming Data into Evidence,” examines several safety examples from the literature, including intussusception for rotavirus vaccines, bleeding events in anticoagulant drugs, and several safety outcomes for patients taking T2DM therapies from the thiazolidinedione class. The case studies presented here illustrate that different products may require distinctive approaches to effectively evaluate patient safety. Despite this body of work, numerous challenges in the analysis of safety endpoints remain. For example, there are open questions regarding how best to address the guidance for safety monitoring for Investigational New Drug safety reporting. Recommendations are also needed on how to most appropriately combine data across the various sources to obtain an overall assessment of patient safety for a particular medical product. And, as we allude to at the end of the first paper, the research presented here focuses on summarizing and responding to safety events as they occur. There are numerous initiatives under way to identify and validate potential biomarkers to predict which patients are likely to experience serious safety events. These and other research topics are being evaluated by the Safety Working Group. The authors would like to take this opportunity to thank the Biopharmaceutical Section of the ASA for their support of the Safety Working Group. We thank Dr Stephen Spielberg and Ms Judy Connors, formally at DIA, for their initial interest in this proposed Special Section and Dr Rick Turner and Dr Ranjini Prithviraj for their continued support of this work. Finally, the authors would like to thank the individual reviewers for their constructive comments and suggestions, which improved the content of these manuscripts.

Sampling Methodology: Implications for Drawing Conclusions from Clinical Research Findings

Generating a random sample from a population is important to minimize the bias of sample estimates in describing the population parameters. Despite the benefits of random sampling for generating appropriate inference, clinical research often relies instead on samples of convenience. Baseline characteristics and study inclusion and exclusion criteria can help identify the study population from which the sample was drawn. It is important to understand the factors that differ between the study sample and the larger population and the potential impact these differences may have on the conclusions of the study and how appropriate it is to apply study results to the larger population.

Review of Clinical Trial Design: Bayesian and Frequentist Adaptive Methods

The subtitle of this book is ‘‘An in-depth guide to writing, editing, tracking, and submitting the original IND and applicable IND amendments.’’ This book is a clear and thorough guide to the preparation, writing, publishing, submission, and monitoring of documents that FDA requires drug sponsors to submit. Although the title and subtitle focus on IND submissions, in fact the book covers in depth a few other types of FDA submissions as well, such as orphan drug applications and FDA meeting requests. The book is set up like a training manual for those who are entering regulatory affairs for drug products, and it is a good one. It would be useful as a textbook for a regulatory affairs course or to study for certification. Although it is very basic, it might also be useful as a desk reference for a regulatory affairs department for a quick look at an applicable chapter when a quick answer is needed. Topics include all aspects of INDs, including administrative aspects of submissions (creating style guides and templates; tips on writing and coordinating writing by a group; FDA forms), paper and electronic submissions, tracking submissions, managing references, CTD format, electronic document management systems, FDA meetings, dispute resolution, expanded use INDs, exploratory INDs, IND amendments, protocols, transfers of obligations, investigator’s brochures, safety reports, fast-track designation, responding to clinical holds, special protocol assessments, statistical analysis plans, filing at clinicaltrials.gov, CMC issues, orphan drug issues, USAN and proprietary name development, inactivating and reactiving an IND, and drug master files. The book does not seem to miss any IND-related topics, and everything I read was very accurate and also easy to read. The organization of the book is roughly along the lines of how one would go about initiating and maintaining an IND. However, surprisingly many introductory concepts are not introduced until chapters 22, 26, and 27. This is a sizable book, with 530 pages of relatively large dimensions (approximately 11’’ 12’’). It has a plastic spiral binding, but that is mounted within an attractive solid hardcover backing. The spiral binding appears intended to allow readers to remove the spiral-bound pages from the hard cover so that pages can be turned back on the spiral. Each of the 62 chapters is marked with a thick-tabbed divider with each topic clearly marked on the tab, which is extremely convenient and makes its use as a reference tool easier, particularly since the book lacks an index. Its 33-page detailed table of contents partially compensates for this as well. One of the most impressive aspects of this book is the accompanying CD-ROM, containing template documents, arranged by chapter, for 54 of the chapters. With artificial ‘‘samples’’ of FDA meeting requests, completed FDA forms, etc, as well as a sample style guide and examples of the types of letters that FDA sends to drug sponsors, this CD-ROM collection could be worthwhile for any new regulatory affairs department to obtain. There is no information about author Meredith BrownTuttle in the book. A quick search on the Internet reveals that she has been active in regulatory affairs as a consultant and at a number of consulting firms over the years, was on the Board of Editors at the Regulatory Affairs Professionals Society (RAPS) for many years, and teaches regulatory affairs at the University of California, Santa Cruz.