David Fram

Comparative performance of two quantitative safety signalling methods: implications for use in a pharmacovigilance department.

AbstractBackground and objectives: There is increasing interest in using disproportionality-based signal detection methods to support postmarketing safety surveillance activities. Two commonly used methods, empirical Bayes multi-item gamma Poisson shrinker (MGPS) and proportional reporting ratio (PRR), perform differently with respect to the number and types of signals detected. The goal of this study was to compare and analyse the performance characteristics of these two methods, to understand why they differ and to consider the practical implications of these differences for a large, industry-based pharmacovigilance department. Methods: We compared the numbers and types of signals of disproportionate reporting (SDRs) obtained with MGPS and PRR using two postmarketing safety databases and a simulated database. We recorded signal counts and performed a qualitative comparison of the drug-event combinations signalled by the two methods as well as a sensitivity analysis to better understand how the thresholds commonly used for these methods impact their performance. Results: PRR detected more SDRs than MGPS. We observed that MGPS is less subject to confounding by demographic factors because it employs stratification and is more stable than PRR when report counts are low. Simulation experiments performed using published empirical thresholds demonstrated that PRR detected false-positive signals at a rate of 1.1%, while MGPS did not detect any statistical false positives. In an attempt to separate the effect of choice of signal threshold from more fundamental methodological differences, we performed a series of experiments in which we modified the conventional threshold values for each method so that each method detected the same number of SDRs for the example drugs studied. This analysis, which provided quantitative examples of the relationship between the published thresholds for the two methods, demonstrates that the signalling criterion published for PRR has a higher signalling frequency than that published for MGPS. Discussion and conclusion: The performance differences between the PRR and MGPS methods are related to (i) greater confounding by demographic factors with PRR; (ii) a higher tendency of PRR to detect false-positive signals when the number of reports is small; and (iii) the conventional thresholds that have been adapted for each method. PRR tends to be more ‘sensitive’ and less ‘specific’ than MGPS. A high-specificity disproportionality method, when used in conjunction with medical triage and investigation of critical medical events, may provide an efficient and robust approach to applying quantitative methods in routine postmarketing pharmacovigilance.

Association of asthma therapy and Churg-Strauss syndrome: an analysis of postmarketing surveillance data.

BACKGROUND Churg-Strauss syndrome (CSS), also known as allergic granulomatous angiitis (AGA), is a rare vasculitis that occurs in patients with bronchial asthma. The nature of the association of CSS with various asthma therapies is unclear. OBJECTIVE This study investigated the associations of different multidrug asthma therapy regimens and the reporting of AGA (the preferred code for CSS in the coding dictionary for the Adverse Event Reporting System [AERS]) by applying an iterative method of disproportionally analysis to th AERS database maintained by the US Food and Drug Administration. METHODS The public-release version of the AERS database was used to identify reports of AGA in patients receiving asthma therapy. Reporting of AGA was examined using iterative disproportionality methods in patients receiving > or =1 of the following drug classes: inhaled corticosteroid (ICS), leukotriene receptor antagonist (LTRA), short-acting beta(2)-agonist (SABA), or long-acting beta(2)-agonist (LABA). The Bayesian data-mining algorithm known as the multi-item gamma poisson shrinker was used to determine the relative reporting rates by calculation of the empirical Bayes geometric mean (EBGM) and its 90% CI (EB05 = lower limit and EB95 = upper limit) for each drug. Subset analyses were performed for each drug with different medication combinations to differentiate the relative reporting of AGA for each. RESULTS A strong association was found between LTRA use and AGA (EBGM = 104.0, EB05 = 95.0, EB95 = 113.8) that persisted with all combinations of therapy studied. AGA was also associated with the ICS, SABA and LABA classes (EBGM values of 27.8, 14.6 and 40.4, respectively). However, the latter associations were mostly dependent on the presence of concurrent LTRA and, to a lesser extemt, oral corticosteroid therapy and became negligible (ie, EB05 < 2) for patients who were not receiving these concurrent treatments. CONCLUSIONS Differences based on relative reporting were observed in the patterns of association of AGA with LTRA, ICS, and beta(2)-agonist therapies. A strong association between LTRA use and AGA was present regardless of the use of other asthma drugs.

Adaptation of Bayesian data mining algorithms to longitudinal claims data: coxib safety as an example.

Introduction:Bayesian data mining methods have been used to evaluate drug safety signals from adverse event reporting systems and allow for evaluation of multiple endpoints that are not prespecified. Their adaptation for use with longitudinal data such as administrative claims has not been previously evaluated or validated. Methods:In this pilot study, we evaluated the feasibility of adapting data mining methods using the empirical Bayes Multi-item Gamma Poisson Shrinkage (MGPS) algorithm to longitudinal administrative claims data. The Medicare Current Beneficiary Survey was used to identify a cohort of Medicare enrollees who were exposed to cyclooxygenase selective (coxib) or nonselective nonsteroidal anti-inflammatory drugs (NS-NSAIDs) from 1999 to 2003. Empirical Bayes MGPS algorithm was used to simultaneously evaluate 259 outcomes associated with current use of coxibs versus NS-NSAIDs while adjusting for key covariates and multiple comparisons. For comparison, a parallel analysis used traditional epidemiologic methods to evaluate the relationship between coxib versus NS-NSAID use and acute myocardial infarction, with the goal of establishing the concurrent validity of the data mining approach. Results:Among 9431 Medicare beneficiaries using NSAIDs and considering all 259 possible outcomes, empirical Bayes MGPS identified an association between current celecoxib use and acute myocardial infarction (Empirical Bayes Geometric Mean ratio 1.91) but not other outcomes. Rofecoxib use was associated with acute cerebrovascular events (Empirical Bayes Geometric Mean ratio 1.85) and several other diagnoses that likely represented indications for the drug. Results from the analyses using traditional epidemiologic methods were similar and indicated that the data mining results were valid. Discussion:Bayesian data mining methods seem useful to evaluate drug safety using administrative data. Further work will be needed to extend these findings to different types of drug exposures and to other claims databases.

Signal detection methodologies to support effective safety management

The increased focus on the safety of medical products, as well as the growing volume of available safety information, has created a need for objective quantitative approaches to supplement the medical review of individual case safety reports. Statistical algorithms can be used to identify trends and relationships in both clinical and postmarketing safety databases in support of safety signal detection. Powerful data visualization tools facilitate the medical review of the complex information generated by these methods. In addition, all these approaches need to be integrated into the daily practice of clinical safety and postmarketing pharmacovigilance.

Development of a mother–child database for drug exposure and adverse event detection in the Military Health System

The aim of this study was to develop a mother‐child linked database consisting of all eligible active duty military personnel, retirees, and their dependents in order to conduct medication‐related analyses to improve the safety and quality of care in the Military Health System (MHS).

Automated Method for Detecting Increases in Frequency of Spontaneous Adverse Event Reports over Time

A statistical methodology—focused on temporal change detection—was developed to highlight excursions from baseline spontaneous adverse event (AE) reporting. We used regression (both smooth trend and seasonal components) to model the time course of a drugs reports containing an AE, and then compared the sum of counts in the past 2 months with the fitted trend. The signaling threshold was tuned, using retrospective analysis, to yield acceptable sensitivity and specificity. The method may enhance pharmacovigilance by providing effective automated alerting of reporting aberrations when databases are small, when drugs have established safety profiles, and/or when product quality issues are of concern.

Molecular clinical safety intelligence: a system for bridging clinically focused safety knowledge to early-stage drug discovery - the GSK experience.

Drug toxicity is a major cause of late-stage product attrition. During lead identification and optimization phases little information is typically available about which molecules might have safety concerns. A system was built linking chemistry, preclinical and human safety information, enabling scientists to lever safety knowledge across multiple disciplines. The system consists of a data warehouse with chemical structures and chemical and biological properties for ∼80000 compounds and tools to access and analyze clinical data, toxicology, in vitro pharmacology and drug metabolism data. Tapping into this safety knowledge enables rapid clinically focused risk assessments of drug candidates. Use of this strategy adds value to the drug discovery process at GSK via efficient triage of compounds based on their potential for toxicity.

Online signal management : A systems-based approach that delivers new analytical capabilities and operational efficiency to the practice of pharmacovigilance

Traditional approaches to postmarketing pharmacovigilance have been primarily qualitative and paper based. We describe the development and implementation of online signal management (OSM), a new systems-based platform for pharmacovigilance that supports prioritization of safety issues, in-stream review and data retrieval, aggregate-level analysis of data patterns, and knowledge management. By providing an integrated view of the information needed to detect and evaluate safety signals, OSM enhances the efficiency of the pharmacovigilance process. The system enables pharmacovigilance professionals to triage incoming data using statistical, visualization, and alerting tools. Safety issues can be flagged and tracked, providing a streamlined approach to the retention and sharing of knowledge. We have deployed OSM to more than 100 users. User surveys and operational metrics demonstrate that OSM increases the efficiency of pharmacovigilance and that users are highly satisfied with OSM and the processes implemented for its use. In summary, this integration of state-of-the-art statistical and visualization tools with traditional, case-evaluation tools and a novel knowledge management system represents a significant advance in pharmacovigilance.