Eric Hung

Assessing the extent and impact of the masking effect of disproportionality analyses on two spontaneous reporting systems databases

Masking is a statistical issue by which signals are hidden by the presence of other medicines in the database. In the absence algorithm, the impact of the masking effect has not been fully investigated.

A conceptual approach to the masking effect of measures of disproportionality

Masking is a statistical issue by which true signals of disproportionate reporting are hidden by the presence of other products in the database. Masking is currently not perfectly understood. There is no algorithm to identify the potential masking drugs to remove them for subsequent analyses of disproportionality.

Pneumothorax as an adverse drug event: an exploratory aggregate analysis of the US FDA AERS database including a confounding by indication analysis inspired by Cornfield's condition.

Introduction: Pneumothorax is either primary or secondary. Secondary pneumothorax is usually due to trauma, including various non-pharmacologic iatrogenic triggers. Although not normally thought of as an adverse drug event (ADE) secondary pneumothorax is associated with numerous drugs, though it is often difficult to determine whether this association is causal in nature, or reflects an epiphenomenon of efficacy or inefficacy, or confounding by indication (CBI). Herein we explore this association in a large health authority drug safety surveillance database. Methods: A quantitative pharmacovigilance (PhV) methodology known as disproportionality analysis was applied to the United States Food and Drug Administration (US FDA) Adverse Event Reporting System (AERS) database to explore the quantitative reporting dependencies between drugs and the adverse event pneumothorax as well the corresponding reporting dependencies between drugs and reported indications that may be risk factors for pneumothorax themselves in order to explore the potential contribution of CBI. Results: We found 1. Multiple drugs are associated with pneumothorax; 2. Surfactants and oncology drugs account for most statistically distinctive associations with pneumothorax; 3. Pulmonary surfactants, pentamidine and nitric oxide have the largest statistical reporting associations 4. CBI may play a prominent role in reports of drug-associated pneumothorax. Conclusions: Disproportionality analysis (DA) can provide insights into the spontaneous reporting dependencies between drugs and pneumothorax. CBI assessment based on DA and Cornfields inequality presents an additional novel option for the initial exploration of potential safety signals in PhV.

A Quantitative Analysis of the Spontaneous Reporting of Congestive Heart Failure‐Related Adverse Events With Systemic Anti‐Fungal Drugs

To investigate spontaneous reporting relationships between representative antifungal agents and congestive heart failure (CHF)‐related adverse events (AE) we performed multiple disproportionality analyses of the US FDA AERS database. Specifically we performed analysis of drug‐AE associations (2D) plus drug–drug–AE and drug–AE–AE‐associations (3D), the latter two to explore the potential contribution of reported pharmacodynamic interactions, overexposure from pharmacokinetic interactions, and drug overdose. Itraconazole displayed a pattern of statistical reporting dependencies across multiple analyses (2D and 3D). Amphotericin B was the only other antifungal that demonstrated a 2D SDR with CHF‐related events. Itraconazole demonstrated multiple SDRs with calcium channel blockers in suspect drug‐only 3D analysis. There was one other SDR with fluconazole and propanolol and three SDRs involving valproate and fluconazole that may have been do at least in part to duplicate reporting. Less specific 3D analysis including both suspect plus concomitant medications showed a greater number and variety of SDRs with multiple antifungals. Statistical reporting dependencies with CHF‐related events did not appear to be a consistent pharmacological (e.g., azole/triazole)/therapeutic (i.e., antifungal) class effect. Itraconzole was unique in the pattern of statistical reporting dependencies with CHF‐related events which is consistent with findings from independent data sets.

Seasonal and Geographic Variation in Adverse Event Reporting

BackgroundMany illnesses demonstrate seasonal and geographic variations. Pharmacovigilance is unique among public health surveillance systems in terms of the clinical diversity of the events under surveillance. Since many pharmacovigilance signal detection methodologies are geared towards looking for increased frequency of spontaneous adverse drug event (ADE) reporting over variable time frames, seasonality of ADEs may have implications for signal detection.ObjectiveThe aim of this study was to investigate whether a set of illnesses that might be expected to display seasonality in general, did so when spontaneously reported as ADEs.MethodsWe performed our analysis with the publically available US FDA Adverse Event Reporting System (FAERS) data. We selected a convenience sample of events possibly triggered by seasonal factors (hypothermia, Raynaud’s phenomenon, photosensitivity reaction, heat exhaustion, heat stroke, and sunburn) and events for which previous literature experience suggests seasonality (anencephaly and interstitial lung disease). Our statistical procedures can be explained in terms of a simple physicogeometric setting: the unit circle divided into 6 (semiannual sinusoidal) or 12 (annual sinusoidal) arcs. When reporting frequencies (weights) are more or less evenly distributed across months, the center of mass of the circle would not be significantly displaced from the origin (0, 0). Distinct seasonal patterns will significantly displace the center of mass of the circle.ResultsVarious patterns of seasonality were identified for some, but not all, events and region–event pairs. USA displayed the most instances of seasonality. Scandinavia did not display seasonality for any events. Seasonality was usually annual sinusoidal. Possible explanations for failure to observe seasonality are briefly considered.ConclusionsUnderstanding seasonality of spontaneous ADE reporting may have public health policy and research implications and may mitigate false-positive and missed true-positive pharmacovigilance signals. More systematic study of seasonality of spontaneous ADE reporting, including additional events with more or less biological rationale for seasonality, is a logical extension of this analysis.

Revisiting the reported signal of acute pancreatitis with rasburicase: an object lesson in pharmacovigilance

Introduction: There is an interest in methodologies to expeditiously detect credible signals of drug-induced pancreatitis. An example is the reported signal of pancreatitis with rasburicase emerging from a study [the ‘index publication’ (IP)] combining quantitative signal detection findings from a spontaneous reporting system (SRS) and electronic health records (EHRs). The signal was reportedly supported by a clinical review with a case series manuscript in progress. The reported signal is noteworthy, being initially classified as a false-positive finding for the chosen reference standard, but reclassified as a ‘clinically supported’ signal. Objective: This paper has dual objectives: to revisit the signal of rasburicase and acute pancreatitis and extend the original analysis via reexamination of its findings, in light of more contemporary data; and to motivate discussions on key issues in signal detection and evaluation, including recent findings from a major international pharmacovigilance research initiative. Methodology: We used the same methodology as the IP, including the same disproportionality analysis software/dataset for calculating observed to expected reporting frequencies (O/Es), Medical Dictionary for Regulatory Activities Preferred Term, and O/E metric/threshold combination defining a signal of disproportionate reporting. Baseline analysis results prompted supplementary analyses using alternative analytical choices. We performed a comprehensive literature search to identify additional published case reports of rasburicase and pancreatitis. Results: We could not replicate positive findings (e.g. a signal or statistic of disproportionate reporting) from the SRS data using the same algorithm, software, dataset and vendor specified in the IP. The reporting association was statistically highlighted in default and supplemental analysis when more sensitive forms of disproportionality analysis were used. Two of three reports in the FAERS database were assessed as likely duplicate reports. We did not identify any additional reports in the FAERS corresponding to the three cases identified in the IP using EHRs. We did not identify additional published reports of pancreatitis associated with rasburicase. Discussion: Our exercise stimulated interesting discussions of key points in signal detection and evaluation, including causality assessment, signal detection algorithm performance, pharmacovigilance terminology, duplicate reporting, mechanisms for communicating signals, the structure of the FAERs database, and recent results from a major international pharmacovigilance research initiative.

Bevacizumab-associated diverticulitis: results of disproportionality analysis

We read the excellent article on the state-of-the-art in signal detection for oncology drugs [1]. It provides a wonderful information source for those looking to make performance assessments of signal detection methodologies in the oncology drug safety domain. Tuccori et al. provide a simple classification of signal detection methods as qualitative versus quantitative, the later referring to data mining methodologies in use for signal surveillance, which at this point, is primarily disproportionality analysis (DA). A clear understanding of when qualitative and quantitative approaches to signal detection yield convergent versus discrepant results will facilitate their joint implementations in a holistic, complementary manner. It therefore is an interesting and important question as to which specific signals are detected versus missed with signal detection approaches based on intense clinical surveillance versus statistical screening of databases. In light of the above, one item in the article which caught our attention was the summary of the study by McKoy et al. (2008) [2] in table 1 [1], because it is the one study reporting purely negative data mining findings. The main results in table 1 include proportional reporting ratio “PRR: not significant” for bevacizumab–diverticulitis. Our initial educated impression was that diverticulitis might be a rare event overall in US FDA Adverse Events Reporting System (FAERS), and thus a relatively small number of observed reports might generate a signal of disproportionate reporting (SDR) [3] using PRRs due to a low ‘expected count’ [4] for this drug–event combination. We therefore decided to see if we could reproduce the reported negative finding after first obtaining the cited reference which confirmed the accuracy of this table entry, stating: ‘Our analysis sought to determine whether these cases had proportional reporting ratios (PRRs) that would suggest a relationship with bevacizumab or carboplatin, finding no evidence of signals for these drugs’ [2]. Using an freedom of information extract of FAERS through the first quarter of 2014, we performed DA on cumulative time slices of the data representing spontaneous reports (i.e., with reports marked as clinical trial cases removed). We found that there was an SDR using a commonly accepted PRR protocol (PRR > 2, c > 3.84 and n > 2) [5] as early as 2006 based on 11 cumulative reports (PRR = 3.047, c = 14.899) in an analysis of suspect drugs only (S). It is unclear why McKoy et al. did not identify a PRR signal as we did. Despite early inflated claims by some of the objectivity and accuracy of these methods when applied to spontaneous report databases, subsequent investigations uncovered important limitation and how variations in analytical configuration and/or even software vendor selection can result in different findings [6]. Thus, DA entails series of analytical choices, which can impact results [6], and this provides a basis for exploring possible explanations for their negative PRR finding given the positive finding we obtained in our initial

The impact of database restriction on pharmacovigilance signal detection of selected cancer therapies

Background: The aim of this study was to investigate whether database restriction can improve oncology drug pharmacovigilance signal detection performance. Methods: We used spontaneous adverse event (AE) reports in the United States (US) Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS) database. Positive control (PC) drug medical concept (DMC) pairs were selected from safety information not included in the product’s first label but subsequently added as label changes. These medical concepts (MCs) were mapped to the Medical Dictionary for Regulatory Activities (MedDRA) preferred terms (PTs) used in FAERS to code AEs. Negative controls (NC) were MCs with circumscribed PTs not included in the corresponding US package insert (USPI). We calculated shrinkage-adjusted observed-to-expected (O/E) reporting frequencies for the aforementioned drug–PT pairs. We also formulated an adjudication framework to calculate performance at the MC level. Performance metrics [sensitivity, specificity, positive and negative predictive value (PPV, NPV), signal/noise (S/N), F and Matthews correlation coefficient (MCC)] were calculated for each analysis and compared. Results: The PC reference set consisted of 11 drugs, 487 PTs, 27 MCs, 37 drug–MC combinations and 638 drug–event combinations (DECs). The NC reference set consisted of 11 drugs, 9 PTs, 5 MCs, 40 drug–MC combinations and 67 DECs. Most drug–event pairs were not highlighted by either analysis. A small percentage of signals of disproportionate reporting were lost, more noise than signal, with no gains. Specificity and PPV improved whereas sensitivity, NPV, F and MCC decreased, but all changes were small relative to the decrease in sensitivity. The overall S/N improved. Conclusion: This oncology drug restricted analysis improved the S/N ratio, removing proportionately more noise than signal, but with significant credible signal loss. Without broader experience and a calculus of costs and utilities of correct versus incorrect classifications in oncology pharmacovigilance such restricted analyses should be optional rather than a default analysis.

An exploratory factor analysis of the spontaneous reporting of severe cutaneous adverse reactions

Background: Severe cutaneous adverse reactions (SCARs) are prominent in pharmacovigilance (PhV). They have some commonalities such as nonimmediate nature and T-cell mediation and rare overlap syndromes have been documented, most commonly involving acute generalized exanthematous pustulosis (AGEP) and drug rash with eosinophilia and systemic symptoms (DRESS), and DRESS and toxic epidermal necrolysis (TEN). However, they display diverse clinical phenotypes and variations in specific T-cell immune response profiles, plus some specific genotype–phenotype associations. A question is whether causation of a given SCAR by a given drug supports causality of the same drug for other SCARs. If so, we might expect significant intercorrelations between SCARs with respect to overall drug-reporting patterns. SCARs with significant intercorrelations may reflect a unified underlying concept. Methods: We used exploratory factor analysis (EFA) on data from the United States Food and Drug Administration Adverse Event Reporting System (FAERS) to assess reporting intercorrelations between six SCARs [AGEP, DRESS, erythema multiforme (EM), Stevens–Johnson syndrome (SJS), TEN, exfoliative dermatitis (ExfolDerm)]. We screened the data using visual inspection of scatterplot matrices for problematic data patterns. We assessed factorability via Bartlett’s test of sphericity, Kaiser-Myer-Olkin (KMO) statistic, initial estimates of communality and the anti-image correlation matrix. We extracted factors via principle axis factoring (PAF). The number of factors was determined by scree plot/Kaiser’s rule. We also examined solutions with an additional factor. We applied various oblique rotations. We assessed the strength of the solution by percentage of variance explained, minimum number of factors loading per major factor, the magnitude of the communalities, loadings and crossloadings, and reproduced- and residual correlations. Results: The data were generally adequate for factor analysis but the amount of variance explained, shared variance, and communalities were low, suggesting caution in general against extrapolating causality between SCARs. SJS and TEN displayed most shared variance. AGEP and DRESS, the other SCAR pair most often observed in overlap syndromes, demonstrated modest shared variance, along with maculopapular rash (MPR). DRESS and TEN, another of the more commonly diagnosed pairs in overlap syndromes, did not. EM was uncorrelated with SJS and TEN. Conclusions: The notion that causality of a drug for one SCAR bolsters support for causality of the same drug with other SCARs was generally not supported.