Failure of a Meta-analysis: A Commentary on Glen Spielmans’s “Re-Analyzing Phase III Bremelanotide Trials for ‘Hypoactive Sexual Desire Disorder in Women’”

Abstract

In a “re-analysis” of Kingsberg et al.’s (2019) article, “Bremelanotide for the Treatment of Hypoactive Sexual Desire Disorder: Two Randomized Phase III Trials,” Spielmans (2021) offers inaccurate assertions. What this journal has elected to publish is a biased, retrospective analysis of the published results from two successful, well-controlled clinical trials. In the author’s own words: “While metaanalysis offers a standardized method of data analysis, results may be interpreted in various ways.” Following this declaration, Spielmans provides his limited personal perspective on the retrospective analysis, strongly suggesting that his uninformed personal interpretation of the meta-analyses supersedes the individual trial analyses conducted using patientlevel data by the trial sponsor and corroborated by the Food and Drug Administration (FDA) as part of the drug approval process. In the limitations section of Spielmans’s article, there is an acknowledgment that “Bremelanotide (BMT) appears to offer modest benefits on the Female Sexual Function Index-Desire Domain (FSFI-D) (Rosen et al., 2000) and the Female Sexual Distress Scale-Desire, Arousal, Orgasm, Item 13 (FSDS-DAO #13)” (Derogatis et al., 2008). Spielmans is not a treating clinician and is unaware of the validation that was conducted, at the direction of the U.S. FDA, to establish clinically meaningful cutoffs of the various patient-reported outcomes to define clinical benefit (Revicki et al., 2018). It is questionable how Spielmans can both interpret and denigrate the extent of the comprehensive benefit bremelanotide (BMT) provides for a patient. In numerous cases, Spielmans states that placebo outperformed BMT; the authors of this commentary encourage all readers to avail themselves of the published results from Kingsberg et al. (2019), where Figure 3 presents the results of BMT over placebo for eight individual patient-reported outcomes within each of the two Phase III studies. Spielmans’s statement is also factually incorrect in stating, “most reported efficacy outcomes were apparently derived post-hoc”; the extensive statistical analysis plan that was prepared and reviewed by the FDA defined each analysis in detail prior to database lock. The phase III BMT program involving over 1200 women was the culmination of over a decade of research, supported by over a dozen individual studies involving the compound for treating this disorder (e.g., Clayton et al., 2016). The alternative interpretation of the retrospective analyses conducted by Spielmans goes far beyond claiming that the FDA was wrong for approving this drug. For example, he lists as a “problem” that not everything in the study protocol was included in our Kingsberg et al. (2019) paper. In contrast to the Journal of Sex Research that provided 20 pages for Spielmans’s article, many, high-impact, scientifically rigorous journals have a page limit. He also alludes to “data peeking” in his introduction and that the FDA allowed the “sponsor’s request for satisfying sexual events (SSEs) to move from the co-primary to the key secondary outcome . . .. a year after the trials had begun.” What Spielmans omitted is that the FDA published a guidance document (2016) for designing clinical trials in which SSEs were no longer required to be a primary endpoint for HSDD treatment trials. Instead, trials could now include measures reflecting the hallmark criteria of the condition: loss of (i.e., deficiency or absence of) sexual desire (i.e., FSFI-D) and distress about lack of desire (i.e., FSDS-DAO #13). The approval from the FDA to change the primary endpoint, after discussion with the FDA review division, came prior to the data lock in these well-conducted, randomized, double-blind, multicenter placebo-controlled trials, with pre-established statistical analysis plans. All sites received investigational review board (IRB) approval and were conducted according to good clinical practice, and the suggestion of any research malfeasance by Spielmans is baseless and false. A researcher with expertise in sexual medicine would likely know the rigor and oversight of these trials along with the historical context in the change of endpoints that they reflect, the evolution of this field with an improved understanding of how to evaluate this condition (HSDD), and that SSEs had long been deemed by experts as a poor surrogate of desire and instead, a downstream effect of desire (Kingsberg & Althof, 2011). The medical community welcomes the constructive review by subject experts to help improve the process leading to the development of safe and effective medications for patients with few treatment options. However, the retrospective analysis performed by Spielmans fails to acknowledge the decade of diligent research that culminated in a drug that the FDA scrutinized through the development process to a point that