Efficacy of Hydroxychloroquine as Prophylaxis for Covid-19
EEfficacy of Hydroxychloroquine as Prophylaxis for Covid-19
Márcio WatanabeUniversidade Federal Fluminense
Abstract
Limitations in the design of the experiment of Boulware et al are considered in Cohen . Theyare not subject to correction but they are reported for readers' consideration. However, they made ananalysis for the incidence based on Fisher’s hypothesis test for means while they published detailedtime dependent data which were not analyzed, disregarding an important information. Here we makethe analyses with this time dependent data adopting a simple regression analysis. We conclude their randomized, double-blind, placebo-controlled trial presents statisticalevidence, at 99% confidence level, that the treatment of Covid-19 patients with hydroxychloroquine iseffective in reducing the appearance of symptoms if used before or right after exposure to the virus. For0 to 2 days after exposure to virus, the estimated relative reduction in symptomatic outcomes is 72%after 0 days, 48.9% after 1 day and 29.3% after 2 days. For 3 days after exposure, the estimatedrelative reduction is 15.7% but results are not statistically conclusive and for 4 or more days afterexposure there is no statistical evidence that hydroxychloroquine is effective in reducing theappearance of symptoms.Our results show that the time elapsed between infection and the beginning of treatment iscrucial for the efficacy of hydroxychloroquine as a treatment to Covid-19.
1. Introduction
The novel coronavirus disease, Covid-19, caused by the virus SARS-coV-2 has caused a majorpandemic worldwide. No vaccines or specific treatment were available until June 2020. Many existingmedicines have been tested to treat patients, mostly in hospitalized patients under more severe clinicalconditions. Hydroxychloroquine is the most discussed of these drugs and several studies have pointedto different outcomes. Although some randomized clinical trials have shown the inefficacy ofhydroxychloroquine treatment to prevent death in hospitalized patients, the benefits to less severepatients in the beginning of the disease and as a pre-exposure or post-exposure prophylaxis is stillunder discussion. Boulware et al. was the first randomized clinical trial which has testedhydroxychloroquine as a prophylaxis treatment to Covid-19. The authors of the study concluded thereis no statistical evidence of hydroxychloroquine’s efficacy when compared to placebo results. Besidessome limitations in their experimental design, as pointed by Cohen, up to 23th of June 2020, when ourstudy was finished, their results were the most reliable information available about hydroxychloroquinetreatment for Covid-19 as a prophylaxis. However, correctable inaccuracies in their estimates remainedwithout revision. In the section Methods, we discuss some of these inaccuracies and present the corrections.Furthermore, in the section Results, we provide an original analysis of a more detailed time series data,available at their supplementary appendix. – Methods The randomized trial of Boulware et al. aims to test the treatment of Covid-19 infected patientsfor 5 days with hydroxychloroquine as a prophylaxis, measuring its effect by a possible reduction onthe incidence of symptomatic outcomes when compared to results from a placebo group. Adult patientswho had epidemiological linkage with Covid-19 confirmed patients were included if exposure waswithin 4 days at the beginning of the experiment. Initially 921 asymptomatic participants wererandomly assigned to treatment or placebo groups but 100 presented symptoms at the day of beginningof experiment (day 1) and were excluded. The primary measure of effect was incidence of Covid-19compatible symptoms with laboratory or clinical confirmation within 14 days. In the placebo group, 58of 407 participants (14.3%) presented symptoms from day 2 to day 14, while 49 of 414 participants(11.8%) of the treatment group presented symptoms in the same period. No serious adverse reactionswere reported. A two-tailed Fisher exact test was used to obtain a p-value of 0.351 and at 95%confidence level, they have concluded hydroxychloroquine did not prevent illness compatible withCovid-19 when used as a postexposure prophylaxis. Next, we make some considerations on theirstatistical analysis to corroborate with the different choices we make at section Results. Limitations onthe design of their experiment are discussed by Cohen. The drug in test is known to have antiviral and anti-inflammatory activities and to effectivelyinhibit SARS-coV-2 in vitro. When applying a hypothesis test, the statistician responsible for the data analysis must definewhether to use a two-tailed test or a one-tailed test. This definition typically depends on the alternativehypothesis to be tested. In the present problem, the question to be answered is whether the treatment isable to reduce the incidence of symptomatic patients in contrast to the null hypothesis H0 that it will beas good as placebo. Thus, a one-tailed test is the natural choice to the alternative hypothesis H1 in thisproblem, in contrast to the two-tailed test adopted in their study. We emphasize this choice has serious implications in both interpretation and quantitativeresults. In the adopted two-tailed test, if the test result leads to H0 rejection, the conclusion would bethat the treatment with hydroxychloroquine presents a different result than the placebo, where the trueincidence could be either higher or lower. In this two-tailed case, it would not be possible to concludeirectly that the rejection of H0 implies the efficacy of the treatment, despite the observed incidencesfavoring the hydroxychloroquine group. Another important aspect to consider is that the use of the two-tailed test favors null hypothesis to be not rejected in comparison with the one-tailed test. As anexample, the p-value obtained in the article by a two-tailed test is 35.1% while the p-value obtained bythe same method but with a one-tailed test is 17.8% (for complete data see Boulware et al ). Precise treatment group definition is necessary to a proper conduction of the statistical analysisin a clinical trial. In Boulware et al., the treatment group has been apparently defined as the group ofrandomly selected patients which received hydroxychloroquine for five consecutive days in thefollowing scheme: 800mg once, 600mg 6 to 8 hours later, then 600mg daily for 4 days. Treatment effect has been calculated from the incidence differences of symptomatic patientsbetween day 14 and day 1. Note that of 921 eligible patients on day 0, 100 became symptomatic on day1 and were excluded. However, patients who became symptomatic on days 2, 3, 4 and 5 were includedin the calculation of the treatment effect. Note this subgroup of patients, although included in theanalysis by the authors, had not completed the entire treatment when the response variable(symptomatic or asymptomatic) had been defined and once a patient had symptoms it is no longerpossible to change his status. Note that a large proportion of the symptomatic patients in the study have presented firstsymptoms between day 2 and day 5, the period of treatment application (see Table 1 below). Thesepatients did not have the full effect of the treatment. Using a similar logic that had made the authorsexclude symptomatic patients in day 1, we should exclude patients with symptoms in days 2, 3, 4 and5, otherwise treatment definition and statistical method should be changed. Therefore, in order to have a homogeneous sample of patients taking the same dose, anadequate analysis should measure the differences in incidences from days 14 to 5, and not from 14 today 1. The high sensitivity of the response variable chosen by the researchers (symptomatic orasymptomatic) is another reason why people who had symptoms before finishing treatment should beexcluded from measuring the effects on the test.The sample obtained considering only patients who presented first symptoms after the fifth dayis far from ideal because there can be many patients for whom a long time has passed from the day ofinfection to the day of the end of treatment. Unfortunately, the best way to avoid this problem shouldhave been adopted in the design of the experiment. An alternative solution to include patients who haveot used full treatment before the onset of symptoms is to use a statistical method more appropriate to aheterogeneous sample such as a regression method (see section 3).Hence, to test the effect of a complete 5 days treatment we should compare only the incidencedifferences between the groups from day 14 to day 5 (patients who have presented symptoms only afterthe end of complete prophylaxis). Raw data are not available, but with figure 2 of their article we canobtain the following approximation :Table 1: Percentage of symptomatic patients Day 5 Day 14 Difference Symptomatic AsymptomaticTreatment ≈ 7.7% 11.8% 4.1% a=17=49-32 b=365Placebo ≈ 7.4% 14.3% 6.9% c=28=58-30 d=349Applying Fisher’s exact test to this approximate data, for a one-tailed test, we obtain a p-valueof 5.6%, which is considerably different from the 35.1% of their article.
Their study uses absolute difference of the proportions of symptomatic patients betweentreatment and placebo groups as its response variable. However, this variable is not explanatorybecause it is not robust to inclusion of asymptomatic uninfected patients. First, observe that there aremany more asymptomatic than symptomatic patients in the sample (714 and 107). Second, mostpatients in the study were not tested, so a significant proportion of this sample should not be infected,decreasing a possible reduction in the absolute incidence of treatment effect and artificially increasingthe sample size and, as a consequence, the reliability of the statistics in the study. The real number of asymptomatic infected patients is unknown in each group, and Fisher’sexact test cannot be applied to this data, because the test must be applied to symptomatic infectedversus asymptomatic infected in each group, since asymptomatic uninfected are not sensitive to anytreatment, no matter the treatment is effective or not. Hence, in order to verify whether the difference between rates of symptomatic patients in thetwo groups is significant, the ideal sample should have included only confirmed infected patients. Thismajor problem in the design of the experiment can not be changed, but it can be significantlyminimized by selecting a statistic to measure treatment effect which is less affected by the inclusion ofuninfected asymptomatic patients in the sample. The relative difference (Rd) is a more appropriatemeasure in this case: d = rate of symptomatic at hydroxychloroquine group − rate of symptomatic at placebo grouprate of symptomatic at placebo group The relative difference can be interpreted as the negative of a measure of the percentage oftreatment effectiveness. That is, define Treatment efficacy = -Rd. Suppose for example the relativedifference Rd = -0.30. Then, hydroxychloroquine treatment efficacy is 30% in this case, which implies,on average, 30 out of 100 patients who would have presented symptoms if they have taken placebo willno longer develop symptoms if they take hydroxychloroquine. This variable is more informative and itis far less affected by the unknown proportion of uninfected patients in the sample. To illustrate this, consider the following example: Let Sh and Sp be the number of symptomaticpatients in treatment and placebo groups respectively. Suppose N is the total number of patients in eachgroup. Then in this case, absolute difference of incidences is given by |Sh-Sp|/N and relative differenceRd={(Sh/N)-(Sp/N)}/{Sp/N}= (Sh-Sp)/Sp. If we add M=N uninfected patients to each group, then theabsolute difference will be half of the original value interfering in any statistical test used to measuretreatment effect. Nevertheless, relative difference remains invariant regardless of the number M ofuninfected patients included.The treatment efficacy for the entire sample of Boulware’s study is -Rd = -(11.83-14.25) / 14.25= 16.9%, which means that if we consider patients who presented first symptoms from days 2 to 14,then the hydroxychloroquine group decrease the average number of symptomatic patients in 16.9%when compared to placebo group in these conditions. The treatment efficacy including only patients who had symptoms after day 5, which is theappropriate group to measure the effect of complete treatment, is -Rd = -(4.1-6.9) / 6.9 = 40.6% (seesubsection 2.2, in Methods) (for complete data see Boulware et al. ). In this section, we consider the same treatment of Boulware et al, which included in the samplepatients with a post-exposure period from 1 to 4 days before the beginning of the treatment sample,however we discriminate the sample into four sub-samples according to the number of days afterexposure. The data, which have been taken from table S6 in the supplementary appendix of Boulwareet al. , is shown in table 2 below. able 2: Number of patients according to time from exposure to SARS-coV-2 Hydroxychloroquine group Placebo groupDays from exposure Sample size Symptomatic Sample size Symptomatic1 77 5 63 82 100 12 106 183 98 12 117 174 138 20 121 15
The main reason to conduct a regression study is that hypothesis tests for means, such as theFisher’s exact test adopted in Boulware et al. , lose vital information contained in a time series datawhere the effect can be measured with a greater statistical significance for the same sample size.Another important advantage, as indicated in subsection 2.2, is that regression methods are adequate tomeasure the effect in time heterogeneous data such as that described in table 2.Note: The problem discussed in subsection 2.2 is not considered here, as we do not have theinformation necessary to proceed this specific correction. However, we do the analysis with themodifications described in subsections 2.1 and 2.3. Let y= f(x) be the treatment effect, where the explanatory variable x=number of days fromexposure to the beginning of treatment. With data from table S6, we obtain four different treatmenteffects, one for each x=1,2,3,4. That is, we calculate the negative relative differences of incidence ofsymptomatic patients of hydroxychloroquine group to placebo group as a response variable of the daysfrom exposure to the beginning of treatment.Hydroxychloroquine treatment efficacy and 95% confidence intervals for x= 1, 2, 3, 4 are givenby 48.86% [28.97, 72.81], 29.33% [15.45, 44.15], 15.73% [-5.64, 23.06], -16.91% [-34.30, 9.53]respectively. Figure 1 shows the graph of y=f(x), for x=1,2,3,4. It also displays a simple linearregression line for this data with 95% confidence bands. Estimated slope is -21.09 with 95%confidence interval of [-32.81, -9.38]. Predicted efficacy of the treatment if applied at the same day ofexposure (day 0) is 71.98% with 95% confidence interval [39.90, 100]. One-tailed p-value = 0.0081(0.81%). Thus with 99% of confidence, we reject the hypothesis that the slope is greater than or equalto 0. This is a strong statistical evidence that hydroxychloroquine treatment reduces the proportion ofsymptomatic patients when used as a prophylaxis right after exposure, especially if treatment startswithin 2 days. igure 1- Red dots: y = hydroxychloroquine treatment efficacy and x= number of days from exposure to thebeginning of treatment, for x =1,2,3,4. Blue line: Simple linear regression line for the four red dots. Dark grayshadow: 95% confidence bands for the simple linear regression line.
Next, we verify linear model suppositions. First, we assess the effect of heteroscedasticity byusing weights in linear regression fit proportional to the sample sizes for x=1,2,3,4. The estimated slopeis -21.63 and predicted relative difference at day 0 is 73.40%. One-tailed p-value is 0.88% in this case,which maintains the conclusions from homogeneous simple linear regression.To assess for autocorrelation in the residuals we perform a two-tailed Durbin-Watson test usingboth Pan algorithm with 1000 iterations and bootstrap resampling algorithm with 10.000 replicates.Autocorrelation = -0.46, DW statistic = 2.59 and p-value = 89.8%. Thus, at 95% confidence level, thehypothesis H0 (autocorrelation = 0) is not rejected.To assess the normality of the residuals, we obtain kurtosis = 2.09, skewness = 0.77 andperform the Shapiro-Wilkson test. The W statistic = 0.91, p-value = 46.3% and at 95% confidence levelthe normality hypothesis is not rejected. - Discussion
In this study, we discussed some inaccuracies in the statistical analysis of Boulware et al. Wealso add an original statistical analysis by adopting a different method, replacing Fisher’s exact testwith a simple regression analysis. There are two main reasons for this choice: first, the data of the trialis time dependent and a mean type test like Fisher’s ignores this important information whicheventually lead to a different conclusion; second, the number of infected asymptomatic patients, whichis necessary to use Fisher’s exact test, is is unknown in this data, invalidating their results. At Boulware et al. , the authors analysis did not rejected the hypothesis thathydroxychloroquine effect was equal to placebo effect and they concluded that hydroxychloroquine didnot prevent symptoms of Covid-19 as prophylaxis treatment. Note this conclusion cannot be made byany hypothesis test, which only states in this case there is not enough statistical evidence to refuse nullhypothesis, which is different from stating the alternative hypothesis is correct. Their conclusionincorrectly states there is no evidence of efficacy, while the evidence is positive although notconclusive at 95% level with the sample size and methodology used.Applying the modifications we have stated in sections 2 and 3, in particular using a simplelinear regression method to their data, we conclude the randomized trial of Boulware et al. hasstatistical evidence, at 99% confidence level, that hydroxychloroquine treatment is time-dependent witha negative slope. We conclude that, when applied as a prophylaxis, it can significantly reduce therelative proportion of symptomatic patients if used from 0 to 2 days after exposure to the virus (71.98%for 0 days, 48.86% for 1 day and 29.33% for 2 days). The predictive value for day 0 can be seen aslower bound for the expected hydroxychloroquine efficacy if used as a pre-exposure prophylaxis. Thissuggests that pre-exposure prophylaxis can be significantly effective. For 3 and 4 days, we concludethere is no statistical evidence, at 99% level, that hydroxychloroquine treatment reduces the proportionof symptomatic patients. Moreover, our results show that the elapsed time between the exposure to the virus and thebeginning of treatment is vital to the effectiveness of the antiviral use. We expect the treatment will bemore effective when applied to patients in the viral replication period, before viral load reaches its peakwhich occurs around 5 days after symptom onset. Meanwhile, if disease reaches the inflammatoryperiod, typically after 8 days of symptoms onset and after viral load reaches its peak, we can expectedno or little benefit with the antiviral treatment.Therefore, the mean time elapsed from exposure to the virus and the start of treatment in thesample may act as a lurking variable, influencing in a hidden way the efficacy of treatment. This mightxplain why many studies have found no statistical evidence of effectiveness of hydroxychloroquinetreatment when used in hospitalized patients as most of this more severe cases had probably startedtreatment long after 4 days from their exposure to the virus. In addition, it helps to understand whysome studies have shown some positive results of hydroxychloroquine treatment as we can expect thiswhen the proportion of patients in the beginning of the infection is higher in the sample. Hence, asdescribed by Boulware et al., two possible applications would be to apply prophylaxis to healthprofessionals and to contacts of positive patients, since these two groups would have a greaterprobability to benefit from treatment. Our results suggest there is probably little or no benefit if the treatment is used in patientsinfected for too long, like hospitalized severe patients. On the contrary, they also suggest infectedpatients may have a large benefit if treated as early as possible, mostly as pre-prophylaxis treatmentwhere symptoms appear will have an estimated relative reduction of at least 72%.Another important aspect is that the variable of the study, be asymptomatic or be symptomatic,is quite time sensitive. Future trials should adopt less time sensitive variables, such as the number ofdays each patient is symptomatic, which could measure the possible benefit of treatment for patientsthat have been exposed for more than 3 days before the beginning of treatment. Another commonpossibility is to adopt some score system to measure severity of symptoms. However, score systemsare difficult to be scientifically validated because they typically depend on personal judgment, whereasthe variable number of days with symptoms can be more easily replicated by other studies.Furthermore, the hydroxychloroquine prophylaxis should also be investigated with concomitantuse of azithromycin and zinc , as also other antivirals should also be tested as prophylaxis, measuringthe relative efficacy as a measure of the elapsed time after exposure to the virus to the beginning oftreatment. We conducted all statistical analysis with R software, version 4.0.0.We declare no conflict of interests.Author: Márcio Watanabe, Ph.DDepartment of Statistics, Federal Fluminense UniversityAddress: Rua Professor Marcos Waldemar de Freitas Reis, s/nInstituto de Matemática e Estatística – Bloco H – 3 andar – Ala ASão Domingos – 24.2010-201 – Niterói – RJ, BrasilEmail: [email protected] eferences Boulware DR, Pullen MF, Bangdiwala AS, et al. A randomized trial of hydroxychloroquine aspostexposure prophylaxis for Covid-19. N Engl J Med (2020). DOI: 10.1056/NEJMoa2016638. Cohen MS. Editorial - Hydroxychloroquine for the Prevention of Covid-19 - Searching for Evidence.N Engl J Med (2020). DOI: 10.1056/NEJMe2020388. Pastick KA, Okafor EC, Wang F, et al. Review: hydroxychloroquine and chloroquine for treatmentof SARS-CoV-2 (COVID-19). Open Forum Infect Dis 2020; 7: ofaa130. Rosenberg ES, Dufort EM, Udo T, et al. Association of treatment with hydroxychloroquine orazithromycin with in-hospital mortality in patients with COVID-19 in New York State. JAMA 2020May 11 (Epub ahead of print). Liu J, Cao R, Xu M., et al. Hydroxychloroquine, a less toxic derivative of chloroquine, is effective ininhibiting SARS-CoV-2 infection in vitro. Cell Discov 6, 16 (2020). https://doi.org/10.1038/s41421-020-0156-0 Tay, M.Z., Poh, C.M., Rénia, L.et al. The trinity of COVID-19: immunity, inflammation andintervention. Nat Rev Immunol 20,363–374 (2020). https://doi.org/10.1038/s41577-020-0311-8 Gautret, P. et al. Hydroxychloroquine and azithromycin as a treatment of COVID-19: results of anopen-label non-randomized clinical trial. Int. J. Antimicrob. Agents. (2020). https://doi.org/10.1016/j.i ja n timicag . 2020.1059498.