Response to “Safety and effectiveness of SARS‐CoV‐2 vaccines: A systematic review and meta‐analysis”

Abstract

To the Editor, We read with great interest the systematic review and meta‐analysis on the safety and efficacy of SARS‐CoV‐2 vaccines by Ling et al. The authors included nine randomized controlled trials (RCTs) in their analyses and concluded that the incidence of adverse events associated with SARS‐CoV‐2 vaccines is greater versus placebo. We are grateful for the authors up‐to‐date analysis on the efficacy and safety of SARS‐CoV‐2 vaccines, which has garnered considerable attention during the pandemic. However, there are major limitations in the authors methodology and interpretation that need to be addressed to ensure correct application of the findings. First, the authors used several outdated frameworks and did not report essential information regarding their review. Specifically, neither a detailed search strategy including any pertinent RCT filters, nor any identification number for the prospective registration of their review such as a PROSPERO number, was reported. Additionally, the authors only identified 91 items after their systematic searches. A review on the same topic by Pormohammad et al. identified 32 790 initial items, and therefore, it is difficult to assess whether all of the available evidence was included. Furthermore, they did not conduct the review with the most up‐to‐date methodology, such as PRISMA 2020 guidelines, RoB 2, and the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) framework. Lastly, the authors did not indicate that any screening or extraction process was conducted in duplicate. These components are essential to the integrity and reproducibility of systematic reviews, and are requirements in both PRISMA and Cochrane guidelines. Second, the author used I statistics to determine the use of a fixed versus random‐effects model for their meta‐analysis. This approach does not align with the Cochrane handbook which specifically states that “the choice between a fixed‐effect and a random‐effects meta‐analysis should never be made on the basis of a statistical test for heterogeneity.” Rather, this decision should have been made on the basis of the level of similarity between study characteristics. Considering that the authors analyses involved different types of vaccines across different platforms, with varied participant demographics, a random‐effects model should have been implemented, even for analyses with low statistical heterogeneity. Moreover, the authors did not appropriately assess the significance of their findings. Notable examples include the erroneous statements that the odds of developing any and local adverse events is significantly higher in inactivated virus vaccine recipients compared with placebo, despite the 95% confidence interval (CI) (odds ratio [OR], 0.76–7.87 and 0.87–1.71 for any and local adverse events, respectively) crossing the line of no difference in both cases. Similar examples can be found when the authors stated that the vaccine group had a higher incidence of fever, swelling, and erythema compared with placebo despite wide, nonsignificant CIs. Furthermore, the authors observed that the incidence of adverse events is higher in low‐dose and high‐dose vaccine recipients compared with medium‐ dose recipients in several analyses, and concluded the presence of a dose‐dependent relationship. While dose‐dependent relationships between vaccine dosage and adverse event incidence were found in clinical trials, they mostly resemble a typical dose–response curve rather than the V‐shaped curve presented by the authors. In any case, the dose‐dependent variations are small and likely insignificant; for instance, the OR of inactivated virus vaccine adverse events is 1.33, 1.10, and 1.32 for low‐, medium‐ and high‐dosages, respectively. We speculate that these variations may be attributed to different numbers of studies in the analysis of each dosage subgroup, rather than underlying dose‐dependent relationships. Lastly, we believe that this systematic review could be greatly improved if the analyses were stratified by adverse event severity. While the authors did comment on the incidence of severe adverse events, this analysis was only conducted for RNA vaccines and was not reported in the Methods section. Stratifying the analyses by, for example, the Common Terminology Criteria for Adverse Events (CTCAE) grading system which is commonly used by RCTs, would be more effective at communicating the actual safety profiles of SARS‐ CoV‐2 vaccinations. Current evidence suggests that most vaccine side effects are mild and short‐lived, despite the findings of this meta‐analysis. Vaccination is the most effective tool for combating the pandemic. Thus, it is important that rigorous methodologies are utilized in research on the safety of SARS‐CoV‐2 vaccines, especially at a time when vaccine hesitancy and misinformation serve as major barriers to the widespread adoption of vaccination programs. The authors should consider the aforementioned critiques and re‐evaluate their methodology and conclusions.