Considerations, Good Practices, Risks and Pitfalls in Developing AI Solutions Against COVID-19
Alexandra Luccioni, Joseph Bullock, Katherine Hoffmann Pham, Cynthia Sin Nga Lam, Miguel Luengo-Oroz
CConsiderations, Good Practices, Risks and Pitfallsin Developing AI Solutions Against COVID-19
Alexandra Luccioni , , Joseph Bullock , , Katherine Hoffmann Pham , , Cynthia Sin Nga Lam and Miguel Luengo-Oroz Universit´e de Montr´eal Mila Qu´ebec Artificial Intelligence Institute United Nations Global Pulse Institute for Data Science, Durham University NYU Stern School of Business Global Coordination Mechanism on NCDs, [email protected], { joseph,katherine,miguel } @unglobalpulse.org, [email protected] Abstract
The COVID-19 pandemic has been a major chal-lenge to humanity, with 12.7 million confirmedcases as of July 13th, 2020 [1]. In previous work,we described how Artificial Intelligence can beused to tackle the pandemic with applications atthe molecular, clinical, and societal scales [2]. Inthe present follow-up article, we review these threeresearch directions, and assess the level of matu-rity and feasibility of the approaches used, as wellas their potential for operationalization. We alsosummarize some commonly encountered risks andpractical pitfalls, as well as guidelines and bestpractices for formulating and deploying AI appli-cations at different scales.
In recent months, an already remarkable body of researchon potential applications of Artificial Intelligence (AI) andMachine Learning (ML) to address the health and societalchallenges created by the COVID-19 pandemic has been pro-duced. In a recent survey of the literature [2], we identifiedover 300 relevant journal articles and preprints with applica-tions ranging from tracking the spread of misinformation tomolecular modeling of the SARS-CoV-2 virus. These appli-cations are emerging in a wide array of settings, increasinglydriven by the priorities of responders and multidisciplinarycollaborations.In this article, we present a more in-depth analysis of AIapplications at the molecular, clinical, and societal scales.We build on our previous work by sharing some high-levelobservations on the broader landscape of AI for COVID-19and providing a critical take on best practices and applicationstrategies. While the speed and scale of the AI community’sresearch response to the pandemic is unprecedented, we feelnonetheless that there are a number of common, structuralchallenges faced by many of these projects. We extract shareddesign considerations and promising directions, as well as re-curring problems, risks, and areas of improvement that couldhelp researchers and funding bodies when developing furtherapplications and research agendas.
Promising Directions
AI has been applied in a wide spec-trum of molecular research – ranging from better understand-ing the structure of the SARS-CoV-2 virus to assisting drugdevelopment and improving molecular diagnosis.Some of the earliest responses to the pandemic were pro-duced by AI researchers leveraging existing protein structureprediction algorithms [3] and drug discovery pipelines [4], orexploring pre-built knowledge graphs [5]. Having the infras-tructure for this research already in place facilitated a rapidresponse to the new challenge of COVID-19. Open science,based on previously published peer-reviewed papers detail-ing the methodologies involved, has helped to accelerate theevaluation of these approaches.A number of projects studied drug re-purposing, in thehope of discovering a therapy for which the engineering ofnovel compounds is not needed, clinical trials have been au-thorized, or the compound’s use has already been approved(e.g., [6]). Others used ML approaches to reduce the com-putational burden of docking simulations by narrowing downthe set of candidate compounds which needed to be docked(e.g., [7]). Another notable trend in molecular applicationsinvolves identifying desirable or undesirable properties of acandidate compound – e.g., novelty, drug likeness, or toxicity– and then training drug discovery models to suggest com-pounds that meet these desiderata (e.g., [8]). These applica-tions allowed creative exercises from the modeling perspec-tive, in which researchers experimented with different repre-sentations of the same data to design different AI pipelines.Many papers relied on a common set of open datasets suchas PDBbind, ChemBL, and DrugBank, which makes theirmethodologies more accessible and replicable for other re-searchers. Several even posted models or results on Github orother web pages, providing open access to all (e.g., [9]).
Risks and Pitfalls
Many ML applications in molecular sci-ence remain at the research level. It is understandable thatmolecular research is part of a multidisciplinary and multi-step process; nonetheless, few of the efforts we observed hadadvanced through the drug development pipeline to more for-mal evaluation. One challenge is that synthesizing and testingcompounds is costly and time-consuming, so researchers whoare not embedded within a larger infrastructure are often un-able to execute these later steps required to translate research a r X i v : . [ c s . C Y ] A ug igure 1: AI applications for the COVID-19 response organized at three levels: the molecular scale, the clinical scale, and the societal scale. to practice.Another barrier in this research-to-practice process couldbe limitations in data and model sharing. To our knowledge,two candidate vaccines that reported the use of ML in theirdevelopment have been approved for clinical evaluation [10].However, both vaccines came from corporations which pub-lished limited information on their approach and the extentto which ML was utilized in vaccine development. Moregenerally, we found that some of the research produced byprivate-sector entities relied on datasets such as knowledgegraphs which are not publicly accessible, describing propri-etary models only in vague or high-level terms. Hence, wewould like to reiterate the importance of open science in en-suring accessible vaccines and treatments for vulnerable com-munities. Promising Directions
From a clinical perspective, AI andML have already been used to assist and improve patient-levelassessment of COVID-19. One major application for ML inthis field has been the analysis of medical imagery such as CTand X-Ray scans with the help of common neural network ar-chitectures, either to provide a supplementary data point tocorroborate COVID-19 diagnosis [11], or to assess the sever-ity and progression of the disease [12]. Some of these ap-proaches are now fully operational and have received institu-tional approval for deployment in hospitals as triage tools [13]or as human-in-the-loop systems for radiologists [14].Approaches using complementary data sources have alsobeen proposed, ranging from wearable devices and mobilephones for detecting symptoms [15], to electronic healthrecords to improve diagnosis and outcome prediction. Wefind one of the most promising applications of AI at a clinicallevel to be the prediction of patient outcomes and the proposalof triage approaches based on features extracted from medi-cal data. Such approaches are transparent and practicable,pinpointing key measurable features which enable hospitalsto plan the use of resources such as ventilators and ICU beds.Furthermore, there are promising hybrid studies that lever-age both medical imagery and clinical features to predictpatient-level characteristics such as the severity of COVID-19 [16]. These approaches leverage different complementarysources of data in order to make more precise and more gener-alizable predictions of patients’ prognosis. In fact, the clinical features identified by studies using solely clinical data werealso corroborated by hybrid studies; e.g. both sets of studiesfound that high levels of substances such as lactic dehydro-genase (LDH) and high-sensitivity C-reactive protein (CRP)were correlated with mortality risk and longer hospitalization.
Risks and Pitfalls
While many studies are being carriedout in situ on COVID-19 patients, there is still much that wedo not know about the virus itself and the factors that canput patients at risk for hospitalization, developing acute res-piratory distress syndrome (ARDS), and eventual death fromrespiratory failure. For instance, the extent to which medicalimagery alone can be used for the diagnosis of COVID-19 isstill debated by the medical community [17]. Above and be-yond the feasibility of diagnosis, many of the medical imag-ing papers we reviewed had methodological issues, relying onsmall and poorly-balanced datasets that mix data from severalpopulations, coupled with flawed evaluation procedures [18].Most also presented no plan for inclusion in clinical work-flows and no attempt to provide a transparent explanation forthe diagnosis, which is especially important in patient-levelapplications of AI [19]. Finally, while approaches that lever-age ML to analyze non-invasive measurements are potentiallypromising given the ubiquity and accessibility of sensor tech-nologies, we found that these approaches are not sufficientlymature to evaluate their performance. We would advocatemore extensive testing and clinical investigations to validatetheir performance in deployment.
Promising Directions
From a societal perspective, AI hasbeen applied to the field of epidemiological modelling, aswell as to understanding and combating the “infodemic”spread of misinformation [20]. At the epidemiological level,many studies have sought to produce forecasting models fornational and regional level statistics. While a vast body ofliterature on modeling already exists, AI based models couldaugment classical models in situations where analytic trans-mission equations are not well known, such as when mod-elling the effects of public policy measures such as social dis-tancing and self-quarantine [21]. In addition, AI methods canalso be used to incorporate new data sources, such as socialmedia and search information [22]. Other works use AI toidentify similarities and differences in the evolution of thepandemic between regions. These approaches have leveragedoth supervised and unsupervised techniques, and may helpinform policy makers at a high level and highlight areas formore detailed exploration.AI has also been applied to investigate the scale and spreadof the infodemic in order to address the propagation of mis-information and disinformation including the emergence ofhate speech. Given the vast amount of information now beingdisseminated and shared, there is a need for tools to help iden-tify and promote reliable information sources, and understandthe spread of misinformation. Promising work has analyzedpatterns in the transmission of such information and devel-oped infodemic risk scoring algorithms (e.g., [23]). More-over, there has been an increasing focus on assessing theemergence of hate speech, particularly using network anal-ysis techniques, which could help inform preventative effortsor contribute to the development of continuous monitoringplatforms [24].
Risks and Pitfalls
Policy decisions must be based on justi-fiable models which stand up to public scrutiny. Since muchof the data collected for COVID-19 epidemiological mod-eling tasks is extremely limited, the choice of models anddatasets can have significant effects on overall performanceand models may lack generalizability. A significant limitationof many articles in this category is the heterogeneous data col-lection in different countries due to multiple factors includingvariations in testing, case tracking, and reporting quality andstandards. Moreover, applying models trained in one contextto another raises concerns surrounding the model’s ability tocapture aspects such as different cultural norms which mayimpact the spread and effect of the virus. In such examples,a transferred model will have to be tailored for local contextsgiven that there may be different demographic characteristicsand behaviors. Indeed, developing proper model transferabil-ity procedures and guidelines is especially important for datapoor regions. While synthetic data approaches have been pro-posed, these should be applied with caution.In order to understand and tackle the infodemic, it is im-portant to capture and analyze information from a diverserange of sources. While much of the work on this topic usesdata from online sources such as social media and Googlesearches, information propagated through alternative chan-nels such as radio is important for capturing wider trends.Moreover, many of the existing approaches rely on languagemodelling techniques developed for English or other widelyspoken languages, but relying on such models might leavemany populations behind, including some of those most vul-nerable. Finally, we note that while numerous methods havebeen proposed for identifying hate speech, further research isneeded to identify the targeted groups and to learn how to usethe insights gained from these techniques in an effective way(e.g. see Section II [25]).
We believe that there are several considerations to keep inmind when applying AI to a global problem such as theCOVID-19 pandemic. These include:1.
Application relevance and context : Does the applica-tion make sense from both an application and a method- ological perspective? Were domain experts consulted toproperly assess the needs and define the problem to besolved? Does the solution serve its target audience?2.
Data availability and quality : Has the quantity of databeing used to train and evaluate the model been assessedand deemed to be of a reasonable size and diversity tojustify the claims made? Has bias in the data been con-sidered and documented? Were privacy measures taken?3.
AI methodology and complexity : Is the approach pro-posed justified? If an AI based model, has it been bench-marked against more traditional approaches? Has theapproach been validated by other researchers and, if pos-sible, peer-reviewed?4.
Transparency and explainability : Have efforts beenmade to render the results and the approach understand-able by humans? Is it possible to identify the key fea-tures being used by the algorithm?5.
Dissemination of knowledge : Are the data, code andmodels being shared in any form? Are there reportingguidelines and standards that should be followed?6.
Operationalization and performance : Can the approachbe integrated into decision-making workflows? What isthe implementation plan? Are there regulatory frame-works that have to be taken into account? What are thepotential risks? Will the models incorporate user feed-back, and if so, how?The COVID-19 pandemic is a global emergency that hasoverstretched health care networks and posed significanthealth, economic and social challenges to humanity. AI canplay an important role in alleviating this pressure, but wewould like to reiterate that in order for any technological solu-tion to make an impact, it must be deployed contextually andappropriately. We advocate for aspiring research initiatives tobe carried out in partnership with stakeholders who have thenecessary domain knowledge. It is also important to investi-gate how proven solutions can be adapted to local contexts toaddress unmet needs, particularly in areas of the world withfewer resources. This requires developing appropriate modeland data sharing solutions, and specific measures to addressdata scarcity (see e.g. [26] for more details).Finally, given the rapidly changing nature of human under-standing regarding the pandemic, and therefore the inabilityto fully validate many approaches, we suggest that modelsshould not be designed to process data in an end-to-end fash-ion at this stage, but rather to augment human decision mak-ing. With careful attention to the implementation context andoperational needs, we believe that AI solutions can be a valu-able asset in the fight against the pandemic.
Acknowledgements
United Nations Global Pulse is supported by the Govern-ments of Netherlands, Sweden and Germany and the Williamand Flora Hewlett Foundation. JB also is supported by theUK Science and Technology Facilities Council (STFC) grantnumber ST/P006744/1. AL is supported by funding fromIVADO and Mila. eferenceseferences