An early prediction of covid-19 associated hospitalization surge using deep learning approach
AA N EARLY PREDICTION OF COVID -19
ASSOCIATEDHOSPITALIZATION SURGE USING DEEP LEARNING APPROACH
Yuqi Meng
Northeast Yucai Foreign Language SchoolShenyang, Liaoning, China
Ying Zhao
Department of Engineering Science and Applied MathNorthwestern UniversityEvanston, IL, U.S.A [email protected]
Zhixiang Li
Department of Biomedical EngineeringShenyang Pharmaceutical UniversityShenyang, Liaoning, China A BSTRACT
The global pandemic caused by COVID-19 affects our lives in all aspects. As of September 11, morethan 28 million people have tested positive for COVID-19 infection, and more than 911,000 peoplehave lost their lives in this virus battle. Some patients can not receive appropriate medical treatmentdue the limits of hospitalization volume and shortage of ICU beds. An estimate future hospitalizationis critical so that medical resources can be allocated as needed. In this study, we propose to use 4recurrent neural networks to infer hospitalization change for the following week compared with thecurrent week. Results show that sequence to sequence model with attention achieves a high accuracyof 0.938 and AUC of 0.850 in the hospitalization prediction. Our work has the potential to predict thehospitalization need and send warning to medical providers and other stakeholders when a re-surgeinitializes.
COVID-19, the disease caused by SARS-CoV-2 virus, is affecting our daily life from financing, health, employment,travel restriction, family apart to every details [1]. According to the study of John Hopkins University [2], by September11th, more than 28 million people have been tested positive of COVID-19 infection. We lost more than 911,000 lives inthis battle of the virus. United States of America has the most serious community transmission and fatal cases. Solely,in U.S.A, more than 6 million people are infected with COVID-19, which accounts for more than 196,000 death. Untila efficient, risk-free and large-scale available vaccine is developed, we have to keep fighting with the disease and can’tresume to our daily life before the global pandemic.Most patients infected with COVID-19 only have mild symptoms such as cough, fever, muscle pain and so on [3]. Thesepatients could recover with self-quarantine and medical treatment by their patients. However, people aged over 65 orhaving underlying medical condition such as lung disease, chronic disease, diabetes, are more at risk for complicationsfrom the virus [4]. Those patients with serious symptoms should receive hospitalization under the guidance of medicalprovider. According to the data of US Centers for Disease Control and Prevention, the overall cumulative hospitalizationrate associated with COVID-19 is 166.9 per 100,000 with the highest rates in people aged 65 years and older (451.2per 100,000) followed by the age group of 60-64 years (249.8 per 100,000). A high hospitalization rate may cause theshortage of health resources, higher risk of community transmission and extra burden to health provider. Therefore, aprecise estimate of surge of hospitalization is essential.Numerous models are being used to predict the hospitalization rates for the following four weeks [5]. These modelscan be formed to 2 groups. In one group, machine learning and deep learning are widely used to infer the trend ofhospitalization change. In the other group, traditional statistics, such as naive estimator and auto regression model are a r X i v : . [ c s . L G ] S e p Figure 1: 35 days running hospitalization of U.S.A starting from July 1st, 2020being applied to make predictions. In this work, we will predict the hospitalization change of the next one week frommulti-modal data using different deep learning configurations.
Thanks to the fast development of computational technology, deep learning and machine learning has been widely usedin the different aspects. Healthcare also benefits a lot from many deep learning and machine learning models built byresearchers including performance improvement of disease early prediction [6, 7], automatic interpretation of medialimages [8, 9], RNA sequence pattern identification [10] and so on. In terms of epidemic prediction, time series deeplearning models are also widely involved. Ning et al [11]. apply a 2-step augmented regression to estimate regionalinfluenza epidemics. Kondo et al. [12] use sequence-to-sequence model to achieve state-of-the-art Pearson score inthe influenza prevalence prediction. Particularly to COVID-19, the Karlen Working Group uses the rate of reportedinfections to estimate the number of new hospitalizations in a given jurisdiction. The Institute of Health Metrics andEvaluation infers numbers of new hospitalizations based on numbers of predicted deaths. Unlike these work, we willuse statistical figure of the disease of the past week to infer whether the hospitalization of the following week will surgecompared with current week. This work will contribute to the medical resource allocation and disease prevention.
The dataset we use is named COVIDTracking and is published at https://github.com/google-research/open-covid-19-data/. 20 epidemiological statistics of U.S.A including positive cases, death, test number, cumulative hospitalizationand other are collected from January 1st, 2020. The dataset is updated everyday. An example of 35 days runninghospitalization is shown in Figure 1. Our research problem is defined as follows. We are trying to infer the averagehospitalization change from the epidemiological statistics of past 4 weeks. The average hospitalization change is abinary label and defined as the comparison between averaged next 7 days running hospitalization and averaged pastweek (including today) running hospitalization. We used Figure 1 to illustrate our task. Suppose the reference date isJuly 28th, 2020. The input features are 20 epidemiological statistics from July 1st to July 28th, 2020. The predictedoutcome is 0 in that the running averaged hospitalization from July 29th to August 4th (bars with horizontal strips) issmaller than that of July 22th to July 28th (bars with diagonal strips). As the cases number and other factors beforeMarch 1st are highly limited in U.S. Therefore, we will only use the data after March 1st. Given that our time lag is 4weeks in the prediction, our reference date is from April 1st, 2020 to August 30th, 2020. There are 152 days in total.2able 1: Performance of 4 recurrent neural networks on the prediction of hospitalization change associated withCOVID-19
Model Accuray AUC
Long Short-Term Memory 0.639 0.539Stacked Long Short-Term Memory 0.738 0.773Bi-directional Long Short-Term Memory 0.885 0.767Sequence-to-sequence Model with Attention 0.934 0.850
Recurrent neural networks (RNN) are widely used in the time series prediction. In our study, we will use a singlelayer of Long short-term memory (LSTM) [13] with 64 neurons as the baseline. On top of this we will try some otherconfigurations with augmentation and attention. First, we will try a 3-layer stacked LSTM model [14] with 128, 64 and32 neuron at each LSTM layer. Next, we adopt Bidirectional LSTM (Bi-LSTM) [15]. The bidirectional structure allowsthe network to transmit both forward and backward at every time step. We also try sequence-to-sequence model withattention. In this model, LSTM layers are used as both encoder and decoder. This model yields state-of-the-arts modelin influenza prediction. We expect it can also boost the performance of naive LSTM models. For the technical detailswe will use stochastic gradient descent optmizier with learning rate 0.01 and a momentum of 0.9. As our problem isa binary classification, we employ binary cross-entropy as our loss function. For the stacked LSTM and the LSTMcomponents of seq-to-seq model, the LSTM layer is followed by a rectifier activation and dropout layer with dropoutrate 0.2. We will train each classifier for 200 epochs with early termination conditioned on the loss of developing set.
We first split all data into training set and test set. In the training set, the reference date is from April 1st to Jun 30th,2020 (91 days in total). In the test set, the reference date is from July 1st to August 30th, 2020 (61 days in total). Werandomly select 10 days from training set as a development set for hyper-parameters tuning purposes. We evaluate theheld-out test set using accuracy and area under the receiver operating characteristic (AUC).
We build the entire pipeline using Python 3.6. Keras is used to design deep neural networks. Pandas and Numpy isapplied to pre-process data. Google Colab with GPU runtime is used to reduce training time. Scikit-learn package isused to evaluate the model performance.
The results are shown in Table 1. Sequence to sequence model with attention yields the best accuracy of 0.934 and AUCof 0.850, followed by bidirectional LSTM, which achieves an accuracy of 0.885 and AUC of 0.767. We would liketo note that the test sample is only 61. Bi-LSTM correctly predict 56 out of 61 hospitalization change while Seq2seqoutperforms a bit that correctly infer one more hospitalization change. LSTM model and stacked LSTM, due to itssimple architecture, don’t achieve optimal results.
We visualized the predicted trend compared with ground truth. We can see that LSTM model simply predicts all thetime series to negative. This is as expectation as the simple LSTM network will cause over-fitting on the trainingset. In the real case, running hospitalization change flip over from increaing to decreasing on July 22nd, 2020. BothSeq2seq model and Bi-LSTM could give an estimate of this flipping over trending. When talking about the practicaluse, our time series classifier could give an estimate of the hospitalization change. This may help the health providerand stakeholder to allocate the limited medical source. Moreover, it can also send warning to people when there is aCOVID-19 re-surge.Our work also have following limitations. First, as COVID-19 is caused by a novel virus and only traced recently,we only have a sample size of 152. A larger dataset could potential improve our model performance. Secondly, wedon’t apply some other trending time series prediction model such as deep transformers [16], which has shown great3 / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / Ground Truth LSTM Stacked LSTM Bi-LSTM Seq2Seq wAttHospitalization IncreasedHospitalization Decrease
Figure 2: Predicted trends from 4 recurrent neural networks compared with the ground truth trendsuccess in influenza prediction [17]. Finally, we don’t try other configurations such as 2 week time lag, the monthlyhospitalization change and so on. In the future study, we will keep monitoring this global pandemic and update as wellas upgrade our model to achieve better hospitalization change prediction.
All authors discuss and define the research problem. Y.M. extracts and preprocess all the research data. Y.M. and Y.L.implement all deep neural networks, analyze and visualize the results. Y.M. and Y.L. write the manuscript advised byZ.L., who also organizes the manuscript to a better academic standard.4 eferences [1] Naomi Laventhal, Ratna Basak, Mary Lynn Dell, Douglas Diekema, Nanette Elster, Gina Geis, Mark Mercurio,Douglas Opel, David Shalowitz, Mindy Statter, et al. The ethics of creating a resource allocation strategy duringthe covid-19 pandemic.
Pediatrics , 146(1), 2020.[2] Ensheng Dong, Hongru Du, and Lauren Gardner. An interactive web-based dashboard to track covid-19 in realtime.
The Lancet infectious diseases , 20(5):533–534, 2020.[3] Wenjie Yang, Qiqi Cao, Le Qin, Xiaoyang Wang, Zenghui Cheng, Ashan Pan, Jianyi Dai, Qingfeng Sun, FengquanZhao, Jieming Qu, et al. Clinical characteristics and imaging manifestations of the 2019 novel coronavirus disease(covid-19): A multi-center study in wenzhou city, zhejiang, china.
Journal of Infection , 2020.[4] Team CDC COVID and Response Team. Severe outcomes among patients with coronavirus disease 2019(covid-19)-united states, february 12-march 16, 2020.
MMWR Morb Mortal Wkly Rep , 69(12):343–346, 2020.[5] Fotios Petropoulos and Spyros Makridakis. Forecasting the novel coronavirus covid-19.
PloS one , 15(3):e0231236,2020.[6] Yikuan Li, Liang Yao, Chengsheng Mao, Anand Srivastava, Xiaoqian Jiang, and Yuan Luo. Early predictionof acute kidney injury in critical care setting using clinical notes. In , pages 683–686. IEEE, 2018.[7] Hanyin Wang, Yikuan Li, Hongyan Ning, John Wilkins, Donald Lloyd-Jones, and Yuan Luo. Using machinelearning to integrate socio-behavioral factors in predicting cardiovascular-related mortality risk. In
MedInfo , pages433–437, 2019.[8] Yikuan Li, Hanyin Wang, and Yuan Luo. A comparison of pre-trained vision-and-language models for multimodalrepresentation learning across medical images and reports. arXiv preprint arXiv:2009.01523 , 2020.[9] Pranav Rajpurkar, Jeremy Irvin, Kaylie Zhu, Brandon Yang, Hershel Mehta, Tony Duan, Daisy Ding, Aarti Bagul,Curtis Langlotz, Katie Shpanskaya, et al. Chexnet: Radiologist-level pneumonia detection on chest x-rays withdeep learning. arXiv preprint arXiv:1711.05225 , 2017.[10] Babak Alipanahi, Andrew Delong, Matthew T Weirauch, and Brendan J Frey. Predicting the sequence specificitiesof dna-and rna-binding proteins by deep learning.
Nature biotechnology , 33(8):831–838, 2015.[11] Shihao Yang, Shaoyang Ning, and SC Kou. Use internet search data to accurately track state-level influenzaepidemics. arXiv preprint arXiv:2006.02927 , 2020.[12] Kenjiro Kondo, Akihiko Ishikawa, and Masashi Kimura. Sequence to sequence with attention for influenza preva-lence prediction using google trends. In
Proceedings of the 2019 3rd International Conference on ComputationalBiology and Bioinformatics , pages 1–7, 2019.[13] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory.
Neural computation , 9(8):1735–1780, 1997.[14] Wei Bao, Jun Yue, and Yulei Rao. A deep learning framework for financial time series using stacked autoencodersand long-short term memory.
PloS one , 12(7):e0180944, 2017.[15] Shouxiang Wang, Xuan Wang, Shaomin Wang, and Dan Wang. Bi-directional long short-term memory methodbased on attention mechanism and rolling update for short-term load forecasting.
International Journal ofElectrical Power & Energy Systems , 109:470–479, 2019.[16] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectionaltransformers for language understanding. arXiv preprint arXiv:1810.04805 , 2018.[17] Neo Wu, Bradley Green, Xue Ben, and Shawn O’Banion. Deep transformer models for time series forecasting:The influenza prevalence case. arXiv preprint arXiv:2001.08317arXiv preprint arXiv:2001.08317