Use Internet Search Data to Accurately Track State-Level Influenza Epidemics
UU SE I NTERNET S EARCH D ATA TO A CCURATELY T RACK S TATE -L EVEL I NFLUENZA E PIDEMICS
A P
REPRINT
Shihao Yang ∗ Shaoyang Ning † S. C. Kou ‡ June 5, 2020 K EY P OINTS
Big data from the Internet has great potential to track social and economic events at multiplegeographical levels. Here for real-time estimation of state-level influenza activities in the UnitedStates, we propose a statistical model that efficiently combines publicly available Internet search dataat multiple resolutions (national, regional, and state-level) with traditional influenza surveillance datafrom the Centers for Disease Control and Prevention. Our method, across all states, outperforms allexisting time-series-based influenza tracking methods. Our model is robust and easy to implement,with the flexibility to incorporate additional information from other sources and resolutions, makingit generally applicable to tracking other social, economic or public health events at the state or locallevel. K eywords Big data | Infectious disease tracking | Influenza-like-illness | Spatial correlation | Real-time estimation | error reduction ∗ Joint first author. Department of Biomedical Informatics, Harvard Medical School, Boston, MA 02115, USA † Joint first author. Department of Mathematics and Statistics, Williams College, Williamstown, MA 01267, USA ‡ To whom correspondence should be addressed. Department of Statistics, Harvard University, Cambridge, MA 02138, [email protected] a r X i v : . [ s t a t . A P ] J un PREPRINT - J
UNE
5, 2020 A BSTRACT
For epidemics control and prevention, timely insights of potential hot spots are invaluable. Alternativeto traditional epidemic surveillance, which often lags behind real time by weeks, big data from theInternet provide important information of the current epidemic trends. Here we present a methodology,ARGOX (Augmented Regression with GOogle data CROSS space), for accurate real-time trackingof state-level influenza epidemics in the United States. ARGOX combines Internet search data at thenational, regional and state levels with traditional influenza surveillance data from the Centers forDisease Control and Prevention, and accounts for both the spatial correlation structure of state-levelinfluenza activities and the evolution of people’s Internet search pattern. ARGOX achieves on average28% error reduction over the best alternative for real-time state-level influenza estimation for 2014 to2020. ARGOX is robust and reliable and can be potentially applied to track county- and city-levelinfluenza activity and other infectious diseases.
Introduction
Each year in the United States (US) alone, the seasonal influenza (flu) epidemics may claim up to 61,000 deaths [1].Quick responses and preventive actions to changes in flu epidemics rely on timely and accurate information on thecurrent flu severity. In particular, due to the geographically varying timing and intensity of disease epidemics, mostpublic health decisions and executive orders for disease control and prevention are made at the state or local level.Accurate real-time flu tracking at the state/local level is thus indispensable. Traditional flu surveillance, such as thoseconducted by the US Centers for Disease Control and Prevention (CDC), however, often lags behind real time by upto two weeks. Here we propose a statistically principled, self-coherent framework ARGOX (Augmented Regressionwith GOogle data CROSS space) for real-time, accurate flu estimation at the state level. ARGOX efficiently combinespublicly available Internet search data with traditional flu surveillance data and coherently utilizes the data from multiplegeographical resolutions (national, regional, and state levels).For the last two decades, tracking of flu activities in the US mainly relies on traditional surveillance systems, such asthe US Outpatient Influenza-like Illness Surveillance Network (ILINet) by the CDC. Through the ILINet, thousands ofhealthcare providers across the US report their numbers of outpatients with Influenza-like Illness (ILI) to CDC on aweekly basis. CDC then aggregates the data and publishes the ILI percentages (%ILI, i.e., the percentages of outpatientswith ILI) in its weekly reports at the national and regional levels (there are ten Health and Human Services (HHS)regions in the US, each consisting of multiple states). Starting from 2017, the state-level %ILI reports became availablefor selected states, and in late 2018 the state-level %ILI reports became available for all states except Florida. Owing tothe time for administrative processing and aggregation, CDC’s flu reports typically lag behind real time for up to 2weeks and are also subject to subsequent revisions. Such delay and inaccuracy are far from optimal for public healthdecision making, especially in the face of epidemic outbreaks or pandemics.Big data from the Internet offers the potential of real-time tracking of public health or social events. In fact, valuableinsights have been gained from the Internet data about current social and economical status of a nation, includingepidemic outbreaks [2] and macro economic indices [3, 4]. Furthermore, real-time data from the Internet could alsooffer insights at the regional, state, or local level. Examples include foreshadowing state-wise housing price index in theUS [5], estimating New York City flu activity [6], estimating real-time county-level unreported COVID-19 severity inthe US [7] among others. For epidemic surveillance, such real-time digital data at local level can be potentially usedto provide insights for early epidemic hot-spot detection and timely public health resource allocation (e.g. vaccinecampaigns) as well as to gather information on the overall disease prevalence.Various models have been proposed to utilize Internet data, especially Internet search volume data, to provide real-timeestimation of the current flu activity at the national level. Google Flu Trends (GFT), as one of the early examples,uses the search frequency of selected query terms from Google to estimate the real-time %ILI [2]. Recent models oncombining CDC’s surveillance data with Internet-derived data appear to work well at the national level [8, 9]. Othermethods, primarily targeting national flu epidemics, were also developed based on traditional epidemiology data andmechanistic models, such as susceptible-infectious-recovered-susceptible model with ensemble adjustment Kalmanfilter (SIRS-EAKF) [6, 10, 11, 12, 13].Compared to estimation at the national level, %ILI estimation at the regional or state level is much more challenging, asdocumented by FluSight, the CDC-sponsored Flu Prediction Initiative [14]. Due to factors like geographical proximity,transportation connectivity, and public health communication, the state-wise epidemic spread exhibits strong spatialstructure. However, many digital flu estimation methods [11, 15, 16], including GFT, ignore such spatial structureand apply the same national-level method to regional, and/or state-level flu estimation. A few attempts have beenmade to incorporate the dependent geographical structure. For example, [17] studied the estimation of ILI activity2
PREPRINT - J
UNE
5, 2020in the boroughs and neighborhoods of New York City (using traditional epidemiological mechanistic model withoutInternet-derived data) and concluded that the spatial network is helpful at the borough scale but not at the neighborhoodscale; [18] utilized an ordinary-least-squares-based network model to improve upon the output of GFT; [19] employs amulti-task nonlinear regression method for regional %ILI estimation; [20] uses a network approach for %ILI estimationin a few selected states; [21] shows that careful spatial structure modeling can lead to much improved accuracy in %ILIestimation at the regional level.Nevertheless, at the state level, no existing methods provide real-time flu tracking with satisfactory accuracy andreliability. (i) There are no unified approaches to combine multi-resolution and cross-state information effectively toprovide national, regional, and state-level estimates within the same framework. (ii) Few existing models can outperforma naive estimation method, which, for each state, without any modeling effort, simply uses CDC’s reported %ILI fromthe previous week as the %ILI estimate for the current week (see Figure 1 for an illustration). This would be particularlyworrisome for public health officials who rely on accurate flu estimation at the local level to make informed decisions.In this article we introduce ARGOX, a unified spatial-temporal statistical framework that combines multi-resolution,multi-source information to provide real-time state-level %ILI estimates while maintaining coherency with %ILIestimation at the regional and national levels (in a cascading fashion). To illustrate the underlying idea of ARGOX, letus take estimating the %ILI in California as an example. The real-time Google search volumes for flu-related terms like"flu symptoms" or "flu duration" from California reflect its current state-level flu intensity to some extent. In addition,California’s flu epidemics could be highly correlated with flu epidemics of nearby states such as Oregon and Nevada, aswell as with geographically distant but transportation-wise well-connected states such as Illinois. California’s currentflu situation may also depend heavily on the recent trends of flu epidemics, in particular, the overall national andPacific-west regional flu trends. Taken these considerations together, ARGOX operates in two steps: at the first step, itextracts Google search information of most relevant query terms at three geographical resolutions – national, regional,and state levels; at the second step, the cross-time, cross-resolution, cross-state information mentioned above, togetherwith Internet-extracted information, are integrated through careful modeling of their temporal-spatial dependencestructure, which yields significant enhancement in the estimation accuracy.Through the ARGOX framework, the state-level flu activity estimates are produced in a unified and coherent way withthe national and regional estimates. ARGOX achieves on average 28% mean squared error (MSE) reduction comparedto the best alternative and shows strong advantages over all benchmark methods, including GFT, time-series-basedvector autoregression (VAR), and another recent Internet-search-based method developed in Lu et al. (2019) [20].ARGOX achieves its high estimation accuracy through a few features: (i) it automatically selects the most relevantsearch queries to address the problem of lower-quality Google search information at state or regional level; (ii) itincorporates time-series momentum of flu activity; (iii) it pools the multi-resolution information by combining thenational-, regional-, and state-level data; (iv) it explicitly models the spatial correlation structure of state-level fluactivities; (v) it adapts to the evolution in people’s search pattern, Google’s search engine algorithms, epidemic trends,and other time-varying factors [22] with a dynamic two-year rolling window for training; and (vi) it achieves selectivepooling of most immediately relevant information for a handful of stand-alone states (details in Materials and Methods).
Results
We conducted retrospective estimation of the weekly %ILI at the US state level – 50 states excluding Florida whose ILIdata is not available from CDC, plus Washington DC and New York City – for the period of Oct 11, 2014 to March 21,2020. For each week during this period, we only used the data that would have been available – the historical CDC’sILI reports up to the previous week and Google search data up to the current week – to estimate state-level %ILI of thecurrent week. To evaluate the accuracy of our estimation, we compared the estimates with actual %ILI released byCDC weeks later in multiple metrics, including the mean squared error (MSE), the mean absolute error (MAE), and thecorrelation with the actual %ILI (detailed in Materials and Methods). We also compared the performance of ARGOXwith several benchmark methods, including (a) GFT (last estimate available: the week ending on August 15, 2015), (b)estimates by the lag-1 vector autoregressive model (VAR model), (c) the naive estimates, which for each state withoutany modeling effort simply use CDC’s reported %ILI of the previous week as the estimate for the current week, and(d) a recent Internet-search-based state-level estimation model developed in Lu et al. (2019) [20]. As ARGOX usesa two-year training window, for fair comparison we keep the same two-year training window for VAR as well. Alsofor fair comparison, the numerical results of the method of Lu et al. (2019) were directly quoted from the article [20](which reported results through May 14, 2017).Table 1 summarizes the overall results of ARGOX, VAR, GFT, and the naive method, averaging over the 51states/district/city for the whole period of 2014 to 2020 (up to March 21, 2020). Table 2 summarizes the com-parison between ARGOX and the method of Lu et al. (2019), averaging over 37 states for the period of 2014 to 2017.3
PREPRINT - J
UNE
5, 2020Whole period ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 1.556 1.606 0.819 1.629 2.615 1.277 3.747GFT – 2.186 – – – – –naive 0.473 0.665 0.257 0.551 0.779 0.434 1.150MAEARGOX
VAR 0.597 0.633 0.516 0.693 0.825 0.668 1.058GFT – 0.944 – – – – –naive 0.393 0.435 0.340 0.464 0.547 0.443 0.696CorrelationARGOX
VAR 0.857 0.806 0.693 0.752 0.854 0.813 0.772GFT – 0.904 – – – – –naive 0.931 0.885 0.803 0.842 0.902 0.890 0.874Table 1: Comparison of different methods for state-level %ILI estimation. The evaluation is based onthe average of 51 US states/district/city in multiple periods and multiple metrics. The MSE, MAE, andcorrelation are reported. The method with the best performance is highlighted in boldface for each metric ineach period. Methods considered here include ARGOX, VAR, GFT, and the naive method. All comparisonsare conducted on the original scale of CDC’s %ILI. The whole period is Oct 11, 2014 to March 21, 2020.Columns 3 to 8 correspond to the regular flu seasons (week 40 to week 20 next year, defined by CDC’sMorbidity and Mortality Weekly Report; 19’-20’ season is up to March 21, 2020).Overall (’14-’17) ’14-’15 ’15-’16 ’16-’17MSE ARGOX
Lu et al. (2019) [20] 0.418 0.467 0.528 0.544CorrelationARGOX
Lu et al. (2019) [20] 0.912 0.912 0.808 0.858Table 2: Comparison of ARGOX to the method of Lu et al. (2019) [20] for state-level %ILI estimation.The numbers of Lu et al. (2019) are directly obtained from [20], which reported its estimation results of 37states over three flu seasons: 2014 to 2017. For fair comparison, the result of ARGOX is restricted to thesame 37 states and the same time period to match [20]. The method with best performance for each metricin each period is highlighted in boldface.We need to compare ARGOX with Lu et al. (2019) in a separate Table 2 because the results of Lu et al. (2019) are onlyavailable for 37 states and only for the period of 2014 to 2017.Table 1 shows that ARGOX gives the leading performance uniformly through all flu seasons in all metrics. Particularly,ARGOX achieves up to 28% error reduction in MSE and about 15 % error reduction in MAE than the best alternativein the whole period. ARGOX also keeps consistent season-by-season performance, with at least 15% error reductionin MSE compared to the best alternative method in every season from 2014 to 2019. For the current 2019-2020 fluseason with the (onset of) COVID-19 pandemic, ARGOX’s accuracy still maintains. Compared with other benchmarks,ARGOX’s advantages in state-level flu tracking are substantial. VAR and GFT fail to outperform the naive method inany of the evaluated flu seasons. Both methods would give 2-to-3 folds of MSE compared with the naive method. Table2 shows that ARGOX also uniformly outperforms Lu et al. (2019) in all three seasons when the benchmark is available.More detailed results comparing ARGOX with the benchmarks can be found in the supplementary material (Table S4).Among all the methods that we numerically compared, ARGOX is the only one that uniformly outperforms the naivemethod in all 51 states/district/city in terms of MSE for the whole period of evaluation. Figure 1 plots the state-by-stateestimation results, showing the ratio of the MSE of a given method to the MSE of the naive method. The results of fourmethods are plotted: ARGOX, VAR, GFT, and Lu et al. (2019). For each state, a blue color means that the MSE ofa method is smaller (better) than the MSE of the naive method for that state, and a red color means the MSE of themethod is larger (worse) than the MSE of the naive method. Darker blue means more advantage over the naive method,while darker red means more disadvantage than the naive method. It is noteworthy that ARGOX with all blue colors is4
PREPRINT - J
UNE
5, 2020
ALAK AZ ARCA CO CTDEDCFLGAHIID IL INIAKS KYLA MEMD MAMIMN MSMOMT NENV NHNJNM NYNCND OHOKOR PA RISCSD TNTXUT VTVAWA WVWIWY
Relative MSE NA ARGOX
ALAK AZ ARCA CO CTDEDCFLGAHIID IL INIAKS KYLA MEMD MAMIMN MSMOMT NENV NHNJNM NYNCND OHOKOR PA RISCSD TNTXUT VTVAWA WVWIWY
VAR
ALAK AZ ARCA CO CTDEDCFLGAHIID IL INIAKS KYLA MEMD MAMIMN MSMOMT NENV NHNJNM NYNCND OHOKOR PA RISCSD TNTXUT VTVAWA WVWIWY
GFT
ALAK AZ ARCA CO CTDEDCFLGAHIID IL INIAKS KYLA MEMD MAMIMN MSMOMT NENV NHNJNM NYNCND OHOKOR PA RISCSD TNTXUT VTVAWA WVWIWY
Lu et al. (2019)
Figure 1: State-by-state Heatmap of Relative Mean Squared Error of ARGOX, VAR, GFT, and Lu et al.(2019) [20] to the naive method. The relative MSE is the ratio of the MSE of a given method to that of thenaive method. Blue color means smaller MSE (i.e., better performance) than the naive method; red colormeans larger MSE (i.e., worse performance ) than the naive method; grey color means result not available.ARGOX with all blue colors uniformly dominates the naive method, while mixed colors in the rest of theplots show that VAR, GFT, and Lu et al. (2019) were worse than the naive method in a large proportion ofstates. ARGOX and VAR are evaluated for the whole period of Oct 11, 2014 to March 21, 2020; GFT isevaluated for the period of Oct 11, 2014 to August 15, 2015 due to GFT data availability; Lu et al. (2019) isevaluate from Oct 11, 2014 to May 14, 2017 due to its availability.the only method that gives uniformly better performance than the native method across all states. All other methods incomparison fail to do so for a large portion of the states investigated. Note that the naive method provides a model-freebaseline benchmark that solely relies on information from CDC’s flu reports. Therefore, ARGOX is the only methodthat effectively utilizes the Internet data to uniformly improve flu tracking from the traditional surveillance system,indicating ARGOX’s reliability and adaptability. With its universally enhanced accuracy over the alternative methodsfor real-time state-level flu situation estimate, it appears that ARGOX could help timely, proper public health decisionmaking for the local control of the disease.Detailed numerical results for each state and for each flu season are reported in Tables S5-S55 and the figures in SI,where ARGOX holds lead over other methods in the vast majority of the cases, further revealing its robustness overgeographical and seasonal variability in flu epidemics.In addition to the point estimate, ARGOX also provides 95% confidence intervals for each week’s estimates. For theentire period from 2014 to 2020, over all 51 states/district/city, the intervals provided by ARGOX successfully coverthe actual %ILI in 92.5% of the cases (Table S1), which is close to the nominal 95%, demonstrating ARGOX’s accurateuncertainty quantification.
Discussion
ARGOX effectively combines state-, regional-, and national-level publicly available data from Google searches andCDC’s traditional flu surveillance system. It incorporates geographical and temporal correlation of flu activities to5
PREPRINT - J
UNE
5, 2020provide accurate, reliable real-time flu tracking at the state level. Across all the available states, ARGOX outperformstime-series-based benchmark models, GFT, and the method of Lu et al. (2019). ARGOX’s weekly %ILI estimationsare accompanied by reliable interval estimates as a measure for uncertainty. The state-level real-time tracking of fluepidemics by ARGOX could help public health officials be aware of potential epidemic hot spots and thereby optimizeresource allocation across the nation.ARGOX’s adaptive pooling of the most-relevant information among the 51 US states/district/city plays an importantrole in its performance. To avoid the possibility of overfitting, a structured covariance matrix on the %ILI increments isutilized. Such structured dynamic modeling of the cross-state covariance serves to capture the ever-changing geographicspread pattern of the flu. It aggregates state-to-state, time-varying connectivity factors such as commuting traffic,airline frequency, geographic proximity, and climatic patterns. The utilization of cross-state correlation also helps poolinformation from different states, regions and the entire nation in addition to the information at a given state. Thepooling from national and regional level estimates incorporates the shared seasonality component in flu trends across allthe states, which further helps reduce the risk of overfitting.The two-step design of ARGOX has broad applicability. The first step could be substituted by other models or includeother data sources, while the second step remains adaptable for multi-resolution spatial-temporal boosting. A widespectrum of flu estimation models, including susceptible-infectious-recovered-susceptible model [6], empirical Bayesmethod [15], Wisdom-of-crowds forecast [16], or ensemble of them [23] can be fitted into the cross-state boosting step(the second step) of ARGOX.Like all big-data-based models, our result has certain limitations. ARGOX’s accuracy depends on the reliability of itsinputs – Google Trends data and historical %ILI data from CDC. Google Trends data have increasing amount of missingdata and zero counts as the resolution goes from national to regional and state levels (Table S3). Such degeneracy indata quality is a challenge for high-resolution inference. Google search information could also be sensitive to mediacoverage [24, 25]. Fortunately, the L penalty and the dynamic training of ARGOX effectively addressed the sparsityand over-shooting problem of Google data. In addition, we should be aware that our estimation target, the CDC’s%ILI, is only a proxy for the true flu incidence in the population, as it’s calculated from a sample of outpatient visitswith influenza-like symptoms. The reported %ILI at the state level could have (1) high noise due to its limited samplesize, (2) subsequent revision when healthcare providers update their information, and (3) bias towards those with easyhealthcare access. Nevertheless, accurate estimation of CDC’s %ILI at the state level is valuable for optimizing resourceallocations. More detailed discussion about the importance of alternative indicators for flu incidence in the populationcan be found in [26, 27, 28].ARGOX is accurate, reliable, flexible and generalizable, making it adaptable to other spatial and temporal resolutionsfor tracking or forecasting other diseases and social/economic events that leave traces on people’s Internet activityrecords. The ARGOX framework can be potentially adapted for COVID-19 tracking by incorporating additionalcoronavirus-related query terms at city, state, regional, and national level [29]. With the current development ofCOVID-19 pandemic, it is likely that the coronavirus would come back in the winter of 2020/2021. In light of this,accurate localized tracking of epidemic activity has become more important than ever before. Materials and Methods
CDC’s ILINet data
Every Friday, CDC releases a report of %ILI for the previous week, which gives the percent of outpatient visits withinfluenza-like illness for the whole nation, each HHS region, each state (except Florida), Washington DC, and New YorkCity (separated from New York State) ( ). CDC also revises theinitial report numbers in the subsequent weeks when more information become available ( gis.cdc.gov/grasp/fluview/fluportaldashboard.html ). Consequently, CDC’s %ILI data lag behind real-time for up to 2 weeks andare less accurate for more recent weeks. CDC’s %ILI data for this study were downloaded on Mar 27, 2020.
Google Data
The Internet search volume data from Google are publicly available through Google Trends ( trends.google.com ). Auser can specify the desired query term, geographical location, and time frame on Google Trends; the website then willreturn a (weekly) time series in integer values from 0 to 100, which corresponds to the normalized search volume of thequery term within the specified time frame, where 100 represents the historical maximum, and 0 represents missing datadue to inadequate search intensity. This integer-valued time series from Google Trends is based on sampling Google’sraw search logs. 6
PREPRINT - J
UNE
5, 2020The search query terms that we use are based on previous work for national and regional flu estimation [8, 21]. We alsoincluded several additional queries and topics in this study, which were obtained from “Related queries” and “Relatedtopics” on the Google Trends website when searching for flu related information. Table S2 in the SupplementaryMaterial lists these search terms.As one benchmark, we downloaded the discontinued Google Flu Trends (GFT) data ( ). GFT has national, regional, and state-level prediction for the weekly%ILI from Jan 1, 2004 to August 9, 2015.
Regional-Enrichment of state-level Google search data
Google Trends provides (normalized) search volume data at both national and state levels. However, for the state-leveldata, there is a high level of sparsity (i.e., zero observations) among the returned integer-valued time series (see TableS3). These zeros, which correspond to missing data due to inadequate search intensity, significantly lower the dataquality at the state level (compared to the national level), which in turn severely reduces the prediction accuracy at thestate level. To enhance the predictive power of state-level Google data, we use a simple approach to borrow informationfrom the regional level. First, we reconstruct regional-level search frequency for each region in the US by weightingthe state-level search frequencies within a given region, where the weights are proportional to the state’s population.Second, instead of using the state-level Google Trends time-series, for each search term, we use a weighted averageof the state-level search frequency (2/3 weight) and the regional-level search frequency (1/3 weight) as the input forstate-level %ILI estimation. We carry out this regional-enrichment process for all states/district/city, except seven states– Hawaii (HI), Alaska (AK), Vermont (VT), Montana (MT), North Dakota (ND), Maine (ME), and South Dakota (SD) –because these seven states are modeled with a separate stand-alone model (as detailed in the following sections). Forthese seven states, the raw Google Trends state-level times series, not the regional-enriched time series, are used asinput.
Evaluation metrics
We use three metrics to evaluate the accuracy of an estimate against the actual %ILI released by CDC: the mean squarederror (MSE), the mean absolute error (MAE), and the Pearson correlation (Correlation). MSE between an estimate ˆ p t and the true value p t over period t = 1 , . . . , T is T (cid:80) Tt =1 (ˆ p t − p t ) . MAE between an estimate ˆ p t and the truevalue p t over period t = 1 , . . . , T is T (cid:80) Tt =1 | ˆ p t − p t | . Correlation is the Pearson correlation coefficient between ˆ p = (ˆ p , . . . , ˆ p T ) and p = ( p , . . . , p T ) . Prediction model of ARGOX
ARGOX operates in two steps: the first step extracts Internet search information at the state level, and the second stepenhances the estimates using cross-state and cross-resolution information.At the second step, we take a dichotomous approach for the 51 US states/district/city (50 states except Florida, whichdoes not have %ILI data, plus Washington DC and New York City). We set apart seven states: HI, AL, VT, MT, ND,ME, and SD. The first two (HI and AL) are geographically separated from the contiguous US. The last five (VT, MT,ND, ME, and SD) are the states that have the lowest multiple correlations (a.k.a. the R ) in %ILI to the %ILI of theentire nation, the %ILI of the other states, and the %ILI of the other regions (detailed calculation method is given inSupplementary Material). A low multiple correlation of a state implies that the state’s flu activity is not well correlatedwith other states’ or other regions’. For these seven states, due to either the geological discontinuity or the low multiplecorrelation, it is not clear if using information cross the other states or other regions can help the state-level %ILIestimation. Therefore, we adopt the dichotomous approach: For the 44 states/district/city (the vast majority), we applya joint estimation approach at the second step to enhance the state-level %ILI estimation by using all information,including information from other states and other regions; for the above-mentioned seven states, we use a stand-aloneestimation approach at the second step to enhance the %ILI estimation (not using information from other states andregions). The two steps of ARGOX are detailed below. First step: extracting Internet search information at the state level
This step concerns extracting Google search information at each state. In particular, for a given state/district/city m , m = 1 , . . . , , let X i,t,m be the logarithm of 1 plus the state-level Google Trends data of search term i at week t (note: 1 is added to each state-level Google Trends data point to avoid taking logarithm of zero); let y t,m be thelogit-transformation of CDC’s %ILI at time t for state m . To estimate y T,m , an L regularized linear estimator is used7 PREPRINT - J
UNE
5, 2020in the first step based on the vector X T,m = ( x i,T,m ) : ˆ y T,m = ˆ β ,m + X (cid:124) T,m ˆ β m , where the coefficients ( ˆ β ,m , ˆ β m ) are obtained via arg min β ,m , β m T − (cid:88) t = T − N (cid:16) y t,m − β ,m − X (cid:124) T,m β m (cid:17) + λ (cid:107) β m (cid:107) . (1)We set N = 104 , i.e., a two-year window, as recommended in previous studies [8, 21, 22]. We set λ throughcross-validation.In addition, we obtain an accurate estimate ˆ p natT for the national %ILI by using the ARGO method [8], which usesnational level Google search data. We also obtain an estimate (ˆ p regT, , . . . , ˆ p regT, ) for the ten HHS regional %ILI by theARGO2 method [21], which uses aggregated regional level Google search data. Second step: joint model for the 44 states/district/city other than HI, AK, ND, VT, MT, ME, and SD
For the 44 states, let p t = ( p t, , . . . , p t, ) (cid:124) denote CDC’s %ILI at the state level; they are related to y t,m through p t,m = exp( y t,m ) / (1 + exp( y t,m )) . Our raw estimate for p t from the first step is ˆ p GTt = (ˆ p t, , . . . , ˆ p t, ) (cid:124) , where ˆ p t,m = exp(ˆ y t,m ) / (1 + exp(ˆ y t,m )) . Our estimate of the national %ILI from the first step is ˆ p natt . Let the boldface ˆ p natt denote the length-44 vector ˆ p natt = (ˆ p natt , . . . , ˆ p natt ) (cid:124) . We also have the regional %ILI estimate (ˆ p regt, , . . . , ˆ p regt, ) from the first step. Let ˆ p regt denote the length-44 vector ˆ p regt = (ˆ p regt,r , . . . , ˆ p regt,r ) (cid:124) , where r m is the region number forstate m .Estimating p t is equivalent to estimating the time series increment ∆ p t = p t − p t − . We denote Z t = ∆ p t fornotational simplicity. For the estimation of Z t , we want to incorporate the cross-state, cross-source correlations.We have four predictors for Z t after the first step: (i) Z t − = ∆ p t − , (ii) ˆ p GTt − p t − , (iii) ˆ p regt − p t − , and (iv) ˆ p natt − p t − ; they represent time series information, information from the state level Google search, information fromthe regional level estimation, and information from the national level estimation, respectively. Let W t denote thecollection of these four vectors W t = ( Z (cid:124) t − , ( ˆ p GTt − p t − ) (cid:124) , ( ˆ p regt − p t − ) (cid:124) , ( ˆ p natt − p t − ) (cid:124) ) (cid:124) .To combine the four predictors, we use the best linear predictor formed by them: ˆ Z t = µ Z + Σ ZW Σ − W W ( W t − µ W ) , (2)where µ Z and µ W are the mean vectors of Z and W respectively, and Σ ZZ , Σ ZW , and Σ W W are the covariancematrices of and between Z and W . The best linear predictor gives the optimal way to linearly combine the fourpredictors to form a new one. The variance of ˆ Z t is Var( ˆ Z t | W t ) = Σ ZZ − Σ ZW Σ − W W Σ W Z . (3)Consistent with the first step, we adopt a sliding two-year training window to estimate µ Z , µ W , Σ ZZ , Σ ZW , and Σ W W in Eq. (2) and (3). For µ Z and µ W , we use the empirical mean of the corresponding variables as the estimates. However,for the covariance matrices, due to their large sizes and the small number of observations, we need to structure thecovariance matrices for reliable estimation.We assume the following structure:1. The covariances between the time series increments satisfy Var( Z t ) = Var( Z t − ) = Σ ZZ and Cov( Z t , Z t − ) = ρ Σ ZZ , where < ρ < . This essentially assumes that the time series incrementsare stationary and have a stable autocorrelation across time and states.2. Independence among the different sources of information: time series increment, the estimation error of thefirst-step state-level estimate, the estimation error of the regional estimate, and the estimation error of thenational estimate, i.e., Z t , ˆ p GTt − p t , ˆ p regt − p t , ˆ p natt − p t are all mutually independent.The covariance matrices are thereby simplified as: Σ ZW = ( ρ Σ ZZ Σ ZZ Σ ZZ Σ ZZ ) (4) Σ W W = Σ ZZ ρ Σ ZZ ρ Σ ZZ ρ Σ ZZ ρ Σ ZZ Σ ZZ + Σ GT Σ ZZ Σ ZZ ρ Σ ZZ Σ ZZ Σ ZZ + Σ reg Σ ZZ ρ Σ ZZ Σ ZZ Σ ZZ Σ ZZ + Σ nat (5)8 PREPRINT - J
UNE
5, 2020where Σ reg = Var( ˆ p regt − p t ) , Σ nat = Var( ˆ p natt − p t ) , and Σ GT = Var( ˆ p GTt − p t ) . To further control theestimation stability, we incorporate a ridge-regression-inspired shrinkage [30] to the linear predictor (2), replacing thejoint covariance matrix of ( Z (cid:124) t , W (cid:124) t ) (cid:124) by the average of the structured covariance matrix and its empirical diagonal.Effectively, in Eq. (2), Σ ZW is replaced by Σ ZW , and Σ W W is replaced by ( Σ W W + D W W ) , where D W W is thediagonal of the empirical covariance of W t : ˆ Z t = µ Z + 12 Σ ZW ( 12 Σ W W + 12 D W W ) − ( W t − µ W ) . (6) Σ ZZ , Σ nat , Σ reg , Σ GT and D W W are estimated by the corresponding sample covariance from the data in the mostrecent 2-year training window; ρ is estimated by minimizing the Frobenius norm ( L distance) between the empiricalcorrelation and structured correlation. Based on Eq. (3), the variance estimate is similarly updated by Var( ˆ Z t | W t ) = Σ ZZ −
12 Σ ZW ( 12 Σ W W + 12 D W W ) −
12 Σ
W Z . Our final state-level %ILI estimate for week T after the second step is: ˆ p T = p T − + ˆ µ Z + ˆΣ ZW ( ˆΣ W W + ˆ D W W ) − ( W T − ˆ µ W ) , (7)with corresponding 95% interval estimate (cid:34) ˆ p T ± . · (cid:115) diagonal (cid:18) ˆΣ ZZ −
12 ˆΣ ZW ( ˆΣ W W + ˆ D W W ) − ˆΣ W Z (cid:19)(cid:35) . Second step: stand-alone model for HI, AK, ND, VT, MT, ME and SD
For m ∈ { HI, AK, ND, VT, MT, ME, SD } , we take a stand-alone modeling approach. For each of these states, whichis either non-contiguous or has the lowest multiple correlation with out-of-state %ILI (detailed in SupplementaryMaterial), we focus on estimating the individual state’s %ILI by integrating the within-state and national information inthe second step. Thereby, our target is a scalar Z ( m ) t = p t,m − p t − ,m , the state’s %ILI increment at the current week.The predictor vector in the second step for state m is W ( m ) t = ( Z ( m ) t − , (ˆ p GTt,m − p t − ,m ) , (ˆ p natt − p t − ,m )) , where theregional terms are dropped. The best linear predictor with ridge-regression inspired shrinkage is then used to get thefinal estimate ˆ Z ( m ) t = µ ( m ) Z + 12 Σ ( m ) ZW ( 12 Σ ( m ) W W + 12 D ( m ) W W ) − ( W ( m ) t − µ ( m ) W ) . (8)The corresponding covariance matrices between the components Σ ( m ) ZW = Cov( Z ( m ) , W ( m ) ) , Σ ( m ) W W = Var( W ( m ) ) ,and D ( m ) W W = diagonal(Σ ( m ) W W ) are estimated by the corresponding sample covariance from the data in the most recent2-year training window.The final state-level %ILI estimate for week T after the second step for m ∈ { HI, AK, ND, VT, MT, ME, SD } is: ˆ p T,m = p T − ,m + ˆ µ ( m ) Z + ˆΣ ( m ) ZW ( ˆΣ ( m ) W W + ˆ D ( m ) W W ) − ( W ( m ) t − ˆ µ ( m ) W ) , (9)with corresponding 95% interval estimate (cid:34) ˆ p T,m ± . · (cid:114) ˆΣ ( m ) ZZ −
12 ˆΣ ( m ) ZW ( ˆΣ ( m ) W W + ˆ D ( m ) W W ) − ˆΣ ( m ) W Z (cid:35) , where Σ ( m ) ZZ = Var( Z ( m ) ) is the scalar variance of the univariate time series Z ( m ) t . Availability of data and material
All analyses were performed with the R statistical software [31]. The R package that implements the ARGOX methodis available on CRAN at https://cran.r-project.org/web/packages/argo/ , which uses the glmnet package[32]. All datasets analyzed in the current study are available in the Harvard Dataverse repository, doi:XXX/XXX/XXXX.
Acknowledgements
SCK’s research was supported in part by National Science Foundation grant DMS-1810914. The authors thank ProfessorHerman Chernoff for helpful comments. 9
PREPRINT - J
UNE
5, 2020
References [1] US Centers for Disease Control and Prevention (CDC) (2020) Past seasons estimated influenza disease burden( ). Accessed: 2020-05-07.[2] Ginsberg J, et al. (2009) Detecting influenza epidemics using search engine query data.
Nature
International Journal ofMathematical Modelling and Numerical Optimisation
Economic Analysisof the Digital Economy , eds. Goldfarb A, Greenstein SM, Tucker CE. (University of Chicago Press), pp. 119–135.[5] Wu L, Brynjolfsson E (2015) The future of prediction: how Google searches foreshadow housing prices and salesin
Economic Analysis of the Digital Economy , eds. Avi Goldfarb SG, Tucker C. (University of Chicago Press), pp.89–118.[6] Shaman J, Karspeck A (2012) Forecasting seasonal outbreaks of influenza.
Proceedings of the National Academyof Sciences ). Accessed: 2020-04-12.[8] Yang S, Santillana M, Kou SC (2015) Accurate estimation of influenza epidemics using google search data viaargo.
Proceedings of the National Academy of Sciences
BMC Infectious Diseases
Proceedings of the National Academy of Sciences
Nature Communications
PLoS Comput Biol
PLoS currents outbreaks . doi:10.1371/currents.outbreaks.8a6a3df285af7ca973fab4b22e10911e.[14] (2020) Flusight: Flu forecasting | CDC ( ). Ac-cessed: 2020-04-12.[15] Brooks LC, Farrow DC, Hyun S, Tibshirani RJ, Rosenfeld R (2015) Flexible modeling of epidemics with anempirical Bayes framework.
PLoS Comput Biol
PLoS Comput. Biol.
PLoS Computational Biology
Scientific Reports
Proceedings ofthe 2018 World Wide Web Conference . pp. 87–96.[20] Lu FS, Hattab MW, Clemente CL, Biggerstaff M, Santillana M (2019) Improved state-level influenza nowcastingin the united states leveraging internet-based data and network approaches.
Nature communications
Scientificreports
Statistics inMedicine
PLoS Computational Biology
Science
Nature
PREPRINT - J
UNE
5, 2020[26] Lipsitch M, et al. (2011) Improving the evidence base for decision making during a pandemic: the example of2009 influenza A/H1N1.
Biosecurity and bioterrorism: biodefense strategy, practice, and science
Influenza and other respiratory viruses
PloS One ). Accessed: 2020-05-07.[30] Hoerl AE, Kennard RW (1970) Ridge regression: Biased estimation for nonorthogonal problems.
Technometrics
R: A Language and Environment for Statistical Computing (R Foundation for StatisticalComputing, Vienna, Austria).[32] Friedman J, Hastie T, Tibshirani R (2010) Regularization paths for generalized linear models via coordinatedescent.
Journal of Statistical Software
PREPRINT - J
UNE
5, 2020
Supplementary Material
This Supplementary Material is organized as following: (1) the detailed calculation procedure for the multiple correlationbehind the stand-alone modeling of HI, AK, ND, VT, MT, ME, and SD is presented; (2) a table for the confidenceinterval coverage is presented; (3) all the Google query terms used in this study are listed; (4) Google Trends dataquality at different geographic area is studied; (5) full comparison to another Google-search-based benchmark methodis presented; (6) detailed estimation results for each of 51 studied states/district/city are reported in tables and plotted infigures.
Multiple correlation
For each state, the multiple correlation of its flu activity level to the other states’, other regions’ and the national fluactivity levels is calculated as follows. First, the states of HI and AK are excluded because they are not part of thecontiguous US; the state of FL is excluded because FL data is not available from CDC. Then for the in-sample timeperiod of 2010-10-09 to 2014-09-27, we regress each state’s %ILI to (i) all other 48 states’ %ILI (including DC andNYC but excluding FL, HI, and AK), (ii) all the other 9 regions’ %ILI (i.e., regions other than the one that the specificstate belongs to), and (iii) the national %ILI. After the regression, we obtain the R-squared, which is the square ofmultiple correlation. The five states with the lowest multiple correlations are ND, VT, MT, ME, and SD. We, therefore,would not use spatial pooling on HI, AK, ND, VT, MT, ME, and SD. Instead, we only use the state-specific datatogether with national level data for cross-resolution boosting on those seven aforementioned states in the second stepof ARGOX.
Confidence interval coverage
We study the goodness of our confidence intervals by examining its actual coverage (coverage of the actual %ILIreleased by CDC weeks later). The result is shown in Table S1. In general, the coverage of 95% confidence interval isquite close to the nominal value, suggesting that our model quantifies the uncertainty reasonably well.
AL AK AZ AR CA CO CT DE DC GA0.919 0.916 0.909 0.930 0.944 0.905 0.909 0.930 0.923 0.926HI ID IL IN IA KS KY LA ME MD0.947 0.958 0.930 0.926 0.926 0.940 0.867 0.916 0.916 0.947MA MI MN MS MO MT NE NV NH NJ0.937 0.947 0.940 0.930 0.923 0.937 0.909 0.916 0.898 0.930NM NY NC ND OH OK OR PA RI SC0.933 0.902 0.951 0.944 0.926 0.937 0.926 0.930 0.909 0.926SD TN TX UT VT VA WA WV WI WY0.940 0.919 0.926 0.898 0.947 0.951 0.905 0.909 0.937 0.874NYC0.912
Table S1: The actual coverage of the confidence intervals by ARGOX for 51 states/district/city. The coverage is for95% nominal confidence level. The average coverage over all the 51 states/district/city is 92.5%.12
PREPRINT - J
UNE
5, 2020
Query terms for Google Trends
Table S2 lists all query terms/phrases used in this study. Most of them are taken from previous studies [8, 21] with afew additional terms identified through “Related topics” and “Related queries” from Google Trends when search forflu-related information.Table S2: All search query terms used in this study. The last 21 terms separated by a horizontal line from the first 140terms are new “Related topics” and “Related queries” identified from Google Trends. flu incubation flu incubation period influenza type a symptoms of the fluflu symptoms influenza symptoms flu contagious influenza aa influenza symptoms of flu flu duration influenza incubationtype a influenza flu treatment symptoms of influenza influenza contagiousflu in children cold or flu symptoms of bronchitis flu recoverytessalon influenza incubation period symptoms of pneumonia tussionexsigns of the flu flu treatments remedies for the flu walking pneumoniaflu test tussin upper respiratory respiratory fluacute bronchitis bronchitis sinus infections flu reliefpainful cough how long does the flu last flu cough sinusexpectorant strep strep throat influenza treatmentflu reports flu remedy robitussin rapid flutreatment for the flu chest cold cough fever oscillococcinumflu fever treat the flu how to treat the flu over the counter fluhow long is the flu flu medicine flu or cold normal bodyis flu contagious treat flu body temperature reduce feverflu vs cold how long is the flu contagious fever reducer get over the flutreating flu having the flu treatment for flu human temperaturedangerous fever the flu remedies for flu influenza a and bcontagious flu fever flu flu remedies how long is flu contagiouscold vs flu braun thermoscan fever cough signs of fluhow long does flu last normal body temperature get rid of the flu i have the flutaking temperature flu versus cold how long flu flu germsflu and cold thermoscan flu complications high feverflu children the flu virus how to treat flu pneumoniaflu headache ear thermometer how to get rid of the flu flu how longcold and flu over the counter flu medicine treating the flu flu carehow long contagious fight the flu reduce a fever cure the flumedicine for flu flu length cure flu exposed to flulow body early flu symptoms flu report incubation period for flubreak a fever flu contagious period cold versus flu what to do if you have the flumedicine for the flu flu and fever flu lasts incubation period for the fludo i have the flu type a flu symptoms flu texas how long am i contagious with the fluhow to break a fever fever breaks type a flu how to bring a fever downhow to treat the flu at home flu how long are you contagious flu a symptoms fluInfluenza vaccine Influenza Fever Influenza A virusInfluenza B virus Common cold Cough Sore throatVirus Avian influenza Spanish flu HeadacheNausea Flu season Oseltamivir Nasal congestionCanine influenza Rapid influenza diagnostic test Theraflu DextromethorphanRhinorrhea PREPRINT - J
UNE
5, 2020
Google Trends data quality
As stated at trends.google.com , the numbers in Google Trends “represent search interest relative to the highest pointon the chart for the given region and time; a value of 100 is the peak popularity for the term; a value of 50 means thatthe term is half as popular; a score of 0 means there was not enough data for this term.” As such, the proportion ofzeros in the Google Trends data reflects the data quality: higher proportion of zeros indicates lower quality of GoogleTrends data. Table S3 summarizes the average proportion of zeros for the query terms listed in Table S2 in each ofthe geographic areas. As we can see, Google Trends data at the US national level have far fewer zeros than any of thestates, implying a significant drop in quality from national-level data to state-level data.Table S3: Average proportion of zeros in Google Trends data for the query terms in Table S2. Higher proportion ofzeros indicates lower quality of Google Trends data since “a score of 0 means there was not enough data for this term”( trends.google.com ). The proportion of zeros in Google Trends at the US national level is in the upper sub-table,while the proportions of zeros at state/district/city level are in the lower sub-table.
US Nationalproportion of zeros 1.37%AK AL AR AZ CA CO CT DC DE FL78.33% 46.13% 56.16% 39.53% 13.52% 42.77% 50.99% 63.00% 73.34% 20.93%GA HI IA ID IL IN KS KY LA MA29.86% 67.14% 54.90% 64.58% 26.39% 40.78% 55.50% 47.06% 48.70% 37.68%MD ME MI MN MO MS MT NC ND NE41.26% 67.47% 31.78% 41.99% 40.67% 56.99% 73.07% 30.69% 75.61% 59.34%NH NJ NM NV NY OH OK OR PA RI67.21% 34.86% 63.73% 55.14% 18.92% 30.64% 50.78% 48.99% 30.17% 69.28%SC SD TN TX UT VA VT WA WI WV45.97% 74.43% 38.48% 16.77% 52.78% 32.98% 77.42% 37.85% 42.78% 64.07%WY NYC79.82% 18.25% PREPRINT - J
UNE
5, 2020
More comparison with the result of Lu et al. (2019)
Lu et al. (2019) [20] proposed another Google-search-based method for state-level influenza tracking, utilizing anetwork approach. We compare ARGOX with Lu et al. (2019) together with other methods here. The retrospectiveresults of Lu et al. (2019) are available for seasons 2014-15, 2015-16, and 2016-17, and it only studied 37 selectedstates, which are: AK, AL, AR, AZ, DE, GA, ID, KS, KY, LA, MA, MD, ME, MI, MN, NC, ND, NE, NH, NJ, NM,NV, NY, OH, OR, PA, RI, SC, SD, TN, TX, UT, VA, VT, WA, WI, WV. For completeness, ARGOX, VAR, GFT, andthe naive method are compared here for the same period and for the same 37 states. Overall ARGOX takes the lead inthis subset of 37 states for all three seasons of 2014-15, 2015-16 and 2016-17.Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17MSEARGOX
VAR 0.873 1.234 0.503 1.214GFT – 1.464 – –Lu et al. (2019) 0.418 0.467 0.528 0.544naive 0.383 0.618 0.201 0.471CorrelationARGOX
VAR 0.808 0.809 0.684 0.754GFT – – –Lu et al. (2019) 0.912 0.912 0.808 0.858naive 0.894 0.880 0.806 0.860Table S4: Comparison of different methods for state-level %ILI estimation, averaging over the 37 states, for the periodof 2014 to 2017, due to the availability of Lu et al. (2019). The MSE and Correlation are reported. The method with thebest performance is highlighted in boldface for each metric in each period. Methods considered here include ARGOX,VAR, GFT, Lu et al. (2019), and the naive method. 15
PREPRINT - J
UNE
5, 2020
Detailed estimation results for each state/district/city
Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 1.214 1.378 2.617 0.551 1.226 1.611 1.650 1.553GFT – – 7.441 – – – – –Lu et al. (2019) – 4.875
VAR 0.707 0.801 1.055 0.629 0.861 0.780 0.788 0.912GFT – – 1.750 – – – – –Lu et al. (2019) – – – – – – – –naive 0.593 0.709 1.200
VAR 0.929 0.912 0.924 0.844 0.883 0.959 0.906 0.846GFT – – – – – – –Lu et al. (2019) – 0.885 0.924 0.683 – – –naive 0.923 0.888 0.867 − Jan 2018 − Jan 2020 − Jan % I L I US.AL −
505 2016 − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S1: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Alabama (AL).16
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 4.902 4.659 6.417 4.153 5.154 10.475 2.379 8.727GFT – – 0.918 – – – – –Lu et al. (2019) – 0.510 0.450
VAR 1.338 1.299 1.332 1.331 1.546 2.094 1.128 1.768GFT – – 0.780 – – – – –Lu et al. (2019) – – – – – – – –naive 0.619 0.510 0.461 0.596 0.604 0.504 0.821 1.326CorrelationARGOX
VAR 0.593 0.291 0.166 0.363 0.227 0.380 0.754 0.457GFT – – 0.638 – – – – –Lu et al. (2019) – 0.781 0.723 0.815 0.750 – – –naive 0.865 0.797 − Jan 2018 − Jan 2020 − Jan % I L I US.AK − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S2: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Alaska (AK).17
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.663 0.188 0.062 0.270 0.362 3.360 0.262 0.781GFT – – 1.213 – – – – –Lu et al. (2019) – 0.126
VAR 0.459 0.319 0.199 0.376 0.497 0.999 0.403 0.543GFT – – 0.940 – – – – –Lu et al. (2019) – – – – – – – –naive 0.297 0.230 0.198 0.293
VAR 0.835 0.923 – – –naive 0.941 0.956 0.949 0.939 0.865 0.886 0.901 0.810Table S7: Comparison of different methods for state-level %ILI estimation in Arizona (AZ). The MSE, MAE, andcorrelation are reported. The method with the best performance is highlighted in boldface for each metric in each period. − Jan 2018 − Jan 2020 − Jan % I L I US.AZ − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S3: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Arizona (AZ).18
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 2.487 1.529 2.202 0.879 2.109 9.018 2.619 3.827GFT – – 1.820 – – – – –Lu et al. (2019) – 0.679 1.325 0.341 – – – –naive 1.201 0.922 1.570 0.418 1.166 2.024 0.974 4.553MAEARGOX
GFT – – 1.153 – – – – –Lu et al. (2019) – – – – – – – –naive 0.647 0.548 0.677 0.472 0.708 1.016 0.729 1.523CorrelationARGOX
VAR 0.856 0.860 0.907 0.685 0.783 0.842 0.884 0.633GFT – – – – – – –Lu et al. (2019) – 0.928 0.909 0.875 – – – –naive 0.906 0.906 0.889 0.866 0.893 0.920 0.903 0.620Table S8: Comparison of different methods for state-level %ILI estimation in Arkansas (AR). The MSE, MAE, andcorrelation are reported. The method with the best performance is highlighted in boldface for each metric in each period. − Jan 2018 − Jan 2020 − Jan % I L I US.AR − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S4: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Arkansas (AR).19
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.169 0.162 0.172 0.190 0.190 0.409 0.093 0.257GFT – – 0.266 – – – – –Lu et al. (2019) – – – – – – – –naive 0.123 0.092 0.124 0.125
VAR 0.288 0.284 0.272 0.320 0.310 0.457 0.255 0.379GFT – – 0.388 – – – – –Lu et al. (2019) – – – – – – – –naive 0.224 0.214 0.233 0.277 0.194 0.381 0.220 0.330CorrelationARGOX
VAR 0.941 0.938 0.949 0.908 0.872 0.926 0.924 0.931GFT – – 0.927 – – – – –Lu et al. (2019) – – – – – – – –naive 0.952 0.947 0.944 0.907 0.917 0.892 0.940 0.943Table S9: Comparison of different methods for state-level %ILI estimation in California (CA). The MSE, MAE, andcorrelation are reported. The method with the best performance is highlighted in boldface for each metric in each period. − Jan 2018 − Jan 2020 − Jan % I L I US.CA − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S5: Plots of the %ILI estimates (top) and the estimation errors (bottom) for California (CA).20
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.770 0.456 0.514 0.070 1.042 0.814 1.971 2.211GFT – – 0.585 – – – – –Lu et al. (2019) – – – – – – – –naive 0.225 0.201 0.204 0.043 0.470 0.071
VAR 0.531 0.374 0.363 0.204 0.676 0.578 1.062 1.085GFT – – 0.420 – – – – –Lu et al. (2019) – – – – – – – –naive 0.310 0.287 0.279 0.166 0.506 0.213
VAR 0.888 0.665 0.754 0.824 0.461 0.811 0.661 0.784GFT – – 0.850 – – – – –Lu et al. (2019) – – – – – – – –naive 0.964 0.832 0.908 0.885 0.374 0.961 0.904 0.930Table S10: Comparison of different methods for state-level %ILI estimation in Colorado (CO). The MSE, MAE, andcorrelation are reported. The method with the best performance is highlighted in boldface for each metric in each period. − Jan 2018 − Jan 2020 − Jan % I L I US.CO − − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S6: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Colorado (CO).21
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 3.626 5.482 15.079 1.046 0.512 3.095 1.515 3.967GFT – – 1.944 – – – – –Lu et al. (2019) – – – – – – – –naive 0.330 0.230
VAR 0.891 0.925 1.584 0.845 0.584 1.090 0.910 1.395GFT – – 0.999 – – – – –Lu et al. (2019) – – – – – – – –naive 0.375
VAR 0.699 0.443 0.222 0.674 0.855 0.785 0.767 0.822GFT – – 0.774 – – – – –Lu et al. (2019) – – – – – – – –naive 0.955 − Jan 2018 − Jan 2020 − Jan % I L I US.CT − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S7: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Connecticut (CT).22
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.644 0.313 0.495 0.116 0.486 2.306 0.239 2.213GFT – – 3.978 – – – – –Lu et al. (2019) – 0.163 0.377 0.073 0.053 – – –naive 0.211
VAR 0.359 0.277 0.333 0.230 0.388 0.673 0.340 0.994GFT – – 1.783 – – – – –Lu et al. (2019) – – – – – – – –naive 0.232
VAR 0.748 0.718 0.760 0.659 0.730 0.848 0.779 0.457GFT – – 0.747 – – – – –Lu et al. (2019) – 0.843 0.841 − Jan 2018 − Jan 2020 − Jan % I L I US.DE − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S8: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Delaware (DE).23
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 15.898 10.397 5.739 15.635 17.804 14.672 7.150 1.439GFT – – 25.894 – – – – –Lu et al. (2019) – – – – – – – –naive 2.081 2.907 3.124 2.787 4.811
VAR 2.126 2.249 1.741 2.723 3.370 2.132 1.775 0.921GFT – – 4.759 – – – – –Lu et al. (2019) – – – – – – – –naive
VAR 0.554 0.605 0.602 0.580 0.475 0.602 0.243 0.656GFT – – 0.728 – – – – –Lu et al. (2019) – – – – – – – –naive − Jan 2018 − Jan 2020 − Jan % I L I US.DC − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S9: Plots of the %ILI estimates (top) and the estimation errors (bottom) for District of Columbia (DC).24
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.715 0.416 0.493 0.469 0.445 1.595 0.986 2.239GFT – – 0.600 – – – – –Lu et al. (2019) – 0.283
VAR 0.527 0.454 0.451 0.484 0.518 0.834 0.666 0.954GFT – – 0.470 – – – – –Lu et al. (2019) – – – – – – – –naive 0.442 0.347 0.442 0.275 0.418 0.837 0.488 1.085CorrelationARGOX
VAR 0.943 0.889 0.908 0.352 0.870 0.956 0.846 0.869GFT – – 0.901 – – – – –Lu et al. (2019) – − Jan 2018 − Jan 2020 − Jan % I L I US.GA − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S10: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Georgia (GA).25
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 2.395 2.945 7.087 1.148 0.893 4.532 0.409 3.337GFT – – 15.289 – – – – –Lu et al. (2019) – – – – – – – –naive 0.880
VAR 0.989 1.093 1.805 0.813 0.765 1.354 0.513 1.420GFT – – 3.292 – – – – –Lu et al. (2019) – – – – – – – –naive 0.662 0.717
VAR 0.871 0.891 0.870 0.832 0.632 0.681 0.854 0.720GFT – – 0.899 – – – – –Lu et al. (2019) – – – – – – – –naive 0.933 − Jan 2018 − Jan 2020 − Jan % I L I US.HI − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S11: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Hawaii (HI).26
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX – – – –naive 0.322 0.369 0.675 0.221 0.228 0.398 0.673
MAEARGOX
CorrelationARGOX
VAR 0.744 0.687 0.732 0.383 0.274 0.816 – – – – –Lu et al. (2019) – – – – –naive 0.900 0.909 0.904 0.692 0.750 0.818 0.754 0.863Table S16: Comparison of different methods for state-level %ILI estimation in Idaho (ID). The MSE, MAE, andcorrelation are reported. The method with the best performance is highlighted in boldface for each metric in each period. − Jan 2018 − Jan 2020 − Jan % I L I US.ID − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S12: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Idaho (ID).27
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.203 0.157 0.259 0.068 0.146 0.327 0.122 0.829GFT – – 0.414 – – – – –Lu et al. (2019) – – – – – – – –naive 0.163 0.117 0.188 0.073 0.129 0.381 0.155 0.479MAEARGOX
VAR 0.289 0.285 0.357 0.203 0.292 0.365 0.228 0.680GFT – – 0.574 – – – – –Lu et al. (2019) – – – – – – – –naive 0.254 0.230 0.262 0.221 0.268 0.396 0.285 0.523CorrelationARGOX
VAR 0.952 0.941 0.933 0.956 0.931 0.955 0.913 0.905GFT – – 0.947 – – – – –Lu et al. (2019) – – – – – – – –naive 0.962 0.951 0.939 0.939 0.936 0.939 0.889 0.945Table S17: Comparison of different methods for state-level %ILI estimation in Illinois (IL). The MSE, MAE, andcorrelation are reported. The method with the best performance is highlighted in boldface for each metric in each period.
246 2016 − Jan 2018 − Jan 2020 − Jan % I L I US.IL − −
101 2016 − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S13: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Illinois (IL).28
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 1.266 0.761 1.410 0.463 0.682 3.456 1.959 2.641GFT – – – – – – –Lu et al. (2019) – – – – – – – –naive 0.537 0.515 0.751 0.315 0.689 0.928 0.613 1.094MAEARGOX
VAR 0.636 0.531 0.638 0.499 0.640 1.056 0.940 0.992GFT – – – – – – –Lu et al. (2019) – – – – – – – –naive 0.459 0.464 0.486 0.433 0.596 0.608 0.540 0.685CorrelationARGOX
VAR 0.857 0.869 0.907 0.707 0.857 0.884 0.668 0.609GFT – – – – – – –Lu et al. (2019) – – – – – – – –naive 0.914 0.877 0.885 0.776 0.865 0.925 0.857 0.840Table S18: Comparison of different methods for state-level %ILI estimation in Indiana (IN). The MSE, MAE, andcorrelation are reported. The method with the best performance is highlighted in boldface for each metric in each period. − Jan 2018 − Jan 2020 − Jan % I L I US.IN − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S14: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Indiana (IN).29
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.446 0.201
VAR 0.350 0.279 0.301 0.249 0.381 0.516 0.450 0.933GFT – – 1.237 – – – – –Lu et al. (2019) – – – – – – – –naive 0.265 0.250
VAR 0.825 0.787 − Jan 2018 − Jan 2020 − Jan % I L I US.IA
05 2016 − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S15: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Iowa (IA).30
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 1.489 0.909 1.504 0.616 1.045 2.389 0.866 7.667GFT – – 1.929 – – – – –Lu et al. (2019) – 0.328 0.607 0.118 – – –naive 0.461 0.492 0.719 0.089 0.942 1.013 0.307 0.642MAEARGOX
VAR 0.664 0.562 0.733 0.530 0.675 1.038 0.692 1.673GFT – – 0.983 – – – – –Lu et al. (2019) – – – – – – – –naive 0.405 0.397 0.481
VAR 0.919 0.918 0.919 0.697 0.919 0.954 0.871 0.699GFT – – 0.962 – – – – –Lu et al. (2019) – 0.965 0.960 0.860 – – –naive 0.965 0.948 0.945 0.874 0.929 0.965 0.937 0.959Table S20: Comparison of different methods for state-level %ILI estimation in Kansas (KS). The MSE, MAE, andcorrelation are reported. The method with the best performance is highlighted in boldface for each metric in each period. − Jan 2018 − Jan 2020 − Jan % I L I US.KS − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S16: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Kansas (KS).31
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 3.082 0.823 0.508 0.076 2.562 9.726 3.600 13.010GFT – – 7.601 – – – – –Lu et al. (2019) – 0.415 0.106 0.076 1.433 – – –naive 0.724 0.447 0.137 0.079 1.494 1.641 1.078 2.174MAEARGOX
VAR 0.777 0.408 0.288 0.194 1.014 1.715 1.293 2.222GFT – – 1.951 – – – – –Lu et al. (2019) – – – – – – – –naive 0.466 0.351 0.197 0.190 0.913 0.851 0.680 1.072CorrelationARGOX
VAR 0.879 0.876 0.905 0.922 0.825 0.899 0.857 0.617GFT – – – – – – –Lu et al. (2019) – 0.941 0.940 0.923 0.907 – – –naive 0.955 0.936 0.910 0.908 0.898 0.943 0.934 0.891Table S21: Comparison of different methods for state-level %ILI estimation in Kentucky (KY). The MSE, MAE, andcorrelation are reported. The method with the best performance is highlighted in boldface for each metric in each period. − Jan 2018 − Jan 2020 − Jan % I L I US.KY − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S17: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Kentucky (KY).32
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.554 0.310 0.654 0.194 0.115 0.574
VAR 0.483 0.366 0.541 0.336 0.260 0.562 0.622 1.375GFT – – 0.676 – – – – –Lu et al. (2019) – – – – – – – –naive 0.392 0.249 0.353 0.195 0.272 0.591 0.705 1.033CorrelationARGOX
VAR 0.960 0.942 0.947 0.711 0.951 0.972 – – – – –Lu et al. (2019) – 0.963 0.955 0.796 0.958 – – –naive 0.968 0.960 0.954 0.876 0.948 0.954 0.927 0.833Table S22: Comparison of different methods for state-level %ILI estimation in Louisiana (LA). The MSE, MAE, andcorrelation are reported. The method with the best performance is highlighted in boldface for each metric in each period. − Jan 2018 − Jan 2020 − Jan % I L I US.LA −
202 2016 − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S18: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Louisiana (LA).33
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.176 0.172 0.230 0.159 0.132 0.139 0.317 0.401GFT – – 0.497 – – – – –Lu et al. (2019) – – – –naive 0.097 0.105 0.118 0.141 0.088 0.062
VAR 0.313 0.318 0.368 0.301 0.277 0.303 0.460 0.520GFT – – 0.448 – – – – –Lu et al. (2019) – – – – – – – –naive 0.222 0.247 0.264 0.290 0.229 0.204 – – – – –Lu et al. (2019) – – – –naive 0.951 0.831 0.885 0.257 0.789 0.780 0.940
Table S23: Comparison of different methods for state-level %ILI estimation in Maine (ME). The MSE, MAE, andcorrelation are reported. The method with the best performance is highlighted in boldface for each metric in each period. − Jan 2018 − Jan 2020 − Jan % I L I US.ME − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S19: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Maine (ME).34
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.831 0.846 1.053 0.645 0.846 0.463 0.620 2.937GFT – – 1.504 – – – – –Lu et al. (2019) – 0.347 0.365 0.347 0.441 – – –naive 0.376 0.433 0.507 0.432 0.520 0.376 0.389 0.678MAEARGOX
VAR 0.595 0.639 0.640 0.597 0.661 0.503 0.567 1.131GFT – – 0.968 – – – – –Lu et al. (2019) – – – – – – – –naive 0.423 0.452 0.456 0.494 0.510 0.441 0.496 0.594CorrelationARGOX
VAR 0.837 0.653 0.741 0.585 0.788 0.905 0.788 0.819GFT – – 0.816 – – – – –Lu et al. (2019) – 0.824 − Jan 2018 − Jan 2020 − Jan % I L I US.MD − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S20: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Maryland (MD).35
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.155 0.101 0.133 0.076 0.137 0.315 0.160 0.510GFT – – 0.112 – – – – –Lu et al. (2019) – – – –naive 0.126 0.070 0.100 0.066 0.073 0.284 0.105 0.528MAEARGOX
VAR 0.266 0.234 0.263 0.219 0.283 0.393 0.293 0.540GFT – – 0.263 – – – – –Lu et al. (2019) – – – – – – – –naive 0.217 0.181 0.214 0.181 0.203 0.358 0.231 0.530CorrelationARGOX
VAR 0.938 0.874 0.869 0.845 0.818 0.927 0.884 0.928GFT – – 0.947 – – – – –Lu et al. (2019) – – – –naive 0.950 0.912 0.891 0.870 0.908 0.921 0.919 0.923Table S25: Comparison of different methods for state-level %ILI estimation in Massachusetts (MA). The MSE, MAE,and correlation are reported. The method with the best performance is highlighted in boldface for each metric in eachperiod. − Jan 2018 − Jan 2020 − Jan % I L I US.MA − −
101 2016 − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S21: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Massachusetts (MA).36
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.177 0.168 0.174 0.174 0.249 0.318 0.181 0.317GFT – – 1.313 – – – – –Lu et al. (2019) – 0.132 0.246 0.081 0.174 – – –naive 0.130 0.146 0.260 0.066 0.169 0.232 0.108 0.177MAEARGOX
VAR 0.292 0.293 0.284 0.309 0.403 0.413 0.314 0.380GFT – – 0.957 – – – – –Lu et al. (2019) – – – – – – – –naive 0.236 0.227 0.284 0.184 0.299 0.392 0.255 0.308CorrelationARGOX
VAR 0.934 0.928 0.938 0.905 0.874 0.940 0.795 0.871GFT – – – – – – –Lu et al. (2019) – 0.936 0.919 0.908 0.928 – – –naive 0.945 0.928 0.868 0.917 0.917 0.939 0.837 0.930Table S26: Comparison of different methods for state-level %ILI estimation in Michigan (MI). The MSE, MAE, andcorrelation are reported. The method with the best performance is highlighted in boldface for each metric in each period. − Jan 2018 − Jan 2020 − Jan % I L I US.MI − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S22: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Michigan (MI).37
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.953 1.049 1.552 0.634 1.414 0.387 1.047 2.487GFT – – 0.544 – – – – –Lu et al. (2019) – 0.287 0.472 0.370 0.215 – – –naive 0.499 0.443 0.722 0.510 0.277 0.247 0.289 2.395MAEARGOX
VAR 0.644 0.640 0.713 0.608 0.779 0.475 0.779 1.143GFT – – 0.527 – – – – –Lu et al. (2019) – – – – – – – –naive 0.441 0.440 0.535 0.544 0.378 0.388 0.402 0.965CorrelationARGOX
VAR 0.815 0.779 0.839 0.576 0.536 0.948 0.662 0.602GFT – – – – – – –Lu et al. (2019) – 0.907 0.910 0.703 0.915 – – –naive 0.888 0.865 0.855 0.629 0.888 0.960 0.807 0.575Table S27: Comparison of different methods for state-level %ILI estimation in Minnesota (MN). The MSE, MAE,and correlation are reported. The method with the best performance is highlighted in boldface for each metric in eachperiod. − Jan 2018 − Jan 2020 − Jan % I L I US.MN − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S23: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Minnesota (MN).38
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.821 0.462 0.663 0.439 0.428 2.412 0.989 1.619GFT – – 1.936 – – – – –Lu et al. (2019) – – – – – – – –naive 0.622 0.476 0.931 0.200 0.422 1.690 0.706 1.043MAEARGOX
VAR 0.614 0.510 0.571 0.533 0.531 1.032 0.724 0.939GFT – – 1.220 – – – – –Lu et al. (2019) – – – – – – – –naive 0.508 0.450 0.606 0.344 0.519 0.882 0.601 0.851CorrelationARGOX
VAR 0.932 0.932 0.947 0.731 0.879 0.931 0.925 0.778GFT – – 0.960 – – – – –Lu et al. (2019) – – – – – – – –naive 0.940 0.926 0.923 0.835 0.883 0.930 0.912 0.822Table S28: Comparison of different methods for state-level %ILI estimation in Mississippi (MS). The MSE, MAE,and correlation are reported. The method with the best performance is highlighted in boldface for each metric in eachperiod. − Jan 2018 − Jan 2020 − Jan % I L I US.MS − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S24: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Mississippi (MS).39
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 2.090 0.934 0.843 0.969 1.633 6.039 1.068 9.347GFT – – 0.482 – – – – –Lu et al. (2019) – – – – – – – –naive 0.767 0.412 0.741 0.155 0.478 1.818 1.017 2.692MAEARGOX
VAR 0.712 0.611 0.637 0.639 0.795 1.068 0.691 2.100GFT – – 0.570 – – – – –Lu et al. (2019) – – – – – – – –naive 0.499 0.392 0.486
VAR 0.866 0.852 0.883 0.626 0.844 0.897 0.901 0.658GFT – – – – – – –Lu et al. (2019) – – – – – – – –naive 0.935 0.916 0.898 0.761 0.919 0.938 0.890 0.895Table S29: Comparison of different methods for state-level %ILI estimation in Missouri (MO). The MSE, MAE, andcorrelation are reported. The method with the best performance is highlighted in boldface for each metric in each period. − Jan 2018 − Jan 2020 − Jan % I L I US.MO − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S25: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Missouri (MO).40
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.198 0.157 0.275 0.080 0.136 0.037 0.351 0.885GFT – – 1.095 – – – – –Lu et al. (2019) – – – – – – – –naive 0.061 0.049 0.079 0.067 0.024 0.029
VAR 0.248 0.232 0.330 0.216 0.212 0.140 0.449 0.598GFT – – 0.870 – – – – –Lu et al. (2019) – – – – – – – –naive 0.144 0.132
VAR 0.877 0.552 0.585 0.367 0.398 0.668 0.897 0.802GFT – – – – – – –Lu et al. (2019) – – – – – – – –naive 0.963 0.821 0.865 0.341 0.499 0.736 0.916 0.969Table S30: Comparison of different methods for state-level %ILI estimation in Montana (MT). The MSE, MAE, andcorrelation are reported. The method with the best performance is highlighted in boldface for each metric in each period. − Jan 2018 − Jan 2020 − Jan % I L I US.MT − − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S26: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Montana (MT).41
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 2.520 1.172 3.148 0.093 0.353 2.474 4.081 13.168GFT – – 0.678 – – – – –Lu et al. (2019) – 0.265 0.605 0.141 0.251 – – –naive 0.497 0.303 0.636
VAR 0.779 0.490 0.881 0.253 0.373 1.175 1.295 1.846GFT – – 0.501 – – – – –Lu et al. (2019) – – – – – – – –naive 0.434 0.318 0.460
VAR 0.769 0.495 0.255 0.498 0.818 0.723 0.628 0.735GFT – – 0.846 – – – – –Lu et al. (2019) – − Jan 2018 − Jan 2020 − Jan % I L I US.NE − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S27: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Nebraska (NE).42
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX – – –naive 0.115 0.109 0.179 0.131 0.060 0.122 0.184
MAEARGOX
CorrelationARGOX – – –naive 0.929 0.919 0.929 0.859 0.865 0.933 0.898
Table S32: Comparison of different methods for state-level %ILI estimation in Nevada (NV). The MSE, MAE, andcorrelation are reported. The method with the best performance is highlighted in boldface for each metric in each period. − Jan 2018 − Jan 2020 − Jan % I L I US.NV − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S28: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Nevada (NV).43
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.363 0.081 0.113 0.048 0.086 – – –naive 0.201 0.056
VAR 0.357 0.198 0.226 0.168 – – – – –Lu et al. (2019) – – – –naive 0.904 0.854 0.902 0.883 0.692 0.887 0.743
Table S33: Comparison of different methods for state-level %ILI estimation in New Hampshire (NH). The MSE, MAE,and correlation are reported. The method with the best performance is highlighted in boldface for each metric in eachperiod. − Jan 2018 − Jan 2020 − Jan % I L I US.NH − −
202 2016 − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S29: Plots of the %ILI estimates (top) and the estimation errors (bottom) for New Hampshire (NH).44
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.757 0.797 1.205 0.440 1.098 1.405 0.257 1.542GFT – – 0.644 – – – – –Lu et al. (2019) –
VAR 0.562 0.621 0.775 0.462 0.836 0.727 0.350 0.811GFT – – 0.672 – – – – –Lu et al. (2019) – – – – – – – –naive 0.387 0.384 0.412 0.376
VAR 0.907 0.826 0.638 0.835 0.766 0.911 0.926 0.888GFT – – 0.896 – – – – –Lu et al. (2019) – − Jan 2018 − Jan 2020 − Jan % I L I US.NJ − − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S30: Plots of the %ILI estimates (top) and the estimation errors (bottom) for New Jersey (NJ).45
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.598 0.477 0.700 0.401 0.519 0.751 0.825 1.958GFT – – 1.944 – – – – –Lu et al. (2019) – 0.164 0.293 0.213 – – –naive 0.417 0.247 0.399 0.234 0.167 0.807 0.560 1.499MAEARGOX
VAR 0.478 0.454 0.475 0.487 0.516 0.538 0.611 0.928GFT – – 1.222 – – – – –Lu et al. (2019) – – – – – – – –naive 0.389 0.329 0.368 0.368 0.310 0.580 0.501 0.806CorrelationARGOX
VAR 0.926 0.871 0.895 0.843 0.845 0.942 0.888 0.903GFT – – – – – – –Lu et al. (2019) – 0.943 0.923 0.927 – – –naive 0.946 0.912 0.881 0.916 0.935 0.926 0.924 0.925Table S35: Comparison of different methods for state-level %ILI estimation in New Mexico (NM). The MSE, MAE,and correlation are reported. The method with the best performance is highlighted in boldface for each metric in eachperiod. − Jan 2018 − Jan 2020 − Jan % I L I US.NM − − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S31: Plots of the %ILI estimates (top) and the estimation errors (bottom) for New Mexico (NM).46
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 1.863 2.812 0.792 0.522 9.831 1.794 0.431 2.434GFT – – 0.686 – – – – –Lu et al. (2019) – 1.228
VAR 0.690 0.755 0.594 0.544 1.581 0.952 0.486 1.003GFT – – 0.630 – – – – –Lu et al. (2019) – – – – – – – –naive 0.426 0.489 0.514 0.432
VAR 0.849 0.769 0.859 0.756 0.676 0.921 0.817 0.856GFT – – 0.903 – – – – –Lu et al. (2019) – 0.856 − Jan 2018 − Jan 2020 − Jan % I L I US.NY − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S32: Plots of the %ILI estimates (top) and the estimation errors (bottom) for New York (NY).47
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 1.006 1.072 2.404 0.297 0.740 2.260 1.597 0.279GFT – – 0.374 – – – – –Lu et al. (2019) –
VAR 0.547 0.607 0.887 0.456 0.625 0.927 0.617 0.422GFT – – 0.420 – – – – –Lu et al. (2019) – – – – – – – –naive 0.408 0.446 0.474 0.348 0.680 0.707 0.374 0.431CorrelationARGOX
VAR 0.874 0.769 0.569 0.831 0.890 0.954 0.911 0.957GFT – – – – – – –Lu et al. (2019) – 0.942 0.958 0.890 0.919 – – –naive 0.925 0.887 0.860 0.881 0.881 0.910 0.940 0.941Table S37: Comparison of different methods for state-level %ILI estimation in North Carolina (NC). The MSE, MAE,and correlation are reported. The method with the best performance is highlighted in boldface for each metric in eachperiod. − Jan 2018 − Jan 2020 − Jan % I L I US.NC − − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S33: Plots of the %ILI estimates (top) and the estimation errors (bottom) for North Carolina (NC).48
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 3.213 1.877 2.947 0.544 3.125 5.011 1.006 8.291GFT – – 0.938 – – – – –Lu et al. (2019) – 0.806 1.496 0.230 – – –naive 0.816 0.776 0.819 0.245 1.802 0.992 0.618 2.408MAEARGOX
VAR 0.874 0.724 0.923 0.521 1.080 1.165 0.796 1.894GFT – – 0.669 – – – – –Lu et al. (2019) – – – – – – – –naive 0.567 0.499 0.495 0.405 0.899 0.735 0.607 1.210CorrelationARGOX
VAR 0.671 0.799 – – –naive 0.846 0.800 0.869 0.687 0.575 0.740 0.808 0.769Table S38: Comparison of different methods for state-level %ILI estimation in North Dakota (ND). The MSE, MAE,and correlation are reported. The method with the best performance is highlighted in boldface for each metric in eachperiod. − Jan 2018 − Jan 2020 − Jan % I L I US.ND − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S34: Plots of the %ILI estimates (top) and the estimation errors (bottom) for North Dakota (ND).49
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.248 0.312 0.784 0.063 0.119 0.232 0.110 0.596GFT – – 0.819 – – – – –Lu et al. (2019) –
VAR 0.280 0.280 0.436 0.203 0.237 0.344
VAR 0.885 0.846 0.835 0.829 0.919 0.957 – – – – –Lu et al. (2019) – − Jan 2018 − Jan 2020 − Jan % I L I US.OH − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S35: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Ohio (OH).50
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 8.691 4.993 3.314 2.556 13.327 5.961 9.482 52.931GFT – – 3.906 – – – – –Lu et al. (2019) – – – – – – – –naive 0.965
VAR 1.415 1.282 1.186 1.253 2.139 1.349 1.956 4.176GFT – – 1.390 – – – – –Lu et al. (2019) – – – – – – – –naive 0.646
VAR 0.808 0.803 0.858 0.493 0.761 0.914 0.890 0.610GFT – – 0.856 – – – – –Lu et al. (2019) – – – – – – – –naive 0.951 − Jan 2018 − Jan 2020 − Jan % I L I US.OK − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S36: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Oklahoma (OK).51
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 1.188 0.616 0.281 0.559 1.463 6.474 0.349 0.717GFT – – 0.372 – – – – –Lu et al. (2019) – 0.408 0.253
VAR 0.522 0.480 0.356 0.598 0.716 1.156 0.436 0.636GFT – – 0.516 – – – – –Lu et al. (2019) – – – – – – – –naive 0.300 0.341
VAR 0.794 0.739 0.745 0.591 0.702 0.825 0.878 0.872GFT – – 0.739 – – – – –Lu et al. (2019) – 0.802 0.779 − Jan 2018 − Jan 2020 − Jan % I L I US.OR − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S37: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Oregon (OR).52
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.311 0.511 1.013 0.224 0.446 0.167 0.106 0.342GFT – – 0.329 – – – – –Lu et al. (2019) –
VAR 0.322 0.420 0.561 0.324 0.473 0.302 0.230 0.421GFT – – 0.473 – – – – –Lu et al. (2019) – – – – – – – –naive 0.295 0.361 0.467 0.336 0.396 0.358 0.212 0.440CorrelationARGOX
VAR 0.925 0.871 0.866 0.810 0.866 0.973 0.934 0.935GFT – – 0.950 – – – – –Lu et al. (2019) – – – –naive 0.938 0.898 0.889 0.793 0.900 0.945 0.943 0.938Table S42: Comparison of different methods for state-level %ILI estimation in Pennsylvania (PA). The MSE, MAE,and correlation are reported. The method with the best performance is highlighted in boldface for each metric in eachperiod. − Jan 2018 − Jan 2020 − Jan % I L I US.PA − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S38: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Pennsylvania (PA).53
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.858 0.272 0.349 0.129 0.423 2.802 2.542 1.130GFT – – 0.417 – – – – –Lu et al. (2019) – – – –naive 0.307 0.166 0.201 0.057 0.294 0.661 0.428 1.150MAEARGOX
VAR 0.433 0.313 0.313 0.248 0.494 0.818 0.894 0.737GFT – – 0.565 – – – – –Lu et al. (2019) – – – – – – – –naive 0.314 0.232 0.257
GFT – – 0.944 – – – – –Lu et al. (2019) – − Jan 2018 − Jan 2020 − Jan % I L I US.RI − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S39: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Rhode Island (RI).54
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 1.932 0.912 0.963 0.342 2.115 8.423 1.260 3.850GFT – – 2.269 – – – – –Lu et al. (2019) – 0.383
VAR 0.687 0.531 0.436 0.459 1.085 1.772 0.741 1.029GFT – – 1.235 – – – – –Lu et al. (2019) – – – – – – – –naive 0.516 0.403 0.387 0.292 0.795 0.880 0.683 1.172CorrelationARGOX
VAR 0.906 0.882 0.816 0.574 0.811 0.754 0.922 0.918GFT – – 0.978 – – – – –Lu et al. (2019) – 0.951 – – –naive 0.954 0.934 0.860 0.823 0.888 0.941 0.915 0.874Table S44: Comparison of different methods for state-level %ILI estimation in South Carolina (SC). The MSE, MAE,and correlation are reported. The method with the best performance is highlighted in boldface for each metric in eachperiod. − Jan 2018 − Jan 2020 − Jan % I L I US.SC − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S40: Plots of the %ILI estimates (top) and the estimation errors (bottom) for South Carolina (SC).55
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.331 0.372 0.363 0.371 0.594 0.250 0.228 0.992GFT – – 0.953 – – – – –Lu et al. (2019) – – – –naive 0.124 0.117 0.117 0.095 0.182 0.285 0.141 0.095MAEARGOX
VAR 0.373 0.397 0.405 0.421 0.497 0.395 0.369 0.629GFT – – 0.476 – – – – –Lu et al. (2019) – – – – – – – –naive 0.256 0.252 0.239 0.244 0.318 0.416 0.284 0.220CorrelationARGOX
VAR 0.880 0.828 0.828 0.674 0.832 0.922 0.725 0.843GFT – – 0.906 – – – – –Lu et al. (2019) – – – –naive 0.943 0.918 0.907 0.840 0.924 0.908 0.844 0.970Table S45: Comparison of different methods for state-level %ILI estimation in South Dakota (SD). The MSE, MAE,and correlation are reported. The method with the best performance is highlighted in boldface for each metric in eachperiod. − Jan 2018 − Jan 2020 − Jan % I L I US.SD − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S41: Plots of the %ILI estimates (top) and the estimation errors (bottom) for South Dakota (SD).56
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 1.752 1.695 2.354 0.767 2.342 3.022 1.295 4.688GFT – – 1.407 – – – – –Lu et al. (2019) – 0.391 0.440 0.282 – – –naive 0.748 0.568 0.705 0.257 0.983 1.427 0.500 2.811MAEARGOX
VAR 0.726 0.776 0.729 0.633 1.079 0.787 0.713 1.469GFT – – 0.947 – – – – –Lu et al. (2019) – – – – – – – –naive 0.509 0.481 0.480
VAR 0.853 0.797 0.867 0.684 0.671 0.813 0.824 0.803GFT – – – – – – –Lu et al. (2019) – 0.936 0.953 0.841 – – –naive 0.929 0.900 0.898 0.836 0.822 0.869 0.869 0.881Table S46: Comparison of different methods for state-level %ILI estimation in Tennessee (TN). The MSE, MAE, andcorrelation are reported. The method with the best performance is highlighted in boldface for each metric in each period. − Jan 2018 − Jan 2020 − Jan % I L I US.TN −
505 2016 − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S42: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Tennessee (TN).57
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 1.783 1.736 3.191 1.162 1.448 3.269 2.243 3.249GFT – – – – – – –Lu et al. (2019) –
VAR 0.738 0.757 0.961 0.709 0.829 0.959 0.858 1.152GFT – – 0.772 – – – – –Lu et al. (2019) – – – – – – – –naive 0.598 0.563 0.801
VAR 0.920 0.839 0.827 0.641 0.866 0.937 0.931 0.876GFT – – – – – – –Lu et al. (2019) – − Jan 2018 − Jan 2020 − Jan % I L I US.TX −
505 2016 − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S43: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Texas (TX).58
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.875 0.742 0.960 1.257 0.363 0.232 1.888 2.574GFT – – 1.059 – – – – –Lu et al. (2019) – 0.185 0.235 0.221 0.217 – – –naive 0.255 0.182 0.238 0.184
VAR 0.633 0.594 0.698 0.881 0.392 0.356 1.086 1.313GFT – – 0.930 – – – – –Lu et al. (2019) – – – – – – – –naive 0.334 0.293 0.334 0.329
VAR 0.840 0.818 0.795 0.698 0.858 0.847 0.704 0.720GFT – – 0.949 – – – – –Lu et al. (2019) – 0.921 0.930 0.849 0.880 – – –naive 0.951 0.923 0.923 0.885 − Jan 2018 − Jan 2020 − Jan % I L I US.UT −
202 2016 − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S44: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Utah (UT).59
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.579 0.597 0.655 0.692 0.597 0.801 0.967 0.355GFT – – 1.330 – – – – –Lu et al. (2019) – 0.317 0.845 – – –naive 0.286 0.315 0.606 0.220 0.127 0.333 0.348 0.323MAEARGOX
VAR 0.558 0.580 0.615 0.661 0.510 0.670 0.727 0.471GFT – – 0.761 – – – – –Lu et al. (2019) – – – – – – – –naive 0.380 0.384 0.521 0.360 0.264 0.451 0.447 0.400CorrelationARGOX
VAR 0.824 0.790 0.883 0.609 0.478 0.840 0.683 0.928GFT – – 0.851 – – – – –Lu et al. (2019) – 0.889 0.845 − Jan 2018 − Jan 2020 − Jan % I L I US.VT − −
202 2016 − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S45: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Vermont (VT).60
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.325 0.263 0.438 0.048 0.408 0.651 0.297 0.975GFT – – 0.984 – – – – –Lu et al. (2019) – – – –naive 0.396 0.389 0.798 0.084 0.417 0.952 0.314 0.695MAEARGOX
VAR 0.327 0.299 0.373
VAR 0.965 0.946 0.956 0.937 0.892 0.960 0.959 0.929GFT – – 0.977 – – – – –Lu et al. (2019) – – – –naive 0.955 0.907 0.895 0.889 0.885 0.941 0.956 0.948Table S50: Comparison of different methods for state-level %ILI estimation in Virginia (VA). The MSE, MAE, andcorrelation are reported. The method with the best performance is highlighted in boldface for each metric in each period. − Jan 2018 − Jan 2020 − Jan % I L I US.VA − − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S46: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Virginia (VA).61
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.561 0.257 0.305 0.213 0.401 0.558 0.322 3.864GFT – – 0.562 – – – – –Lu et al. (2019) – – – –naive 0.263 0.157 0.145 0.114 0.322 0.551 0.302 0.966MAEARGOX
VAR 0.406 0.337 0.396 0.364 0.383 0.450 0.329 1.431GFT – – 0.654 – – – – –Lu et al. (2019) – – – – – – – –naive 0.304 0.262 0.277 0.283
VAR 0.861 0.836 0.867 0.604 0.796 0.848 0.882 0.698GFT – – 0.963 – – – – –Lu et al. (2019) – − Jan 2018 − Jan 2020 − Jan % I L I US.WA − −
202 2016 − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S47: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Washington (WA).62
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.600 0.307 0.341 0.222 0.523 1.791 0.489 2.093GFT – – 0.974 – – – – –Lu et al. (2019) – 0.179
VAR 0.458 0.385 0.405 0.355 0.529 0.778 0.522 1.101GFT – – 0.909 – – – – –Lu et al. (2019) – – – – – – – –naive 0.355 0.317 0.442 0.281
VAR 0.920 0.939 0.964 0.787 0.896 0.894 0.909 0.835GFT – – 0.969 – – – – –Lu et al. (2019) – − Jan 2018 − Jan 2020 − Jan % I L I US.WV − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S48: Plots of the %ILI estimates (top) and the estimation errors (bottom) for West Virginia (WV).63
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.870 0.802 1.446 0.474 0.681 1.958 0.343 0.788GFT – – 0.562 – – – – –Lu et al. (2019) – 0.271 0.630 0.144 0.162 – – –naive 0.203 0.274 0.458 0.120 0.180 0.204 0.086 0.243MAEARGOX
VAR 0.583 0.618 0.881 0.557 0.530 0.891 0.432 0.666GFT – – 0.604 – – – – –Lu et al. (2019) – – – – – – – –naive 0.308 0.360 0.418 0.298 0.344 0.344 0.224 0.381CorrelationARGOX
VAR 0.779 0.738 0.695 0.543 0.757 0.749 0.475 0.860GFT – – 0.897 – – – – –Lu et al. (2019) – 0.898 0.868 0.696 0.895 – – –naive 0.937 0.896 0.902 0.743 0.869 0.922 0.885 0.962Table S53: Comparison of different methods for state-level %ILI estimation in Wisconsin (WI). The MSE, MAE, andcorrelation are reported. The method with the best performance is highlighted in boldface for each metric in each period. − Jan 2018 − Jan 2020 − Jan % I L I US.WI − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S49: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Wisconsin (WI).64
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
VAR 0.739 0.473 0.489 0.290 0.946 1.503 1.603 1.554GFT – – 0.318 – – – – –Lu et al. (2019) – – – – – – – –naive 0.325 0.156
CorrelationARGOX
VAR 0.880 0.846 0.930 0.728 0.740 0.848 0.862 0.803GFT – – 0.891 – – – – –Lu et al. (2019) – – – – – – – –naive 0.939 0.918 − Jan 2018 − Jan 2020 − Jan % I L I US.WY − − Jan 2018 − Jan 2020 − Jan % I L I Methods
ARGOXCDCGFTVAR
Figure S50: Plots of the %ILI estimates (top) and the estimation errors (bottom) for Wyoming (WY).65
PREPRINT - J
UNE
5, 2020Whole period ’14-’20 Overall ’14-’17 ’14-’15 ’15-’16 ’16-’17 ’17-’18 ’18-’19 ’19-’20MSEARGOX
GFT – – 0.257 – – – – –Lu et al. (2019) – – – – – – – –naive 0.269 0.072 0.048 0.069 0.154 0.581 0.105 1.768MAEARGOX
VAR 0.301 0.238 0.223 0.250 0.299 0.386 0.336 0.745GFT – – 0.436 – – – – –Lu et al. (2019) – – – – – – – –naive 0.267 0.188 0.168 0.207
GFT – – 0.949 – – – – –Lu et al. (2019) – – – – – – – –naive 0.944 0.949 0.950 0.929 0.902 0.928 0.929 0.878Table S55: Comparison of different methods for state-level %ILI estimation in New York City (NYC). The MSE, MAE,and correlation are reported. The method with the best performance is highlighted in boldface for each metric in eachperiod. − Jan 2018 − Jan 2020 − Jan % I L I US.NYC − −
202 2016 − Jan 2018 − Jan 2020 − Jan % I L I Methods