Epidemiologically and Socio-economically Optimal Policies via Bayesian Optimization
Amit Chandak, Debojyoti Dey, Bhaskar Mukhoty, Purushottam Kar
EEpidemiologically and Socio-economically Optimal Policies viaBayesian Optimization
Amit Chandak Debojyoti Dey Bhaskar MukhotyPurushottam KarIndian Institute of Technology Kanpur { amitch,debojyot,bhaskarm,purushot } @cse.iitk.ac.in June 16, 2020
Abstract
Mass public quarantining, colloquially known as a lock-down , is a non-pharmaceutical intervention tocheck spread of disease. This paper presents
ESOP (Epidemiologically and Socio-economically OptimalPolicies) , a novel application of active machine learning techniques using Bayesian optimization, thatinteracts with an epidemiological model to arrive at lock-down schedules that optimally balance publichealth benefits and socio-economic downsides of reduced economic activity during lock-down periods. Theutility of ESOP is demonstrated using case studies with
VIPER (Virus-Individual-Policy-EnviRonment),a stochastic agent-based simulator that this paper also proposes. However,
ESOP is flexible enough tointeract with arbitrary epidemiological simulators in a black-box manner, and produce schedules thatinvolve multiple phases of lock-downs.
Disclaimer : This paper makes no recommendation to individuals and its results should not be interpretedby individuals to modulate personal behavior. The authors recommend that individuals continue to followguidelines offered by local governments with respect to lock-downs and social distancing, and those offeredby medical professionals with respect to personal hygiene and treatment.
Infectious diseases that are contagious pose a threat to public safety once they attain pandemic status. Severalhistorical instances of such pandemics have taken a heavy toll on human lives. Prominent examples includethe H1N1 (Spanish flu) pandemic of 1918 ( >
50 million fatalities), the H3N2 (HongKong flu) pandemic of1968 ( ≈ ≈
32 million fatalities) [Kimball and Bose, 2020], thenovel influenza-A H1N1 (swine flu) pandemic of 2009 ( ≈ ≈ mitigation policies such as human surveillance and contact tracing,and 2) suppression policies such as social distancing or its more extreme form colloquially known as a lock-down [Aledort et al., 2007]. However, their benefits with respect to public health outcomes notwithstanding,severe and extended applications of suppression policies such as lock-downs negatively impact livelihoodsand the economy. For instance, Scherbina [2020] estimates the cost of extensive suppression measures to theUS economy at $9 trillion, or about 43% of its annual GDP.Peak et al. [2017] demonstrate the need for policy decisions to balance suppression and mitigation mea-sures in terms of the epidemiological characteristics of the pandemic, pointing out that suppression measures All code used for this study is available at the following GitHub Repository https://github.com/purushottamkar/esop a r X i v : . [ q - b i o . P E ] J un old most benefit for fast-course diseases whereas effective mitigation measures may suffice for others at muchless socio-economic cost. This points to a need for techniques that can take the disease progression charac-teristics of a certain outbreak and suggest policies that optimally use suppression and mitigation techniquesto offer acceptable health outcomes as well as socio-economic risks within acceptable limits. In fact there isan emerging area variously termed “economic epidemiology” or “epidemiological economics” Perrings et al.[2014] that seeks to develop models that can address the interplay of disease and host behavior. Several works exist on modelling epidemic and pandemic progressions using differential equation-based mod-els such as SIR or SEIR and using them to make predictions. Some examples include [Efimov and Ushirobira,2020, Lyra et al., 2020, Sardar et al., 2020, Vyasarayani and Chatterjee, 2020]. Most of these studies utilizedifferential equation-based models such as SIR and SEIR variants. This paper instead uses a stochasticagent-based model called
VIPER that is proposed in Sec 2. The reason behind this choice was to demon-strate the effectiveness of our proposed techniques when working with epidemiological models that are notdescribed compactly using a few equations and thus, harder to analyze.Micro-simulation studies using UK [Ferguson et al., 2020] and Indian [Singh and Adhikari, 2020] dataconclude that multiple short-term suppression rounds may offer acceptable health outcomes when a singleextended period of suppression is infeasible. However, these works do not offer ways to find either the optimalmoment to initiate suppression measures or their duration.This is important since Morris et al. [2020], Patterson-Lomba [2020] show that the optimal initiation pointand duration of a suppression may depend substantially on the disease and social characteristics themselves(for instance the disease incubation period and the basic reproduction number R ). This is understandablesince premature suppression would slow down the depletion of the pool of susceptible individuals leavingroom open for a second wave of infections whereas delayed suppression may cause the initial wave to bewidespread in itself. Scherbina [2020] additionally considers the economic impact of these measures andsuggests durations for lock-down periods and their associated economic costs in medical expenses as well aslost value of statistical life.Prior works offering actual policy advice fall into two categories: 1) those that offer only broad principleson how to target interventions e.g. by identifying simple rules of thumb [Wallinga et al., 2010], and 2) thosethat do offer actionable advice e.g. when to initiate suppression [Klepac et al., 2011, Morris et al., 2020,Torre et al., 2019, Zhao and Feng, 2019]. However, the latter often do not take the socio-economic impactof these measures into account and moreover, consider only simple theoretical models e.g. SIR that are notvery expressive.A subclass of the latter approaches [for example, Bussell et al., 2019] advocate first fitting an approximatemodel to the actual simulator (to make it simple enough to enable mathematical analyses) before applyingoptimal control strategies. Such approximations may introduce unmodelled errors into the prediction pipelineand adversely affect their outcome. ESOP instead directly models intervention outcomes in terms of thesimulator outputs.
This paper presents
ESOP (Epidemiologically and Socio-economically Optimal Policies), a system that usesBayesian optimization to automatically suggest suppression policies that optimally balance public healthand economic outcomes.
ESOP interacts with epidemiological and economic models to automatically sug-gest policy decisions. The paper also presents
VIPER (Virus-Individual-Policy-EnviRonment), an iterative,stochastic agent-based model (ABM) with which case studies are conducted to showcase the utility of
ESOP .We note however, that
ESOP can readily interact with other epidemiological models, e.g. those that incor-porate stratification based on region and age e.g. INDSCI-SIM [Shekatkar et al., 2020], COVision [Nagoriet al., 2020], ABCS [Harsha et al., 2020] and IndiaSim [Megiddo et al., 2014]. Although machine learn-ing techniques have been used in epidemiological forecasting [Lindstr¨om et al., 2015] and estimating model2 ttr Description Range Def
Viral Model
INC incubation period N ,
1] 0.05DPR disease progression rate [0 ,
1] 0.1XTH VLD threshold for expiry [0 ,
1] 0.7BXP expiry probability at XTH [0 ,
1] 0.0
Environment Model
BCR contact radius b/w individuals [0 ,
1] 0.25BIP prob. infection upon contact [0 ,
1] 0.5BTR prob. of an individual traveling [0 ,
1] 0.01BTD maximum travel distance [0 ,
1] 1.0INI initial rate of infection [0 ,
1] 0.01
Attr Description Range Ini
Individual Model
SUS susceptibility to infection [0 ,
1] rndRST resistance to disease progression [0 ,
1] rndVLD current viral load [0 ,
1] 0.0RLD current recovery load [0 ,
1] 0.0STA current state SEIRX SQRN quarantine status 0 or 1 0X, Y current location [0 , rnd Policy Model
QTH VLD threshold for quarantine [0 ,
1] 0.3BQP quarantine probability at QTH [0 ,
1] 0.0 l ( t ) lock-down level at time t [0 ,
5] —
Table 1:
VIPER model attributes, valid ranges and default/initial values. See Sec 2 for details.parameters [Dandekar and Barbastathis, 2020], we are not aware of prior work using machine learning inepidemiological policy design. VIPER : An Iterative Stochastic Agent-based EpidemiologicalModel
VIPER (Virus-Individual-Policy-EnviRonment) models an in-silico population of individuals, supports com-partments of the SEIR model [Keeling and Rohani, 2008], and allows travel and quarantining of individuals.Being an ABM rather than an ODE-based model,
VIPER can model disease progression within each individ-ual separately and thus, quarantine or expire individuals based on their stage of the disease, something thatis difficult to do in ODE-based models. Stochastic ABMs allow diverse socio-medico-economic traits to bemodeled at the individual level but cannot be easily represented by a concise system of ODEs. Thus, workssuch as [Morris et al., 2020] do not apply here.Details of the
VIPER model are described below and succinctly enumerated in Tab 1.
VIPER consistsfour sub-models , one each devoted to modelling individuals, the virus, environment parameters and policyparameters.
Individual Model : an individual is characterized by their susceptibility to infection (SUS), resistance todisease progression (RST), viral load (VLD), recovery load (RLD), current state (STA), quarantine status(QRN) and location (X,Y). SUS, RST, VLD DLD, X and Y are real numbers between 0 and 1, whereasQRN takes Boolean values. The state of an individual STA can be either S (susceptible), E (exposed), I(infectious), R (recovered) or X (expired/deceased). Individuals are initialized with random values for RST,SUS and their location within the 2-D box [0 , . Individuals progress from state S → E → I, can beoptionally quarantined while in state I, and then move to either state R or X.
Viral Model : the virus is characterized by its incubation period (INC), the base viral load in an individualat the end of the incubation period (BVL), the disease progression rate (DPR), the viral load over whichan individual’s chances of getting expired start increasing (XTH) and the base removal probability of anindividual with viral load at XTH (BXP). BVL, DPR, XTH and BXP are real numbers between 0 and 1whereas INC is a natural number.
Environment Model : the environmental factors are modeled using the typical contact radius betweenindividuals (BCR), the probability that a contact between an infectious and susceptible individual will leadto a successful infection (BIP), the fraction of the population that travels at any time instant (BTR), the3aximum distance to which they travel (BTD), and the fraction of population that is infected with the virusat start of the simulation (INI). All these values are represented as real numbers between 0 and 1.
Policy Model : the policy model comprises the viral load over which an individual’s chances of gettingquarantined start increasing (QTH) and the base quarantining probability of an individual with viral loadat QTH (BQP). Both are real numbers between 0 and 1. Additionally, the policy prescribes a lock-down level which is a real number between 0 and 5. The lock-down level is specified at every time instant t of thesimulation. A lock-down level of l at a certain time instant causes BTD as well as BCR to go down by afactor of exp( − l ). Thus, at a high lock-down level, individuals are neither able to travel much, nor interactwith other individuals far off from their current location. Modelling disease-progression dynamics in
VIPER : The viral load (VLD) of an individual representsthe extent of infection within their system. At the end of the incubation period (INC), an exposed individualalways has a “base” viral load of BVL. The virus attempts to increase this viral load according to the diseaseprogression rate (DPR) whereas the individual resists this according to their resistance level (RST) byconverting viral load to recovery load (RLD). Disease progression in every infected individual is governed byan SIR-like model with d VLD( t ) dt = − RST · VLD( t ) + DPR · (1 − VLD( t ) − RLD( t ))and d RLD( t ) dt = RST · VLD( t )Thus, VIPER allows individuals to experience disease progression, as well as associated effects like quaran-tining or expiry, in a completely individualized manner, something that is readily possible in agent-basedmodels but much more difficult to express in terms of a compact set of differential equations.An infected individual whose VLD falls below BVL moves on to state R, i.e. recovers. An individualwith VLD equal to QTH (resp. XTH) has a probability BQP (resp. BXP) of getting quarantined (resp.expired). An individual with VLD greater than QTH has the following probability of getting quarantined P [quarantine] = BQP + (1 − BQP) · VLD − QTH1 − QTH , i.e. the probability of getting quarantined increases linearly to 1 as the individual’s VLD goes up. At everytime step t , a coin is tossed for all individuals with VLD greater than QTH which lands heads with thisparticular probability. If the coin does indeed land heads, the individual is deemed quarantined. Note thatthis allows VIPER to model asymptomatic transmission since it allows individuals with low VLD levels (esp.below QTH) to avoid detection with high probability but makes it difficult for those in advanced stages ofthe disease to avoid quarantine.Similarly, an individual with VLD greater than XTH has the following probability of getting expired P [expiry] = BXP + (1 − BXP) · VLD − XTH1 − XTH . At every time step, a coin is similarly tossed to decide on whether an individual with VLD greater than XTHgets expired or not. The lock-down level needs to be specified at every time instant t of the simulation. Alock-down level of l ( t ) causes BTD as well as BCR at that time t to go down by a factor of exp( − l ( t )). Thus,at high lock-down levels, individuals are neither able to travel much, nor have contact with other individualsfar off from their current location. The lock-down level has no effect on the quarantining or expiry processesdescribed above which continue the same way irrespective of the lock-down level.4 y next query point (a) x y next query point (b) true function f ( x ) mean estimate ˆ f ( x ) uncertainty region ˆ f ( x ) ± σ ( x ) observations ˆ x query Update mean መ 𝑓 and uncertainty 𝜎 estimatesOptimize acquisition function to find next query point ො𝐱 query Get (estimate of) 𝑓 ො𝐱 query from simulators
Figure 1:
An illustration of the Bayesian optimization process . See Sec 3 for details. ESOP : Epidemiologically and Socio-economically Optimal Poli-cies
ESOP encodes interventions as vectors and their health and socio-economic outcomes as functions, e.g., thecoordinates of a 2-D vector x = [ x , x ] ∈ N may encode the starting point ( x ) and duration ( x ) ofa lock-down. Next, consider a function f epi : N → [0 ,
1] encoding health outcomes with f epi ( x ) equal tothe peak infection rate (the largest fraction of the total population infected at any point of time) if theintervention x is applied. Similarly, let f eco : N → [0 ,
1] encode economic outcomes with f eco ( x ) being thefraction of population that would face unemployment if lock-down were indeed to last x days. We stress thatthe functions f epi , f eco described here are examples and other measurable outcomes, e.g. cumulative deathrate, loss to GDP, can also be used. Predicted estimates for f epi would be obtained from epidemiologicalsimulators such as INDSCI-SIM or IndiaSim (we will use VIPER ) and those for f eco would be obtained fromeconomic models. Our goal is to balance health and economic outcomes by solving the following optimizationproblem: x ∗ := arg min x ∈ N f ( x ) where f ( x ) = f epi ( x ) + f eco ( x )However, it is challenging to perform this optimization using standard descent techniques [Boyd and Van-denberghe, 2004] since even obtaining values of the function f at specific query points (let alone gradients)is expensive as it involves querying simulators such as VIPER . An acceptable solution in this case is Bayesianoptimization [Jones et al., 1998] which is an active machine learning technique used to optimize functionswhich are expensive to evaluate and to which, moreover we do not have access to gradients.Fig 1 presents a visual depiction of the key processes in Bayesian optimization. The technique adaptivelyqueries the function at only a few locations to quickly approximate the solution to the optimization problem.The algorithm uses the current observations and Gaussian process regression [Rasmussen and Williams,2006] to obtain a mean estimate ˆ f ( x ) (dashed line in Fig 1) of the true function f ( x ) (bold line in Fig 1) aswell as an estimate σ ( x ) of the uncertainty in that estimate (the blue shaded region depicts ˆ f ( x ) ± σ ( x ) inFig 1). Notice that uncertainty drops around observation points since (a good estimate of) the true functionvalue is known there.Using these, an acquisition function is created. Fig 1(a) uses the simple LCB (lower confidence bound)acquisition function defined as a ( x ) = ˆ f ( x ) − σ ( x ). Other possibilities include EI (expected improvement)and KG (knowledge gradient). The (estimated) function value at the point ˆ x query := arg min a ( x ) is nowqueried. Using f (ˆ x query ), the mean and uncertainty estimates ˆ f , σ are updated as shown in Fig 1(b) and theprocess is repeated. At the end, the query point with the lowest function value is returned as the estimatedminimum. Note that ESOP can only query
VIPER but does not have access to its internal attribute values.Lack of space does not permit a detailed overview and we refer the reader to [Frazier, 2018] for anexcellent review.
ESOP also employs multi-scale search and caching techniques to accelerate computations.Bayesian optimization routines often enjoy provable convergence bounds which we briefly discuss in Sec 4.3.5
100 200 300 400 500
Time N u m b e r o f I n d i v i d u a l s (a) Susceptible (S)Exposed + Infectious (E + I)Recovered (R)Expired (X)Non-quaran. Infectious (I - Q) 0 100 200 300 400 500
Time N u m b e r o f I n d i v i d u a l s (b) L o c k - d o w n l e v e l lockdown level0 100 200 300 400 500 Time N u m b e r o f I n d i v i d u a l s Susceptible (S)Exposed + Infectious (E + I)Recovered (R)Expired (X)Non-quaran. Infectious (I - Q) (c)
Time N u m b e r o f I n d i v i d u a l s L o c k - d o w n l e v e l lockdown level (d) Figure 2:
Using
ESOP to discover the optimal point at which to initiate a lock-down . Figs 2(a),(b)consider a virus strain with an incubation of 3 days whereas Figs 2(c),(d) consider a strain with a 10 dayincubation period. The optimal initiation point of a lock-down depends strongly on viral characteristics e.g.incubation period.
ESOP is able to discover a near-optimal initiation point in both cases within very fewiterations (see Fig 3. See Sec 4.2 for details.
Time step at which lock-down is imposed P e a k E + I I n d i v i d u a l s optimumby ESOP O b j e c t i v e v a l u e (a) Time step at which lock-down is imposed P e a k E + I I n d i v i d u a l s optimum by ESOP O b j e c t i v e v a l u e (b) Figure 3:
Convergence rates offered by
ESOP . Figs 3(a),(b) show (see black curve)
ESOP ’s convergencerate in the 3 and 10 day incubation period cases in Fig 2. The optimal initiation point varies significantlywith the incubation period (33 vs 62 days). See 4.3 for details.6
Experimental Case Studies
We present case studies with
VIPER simulating an in-silico population of 20000 individuals (
ESOP scales tolarger populations too). The default attributes settings in
VIPER are given in Tab 1. Any modifications tothese are mentioned below. f epi and f eco A lock-down is represented as a 3-D vector x = ( i, p, l ) of its initiation point ( i ), period ( p ) and level ( l ). Ourobjective is to minimize f epi + f eco where f epi ( x ) is the peak of the E+I curve. We use f eco ( x ) = l · p · N to estimate job losses due to the lock-down assuming that a level- l lock-down forces an l percent of thepopulation of N = 20000 into unemployment each day for p days. Note that most natural definitions of f epi and f eco would conflict with each other since f epi would promote aggressive and sustained lock-downswhereas f eco would oppose them.The notion of f eco ( · ) used above is merely demonstrative and ESOP can instead use a more realisticdefinition of f eco that might be a non-linear function of x , by asking an economic simulator, just as it asksvalues of f epi from an epidemiological simulator such as VIPER . Also, defining the objective as an additivesum f epi + f eco of the two functions is not necessary essential for ESOP to function and users may insteadprefer other formulations e.g. f α eco · f β epi for some α, β >
0, etc.All that
ESOP requires is black-box access to values of the objective function, however it may be defined.As Fig 3 indicates,
ESOP is capable of optimizing highly non-linear functions as well. However, the algorithmwould naturally require more iterations if the objective becomes highly convoluted and sensitive to changesin the input (see Sec 5 for a brief discussion).
ESOP to Find the Optimal Initiation Point of a Lock-down
Fig 2 considers a simple case where we have decided to impose a 30 time step lock down at level 5 but areunsure when to initiate the lock down for optimal effect. The objective here is to minimize f epi alone and f eco is not considered since we have already decided the duration and intensity of the lock-down in this caseand hence resigned to a predictable economic outcome. For any initiation point i ∈ N , let f epi ( i ) be thelargest number of individuals in E and I states at any given point of time (the so called peak of the curve)if a level 5 lock-down is initiated at t = i .Fig 2(a) shows the daily count of individuals in various categories if no suppression is used. Note thelarge number of non-quarantined yet infectious individuals (I-Q) who are responsible for disease spread. Thenumber of infected (E+I) individuals peaks at around 15500 at t = 58. For Fig 2(b), ESOP was asked tosuggest when to start a 30 day lock-down at level 5. It suggested starting at t = 33 which brings the peakdown to 8200 cases, a reduction of 47%. Figs 2(c), (d) show similar results but for a viral strain that has anincubation of 10 days instead of 3 days.The results show that the optimal initiation point depends significantly on disease characteristics, e.g.,incubation period of the virus. Nevertheless, ESOP discovers a near-optimal solution, offering far superiorhealth outcomes compared to a no-lock-down scenario. Note that the number of non-quarantined infectiousindividuals continues to rise (see Fig 2(b), (d) insets) even after imposition of lock-down due to the incubationperiod of the disease.
ESOP
The topic of how fast do Bayesian optimization routines converge to the optimal solution is a subject ofintense study but one beyond the scope of this paper. Under appropriate assumptions, Bayesian optimizationroutines, within T queries to the underlying function (e.g., as ESOP queries f epi via VIPER ), are able to offera solution that is only (cid:15) T worse than the optimal solution. The sub-optimality (cid:15) T goes down with the numberof queries T at a rate T α where α > kernel
50 100 150 200 250 300
Time N u m b e r o f I n d i v i d u a l s Exposed + Infectious (E + I)Expired (X)Non-quaran. Infectious (I - Q) L o c k - d o w n l e v e l (a) lockdown level 0 50 100 150 200 250 300 Time N u m b e r o f I n d i v i d u a l s L o c k - d o w n l e v e l (b) Time N u m b e r o f I n d i v i d u a l s L o c k - d o w n l e v e l (c) lock-down Time N u m b e r o f I n d i v i d u a l s L o c k - d o w n l e v e l (d) earlier lock-downslock-down Figure 4:
Optimizing (multi-phase) lock-downs under constraints . See Sec 4.4 for details.
ESOP is able to shift from a containment strategy (e.g. in Fig 4(a)) to a mitigation strategy (e.g. in Fig 4(b))depending on the constraints. When multiple spikes are inevitable due to constraints on the lock-down (e.g.upper limits on the duration of the lock-downs),
ESOP balances the heights of the spikes to ensure thatinfections are evenly distributed among them.used by the Bayesian optimization procedure (all experiments with
ESOP use the Matern kernel). We referthe reader to [Srinivas et al., 2010] for technical details of these convergence results.Figs 3(a), (b) show that
ESOP achieved the globally optimal initiation point for the two cases consideredin Fig 2 within just 12 and 4 calls to
VIPER (see inset figures). To plot the orange curves in Fig 3,
VIPER was queried with all i ∈ [0 , ESOP does indeed reach the global optimum. However,
ESOP itself requires far fewer, e.g. 12 or4 calls to
VIPER to reach the global minimum.
ESOP to Optimize (Multi-phase) Lock-downs under Constraints
Administrative and logistical considerations put constraints on lock-downs, e.g., on when a lock-down canbe initiated or, on how long it can last. Another constraint might be taking into account previously appliedlock-downs. We demonstrate how
ESOP can be used to design lock-down schedules that are optimal amongthose that satisfy a given set of constraints. Fig 4 considers finding the optimal initiation, period and levelof a lock-down, subject to constraints.
Single-phase Lock-downs under Constraints.
For Fig 4(a),
ESOP was asked to suggest a lock-down(no constraints on duration etc). It suggested one at level 3.5 starting t = 3, lasting 69 steps, resulting in apeak of 303 infections (much smaller than the peaks in Fig 2) and 1092 predicted cases of unemployment. Wenote that f eco prevents the lock-down from going on indefinitely. The suggested lock-down seems to adopt acontainment strategy, initiating a moderate-level suppression very early on to deplete the pool of infectiousindividuals. Note that the pool of non-quarantined infectious individuals is indeed exhausted by the timethe suggested lock-down is over, preventing a second wave of infections. For Fig 4(b), ESOP was forced to8uggest a lock-down starting no earlier than t = 12 and lasting no longer than 40 days. Since infections arealready rampant by t = 12, ESOP instead adopts a mitigation strategy of starting a lock-down at t = 30for 40 steps at level 3.5, causing a peak of 7353 infections and 560 cases of unemployment. Notice that thelock-down is strategically delayed so that the second wave does not have a higher peak, thus balancing thetwo peaks indeed as dictated by f epi . Multi-phase Lock-downs.
For Fig 4(c),
ESOP was given a situation where an earlier lock-down (greenshading) had already taken place but was ineffective and left alone, would have caused a massive secondwave with a peak of 14384 (dotted orange curve).
ESOP was asked to suggest a new lock-down that starts noearlier than 10 days after the previous lock-down ended and lasting no longer than 40 days.
ESOP suggesteda second lock-down starting at t = 56 lasting 35 steps at level 4 which brings the peak down to 8447 (areduction of 40%) and causing 560 additional cases of unemployment. Fig 4(d) considers a scenario wherethe policy makes are still dissatisfied with the outcomes, and request a third lock-down starting no earlierthan 10 days after the second lock-down ended and lasting no longer than 40 days. ESOP suggests a thirdmilder lock-down at level 3.5 starting at t = 114 and lasting 25 days. This brings the peak to a much lowernumber (6052 i.e. a further 28% reduction) and 350 additional cases of unemployment. ESOP to offer less severe lock-downs?
Intuitively, if aggressive quarantining is applied, then it should be possible to avert an epidemic with milderlock-downs. Fig 5 verifies this claim by considering at the situation in Fig 4(b) (i.e. starting a lock-down noearlier than t = 12 days and lasting no longer than 40 days), but with varying quarantine aggressiveness. AsTab 1 and Sec 2 explain, individuals with viral loads over a threshold QTH get quarantined with a certainprobability profile. Using greater screening and public awareness, this profile can be altered.Fig 5(a) considers sluggish quarantining with QTH = 0.9 and BQP = 0.0 (the quarantining probabilityprofile is shown in Fig 5(a) as an inset). Almost no individuals get quarantined and despite its best efforts, ESOP is only able to offer 11338 peak infections and 800 unemployment cases using a lock-down starting at t = 32 (level 5, 40 steps). With stronger quarantining at QTH = 0.4 and BQP = 0.3, the situation improvesin Fig 5(b) where ESOP offers fewer infections (7930) and job losses (560) using a lock-down at a reducedlevel of 3.5 (starting t = 30, 40 steps). With still stronger quarantining (QTH = 0.0, BQP = 0.0) in Fig 5(c), ESOP offers a peak of 871 and 378 job losses using a lock-down lasting fewer (27) steps (starting t = 15, level3.5). Fig 5(c) reports the outcomes with even stronger quarantining at QTH = 0.2 and BQP = 0.5 where ESOP offers a peak of just 706 and 192 job losses using a lock-down lasting just 16 steps(level 3 starting t = 12). We also consider location (X, Y), susceptibility (SUS) and resistance (RST) values (see Tab 1) which arenot set uniformly but fitted to statistical data for India.
Population Clusters.
Instead of distributing individuals uniformly in the box [0 , , as we did earlier,we now distribute 34% of the population into 4 Gaussian clusters with standard deviation 0.1 (to simulatecrowded urban areas). India does have a similar proportion of urban population [Bank, 2018]. We distributethe rest 66% of the population uniformly (to simulate scattered non-urban population). Fitted Demographic Data.
Instead of distributing SUS and RST values uniformly in [0 ,
1] as we didearlier, we now distribute these according to age and co-morbidity statistics for India. [Joshi, 2020, Table1] reports that the four factors with highest risk score for CoViD-19 patients are age above 55 years, malegender, hypertension, and diabetes. We collected age-stratified statistics for all these factors from respectively[Home Affairs India, 2016, Detailed Tables], [Ramakrishnan et al., 2019, Table 1] and [Group, 2003, Table2]. After assigning age, gender, and co-morbidity values to alll individuals so as to fit the Indian statistics,individuals were assigned a point each for being male, over 55, and suffering from each of the two co-morbidities. Thus, an individual could clock up a maximum of 4 points. An individual’s SUS value was then9
50 100 150 200 250 300
Time N u m b e r o f I n d i v i d u a l s VLD of the Individual Q u a r a n t i n e P r o b . L o c k - d o w n l e v e l (a) Time N u m b e r o f I n d i v i d u a l s VLD of the Individual Q u a r a n t i n e P r o b . L o c k - d o w n l e v e l (b) Time N u m b e r o f I n d i v i d u a l s VLD of the Individual Q u a r a n t i n e P r o b . L o c k - d o w n l e v e l (c) Time N u m b e r o f I n d i v i d u a l s VLD of the Individual Q u a r a n t i n e P r o b . L o c k - d o w n l e v e l (d) Figure 5:
Effect of quarantine policy on
ESOP ’s ability to offer desirable outcomes . The areaunder the black curve in the insets roughly measures the aggressiveness of the quarantining policy. SeeSec 4.5 for details.
ESOP offers better outcomes, both epidemiologically as well as economically, and thattoo with milder lock-downs, if strong quarantining is applied.
Age Group N u m b e r o f I n d i v i d u a l s Seniority Threshold (a)
RST Value N u m b e r o f I n d i v i d u a l s (b) Figure 6:
Demographic data fitted to Indian statistics . The demographic advantage of the Indianpopulation is evident from both curves. Fig 6(a) shows that the vast majority of the population do not crossthe risk-factor forming seniority threshold whereas Fig 6(b) shows that most individuals have high RSTvalue i.e. resistance to infection, as calculated in Sec 4.6.10
50 100 150 200 250 300
Time N u m b e r o f I n d i v i d u a l s Exposed + Infectious (E + I)Expired (X)Non-quaran. Infectious (I - Q) L o c k - d o w n l e v e l (a) lockdown level 0 50 100 150 200 250 300 Time N u m b e r o f I n d i v i d u a l s L o c k - d o w n l e v e l (b) Time N u m b e r o f I n d i v i d u a l s L o c k - d o w n l e v e l (c) Time N u m b e r o f I n d i v i d u a l s L o c k - d o w n l e v e l (d) Figure 7:
ESOP with fitted demographic and location data . See Sec 4.6.1 for details. The dottedlines in all figures gives the infection curve had no lock-down been applied.decided as 0 . . × − SUS. Fig 6(b) shows the resultingdistributions.
Fig 7(a) is identical to Fig 4(b) (where
ESOP was asked for a lock-down lasting no longer than 40 daysand starting no earlier than t = 12) except that Fig 7(a) also shows (using a dotted orange line) what theinfection curve would have been, had no lock-down been applied at all. Recall that with the lock-downsuggested by ESOP , we had a peak of 7353 infections and 560 cases of unemployment. Fig 7(b) presentsthe situation where we use population clusters as well as fitted demographic data. The improvement with
ESOP ’s new suggestion is extremely significant with a peak of just 1653 infections (more than 4 × reduction)and 480 cases of unemployment, that too, using a milder lock-down at level 3.0.To ascertain whether this improvement was due to the demographic change or the population clusters,Figs 7(c),(d) present ablation studies. In Fig 7(c), only the demographic distributions of RST and SUS arefitted to Indian data but locations are kept uniformly distributed over the 2-D box [0 , as we were doingearlier. In Fig 7(d), RST and SUS values are uniformly distributed over [0 ,
1] as we were doing earlier, but34% of the population is clustered into “cities”.It is clear that changing the location distribution of the individuals alone worsens the outcome (Fig 7(d)reports a peak of 7657 that is higher than Fig 7(a)). This is to be expected since people are now crowdedinto cities . However, changing the demographics of the population to match that of India greatly improvesthe outcome (Fig 7(c) reports a peak of just 2083 and just 240 cases of unemployment). This is also to beexpected since India has an extremely favorable age structure. Since applying both changes together, as inFig 7(b), also presents a significant improvement over Fig 7(a), this seems to suggest that the demographicadvantages more than overcome the disadvantage due to crowding in cities.11
Concluding Remarks
Incorporating age stratification and climate into
VIPER and augmenting
ESOP to suggest “personalized” age-and climate-specific policies would be interesting. It would also be interesting to develop epidemiologicalmodels that simultaneously track multiple diseases with similar or confounding symptoms, such as CoViD-19and ILI (Influenza-like illnesses) as it would allow policy models e.g. testing and quarantining models tobe checked for false positive and missed detection rates. Lastly, non-linear systems with negative feedbackloops such as epidemiological models, are known to exhibit chaotic behavior [Bolker, 1993, Eilersen et al.,2020]. It is interesting how techniques like
ESOP can be adapted to handle such systems.
Given the evolving nature of the current CoViD-19 pandemic, a technique like
ESOP helps in optimallydesigning multi-phase lock-downs, thus avoiding speculation and human error. As shown in Sec 4, whenevermultiple waves of infection are unavoidable due to constraints on lock-downs,
ESOP offers lock-down schedulesthat balance the peaks of these multiple outbreaks, ensuring no peak is too high. To maximize the impact ofmethods such as
ESOP , close interaction and collaboration is needed with experts in the epidemiological andsocial sciences to better align
ESOP with professional epidemiological forecasting and economic forecastingmodels.
ESOP ’s interaction with these models is of a black-box nature which makes integration smootherand simpler.
Acknowledgements
D.D. is supported by the Visvesvaraya PhD Scheme for Electronics & IT (FELLOW/2016-17/MLA/194).P.K. thanks Microsoft Research India and Tower Research for research grants.
Conflict of Interest
The authors declare that they have no conflict of interest.
Code Availability
All code used for this study is available at the following GitHub Repository https://github.com/purushottamkar/esop
References
Julia E Aledort, Nicole Lurie, Jeffrey Wasserman, and Samuel A Bozzette. Non-pharmaceutical public healthinterventions for pandemic influenza: an evaluation of the evidence base.
BMC Public Health , 7(208), 2007.The World Bank. Urban population (% of total population) - India. https://data.worldbank.org/indicator/SP.URB.TOTL.IN.ZS?locations=IN , 2018.Benjamin Bolker. Chaos and complexity in measles models: A comparative numerical study.
IMA Journalof Mathematics Applied in Medicine & Biology , 10:83–95, 1993.Stephen Boyd and Lieven Vandenberghe.
Convex Optimization . Cambridge University Press, 2004.E. H. Bussell, C. E. Dangerfield, C. A. Gilligan, and N. J. Cunniffe. Applying optimal control theory tocomplex epidemiological models to inform real-world disease management.
Philosophical Transactions ofthe Royal Society B , 374(1776), 2019. 12aj Dandekar and George Barbastathis. Neural Network aided quarantine control model estimation of globalCovid-19 spread. arXiv:2004.02752v1 [q-bio.PE], April 2020.Denis Efimov and Rosane Ushirobira. On an interval prediction of COVID-19 development based on a SEIRepidemic model. Technical report, Inria Lille Nord Europe - Laboratoire CRIStAL - Universit´e de Lille,April 2020. hal-02517866v4.Andreas Eilersen, Mogens H. Jensen, and Kim Sneppen. Chaos in disease outbreaks among prey.
ScientificReports , 10(3907), 2020.Neil M Ferguson, Daniel Laydon, Gemma Nedjati-Gilani, Natsuko Imai, Kylie Ainslie, Marc Baguelin,Sangeeta Bhatia, Adhiratha Boonyasiri, Zulma Cucunub´a, Gina Cuomo-Dannenburg, Amy Dighe, IlariaDorigatti, Han Fu, Katy Gaythorpe, Will Green, Arran Hamlet, Wes Hinsley, Lucy C Okell, Sabine vanElsland, Hayley Thompson, Robert Verity, Erik Volz, Haowei Wang, Yuanrong Wang, Patrick GT Walker,Caroline Walters, Peter Winskill, Charles Whittaker, Christl A Donnelly, Steven Riley, and Azra C Ghani.Impact of non-pharmaceutical interventions (NPIs) to reduce COVID-19 mortality and healthcare demand.Technical report, Imperial College London, 2020.Peter I. Frazier. A Tutorial on Bayesian Optimization. arXiv:1807.02811v1 [stat.ML], July 2018.The DECODA Study Group. Age- and Sex-Specific Prevalence of Diabetes and Impaired Glucose Regulationin 11 Asian Cohorts.
Diabetes Care , 26(6):1770–1780, 2003.Prahladh Harsha, Sandeep Juneja, Preetam Patil, Nihesh Rathod, Ramprasad Saptharishi, A. Y. Sarath,Sharad Sriram, Piyush Srivastava, Rajesh Sundaresan, and Nidhin Koshy Vaidhiyan. COVID-19 EpidemicStudy II: Phased Emergence From the Lockdown in Mumbai. arXiv:2006.03375 [q-bio.PE], 2020.Office of Ministry Home Affairs India. Sample Registration System Statistical Report 2016. https://censusindia.gov.in/vital_statistics/SRS_Reports__2016.html , 2016. Accessed 14 June, 2020.Donald R. Jones, Matthias Schonlau, and William J. Welch. Efficient Global Optimization of ExpensiveBlack-Box Functions.
Journal of Global Optimization , 13:455–492, 1998.Shashank R Joshi. Indian COVID-19 Risk Score, Comorbidities and Mortality.
Journal of The Associationof Physicians of India , 68(5):11–12, 2020.Matt J. Keeling and Pejman Rohani.
Modeling Infectious Diseases in Humans and Animals . PrincetonUniversity Press, 2008.Ryder Kimball and Debanjali Bose. 11 pandemics that changed the course of hu-man history, from the Black Death to HIV/AIDS - to coronavirus. Busi-ness Insider India, March 2020. . Accessed 19 April 2020.Petra Klepac, Ramanan Laxminarayan, and Bryan T. Grenfell. Synthesizing epidemiological and economicoptima forcontrol of immunizing infections.
Proceedings of the National Academy of Sciences , 108(34):14366–14370, 2011.Tom Lindstr¨om, Michael Tildesley, and Colleen Webb. A Bayesian Ensemble Approach for EpidemiologicalProjections.
PLOS Computational Biology , 11(4):e1004187, 2015.Wladimir Lyra, Jos´e-Dias do Nascimento Jr., Jaber Belkhiria, Leandro de Almeida, Pedro Paulo M.Chrispim, and Ion de Andrade. COVID-19 pandemics modeling with SEIR(+CAQH), social dis-tancing, and age stratification. The effect of vertical confinement and release in Brazil. medRxiv:2020.04.09.20060053, April 2020. 13tamar Megiddo, Abigail R. Colson, Arindam Nandi, Susmita Chatterjee, Shankar Prinja, Ajay Khera,and Ramanan Laxminarayan. Analysis of the Universal Immunization Programme and introduction of arotavirus vaccine in India with IndiaSim.
Vaccine , 325:A151–A161, 2014.Dylan H. Morris, Fernando W. Rossine, Joshua B. Plotkin, and Simon A. Levin. Optimal, near-optimal,and robust epidemic control. arXiv:2004.02209v1 [q-bio.PE], April 2020.Aditya Nagori, Raghav Awasthi, Vineet Joshi, Suryatej Reddy Vyalla, Akhil Jarodia, Chandan Gupta,Amogh Gulati, Harsh Bandhey, Ponnurangam Kumaraguru, and Tavpritesh Sethi. Less Wrong COVID-19 Projections With Interactive Assumptions. medRxiv: 2020.06.06.20124495, 2020.Oscar Patterson-Lomba. Optimal timing for social distancing during an epidemic. medRxiv:2020.03.30.20048132, April 2020.Corey M. Peak, Lauren M. Childs, Yonatan H. Grad, and Caroline O. Buckee. Comparing nonpharmaceuticalinterventions forcontaining emerging epidemics.
Proceedings of the Nat. Acad. Sci. USA Sciences of theUSA , 114(15):4023–4028, 2017.Charles Perrings, Carlos Castillo-Chavez, Gerardo Chowell, Peter Daszak, Eli P. Fenichel, David Finnoff,Richard D. Horan, A. Marm Kilpatrick, Ann P. Kinzig, Nicolai V. Kuminoff, Simon Levin, BenjaminMorin, Katherine F. Smith, and Michael Springborn. Merging Economics and Epidemiology to ImprovethePrediction and Management of Infectious Disease.
Ecohealth , 11(4):464–475, 2014.Sivasubramanian Ramakrishnan et al. Prevalence of hypertension among Indian adults: Results from thegreat India blood pressure survey.
Indian Heart Journal , 71(4):309–313, 2019.Carl Edward Rasmussen and Christopher K. I. Williams.
Gaussian Processes for Machine Learning . MITPress, 2006.Robert Roos. CDC estimate of global H1N1 pandemic deaths: 284,000. Newsletter of the Center forInfectious Disease Research and Policy, University of Minnesota, June 2012. . Accessed 19April 2020.Tridip Sardar, Sk Shahid Nadimb, and Joydev Chattopadhyay. Assessment of 21 Days Lockdown Ef-fect in Some States and Overall India: A Predictive Mathematical Study on COVID-19 Outbreak.arXiv:2004.03487v1 [q-bio.PE], April 2020.Anna Scherbina. Determining the Optimal Duration of the COVID-19 Suppression Policy https://dx.doi.org/10.2139/ssrn.3562053 . Accessed 19 April 2020., March 2020.Snehal Shekatkar, Bhalchandra Pujari, Mihir Arjunwadkar, Dhiraj Kumar Hazra, Pinaki Chaudhuri,Sitabhra Sinha, Gautam I Menon, Anupama Sharma, and Vishwesha Guttal. INDSCI-SIM: A state-levelepidemiological model for India, 2020. Ongoing Study at https://indscicov.in/indscisim .Rajesh Singh and Ronojoy Adhikari. Age-structured impact of social distancing on the COVID-19 epidemicin India. arXiv:2003.12055v1 [q-bio.PE], March 2020.Niranjan Srinivas, Andreas Krause, Sham Kakade, and Matthias Seeger. Gaussian Process Optimizationin the Bandit Setting: No Regret and Experimental Design. In
Proceedings of the 27th InternationalConference on Machine Learning (ICML) , 2010.Davide La Torre, Tufail Malik, and Simone Marsiglio. Optimal Control of Prevention and Treatment in aBasic Macroeconomic-Epidemiological Model. arXiv:1910.03383 [econ.TH], 2019.C. P. Vyasarayani and Anindya Chatterjee. New approximations, and policy implications, from a delayeddynamic model of a fast pandemic. arXiv:2004.03487v1 [q-bio.PE], April 2020.14acco Wallinga, Michiel van Boven, and Marc Lipsitch. Optimizing infectious disease interventions duringan emerging epidemic.
Proceedings of the Nat. Acad. Sci. USA Sciences of the USA , 107(2):923–928, 2010.WHO. Coronavirus disease (COVID-19) Pandemic, April 2020. . Accessed 07 June 2020.Henry Zhao and Zhilan Feng. Identifying Optimal Vaccination Strategies via Economic and EpidemiologicalModeling.