Game theory to enhance stock management of Personal Protective Equipment (PPE) during the COVID-19 outbreak
Khaled Abedrabboh, Matthias Pilz, Zaid Al-Fagih, Othman S. Al-Fagih, Jean-Christophe Nebel, Luluwah Al-Fagih
GGame theory to enhance stock management of personalprotective equipment (PPE) during the COVID-19 outbreak
Khaled Abedrabboh , Matthias Pilz , Zaid Al-Fagih , Othman S. Al-Fagih ,Jean-Christophe Nebel , Luluwah Al-Fagih College of Science and Engineering, Hamad Bin Khalifa University, Doha 34110, Qatar Independent Researcher, e-mail: [email protected] Independent Researcher, e-mail: [email protected] NHS Health Education East of England, Cambridge, CB21 5XB, UK School of Computer Science and Mathematics, Kingston University, London KT12EE, UK*Corresponding author. Email: [email protected]
Abstract
Since the outbreak of the COVID-19 pandemic, many healthcare facilities have sufferedfrom shortages in medical resources, particularly in Personal Protective Equipment(PPE). In this paper, we propose a game-theoretic approach to schedule PPE ordersamong healthcare facilities. In this PPE game, each independent healthcare facilityoptimises its own storage utilisation in order to keep its PPE cost at a minimum.Such a model can reduce peak demand considerably when applied to a variable PPEconsumption profile. Experiments conducted for NHS England regions using actual dataconfirm that the challenge of securing PPE supply during disasters such as COVID-19can be eased if proper stock management procedures are adopted. These procedures caninclude early stockpiling, increasing storage capacities and implementing measures thatcan prolong the time period between successive infection waves, such as social distancingmeasures. Simulation results suggest that the provision of PPE dedicated storage spacecan be a viable solution to avoid straining PPE supply chains in case a second wave ofCOVID-19 infections occurs.
Novel infectious diseases pose a serious challenge to policy makers and healthcare systems.Emerging from Wuhan, China, the ongoing Coronavirus Disease 2019 (COVID-19)pandemic, caused by the Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has wreaked havoc globally. This is due to its rapid rate of transmission, its virulenceand the inability of most countries to adequately prepare for such a disease [1]. Identifiedin December 2019, the disease now has a global distribution with over 30 million confirmedcases and almost one million confirmed deaths as of September 2020 according to theWorld Health Organisation (WHO) [2]. COVID-19 is primarily transmitted throughrespiratory droplets and the WHO has identified two principal routes through whichthese are carried between people. The first mode of transmission involves a person beingin direct, close contact with someone who carries the virus (within one metre) where theybecome directly exposed to potentially infectious respiratory droplets. The second modeof transmission involves contact with fomites in the immediate vicinity of the infectedSeptember 28, 2020 1/21 a r X i v : . [ c s . C Y ] S e p erson [3]. Nguyen et al. estimated that frontline healthcare professionals (HCPs) hada 3.4 times higher risk than non-healthcare workers of contracting COVID-19, evenwhen adjusting for the probability of being tested [4]. Indeed, approximately 10% ofthe confirmed cases in China [5] and up to 9% of all cases in Italy have been amonghealthcare professionals [6] as of the date of publication of these studies. This increasedrisk does not only pose a problem for the HCPs themselves, but also poses a majorthreat to the elderly and vulnerable populations they care for, since outbreaks withinhealthcare settings are important amplifiers of infection [7]. One of the most crucial waysby which transmission of this virus (and other infectious diseases) is reduced is the useof proper Personal Protective Equipment (PPE) [7]: the WHO recommends a surgicalmask, goggles, or face shield, gown, and gloves to be worn as PPE in their COVID-19PPE guidelines. If an aerosol-generating procedure is performed, the surgical mask isreplaced with an N95 or FFP2/3 respirator which provides a greater level of filtrationthan surgical masks [8]. These guidelines are replicated globally with minor differences.In the United Kingdom (the case study reported in this paper), the most recent PublicHealth England guidelines are essentially the same [9]. Globally, medical resources,particularly PPE have come under unprecedented demand. This has led to shortagesin many countries with some rationing their use of PPE and in some cases reusingdisposable material [10]. The natural consequence of this has been the disproportionatelyhigh rate of infection amongst HCPs which in turn contributes to disease spread. Thismay not necessarily result from a national shortage of PPE but rather local shortagesresulting from inefficient distribution of resources in timely manner [11].In spite of the limited applicability of resource allocation methods, as choices have tobe made, they can aid in making the decisions that achieve the best health outcomes(see [12] for a review on the use of such methods in epidemic control). Single- andmulti-objective optimisation methods have been used frequently to address problems ofresource allocation and scheduling of purchase orders for medical supplies (see [13] for acomprehensive survey and taxonomy). Such methods are centralised in the sense thata central planner seeks to optimally coordinate supply activities for the entire system,e.g. minimise overall costs for the central planner. In contrast to this, game theory isa tool that allows decentralised decision-making. That means, different entities in thesystem can make decisions based on their individual preferences. For instance, they canschedule their orders to minimise their own costs. The decentralised approach leaves theactors more freedom and is the more applicable direction for our scenario.Game theory has been applied to a variety of subject areas such as biology [14],economics [15], computer science [16] and energy [17, 18]. In general, game theory isused to mathematically model systems of competing agents. Usually these individualsact in a selfish and rational manner. In this context, selfish can be understood asbeing only interested in their own good, i.e. an agent strives to maximise its outcomeirrespective of the outcome of others. Furthermore, rational means that there is a clearlogical reasoning behind every decision. The most widely used solution concept for anon-cooperative game is the Nash equilibrium [19]. It is achieved when none of theplayers has an incentive to change their strategy unilaterally.Despite its various applications in supply chain management (cf. [20] and the referencestherein), only few studies have applied game theory to the management of medicalsupplies. To the best of our knowledge, a 2008 study [21] (an extended version was laterpublished in [22]) was the first to develop a game theoretic approach for stockpiling ofcritical medical items. In their model, the authors propose a non-cooperative game wherehospitals stockpile critical medical resources in preparation for disasters. It is proposedthat these hospitals determine their individual stockpile levels strategically in a way thatminimises their expected total spend. Their model uses a cost function that consistsof the cost of ordering, borrowing, storage and a penalty cost in the case of shortages.September 28, 2020 2/21lthough their model proves to give some breathing space to the medical supply chainat the onset of an epidemic, it is only intended as a preparation scheme, and thereforemay suffer from inapplicability if an epidemic lasts longer than the planning periodconsidered. The model assumes a given likelihood of the occurrence and severity of apandemic, which is, in practice, heavily unpredictable. Their work was further developedin [23] by introducing network constraints, thus giving realistic hospital sharing policies.The authors find that deficits between stockpiles and demand can be reduced throughcentral stockpiling and through increasing penalties for deficits.Game theory has also been applied to the problem of drug allocation [24], wherecountries behave selfishly to minimise their expected number of infections. Unlike in [25]where the authors propose a similar resource competition among countries, their modelimitates the stochasticity of infection transmission parameters. The resulting comparisonbetween this selfish allocation scheme and a utilitarian division scheme, where a centralplanner (e.g. WHO) makes all of the allocation decisions, shows that having a centralplanner reduces the total number of worldwide infections considerably. Although theirproposed model can aid in understanding how countries, acting in their own self interest,would behave in an epidemic, it does not address the problem of resource distributionwithin a given country. [26] compares selfish vaccination coverage, which the authorscall the ‘Nash vaccination‘, with group optimal coverage, which they call the ‘utilitarianvaccination’. The authors find that the cost of vaccination would be a pivotal factor indetermining the effectiveness of Nash vs. utilitarian coverage. However, this can resultin different outcomes depending on the level of disease severity according to age, such asin the case of chickenpox.Most recently, Nagurney et al. proposed a novel decentralised model for medicalsupply chain with multiple supply and demand points [27]. In their model, selfishconsumers who have stochastic demands make purchasing decisions that minimise theirtotal expenditure. The authors assume that the disutility function of a consumerconsists of a linear cost of demand, a quadratic cost of transportation and a penalty forshortages/surpluses. Although the quantification of the shortage/surplus penalty can bedebated, the authors conclude by suggesting that shortages in supply can be avoided byredirecting global production efforts to instead adding or increasing local productioncapabilities.In this paper, we propose a game theoretic approach for managing PPE suppliesduring a pandemic. We take inspiration from the electricity storage scheduling gamedeveloped by the co-authors in [28], where a decentralised system of individually ownedhome energy systems served by the same utlity company schedule their day-ahead batteryusage over a full year. This decentralised system resulted in improved energy efficiencyfor the utility company and cost savings for the participating users.This work proposes a centralised-decentralised approach to the PPE supply chain(cf. Section 2). In the proposed architecture, healthcare facilities report their demand to acentral entity. This central entity is assumed to have the commitment power to fulfill itsorders and set the costs for PPE. Given the actions of all game participants, healthcarefacilities optimise their PPE orders by making stockpiling decisions individually. Ourapproach is centralised in the sense that cost and supply is controlled by a central entity.It is also decentralised because healthcare facilities make their stockpiling decisionsindependently. By adapting the model developed in [28] and applying it to COVID-19related PPE demand in England, we study the effects of early stockpiling as well asincreasing storage capacities on PPE supply in challenging circumstances. Additionally,we examine the impact of a putative second wave on PPE supply and investigate whetherdelaying this putative second wave can ease the challenge of fulfilling the PPE demandof healthcare facilities. Finally, insights and conclusions from this study are drawn andsuggestions regarding PPE supply in disaster management are discussed.September 28, 2020 3/21he contributions of this work can be summarised as follows:1. A game theory-based model designed to enhance stock management of PPEsupply.2. A study of PPE supply management in England during the current COVID-19pandemic demonstrating the benefits of the centralised-decentralised allocationapproach advocated by the proposed model.3. A detailed analysis of how key factors, i.e. stockpiling start date, storage capacityand the date of a putative second wave, impact PPE supply in England.4. Insights and suggestions regarding the handling of a putative second wave interms of PPE supply management.The remainder of this paper is organised as follows. In Section 2 we provide anoverview of the system architecture, explain the chosen cost function and give details onthe game formulation. Section 3 contains information on the experimental setup andpresents an analysis of the results with simulations of different scenarios. This is followedby a discussion and recommendations in Section 4 and finally, Section 5 concludes thepaper and points out future research directions. During the COVID-19 pandemic, most countries have experienced strained healthcareresources and even shortages. The efficient management of personal protective equipment(PPE) supply has proved vital in limiting the spread of the virus and in keeping thehealthcare professionals protected. In this paper, after designing a game theory-basedmodel to enhance stock management of PPE supply, we investigate as a case studyPPE provision in hospitals of the English National Health Service (NHS). England wasselected as not only has it been one of the first and most severely hit countries, but ithas also released COVID-19 data in a transparent and timely manner.
System architecture
Healthcare facilities in England are run by the NHS (NHS England). The NHS is madeup of organisational units named Trusts which mainly serve geographic areas, but canalso serve specialised functions. For a detailed description of the NHS structure inEngland, please see [29]. Prior to the outbreak of COVID-19 in England, procurementof medical supplies to NHS Trusts, including PPE, was centralised where each Trustwould ‘order’ supplies via NHS Supply Chain [30]. However, in the initial period of thepandemic, the NHS was unable to fulfil Trusts’ demand centrally leading to a chaoticalbeit temporary decentralised supply chain; some of which was unconventional [31]. On01 May, NHS Supply Chain introduced a dedicated PPE supplies channel separate toother medical supplies to address this issue [32].The NHS openly expresses its commitment to improve efficiency [33] and, via itsNHS Improvement department, it has advocated the use of modelling and novel ideas tofacilitate this [34]. In line with favoured practice by the NHS, we propose a centralised-decentralised PPE supply chain architecture, shown in Fig 1, where a central entity(controlled by the NHS) has the commitment power to set the national cost of PPEbased on the market price of the sourced PPE and to make PPE deliveries as orderedby the NHS Trusts. In this architecture, NHS Trusts, each selfishly concerned with theirown interests of satisfying their demand and minimising their cost, manage the use ofSeptember 28, 2020 4/21heir storage capabilities so that their PPE cost is minimised. After optimising theirstorage schedules independently, NHS Trusts then report their optimal PPE orders tothe central entity, which in turn fulfils their demand and charges them their share ofPPE cost.
Fig 1.
System architecture for the proposed model, showing thecentralised-decentralised approach to PPE supply chain.
Cost function
In order to ensure security of supply and avoid burdening the supply chain, suddensurges in demand should be eliminated or at least planned for. It is therefore beneficialfor the central entity to aim at having a relatively flat demand profile. This can be doneby incentivising Trusts to utilise their storage in an optimal manner so that their overalldemand is level. Accordingly, the following assumptions are made regarding the costfunction of PPE, C ( Q ), where Q is the total quantity of PPE consumption:1. The cost function of PPE C ( Q ) is strictly increasing in consumption Q ,d C d Q > , (1)i.e. the higher the consumption the higher the cost.2. The marginal cost of PPE is strictly increasing in consumption,d C d Q > , (2)i.e. the rate of rise of cost increases when consumption increases. This assump-tion can be justified by considering that an increase in demand will likely requirenew supply routes or new production facilities be established.September 28, 2020 5/21. Zero consumption yields (incurs) zero cost C (0) = 0 . (3)While many functions satisfy the above assumptions, this work adopts a quadraticcost function where C ( Q ) = aQ + bQ (4)and where a > , b ≥ Game formulation
Within our model the consumption Q (as introduced in the previous section) consistsof two separate quantities. On the one hand, there is the demand d for PPE accordingto the number of patients that are currently treated. This number cannot directly beinfluenced within the game formulation. On the other hand, there is the number of PPEkits a that are put into the storage (or taken from the storage), which is our decision tomake. Thus we have: Q = d + a . (5)While d will always be larger than zero, a can take values in a range from max ( − d, − s )to s max − s . Formally, this can be summarised by the following constraint: h ( s, a ) = (cid:20) a − ( s max − s ) − a + min( d, s ) (cid:21) . (6)That means, the largest amount of PPE that can be taken from the storage is limited bythe current demand and the amount of available stored PPE. The upper bound is dueto the finite amount of space in the storage facilities. The chosen action a then directlyaffects the stored PPE leading to the following transition equation: f ( s old , a ) = s new = s old + a . (7)Similar to [28], we propose a discrete time dynamic game, where the decisions of theplayers (the Trusts) of how much to put/take in/from the PPE storage are performedsequentially in stages. These stages (also called intervals) are defined according to theactual demand variations, i.e. if the demand changes on a daily basis, each intervalwould cover a one day period. Furthermore we introduce the state of the game (for eachinterval) and how it interacts with the decisions of the players. Overall the goal of theplayers is to minimise their own costs of PPE, i.e. their utility function which is closelyrelated to the cost function discussed in the previous section (see Eq 4). To summarise,the game consists of:1. A set of players (Trusts) N = { , . . . , N } , where N is the total number ofparticipants. All players are assumed to be selfish and rational.2. A set of intervals (days), T = { , . . . , T − } where T is the number of intervalsthat comprise the intended time cycle of the game.September 28, 2020 6/21. Scalar state variables s tn ∈ S n ⊂ IR denoting the amount of stored PPE of the n th player at stage t ∈ T ∪ { T } . Collectively, we denote the state variables ofall players at stage t by s t := [ s t , . . . , s tN ] ∈ S := S × · · · × S N ⊂ IR N . In theopen-loop information structure it is assumed that the initial state s is knownto all players n ∈ N .4. Scalar decision variables a tn ∈ H tn ( s tn ) ⊂ A n ⊂ IR (for definition of H tn seeitem (5)) denoting the usage of the stored PPE of the n th player at time t ∈ T . Collectively, we denote the decision variables of all players at stage t by a t := [ a t , . . . , a tN ] ∈ A := A × · · · × A N ⊂ IR N . Furthermore we define the schedule of PPE usage of an individual player n ∈ N as a collection of all itsdecisions in the stages of the game by a n := (cid:2) a n , . . . , a T − n (cid:3) . A strategy profile is denoted by a := [ a , . . . , a N ].5. A set of admissible decisions H n (cid:0) s n (cid:1) := { a n | h tn ( s tn , a tn ) ≤ , t ∈ T } ⊂ IR T for the n th player. The function h tn ( s tn , a tn ) has been defined in Eq 6, cap-turing the restrictions posed by the storage facilities. We denote H tn ( s tn ) := { a tn | h tn ( s tn , a tn ) ≤ } ⊂ IR6. A state transition equation s t +1 n = f tn (cid:0) s tn , a tn (cid:1) , t ∈ T , n ∈ N , (8)governing the state variables { s t } Tt =0 . The function f tn ( s tn , a tn ) is the discretisedversion of the transition equation (Eq 7), showing how a decision of the playerinfluences the state of its PPE storage for the upcoming stage.7. A stage additive utility function u n (cid:0) s n , [ a n , a − n ] (cid:1) = − C Tn (cid:0) s Tn (cid:1) − T − (cid:88) t =0 C tn (cid:0) s tn , (cid:2) a tn , a t − n (cid:3)(cid:1) (9)for the n th player, where a − n := [ a , . . . , a n − , a n +1 , . . . , a N ] denotes the deci-sions of all other players. The function C tn (cid:0) s tn , (cid:2) a tn , a t − n (cid:3)(cid:1) fulfils the assumptionsas denoted in Eq 4 capturing the costs to the n th player at the t th stage. Notethat the utility function depends only on the initial state variable s n , since thesubsequent states s tn are determined by Eq 8. The function C Tn (cid:0) s Tn (cid:1) = s Tn (10)can be interpreted as a penalty for the n th Trust that is incurred by endingup in state s Tn , i.e. its overbought PPE capacity, at the end of the schedulingperiod.The objective of rational players is to maximise their total utility, i.e. minimisetheir overall costs, over the complete time cycle T . We represent the decision problem G n of the n th player (given the actions a − n of all the other players) as the followingoptimisation problem: G n ( a − n ) given s ∈ S maximise a n u n (cid:0) s n , [ a n , a − n ] (cid:1) subject to a tn ∈ H tn (cid:0) s tn (cid:1) s t +1 n = f tn (cid:0) s tn , a tn (cid:1) ∀ t ∈ T ∪ { T } (11)Moreover, the game is referred to as { G , . . . , G N } , which denotes the simultaneousSeptember 28, 2020 7/21and linked) decision problems for all the Trusts. Within this game formulation we canformally define the Nash equilibrium (cf. Section 1) by:A strategy profile ˆ a = [ˆ a , . . . , ˆ a N ] is a Nash equilibrium for the game { G , . . . , G N } if and only if for all players n ∈ N we have u n (cid:0) s n , [ˆ a n , ˆ a − n ] (cid:1) ≥ u n (cid:0) s n , [ a n , ˆ a − n ] (cid:1) , ∀ a n ∈ H n (cid:0) s n (cid:1) . (12) Data sets and experimental setup
In this paper, we propose a game-theoretic approach for the supply of PPE to healthcarefacilities. We consider the case study of England as it was one of the countries severelyimpacted by the COVID-19 pandemic and where shortages in medical resources, especiallyPPE, were reported [10]. The players in this game structure are assumed to reporttheir demand and are committed to paying their respective invoices. The players arealso assumed to have the financial independence that inspires their selfish behaviour.Although this game is intended for independent healthcare providers, such as NHS Trustsin England, only region-level COVID-19 data are publicly available. Therefore, in thisexperiment, we assume that each of the seven NHS England regions can act selfishly ina way that represents the interests of its accommodated Trusts. We believe this has noeffect on the insights drawn from this experiment since a region is merely a group ofTrusts.
Demand profiles until 01 Aug
The region demand profiles used in this experiment were originated from daily COVID-19 occupied hospital beds data, which is available for the seven NHS England regionsand published by the UK government [36]. Fig 2 shows the peak and total (up to 01Aug) COVID-19 occupied hospital beds for each of the seven NHS England regions.We assume this information can represent the extent of the regions’ response to thepandemic. Therefore, we assume that the PPE consumption that is related to allCOVID-19 activities within these regions can be estimated from their daily COVID-19occupied hospital beds data.On 15 Apr, the UK government published its estimated PPE deliveries in Englandsince the start of the outbreak in the country: “Since 25 February 2020, at least 654million items of PPE have been supplied in this way” [37]. As this number counts apair of hand gloves as two items and includes consumables that are not considered forthis study as mentioned in Section 1, e.g. body bags and swabs, we used the number ofdelivered aprons (135 M) as our reference for consumed PPE kits. However, only 86%of these items were delivered to the NHS Trusts, whereas the rest were distributed toprimary care providers (GPs), pharmacies, dentists and social care providers, as wellas to other sectors [37]. Additionally, although these deliveries took place during theperiod between 25 Feb to 15 Apr, the first COVID-19 hospitalization only took place on20 Mar. Also, given that PPE supply chains were heavily burdened at the beginning ofthe outbreak, the UK government was led to change PPE usage guidance that healthand social care workers should use in different settings when caring for people withCOVID-19. The most notable of these was on 02 Apr when a single kit of PPE wasadvised to be used per session rather than per patient [38]. This change of guidancealong with the fact that our model is only concerned with COVID-19 related PPEconsumption have led us to assume that 50% of the PPE kits delivered to NHS Trustsbetween 25 Feb to 15 Apr were consumed for COVID-19 related activities in the periodSeptember 28, 2020 8/21 ig 2.
Total and peak occupied hospital beds with illness related to COVID-19 in theseven regions of NHS England up to and including 01 Aug 2020.between 20 Mar and 15 Apr. This results in an estimated PPE consumption of 58 Mkits for COVID-19 related hospitalisation cases. Dividing this number by the aggregateddaily COVID-19 occupied beds in England for the aforementioned period results in arough estimate of 195 PPE kits daily consumption per occupied hospital bed. Takingthis into consideration, along with the mentioned change in guidance, we were able toextrapolate the publicly available daily COVID-19 occupied beds region data into PPEconsumption by using a factor randomised between 210 and 240 kits per bed until 02Apr, and between 150 and 180 from then onwards.
Storage Capacity
According to the experience of our physician co-authors who have worked in several NHShospitals, it can be estimated that each set of 20 hospital beds is supported by a PPEstorage space of three shelves, the dimensions of each are 4 . m × . m × . m . Ourextrapolation in m to the storage capacity of the seven NHS England regions is listed inTable 4. Moreover, using the volumes supplied by PPE suppliers for bulk orders [39–42],the volume of a thousand of PPE kits, each consisting of a face shield (visor), a facemask, an apron and a pair of gloves, can be approximated to 1.35 m . Consequently,the storage capacities in terms of PPE kits can be estimated for each region as seen inTable 4.Although we are aware that our estimates are very coarse, we believe that theyrepresent a reasonable attempt at quantifying those largely unpublished quantities. InSeptember 28, 2020 9/21ny case, they are only used as a baseline for our modelling as four other values ofstorage capacities are also considered for each region, i.e. the baseline storage multipliedby a factor of 5, 10, 15 and 20. Table 4.
Storage capacities of the seven NHS England regions.
Region Maximum COVID-19occupied beds Storagein m Storage inPPE kits
East of England 1 ,
484 427 316 , ,
813 1 ,
386 1 , , ,
101 893 661 , ,
567 739 547 , ,
890 832 616 , ,
073 597 442 , , Second wave
Given that several countries have already experienced a second wave of COVID-19infections (e.g. Spain and France) [43] and several studies [44–46] predict that this ismost likely to occur in England as well, our model is used to investigate how PPEdemand should be handled in such a situation. Those simulations rely on the modelproposed by [44], where the authors predict that a second surge of COVID-19 caseshappening in England can be of a lesser magnitude than the first wave, depending onthe control measures in place (e.g. lockdown and social distancing measures), but willmost likely last longer than the first wave.Therefore, a second surge in PPE demand was introduced to the regions’ demandprofiles from 01 Aug. Fig 3 shows an examples of a second wave PPE demand profileexpected to peak in mid October 2020. As the time of occurrence of a putative secondwave is largely unknown, we considered in our experiments waves peaking at five differentdates, i.e. mid October, mid November, mid December 2020 and mid January and midFebruary 2021. As can be shown from [44], it is highly unlikely that a putative secondwave would last longer than 100 days after peaking. This has led us to assume that PPEdemand for the second wave would come to an end 100 days after the second peak.
Stockpiling starting dates
In this study, we investigate whether shortages in PPE could have been avoided if thestockpiling game had been initiated at an earlier stage. Five different starting datesare considered as each of these had some importance regarding the development of theCOVID-19 pandemic in the UK. Consequently, each could have triggered the start ofthe PPE stockpiling process:1. 20 March 2020: data on COVID-19 cases and occupied hospital beds in Englandstart to be released to the public [36].2. 11 March 2020: the WHO declared COVID-19 a pandemic [47].3. 28 February 2020: the European Union proposed to the UK a scheme tobulk-buy PPE [48].4. 07 February 2020: WHO warned of PPE shortages [49].5. 31 January 2020: the first COVID-19 case was confirmed in the UK [50].September 28, 2020 10/21 ost parameters
As mentioned in Section 2, considerable rises in PPE costs were reported during peakCOVID-19 cases in England [35]. This led us to use a quadratic cost function in ourmodel to express cost as a function of demand (see Eq 4). Since our proposed costfunction does not have any cost terms other than the cost of ordering PPE, schedulingdecisions are fairly independent of the cost parameters. In fact, the authors in [28]showed that the sensitivity of similar scheduling games in relation to cost parameters isquite low. In the experiments conducted in this study, we use 8 × − as the quadraticterm parameter and 1 × − as the linear term parameter. Usage of these parametershas resulted in a 300% increase in PPE cost between peak and average demand. Webelieve this is in line with what has been reported in the literature [35], where the costsof certain PPE items increased by up to 1000% between pre-COVID and peak COVIDperiods. Experiment implementation
In order to show the outcome of the game and whether it has the capability to ease thechallenge of securing PPE during peak demand, this experiment was implemented in twostages. In the first stage, we considered the overall period using five different stockpilingstart dates (as stated above), to the end of a putative second wave, which also has fivedifferent peak dates. Additionally, we used five different storage capacities for eachregion in order to analyse the effects of adding extra PPE storage space. This meansthat 125 games were played in the first stage, the results of which will be discussed in thefollowing sections. In the second stage of this experiment, we only examined the effectsof running the game for the duration of the second wave; with the simulation startingfrom 01 Aug, (i.e. the cut-off date of the data used to simulate demand), and ending 100days after the peak of the second wave. We have simulated five different dates for thepeak of the second wave. Furthermore, we used the same five storage capacities for thisstage as already employed before. The results of the 25 games that were simulated inthe second stage will be discussed in the following sections. In both stages, we assumethat regions start the scheduling game with an empty store.A Python code was written to find the Nash equilibrium of this game, where playerssequentially update their PPE orders to minimise their costs. The optimisation package scipy.optimize was used for each player to find their optimal scheduling decisions ateach game iteration. The game converges once the mean squared error (MSE) betweensuccessive game iterations drop below a threshold. Given that peak PPE demand is inthe order of millions of sets, we used a maximum value of 10 for the MSE to obtainaccurate results.
Analysis of outputs generated by the game
In order to illustrate the outputs generated by the game, we show results from runningone of the games in Fig 3. The outcome of the game is a series of daily decisions thateach English NHS region should take regarding their PPE requirements: order PPE, usePPE from their storage or stockpile PPE in their storage. The result of those decisionscan be visualised by monitoring the status of their storage levels. Fig 3 displays theamount of PPE sets available in the storage for each region (coloured areas), the PPEdemand (dotted line) and PPE orders (bold line). From the stockpiling date, a constantset of PPE is ordered daily, leading to storage reaching full capacity at the start of thefirst wave that triggers PPE demand. However, as storage capacity becomes rapidlyinsufficient to meet the daily demand, available (stored) PPE decreases rapidly anddaily ordering increases until reaching a plateau around the period of the peak of theSeptember 28, 2020 11/21irst wave. Eventually, storage becomes depleted and the daily order equals the dailydemand of a receding wave. Finally, when the daily demand is sufficiently low, storage isrepleted allowing English regions to face the putative second wave with full PPE storagecapacity. From then, patterns observed in terms of order and storage levels are similarto that already described for the first wave.
Fig 3.
Millions of PPE sets required daily nationally (dotted line), ordered dailynationally (bold line) according to the game, and stored in each of the seven regions ofNHS England (coloured areas). Here, stockpiling started on 28 Feb, standard storagecapacity was multiplied by five and the peak of the second wave is expected to happenin mid October. Stored PPE sets for each region is stacked on top of each other in orderto show the cumulative PPE stock of all regions to illustrate the game mechanism. Notethat the areas are scaled down by a factor of five in order to make the figure morereadable.As this model assumes that the cost of ordering PPE is quadratic with respect tothe aggregated PPE order, the game aims at producing a level of orders as constant andlow as the conditions permit. Deviations from this ideal scenario generate additionalcosts which corresponds in the proposed model to an increased challenge for the NHS atdelivering required PPE. This challenge is visible in Fig 4 where a comparison betweenthree different scenarios is shown. The first scenario can be considered as a referencescenario where NHS regions report their PPE demand to NHS without scheduling theirstorage usage. The second and third scenarios are outcomes from the game when regionsstart stockpiling on 11 Mar and 07 Feb, respectively. One should also note that storagecapacity in the third scenario is multiplied by 10. In this figure, the cumulative costfor the second and third scenarios is shown with reference to the first. As shown, a9% saving resulted from the second scenario at the end of the scheduling period whilethe third scenario resulted in a 38% saving. Since a higher cost saving means that lessfluctuations occur in demand, these figures can be directly used as a method to quantifythe challenge of securing PPE. In Fig 4, as in the remaining figures, colours have beenused to illustrate that level of challenge. The associated colour grading goes from darkred to dark green, where dark red expresses a high challenge and dark green indicates arelatively low challenge.September 28, 2020 12/21 ig 4.
Comparison between the outcome of three different scenarios, a referencescenario (without scheduling), when stockpiling starts on 11 Mar and when it starts on07 Feb and storage capacities are multiplied by 10. This shows that cumulative costsaving can represent the challenge level of securing PPE supply. A colour grading isapplied to represent the level of this challenge, where dark red is used for the highestchallenge level and green for the lowest.
Scenario simulations
Using the proposed model, a range of different scenarios were considered to assess how thechallenge in terms of fulfilling PPE demand varies according to the amount of availablestorage capacity, the stockpiling starting date and the peak date of a putative secondwave of COVID-19. In total, 125 scenarios were conducted by considering five differentvalues for each of the three parameters (see subsections above). Fig 5 summarises theoutcomes of these experiments. The figure reveals that in a two-wave pandemic, the twomost critical parameters are the stockpiling date and storage capacity. With the peak ofthe first wave being estimated at 08 Apr [51], and the second wave taking place at leastsix months after the first one, the timing of the second wave thereafter would have onlya moderate impact on easing the PPE challenge. On one hand, Fig 5 highlights thatif the storage capacity is low, an early stockpiling starting date hardly alleviates thePPE challenge and has minimal, if not negligible, cost savings (leftmost column). Onthe other hand, a high storage capacity only slightly mitigates a late stockpiling date(bottom row). Indeed, only early stockpiling and a sizeable increase in storage (top rightarea) would provide the right conditions to lower the PPE challenge overall and indeedwould result in considerable cost savings.In order to focus on the future, an additional 25 scenarios were simulated focusingon the PPE challenges associated with the available storage capacity and the peak dateof a putative second wave. Using the last date of bed occupancy data (01 Aug) as theSeptember 28, 2020 13/21 ig 5.
Challenge in terms of fulfilling PPE demand delivery according to the amountof available storage capacity (x-axis), the stockpiling starting date in 2020 (y-axis), andthe peak date of a putative second wave of COVID-19 (each cell in the grid is divided infive stripes corresponding, from top to bottom, to peak dates in October, November,December 2020, January and February 2021).simulation starting date, five different values were considered for the two parameters(see subsections above). The results of these 25 scenarios are shown in Fig 6. As shown,while the storage capacity remains a key parameter, the impact of the date of the peakof the second wave has become apparent. Indeed, as the period of study is shorter thanin the previous set of simulations and only a single wave of infection is considered, itstiming substantially affects the PPE challenge. Consequently, even if a large amount ofstorage is available (two rightmost columns), only a late 2020 peak or later would behandled well in terms of cost savings.
The model we have constructed to analyse the provision of PPE has important implica-tions for the management of the pandemic hitherto. Firstly, we have shown objectivelythat early stockpiling and increasing storage capacity would have helped to massivelyreduce the costs associated with the containment of the pandemic. Secondly, the early,sufficient and cheap provision of PPE would have had a substantial impact on theability to contain the pandemic, and protect both the most vulnerable patients and thehealthcare professionals that care for them [7].Based on these findings and the mathematics underlying our approach, we haveidentified the key elements that govern the aggregated demand (and therefore, the cost)of PPE in our scenario. The two key parameters involved in saving costs by reducing theSeptember 28, 2020 14/21 ig 6.
Challenge in terms of fulfilling PPE demand delivery according to the amountof available storage capacity (x-axis) and the peak date of a putative second wave ofCOVID-19 (y-axis).demand for PPE in the context of a pandemic that were identified using this approachare: (i) the storage capacity (for PPE) available to the health system. We have shownthat increasing the storage capacity enhances the ability to stockpile which in turnhelps flatten the demand curve (making it easier for suppliers to meet this demand in acost-effective way) ; and (ii) the date when stockpiling of PPE commences. We haveshown that earlier stockpiling would have saved costs and ensured adequate provisionof PPE for the first peak of the pandemic. These two factors will also prove crucialwhen planning for a second wave. Of secondary importance is the ability to predict thedate of the second wave. This is because, providing the second wave occurs after thenew year, the degree of cost saving that occurs diminishes to negligible levels beyondthis point in time. Using our game-theoretic approach, we have found that the NHSwould be able to achieve all the cost-saving advantages if the second wave was to occurin 2021. A practical implication of this is that policymakers may need to keep someof the restricting measures that are in place to control the pandemic until beyond thispoint in time.In order to highlight the significance of having dedicated and sufficient storage forPPE in preparation for a putative second wave, we show a comparison between differentstorage capacities in Fig 7. In this figure, the aggregate demand profile for all NHSEngland regions is shown both with and without storage scheduling for a second wave ofPPE demand that peaks in mid November. As shown, there is a substantial improvementin cost savings between the different storage capacities, especially when it is amplifiedby a factor of five and 10, resulting in an improvement in cost saving of 15.9% and7.7% respectively. This means that the challenge of securing PPE during a second wavecan be eased by a considerable amount when additional storage space is provided forSeptember 28, 2020 15/21tockpiling PPE. The figure also shows that further storage enhancement beyond thiswould result in less cost-saving improvements and would therefore have little impacton the challenge of securing sufficient PPE. This is especially visible when storage isexpanded from 15 times to 20 times the standard storage space. Indeed, enhancing thePPE storage capacity by 15 times, perhaps by using temporary storage facilities as aninterim solution, results in a relatively steady demand profile as shown in Fig 7.
Fig 7.
Effect of increasing storage capacity on the outcome of the game in terms ofaggregate PPE demand. These results are for a second wave where PPE demand peaksin mid November.
In this paper, we developed a game-theoretic model for scheduling PPE supply forhealthcare facilities. In this discrete time dynamic game, healthcare facilities makestockpiling decisions that minimise their PPE ordering cost. Our model adopts acentralised-decentralised approach to the PPE supply chain, where a central entitycontrols the PPE pricing formula, yet is committed to fulfilling the orders independentlyplaced by healthcare providers. Based on publicly available COVID-19 hospitalisationdata for NHS England regions, we performed simulations to investigate the impact ofthree key factors on PPE security of supply. These factors comprise: (i) the stockpilingstart date, (ii) the time of the peak of a putative second wave, and (iii) the amountof storage available for medical PPE. The two most critical parameters were found tobe the storage capacity and the stockpiling start date, while the timing of the secondwave has only had a moderate impact on the challenge of securing PPE. Within ourmodel we observe that enhancing PPE storage capacities by a factor of 15 is sufficientto considerably lower the peak demand at any given day and effectively minimises thestrain on the health care system.While the shortage of PPE supplies for care home workers during the first peakin the UK was a topic of great concern, since those shortages were blamed for carehome outbreaks which led to a large number of lost lives [52], we explicitly excluded Please note that this does not imply that dealing with the pandemic in this scenario is renderedtrivial. It rather represents the most favourable way to act during the crisis.
September 28, 2020 16/21his from our study due to the lack of publicly available data. Access to such datawould provide further insights into demand patterns which can potentially influencethe game. Extensions to our work can include introducing supply constraints wheredemand cannot always be fulfilled, perhaps by adding a shortage penalty to the costfunction. Also, our model assumes that healthcare facilities can predict their demandwithout forecasting errors. This can be enhanced by adding stochastic elements thatcapture the uncertainty in demand [28]. The model can also be extended by investigatingscenarios where hospitals can share their resources by having mutual agreements orusing the selfish sharing approach proposed in [53]. This could be via using a schemeto incentivise matching scheduled regional demand with the production of the supplier.Another possible extension of this research is to propose a model that finds the optimalstorage capacity for each game player.Although the above refinements may give greater insight, our model already hasthe potential to be a useful tool that can help governments and policy makers indecision-making in relation to critical PPE supplies.
Acknowledgments
The authors dedicate this work to healthcare workers that have sadly lost their livesduring this pandemic, and would like to thank all of those working on the front-lines,whether it is in hospital wards, care homes or research laboratories.
References
1. Madabhavi I, Sarkar M, Kadakol N. CoviD-19: A review. Monaldi Archives forChest Disease. 2020;90(2):248–258. https://doi.org/10.4081/monaldi.2020.1298 .2. World Health Organization (WHO). WHO Coronavirus Disease (COVID-19)Dashboard; 2020. Available from: https://covid19.who.int . Accessed 22September 2020.3. Ong SWX, Tan YK, Chia PY, Lee TH, Ng OT, Wong MSY, et al. Air, SurfaceEnvironmental, and Personal Protective Equipment Contamination by SevereAcute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) from a SymptomaticPatient. JAMA - Journal of the American Medical Association. 2020;323(16):1610–1612. https://doi.org/10.1001/jama.2020.3227 .4. Nguyen LH, Drew DA, Graham MS, Joshi AD, Guo CG, Ma W, et al. Risk ofCOVID-19 among front-line health-care workers and the general community: aprospective cohort study. The Lancet Public Health. 2020. https://doi.org/10.1016/s2468-2667(20)30164-x .5. World Health Organization (WHO). Report of the WHO-China JointMission on Coronavirus Disease 2019 (COVID-19); 2020. Availablefrom: . Accessed 01 March 2020.6. Istituto Superiore di Sanit`a (ISS). Sorveglianza Integrata COVID-19 in Italia; 2020.Available from: . Accessed 22 September 2020.September 28, 2020 17/21. European Centre for Disease Prevention and Control (ECDC). Infectionprevention and controland preparednessfor COVID-19 in healthcare settings;2020. Available from: . Accessed 22 September 2020.8. World Health Organization (WHO). Rational use of personal protectiveequipment for coronavirus disease 2019 (COVID-19); 2020. Availablefrom: https://apps.who.int/iris/bitstream/handle/10665/331215/WHO-2019-nCov-IPCPPE_use-2020.1-eng.pdf . Accessed 22 September 2020.9. Public Health England (PHE). COVID-19: infection prevention and control(IPC); 2020. Available from: . Accessed22 September 2020.10. Tabah A, Ramanan M, Laupland KB, Buetti N, Cortegiani A, Mellinghoff J, et al.Personal protective equipment and intensive care unit healthcare worker safetyin the COVID-19 era (PPE-SAFE): An international survey. Journal of CriticalCare. 2020;59:70–75. https://doi.org/10.1016/j.jcrc.2020.06.005 .11. Reuters. UK has enough protective equipment but there may be local shortages:health official; 2020. Available from: https://uk.reuters.com/article/us-health-coronavirus-britain-equipment/uk-has-enough-protective-equipment-but-there-may-be-local-shortages-health-official-idUSKCN2242SL . Accessed 22 September 2020.12. Brandeau ML. In: Brandeau ML, Sainfort F, Pierskalla WP, editors. AllocatingResources to Control Infectious Diseases. Boston, MA: Springer US; 2004. p.443–464. https://doi.org/10.1007/1-4020-8066-2_17 .13. T¨uz¨un S, Topcu YI. In: Kahraman C, Topcu YI, editors. A Taxonomy of Opera-tions Research Studies in Healthcare Management. Cham: Springer InternationalPublishing; 2018. p. 3–21. https://doi.org/10.1007/978-3-319-65455-3_1 .14. Hammerstein P, Selten R. Chapter 28 Game theory and evolutionary biology.vol. 2 of Handbook of Game Theory with Economic Applications. Elsevier; 1994.p. 929 – 993. https://doi.org/10.1016/S1574-0005(05)80060-8 .15. Cournot A. Researches Into the Mathematical Principles of the Theory of Wealth;1838.16. Shoham Y. Computer Science and Game Theory. Commun ACM. 2008;51(8):74–79. https://doi.org/10.1145/1378704.1378721 .17. Pilz M, Al-Fagih L. Recent Advances in Local Energy Trading in the SmartGrid Based on Game–Theoretic Approaches. IEEE Transactions on Smart Grid.2019;10(2):1363–1371. https://doi.org/10.1109/TSG.2017.2764275 .18. Pilz M, Naeini FB, Grammont K, Smagghe C, Davis M, Nebel JC, et al. Securityattacks on smart grid scheduling and their defences: a game-theoretic approach.International Journal of Information Security. 2020;19(4):427–443. https://doi.org/10.1007/s10207-019-00460-z .19. Nash J. Non-Cooperative Games. Annals of Mathematics. 1951;54(2):286–295. https://doi.org/10.2307/1969529 .September 28, 2020 18/210. Vasnani NN, Chua FLS, Ocampo LA, Pacio LBM. Game theory in supply chainmanagement: Current trends and applications. International Journal of AppliedDecision Sciences. 2019;12(1):56–97. https://doi.org/10.1504/IJADS.2019.096552 .21. DeLaurentis PC, Adida E, Lawley M. A game theoretical approach for hospitalstockpile in preparation for pandemics. hospital. 2008;101:1.22. Adida E, DeLaurentis PCC, Lawley MA. Hospital stockpiling for disaster planning.IIE Transactions. 2011;43(5):348–362. https://doi.org/10.1080/0740817X.2010.540639 .23. Lofgren E, Vullikanti A. Hospital stockpiling problems with inventory sharing. In:Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. AAAI’16.AAAI Press; 2016. p. 35–41.24. Sun P, Yang L, De V´ericourt F. Selfish drug allocation for containing an interna-tional influenza pandemic at the onset. Operations Research. 2009;57(6):1320–1332. https://doi.org/10.1287/opre.1090.0762 .25. Wang S, de V´ericourt F, Sun P. Decentralized resource allocation to control anepidemic: A game theoretic approach. Mathematical Biosciences. 2009;222(1):1–12. https://doi.org/10.1016/j.mbs.2009.08.002 .26. Liu J, Kochin BF, Tekle YI, Galvani AP. Epidemiological game-theory dynamicsof chickenpox vaccination in the USA and Israel. Journal of the Royal SocietyInterface. 2012;9(66):68–76. https://doi.org/10.1098/rsif.2011.0001 .27. Nagurney A, Salarpour M, Dong J, Dutta P. Competition for Medical Sup-plies Under Stochastic Demand in the Covid-19 Pandemic: A Generalized NashEquilibrium Framework. Forthcoming in: Nonlinear Analysis and Global Optimiza-tion, Themistocles M. Rassias, and Panos M. Pardalos, Editors, Springer NatureSwitzerland AG. 2020; Available from: https://ssrn.com/abstract=3657994 .28. Pilz M, Al-Fagih L. A dynamic game approach for demand-side management:scheduling energy storage with forecasting errors. Dynamic Games and Applica-tions. 2019; p. 1–33. https://doi.org/10.1007/s13235-019-00309-z .29. National Health Service (NHS) England. Understanding the New NHS -NHS England; 2014. Available from: . Accessed 22 September 2020.30. Supply Chain Coordination Limited (SCCL). NHS Supply Chain - SupplyChain Coordination Limited (SCCL) Timeline;. Available from: . Accessed 22 September 2020.31. Milmo C, Kirby D. PPE chaos: Desperate NHS trusts’ race to secure lifesaving PPEfrom unproven suppliers amid catastrophic distribution failures; 2020. Availablefrom: https://inews.co.uk/news/long-reads/ppe-chaos-investigation-desperate-nhs-trust-race-secure-lifesaving-ppe-unproven-suppliers-revealed-571094 . Accessed 22 September 2020.32. Department of Health & Social Care (DHSC). Letter to procurement leadscentralising procurement of PPE;. Available from: . Accessed 22 September 2020.September 28, 2020 19/213. National Health Service (NHS) Improvement. What we do — NHS Improve-ment;. Available from: https://improvement.nhs.uk/about-us/what-we-do .Accessed 22 September 2020.34. National Health Service (NHS) Improvement. Quality, Service Improvementand Redesign Tools: Modelling and simulation;. Available from: https://improvement.nhs.uk/documents/2136/modelling-simulation.pdf . Accessed22 September 2020.35. Hardy J. Protective equipment companies under fire for ’1,000% price in-crease’; 2020. Available from: . Accessed22 September 2020.36. Public Health England (PHE). Coronavirus (COVID-19) in the UK: Health-care in England; 2020. Available from: https://coronavirus.data.gov.uk/healthcare . Accessed 22 September 2020.37. Department of Health & Social Care (DHSC). COVID-19: personalprotective equipment (PPE) plan - GOV.UK; 2020. Available from: . Accessed 22 September 2020.38. National Audit Office (NAO). Readying the NHS and adult social care in Englandfor COVID-19. Department of Health & Social Care (DHSC); 2020. Availablefrom: . Accessed22 September 2020.39. Amazon.com. Branden Disposable Non-sterile Surgical Face Mask. Avail-able from: . Accessed 22 September 2020.40. Amazon.com. INTCO 100Pcs Disposable Nitrile Glove Powder Free Examina-tion Gloves. Available from: . Accessed 22 September2020.41. Amazon.com. PPE Visor Face Shield. Available from: . Ac-cessed 22 September 2020.42. Amazon.com. 100x Disposable White Polythene Aprons. Availablefrom: . Accessed 22 September 2020.43. Worldometer. Coronavirus Cases; 2020. Available from: . Accessed 22 September 2020.44. Keeling MJ, Hill E, Gorsich E, Penman B, Guyver-Fletcher G, Holmes A, et al.Predictions of COVID-19 dynamics in the UK: short-term forecasting and analysisof potential exit strategies. medRxiv. 2020; https://doi.org/10.1101/2020.05.10.20083683 .45. Panovska-Griffiths J, Kerr CC, Stuart RM, Mistry D, Klein DJ, Viner RM, et al.Determining the optimal strategy for reopening schools, the impact of test andtrace interventions, and the risk of occurrence of a second COVID-19 epidemicSeptember 28, 2020 20/21ave in the UK: a modelling study. The Lancet Child & Adolescent Health. 2020; https://doi.org/10.1016/S2352-4642(20)30250-9 .46. Wise J. Covid-19: Risk of second wave is very real, say researchers. BMJ. 2020;369. https://doi.org/10.1136/bmj.m2294 .47. World Health Organization (WHO). WHO announces COVID-19 outbreaka pandemic; 2020. Available from: . Accessed 22 September2020.48. Boffey D, Booth R. UK missed three chances to join EU scheme to bulk-buy PPE;2020. Available from: . Accessed 22September 2020.49. World Health Organization (WHO). WHO Director-General’s opening remarks atthe media briefing on 2019 novel coronavirus - 7 February 2020; 2020. Availablefrom: . Accessed 22 September 2020.50. Bowden E. Coronavirus in the UK – a timeline; 2020. Available from: .Accessed 22 September 2020.51. Leon DA, Jarvis CI, Johnson AM, Smeeth L, Shkolnikov VM. What can trendsin hospital deaths from COVID-19 tell us about the progress and peak of thepandemic? An analysis of death counts from England announced up to 25 April2020. medRxiv. 2020; https://doi.org/10.1101/2020.04.21.20073049 .52. Brainard JS, Rushton S, Winters T, Hunter PR. Introduction to and spread ofCOVID-19 in care homes in Norfolk, UK. medRxiv. 2020; https://doi.org/10.1101/2020.06.17.20133629 .53. Pilz M, Al-Fagih L. Selfish Energy Sharing in Prosumer Communities: A Demand-Side Management Concept. In: 2019 IEEE International Conference on Communi-cations, Control, and Computing Technologies for Smart Grids (SmartGridComm);2019. p. 1–6. https://doi.org/10.1109/SmartGridComm.2019.8909791https://doi.org/10.1109/SmartGridComm.2019.8909791