Attenuating the Impact of Integrity Attacks on Real-Time Pricing in Smart Grids
AAttenuating the Impact of Integrity Attackson Real-Time Pricing in Smart Grids
Jairo Giraldo and Alvaro C´ardenas and Nicanor Quijano
Abstract
The vulnerability of false data injection attacks onreal-time electricity pricing for the power grid markethas been recently explored. Previous work has focusedon the impact caused by attackers that compromisepricing signals and send false prices to a subset ofconsumers. In this paper we extend previous work byconsidering a more powerful and general adversarymodel, a new analysis method based on sensitivityfunctions, and by proposing several countermeasuresthat can mitigate the negative impact of these attacks.Countermeasures include adding a low-pass filter tothe pricing signals, selecting the time interval betweenprice updates, selecting parameters of the controller,designing robust control algorithms, and by detectinganomalies in the behavior of the system.
1. Introduction
The objective of the power grid is to generateand then deliver enough electric power to match thedemand of consumers. Unlike other critical infrastruc-tures like water or gas distribution networks that canaccommodate a variation in demand by storing theirresource, the power grid cannot store electricity, andthus, electricity must be generated in the exact momentthat it is consumed. If the supply of power is greaterthan the demand, this excess power is stored in theform of kinetic energy in the electricity generators,which produces an acceleration of the generator result-ing in higher rotation frequency; on the other hand,if the supply of power is not enough to match thedemand, generators will have to provide more currentto the system, and the magnetic field associated withthis increased current will slow down the generator–resulting in lower rotation frequency. All the equip-ment in the power grid is meant to operate at a specificfrequency (e.g., 60Hz) and changes in the frequencyof electricity will result in poor power quality andultimately risk of physical equipment damage and ifthe frequency deviation is large enough it may trip circuit breakers and disconnect regions of the gridcausing blackouts.To maintain a balance between optimizing the useof resources and the real-time control requirements forkeeping the frequency and voltage of the power gridat their design levels, the power grid uses a daily andhourly scheduling of generation units to match theforecast electricity load via wholesale market trans-actions. A scheduling coordinator solicits generationthrough some form of auction where lowest biddersgenerate electricity and this in turn creates an eco-nomically optimal schedule of generators. In contrastto these traditional wholesale markets (e.g., betweengeneration utilities and distribution utilities), manyretail markets (e.g., between a distribution utility andan industry consumer of electricity) have traditionallyadopted static pricing schemes such as fixed and time-of-use tariffs, under which consumers have limitedincentives to adapt their electricity consumption tomarket conditions. This lack of incentives results inhigh peak demands that strain infrastructure capacitiesand unnecessarily increase operational costs [1]. Thisapproach is inefficient, since the system infrastructureused to guarantee supply under peak hours is notcompletely used most of the time. According to theThe US Department of Energy, 10% of the wholegenerating capacity and 25% of distribution capacityis used less than the 5% of the time.In an effort to increase the efficiency of thepower grid, many retail-markets are expanding the useof demand-response programs. In their basic form,demand-response programs are a control problemwhere the control signal are the incentives (e.g., real-time pricing), or direct-load control (e.g., the utilitydirectly controlling the set-points of air conditioningsystems in specific cases) for consumers to reduce elec-tricity consumption during peak hours and to shift thisload to off-peak hours (Figure 1). Currently most of theelectricity consumers leveraging demand-response pro-grams are large commercial consumers, but the marketis expanding more and more to smaller industries andeven residential consumers. As the number of smart a r X i v : . [ c s . S Y ] O c t evices necessary to manage this market expands, thepotential attack surface of the market also increases,and therefore we need to begin considering the poten-tial impact of attackers that compromise devices andcommunication channels used in this market. Figure 1. Real-time pricing algorithms try to con-trol the load with price signals.
The security of demand response algorithms withreal-time electricity pricing was recently explored byTan et al. [1]. In their work, they consider an attackerthat has compromised a portion of the communicationchannels used to send price information to consumers,and then study the effects to the power system from scaling and delay attacks, where the prices advertisedto smart meters are compromised by a scaling factor(so consumers use the wrong prices) and by corruptedtiming information (so consumers use old prices).While this previous work is an important step forinitiating the discussion on how to analyze the impactof attacks on real-time pricing, this research has limi-tations on the way it modeled the adversary by limitingattacks to scaling and delays. In addition this previouswork did not discuss any security countermeasuresagainst attacks.In this paper we extend the work of Tan et al. [1]in several directions: • Parametric adversary models (e.g., scaling or de-lay attacks) are a common assumption to keep amathematical analysis of the problem tractable,but constraining the adversary this way is a defi-ciency in modeling realistic attackers which willnot be subject to these constraints. We model amore realistic attacker that can inject an arbitrarymodification to the price received by the con-sumer, and is not constrained to scaling or delayattacks. • Real-time pricing forms a closed-loop controlsystem, and small modifications to the signals in the closed loop made by the adversary canbe iteratively amplified by the feedback. We usesensitivity analysis to identify the attack signalsthat will be amplified and the ones that will beattenuated by the control loop, and thus, we findthe worst-possible attacks for any given bound onthe maximum price deviation introduced by anattacker at any time instant. • In addition to modeling and analyzing the impactof attacks that compromise the price signals, wealso model the effect of attacks that compromisesensor signals (smart meter electricity consump-tion reports). • We propose countermeasures based on changingthe parameter of the original controller by Tanet al. In addition, we propose an estimator anda new robust-control design that estimates theperturbation and computes a new price to attenu-ate the error between supply and demand causedby the attacker. We also introduce a low-passfilter as a solution to attenuate any high-frequencycomponent of an attack, thus guaranteeing thatour robust controller will minimize the differencebetween power generated and power consumedwith, or without attack. • Finally, while the robust controller will minimizethe impact of any attack, it will still be beneficialto notify the operator of the power grid of anypotential indicator of an attack. Thus we pro-pose an attack-detection algorithm and evaluateits effectiveness in a preliminary experiment toidentify the properties of the detector for dif-ferent controller parameters, and different attackfrequencies.
2. Related Work: Impact of Integrity At-tacks in the Power Grid
Our work falls within the scope of integrity at-tacks (or false-data injection attacks) to the sensor orcontrol signals of a cyber-physical system. Integrityattacks have been proposed as a way to analyze thevulnerability of cyber-physical systems in general andthe power grid in particular. Injecting false data tostate estimation algorithms used in bulk of the powergrid was first proposed by Liu et al. [2], and similarintegrity attacks were proposed for compromised smartmeters trying to defraud the electric utility [3].The work on integrity attacks against bad datadetectors for state estimation in the power grid hasgenerated a significant body of follow up work; forexample D´an and Sandberg [4], consider a defenderhat can secure individual sensor measurements by,for example, replacing an existing meter with anothermeter with better security mechanisms such as tamperresistance or hardware security support. Kosut et al. [5]also extend the basic false data injection attack to con-sider attackers trying to maximize the error introducedin the estimate, and defenders with a new detectionalgorithm that attempts to detect false data injectionattacks. Similar false-data injection attacks have beenconsidered for specific devices in the power grid,such as integrity attacks against the Flexible AlternateCurrent Transmission System (FACTS) [6], [7], andAutomatic Generator Control (AGC) [8]–[10]. All thisrelated work has targeted operational data of the powergrid, and is not related to electricity markets.Negrete-Pincetic et al. [11] were one of the first tostudy how false control signals can affect the socialwelfare of the electricity market. Related work by Xieet al. [12] studied how false data injection attacks canbe used to defraud bulk electricity markets by modify-ing Locational Marginal Prices (LMPs), and work byJia et al. [13] studied how false meter data in the bulkof the power grid can be used to cause the largest errorsin LMP estimation. These integrity attacks have beenstudied in the bulk electricity market and specifically,the estimation problem alone; most previous work doesnot consider how the control algorithm can be designedto minimize the impact of integrity attacks, or studiedthe feedback control loop behavior of the system underattack.
3. Preliminaries
We follow the real-time pricing model from Tan etal. [1]. This model considers a market with consumersof electricity, suppliers of electricity, and a third partyentity—an Independent System Operator (ISO)—withthe goal of matching supply and demand by setting themarket price for electricity. The general assumption isthat the ISO determines a clearing price λ k valid forthe period of time [ k · T, ( k + 1) · T ] (this is calledan ex-ante market) every T hours (e.g., T=0.5h) andannounces it to the suppliers and consumers.The electricity demand is characterized by two com-ponents: a baseline electricity consumption b k that cap-tures the electricity consumption that is independent ofthe pricing mechanism, and a price-responsive demand w ( λ k ) , which captures the amount of electricity con-sumption that can be controlled by the pricing signal λ k . The aggregated demand of all consumers is thus: d k ( λ k ) = b k + w ( λ k ) . For simulation purposes b k can be obtained fromhistorical demand curves such as those from the NewYork ISO [14].The Constant Elasticity of Own-price (CEO) hasbeen commonly adopted to characterize the total price-responsive demand. The CEO model is defined by w ( λ k ) = D ( λ k ) (cid:15) (1)where D > and (cid:15) ∈ ( − , are constants. The priceelasticity of demand is captured by (cid:15) .Similarly, for the supply of electricity, Tan et al.,propose a linear regression between supply and cost,a model they validated from the Australian EnergyMarket Operator and the the electricity market in Cali-fornia. Under these assumption the supply of electricitycan be modeled by the following equation: s ( λ k ) = pλ k + q, (2)where p and q are parameters estimated by the histor-ical market data from the area of study. The control objective of the ISO is to send pricesignals λ k to keep the error between supply anddemand of electric power e k = s ( λ k ) − d ( k, λ k ) close to zero for every time instant k . This can beseen as a control problem in which the system tobe controlled is the outcome of a market, the controlvariable is the price signal λ k and the variable that canbe measured is the error e k .The price signal λ k must be carefully designedbecause a direct feedback of the wholesale prices tothe users might cause oscillations or even instability[1], [15]. Transfer functions are a mathematical representationof linear difference (or differential) equations that al-low us to represent the system in a compact way and toevaluate the performance of the system in therms of thefrequency components of the control signals—recallthat every time series has an equivalent representation(a one to one mapping) to a function in the frequencydomain given by the Fourier transform.or our discrete-time system (where sensor and con-trol actions are taken at given time steps k separated bythe sampling period T (e.g., 30 minutes), the transferfunction for the equations modeling the dynamics ofthe system can be obtained by using the z-transform(a transform similar to the Fourier transform).In particular, we can define the transfer function ofthe price stabilization algorithm, the system, and theobservation mechanism as G c ( z ) , G p ( z ) , and H ( z ) ,respectively.To express these transfer functions it is necessaryto approximate the dynamics system at the opera-tion point λ to a linear system. Hence, followingTan et al. [1] we make the following approximationswith the Taylor polynomials of the supply s () anddemand w () : s ( λ ) (cid:39) ˙ s ( λ )( λ − λ ) + s ( λ )= ˙ s ( λ ) λ + s w ( λ ) (cid:39) ˙ w ( λ )( λ − λ ) + w ( λ )= ˙ w ( λ ) λ + w where ˙ f = dfdλ , and where we define the constant (orendogenous) terms with s = s ( λ ) − λ ˙ s ( λ ) and w = w ( λ ) − λ ˙ w ( λ ) .Therefore, the transfer functions can be defined as G s ( z ) = ˙ s ( λ ) = p, with initial condition s and G w ( z ) = ˙ w ( λ ) = D(cid:15) ( λ ) (cid:15) − , with initial condition w .The outcome of the market can be expressed as G p ( z ) = G s ( z ) − G p ( z ) . The price setting control algorithm depends on theprevious price λ k − and the observed error at thecurrent time step e k . If e k is negative, it means thatthere was more power demanded than supplied, andthus the price will increase (to motivate consumers todecrease consumption), while if e k is positive, then theprice will decrease. The precise amount of increaseand decrease of the prices at each time step shouldbe selected carefully as inadequate price updates canmake the system unstable. Tan et al. found that whenwe design a proportional gain η ∈ (0 , in thefollowing price-setting algorithm: λ k = λ k − − η ˙ s ( λ ) − ˙ w ( λ ) e k , The system will remain stable. η is in fact an importantdesign parameter for the control algorithm, and as wewill show, it can also determine the impact to theresiliency of the system under attacks. When properlyselected, it can also attenuate the impact of attacks.Assuming an observation device characterized bya one-step delay transfer function: H ( z ) = z − ,this price control mechanism can be represented bya transfer function as G c ( z ) = 2 η ˙ s ( λ ) − ˙ w ( λ ) 11 − z − .
4. Attacker Model
In contrast to one-shot attacks, where the attackerprovides false information only once [2], [16], inthis work we consider that an attacker compromisesa device or a communication channel, and has thecapability to add false information at any momentand—more importantly—repeatedly over a long periodof time.For example, most of the work on false data in-jection in state estimation finds a value d to insert atan arbitrary point in time [2], however, this previouswork does not consider the evolution of the systemdynamics over time. In this context, the question wewould like to pose from an adversarial point of viewis the following: • What is the worst attack time series d k that canaffect the system while keeping some bounds(prices will be bound by some maximum andminimum values: ∀ k d k ∈ [ d min , d max ] .Tan et al. [1] proposed an adversary model whereone attacker compromised the pricing communicationchannel between the ISO and a percentage ρ of con-sumers. They considered delay attacks and scalingattacks.In a delay attack, the compromised price is an oldversion of the price, i.e., ˆ λ k = λ k − τ , and in a scalingattack, the compromised price is a scaled version ofthe true price, i.e., ˆ λ k = γλ k .While the attacks defined above can be easily ana-lyzed from a theoretical point of view, it is not clearwhy an attacker who has compromised a commu-nications channel will select to launch these attackswhen she has the flexibility of sending any arbitrarytime series ˆ λ k she wants, even one that bears noresemblance to the original time series λ k .Furthermore, scaling attacks and delay attacks arenot strategic, and do not seek to maximize any ob-jective function from the adversary. In this work weollow the generic and more powerful adversary modelintroduced by the false data injection paper [2], andwe expand it to consider a time series. In particular,we model a compromised communication channel as ˆ λ k = λ k + d aK , where ˆ lambda k is the price informationreceived by the victim, and d ak is an arbitrary timesignal that can take any value. It is clear now thatscaling attacks and delay attacks are simple subsetsof this new attack because for every scaling or delayattack possible producing a false price information ˆ λ k , there exists an arbitrary time signal d ak that willproduce the same price ˆ λ k received by the victim.The question we now face is how to determine astrategic attack time series d ak that will try to causeas much damage as possible (i.e., that will try tomaximize the mismatch between power generated andpower consumed). One of our key insights into tacklingthis problem is the fact that for every time series,there is a one-to-one correspondence of the time seriesand its frequency (Fourier transform) representation.Therefore, instead of attempting to analyze the worsttime series d ak in time, we identify the worst-possibleattacks in frequency space.In order to provide a mathematical tool that enablesus to quantify the impact of the attack, we use sensi-tivity analysis. Sensitivity functions have been widelyused to analyze the impact of external disturbancesor parameter changes on the output of a feedbacksystem. In systems and control theory, it is well knownthat feedback can attenuate or amplify disturbances;therefore, using the frequency representation of thesystem (the transfer function), it is possible to obtainthe sensitivity function and observe the response ofthe system to a perturbation of a specific frequency ω . [17].In this work we focus our attention on two types ofadditive attacks: i) additive attack in the price informa-tion, and ii) additive attack in the sensor information.Each type of attack produces different consequencesto the system.In the next section we give the formal incorporationof the attacks against pricing signal, and in the sectionafter that we use sensitivity analysis to identify theimpact of the attacks. We assume that an amount ρ of communicationchannels are compromised, and each of these con-sumers receives the price value ˆ λ k = λ k + d ak , where d a ∈ R corresponds to the additional false information. It is necessary to identify how the inclusion of thisattack affects the system representation of the real-time problem. In particular, we need to identify howthe attack changes the transfer functions of the model(i.e., we need to characterize the new transfer functions G w ( z ) for the consumers who are unaffected, and G w ( z ) for the consumers who receive false informa-tion, as shown in Figure 2.)Let us consider the price response demand basedon the CEO model for the set of compromised nodes ρw k ( λ k , d ak ) = ρD ( λ k + d ak ) (cid:15) . In order to linearizethis model it is necessary to assume that | d k | << λ k and λ k > . As we will discuss towards the endof the paper (the attack-detection formulation), thisis a perfect assumption for an attacker that wants tominimize its chances of being detected (by causingsmall changes to the price | d k | << λ k ) but at the sametime wants to find the best way to find a small signaldeviation that will maximize the potential damage tothe system.The linearized model is described by: w (ˆ λ k ) = ρw ( λ o + d ao ) + ρ ˙ w ( λ o + d ao ) ( λ k + d ak − λ o − d ao ) +(1 − ρ )( w ( λ o ) + ˙ w ( λ o )( λ k − λ o )) We can group the price-independent terms with b k (the baseline consumption of electricity that isindependent of the price), and then also group theprice-dependent components for the transfer functions. G w ( z ) = (1 − ρ ) ˙ w ( λ o ) , (3)then corresponds to the transfer function of consumerswho receive unmodified price information, and G w ( z ) = ρ ˙ w ( λ o + d o ) , (4)corresponds to the transfer function of the victims.Under the assumption that | d k | << λ k , we can neglectthe term d o in the linearization, such that G w ( z ) = ρ ˙ w ( λ o ) . (5)
5. Sensitivity Analysis
The sensitivity function models how one input to thesystem (in our case the attack) affects another signalin the system (we are mostly interested to see how theattack affects the error in power generated minus thedemand, and to also see the impact on the prices).We start by looking at the impact that a distur-bance d a ( z ) (in the frequency space) can have on theerror E ( z ) . In particular, the sensitivity function for c ( z ) G s ( z ) G w ( z ) G w ( z ) H ( z ) ++ + -- d a ( z ) S o − w o − b k - + ++ E ( z )Λ( z ) Controller ObservationPrice responsive demand
Attack
Supply response
Victims Supply and demanderror
Figure 2. Block diagram of the real-time pricing model under attack. ω (rad/h) | S ε , d ( j ω T ) | ω (rad/h) | S λ , d ( j ω T ) | η =0.1 η =0.25 η =0.4 η =0.55 η =0.7 η =0.85 Figure 3. Left: Sensitivity of the error E ( z ) . Right: Sensitivity of the Price λ ( z ) . We can see that while theattack always amplifies the error between power generated and consumed, the price signals are actuallyattenuated (except for η = 0 . ). This sensitivity analysis uses parameters: ρ = 0 . , p = 31 , q = 917 , and T = 0 . h. The baseline consumption is b = 400 MW, which is proportional to 1 million households, and thebase demand of each consumer is b i ∈ [2 . , . KW . these two time series (denoted as S E ,d ) is the ratio E ( z ) /d a ( z ) : S E ,d = − G w ( z )1 + G c ( z ) H ( z ) G p ( z )= − ρ ˙ w ( λ )( z − z − η ) . (6)As stated before, our interest is to analyze the effectsof an additive attack in the frequency domain. Wedenote the angular frequency as ω . We then replace z = e jωT for T being the sampling period (the time interval between updating the sensor measurementsand the prices). It is important to notice that the maxi-mum frequency that an attacker can generate is limitedby the sampling period, such that ω max = π/T . Forinstance, if the sampling period is T = 0 . hours, then ω max = 2 π .In order to observe the effects an attack time-serieswith different frequency components in the output error E ( z ) , we obtain the expression | S E ,d ( e jω ) | for ω , theisturbance frequency: | S E ,d ( e jω ) | = | ρ ˙ w ( λ ) | (cid:0) sin ( ω/ − η sin ( ω/
2) + η sin ( ω/ (cid:1) / (cid:0) sin ( ω/ − η sin ( ω/
2) + η (cid:1)
10 20 30 40 50 60 70−300−200−1000100200300
Time (h) S upp l y − de m and m i s m a t c h ω (rad/s) | S ε ( e j ω ) | η =0.01 η =0.1 η =0.2 η =0.3 η =0.4 η =0.5 η =0.6 η =0.7 η =0.8 η =0.8 η =0.1 Figure 4. A smaller control parameter η will beable to attenuate the impact of high-frequencyattacks; however this will come at the cost of longerconvergence times. From this equation we can see that the percentageof compromised channels ρ has a scaling effect onthe sensitivity of the system. Moreover, the selectionof the control parameter η proposed by Tan et al. isfundamental for attenuating the effects of the attack.The left side of Figure 3 shows how the attack can beamplified (or attenuated) as a function of the frequencyof the attack signal. Clearly, the impact the supply-demand mismatch E is severe for most frequencies;however, we can also see how the control parameter η can be selected to attenuate the impact of high-frequency signals: smaller values of η will minimizethe impact of high-frequency components of the attacktime-series—this comes at the cost of a slower controlaction (as seen in Figure 4) which might not be a badidea, as changes in prices will remain small, givingconsumers more predictability in their electricity con-sumption habits.Recall that if the output E is different from zero, thenthere is over demand or over production of electricity,which can affect considerably the system (resulting in large frequency changes). Even if the price variationsare small, the output amplifies the disturbance. Thereis a trade off between the η , ρ , and the frequency ofthe disturbance. An attacker can easily take advantageof this fact, and introduce intelligently false data to aportion of the users. This information can be of smallamplitude, and hardly detected; however, the effects onthe output can be catastrophic.We can also obtain the sensitivity function withrespect to the price. This function reveals how theattack modifies the real price calculated by the ISO.The function is described by S λ,d ( z ) = − G c ( z ) G w ( z ) H ( z )1 + G c ( z ) H ( z ) G p ( z )= − ηρ ˙ w ( λ )( ˙ s ( λ ) − ˙ w ( λ ))( z − η ) , (7)and looking at the magnitude of the frequency compo-nents we obtain: | S λ,d ( e jω ) | = | ηρ ˙ w ( λ ) | ( ˙ s ( λ ) − ˙ w ( λ )) (cid:0) sin ( ω/ − η sin ( ω/
2) + n (cid:1) / . The left side of Figure 3 shows the sensitivity functionwith respect to the price for different values of η , and ρ = 0 . . With this selection of ρ , the real price changesproduced by the attack are attenuated for all η .
25 30 35 40−1−0.500.511.5
Time (h) P r i c e m od i f i c a t i on ( $ / M W h )
25 30 35 40−200−150−100−50050100150200250300
Time (h) S upp l y − de m and e rr o r ( M W ) Scaling attack ( γ =0.95)Additive attack ( ω = π )Delay attack ( τ =8) Figure 5. Modification in the price for the scalingattack with a scale parameter γ = 0 . , delayattack with a delay τ = 8 , and the additive attack d ak = sin(2 πkT ) Now that we have gained some insight into how the“frequency components” of a time series can affect thesystem, we look again at the “time domain” to applythese lessons in the analysis of attacks.
Additive attack in the price
Time (h) $ / M W h
20 22 24 26 28−100−80−60−40−20020406080100
Supply−demand mismatch
Time (h) M W
20 22 24 26 2818.51919.52020.52121.5
Price ( λ k )Time (h) $ / M W h Figure 6. Effects in the supply-demand mismatch (middle) and the price (right) for two attacks i) d ak =sin(3 / πT k ) and ii) d ak = sin( πT k/ for η = 0 .
20 22 24 26 28−1−0.8−0.6−0.4−0.200.20.40.60.81
Additive attack in the price
Time (h) $ / M W h
20 22 24 26 28−25−20−15−10−50510152025
Supply−demand mismatch
Time (h) M W
20 22 24 26 2819.819.8519.919.952020.0520.120.1520.2
Price ( λ k ) Time (h) $ / M W h Figure 7. Effects in the supply-demand mismatch (middle) and the price (right) for two attacks i) d ak =sin(3 / πT k ) and ii) d ak = sin( πT k/ for η = 0 . First we take a look at how the attacks proposedin previous work (scaling and delay attacks) compareto attacks with a frequency designed to maximizethe error between generated and consumed power.Figure 5 shows a typical example of a the effects of ascaling attack, a delay attack, and the attack targetingthe frequency where the maximization of the erroris maximum. The left hand side of the figure showsthree different attack time series: the green signal isthe scaling attack, the blue signal is the delay attack,and the black signal is the new attack designed withour sensitivity function analysis. The right hand sideof Figure 5 shows how previously proposed attacksgenerate a much smaller error than the attack designedwith the help of the sensitivity function.We now look at attacks of different frequencies and their effect on both: (1) the error in generated andconsumed power, and (2) the price signal.Figure 6 shows a high-frequency attack (black) anda low-frequency attack (red) on the left. The controlalgorithm is using η = 0 . and therefore we cansee a large error magnification caused by this controlparameter (as predicted by Figure 3). Similarly, theprice signal is also amplified for the high frequencyattack (as can be seen by the figure on the right).Figure 7 shows a high-frequency attack (black) anda low-frequency attack (red) on the left. The controlalgorithm is using in this case the parameter η = 0 . ,and it can be seen (in the middle figure) how the impactof the error between supply and demand is attenuatedwhen compared to Figure 6. The other interesting thingto observe on the figure at the right is that (as predictedby Figure 3) the price signal is attenuated for highrequencies when we use small η .
6. Modeling Attacks on Sensors
Previous work has only considered integrity attacksto the price signals, but the sensors (or smart meters)can also be compromised and can be used to send falseelectricity consumption reports to the controller. Thisnew attack model requires a new mathematical analysisof the attacks.Now we assume that the attack occurs in the infor-mation that each consumer sends to the ISO, where N sensors are compromised (Figure 8). We can ob-serve that the main difference between attacking pricesignals (i.e., control commands) and sensor signals, isthe fact that sensor signals are going to be aggregatedin this case, and therefore we do not need to modeltwo different transfer functions for compromised con-sumers, and uncompromised consumers (as we had todo when the price signal was attacked).We define n ak as the attack over one sensor, andstudy the sensitivity for one attack, and due to thelinearity of the model and the assumption of homo-geneous attacks, we scale the analysis by a factor N ∈ Z + , which is the number of compromisedsensors. G c ( z ) G s ( z ) G w ( z ) H ( z ) + - S o − w o − b k - + ++ E ( z ) Λ( z ) n a ( z ) + - Figure 8. Block diagram of the real-time pricingmodel with an attack n a on the sensor values. The sensitivity function that relates the output E ( z ) with respect to the sensor additive attack n a is givenby S E ,n = − N G p ( z ) H ( z ) G c ( z )= − N z − z − η . (8)Evaluating z = e jω , we obtain the frequency responseof the sensitivity function as | S E ,d ( e jω ) | = N (cid:0) sin ( ω/ − η sin ( ω/
2) + η sin ( ω/ (cid:1) / (cid:0) sin ( ω/ − η sin ( ω/
2) + η (cid:1) ω (rad/s) | S ε , n ( j ω ) | ω (rad/s) | S λ , n ( j ω ) | η =0.1 η =0.25 η =0.4 η =0.55 η =0.7 η =0.85 Figure 9. Sensitivity with respect to the supply-demand error (left) and price (right) for only onecompromised sensor.
Similarly, we evaluate the effects of the price vari-ations provoked by the false sensor information: S λ,n ( z ) = − N Gc ( z ) H ( z )1 + G p ( z ) H ( z ) G c ( z )= N η ( ˙ s ( λ ) − ˙ w ( λ ))( z − η ) (9) | S λ,n ( e jω ) | = N | η | ( ˙ s ( λ ) − ˙ w ( λ )) (cid:0) sin ( ω/ − η sin ( ω/
2) + n (cid:1) / . (10)Figure 9 shows the graphical representation of bothsensitivity functions. Clearly, the sensitivity functionswith respect to the sensor attack are scaled versionsof the sensitivity with additive attack in the price.Therefore, if the additive attack in the price occurswith ρ = 1 , for a total number of consumers N T >> | ˙( λ o ) | , the effects of the same attack in all the sensors( N = N T ) will produce a larger impact.In order to illustrate the different impacts for bothtypes of attacks (control signals vs. sensor signals) weassume a total number of consumers N T = 1000000 .We analyze two different cases: i) an attack in theprice information with d ak = 0 .
25 sin( πk/ /M W ,ii) an attack on the sensor measurements with n ak =0 . πk/ kW/h .Figure 10 shows the maximum value of the output E when a disturbance of the form d k = 0 .
25 sin( πk/ is introduced to the price value and to the sensors,for different values of η , and for different amount ofcompromised consumers (for both types of attacks).We can see that for the same number of communicationchannels compromised, the attacks can be actually M W Maximum supply−demand mismatch
Attack in sensors η =0.1Attack in sensors η =0.4Attack in sensors η =0.7Attack in price η =0.1Attack in price η =0.4Attack in price η =0.7 Figure 10. Comparison between the additive at-tack over the price information and over the sen-sors. much worse if the attacker decides to compromisedsensors vs. compromising the control signals.
7. Designing an Attack-Resilient Con-troller
Previous work only studied the effects of the attack,but did not propose new control mechanisms to mit-igate possible attacks. We know discuss how we canstart designing attack-resilient controllers.In order to design an attack-resilient controller, wecan leverage the fact that the ISO has historical datashowing the behavior of the system which can be usedfor learning the dynamics (parameters) of the system.Whenever the controller commands do not have theexpected effect, or when the sensor signals to notreflect the normal evolution of the system we can tryto identify these problems and design a controller thatminimizes the impact of price or sensor attacks.As the attack are unknown inputs into the system, wecan use a type of disturbance estimators. Disturbanceobservers have been studied in literature but we focusour attention in the one introduced by Kim et al. [18]for discrete-time systems.We assume that the ISO possesses the informationabout the supply-demand error E k − and we try todetect an attack using the observer (an observer isanother name for a “state estimator”).We first present the attack-resilient controller for ageneral discrete-time system, and in the next sectionwe show how to apply it to our real-time pricing model.Let us consider a generic linear discrete-time system for a sampling period T > of the form x k +1 = Ax k + Bu k + Γ d k y k = Cx k (11)where x k ∈ R n , u k ∈ R m , d k ∈ R q , and y k ∈ R l arethe state variable, the control input, the disturbance,and the measurement output, respectively. Thematrices A, B, Γ , C are of adequate dimension.For d k = ( d k , . . . d qk ) , the disturbance is slowlytime-varying, such that d ik +1 − d ik < T µ i , ∀ i =1 , . . . , q . Given a K ∈ R q × n and C = I n , the observeris described as follows z k +1 = z k + K (cid:16) ( A − I n ) x k + Bu k + Γ ˆ d k (cid:17) ˆ d k = Kx k − z k (12)Under the assumption that Γ is invertible, we canchoose K = ( I q − Φ)Γ − for Φ = [ φ , . . . , φ q ] (cid:62) , and φ i ∈ ( − , . The estimation error is then bounded by e ∞ = T µ i − | φ i | for φ ∈ (0 , , and µ i > .However, as the ISO possesses only past informationabout the state (i.e., x k − = E k − ) the estimatorhas to be slightly modified in order to estimate thedisturbance using only x k − . As a consequence, theestimation is always a delayed version of the realdisturbance. Therefore, the modified estimator is givenby ˆ d k = Kx k − − z k − z k +1 = z k − + K (cid:16) ( A − I n ) x k − + Bu k + Γ ˆ d k (cid:17) (13) Let G p = ˙ s ( λ ) − ˙ w ( λ ) and d k = d ak to simplifynotation. We can write the feedback real-time pricingproblem using a discrete-time state space representa-tion as follows E k +1 = G p u k − ρ ˙ w ( λ ) d k y k = E k (14)Note that comparing Equation (14) with Equation (11),we have A = 0 , B = G p , Γ = − ρ ˙ w ( λ ) , x k = E k and u k = λ k .Note that to compute the state estimation, it isnecessary to know Γ , which means that we would needprior knowledge about the amount of compromisednodes. Obviously, this requirement seems unrealistic as will remain unknown to the defender; however, wecan exploit a very interesting property of the estimatorwe found to perform state estimation without knowing ρ , as stated in the following proposition. Proposition 7.1:
Let us consider the disturbanceestimator described in (13) for the real-time pricingmodel in Equation (14). The rate of change of thedisturbance ∆ k = d k − d k − is bounded such that ∆ d k ≤ T µ for some constant µ and T the samplingperiod. We define ˆΓ as an approximate value of Γ and ˆ e k = Γ d k − ˆΓ ˆ d k as an error between thereal effect of the disturbance and its estimate. If K = ˆΓ − (1 − φ ) for φ ∈ ( − , , the error convergesand is bounded by | ˆ e ∞ | ≤ | Γ | T µ − | φ | Proof:
The error evolution is ˆ e k +1 = Γ d k +1 − ˆΓ ˆ d k +1 = Γ d k +1 − ˆΓ( Kx k − z k )= Γ d k +1 − ˆΓ K ( G p u k − + Γ d k − )+ ˆΓ( z k − − Kx k − + K Γ pu k − + K ˆΓ ˆ d k − )= Γ d k +1 − ˆΓ K Γ d k − − ˆΓ d k − + ˆΓ K ˆΓ ˆ d k − = Γ d k +1 − Γ d k − − ˆΓ K (Γ d k − − ˆΓ ˆ d k − ) − ˆΓ d k − + Γ d k − = 2Γ∆ d k +1 + (1 − ˆΓ K )ˆ e k − As K = (1 − φ ) / ˆΓ , in the equilibrium when ˆ e k +1 =ˆ e k − , ˆ e ∞ is bounded by | ˆ e ∞ | = 2Γ∆ d k +1 − | φ | ≤ T µ − | φ | (cid:4) Remark 7.1:
If the portion of compromised nodesis identified, then the estimation error e k = d k − ˆ d k converges and is bounded by e ∞ ≤ T µ − | φ | Similar to the previous case, estimating the distur-bance n a does not require prior knowledge of ρ dueto the fact that the attack modifies the informationthat consumers provide about its consumption and thisaffects directly the demand.The state estimation can then be given by z k +1 = z k − + K ( −E k − + G p λ k + Γˆ n ak )ˆ n ak = K E k − − z k − (15) where Γ = − . As it was proven before, ˆ e k is boundedindependent of ˆΓ . This fact will be useful to detectattacks without knowing its exact location, i.e., withoutknowing if the attack is modifying the price or thesensor information, and we can do this using thesame estimator (of course if the attacker controls both:all price signals, and all sensor signals then there isnothing we can do as we have lost any hope of gettingsituational awareness from the system).Before introducing the proposed detection mech-anism, we will show how to improve disturbancerejection of the system using the estimator. It is possible to modify the disturbance rejectionusing an add-on compensator in the controller of theform u k = u nom − B − Γ ˆ d k = λ k − Gp − ˆΓ ˆ d k where u nom is the controller under normal conditions.The mismatch between the supply and the demandis described by E k +1 = G p λ k + Γ d k − ˆΓ ˆ d k . Clearly, if ˆ e k is small, disturbances are attenuated.Including the robust controller in the system pro-duces an improvement in the estimation, leading tothe following result. Proposition 7.2:
For the RTP system under ad-ditive attack, and the proposed robust controller ˆ λ k = λ k − G − p ˆΓ ˆ d k , where ˆ d k is estimated accordingto (13), the estimation error is bounded by | ˆ e ∞ | ≤ | Γ | T µ − | φ | Proof:
The proof is similar to Proposition 7.1, but because ˆ λ k = λ k − ˆ G k is the input, it leads to ˆ e k +1 = Γ d k +1 − ˆΓ K ˆ e k − ˆΓ ˆ d k = Γ∆ d k +1 + ˆ φe k As K = (1 − φ ) / ˆΓ , in the equilibrium when ˆ e k +1 =ˆ e k − , ˆ e ∞ is bounded by | ˆ e ∞ | = Γ∆ d k +1 − | φ | ≤ Γ T µ − | φ | which satisfies the proof. (cid:4) According to Proposition 7.2, the z transform of theerror ˆ e k is e ( z ) = G ( z − z − φ d ( z ) and the new sensitivity function (cid:101) S (cid:15),d ( z ) can be ob-tained as follows E ( z ) = − η E ( z )(1 − z − ) z + ˆ e ( z ) Dividing by d ( z ) and factorizing we obtain E ( z ) d ( z ) = ˆ S (cid:15),d = G ( z − ( z − φ )( z − η ) (16). Figures 11 and 12 illustrate the maximum supply-demand mismatch with the robust controller and thenominal controller for an attack of the form sin( ωkT ) .Note that for frequencies below 1.9, the attack attenu-ation is better than without the add-on compensator;however, for high frequencies, the inclusion of thecompensator increases the impact of the attack. ω (rad/h) M W Maximum supply−demand mismatch
Robust control and η =0.1Robust control and η =0.9Nominal control and η =0.1Nominal control and η =0.9 Figure 11. Maximum supply-demand mismatchwith the nominal controller and with the robustcontroller. We can see that our robust controllerdesign can attenuate the errors caused by theattack; however, at high frequencies it increasesthe errors. In a later section we propose the use oflow-pass filters to prevent an attacker from usinghigh-frequency attacks.
We can obtain the frequency at which the robustcontroller stops improving the system response underattacks. To do this, we need to find ω c = ω : | S (cid:15),d ( jωT ) | = | ˆ S (cid:15),d ( jωT ) | . Taking Equation (6) and(16), we have that | z − | = | z − φ |
20 25 30 35 40 45−1−0.8−0.6−0.4−0.200.20.40.60.81
Additive attack in the price information
Time (h) $ / M W h
20 25 30 35 40 45−15−10−5051015
Supply−demand mismatch
Time (h) M W Nominal controlRobust control
Figure 12. Supply-demand mismatch for an attackof d k = sin( π/ kT ) . We can see that our newrobust controller attenuates the supply-demandmismatch better than the nominal controller. Replacing z = e jωT and solving for w c , we obtain ω c = 1 T arccos (cid:18) φ − φ − (cid:19) This relationship is shown in Figure 13. Note thatthis frequency depends on φ . ω c is larger when φ approaches − . However, the pole corresponding to z − φ would approach the unit circle, compromisingthe exponential stability of the system. −1 −0.5 0 0.5 100.511.522.533.5 φ ω c Figure 13. Cut-off frequency (for the usefulness ofthe robust controller) depends on φ . According to Figure 13, the maximum frequencywhere our proposed robust controller can improve theperformance and attenuate the supply-demand errorunder our attacks is ω c = π/ (2 T ) . The maximumrequency at which an attacker can generate anadditive attack is ω max = π/T . So, there is a range offrequencies that are amplified by the robust controller.To mitigate this issue we propose the use of a digitallow-pass filter in the smart meters, in order to filterprice information with high frequency components.The same filter has to be implemented by the thirdparty that calculates the price. The cut-off frequencyis given by ω c . Therefore, for our robust controller towork, we conclude that every frequency greater than ω c should be attenuated by the low-pass filter.We now compare the performance and robustnessof the real-time pricing model including a digital IIRlow pass filter (Figure 14). The mathematical analysisfor designing the filter is omitted because this topic isout of the scope of this manuscript. The reader onlyneeds to know that there are filters that can eliminatehigh-frequency components of any signal.Admittedly we could also have proposed deployinglow-pass filters at the beginning of the paper (beforethe design of the robust controller), and we could alsohave seen a significant improvement in minimizing themaximum error that an attacker can create. However,as Figure 11 shows, the performance of the robustcontroller for low-frequency signals is still better thanthe performance of the controller proposed in previouswork; therefore with the combination of low-passfilters and robust controllers we seem to have obtainedan ideal combination of protection mechanisms.
20 22 24 26 28−40−30−20−10010203040
Supply−demand mismatch
Time (h) M W
20 22 24 26 28−1−0.8−0.6−0.4−0.200.20.40.60.81
Additive attack in the price information
Time (h) $ / M W h Nominal control(cid:9)Robust controlRobust + Filter
Figure 14. Supply-demand mismatch for d k =sin(2 πkT ) using the nominal controller, the robustcontroller, and the robust+filter control. In summary, the combination of a low-pass filterdeployed at all smart meters (or all devices receivingprice signals) in addition to a robust controller seemsto be the best solution to attenuate any type of attackagainst our system. We believe this is one of thefew instances where a proposed attack-resilient control algorithm does not pose significant negative perfor-mance impacts on the system (when the system is notunder attack), but we plan to continue evaluating ouralgorithm in other realistic real-time pricing settings toidentify any limitations.
8. Detection mechanism
We have designed a new real time pricing algorithmthat not only assures stability, but also minimizes theimpact of attacks. However, in practice, while we haveattenuated the attack, it would still be desirable toknow if we are under attack or not, so we can removecompromised devices from our system.The ISO calculates a clearing price each time period,but even in the presence of an attack, the price changesare small (see Figures 6 and 7). However, the stateestimator used in our robust controller can give infor-mation about the presence of an attacker, by analyzingthe statistical behavior of the state estimator over longperiods of time.The detection mechanism that we propose is basedon the accumulation of the rate of change of the esti-mated signal ˆ d k . This is known as the non-parametricCUSUM change detection statistic, and it is definedas: S = 0 S k +1 = ( S k + | ˆ G ˆ d k − ˆ G ˆ d k − | − α k ) + (17)where S k is the accumulated impact of the disturbance,and α k is the rate of change of S k under normal condi-tions (without attacks). The use of the the error ˆΓ ˆ d k isdue to the fact that the ISO does not have knowledgeabout Γ . An attack is detected when S k > δ . δ hasto be selected such that the number of false alarms islow. As it is based on the rate of change, then highfrequency attacks are detected faster. We assume a populated area with 1 million house-holds, each one receiving information about the priceevery 30 minutes. To improve the realism of thesimulations, we assume that the parameters D and b k change each time period according to a half-hourlybaseline demand profile provided by AEMO from July21st to 27th, in NSW, Australia. The baseline loadper house is a scaled version of the real whole NSWregion. The parameters of the linear CEO model are p = 31 and q = 917 during the simulation time.We assume that an attack is launched and modifiesthe price information of 50% of the households. Thettack is of amplitude . $ /MWh, and a frequency ω .The estimation is based on prior information ofthe baseline load. However, we assume an error inthe real-time baseline consumption, such that the ISOcalculates the estimation and the robust control basedon an approximate load profile, and not the real timeconsumption. Despite that limitation, the detectionalgorithm is able to detect an attack when a thresholdis achieved.Figure 15 illustrates the time that it takes to detectan attack depending on the frequency of the attackfor a threshold δ = 10 , which is selected with resultswithout attack in order to avoid false alarms. Note thatfor high frequency, the time of detection is low, whichis an advantage in order to start a scan in the smartmeters and find the victims of the attack. Time of detection (h) ω (r ad / s ) η =0.1 η =0.4 η =0.7 Figure 15. Detection for different frequency valuesof the attack.
We can also observe that the detection time does notdepend on the η .Our work on detection is preliminary, and in futurework we plan to identify the tradeoffs the attacker willface when deciding to launch attacks that maximize theerror between power generated and consumed whilealso maintaining the attack undetected.
9. Conclusions
In this work we used the theory from sensitivityanalysis to understand how previously proposed attackscould be generalized and evaluated in a formal setting.In particular we showed how to find better attacks thanpreviously proposed, and how to design robust controlsystems that can mitigate a large number of attacks. We also found that the design of the price adjustmentmechanism is fundamental in the resiliency of thesystem. In particular, low values of η reduce the effectof the attacks on both the prices and sensors.Another of our contributions was the model tosensor attacks, and how they can have potentially moredamaging effects than attacks on the pricing signal.We also proposed an attack-resilient controller andseveral mitigation mechanisms, such as the use oflow-pass filters to prevent high-frequency signals, andattack detection mechanisms. We believe we are oneof the few research papers focusing on the importantaspect of designing robust control algorithms againstfalse data injection, as much of the previous work tendsto focus on state estimation but does not consider thecontrol actions of the system under attack, and how todesign an controller that mitigates these attacks.Our results show principled ways to use controltheory in the design of attack-resilient cyber-physicalsystems. In general we believe that a well-designeddefense-in-depth mechanism for cyber-physical sys-tems will have to leverage not only information se-curity expertise, but control theory to detect, respond,and reconfigure systems that can survive partial com-promises.Successfully compromising computers and embed-ded systems participating in controlling the power gridis only the first step to a successful attack. To havea predictable physical modification to the power grid(e.g., strategically manipulating voltages, or loads),the attacker needs to understand how control systemsoperate.Defenders that leverage only information securitymechanisms in their protection strategy will have lim-ited success against these sophisticated attackers. Todevelop a defense-in-depth security strategy, defendersneed to incorporate control models of the power gridto understand the vulnerabilities and fragility of thesystem they are trying to protect (e.g., not all compro-mised devices can drive a system to an unsafe state),and to design attack-resilient control algorithms thatcan survive a partial compromise of the system. Ourwork shows a direction of how to pursue this goalfurther and in general we hope these formalisms canhelp mitigate attacks not only against the power-gridbut against other cyber-physical systems.One interesting area of future research that wedid not address in this paper are the possible attackstrategies that can be achieved by combining attacksto both: sensors and control signals. All our modelsassumed the attacker compromised either the pricesignals, or the sensor signals, but not both. It is clearhat if the attacker controls all control signals and allsensor signals then there is nothing we can do, but ifthe attacker has partial compromise of controllers andsensors, then the defender might still be able to designa robust algorithm that attenuates the attacks. We planto look into this area in future work. References [1] R. Tan, V. Badrinath Krishna, D. K. Yau, and Z. Kalbar-czyk, “Impact of integrity attacks on real-time pricingin smart grids,” in
Proceedings of the 2013 ACMSIGSAC conference on Computer & communicationssecurity . ACM, 2013, pp. 439–450.[2] Y. Liu, M. K. Reiter, and P. Ning, “False data injectionattacks against state estimation in electric power grids,”in
CCS ’09: Proceedings of the 16th ACM conferenceon Computer and communications security . New York,NY, USA: ACM, 2009, pp. 21–32.[3] D. Mashima and A. A. C´ardenas, “Evaluating electric-ity theft detectors in smart grid networks,” in
Researchin Attacks, Intrusions, and Defenses (RAID) . SpringerBerlin Heidelberg, 2012, pp. 210–229.[4] G. D´an and H. Sandberg, “Stealth Attacks and Protec-tion Schemes for State Estimators in Power Systems,”in
First IEEE Smart Grid Commnunications Conference(SmartGridComm) , October 2010.[5] O. Kosut, L. Jia, R. Thomas, and L. Tong, “MaliciousData Attacks on Smart Grid State Estimation: AttackStrategies and Countermeasures,” in
First IEEE SmartGrid Commnunications Conference (SmartGridComm) ,October 2010.[6] L. Phillips, M. Baca, J. Hills, J. Margulies, B. Tejani,B. Richardson, and L. Weiland, “Analysis of operationsand cyber security policies for a system of cooperatingflexible alternating current transmission system,” Dec.2005.[7] S. Sridhar and G. Manimaran, “Data integrity attackand its impacts on voltage control loop in power grid,”in
Prc. IEEE Power Energy Soc. General Meeting , Jul.2011.[8] P. Mohajerin Esfahani, M. Vrakopoulou, K. Margellos,J. Lygeros, and G. Andersson, “Cyber attack in a two-area power system: Impact identification using reacha-bility,” in
American Control Conference (ACC), 2010 ,30 2010-july 2 2010, pp. 962 –967.[9] P. Esfahani, M. Vrakopoulou, K. Margellos, J. Lygeros,and G. Andersson, “A robust policy for automaticgeneration control cyber attack in two area powernetwork,” in
Decision and Control (CDC), 2010 49thIEEE Conference on , dec. 2010, pp. 5973 –5978. [10] S. Sridhar and G. Manimaran, “Data integrity attacksand their impacts on SCADA control system,” in
Prc.IEEE Power Energy Soc. General Meeting , Jul. 2010.[11] M. Negrete-Pincetic, F. Yoshida, and G. Gross, “To-wards quantifying the impacts of cyber attacks in thecompetitive electricity market environment,” in , June 2009.[12] L. Xie, Y. Mo, and B. Sinopoli, “False Data InjectionAttacks in Electricity Markets,” in
First IEEE SmartGrid Commnunications Conference (SmartGridComm) ,October 2010.[13] J. Liyan, R. J. Thomas, and L. Tong, “Impacts ofmalicious data on real-time price of electricity marketoperations,” in
Energy Market (EEM), 2011 8thInternational Conference on the European , may 2011,pp. 250–255.[16] A. Teixeira, G. Dan, H. Sandberg, R. Berthier,R. Bobba, and A. Valdes, “Security of smart distribu-tion grids: Data integrity attacks on integrated volt/varcontrol and countermeasures,” in
Proceedings of theAmerican Control Conference (ACC) , 2014.[17] J. C. Doyle, B. A. Francis, and A. R. Tannenbaum,
Feedback control theory . Courier Dover Publications,2013.[18] K.-S. Kim and K.-H. Rew, “Reduced order disturbanceobserver for discrete-time linear systems,”