Distributed Demand Response and User Adaptation in Smart Grids
aa r X i v : . [ c s . N I] J u l Distributed Demand Response and User Adaptationin Smart Grids
Zhong Fan,
Senior Member, IEEE
Abstract —This paper proposes a distributed framework fordemand response and user adaptation in smart grid networks.In particular, we borrow the concept of congestion pricing inInternet traffic control and show that pricing information is veryuseful to regulate user demand and hence balance network load.User preference is modeled as a willingness to pay parameterwhich can be seen as an indicator of differential quality of service.Both analysis and simulation results are presented to demonstratethe dynamics and convergence behavior of the algorithm.
Index Terms —Smart grid, demand response, pricing, utility.
I. I
NTRODUCTION
A smart grid is an intelligent electricity network that in-tegrates the actions of all users connected to it and makesuse of advanced information, control, and communicationtechnologies to save energy, reduce cost and increase relia-bility and transparency. Recently, many countries have startedmassive efforts on research and developing smart grids. Forexample, the smart grid is a vital component of PresidentObama’s comprehensive energy plan: the American Recoveryand Investment Act includes 11 billion USD in investmentsto “jump start the transformation to a bigger, better, smartergrid”.In electricity grids, demand response (DR) is a mechanismfor achieving energy efficiency through managing customerconsumption of electricity in response to supply conditions,e.g., having end users reduce their demand at critical timesor in response to market prices. In the future smart grid, thetwo way communications between energy provider and endusers enabled by advanced communication infrastructure (e.g.,wireless sensor networks and power line communications) andprotocols will greatly enhance demand response capabilitiesof the whole system. In contrast to the current simple time-of-use (TOU) pricing (e.g. peak time vs. off-peak time), itcan be envisaged that a more dynamic, real-time adaptationto market prices would not only enable consumers to savemore energy and money, as well as manage their usage pref-erences more flexibly, but also facilitate the grid move closertowards its optimal operating point. For a recent overview ofchallenges and issues of enabling communication technologiesin this area, please refer to [1]. The authors of [2] argue thatdemand response and distributed energy storage can be seen asdistributed energy resources and are main drivers of smart grid.While DR can help the industry to achieve market efficiency
Z. Fan is with Toshiba Research Europe Limited, TelecommunicationsResearch Laboratory, 32 Queen Square, Bristol, BS1 4ND, UK. e-mail:[email protected] and operational reliability, there are also challenges ahead inimplementing DR under smart grid and market paradigms.There are a few papers recently on smart grid DR using loadscheduling. In [3], user preferences are taken into account withthe concept of discomfort level and an optimization problemis formulated to balance the load and minimize the user in-convenience caused by demand scheduling. Several ideas fromthe distributed computing area such as makespan have beenintroduced to energy consumption optimization. Similarly, in[4], an energy consumption scheduling problem is establishedto minimize the overall energy cost. Techniques similar tothose used in wireless network resource allocation have beenapplied here to solve the underlying optimization problem. Inboth works, the user demands are known beforehand and theoptimization problem is solved in numerical iterations.In this paper, we consider a fully distributed system wherethe only information available to the end users is the currentprice which is dependent on the overall system load. Based onthis information, the users try to adapt their demands so as tomaximize their own utility. There is no central control entity.Inspired by the well-established work on congestion pricing inIP networks, we propose a simple adaptation strategy based onprice feedbacks and show that it is very effective in achievingdemand response.The rest of the paper is organized as follows. Section 2introduces our DR model and the adaptation algorithm. Wepresent some simulation results in Section 3. Conclusions andfuture work are presented in Section 4.II. D
EMAND RESPONSE MODEL
A. Congestion pricing background
In this paper we propose to apply the principle of congestionpricing in IP networks to demand response in the electricitygrid. In their seminal paper [5], Kelly et al. have proposedthe proportionally fair pricing (PFP) scheme in which eachuser declares a price per unit time that he is willing to payfor his flow. In that sense the network capacity is sharedamong the flows of all users in proportion to the prices paidby the users. It has been shown in [5] that in a weightedproportionally fair system where the weights are the prices theusers pay per unit time, when each user chooses the price thatmaximizes the utility she gets from the network, the systemconverges to a state where the total utility of the network ismaximized. In other words, in an ideal environment, the PFPproposal is able to decentralize the global optimal allocationof congestible resources. Another important result of [5] isthat rate control (such as TCP) based on additive increase and multiplicative decrease achieves proportional fairness. It hasbeen proved in [6] that the decentralized congestion controlmechanism is stable even under arbitrary network topologiesand heterogeneous round trip times (feedback delays).In Kelly’s approach, the philosophy is that users who arewilling to pay more should get more. As the network makesno explicit promises to the user, there is no need for overprovisioning in the core of the network. One implementationof PFP is to give control to end systems (users). In this scheme,the TCP algorithm is modified to incorporate congestion pricesby means of protocols like explicit congestion notification(ECN) [7]. Upon receiving feedback signals, f ( t ) , which arerelated to shadow prices (in terms of packet marks), the usersare free to react as they choose, but will incur charges whenresources are congested. An end system can adjust its rate x ( t ) using a willingness to pay (WTP) parameter w : x ( t + 1) = x ( t ) + α ( w − f ( t )) , (1)where α affects the rate of convergence of the algorithm.In [8], explicit prices instead of marks are fed back to theend users as incentives and users adapt their rates accordingly.It has been shown that the system converges to an optimalallocation of bandwidth: the users’ price predictions convergeto the actual price and their bandwidth allocations convergeto levels which equalize their marginal utility of bandwidth tothe price of bandwidth. B. The DR model and user adaptation
We consider a discrete time slot system where N usersshare some energy resources. In each time slot n , user i has ademand of x i ( n ) (e.g. hourly energy consumption if the timegranularity is one hour). The unit price of energy in a timeslot is a function of the aggregate demand: p ( n ) = f ( N X i =1 x i ( n )) . (2)The price function (spot market price) can be of the followingform [3][9]: f ( x ) = a ( xC ) k , (3)where a and k are constants, and C is the capacity of themarket.Each user i is associated with a utility function u i ( x i ( n )) in time slot n , which is a concave, non-decreasing function ofits demand. A typical logarithmic utility function is given by[5]: u i ( x ) = w i log x, (4)where w i is the willingness to pay parameter. Hence user i chooses its demand x i ( n ) to seek to maximize u i ( x i ( n )) − x i ( n ) p ( n ) . (5)We would like to elaborate on a few assumptions madein the above model. Firstly, in our model the demand x is acontinuous variable, which may not be realistic in practice. Forexample, in real life, the daily usages of a washing machineand a dryer are (fixed as) 1.49 kWh and 2.50 kWh respectively. Here the adaptation of x can be seen as an action of loadscheduling: for example, re-scheduling a dryer operation fromtime slot n = 1 to slot n = 3 leads to x (1) reduced by2.50 kW per hour and x (3) increased by 2.50 kW per hour.Secondly, how to characterize user preference is an open issue.For instance, a user may prefer his washing done at 6pm whichis a typical peak time. To some extent, this preference can bereflected in the WTP parameter w in (4): when a user is willingto pay more, he/she can have a higher demand. However, thedelay (or waiting time) incurred due to rescheduling is notconsidered in this model. Thirdly, as pointed out in [5], log-arithmic utility functions lead to proportional fairness. Thereare other types of utility functions available corresponding todifferent fairness criteria, e.g. u i ( x ) = w i x β − β , β < asproposed in [8]. How to choose a most suitable utility functionin DR applications is an open issue, e.g. how to factor in thewaiting time and user discomfort level [3].User i adapts its demand according to the following equa-tion: x i ( n + 1) = x i ( n ) + α i ( w i − x i ( n ) p ( n )) , (6)where α i is a parameter that controls the rate of convergenceof the algorithm. It is clear that the user adjusts her demandaccording to the price information ( p ( n ) ) and her own will-ingness to pay preference ( w ).To show that the above adaptation converges to the useroptimum, let us assume that the equilibrium price is q . Thenby solving u ′ i ( x i ( n )) = q , we have the optimal demand x ∗ i as x ∗ i = w i q . (7)Given (6), the error of demand estimate, e i ( n + 1) , is givenby e i ( n + 1) = x i ( n + 1) − x ∗ i = x i ( n ) + α i ( w i − x i ( n ) q ) − w i q . (8)Then it follows that e i ( n + 1) = (1 − α i q )( x i ( n ) − w i q ) = (1 − α i q ) e i ( n ) . (9)Therefore e i ( n ) is a geometric series, and when | − α i q | < , lim n →∞ e i ( n ) = 0 . This has established that with properlychosen α i , the adaptation will converge to the optimum.Following [5], it is also straightforward to establish the globalstability of the algorithm in a differential equation form (10)using an appropriate Lyapunov function. ddt x i ( t ) = α i ( w i − x i ( t ) p ( t )) . (10) C. Implementation considerations
In a residential energy management scenario, we envisagethat each user in our model is represented by an entity orsoftware agent called home energy manager (HEM) at aconsumer’s home. Appliances in the home are equipped withsmart meters, and they communicate with HEM via low powerwireless such as ZigBee. HEM is further connected to the grid(supplier) via either wired or wireless links. Based on the priceinformation it receives, HEM calculates demand in the next
Fig. 1. Distributed demand response time slot and distributes it to different appliances. The overallarchitecture is shown in Figure 1.We note that some appliances like refrigerator and heatinghave hard consumption scheduling requirements, while otherssuch as washing machine have soft requirements [4]. WhenHEM has to shift the demand to another time slot, it mayapply only to soft appliances. For example, HEM obtains x ( n + 1) based on (6) with WTP parameter w . If it cansatisfy the demand from the hard appliances (denoted by h ), itcan re-schedule the demand from some of the soft appliances(denoted by s ) so that x ( n + 1) = h ( n + 1) + s ( n + 1) .On the other hand, if it cannot meet the demand from thehard appliances, HEM may have to increase w and recalculate x ( n + 1) . III. S IMULATION RESULTS
In this section, we use simulations to study the behavior anddynamics of the proposed algorithm. There are N = 10 usersand without loss of generality we assume that the capacity C is 1. For the price function (3), a = 1 , k = 4 . A. Basic simulation
Here all the users initiate their demands at 0.02, and theirwillingness to pay parameters range from 0.11 (user 1) to 0.20(user 10). All the users have the same adaptation parameter α of 0.1. Figure 2 shows the demand changes with time for10 users. After a short transient period, each user demandconverges to a stable value (determined by different w values).It is also evident that w is a crucial factor in determining howaggressive a user should be responding to the price signals.Figure 3 clearly shows that the price converges to the optimalvalue. When the system reaches its equilibrium (assuming a =1 , C = 1 ), we have x i = w i ( P Ni =1 x i ( n )) k . (11) de m and Fig. 2. Simulation III-A: demand adaptation of 10 users p r i c e Fig. 3. Simulation III-A: price evolution
Summing over i on both sides of (11), it is easy to verify thatthe price at equilibrium is p = ( N X i =1 w i ( n )) kk +1 . (12)In this case, p = 1 . as shown in Figure 3. B. The effect of α In this simulation experiment we study the effect of α onsystem performance. Figure 4 and Figure 5 depict the demandand price evolution versus time respectively for α = 0 . .Compared with Figure 2 and Figure 3, it can be seen that witha larger α , it takes much longer to converge. Therefore α isan important system parameter that controls the convergencespeed of the process. C. Heterogeneous initial demands
In this simulation experiment we study the effect of hetero-geneity of initial demands, i.e., ten users start with demandsranging from 0.01 to 0.10 respectively. The results are shownin Figure 6 and Figure 7. We observe that different initialconditions do not affect the system stability and convergenceto equilibrium. de m and Fig. 4. Simulation III-B: demand adaptation of 10 users p r i c e Fig. 5. Simulation III-B: price evolution de m and Fig. 6. Simulation III-C: demand adaptation of 10 users p r i c e Fig. 7. Simulation III-C: price evolution de m and Fig. 8. Simulation III-D: demand adaptation of 10 users
D. Heterogeneous initial demands and adaptation rates
In addition to heterogeneous initial demands as in lastsimulation, here users also have different adaptation rates α i :ranging from 0.11 to 0.20. The results are shown in Figure 8and Figure 9, where we can see that the system still convergesto the equilibrium. E. Time-varying w To model the situation where users change their WTP w on-the-fly to accommodate their energy needs, we change w i ’s attime slot 100 by adding a random number within the regionof ( − . , . . Figure 10 and Figure 11 clearly show thatafter time 100, the system tracks the change nicely to a newequilibrium. F. The effect of C The energy provider can influence the price by adjustingthe capacity C . As shown in Figure 12 and Figure 13, when C is doubled, the price will drop by − C kk +1 which is ,and each user’s demand will increase accordingly. p r i c e Fig. 9. Simulation III-D: price evolution de m and Fig. 10. Simulation III-E: demand adaptation of 10 users p r i c e Fig. 11. Simulation III-E: price evolution de m and Fig. 12. Simulation III-F: demand adaptation of 10 users p r i c e Fig. 13. Simulation III-F: price evolution
G. Inaccurate price signals
The feedback price signals are transmitted via a communi-cation network (e.g. GPRS) to the HEM, during which packetloss and delay could occur. In this case users may have toadapt their demands based on outdated or inaccurate priceinformation. We model this situation as a small perturbationto the price signal and study its effect on the system behavior.As shown in Figure 14 and Figure 15, the price and demandsstill converge to the means of the equilibrium values, but withsmall fluctuations. A more detailed perturbation analysis ispart of our future work.IV. C
ONCLUSION AND FUTURE WORK
This paper proposes a distributed framework for demandresponse and user adaptation in smart grid networks. Morespecifically, we have applied the concept of congestion pricingin Internet traffic control to the DR problem and shown that itis possible that the burden of load leveling can be shifted fromthe grid (or supplier) to end users via pricing. Individual usersadapt to the price signals to maximize their own benefits. Userpreference is modeled as a willingness to pay parameter whichcan be seen as an indicator of differential quality of service. de m and Fig. 14. Simulation III-G: demand adaptation of 10 users p r i c e Fig. 15. Simulation III-G: price evolution
The convergence of the algorithm has been demonstrated byboth analysis and simulation results.This paper is just a first step towards our vision of fullydistributed demand response. There are a number of directionsfor future research. Firstly, the proposed model fits nicely intothe game theory framework. In fact, there is already a richliterature in the networking community on game theoreticalanalysis of congestion pricing, e.g. [10]. In Kelly’s framework,the user and social optima coincide if the prices are right,and the social optimum is a Nash bargaining solution withthe logarithmic utility function. More recently, researchershave applied intelligent agents and game theory to micro-storage management for the smart grid [11], in which Nashequilibrium is reached when the agents are able to optimizethe energy usage and storage profile of the dwelling and learnthe best storage profile given market prices at any particulartime. Similarly, based on our model, it would be interestingto study the system dynamics and user interaction in a largescale energy demand game context. There are two types ofuser strategies: price taking users and price anticipating users[12]. A price taking user assumes that he has no effect on theprice of the energy, whereas a price anticipating user realizesthat his own choice of w i affects the price. An interesting observation is that price anticipating users tend to pay less.One important element of intelligence in the smart gridis the learning capability of various components. In demandresponse, if users can learn from past observations (e.g. pricesand load profiles), then they can predict the future load andprice and adjust their strategies accordingly (e.g. adjusting α i and w i ). In this context, Bayesian networks and reinforcementlearning are some of the powerful tools we can leverage toenable learning in this highly dynamic environment.As mentioned earlier, demand scheduling can be formulatedas a typical resource allocation problem, for which a widerange of techniques (many of them have been applied inthe networking field) are available. We are currently inves-tigating the feasibility of applying some convex optimizationtechniques such as water-filling to demand side management,where a certain cost function is to be minimized subject to anumber of system constraints.A CKNOWLEDGMENT
The author would like to thank his colleagues at ToshibaResearch Europe for helpful discussions and its Directors fortheir support of this work.R
EFERENCES[1] Z. Fan, G. Kalogridis, C. Efthymiou, M. Sooriyabandara, M. Serizawa,and J. McGeehan, “The new frontier of communications research: Smartgrid and smart metering,” in
ACM e-Energy , 2010.[2] F. Rahimi and A. Ipakchi, “Demand response as a market resource underthe smart grid paradigm,”
IEEE Trans. Smart Grid , Jun. 2010.[3] C. Wang and M. de Groot, “Managing end-user preferences in the smartgrid,” in
ACM e-Energy , 2010.[4] A. Mohsenian-Rad, V. Wong, J. Jatskevich, and R. Schober, “Optimaland autonomous incentive-based energy consumption scheduling algo-rithm for smart grid,” in
IEEE PES Conference on Innovative SmartGrid Technologies , 2010.[5] F. Kelly, A. Maulloo, and D. Tan, “Rate control for communicationnetworks: shadow prices, proportional fairness and stability,”
Journal ofthe Operational Research Society , 1998.[6] G. Vinnicombe, “On the stability of networks operating TCP-likecongestion control,” in
IFAC , 2002.[7] S. Floyd, “TCP and explicit congestion notification,”
ACM CCR , Oct.1994.[8] A. Ganesh, K. Laevens, and R. Steinberg, “Congestion pricing and useradaptation,” in
IEEE Infocom , 2001.[9] H. Bessembinder and M. Lemmon, “Equilibrium pricing and optimalhedging in electricity forward markets,”
Journal of Finance , Jun. 2002.[10] P. Key and D. McAuley, “Differential QoS and pricing in networks:where flow control meets game theory,”
IEE Proceedings Software , Feb.1999.[11] P. Vytelingum, T. Voice, S. Ramchurn, A. Rogers, and N. Jennings,“Agent-based micro-storage management for the smart grid,” in
AAMAS ,2010.[12] R. Gibbens and F. Kelly, “Resource pricing and the evolution ofcongestion control,”
Automatica , 1999.