Collaboratively Optimizing Power Scheduling and Mitigating Congestion using Local Pricing in a Receding Horizon Market
Cornelis Jan van Leeuwen, Joost Stam, Arun Subramanian, Koen Kok
aa r X i v : . [ c s . M A ] S e p Collaboratively Optimizing Power Scheduling andMitigating Congestion using Local Pricing in aReceding Horizon Market
Coen van LeeuwenTNOGroningen, [email protected] Joost StamUniversity of TwenteEnschede, [email protected] Arun SubramanianTNOThe Hague, [email protected] Koen KokTNOGroningen, [email protected]
Abstract —A distributed, hierarchical, market based approachis introduced to solve the economic dispatch problem. Theapproach requires only a minimal amount of information tobe shared between a central market operator and the end-users. Price signals from the market operator are sent downto end-user device agents, which in turn respond with powerschedules. Intermediate congestion agents make sure that localpower constraints are satisfied and any potential congestion isavoided by adding local pricing differences. Our results show thatin 20% of the evaluated scenarios the solutions are identical tothe global optimum when perfect knowledge is available. In theother 80% the results are not significantly worse, while providinga higher level of scalability and increasing the consumer’s privacy.
Index Terms —Smart Grid, Multi-Agent Systems, Market, Self-Organization
I. I
NTRODUCTION
Three trends set a challenge for future power grids. Firstly,the transition towards sustainable energy sources leads to morerenewable energy, but also to a larger fraction of unpredictableand intermittent production. Secondly, the electrification ofvarious systems such as transport (electric vehicles), heat-ing (heat pumps), and in general an increase of electricity-consuming devices leads to a huge growth of power consump-tion. And thirdly, the distribution of energy generation leads toa very different pattern in the load of the power transmissiongrid, than it was designed for.The control of a vast number of small power units, both con-suming and producing, is extremely difficult to do completelytop-down, so a centralized control strategy cannot be used [1].At the same time the power infrastructure is aging, and wasnot built for the emerging pattern of distributed prosumers [2].This is why we need self-organizing control algorithms toschedule the use of electric devices while taking into accountconstraints of the power distribution infrastructure. This aids distribution system operators (DSO) and transport systemoperators (TSO) to maintain power balance and make surethere is no congestion, i.e. the grid capacity is not overloaded.II. P
ROBLEM S TATEMENT
The problem at hand is a variation of the economic powerdispatch problem [3], [4], where the operation of a set of generators is optimized, such that power is provided toconsumers in the most cost-effective manner. Traditionallythis problem would include controllable generators (powergeneration plants), constraining transmission resources (trans-formers, stations, cables), and end-consumers having a staticload. Currently, with distributed energy resources, and demandresponse—the possibility to control the load of consumersusing flexibility of smart devices—the problem changes sig-nificantly.A formal definition of the problem is as follows: min x X i ∈N J i ( x i ) , (1a)s.t. X i ∈N x i = Θ , (1b) Ax ≤ b , (1c) x min i ≤ x i ≤ x max i , ∀ i ∈ N , (1d) C i ( x i ) = 0 , ∀ i ∈ N . (1e)With all parameters defined in Table I, the objective functionas defined in (1) is the same as for the traditional economicdispatch, which can be summarized as: to find a set ofelectrical powers x i for every device i ∈ N in the grid, thatminimizes the sum of all costs J i ( x i ) . In the original dispatchproblem defined, N defines only the producers. However, inour problem formulation (1) N denotes the full set of devicesin the grid, including producers and consumers . This meansthat contrarily to the traditional economic dispatch problem,not only the generators are controllable, but also the flexibleloads of end consumers.Note that the costs in (1a) do not necessarily refer to thefinancial cost of a dispatch. Rather, this generic model onlyoptimizes some social welfare , which defines a desirable out-come for all participants involved. Depending on the situationat hand, one could minimize the amount of greenhouse gassesemitted, or maximize the amount of renewable energy used.However, for the rest of this paper the costs are defined asthe energy losses of the devices. By minimizing the amountof energy losses we attempt to find a dispatch that is botheconomic and sustainable. TABLE IU
SED NOTATIONS AND PARAMETERS IN THIS PAPER .Symbol Description Typical value Unit A topology matrix of child relations α forecast accuracy . b vector of congestion thresholds [2 , , . . . , . × W β i congestion threshold of i × W C i constraint function of i C Z i constraint function of i of type Z ∗ γ i PV operation cost of i . e i × T vector of energy of i [0 . , . , . . . , − . × Wh e it energy of i at t Wh e min i minimum energy of i . Wh e max i maximum energy of i × Wh ǫ × T vector of errors [0 , , . . . , − W ǫ max upper bound on the error − W η i storage efficiency of i . target power of the cluster [1 . , − , . . . , . × W i an agent index , , . . . , nJ i cost function of iJ Z i cost of i of type Z ∗ λ i storage leakage of i W M i set of children of i N set of all agents ρ × T vector of prices [0 . , . , . . . , . T time horizon t time slot index , , . . . , Tτ duration of a time slot 1 h x i × T vector of scheduled powers of i [1 . , − . , . . . , . × W ¯ x × T vector of aggregated powers of child nodes [24 . , − . , . . . , . × W x ′ i × T vector of expected powers of i [1 . , − . , . . . , . × W ˆ x × T vector of average historic powers [0 . , . , . . . , − . × W x it scheduled power of i at t − . × W x min i minimum power for i − × W x max i maximum power for i × W ψ i heat pump coefficient of performance (COP) of i . Note that powers are indicated from the grid perspective. That means positive powers indicate power going from the grid to the device, and negative powersare from the device back to the grid. ∗ Z indicates a type of agent, which can either be MO for market operator, CO for congestion, LOAD for a static load agent, PV for an agent with photovoltaicsolar panels, or ST for a storage agent.
Let us discuss the constraints of (1), i.e. (1b–1e). Theconstraint (1b) states that the sum of all powers in the systemhas to meet a specific target Θ . In a balanced (island) grid thistarget has to be zero for every time slot, but in a connectedgrid this target is the contracted load with the transmissionoperator. Put differently, Θ is the net power input of the gridto the rest of the world.Constraint (1c) defines the limitations of the physical powergrid; a topology matrix A specifies which device is connectedto which grid components, and b is the rating of thosecomponents—the maximum amount of power that it can safelytransmit.Constraints (1d) and (1e) represent the constraints of thedevices in the grid. Specifically, constraint (1d) states thata device cannot produce or consume more power than itphysically can, which is represented by lower and upperbound, x min i and x max i , respectively. However, (1e) also takesinto account another private constraint C . We refer to thisconstraint as private, as it only concerns the state of a singledevice, and its state may concern information that we shouldnot require to share with other parties. E.g. an end-user shouldnot need to share its intent to run his or her washing machinewith the rest of the neighborhood. This constraint differs per device, and specifies device limitations such as a battery thatcannot hold more than a certain amount of energy, and cannotdischarge when already empty. Flexible loads also may haveconstraints considering the time at which they can turn on oroff based on the consumer’s settings. We will elaborate onthese constraints, as well as the cost functions of the deviceagents in Section III-B. A. Related Work
In existing studies, different strategies are used for energymanagement. Four different main categories are defined in [1]based on whether there is a distributed aspect of decision-making and whether there is one- or two-way communication.One of the strategies defined in [1] uses
Transactive Con-trol , where distributed systems decide locally on their devicemanagement, using two-way communication in a market-based control scheme. The authors compare this approach withtraditional top-down switching, price-reaction and centralizedoptimization strategies and show that transactive control iscapable of using the full flexibility potential of the smart griddevices, while maintaining the end-user privacy. This claimis consistent with earlier studies [5], [6] that have shownthat using a market-based control in a multi-agent system can provide equally optimal results as a centrally optimizedsystem, under certain conditions.Multi-agent based methods are already getting attention inthe smart grid domain due to many desirable properties: ro-bustness, user-friendliness, attack resistance and scalability [7],[8]. The economic dispatch problem is very well suited to berepresented using multi-agent systems [9]. There are manystudies that use a multi-agent based approach to model andsolve the problem, such as the two-way message passingagents using consensus algorithms to find an allocation thatis optimal [10]. Other methods use a completely decentral-ized method involving reinforcement [11], utility maximiza-tion [12] or model predictive control [13].In [14], a strategy is proposed to schedule the powerconsumption and production of generators and loads basedon a method called
Negotiated Predictive Dispatch . In this ap-proach wind and conventional generators, as well as static andflexible loads, are controlled on the transmission level. Agentspropose power schedules, which are aggregated by a centralmarket operator, which then updates a price program usinggradient descent, meaning the price of congested timeslotsincrease, and that of underused time slots decrease. In doing sothe authors show that they are able to balance power produc-tion and consumption, while satisfying power grid constraints.However, this approach focuses on the transmission level,whereas we consider the distribution level power grid to be ata much more imminent risk of congestion. At the transmissionlevel, the scale is much larger than at the distribution level,both geographically and considering the power levels involved.Moreover, the power grid has a very different topology at thetransmission level. A major drawback of this approach whenapplied at the distribution level, is that using a global powerprice for congestion mitigation is not only unfair for agentsin non-congested areas, but it would also be converging to asolution much too slowly. A similar approach to the negotiateddispatch is provided by [15] in which a gradient descent onthe pricing is used, followed by a local optimization of agents.Another approach is given by [16], in which power pro-grams are proposed by device agents that are able to satisfycluster constraints to reach a target energy consumption, whilealso staying within the limits of the grid. In their approach,called
Profile Steering , agents are only motivated to accom-plish the cluster goal, which is to consume or produce energyat a specific target amount, and will propose alternative powerprograms whenever constraints are violated. Proposals thatreduce the constraint violations the most, are then selectedas the new candidate programs. In our view, the problem ofthis approach lies in the lack of motivation for the participantto sacrifice private rewards for running an alternative program.The only objective is the cluster goal, which means that theprice of having limited resources is paid by a select set ofindividuals that offer the most flexibility, or conversely, theprofit is reaped by those who are able to maximally make useof remaining capacity.The authors of [17] propose an approach based on DCOPs(Distributed Constraint Optimization Problem) to solve theeconomic dispatch problem, as well as the real-time demand-response balancing problem. Their algorithm is able to find optimal solutions for a system of controllable generators and(predicted) loads, taking into account transmission networkconstraints. They show that finding the optimal solution usingDynamic DCOPs is possible, but their solution does not scalevery well. A relaxed version of the problem, in which softconstraints may be (temporarily) violated, scales better, butstill for relatively small time horizons, even considering theirimplemented solution on a GPU.We note that related work on the topic of this chapterlisted here is not exhaustive. For further detailed background,a comprehensive overview of different mechanisms for solv-ing optimization problems in smart grids, particularly wheredemand response is involved, we refer to [18], [19], [20].III. S
ELF -O RGANIZING E CONOMIC D ISPATCH
Our contribution is a self-organizing method for coordi-nating the power scheduling of smart grid devices on thedistribution level energy grid. This is a multi-agent basedheuristic approach for find a solution to the economic dispatchproblem as defined in (1). This multi-agent based approachallows for great scalability, while ensuring privacy and finalcontrol of the end-user.At the distribution grid level, devices are typically consumerdevices, such as PV-panels, household batteries, heat pumps,ventilation or air-conditioning units. The controllability, orflexibility of such devices is often limited, and bound by thedevice limitations and the user preferences. With increasingnumbers of such devices, using a strictly “top-down” approachis intractable, since the search space grows exponentially witheach added device. For this reason we use a hierarchicalapproach, in which the market operator delegates its task tointermediate “congestion” agents which then independentlysolve subproblems—which is to make sure that the totalpower throughput at their allocated point does not exceed acertain threshold. Making such subdivisions is justified, sincein the power grid, transformers are effectively branches ofthe topology, and two nodes under different transformers areindependent of one another; hence, transformers are the logicalpoint to place congestion agents in the control hierarchy.Our approach, denoted as Local Pricing Receding Horizon(LP-RH) is based on the economic incentive that end-userdevices should have, to provide its owner with a service, inthe most affordable way. Put differently, the system will useprice differences to stimulate agents to schedule their powerconsumption or production in a balanced way, such that anygrid constraints are satisfied. The overall scheme is simply togather expected power programs from the connected devices,and then iteratively adjust energy prices to steer the agentsinto a certain power program. The method is explained in moredetail in the following section, and is similar to the NegotiatedPrice Dispatch proposed by [14].
A. Local Pricing Receding Horizon
In order to create a power planning taking into accountforecasts and/or predicted power programs of clients, we usea Receding Horizon (RH) approach, depicted in Fig. 1 whichmeans that at any point we only take a fixed horizon of T t t t . . . t T − t T − t T t t t . . . t T − t T t T +1 t t t . . . t T t T +1 t T +2 Fig. 1. The principle of a receding horizon market is that every time stepthe time horizon for optimization shifts by one. Planned power programs(white) are turned into fixed contracts for the current time step (green), whichdetermines the outcome of the algorithm. This figure is redrawn from [14].
Algorithm 1
LP-RH Algorithm x i = createPowerprogram ( i, ρ ) if i is a Device agent then x i = arg min x C i ( x , ρ ) { Run local optimization } else repeat for all j ∈ M i do x j = createPowerprogram ( j, ρ ) end for x i = P j ∈M i x j ǫ = C i ( x i ) ρ = adjustPrices ( ǫ, ρ ) { Using gradient descent } until ǫ ≤ ǫ max end if return x i program time units (PTU) into account. A PTU is typically 15minutes or one hour, throughout this paper we will use PTUlength of one hour so T = 24 . However, none of the mentionedapproaches are limited to this convention. When the algorithmhas converged and a power program is found for the next T PTUs that satisfies all constraints, the first PTU becomes the“current” situation, and the projected power program becomesa contract. The time horizon now shifts by one, and the entiresystem starts again.The Local Pricing Receding Horizon algorithm is shownas pseudocode in Algorithm 1; an example graph on whichthe algorithm could run is shown in Fig. 2. The algorithm de-scribes how any agent finds a new power program x i to satisfyits local constraint C i . Every device agent in the grid does thisby solving a local optimization problem (line 2), taking intoaccount local constraints as explained in Section III-B.Prices ρ are updated by the market operator agent and MC C D D D D D Fig. 2. Simple example topology of a graph which could be running theLP-RH algorithm. One market operator ( M ) is connected to two congestionagents ( C i ), which are connected to a total of five device agents ( D i ). Thedevice agents can represent PV panels, consumer loads, storage agents, orany other leaf nodes in the power grid. Topologies where congestion agentsare connected to more congestion agents are also possible. the congestion agents in order to get a power program x satisfying the grid constraints. When the market operator orany congestion agent runs the LP-RH algorithm, all j ∈ M i ,where M i are the immediate children of agent i , are requestedto propose a power program based on the price ρ (line 6). Herethe function is called recursively until all agents have deter-mined a new power program. Note, that if any of the childrenare congestion agents, they determine their power programby forwarding the prices to their children and returning thesum of all received programs. Many agents will not knowexactly how they will function in the future, as contexts mightchange, a user might behave differently than anticipated, orweather conditions may act up; this is why receding horizonmethod updates the time iteratively. With the sum of all powerprograms, the market operator can determine the error ǫ , whichis the sum of all local constraint violations C for every t ∈ T .In line 9 the constraint C i is used to compute the localconstraint violation and stored as an error variable ǫ . The valueof ǫ is taken into account to adjust the prices in line 10. In the“adjustPrices” function, a gradient descent approach is used,which linearly interpolates the last two errors as a functionof the price. We then choose the price at which the error isprojected to reach zero. Note that this means we assume thereaction of the devices linearly depends on the prices, whichwill only hold under very specific circumstances. To overcomethis issue, we iteratively repeat the process until ǫ ≤ ǫ max and assume that a non-linear response can be described as aseries of linear pieces. The parameter ǫ max then represents anupper bound on the error, which we can use as a convergencecriterion. B. Agent Behavior
Agents in the system are characterized by the device thatthey have to assign a power program for. In the distributedcase, every agent locally optimizes a local cost function J i ,which is part of the global optimization problem (1). Also,local constraints C i have to be taken into account which arerepresented by constraint (1e). The behavior of the agents canbe defined by three functions for the costs, the local constraintand the power program; an overview of this is shown inTable II. In our model we consider the following types ofagents:
1) Market Operator:
This is the root node of the tree ( M in Fig. 4), as far as the local distribution grid concerns. In thephysical grid, it corresponds to the transformer that connectsthe local low voltage (LV) grid to the medium voltage (MV)grid. We assume that there is no energy loss at the marketoperator, so J MO = 0 . (2)Its goal is to find a solution to (1), and its private cost would bethe same as the target constraint in (1b). The market operatorruns Algorithm 1, using a deviation of the target profile Θ asan error, which is minimized by the algorithm. Hence, its localconstraint is defined as C MO = ¯ x − Θ , (3) TABLE IIA
SUMMARY SHOWING THE THREE FUNCTIONS SPECIFYING THE BEHAVIORS OF THE AGENTS .Agent type Cost Constraint PowerMarket Operator J MO = 0 C MO = ¯ x − Θ ¯ x = P i ∈N x i Congestion agent J CO i = 0 C CO = ¯ x i − β i iff | ¯ x i | < β i ¯ x i = P j ∈M i x j Load agent J LOAD i = 0 C LOAD i = 0 x i PV agent J PV i = x i − x ′ i C PV i = 0 iff τ x i ρ ≥ γ i x i = x ′ i or Storage agent J ST i = x i (1 − η ′ i ) C ST i = 0 iff e min i < e i < e max i x i = f ( ρ ) where ¯ x denotes the power program of the market operator.Since the market operator is not a device in the grid itself, itspower is the sum of the powers of its children, which in caseof the market operator are all nodes in the cluster: ¯ x = X i ∈N x i . (4)The value of (3) can either be negative or positive, whichrespectively means either the total power production or thetotal consumption is too high. The market operator sets aninitial price of ρ = 0 . for all timeslots, and then usesAlgorithm 1 to minimize the constraint value until it reacheszero, in order to satisfy the global constraint (1b).
2) Congestion Agent:
This is an intermediate node on thegrid tree ( C in Fig. 4) connected to a parent node who iseither the market operator or another congestion agent. Itcorresponds to a component in the grid where congestionmight potentially occur, such as a transformer. Equivalentlyto the market operator, we assume that no energy is lost here,so: J CO i = 0 . (5)This agent has a constraint that aims to limit the powerusage of that part of the grid, this corresponds to the con-straint (1c). The congestion agent also uses Algorithm 1 tominimize the error of its local constraint, which is defined as C CO i = ( ¯ x i − β i , if | ¯ x i | ≥ β i , , otherwise . (6)Here β i is the power limit of the agent i , which meansit is the maximum power throughput of the grid at thepoint i represents. Again ¯ x i denotes the power profile of thecongestion agent, but is now equal to the sum of all devicesunder the current node: ¯ x i = X j ∈M i x j , (7)Similar to the constraint of the market operator in (3) theconstraint value can become negative or positive, and is usedto compute the error ǫ in Algorithm 1.
3) Load Agent:
This is an agent responsible for an uncon-trollable load in the grid (represented by any device agent D in Fig. 4). This could be a consumer household, an office,street lighting, or any other non-flexible load, and thus onlyparticipates in the problem as part of constraint (1b). In thedistributed system however, it is responsible for making aforecast of the power usage, and this forecast will be updatedwith more accurate information as the time horizon shifts. The load agent has no attached cost in the global problem,and no need to locally compute any optimal behavior. This isequivalent to stating that its cost and constraint correspond to J LOAD i = 0 , (8) C LOAD i = 0 . (9)Its corresponding load profile x i is fixed to some profile thatconstrains the global problem (1). In our experiments its valuesare taken from real households as described in Section IV-B.
4) PV Agent:
The PV agent (any D in Fig. 4) has someflexibility to offer to the optimization function by allowingcurtailment in reference to the expected generation. We assumethat curtailment is binary, in that either the PV generates poweras normal, or it is switched off and produces no power at all.Curtailing means that there is potential energy lost, and hencethe cost function of a PV agent is defined as J PV i = x i − x ′ i , (10)where x ′ i indicates the expected power, when not curtailing.This expected power is taken from the scenario, which will bedetailed in the Section IV.When reacting to prices in the distributed system, a decisionis made in order to decide whether to curtail based on the priceprofile. If the operational running costs γ i of the PV is more than the power that would be generated by it, there is no pointin running the generator (from an economic point of view).Hence, the local constraint and its corresponding decision ruleof a PV agent can be written as C PV i = ( if τ x i ρ < γ i , otherwise , (11) x i = ( if τ x ′ i ρ < γ i , x ′ i otherwise . (12)For the PV agent it also holds that the corresponding expectedpower profile x ′ i determines the global problem (1). Its valuesin our experiments are taken from real PV panels as describedin Section IV-B.
5) Storage Agent:
A storage agent (again a leaf node D inFig. 4) provides flexibility by allowing to store some energy ina local storage like a battery or a heat buffer. There are limitsto the amount of energy that can be stored, either becauseof the physical limitations of the storage device, or becauseof the end-user settings. Moreover, a storage agent has someefficiency, which defines energy loss when energy is put intoit, or out from it. This means that we can define the cost function of the storage agent as the energy lost during chargingor discharging J ST i = x i (1 − η ′ i ) , (13)where η ′ i = ( η i if x i ≥ ,η − i otherwise . (14)This difference makes sure that the loss is correlated to theinternal power of the battery when charging or discharging.I.e. x i defines the power at the grid side of the storage, whencharging a lower power effectively charges the battery, andconversely when discharging a higher power is required toprovide some power level to the grid.In order to define the constraints of the storage agent wemust define the update function for the amount of energy e i stored as the cumulative sum of the powers e it = e i + τ t ′ = t X t ′ = t η ′ i x it ′ − λ i . (15)Here λ i represents the leakage or self-discharge rate of thestorage agent i , and e i is the stored energy at the start of theexperiment. Let us now define the following constraints on thestorage agent C ST i = ( , if e min i < e i < e max i , , otherwise. (16)The minimum and maximum energy values are defined by e min i and e max i , respectively.For a storage device, the power limits denote the maximumcharge and discharge rates. They are defined by x max i and x min i ,respectively, in (1d). A special case of a storage device isa heat pump, which (electrically powered) heats a house inorder to keep the temperature within comfortable levels. Theheat pump only allows charging and only discharges throughleakage, hence for a heat pump x min i = 0 and λ i > .When a storage agent has to update its expected powerprofile in line 2 of Algorithm 1, some response function ( f ( ρ ) in Table II) is required, which is “economically sane” and hasthe following characteristics: • return x max i for low prices and x min i for high prices, • be a monotonically decreasing for increasing prices, • have a “plateau” of zero power response for some inter-mediate price ( ρ = 0 . ), which is wider for less efficientdevices.The final characteristic allows more efficient devices torespond to subtle price change, and have less efficient devicesrespond to more extreme prices. This way devices with ahigher efficiency are used first when flexibility is needed,and (when parameterized correctly) less efficient devices willonly be used when required. In our implementation we chosethe simplest response, where the agent linearly decreases itspower from x max i to zero at ρ = η i / . It then stays at zerosymmetrically around ρ = 0 . until ρ = 0 . /η i , and thendecreases its power response linearly to x min i . This power/pricerelation is depicted in Fig. 3. . . . . − Price (steering signal) P o w e r( W ) Fig. 3. The strategy of the storage agent with x max i = 200 W, x min i = − Wand η i = 0 . shows the power response for an increasing price. The plateau at W extends from ρ = η i / to ρ = 0 . /η i . An alternative strategy for the storage agent would be tomake use out of any differences in the price, charging anddischarging as soon as the price differences are large enough toovercome its efficiency. From a strictly economic perspective,this is the optimal strategy to maximize its own benefit.However, this leads to very “binary” behavior with minimaland maximal charging rates [12] and thus, little room foroptimizing from the market operator and congestion agent.Therefore, a linear strategy is implemented as depicted inFig. 3, allowing to solve the overall optimization problem (1).IV. E
XPERIMENT S ETUP
The LP-RH algorithm was empirically evaluated by runninga simulation of an LV grid with a set of realistic householdload profiles and PV production profiles for a series of 24PTUs.
A. Distribution Network Topology
For the topology of the network we use the European LowVoltage Test Feeder [21] network. This dataset is used tobenchmark power and energy algorithms on realistic Europeandistribution networks. In this paper we superimposed sixpoints on the topology, where we monitor and mitigate anypotential congestion. These points are strategically chosento separate the problem into independent subproblems. Theresulting topology with the congestion points are shown inFig. 4.
B. Household Load and PV Profiles
The household consumption and production profiles aretaken from a pilot study [22], in which 92 residential con-sumers were monitored over the course of a year (from Marchthrough November 2018). The data was preprocessed suchthat we have separated information on the consumption ofhouses, and of the PV installations. Data is anonymized andrandomized per month, so that we can select data from anyspecific month for a base load of a household, or a residentialPV installation.In the experiment the 54 households from Fig. 4 wereassigned a random instance of the load and
PV profiles fromthe same month (i.e. all households were equipped with PVpanels). The daily load consumption varied between 4.98 kWhand 29.39 kWh, and the total PV production varied between826 Wh and 18.8 kWh. Furthermore, 16 randomly selected
Market operatorCongestion agentHousehold
Fig. 4. The topology of the IEEE LV feeder network used for the simulation.The figure depicts the connections between household consumers, the inter-mediate congestion agents, and the root market operator. The “filled” markersrepresent the agents from Section VI. households were assigned a household battery, and again 16were chosen to have a heat pump installation.Every household and PV installation in the simulationwould select a random profile from the dataset, which wasused as its power profile. The objective Θ was set to a totalnet consumption of the LV grid using the mean total powerconsumption of the houses including PV production. Choosingthe target profile Θ in this way corresponds to a situationin which the energy provider of the simulated neighborhoodwould agree to a contract for the average behavior of thehouseholds, and subsequently attempts to use flexibility toaccount for any deviations from the normal.The batteries were all dimensioned with storage capacitiesof e max = 10 . kWh, and maximum charge and discharge ratesof x max = 4000 W and x min = − W, respectively. Thebatteries charging efficiencies were all set at η = 0 . . The heatpumps were estimated to have a working energy capacity of e max = 2 kWh, this means the difference between the thermalenergy capacity of the house at the minimum and maximumcomfortable user temperature is ψ kWh, where ψ = 4 . is thecoefficient of performance of the heat pump. Then, the heatpumps have a x max = 1600 W and x min = 0 W, an efficiencyof η = 1 and a constant leakage rate of λ = 360 W. Finally, toensure convergence, the maximal error value is set to ǫ max =10 − in the experiments for this paper C. Forecast Uncertainty
The predicted load and production power profiles ˆ x of theload and PV agents were generated by taking average profilesof the complete dataset. These average profiles are consideredas taken from historic data and hence, provide a ground forpredicting the power program of future PTUs. When agent i determines its power prediction x ′ i , it will compute a weightedaverage between its assigned power profile x i and the average profile ˆ x , such that: x it = (1 − α ) x ′ it + α ˆ x t , (17) α = r t − T − , (18)such that at t = 1 the prediction equals the selected profile x ′ it . At t = T , the prediction is simply the average power ˆ x t .V. C ENTRALIZED S OLVER
In order to address the performance of the LP-RH algorithm,a centralized optimization approach is also introduced toprovide lower bounds to its results. A mixed integer linearprogram (MILP) solver was used to find these bounds for theproblem stated in (1). The centralized optimization approachconsiders the exact same scenario that was solved by thedecentralized algorithm; the energy loss is minimized and thesame set of constraints apply. A fundamental difference lies inthe availability of information. Whereas detailed informationis only shared locally in the LP-RH algorithm, the centralizedoptimization approach assumes complete knowledge of thecurrent system state for the decision-maker; i.e., no limits areimposed on the spacial flow of information within the network.With these features in mind, two versions of the MILP wereformulated: Receding Horizon Centralized Solver and PerfectInformation Centralized Solver.
A. Receding Horizon Centralized Solver
The receding horizon centralized solver (RHCS) is themost similar to the LP-RH algorithm. It uses a recedinghorizon approach (as depicted in Fig. 1) to find an idealdispatch solving the consecutive sub-problems. Uncertaintyabout future device states is again simulated by (17). Becausethis solver has perfect information at the current state for eachiteration of the receding horizon, its solution represents a lowerbound on the solution found by the LP-RH algorithm.
B. Perfect Information Centralized Solver
In the perfect information centralized solver (PICS), a singlecentralized optimization problem is solved for the completedispatch. In addition to perfect spacial information, this ver-sion of the MILP also has perfect temporal information aboutthe complete system state; this means that no uncertaintyabout future states is simulated. Referring back to (17), this isequivalent to setting α = 1 , which results in perfect predictionsfor each profile. The solution of this MILP represents anabsolute lower bound for problem (1).VI. R ESULTS
In this section the results of the described experiments areshown. Before comparing the performance of the differentsolvers, which is done in Section VI-A, the behavior of theLP-RH algorithm is shown in this section. The graphs in thissection show examples of single simulation runs, demonstrat-ing the behavior of the different agent types. Powers are shownas power consumption, this means a net consumption is shownas positive power, and conversely negative powers indicates a − Time (PTU) P o w e r( k W ) ProgramTarget . . . . . Time (PTU) P r i ce Fig. 5. The power program of the market operator and the price profile asthe outcome of Algorithm 1, show the results of a 24-hour simulation of theproblem with the LV feeder network and 92 random households. net power production. In Fig. 5 the final result of the marketoperator is shown, where the market operator has found a priceprofile such that the target profile is met exactly.Fig. 6 shows the power profile of one of the three congestionagents that is directly connected to the market operator. Itspower congestion limits are set such that in the peak momentsof the day there is some congestion expected. This results ina price difference shown in the power program, as around thepeak PV production ( t = 12 , , ) the local price is slightlylower than the market price, leading to a lower net powerproduction. Similarly, at the end of the day ( t = 19 , , , )a positive power congestion was mitigated by increasing theprice.The power program of a heat pump agent is shown inFig. 7. This heat pump is connected directly to the congestionagent from Fig. 6, hence its price profile should be identical.What is most obvious in this graph is that the high priceat t = 12 leads to a zero power consumption, and quicklyafter that, the power consumption rises in order to maintaincomfortable temperatures. Again, around t = 19 , , therelatively high prices lead to a power consumption of zero,which was apparently feasible because of the high powerconsumption leading up to it.Finally, Fig. 8 shows the total power programs of alldevices of the four classes in this experiment summed up.In this figure, the power profiles of the loads and the PVsare the direct result of the chosen profiles, and are the inputfor problem (1). We can see that for the majority of theexperiment, all used flexibility is from the heat pumps withthe high efficiency. Only at times of the congestion will theless efficient battery agents be used. A. Comparison with Central Solvers
In a validation experiment 105 random problem instances(15 permutations for seven months) were created, and solvedby the three different algorithms. In Fig. 9 the results areshown for the feasible instances. A solution is considered − Time (PTU) P o w e r( k W ) ProgramBounds . . . . . Time (PTU) P r i ce Local priceMarket price
Fig. 6. The power program of a congestion agent shows some periods ofcongestion at the production and consumption bounds (red dashed lines), andthe corresponding changes in price profiles (from Algorithm 1) relative to theglobal price of the market operator. . . . . Time (PTU) P o w e r( k W ) . . . . Time (PTU) P r i ce Fig. 7. The power program of a heat pump agent shows the power beingmostly used at moments where the price is low, relieving the need to chargewhen the price is high; or considering the view of the grid, when a lowerpower consumption is required. feasible if all three solvers were able to find a solution thatsatisfies all constraints. The LP-RH algorithm did not finda correct solution for 26 problem instances, 11 of whichwere found to be overconstrained according to RHCS. Forthe feasible instances the LP-RH algorithm found solutionsthat were not significantly worse than the PICS, and in 21instances found a solution with the exact same cost. In 27instances LP-RH found a solution that was equally well asthe RHCS or even slightly better—this seems to contradictthe initial statement that RHCS acts as a lower bound forLP-RH; however, this is due to the way both solvers dealwith uncertainty. The LP-RH algorithm allows the congestionagents to violate power constraints for future PTUs, as longas eventually they are not actually congested due to imperfectforecasts according to (17). The RHCS algorithm does not al-low power constraints to be violated at any point in the future, − − − Time (PTU) P o w e r( k W ) LoadsPVsHeat pumpsBatteries
Fig. 8. The total power consumption per device type show that the batteriesare used far less than the heat pumps, since they have a lower efficiency. ThePV panels did not have to be curtailed in this run.
PICSRHCSLP-RH
Power loss (kW)
Fig. 9. Compared with centralized optimization solvers, the LP-RH algorithmperforms equally well. In this box plot the median is shown as a line, the boxesindicate the 25% and 75% percentiles, and the tails are capped at a maximumlength of the box width; outliers are drawn separately. and hence might react too strict when a future congestion ismistakenly predicted.In the set of infeasible problems, i.e. problems that do nothave a valid solution satisfying all constraints, LP-RH doesfind solutions that minimize the power loss. However, sincethese solutions either temporarily overload congestion agents(where x a gen > β i ), or does not match the target profile Θ exactly, they are not a fair comparison, since they do notstrictly solve (1). In a separate run the problems were relaxed,by increasing the power rating of the congestion agents. Thisresulted in the LP-RH not being able to find a feasible solutionin only 14 problem instances, but the results were otherwisevery similar to the ones reported here in Fig. 9.VII. C ONCLUSIONS
We have introduced an algorithm for self-organizing smartgrids, by solving the economic dispatch problem using adecentralized market based approach with local pricing. TheLP-RH algorithm using a hierarchical approach was shownto be able to solve the problem using a fairly simple in-teraction scheme in which pricing information is sent downthe hierarchy tree, and planned or forecasted power programsare sent back up. Using a gradient descent approach, themarket operator is capable of tuning the pricing to find feasiblesolutions to minimize the power losses in the grid.In our experiments we found that in 20% of the problemsLP-RH did not perform any worse than a perfect-information centralized solver. In the other 80% our algorithm did notperform significantly worse. The benefit of LP-RH over acentralized solver are in the robustness and scalability ofthe solution, as well as in the preserved privacy of the end-consumers.In the implementation of the response of the storage agent,we intentionally did not choose to respond with an optimalpower program given the price signal. Particularly, when a highprice is expected in the future, the agent will not “proactively”charge to avoid having to charge later, or vice versa. Thisbehavior could be implemented at the agent quite easily usinga dynamic programming approach, but it would lead to veryextreme behavior, e.g. very binary behavior of charging ornot-charging at full capacity even for small price differences.This binary behavior is hard to deal with in the rest of thehierarchical tree, and does not lead to any problems per se,but might be improved upon in a future continuation of thiswork.Other variations of the problem may include other devicetypes, for instance time-shiftable devices such as washingmachines or dishwashers. Also, using electric vehicles (EV)as an additional type of agent, providing energy flexibilityis a very interesting extension, which will undoubtedly leadto other complications because of their high power ratings.Finally, an integration with a real time balancing algorithmsuch as [23] would be very fruitful to complete the needs ofthe future smart grid. R
EFERENCES[1] K. Kok and S. Widergren, “A society of devices: Integrating intelligentdistributed resources with transactive energy,”
IEEE Power Energy Mag. ,vol. 14, no. 3, pp. 34–45, 2016.[2] Q. Gemine, D. Ernst, and B. Corn´elusse, “Active network managementfor electrical distribution systems: Problem formulation, benchmark, andapproximate solution,”
Optimization and Engineering , vol. 18, no. 3, pp.587–629, Sep. 2017.[3] D. W. Ross and S. Kim, “Dynamic economic dispatch of generation,”
IEEE Trans. Power App. Syst. , vol. PAS-99, no. 6, pp. 2060–2068, 1980.[4] M. A. Abido, “Environmental/economic power dispatch using multiob-jective evolutionary algorithms,” in
Power Engineering Society GeneralMeeting . Toronto, Canada: IEEE, Jul. 13–17 2003, pp. 920–925.[5] F. Ygge and H. Akkermans, “Decentralized markets versus centralcontrol: A comparative study,”
Journal of Artificial Intelligence , vol. 11,no. 1, pp. 301–333, 1999.[6] H. Akkermans, J. Schreinemakers, and K. Kok, “Emergence of controlin a large-scale society of economic physical agents,” in
Proc. AAMAS ,New York, NY, USA, Jul. 19–23 2004, pp. 1232–1234.[7] M. Pipattanasomporn, H. Feroze, and S. Rahman, “Multi-agent systemsin a distributed smart grid: Design and implementation,” in
PowerSystems Conference and Exposition . Seattle, WA, USA: IEEE, Mar. 15–18 2009, pp. 1–8.[8] C. M. Colson and M. H. Nehrir, “Comprehensive real-time microgridpower management and control with distributed agents,”
IEEE Trans.Smart Grid , vol. 4, no. 1, pp. 617–627, 2013.[9] N. Cai, N. T. T. Nga, and J. Mitra, “Economic dispatch in microgridsusing multi-agent system,” in
North American Power Symposium . Cham-paign, IL, USA: IEEE, Sep. 9–11 2012, pp. 1–5.[10] M. Hu, J.-W. Xiao, S.-C. Cui, and Y.-W. Wang, “Distributed real-time demand response for energy management scheduling in smartgrid,”
International Journal of Electrical Power & Energy Systems ,vol. 99, pp. 233–245, 2018.[11] W. Liu, P. Zhuang, H. Liang, J. Peng, and Z. Huang, “Distributedeconomic dispatch in microgrids based on cooperative reinforcementlearning,”
IEEE Trans. Neural Netw. Learn. Syst. , vol. 29, no. 6, pp.2192–2203, 2018. [12] N. Li, L. Chen, and S. H. Low, “Optimal demand response based onutility maximization in power networks,” in Power Engineering SocietyGeneral Meeting . Detroit, MI, USA: IEEE, Jul. 24–28 2011, pp. 1–8.[13] M. A. Velasquez, J. Barreiro-Gomez, N. Quijano, A. I. Cadena, andM. Shahidehpour, “Distributed model predictive control for economicdispatch of power systems with high penetration of renewable energyresources,”
International Journal of Electrical Power & Energy Systems ,vol. 113, pp. 607–617, 2019.[14] J. Warrington, S. Mari´ethoz, and M. Morari, “Negotiated predictivedispatch: Receding horizon nodal electricity pricing for wind integra-tion,”in
International Conference on the European Energy Market . IEEE,May 25–27 2011, pp. 407–412.[15] G. Binetti, A. Davoudi, F. L. Lewis, D. Naso, and B. Turchiano, “Dis-tributed consensus-based economic dispatch with transmission losses,”
IEEE Trans. Power Syst. , vol. 29, no. 4, pp. 1711–1720, 2014.[16] T. van der Klauw, M. E. T. Gerards, G. Hoogsteen, G. J. M. Smit,and J. L. Hurink, “Considering grid limitations in profile steering,” in
International Energy Conference . IEEE, Apr. 4–8 2016, pp. 1–6.[17] F. Fioretto, W. Yeoh, E. Pontelli, Y. Ma, and S. Ranade, “A distributedconstraint optimization (DCOP) approach to the economic dispatch withdemand response,” in
Proc. AAMAS , S˜ao Paulo, Brazil, May 8–12 2017,pp. 999–1007.[18] J. S. Vardakas, N. Zorba, and C. V. Verikoukis, “A survey on demandresponse programs in smart grids: Pricing methods and optimizationalgorithms,”
IEEE Commun. Surveys Tuts. , vol. 17, no. 1, pp. 152–178,2014.[19] S. Parhizi, H. Lotfi, A. Khodaei, and S. Bahramirad, “State of the art inresearch on microgrids: A review,”
IEEE Access , vol. 3, pp. 890–925,2015.[20] A. R. Jordehi, “Optimisation of demand response in electric powersystems, a review,”
Renewable and Sustainable Energy Reviews , vol.103, pp. 308–319, 2019.[21] IEEE, “European low voltage test feeder network (v2),” Available online:http://sites.ieee.org/pes-testfeeders/files/2017/08/European LV Test Feeder v2.zip,2015, accessed: Nov. 08, 2018.[22] A. Rassa, C. van Leeuwen, R. Spaans, and K. Kok, “Developing localenergy markets: a holistic system approach,”
IEEE Power and EnergyMagazine , vol. 17, no. 5, pp. 59–70, 9 2019.[23] K. Kok and A. Subramanian, “Fast locational marginal pricing forcongestion management in a distribution network with multiple ag-gregators,” in