Profit-Maximizing Planning and Control of Battery Energy Storage Systems for Primary Frequency Control
Ying Jun, Zhang, Changhong Zhao, Wanrong Tang, Steven H. Low
aa r X i v : . [ c s . S Y ] A p r Profit-Maximizing Planning and Control of BatteryEnergy Storage Systems for Primary FrequencyControl
Ying Jun (Angela) Zhang,
Senior Member, IEEE , Changhong Zhao,
Member, IEEE , Wanrong Tang,
StudentMember, IEEE , Steven H. Low,
Fellow, IEEE , Abstract —We consider a two-level profit-maximizing strategy,including planning and control, for battery energy storage system(BESS) owners that participate in the primary frequency control(PFC) market. Specifically, the optimal BESS control minimizesthe operating cost by keeping the state of charge (SoC) in anoptimal range. Through rigorous analysis, we prove that theoptimal BESS control is a “state-invariant” strategy in the sensethat the optimal SoC range does not vary with the state of thesystem. As such, the optimal control strategy can be computedoffline once and for all with very low complexity. Regarding theBESS planning, we prove that the the minimum operating costis a decreasing convex function of the BESS energy capacity.This leads to the optimal BESS sizing that strikes a balancebetween the capital investment and operating cost. Our workhere provides a useful theoretical framework for understandingthe planning and control strategies that maximize the economicbenefits of BESSs in ancillary service markets. N OMENCLATURE c e electricity purchasing and selling price c p penalty rate for PFC regulation failure n interval index I n length of the n th I interval J n length of the n th J interval t sn start time of the n th I interval t en end time of the n th I interval q n indicator variable of n th excursion event P P F C n
PFC power requested in the n th J interval p ( t ) battery charging/discharging power at time tp ac ( t ) power exchanged with the AC bus at time tη battery charging and discharging efficiency E max battery capacity P max maximum charging power of battery s n SoC at the beginning of the n th I interval s en SoC at the end of the n th I interval cost e,n charging cost incurred in the n th I interval cost p,n penalty assessed in the n th J interval This work was supported in part by the National Basic Research Program(973 program Program number 2013CB336701), and three grants from theResearch Grants Council of Hong Kong under General Research Funding(Project number 2150828 and 2150876) and Theme-Based Research Scheme(Project number T23-407/13-N).Y. J. Zhang and W. Tang are with the Department of Information Engineer-ing, The Chinese University of Hong Kong, Hong Kong. They are also withShenzhen Research Institute, The Chinese University of Hong KongC. Zhao and S. Low are with the Engineering and Applied Science Division,California Institute of Technology, Pasadena, CA, 91125, USA.
I. I
NTRODUCTION
The instantaneous supply of electricity in a power systemmust match the time-varying demand as closely as possible.Or else, the system frequency would rise or decline, compro-mising the power quality and security. To ensure a stable fre-quency at its nominal value, the Transmission System Operator(TSO) must keep control reserves compensate for unforeseenmismatches between generation and load. Frequency controlis performed in three levels, namely primary, secondary, andtertiary controls [1]. The first level, primary frequency con-trol (PFC), reacts within the first few seconds when systemfrequency falls outside a dead band, and restores quickly thebalance between the active power generation and consumption.Due to its stringent requirement on the response time, PFCis the most expensive control reserve. This is because PFCis traditionally performed by thermal generators, which aredesigned to deliver bulk energy, but not for the provision offast-acting reserves. To complement the generation-side PFC,load-side PFC has been considered as a fast-responding andcost-effective alternative [2]–[6]. Nonetheless, the provision ofload-side PFC is constrained by end-use disutility caused byload curtailment.Battery energy storage systems (BESSs) have recently beenadvocated as excellent candidates for PFC due to their ex-tremely fast ramp rate [7], [8]. Indeed, the supply of PFCreserve has been identified as the highest-value applicationof BESSs [9]. According to a 2010 NREL report [10], theannual profit of energy storage devices that provide PFCreserve is as high as US$236-US$439 per KW in the U.S.electricity market. The use of BESS as a frequency controlreserve in island power systems dates back to about 20 yearsago [11]. Due to the fast penetration of renewable energysources, the topic recently regained research interests in bothinterconnected power systems [8], [12] and microgrids [13],[14].In view of the emerging load-side PFC markets institutedworldwide [15], [16], we are interested in deriving profit-maximizing planning and control strategies for BESSs thatparticipate in the PFC market. In particular, the optimal BESScontrol aims to minimize the operating cost by schedulingthe charging and discharging of the BESS to keep its stateof charge (SoC) in a proper range. Here, the operating costincludes both the battery charging/discharging cost and thepenalty cost when the BESS fails to provide the PFC service according to the contract with the TSO. We also determine theoptimal BESS energy capacity that balances the capital costand the operating cost. Previously, [8], [13] investigated theproblem of BESS dimensioning and control, with the aim ofmaximizing the profit of BESS owners. There, the BESS ischarged or discharged even when system frequency is withinthe dead band to adjust the state of charge (SoC). This isto make sure that the BESS has enough capacity to absorb orsupply power when the system frequency falls outside the deadband. A different approach to correct the SoC was proposedin [12], where the set point is adjusted to force the frequencycontrol signal to be zero-mean.To complement most of the previous work based on simula-tions or experiments, we develop a theoretical framework foranalyzing the optimal BESS planning and control strategy inPFC markets. In particular, the optimal BESS control problemis formulated as a stochastic dynamic program with continuousstate space and action space. Moreover, the optimal BESSplanning problem is derived by analyzing the optimal value ofthe dynamic programming, which is a function of the BESSenergy capacity. A key challenge here is that the complexityof solving a dynamic programming problem with continuousstate and action spaces is generally very high. Moreover,standard numerical methods to solve the problem do not revealthe underlying relationship between the operating cost andthe energy capacity of the BESS. Our main contributions inaddressing this challenge are summarized as follows. • We prove that with slow-varying electricity price, theoptimal BESS control problem reduces to finding anoptimal target SoC every time the system frequency fallsinside the dead band. In other words, the optimal decisioncan be described by a scalar, and hence the dimension ofthe action space is greatly reduced. • We show that the optimal target SoC is a range thatis invariant with respect to the system state at eachstage of the dynamic programming. Moreover, the rangereduces to a fixed point either when the battery charg-ing/discharging efficiency approaches 1 or when theelectricity price is much lower than the penalty rate forregulation failure. This result is extremely appealing, forthe optimal target SoC can be calculated offline once andfor all with very low complexity. • We prove that the minimum operating cost is a decreasingconvex function of the BESS energy capacity. Based onthe result, we discuss the optimal BESS planning strategythat strikes a balance between the capital cost and theoperating cost.The rest of the paper is organized as follows. In Section II,we describe the system model. The BESS operation problemis formulated as a stochastic dynamic programming problemin Section III. In Section IV, we derive the optimal BESSoperation strategy, which is a range of target SoC independent of the system state. The optimal BESS planning is discussedin Section V. Numerical results are presented in Section VI.Finally, the paper is concluded in Section VII. (cid:1872) (cid:1835) (cid:2868) (cid:1836) (cid:2868) (cid:1835) (cid:3041) (cid:1836) (cid:3041) (cid:1835) (cid:3041)(cid:2878)(cid:2869) (cid:1836) (cid:3041)(cid:2878)(cid:2869) (cid:1871) (cid:2868) (cid:1871) (cid:3041) (cid:1871) (cid:3041)(cid:2878)(cid:2869) … … … … … … … … (cid:1872) (cid:2868)(cid:3046) (cid:1872) (cid:2868)(cid:3032) (cid:1872) (cid:3041)(cid:3046) (cid:1872) (cid:3041)(cid:2878)(cid:2869)(cid:3046) (cid:1872) (cid:3041)(cid:3032) (cid:1872) (cid:3041)(cid:2878)(cid:2869)(cid:3032)
Fig. 1. System time line.
II. S
YSTEM M ODEL
We consider a profit-seeking BESS selling PFC service inthe ancillary service market. The BESS receives remunerationfrom the TSO for providing PFC regulation, and is liable toa penalty whenever the BESS fails to deliver the service asspecified in the contract with the TSO. We endeavour to findthe optimal planning and control of the BESS to maximize itsprofit in the PFC market.
A. System Timeline
Most of the time, the system frequency stays inside a deadband (typically 0.04%) centred around the nominal frequency.Once the system frequency falls outside the dead band, theTSO sends regulation signals to regulating units, including theBESS. The BESS needs to supply power (i.e., be discharged)in a frequency under-excursion event and absorb power (i.e.,be charged) in a frequency over-excursion event.The system time can be divided into two types of intervalsas illustrated in Fig. 1. The I intervals are the ones duringwhich PFC is not needed, i.e., when the system frequencystays inside the dead band or when the frequency is regulatedby secondary or tertiary reserves. An I interval ends and a J interval starts, when a frequency excursion event occurs.The lengths of the J intervals are the PFC deployment timesrequested by the TSO.The lengths of the n th I and J intervals are denoted as I n and J n , respectively. Suppose that I n ’s are independently andidentically distributed (i.i.d.) with probability density function(PDF) f I ( x ) and complimentary cumulative distribution func-tion (CCDF) ˜ F I ( x ) . Likewise, J n ’s are i.i.d. with PDF f J ( x ) and CCDF ˜ F J ( x ) . Note that f I ( x ) = − d ˜ F I ( x ) dx and f J ( x ) = − d ˜ F J ( x ) dx . Moreover, define indicator variables q n such that q n = 1 and − when the n th frequency excursion event is anover-excursion event and under-excursion event, respectively.Let p = Pr { q n = 1 } and p − = 1 − p = Pr { q n = − } . B. BESS Operation
Suppose that the BESS has an energy capacity E max (kWh)and maximum charging and discharging power limits P max (kW). The charging and discharging efficiency is < η ≤ .Moreover, let e ( t ) denote the amount of energy stored in thebattery at time t , and p ( t ) denote the battery charging ( p ( t ) > ) or discharging ( p ( t ) < ) power at time t . Due to thecharging and discharging efficiency η , the power exchangedwith the AC bus, denoted by p ac ( t ) , is p ac ( t ) = ( p ( t ) /η if p ( t ) > p ( t ) η if p ( t ) < . (1) In the n th frequency excursion event, the BESS is obligedto supply or absorb P P F C,n kW regulation power for theentire period of J n . Here, P P F C,n ’s are i.i.d. random variableswith pdf f P PFC ( x ) and CCDF ˜ F P PFC ( x ) . Typically, P P F C,n takes value in [0 , R ] , where R is the standby reserve capacityspecified in the contract with the TSO. In return, the BESSis paid for the availability of the standby reserve. That is, theremuneration is proportional to R and the tendering period,but independent of the actual amount of PFC energy suppliedor consumed.Let s n and s en denote the SoC (normalized the energycapacity E max ) of the BESS at the beginning and end of I n ,respectively. Obviously, when s en is too low or too high, theBESS may fail to supply or absorb the amount PFC energyrequested by the TSO in the subsequent J n interval, resultingin a regulation failure. In this case, the BESS is assessed apenalty that is proportional to the shortage of PFC energy. Let c p be the penalty rate per kWh PFC energy shortage. Then,the penalty assessed in the n th frequency excursion event is cost p,n ( s en ) = c p (cid:16) E P F C,n − E max (1 − s en ) η (cid:17) + if q n = 1 c p ( E P F C,n − ηE max s en ) + if q n = − , (2)where ( x ) + = max( x, and E P F C,n = P P F C,n J n is an aux-iliary variable indicating the PFC energy supplied or absorbedduring J n . Since P P F C,n ’s and J n ’s are i.i.d., respectively, E P F C,n are also i.i.d. variables with PDF f E PFC ( x ) andCCDF ˜ F E PFC ( x ) . Due to the battery charging/dischargingefficiency, ηE P F C,n and E PFC,n η are the energy charged to ordischarged from the BESS during the PFC deployment time.To avoid penalty, the BESS must be charged or dischargedduring I intervals to maintain a proper level of SoC. Supposethat the electricity purchasing and selling price, denoted by c e , varies at a much slower time scale (i.e., hours) than thatat which the PFC operates (i.e., seconds to minutes), and thuscan be regarded as a constant during the period of interest.Then, the battery charging cost incurred in I n is calculated as cost e,n = c e Z t sn + I n t sn p ac ( t ) dt, (3)where p ac is given in (1). cost e,n > corresponds to a costdue to power purchasing, and cost e,n < corresponds toa revenue due to power selling. Notice that the BESS SoCis bounded between 0 and 1. Thus, p ( t ) is subject to thefollowing constraint ≤ s n E max + Z τt sn p ( t ) dt ≤ E max ∀ τ ∈ [ t sn , t en ] , (4)where t sn and t en are the starting and end times of I n ,respectively. As a result, s n and s en are related as s en = s n E max + R t sn + I n t sn p ( t ) dtE max , (5)subject to the constraint in (4). Likewise, the SoC at the SoC at time t is defined as s ( t ) = e ( t ) E max . Obviously, s ( t ) ∈ [0 , . beginning of the next I interval, s n +1 , is related to s en as s n +1 = " s en E max + ( q =1 η − q = − η ) E P F C E max , (6)where [ x ] = min(1 , max(0 , x )) and A is an indicatorfunction that equals 1 when A is true and 0 otherwise.III. P ROBLEM F ORMULATION
As mentioned in the previous section, the remunerationthe BESS receives from the TSO is proportional to thestandby reserve capacity R and the tendering period, butindependent of the actual amount of PFC energy suppliedor absorbed. With fixed remuneration, the problem of profitmaximization is equivalent to the one that minimizes thecapital and operating costs. In this section, we formulate theoptimal BESS control problem that minimizes the operatingcost P n ( cost e,n + cost p,n ) for a given BESS capacity E max .The optimal BESS planning problem that finds the optimal E max will be discussed later in Section V.At the beginning of each interval I n , the optimal p ( t ) duringthis I interval is determined based on the observation of s n .When making the decision, the BESS has no prior knowledgeof the realizations of I k , E P F C,k , and q k for k = n, n +1 , · · · . As such, the problem is formulated as the followingstochastic dynamic programming, where s n is regarded as thesystem state at the n th stage, and the state transition from s n to s n +1 is determined by the decision p ( t ) as well the exogenousvariables I n , E P F C,n , and q n .At stage n , solve H ∗ n ( s n ) = min p ( t ) ,t ∈ [ t sn ,t en ] E I n ,E PFC,n ,q n [ cost e,n + cost p,n ( s en )]+ α E I n ,E PFC,n ,q n (cid:2) H ∗ n +1 ( g ( s n , p ( t ) , I n , E P F C,n , q n )) (cid:3) s.t. (1) , (4) , and − P max ≤ p ( t ) ≤ P max ∀ t ∈ [ t sn , t en ] , (7)where cost p,n , cost e,n and s en are defined in (2), (3), and (5),respectively. H ∗ n ( s n ) is the optimal value at the n th stage ofthe multi-stage problem, α ∈ (0 , is a discounting factor,and g ( s n , p ( t ) , I n , E P F C,n , q n ) := s n +1 describes the statetransition given by (5) and (6).In practice, the tendering period of the service contractsigned with the TSO (in the order of months) is much longerthan the duration of one stage in the above formulation (inthe order of seconds or minutes). Moreover, the distributionsof I n , E P F C,n , and q n are i.i.d. Thus, Problem (7) can beregarded as an infinite-horizon dynamic programming problemwith stationary policy. In other words, the subscripts n and n + 1 in (7) can be removed.Problem (7) requires the optimization of a continuous timefunction p ( t ) . When the electricity price c e remains constantwithin an I period, there always exists an optimal solutionwhere battery is always charged or discharged at the fullrate P max until a prescribed SoC target has been reachedor the I interval has ended. Then, finding the optimal charg-ing/discharging policy is equivalent to finding an optimal targetSoC π ∈ [0 , . This is because charging/discharging cost during an I period is only related to the total energy chargedor discharged, regardless of when and how fast the chargingor discharging is.Under the full-rate policy, the battery charges/discharges ata rate P max until the target SoC has been reached or the I interval has ended. Thus, the charging cost (3) during I n isequal to the following, where π is the target SoC. cost e ( s n , π ) = (8) c e η min( P max I n , ( π − s n ) E max ) if s n < π − c e η min( P max I n , ( s n − π ) E max ) if s n > π if s n = π . Likewise, (5) can be written as a function of s n and π : s en ( s n , π ) = s n + sgn( π − s n ) min (cid:18) P max I n E max , | π − s n | (cid:19) , (9)where sgn( · ) is the sign function.We are now ready to rewrite Problem (7) into the followingBellman’s equation, where subscript n is omitted because theproblem is an infinite-horizon problem with stationary policy. H ∗ ( s ) = min π ∈ [0 , h ( s, π ) + α E I,E
PFC ,q [ H ∗ ( g ( s, π, I, E PFC , q ))] , (10)where h ( s, π ) = E I [ cost e ( s, π )] + E I,E
PFC ,q [ cost p ( s e ( s, π ))] (11)is the expected one stage cost. With a slight abuse of notation,define g ( s, π, I, E P F C , q ) = (cid:20) E max (cid:18) sE max + sgn( π − s ) min ( P max I, | π − s | E max )+( q =1 η − q = − η ) E PFC (cid:19)(cid:21) as the state transition. More specifically, in (11) E I [ cost e ( s, π )] (12) = (cid:18) π>s η − π
PFC ,q [ cost p ( s e )] = E I (cid:2) cost p ( s e ) (cid:3) (14) = R Q cost p (cid:16) s + P max xE max (cid:17) f I ( x ) dx + cost p ( π ) ˜ F I ( Q ) s ≤ π R Q cost p (cid:16) s − P max xE max (cid:17) f I ( x ) dx + cost p ( π ) ˜ F I ( Q ) s > π , where cost p ( s e ) = E E PFC ,q [ cost p ( s e )] (15) = c p p E E PFC "(cid:18) E P F C − E max (1 − s e ) η (cid:19) + + c p p − E E PFC h ( E P F C − ηE max s e ) + i is the expected regulation failure penalty in the case that theSOC is s e when the frequency excursion occurs.IV. O PTIMAL
BESS C
ONTROL
In general, the optimal decision at each stage of a dynamicprogramming is a function of the system state observed atthat stage. That is, we need to calculate the optimal chargingtarget π ∗ ( s ) as a function of the BESS SoC s observed at thebeginning of each I interval. Interestingly, this is not necessaryin our problem. The following theorem states that the optimaltarget SoC is a range that is invariant with respect to theBESS SoC s at each stage. Furthermore, the range convergesto a single point π ∗ that is independent of s when η → or c e ≪ c p ,. This result is extremely appealing: we can pre-calculate π ∗ for all stages offline. This greatly simplifies thesystem operation. Theorem 1.
The optimal target SoC that minimizes the cost H ∗ ( s ) in (10) is a range [ π ∗ low , π ∗ high ] , where π ∗ low and π ∗ high are fixed in all stages regardless of the system state s . Duringeach I interval, the BESS is charged or discharged when itsSoC falls outside the range, and remains idle when its SoC isin the range. In other words, at each stage, the optimal targetSoC π ∗ is set as π ∗ := π ∗ ( s ) = π ∗ low if s < π ∗ low π ∗ high if s > π ∗ high s if s ∈ [ π ∗ low , π ∗ high ] . (16) Moreover, π ∗ low and π ∗ high converge to a single point π ∗ when η → or c e → . To prove Theorem 1, let us first characterise the sufficientand necessary conditions for optimal π ∗ . For convenience,rewrite (10) into H ∗ ( s ) = min π ∈ [0 , H ( s, π ) , where H ( s, π ) = h ( s, π ) + α E I,E
PFC ,q [ H ∗ ( g ( s, π, I, E P F C , q )] . (17)Taking the first order derivative ∂H ( s,π ) ∂π , we obtain the fol-lowing after some manipulations. ∂H ( s, π ) ∂π = ∂h ( s, π ) ∂π (18) + αp ˜ F I ( Q ) R (1 − π ) Emaxη ∂H ∗ ( s ) ∂s (cid:12)(cid:12)(cid:12)(cid:12) s = π + ηeEmax f E PFC ( e ) de + αp − ˜ F I ( Q ) R ηπE max ∂H ∗ ( s ) ∂s (cid:12)(cid:12)(cid:12)(cid:12) s = π − eηEmax f E PFC ( e ) de . Specifically, ∂h ( s, π ) ∂π = ∂∂π E I [ cost e ( s, π )] + ∂∂π E I (cid:2) cost p ( s e ( s, π )) (cid:3) , (19)where ∂∂π E I [ cost e ( s, π )] = ( η c e E max ˜ F I ( Q ) if π > sηc e E max ˜ F I ( Q ) if π < s (20) as a result of differentiating (12), and ∂∂π E I [ cost p ( s e )] = ∂cost p ( π ) ∂π ˜ F I ( Q ) (21) = (cid:18) c p p E max η ˜ F E PFC (cid:18) E max (1 − π ) η (cid:19) − c p p − ηE max ˜ F E PFC ( ηE max π ) (cid:17) ˜ F I ( Q ) as a result of differentiating (14)(15). Note that E I [ cost e ( s, π )] is not differentiable at π = s unless η c e = ηc e (or equivalentlywhen η = 1 or c e = 0 ).Substituting (20) and (21) to (18), we have ∂H ( s, π ) ∂π = (cid:0) r ( s, π ) E max + u ( π ) (cid:1) ˜ F I ( Q ) , where r ( s, π ) is defined in (22) and u ( π ) = αp Z (1 − π ) Emaxη ∂H ∗ (cid:16) π + ηeE max (cid:17) ∂π f E PFC ( e ) de + αp − Z ηπE max ∂H ∗ (cid:16) π − eηE max (cid:17) ∂π f E PFC ( e ) de. (23)To avoid trivial solutions, we assume that the CCDF ˜ F I ( Q ) > for all s, π . Thus, the sign of ∂∂π H ( s, π ) isdetermined by that of r ( s, π ) E max + u ( π ) . As a result, thenecessary condition for optimal π ∗ is r ( s, π ∗ ) E max + u ( π ∗ ) = r (0) E max + u (0) ≥ if π ∗ = 0= r ( π ∗ ) E max + u ( π ∗ ) = 0 if π ∗ ∈ (0 , s )= r ( π ∗ ) E max + u ( π ∗ ) = 0 if π ∗ ∈ ( s, r (1) E max + u (1) ≤ if π ∗ = 1 , (24)when π ∗ = s . On the other hand, when π ∗ = s , r ( s + ) E max + u ( s + ) > and r ( s − ) E max + u ( s − ) < . (25)Now we proceed to show that the necessary conditions (24)and (25) are also sufficient conditions for optimal π ∗ . To thisend, let us first prove the convexity of H ∗ ( s ) in the followingproposition. Proposition 1. H ∗ ( s ) is convex in s . In other words, ∂ H ∗ ( s ) ∂s ≥ for all s . A key step to prove Proposition 1 is to show that ∂ H ∗ ( s ) ∂s isthe fixed point of equation f ( s ) = T f ( s ) , where operator T is a contraction mapping. The details of the proof are deferredto Appendix A.Proposition 1 implies the following Lemma 1, which furtherleads to Proposition 2. Lemma . Both r ( π ) E max + u ( π ) and r ( π ) E max + u ( π ) are increasing functions of π . Moreover, r ( s, π ) E max + u ( π ) is an increasing function of π .The proof of the lemma is deferred to Appendix B. Proposition 2. H ( s, π ) is a quasi-convex function of π . Inother words, one of the following three conditions holds. (a) ∂∂π H ( s, π ) ≥ for all π . (b) ∂∂π H ( s, π ) ≤ for all π . (c) There exists a π ′ such that ∂∂π H ( s, π ) ≤ when π < π ′ and ∂∂π H ( s, π ) ≥ when π > π ′ . The quasi-convexity of H ( s, π ) is straightforward fromLemma 1. It ensures that the necessary condition (24) and(25) is also sufficient. We are now ready to prove our mainresult Theorem 1. Proof of Theorem 1:
We calculate the optimal π ∗ asfollows. Let π ∗ low ∈ [0 , be the root of the equation r ( π ) E max + u ( π ) = 0 . In case the root does not exist , set π ∗ low = 0 if r (0) E max + u (0) > , and π ∗ low = 1 if r (1) E max + u (1) < . Similarly,define π ∗ high ∈ [0 , as the root of the equation r ( π ) E max + u ( π ) = 0 . In case the root does not exist, set π ∗ high = 0 if r (0) E max + u (0) > , and π ∗ high = 1 if r (1) E max + u (1) < .From the definition, r ( π ) E max + u ( π ) > r ( π ) E max + u ( π ) for any given π . Thus, it always holds that π ∗ low ≤ π ∗ high .From the sufficient and necessary conditions in (24) and (25),we can conclude that π ∗ = π ∗ low if s < π ∗ low π ∗ high if s > π ∗ high s if s ∈ [ π ∗ low , π ∗ high ] . (26)In other words, the optimal target SoC is a range [ π ∗ low , π ∗ high ] .Since r ( π ) E max + u ( π ) and r ( π ) E max + u ( π ) are notfunctions of s , π ∗ low and π ∗ high are independent of s . Thus,the range [ π ∗ low , π ∗ high ] is fixed for all stages regardless of thesystem state s .Furthermore, when η = 1 or c e = 0 , r ( π ) = r ( π ) for all π . In this case, π ∗ low = π ∗ high . Thus, the optimal π ∗ becomes asingle point that remains constant for all system states s . Thiscompletes the proof. Remark . Usually, infinite-horizon dynamic programmingproblems are solved by value iteration or policy iterationmethods [17]. Therein, an N -dimensional decision vector isoptimized in each iteration, with each entry of the vector beingthe optimal decision corresponding to a system state. In ourproblem, the system state s is continuous in [0 , . Discretizingit can lead to a large N . Fortunately, the results in this sectionshow that the optimal decision is characterized by two scalars π ∗ low and π ∗ high that remain constant for all system states. Thus,the calculation of the optimal decision is greatly simplified. Abrief discussion on the algorithm to obtain π ∗ low and π ∗ high canbe found in Appendix D.V. O PTIMAL
BESS P
LANNING
Obviously, the minimum operating cost H ∗ ( s ) is a functionof the BESS energy capacity E max . On the other hand, thecapital cost of acquiring and setting up the BESS increaseswith E max . Let the capital cost be denoted as Q ( E max ) , whichis an increasing function of E max . In this section, we are This happens when r (0) E max + u (0) > , i.e., r ( π ) E max + u ( π ) > for all π , or when r (1) E max + u (1) < , i.e., r ( π ) E max + u ( π ) < for all π . r ( s, π ) = r ( π ) := η c e + c p p η ˜ F E PFC (cid:16) E max (1 − π ) η (cid:17) − c p p − η ˜ F E PFC ( ηE max π ) if π > sr ( π ) := ηc e + c p p η ˜ F E PFC (cid:16) E max (1 − π ) η (cid:17) − c p p − η ˜ F E PFC ( ηE max π ) if π < s . (22)interested in investigating the optimal E max that minimizesthe total expected cost λQ ( E max ) + E s [ H ∗ ( s )] , where λ is a weighting factor that depends on the BESS life time,BESS degradation, and the tendering period. E s [ H ∗ ( s )] isthe expected value of H ∗ ( s ) over all initial SoC s under theoptimal charging operation.The main result of this section is given in Theorem 2 below,which states that H ∗ ( s ) is a decreasing convex function of E max for all s . As a result, E s [ H ∗ ( s )] is also a decreas-ing convex function of E max . In other words, the marginaldecrease of the E s [ H ∗ ( s )] diminishes when E max becomeslarge. This implies the existence of a unique optimal E max ,at which the marginal increase of Q ( E max ) is equal to themarginal decrease of E s [ H ∗ ( s )] , i.e., λ ∂Q ( E max ) ∂E max = − ∂ E s [ H ∗ ( s )] ∂E max . Theorem 2.
The minimum operating cost H ∗ ( s ) given in (10) is a decreasing convex function of E max . The proof of Theorem 2 is deferred to Appendix CVI. N
UMERICAL R ESULTS
In this section, we validate our analysis and investigate howdifferent system parameters affect the optimal BESS operationand planning. The simulations are conducted using the real-time frequency measurement data collectd in Sacramanto,CA, as shown in Fig. 2. The sample rate is 10 Hz (i.e.,1 measurement per 0.1 seconds). The data set, provided byFNET/GridEye [18], includes a total of 2,555,377 samples,accounting for about 71 hours of frequency measurement.Suppose that a frequency excursion event occurs when thesystem frequency deviates outside a dead band of 10mHzaround the normative frequency. The empirical distributionsof I , J , and q derived from the measurement data are plottedin Fig. 3.An underlying assumption of our analysis is that I n , J n and q n are i.i.d. for different n , respectively, and that they aremutually independent. To validate this assumption, we plotthe auto-correlations and cross-correlations of the variablesin Figs. 4 and 5, respectively. As we can see from Fig. 4,the auto-correlations of the variables reach the peak when thetime lag is 0 and are close to zero at non-zero time lags,implying that they are approximately independent for different n . Likewise, Fig. 5 shows that the cross-correlations of thevariables are all close to zero, implying that I , J , and q aremutually independent.Before proceeding, let us verify Proposition 1, the convexityof H ∗ ( s ) with respect to s , which is a key step in theproof of our main result. Unless otherwise stated, we assumethat E max = 0 . MWh, P max = 1 MW, P P F C is uniformlydistributed in [0 . , MW, and the discount factor α = 0 . inthe rest of the section. In Fig. 6, we plot H ∗ ( s ) against s S ys t e m f r equen cy Fig. 2. System frequency measured at Sacramanto, CA x F I ( x ) x F J ( x ) −1 100.5 x P r { q = x } Empirical CDF of I Empirical CDF of J Empirical PMF of q Fig. 3. Empirical distributions of I , J , and q when c e = $0 . / kWh and c p = $10 / kWh. The figure verifiesthat H ∗ ( s ) is indeed a convex function of s , as proved inProposition 1. A. Optimal Target SoC
In this subsection, we investigate the effect of varioussystem parameters on the optimal target SoC π ∗ low and π ∗ high .The settings of system parameters are the same as that in Fig. 6unless otherwise stated. In Fig. 7, π ∗ low and π ∗ high are plottedagainst η . It can be seen that when the battery efficiency η is low, [ π ∗ low , π ∗ high ] is a relatively wide interval. The intervalnarrows when η becomes large, and converges to a single pointwhen η → . This is consistent with Theorem 1. Recall thatthere is no need to charge or discharge the battery during an −4000 −3000 −2000 −1000 0 1000 2000 3000 4000−0.500.51 ∆ n Auto−correlation of I −4000 −3000 −2000 −1000 0 1000 2000 3000 4000−0.500.51 ∆ n Auto−correlation of J −4000 −3000 −2000 −1000 0 1000 2000 3000 4000−0.500.51 ∆ n Auto−correlation of q Fig. 4. Auto-correlations of I , J , and q −4000 −3000 −2000 −1000 0 1000 2000 3000 4000−0.500.5 ∆ n −4000 −3000 −2000 −1000 0 1000 2000 3000 4000−0.500.5 ∆ n Crosscorrelation of I and qCrosscorrelation of I and J−4000 −3000 −2000 −1000 0 1000 2000 3000 4000−0.500.5 ∆ n Crosscorrelation of J and q Fig. 5. Cross-correlations of the variables. H * η = 0.3 η = 0.5 η = 0.7 η = 0.9 Fig. 6. Convexity of H ∗ ( s ) with respect to s when c e = $0 . / kWh and c p = $10 / kWh. η O p t i m a l t a r ge t S o C π low * π high * Fig. 7. π ∗ low and π ∗ high versus η when c e = $0 . / kWh and c p = $10 / kWh. I interval if the SoC at the beginning of the I interval isalready within [ π ∗ low , π ∗ high ] . The result in Fig. 7 is intuitivein the sense that when the battery efficiency is low, adjustingSoC during the I intervals is more costly due to power losses.Thus, the interval [ π ∗ low , π ∗ high ] is wider so that the batterySoC does not need to be adjusted too often.In Fig. 8, π ∗ low and π ∗ high are plotted against c p when c e = $0 . / kWh and BESS efficiency η = 0 . . The figureshows that [ π ∗ low , π ∗ high ] is a relatively large interval when c p is comparable with c e . When c p becomes large compared with c e , π ∗ low and π ∗ high converges to a single point, as proved inTheorem 1. Indeed, π ∗ low and π ∗ high overlap when c p is largerthan $35 /kWh. In practice, the regulation failure penalty c p isusually much larger the regular electricity price c e . Thus, wecan safely regard the optimal target SoC as a single point inpractical system designs.Fig. 9 investigates the effect of battery energy capacity E max on the optimal target SoC π ∗ low and π ∗ high . It can beseen that both π ∗ low and π ∗ high become low when E max is verylarge. This can be intuitively explained as follows. Recall that s e is to denote the BESS SoC at the end of an I interval (or thebeginning of a J interval). If s e E max and (1 − s e ) E max areboth larger than the maximum possible E P F C , then regulationfailures are completely avoided, and the operating cost wouldbe dominated by the charging cost during I intervals. When E max is large, there is a wide range of s e that can completelyprevent regulation failures. Out of this range, smaller s e ’sare preferred, so that the charging cost during I intervals islower. This, the optimal target SoCs must be low when E max becomes large. B. Time Response Comparison
To illustrate the advantage of the proposed BESS controlscheme, we compare the operating cost of our scheme with thefollowing three benchmark algorithms proposed in previouswork, e.g., in [19]. p O p t i m a l t a r ge t S o C π low * π high * Fig. 8. [ π ∗ low , π ∗ high ] vs. c p when c e = $0 . / kWh and η =0.8 max (MWh) O p t i m a l t a r ge t S o C π low * π high * Fig. 9. [ π ∗ low , π ∗ high ] vs. E max when c e = $0 . / kWh and c p = $10 / kWh. • No additional charging during I intervals. Referred to“No recharging” in the figures. • Recharge up to during I intervals. Referred to as“Aggressive recharging” in the figures. • Recharge with upper and lower target SoCs. This schemeis similar to our proposed scheme, except that the targetSoCs are set heuristically (instead of optimized in ouralgorithm). In upper and lower target SoCs are set tobe . and . , respectively in [19]. This scheme isreferred to as “Heuristic recharging” in the figures.In particular, we run a time-response simulation using the real-time frequency measurement data in Fig. 2. The probabilityof encountering regulation failures is plotted in Fig. 10. More-over, the time-aggregate operating costs (without discounting)are plotted in Figs. 11 and 12 when E max = 0 . MWh and E max = 1 . MWh, respectively. It can be seen from Figs. 11and 12 that both ”No recharging” and ”Aggressive recharging”algorithms yield much higher cost than the optimal algorithmproposed in the paper. This is because the battery SoC is −1 E max (MWh) P r obab ili t y o f r egu l a t i on f a il u r e Optimal algorithmNo rechargingAggressive rechargingHeuristic recharging
Fig. 10. Comparison of regulation failure probability when c e = $0 . / kWh, c p = $10 / kWh and η =0.8 Time (hour) C o s t ( $ ) Optimal algorithmNo rechargingAggressive rechargingHeuristic recharging
Fig. 11. Comparison of time-aggregate costs when E max = 0 . MWh, c e = $0 . / kWh, c p = $10 / kWh and η =0.8 often too low (with ”No recharging”) or too high (with ”Ag-gressive charging”), yielding much higher regulation failureprobabilities, as shown in Fig. 10. On the other hand, withoptimal target SoC, the proposed algorithm reduces both theoperating cost and regulation failure probability compared with”Heuristic recharging”. C. Optimal BESS Planning
In Fig. 13, we verify Theorem 2 and investigate the effectof BESS energy capacity E max on the operating cost H ∗ .Here, c e = $0 . / kWh, c p = $10 / kWh, η = 0 . , and E max varies from 0.05MWh to to 10MWh. It can be see that H ∗ ( s ) is a decreasing convex function of E max for all initial SoC s . This implies that there exists an optimal BESS energycapacity E max that hits the optimal balance between thecapital investment and operating cost. Time (hour) C o s t ( $ ) Optimal algorithmNo rechargingAggressive rechargingHeuristic recharging
Fig. 12. Comparison of time-aggregate costs when E max = 1 . MWh, c e = $0 . / kWh, c p = $10 / kWh and η =0.8 max (MWh) H * H*(0.1)H*(0.3)H*(0.5)H*(0.7)H*(0.9)
Fig. 13. H ∗ ( s ) vs. E max when c e = $0 . / kWh, c p = $10 / kWh, and η = 0 . . VII. C
ONCLUSIONS
We studied the optimal planning and control for BESSsparticipating in the PFC regulation market. We show that theoptimal BESS control is to charge or discharge the BESSduring I intervals until its SoC reaches a target value. Wehave proved that the optimal target SoC is a range that isinvariant with respect to the BESS SoC s at the beginning ofthe I intervals. This implies that the optimal target SoC can becalculated offline and remain unchanged over the entire systemtime. Hence, the operation complexity can be kept very low.Moreover, the target SoC range reduces to a point in practicalsystems, where the penalty rate for regulation failure is muchlarger than the regular electricity price. It was also shown thatthe optimal operating cost is a decreasing convex functionof the BESS energy capacity, implying the existence of anoptimal energy capacity that balances the capital investmentof BESS and the operating cost.Other than PFC, BESSs can serve multiple purposes, such asdemand response, energy arbitrage, and peak shaving. Differ- ent services require different energy and power capacities. Forexample, PFC reserves do not require high energy capacity,but are sensitive to regulation failures. On the other hand,high energy capacity is needed for demand response, energyarbitrage, and peak shaving. It is an interesting future researchtopic to study the optimal combining of these services in asingle BESS. A PPENDIX AP ROOF OF P ROPOSITION Proof:
First, calculate ∂H ∗ ( s ) ∂s = ∂h ( s, π ∗ ) ∂s (27) + αp Z Q ∗ Z Q ∗ ∂H ∗ ( Q ∗ ) ∂s f E PFC ( e ) f I ( i ) dedi + αp − Z Q ∗ Z Q ∗ ∂H ∗ ( Q ∗ ) ∂s f E PFC ( e ) f I ( i ) dedi where Q ∗ = (1 − s ) E max − sgn( π ∗ − s ) P max iη , (28) Q ∗ = sE max + sgn( π ∗ − s ) P max i + ηeE max , (29) Q ∗ = η ( sE max + sgn( π ∗ − s ) P max i ) , (30) Q ∗ = sE max + sgn( π ∗ − s ) P max i − e/ηE max , (31)and Q ∗ is the same as Q in (13) except that π is replaced by π ∗ in the definition. After some manipulations, we have H ∗ (2) ( s ) = a ( s ) (32) + αp Z Q ∗ Z Q ∗ H ∗ (2) ( s ′ ) (cid:12)(cid:12) s ′ = Q ∗ f E PFC ( e ) f I ( i ) dedi + αp − Z Q ∗ Z Q ∗ H ∗ (2) ( s ′ ) (cid:12)(cid:12) s ′ = Q ∗ f E PFC ( e ) f I ( i ) dedi, where H ∗ (2) ( s ) := ∂ H ∗ ( s ) ∂s and a ( s ) = − sgn( π ∗ − s ) E max P max (cid:0) r ( s, π ∗ ) E max + u ( π ∗ ) (cid:1) f I ( Q ∗ )+ Z Q ∗ ∂ cost p ( s ′ ) ∂s ′ (cid:12)(cid:12)(cid:12)(cid:12) s ′ = s + PmaxiEmax f I ( i ) di. (33)We claim that a ( s ) is non-negative for all s . To see this, notethat − sgn( π ∗ − s ) (cid:0) r ( s, π ∗ ) E max + u ( π ∗ ) (cid:1) ≥ for all s due to the necessary condition of optimal π ∗ in(24) and (25). Thus, the first term of a ( s ) is non-negative.Moreover, the integrand in the second term of a ( s ) is alwaysnon-negative as: ∂ cost p ( s ) ∂s = c p E max (cid:18) p η f E PFC ( E max (1 − s )) + p − η f E PFC ( E max s ) (cid:19) ≥ , (34) where the equality is obtained by taking the second-orderderivative of (15) over s e at s e = s , and the inequality isdue to the fact that PDF functions are non-negative. Thus, a ( s ) ≥ .Define two operators D and T such that Df ( s ) = αp Z Q ∗ Z Q ∗ f ( s ′ ) (cid:12)(cid:12) s ′ = Q ∗ f E PFC ( e ) f I ( i ) dedi + αp − Z Q ∗ Z Q ∗ f ( s ′ ) (cid:12)(cid:12) s ′ = Q ∗ f E PFC ( e ) f I ( i ) dedi, and T f ( s ) = a ( s ) + Df ( s ) . (35)It will be shown in Lemma 2 that the operator T is acontraction mapping. Thus, H ∗ (2) ( s ) is the fixed point ofequation f ( s ) = T f ( s ) , and the fixed point can be achievedby iteration f ( k +1) ( s ) = T f ( k ) ( s ) . Letting f (0) ( s ) = 0 for all s , we can calculate the fixed pointas H ∗ (2) ( s ) = ∞ X i =0 K i ( s ) , where K ( s ) = a ( s ) and K i ( s ) = DK i − ( s ) . Note that D isa summation of two integrals, and therefore is non-negativewhen the integrand is non-negative. Thus, all K i ( s ) ≥ ,because K ( s ) = a ( s ) ≥ . As a result, H ∗ (2) ( s ) ≥ forall s . This completes the proof. Lemma . The operator T defined in (35) is a contractionmapping.To prove the lemma, we can show that T satisfies followingBlackwell Sufficient Conditions for contraction mapping. • (Monotonicity) For any pairs of functions f ( s ) and g ( s ) such that f ( s ) ≤ g ( s ) for all s , T f ( s ) ≤ T g ( s ) . • (Discounting) ∃ β ∈ (0 ,
1) : T ( f + b )( s ) < T f ( s ) + βb ∀ f, b ≥ , s . Proof:
Obviously, Df ( s ) ≤ Dg ( s ) for any pairs of func-tions f ( s ) ≤ g ( s ) , because the operators is a summation of twointegrals with non-negative integrands. Thus, T f ( s ) ≤ T g ( s ) ,and the Monotonicity condition holds.To prove the discounting property, notice that T ( f + b )( s ) = a ( s ) + D ( f + b )( s ) = a ( s ) + Df ( s ) + Db = T f ( s ) + Db, (36)because integrals are linear operations. Moreover, Db = αb p Z Q ∗ Z Q ∗ f E PFC ( e ) f I ( i ) dedi + p − Z Q ∗ Z Q ∗ f E PFC ( e ) f I ( i ) dedi ! ≤ αb p Z Q ∗ f I ( i ) di + p − Z Q ∗ f I ( i ) di ! ≤ αb ( p + p − )= αb. (37) Here, the inequalities are due to the fact that the integrals ofPDF functions are no larger than 1. Since α is a discountingfactor that is smaller than 1, the Discounting condition holds.A PPENDIX BP ROOF OF L EMMA Proof: ∂u ( π ) ∂π = αp Z (1 − π ) Emaxη H ∗ (2) ( s ) (cid:12)(cid:12) s = π + ηeEmax f E PFC ( e ) de + αp − Z ηπE max H ∗ (2) ( s ) (cid:12)(cid:12) s = π − eηEmax f E PFC ( e ) de ≥ , (38)where the equality is obtained by differentiating (23) over π , and the inequality is due to the fact that H ∗ (2) ( s ) ≥ for all s , as proved in Proposition 1. Thus, u ( π ) increaseswith π . Meanwhile, both r ( π ) and r ( π ) are increasingfunctions of π , because ˜ F E PFC ( x ) is a decreasing function of x . Hence, both r ( π ) E max + u ( π ) and r ( π ) E max + u ( π ) areincreasing functions of π . Moreover, when π increases from s − to s + , r ( s, π ) E max + u ( π ) increases by (cid:16) η − η (cid:17) c e from r ( s − ) E max + u ( s − ) to r ( s + ) E max + u ( s + ) . This completesthe proof. A PPENDIX CP ROOF OF T HEOREM Proof:
The proof of convexity of H ∗ ( s ) with respect to E max is similar to that for Proposition 1, and thus is shortenedhere. We first calculate ∂ H ∗ ( s ) ∂E max =˜ a ( s, E max )+ αp Z Q ∗ Z Q ∗ ∂ H ( s ′ ) ∂E max (cid:12)(cid:12)(cid:12)(cid:12) s ′ = Q ∗ f E PFC ( e ) f I ( i ) dedi + αp ˜ F I ( Q ∗ ) Z (1 − π ∗ ) Emaxη ∂ H ( s ′ ) ∂E max (cid:12)(cid:12)(cid:12)(cid:12) s ′ = π ∗ Emax + ηeEmax f E PFC ( e ) de + αp − Z Q ∗ Z Q ∗ ∂ H ( s ′ ) ∂E max (cid:12)(cid:12)(cid:12)(cid:12) s ′ = Q ∗ f E PFC ( e ) f I ( i ) dedi + αp − ˜ F I ( Q ∗ ) Z ηπ ∗ E max ∂ H ( s ′ ) ∂E max (cid:12)(cid:12)(cid:12)(cid:12) s ′ = π ∗ − eηEmax f E PFC ( e ) de, (39)where ˜ a ( s, E max )= − sgn( π ∗ − s ) | π ∗ − s | P max E max f I ( Q ∗ ) ( r ( s, π ∗ ) E max + u ( π ∗ ))+ c p p (cid:18) − π ∗ η (cid:19) f E PFC (cid:18) (1 − π ∗ ) E max η (cid:19) ˜ F I ( Q ∗ )+ c p p − ( ηπ ∗ ) f E PFC ( ηπ ∗ E max ) ˜ F I ( Q ∗ )+ Z Q ∗ c p p (cid:18) − sη (cid:19) f E PFC ( Q ∗ ) + p − ( ηs ) f E PFC ( Q ∗ ) ! f I ( i ) di. (40) We claim that ˜ a ( s, E max ) ≥ for all s and E max . To seethis, note that the first term is always non-negative, because − sgn( π ∗ − s ) ( r ( s, π ∗ ) E max + u ( π ∗ )) ≥ due to (24) and (25). Moreover, the remaining terms are non-negative due to the non-negativeness of PDFs and CCDFs.Same as the proof in Proposition 1, we can show that theright hand side of (39) is a contraction mapping. Thus, we cancalculate ∂ H ∗ ( s ) ∂E max as a fixed point and get ∂ H ∗ ( s ) ∂E max = ∞ X i =0 ˜ K i ( s, E max ) , where all ˜ K i ( s, E max ) ≥ . This implies that ∂ H ∗ ( s ) ∂E max ≥ ,and thus H ∗ ( s ) is convex with respect to E max .Now we proceed to prove that H ∗ ( s ) is a decreasingfunction of E max . We first show that the optimal single-stage cost h ∗ ( s ) = min π h ( s, π ) decreases with E max . Then,the decreasing monotonicity of H ∗ ( s ) with respect to E max can be proved by the monotonicity property of contractionmapping, which is stated in Lemma 2.Recall that ∂h ( s,π ) ∂π = r ( s, π ) E max ˜ F I ( Q ) , where r ( s, π ) isdefined in (22). Thus, the optimal π that minimizes h ( s, π ) satisfies r ( s, π ) = 0 . (41)Furthermore, we can calculate that ∂h ( s, π ) ∂E max = (cid:18) π ≥ s η − π
PFC ,q [ H ( g ( s, π, I, E P F C , q ))] (44)be the contraction operator corresponding to the Bellmanequation in (10). Then, H ∗ ( s ) = lim k →∞ ( T k H )( s ) for all s .Starting with H ( s ) = 0 , we have H ( s ) = T H ( s ) = h ∗ ( s ) . Let h ∗ + ( s ) (or H + k ( s ) ) and h ∗− ( s ) (or H − k ( s ) ) denote h ∗ ( s ) (or H k ( s ) ) with BESS energy capacity E + max and E − max , respectively. We have proved that h ∗ + ( s ) ≤ h ∗− ( s ) , orequivalently H +1 ( s ) ≤ H − ( s ) , if E + max ≥ E − max . Due to themonotonicity property of contraction mapping, H + k ( s ) ≤ H − k ( s ) as long as H + k − ( s ) ≤ H − k − ( s ) for all k . Taking k to infinity,we have H ∗ + ( s ) ≤ H ∗− ( s ) when E + max ≥ E − max . Thiscompletes the proof. A PPENDIX DA LGORITHM TO O BTAIN π ∗ low AND π ∗ high The traditional algorithms to solve infinite-horizon dynamicprogramming problems, e.g., value iteration and policy itera-tion algorithms, involve iterative steps, where in each iteration,the policy π ( i ) is updated for each system state (i.e., BESSSoC) i . In our problem, the state space is continuous in [0 , .If it is discretized into N levels, i.e., i ∈ { , δ, δ, · · · , } where δ = N − , then N optimization problems, one for each π ( i ) , need to be solved in each iteration.Based on the state-invariant property of π ∗ low and π ∗ high , thecomplexity of solving the dynamic programming problem canbe greatly reduced. Define p ij ( π ) = Pr { s n +1 = j | s n = i, π } ,which can be calculated from the distributions of I , J , q , and E P F C . For any given pair of d = ( π low , π high ) , we have p d ij . = p ij ( π ( i )) = p ij ( π low ) i < π low p ij ( π high ) i > π low p ij ( i ) π low ≤ i ≤ π high (45)Let P d be the matrix of p d ij , and H d be the vector of H d ( i ) .Likewise, define vector h d , whose i th entry is h ( i, π low ) when i < π low , h ( i, π high ) when i > π high , and h ( i, i ) when π low ≤ i ≤ π high . Then, H d can be obtained as the solution of (cid:0) I − α P d (cid:1) H d = h d . (46)The optimal π ∗ low and π ∗ high can then be obtained by solving min π low ,π high β T (cid:0) I − α P d (cid:1) − h d , (47)where β is an arbitrary vector . In contrast to the traditionalvalue iteration and policy iteration approaches, no iteration isrequired here. π ∗ low and π ∗ high can be obtained by solving oneoptimization problem (47) with two scalar variables only.R EFERENCES[1] P. Kundur,
Power System Stability and Control . McGraw-Hill, 1994.[2] F. C. Schweppe, R. D. Tabors, J. L. Kirtley, H. R. Outhred, F. H.Pickel, and A. J. Cox, “Homeostatic utility control,”
IEEE Trans. PowerApparatus and Systems , vol. PAS-99, no. 3, pp. 1151–1163, 1980.[3] C. Zhao, U. Topcu, N. Li, and S. Low, “Design and stability of load-sideprimary frequency control in power systems,”
IEEE Trans. AutomaticControl , vol. 59, no. 5, pp. 1177–1189, 2014.[4] C. Zhao, U. Topcu, and S. H. Low, “Optimal load control via frequencymeasurement and neighborhood area communication,”
IEEE Trans.Power Systems , vol. 28, no. 4, pp. 3576–3587, 2013. The fact that π ∗ low and π ∗ high minimize H ( i ) for all i implies that theoptimal solution to (47) is the same for all β . [5] J. Short, D. G. Infield, L. L. Freris et al. , “Stabilization of grid frequencythrough dynamic demand control,” IEEE Trans on Power Systems ,vol. 22, no. 3, pp. 1284–1293, 2007.[6] A. Molina-Garcia, F. Bouffard, and D. S. Kirschen, “Decentralizeddemand-side contribution to primary frequency control,”
IEEE Trans.Power Systems , vol. 26, no. 1, pp. 411–419, 2011.[7] S.-J. Lee, J.-H. Kim, C.-H. Kim, S.-K. Kim, E.-S. Kim, D.-U. Kim,K. K. Mehmood, and S. U. Khan, “Coordinated control algorithm fordistributed battery energy storage systems for mitigating voltage andfrequency deviations,”
To appear, IEEE Trans. Smart Grid .[8] B. Xu, A. Oudalov, J. Poland, A. Ulbig, and G. Andersson, “BESScontrol strategies for participating in grid frequency regulation,” in
WorldCongress , vol. 19, no. 1, 2014, pp. 4024–4029.[9] A. Oudalov, D. Chartouni, C. Ohler, and G. Linhofer, “Value analysisof battery energy storage applications in power systems,” in
IEEE PESPower Systems Conference and Exposition , 2006, pp. 2206–2211.[10] P. Denholm, E. Ela, B. Kirby, and M. Milligan, “The role of energy stor-age with renewable electricity generation,”
Technical Report, NationalRenewable Energy Laborabory , 2010.[11] D. Kottick, M. Blau, and D. Edelstein, “Battery energy storage forfrequency regulation in an island power system,”
IEEE Trans. EnergyConversion , vol. 8, no. 3, pp. 455–459, 1993.[12] T. Borsche, A. Ulbig, M. Koller, and G. Andersson, “Power and energycapacity requirements of storages providing frequency control reserves,”in
IEEE Power and Energy Society General Meeting (PES) , 2013, pp.1–5.[13] P. Mercier, R. Cherkaoui, and A. Oudalov, “Optimizing a battery energystorage system for frequency control application in an isolated powersystem,”
IEEE Trans. on Power Systems , vol. 24, no. 3, pp. 1469–1477,2009.[14] M. R. Aghamohammadi and H. Abdolahinia, “A new approach foroptimal sizing of battery energy storage system for primary frequencycontrol of islanded microgrid,”
Int. J. Electrical Power and EnergySystems , vol. 54, pp. 325–333, 2014.[15] K. Spees and L. B. Lave, “Demand response and electricity marketefficiency,”
Electricity J. , vol. 20, no. 3, pp. 69–85, 2007.[16] S. Isser, “FERC order 719 and demand response in ISO markets,”
GoodCompany Associates/ERCOT Technical Report , 2009.[17] D. P. Bertsekas,
Dynamic programming and optimal control . AthenaScientific Belmont, MA, 1995, vol. 1 and 2, no. 2.[18] Y. Liu, L. Zhan, Y. Zhang, P. Markham, D. Zhou, J. Guo, Y. Lei,G. Kou, W. Yao, J. Chai, and Y. Liu, “Wide-area measurement systemdevelopment at the distribution level: an fnet/grideye example,”
IEEETransactions on Power Delivery , vol. PP, no. 99, pp. 1–1, 2015.[19] A. Oudalov, D. Chartouni, and C. Ohler, “Optimizing a battery energystorage system for primary frequency control,”