Optimal control of a commercial building's thermostatic load for off-peak demand response
aa r X i v : . [ m a t h . O C ] J un Optimal control of a commercial building’sthermostatic load for off-peakdemand response
Randall Martyr ∗† , John Moriarty ‡ , and Christian Beck † School of Mathematical Sciences, Queen Mary University of London,Mile End Road, London E1 4NS, United Kingdom.
June 6, 2018
Abstract
This paper studies the optimal control of a commercial building’s thermostaticload during off-peak hours as an ancillary service to the transmission system opera-tor of a power grid. It provides an algorithmic framework which commercial build-ings can implement to cost-effectively increase their electricity demand at nightwhile they are unoccupied, instead of using standard inflexible setpoint control.Consequently, there is minimal or no impact on user comfort, while the buildingmanager gains an additional income stream from providing the ancillary service,and can benefit further by pre-conditioning the building for later periods. Theframework helps determine the amount of flexibility that should be offered for theservice, and cost optimized profiles for electricity usage when delivering the service.Numerical results show that there can be an economic incentive to participate evenif the payment rate for the ancillary service is less than the price of electricity.
Keywords—
Optimal control, temperature control, ancillary services, reserve ser-vices, demand response, demand turn up.
Notation
Constant quantitiesNotation Description Units T length of control horizon, the period of time duringwhich the building’s temperature is controlled for theancillary service min ∗ Corresponding author. Email: [email protected] † Financial support received from the UK Engineering and Physical Sciences Research Council (EP-SRC) via Grant EP/N013492/1. ‡ Financial support received from the UK Engineering and Physical Sciences Research Council (EP-SRC) via Grant EP/P002625/1. night-time electricity price p/kWh R utilization payment p/kWh X min night-time lower temperature limit ◦ C X max night-time upper temperature limit ◦ Cˆ X temperature limit used for pre-cooling at time T ◦ C X off asymptotic temperature for the building “off” state ◦ C X on asymptotic temperature for the building “on” state ◦ C τ thermal time constant min C max maximum power limit for the building kW U set of normalized cooling power usage variables u – Time-varying quantitiesNotation Description Units x ( t ) building internal temperature at time t ◦ C C ( t ) power usage of the cooling equipment kW C ref ( t ) reference cooling power usage kW C alt ( t ) alternative power usage used to calculate the level ofreserve kW C cap ( t ) level of reserve capacity kW C ask ( t ) reserve service instructions, a profile of additionalpower usage that must be delivered kW C del ( t ) power usage when delivering the reserve service ac-cording to the instructions C ask kW u ( t ) normalized power cooling usage 1 u ref ( t ) normalized reference cooling power usage 1 u alt ( t ) normalized alternative cooling power usage 1 u del ( t ) normalized delivery cooling power usage 1 Electricity supply and demand on a power system must be balanced continuously in realtime to ensure its stability. The system operator, an independent entity that managesthe transmission system [1, p. 3], balances the power system by: • increasing generation or reducing demand when there is a shortfall in supply; • decreasing generation or increasing demand when there is surplus power.The latter, which we refer to as decremental actions [2], are increasingly relevant for powersystems with high levels of intermittent generation from renewable energy sources [3]. Inorder to carry out its balancing duties, the system operator procures a variety of ancil-lary services from third-party companies [1, p. 106]. There is much interest in enablingelectricity consumers to provide ancillary services [4–6], particularly commercial build-ings due to the large flexible demand from their heating, ventilation and air conditioning(HVAC) systems [7–12]. Replacement reserves , which are given more time to respond andare used as back up for faster acting services, can be most suitable in this case [4, p. 32].This paper is a quantitative study of the potential for a commercial building withflexible thermostatic load to participate in a decremental replacement reserve (DRR)2nitiative that is modelled after a real-world example. The setting of this paper is novelin comparison to previous papers such as [11,12]. In the present work the reserve providerbids a schedule of their reserve capacity (in kW) together with a fixed utilization price(per kWh), rather than a variable price depending on the quantity utilized. We providea mathematical and computational framework that maximizes the benefit gained fromparticipating in the DRR service. We also focus on temperature cooling only and notethat heating can be treated symmetrically. Data centres, in particular, are an importantexample since they account for more than 1% of global electricity usage, and their coolinginfrastructure typically accounts for about 40% to 50% of their electricity usage [13].
Demand response refers to any programme that motivates changes in an electricity con-sumer’s normal power usage, typically in response to incentives regarding electricityprices [5]. It is widely considered as a cost-effective and reliable solution for improv-ing the efficiency, reliability, and safety of the power system [5, 6]. There are severalexamples of initiatives that incentivize electricity consumers to reduce their demand, es-pecially during peak hours, and [7] recently studied the potential for buildings to usetheir HVAC systems to participate in such initiatives. Demand response schemes thatprovide incentives for increased electricity demand are much rarer. The Demand TurnUp (DTU) programme offered by National Grid UK, the transmission system operatorin Great Britain, is one such initiative that is meant to incentivize large electricity con-sumers to increase their demand when there is low overall demand on the network andhigh output from renewable generation [14]. It is particularly relevant during the off-peak, night-time hours of interest to this paper, and below we summarize its key aspects(see [14] for further details).
DTU runs during the British Summer Time (BST) period and there are two routes tomarket for candidates: • Fixed DTU is a medium to long-term procurement process that takes place monthsin advance of BST. • Flexible DTU is a rolling short-term procurement process that takes place duringBST and closer to the period that requires the service.Flexible DTU can be advantageous since it gives candidates the flexibility to adjust theirdeclaration in response to weather and market conditions. Our study is more relevantto flexible DTU since we use a dynamic model of temperature evolution that is moreappropriate over the short-term.DTU candidates declare their availability by specifying a schedule for the adjustmentin electricity usage or generation they can provide, including the payment for utilizationof their service. In addition to the utilization payments, successful candidates receiveguaranteed payments for being available during certain windows. Unsuccessful candidatesdo not receive these guaranteed payments, but have the option to participate in DTU forutilization payments only.National Grid UK sends a contracted DTU provider instructions for the service accord-ing to the capability that was declared. The provider has a deadline for acknowledging3eceipt of the DTU instruction, then a delivery period for responding as instructed. ADTU provider which has declared its availability must be able to deliver the service asinstructed or face a penalty.Settlement is the process of compensation for successful provision of the DTU service.A provider has two options for settlement, forecast or baseline, and its choice is fixedfor the contract’s duration. Both options produce a reference profile to which the actualmetered electricity usage or generation is compared, and the difference is settled againstthe DTU service instruction that was sent. The forecast method uses the provider’sprediction for electricity usage or generation during that service period, whereas thebaseline method uses the average metered output from previous entries for that periodin which the provider did not render a DTU service.
As mentioned previously, commercial buildings have significant potential to provide an-cillary services to the transmission system operator. According to [12, p. 1266], capacity and performance are the two main components of this ancillary service provision, eachof which has a magnitude and cost. Capacity refers to the capability that the building’sHVAC system has to provide an ancillary service, whereas performance refers to the workthat the HVAC system does to provide the ancillary service in response to the systemoperator’s instructions. Several papers have argued that buildings can be incentivizedto participate in ancillary services markets, despite high energy prices or less efficientoperating conditions, provided they are adequately compensated [7–9, 11, 12].Reference [12] proposes a methodology for quantifying flexibility and opportunitycosts arising from the provision of ancillary services by buildings’ HVAC systems. Theauthors identify sources of these opportunity costs, and develop a method of accountingfor them through time that is consistent with current practice for generators. This isdone by recognizing the impacts that intra-hour consumption modification associatedwith ancillary service provision have on daily energy efficiency and costs. The authorsof [11] previously addressed a similar problem, but unlike [12] they focused on a building’scapability to alter its total energy use over a period of time, and the methodology putforward was not intended for real-time dynamic operations [11, p. 654]. Both papers useoptimal control to determine the building’s capability to provide a given level of reserveand the associated opportunity cost. By varying the level of reserve, an opportunity costcurve can be constructed and used in the ancillary services market for the purpose ofdispatch by the system operator, or for bidding purposes by the building manager.
In this paper we study the potential for a commercial building to participate in an ancil-lary service scheme such as Demand Turn Up by controlling the electricity it consumes fortemperature cooling during the night. Unlike the setting studied in [11, 12], the reserveprovider bids a schedule of their reserve capacity (in kW) together with a fixed utiliza-tion price (per kWh), rather than a variable price depending on the quantity utilized.Our main contribution is an analysis of the building manager’s incentives in this novelsetting by using the benefit-cost ratio of utilization payment (benefit) to night-time priceof electricity (cost). By varying this ratio, we can see how it affects the magnitude ofcapacity offered for the ancillary service. 4e approach the overall problem of economically providing the ancillary service bybreaking it up into three smaller problems:1.
Reference: determine an optimal reference power profile to use for settlement of theservice.2.
Capacity: determine an optimal capacity power profile to use for declaration ofavailability for the service.3.
Delivery: determine an optimal delivery power profile that fulfils the service in-structions.We formulate each of these problems as a constrained optimal control problem [15],and use the control parametrization method [16, 17] to obtain approximate numericalsolutions for different scenarios. Optimal control is one of several mathematical techniquesthat can be used to optimize the provision of an ancillary service from a commercialbuilding [7, 18, 19]. Moreover, it can be an effective solution for control of the building’sthermostatic load [5, p. 158].Our methodology is suitable for assessing the building’s ability to participate in anydemand response scheme, for either incremental or decremental reserve, where the uti-lization payment is fixed and the reserve provider bids capacity curves. We are able toidentify the incentives that drive the optimal actions, leading to recommendations thatare intuitive and implementable using a variety of control architectures. Consistent withprevious studies [7–9, 11, 12], we find that, besides the dynamics and constraints for theinternal temperature, the level of participation in the ancillary service depends on howwell the building manager is compensated relative to the additional cost incurred. More-over and counter-intuitively, we show that there is an economic incentive to participateeven when the utilization payment is less than the night-time price of electricity.In the following section we present our mathematical framework for optimizing thereference, capacity and delivery power profiles for the reserve service. This frameworkuses the building’s internal temperature as a controlled variable. In principle, any modelthat describes the temperature dynamics using ordinary differential equations can beused. For realistic applications, the temperature relaxation behaviour of a given buildingis measured and used as input to the optimal control scheme. In Section 2.4 we considera linear model for the temperature dynamics, which is used to obtain the numericalsolutions to the optimization problems presented in Section 3. The paper concludes witha summary of the main results and practical recommendations in Section 4.
The control horizon is a period of time during which the building’s internal temperatureis controlled for the ancillary service. Let
T > x = ( x ( t )) ≤ t ≤ T denote the building’s internal temperature in ◦ C during thistime. 5 .1 Constraints for the internal temperature
We suppose that the internal temperature is kept between lower and upper limits X min and X max overnight where X min < X max , X min ≤ x ( t ) ≤ X max , t ∈ [0 , T ] . (1)A pre-cooling operational strategy refers to the act of increasing cooling power and usingthe building’s thermal inertia to reduce the need for cooling power at later periods [20,21]. We include pre-cooling in our framework by imposing a constraint on the finaltemperature value x ( T ) as follows, X min ≤ x ( T ) ≤ ˆ X, (2)where ˆ X in [ X min , X max ] is set by the building manager. Maximum pre-cooling is achievedby setting ˆ X = X min . In this section we outline our methodology for calculating an optimal reference powerprofile, C ref , and an optimal alternative power profile C alt . These profiles are used tocalculate the optimal instantaneous level of reserve capacity C cap by, C cap ( t ) = C alt ( t ) − C ref ( t ) , t ∈ [0 , T ] . (3)If the right-hand side of (3) is negative then the building is unable to deliver instantaneous decremental reserve at that time. It is important to note, however, that the building canstill deliver total decremental reserve over the control horizon, provided Z T C cap ( t ) dt ≥ . Negative instantaneous reserve exemplifies a possible consequence of demand responseknown as the payback effect [22], which occurs when the building’s cooling equipmentrecovers after deviating from its normal operation in order to satisfy operational con-straints.
Optimizing the reference profile.
Suppose there is no request for decremental re-serve during [0 , T ] and the building manager implements the reference profile C ref . Let-ting P ( p/kW h ), where “p” stands for pence, denote the positive and constant night-timeprice of electricity, the total cost to the building manager is,160 Z T P C ref ( t ) dt, (4)where we divide by 60 since T is given in minutes. It is reasonable to choose C ref so thatit minimizes (4), and we formulate an optimal control problem (13) below to accomplishthis. 6 ptimizing the alternative profile. Let R ( p/kW h ) denote the positive utilizationpayment received as a reward for the electricity consumed in excess of the reference level C ref . When decremental reserve is being delivered according to an alternative profile C alt , the instantaneous net cost is, P C alt ( t ) − R (cid:0) C alt ( t ) − C ref ( t ) (cid:1) + , where y + = max( y, total net cost is therefore,160 Z T (cid:2) P C alt ( t ) − R (cid:0) C alt ( t ) − C ref ( t ) (cid:1) + (cid:3) dt. (5)Given the reference profile C ref , it is reasonable to choose C alt so that it minimizes thetotal net cost (5), and we formulate the second optimal control problem (14) to achievethis. Reserve service instructions are described by a power profile C ask that indicates how muchadditional power the building should consume relative to the reference level C ref ( t ). Wesuppose C ask is of the form, C ask ( t ) = N X i =1 C i,ask [ s i ,e i ) ( t ) , (6)where { ( C i,ask , s i , e i ) } i =1 ,...,N is a sequence of N ≥ delivery amount C i,ask (kW), delivery start time s i (min), and delivery end time e i (min),satisfying, ( i ) 0 ≤ s i < e i ≤ T for i ∈ { , . . . , N } , ( ii ) e i = s i +1 for N ≥ i ∈ { , . . . , N − } , and A denotes the indicator function of a set A , A ( y ) = ( , y ∈ A, , y / ∈ A. Therefore, at any time t ∈ [ s i , e i ) the building’s power usage must be at least C i,ask (kW) more than the reference level C ref ( t ). This leads to the following constraint on anydelivery profile C del that satisfies the reserve service instructions, C del ( t ) ≥ C ref ( t ) + C i,ask , s i ≤ t < e i , i = 1 , . . . , N. (7)We assume that the building’s instantaneous consumption can be less than the referencelevel outside of the reserve service instructions’ times, allowing it to recover from providingthe service as needed. The optimization problem used to determine an optimal C del isformulated as (16) below. 7 .4 Internal temperature modelling Linear dynamics.
We assume that x evolves according to the following linear dynamics[22, 23]: ˙ x ( t ) = − τ [ x ( t ) − X off + ( X off − X on ) u ( t )] ,x (0) ∈ [ X on , X off ] , (8)where ˙ x is the time derivative of x and, • X on and X off are the asymptotic temperatures reached when the cooling equipmentoperates in the “on” and “off” states respectively, with X on < X min and X max
Let U denote the set of normalized power profiles (cid:0) u ( t ) (cid:1) ≤ t ≤ T where u : [0 , T ] → [0 , Constant temperature control.
Suppose the temperature x is constant over an in-terval [¯ t , ¯ t ] with 0 ≤ ¯ t < ¯ t ≤ T . Using (8) any control ¯ u ∈ U that achieves this steadycondition satisfies, − τ [ x (¯ t ) − X off + ( X off − X on )¯ u ( t )] = 0 , t ∈ [¯ t , ¯ t ] , and therefore, ¯ u ( t ) = ¯ u (¯ t ) = X off − x (¯ t ) X off − X on , t ∈ [¯ t , ¯ t ] . (10) Analytic solution for step controls.
Suppose u ∈ U satisfies, u ( t ) = n p X k =1 u k [ t k − ,t k ) ( t ) , t ∈ [0 , T ] , (11)where n p > { t k } n p k =0 is a sequence of time points 0 = t < . . . < t n p = T that partition the control horizon [0 , T ] into n p > x ( t )) ≤ t ≤ T to (8) corresponding to the control (11) is continuous and satisfies, x ( t ) = e − t − tk − τ x ( t k − ) + (cid:16) − e − t − tk − τ (cid:17)(cid:0) X off + ( X on − X off ) u k (cid:1) ,t k − ≤ t < t k , ≤ k ≤ n p . (12)8 .5 Three optimal control problems for off-peak decrementalreplacement reserve provision Problem 1. Optimal reference power usage.
The reference power profile C ref should conduce minimum expenditure if decremental reserve is not requested over thecontrol horizon. The following optimal control problem, formulated using the normalizedreference profile, is suitable for our application.minimize Z T (cid:2) u ref ( t ) + α ref (cid:0) u ref ( t ) (cid:1) (cid:3) dt over u ref ∈ U subject to:( i ) ˙ x ( t ) = f ( t, x ( t ) , u ref ( t )) given by (8) , ( ii ) X min ≤ x ( t ) ≤ X max , t ∈ [0 , T ] , ( iii ) x (0) ∈ [ X min , X max ] and x ( T ) ∈ [ X min , ˆ X ] , (13)where α ref > regularization term (cid:0) u ref ( t ) (cid:1) . The regularizer is used in (13) to disfavour solutions where u ref alternatesrapidly between its minimum and maximum possible values. In the absence of thisregularizer, theory states that this unwanted behaviour can be optimal in (13) sincethe control variable u ref then appears linearly in both the objective function and statedynamics [24]. Problem 2. Optimal alternative power usage.
Given the normalized referencepower profile u ref , night-time electricity price P , and utilization payment R ,minimize J ( u alt ; u ref , P, R ) over u alt ∈ U subject to:( i ) ˙ x ( t ) = f ( t, x ( t ) , u alt ( t )) given by (8) , ( ii ) X min ≤ x ( t ) ≤ X max , t ∈ [0 , T ] , ( iii ) x (0) ∈ [ X min , X max ] and x ( T ) ∈ [ X min , ˆ X ] , ( iv ) Z T (cid:2) u alt ( t ) − u ref ( t ) − RP (cid:0) u alt ( t ) − u ref ( t ) (cid:1) + (cid:3) dt ≤ , (14)where J ( u alt ; u ref , P, R ) is given by, J ( u alt ; u ref , P, R ) = Z T (cid:2) u alt ( t ) − RP (cid:0) u alt ( t ) − u ref ( t ) (cid:1) + (cid:3) dt + α alt Z T (cid:0) u alt ( t ) (cid:1) dt, (15)and, similar to (13) above, α alt > u alt instead of u ref , and is always satisfied when R ≥ P . If “0” onthe right-hand side of (14)–(iv) is replaced by “ − γ ” where γ ≥ γ (cid:0) C max P (cid:1) pence relativeto the reference profile’s cost. 9 roblem 3. Optimal reserve service delivery. Since the building manager is onlycompensated for the additional demand as instructed, it is reasonable to require thatthe building uses no more power than that needed to satisfy the reserve instructions andinternal temperature limits. Using the normalized profiles u ref and u ask , we thereforeformulate the reserve service delivery problem as follows.minimize Z T (cid:2) u del ( t ) + α del (cid:0) u del ( t ) (cid:1) (cid:3) dt over u del ∈ U subject to:( i ) ˙ x ( t ) = f ( t, x ( t ) , u del ( t )) given by (8) , ( ii ) X min ≤ x ( t ) ≤ X max , t ∈ [0 , T ] , ( iii ) x (0) ∈ [ X min , X max ] and x ( T ) ∈ [ X min , ˆ X ] , ( iv ) u del ( t ) ≥ u ref ( t ) + u i,ask , s i ≤ t < e i , i = 1 , . . . , N, (16)where, similar to (13) above, α del > u del ( t ) ≥ u ins ( t ) , t ∈ [0 , T ] , where u ins is the profile of minimal instructed power usage, u ins ( t ) = N X i =1 (cid:2) u ref ( t ) + u i,ask (cid:3) [ s i ,e i ) ( t ) , and we used the property u del ( t ) ≥ Theory, for example [15, 25], guarantees the existence of a solution to each of the prob-lems (13), (14), and (16). In this section we present results of the associated numericalsolutions, which we obtained using the control parametrization method [16, 17] outlinedin Appendix A. The hypothetical building for the experiments has computing equipmentthat generates a significant amount of electricity demand for temperature cooling. Thecontrol horizon is T = 360 minutes long, lasting from 00:00 to 06:00. We assume an inter-nal temperature range of X min = 18 ( ◦ C) to X max = 27 ( ◦ C) must be maintained duringthis time, which is the recommended range in the ASHRAE Standard 90.4-2016 for datacentres [26]. For the simulations we set the utilization payment at either 75%, 100% or125% of the electricity price. According to the 2017 DTU market information [14], theutilization payment was typically in the range R ∈ [6 ,
10] (p / kWh). Table 2 lists valuesfor the parameters used to generate the numerical results.
In this section we highlight important characteristics of the control and temperature tra-jectories corresponding to an optimal reference power profile u ref . These trajectories areintuitive and show that the optimal control uses minimal effort to keep the temperaturewithin its constraints. In particular, 10able 2: Parameters for numerical simulations. Parameter Value Units T
360 minutes τ
120 minutes X off ◦ C X on ◦ C x (0) 27 ◦ C X min ◦ C X max ◦ C RP RP ∈ { , , }
11. Starting at the maximum feasible level X max , the temperature is kept constant atthis level until a time ¯ t ∈ (0 , T );2. From time ¯ t onwards, power consumption is increased steadily until it reaches themaximum level at a time ¯ t ∈ (¯ t , T );3. From time ¯ t to T , maximum power is used to steer the temperature to ˆ X , theupper limit of feasible values at time T .Consequently, when following the optimal reference profile u ref , the building has capacityto provide decremental reserve on [0 , ¯ t ], but is unavailable to provide this service fromtime ¯ t onwards when reference power usage is at its highest. Note that these optimalcharacteristics are sensible for temperature dynamics other than the linear one (8).The linear model allows for easy approximation of the terms described above. Forexample, using (10) the control u ref satisfies, u ref ( t ) ≈ X off − X max X off − X on , t ∈ [0 , ¯ t ] . (17)Using (12), the time ¯ t in the description approximately satisfies,ˆ X = (cid:0) e − T − ¯ t τ (cid:1) X max + (cid:0) − e − T − ¯ t τ (cid:1) X on , which we solve to get, ¯ t ≈ T − τ log (cid:18) X max − X on ˆ X − X on (cid:19) . (18)The time ¯ t can be determined by calculating how long it takes to go from the constantpower level at ¯ t to full power at ¯ t using (17) and (18). Using (18) we see that theduration of unavailability, T − ¯ t , is proportional to the building’s thermal time constant τ , and also increases with the level of pre-cooling at time T , which is controlled by theparameter ˆ X . This section illustrates the numerical results for the reserve capacity problem (14). Fig-ure 1 shows the alternative power usage and capacity profiles corresponding to three11 .000.250.500.751.00 u ( t ) ̄t ̂t0 60 120 180 240 ̂00 ̂60 Timē[min] x ( t )[ ∘ ∘ ] −1.0−0.50.00.51.0 u c a p ( t ) (a) RP = u ( t ) ̄t Timē[min] x ( t )[ ∘ ∘ ] −1.0−0.50.00.51.0 u c a p ( t ) (b) RP = 1 u ( t ) ̄t ̂t0 60 120 180 240 ̂00 ̂60 Timē[min] x ( t )[ ∘ ∘ ] u alt (t)u ref (t)x alt (t)x ref (t)u cap (t)−1.0−0.50.00.51.0 u c a p ( t ) (c) RP = RP010203040506070 NN P ( u a l t ; u r e f , R , P ) (d) Normalized net profit Figure 1: Optimized internal temperature x , alternative power usage u alt and reservecapacity u cap corresponding to three different values for the benefit-cost ratio RP . Thetime ¯ t (cf. (18)) at which the reference profile starts to apply maximum power near thecontrol horizon’s end is shown in each case. In (a) RP = and the building tends to beavailable for decremental reserve at maximum capacity on short intervals. The longestduration of reserve occurs on an interval [ˆ t, ¯ t ] where the temperature is cooled from themaximum allowed value to the minimum one. In (b) RP = 1 and the building’s capacityfor decremental reserve is gently increased up to the maximum value before ¯ t . In (c) RP = and the building is mostly available for decremental reserve. On average, lessthan maximum power is sustained on an interval [ˇ t, ¯ t ] while the temperature is at itsminimum allowed value. Figure (d) shows that, as expected, the normalized net profitincreases with RP . Parameter values are x (0) = 28 for the initial temperature, ˆ X = 18 forthe pre-cooling value, and α ref = α alt = 0 .
01 for the control regularizers. Temperatureand control constraints are shown using dotted horizontal lines.different values of RP . It also shows the normalized net profit relative to the referenceprofile for the three cases, which is defined as the negative of constraint (14)-(iv), N N P ( u alt ; u ref , R, P ) = Z T (cid:2) u ref ( t ) − u alt ( t ) + RP (cid:0) u alt ( t ) − u ref ( t ) (cid:1) + (cid:3) dt. Multiplying this value by (cid:0) C max P (cid:1) gives the total net profit relative to the reference profilein pence. Case 1: RP = In Figure 1(a) the benefit-cost ratio satisfies RP <
1, and the optimal control tends touse short bursts of pre-cooling to minimize the cost of providing reserve capacity. Con-12equently, there are many intervals during which the building is either unavailable fordecremental reserve, or available for short periods at maximum capacity. The longest pe-riod of sustained maximal capacity occurs just before time ¯ t when the reference profilepower usage is at its highest. The normalized level of sustained capacity during this timeis approximately, u cap ( t ) ≈ X max − X on X off − X on , t ∈ [ˆ t, ¯ t ] , (19)where, analogous to (18) above for the reference profile, ˆ t is given by,ˆ t ≈ ¯ t − τ log (cid:18) X max − X on X min − X on (cid:19) . (20)Using the expression for ¯ t in (18) and ˆ X = X min shows,¯ t − ˆ t ≈ T − ¯ t , and u alt essentially time-shifts the final period of maximum consumption that occurredunder the reference profile u ref . It is important to note that the net cost of providingmaximum reserve capacity increases as R decreases. In the presence of thermal losses,this means there may no longer be any profitable solutions to (14) of this form if R is toolow. Case 2: R = P In Figure 1(b) the benefit-cost ratio satisfies RP = 1 and the alternative profile u alt exhibitscomplex bang-bang behaviour early within the control horizon. This complex behaviourcan be explained by noticing that when RP = 1 the instantaneous cost in (15) satisfies, u alt ( t ) − RP (cid:0) u alt ( t ) − u ref ( t ) (cid:1) + + α alt (cid:0) u alt ( t ) (cid:1) = min (cid:0) u alt ( t ) , u ref ( t ) (cid:1) + α alt (cid:0) u alt ( t ) (cid:1) . (21)When α alt is low, the term min (cid:0) u alt ( t ) , u ref ( t ) (cid:1) can dominate (21) and incentivize u alt to use bang-bang control behaviour in order to be more cost effective than the referenceprofile u ref , which already uses minimal effort. This period of complex control behaviouris clearly unsuitable for providing decremental reserve. Moreover, even after this periodthe profile u alt may not be practical since, unlike in Figure 1(a), it slowly ramps up tomaximum consumption and provides only a short period of constant reserve before ¯ t . Case 3: RP = In Figure 1(c) the benefit-cost ratio satisfies RP >
1. The control u alt initially appliesmaximum power to steer the internal temperature to its minimum allowable value X min .Let ˇ t denote the first time that the temperature hits X min . Analogous to (20), ˇ t is givenapproximately by, ˇ t ≈ τ log (cid:18) X max − X on X min − X on (cid:19) , (22)which we recall is equal to T − ¯ t , the length of time that the reference profile u ref appliesmaximum power. The control u alt keeps the temperature at X min from ˇ t until the time¯ t by applying power that is on average equal to (cf. (10)), u alt ( t ) ≈ X off − X min X off − X on , t ∈ [ˇ t, ¯ t ] , u cap ( t ) ≈ X max − X min X off − X on , t ∈ [ˇ t, ¯ t ] . (23) In this section we summarize the numerical results for the optimal reserve service deliveryproblem (16). For further details see Appendix B. The optimized delivery profile usesminimal power to satisfy the temperature constraints and reserve service instructions.After delivering the service, the internal temperature is lower than the reference level,and the cooling equipment can be turned off temporarily while the temperature riseswithin its permitted range. The building manager therefore benefits doubly, by receivingthe utilization payment for delivering the reserve service, and by reducing the electricityusage consequent to pre-cooling. The cooling equipment remains off for a longer durationif less pre-cooling is required at the control horizon’s end.
As the energy transition transforms power grids across the globe, high levels of intermit-tent renewable generation complicate the job of continuously balancing power supply anddemand, which is necessary for the grid’s stability. New ancillary services have emerged inthis regard, such as National Grid UK’s Demand Turn Up (DTU) [14], which is a reserveservice that incentivizes large energy consumers to increase their electricity demand, forexample during overnight periods of high output from renewable generation and low over-all demand. In this paper we explore the optimal participation of a commercial building,through the control of its temperature cooling equipment, in such an ancillary serviceinitiative. We provide a computational framework for solving this problem that takesinto account the economic incentives given. The framework has three main outputs:1. an optimal reference night-time control schedule for the cooling system when it doesnot provide DTU.2. an optimal schedule of DTU capacity relative to the reference for a given remuner-ation.3. an optimal night-time control schedule to fulfil DTU instructions.The framework also takes into account the building’s relaxation dynamics, so that DTUrequests are used as an opportunity to optimally pre-cool the building. In addition tothe DTU payment, this pre-cooling reduces energy usage during the subsequent morningpeak period, which is a financial benefit to the customer and also reduces stress on thegrid.The optimal control schedule used as reference or to fulfil the DTU instructions isintuitive and uses minimal effort to satisfy the temperature and power constraints. Con-sistent with studies such as [11, 12], we find that the level of participation in the ancillaryservice is affected by the dynamics and constraints for the internal temperature, and howwell the building manager is remunerated. Figure 1 shows that the building’s capacityfor DTU depends crucially on the benefit-cost ratio RP , where R is the utilization paymentfor DTU and P is the night-time price of electricity.14 When RP < • When RP = 1 the optimal alternative control schedule also has a complicated struc-ture but is not implementable since it lacks sufficiently long periods of constantreserve. • When RP > RP = ), participation in DTU was found to be profitable. This is becausesimply shifting pre-planned HVAC operation to a time earlier in the night leads to in-creased demand at the earlier time, attracting compensation under DTU. However when RP < R decreases, fluctuatingmore frequently between minimum and maximum power as it “hunts” for profit. De-mand reductions due to payback effects are undesirable as they undermine the purposeof DTU, and frequent rapid power fluctuations may be problematic for system stability.Therefore, in order to economically incentivize a building manager to provide DTU in apractical way, the level of remuneration must be sufficiently high.Possible future extensions of our work would include controlling an ensemble of pos-sibly heterogeneous thermostatic loads [22] and considering measured temperature re-laxation dynamics for each member of the ensemble. The heterogeneity can come fromdifferent rooms in a single building, or from an aggregation of multiple buildings. Ourmodel may also be extended to consider the uncertainty in parameters affecting the in-ternal temperature, such as the external weather conditions, or the uncertainty in beingcalled to provide DTU. References [1] D. S. Kirschen and G. Strbac,
Fundamentals of Power System Economics . Chicester,England: John Wiley & Sons, 2004.[2] D. Z. Szab´o and R. Martyr, “Real option valuation of a decremental regulation ser-vice provided by electricity storage,”
Philosophical Transactions of the Royal SocietyA: Mathematical, Physical and Engineering Sciences , vol. 375, no. 2100, aug 2017.[3] M. Rothleder and C. Loutan, “Case Study - Renewable Integration: FlexibilityRequirement, Potential Overgeneration, and Frequency Response Challenges,” in
Renewable Energy Integration - Practical Management of Variability, Uncertainty,and Flexibility in Power Grids , L. E. Jones, Ed. New York: Elsevier Inc., 2014,ch. 6, pp. 67–79. 154] E. Hirst and B. Kirby, “Unbundling Generation and Transmission Services for Com-petitive Electricity Markets: Examining Ancillary Services,” The National Regu-latory Research Institute, The Ohio State University, Columbus, OH, USA, Tech.Rep., jan 1998.[5] J. S. Vardakas, N. Zorba, and C. V. Verikoukis, “A Survey on Demand ResponsePrograms in Smart Grids: Pricing Methods and Optimization Algorithms,”
IEEECommunications Surveys & Tutorials , vol. 17, no. 1, pp. 152–178, 2015.[6] N. G. Paterakis, O. Erdin¸c, and J. P. Catal˜ao, “An overview of Demand Response:Key-elements and international experience,”
Renewable and Sustainable Energy Re-views , vol. 69, pp. 871–891, mar 2017.[7] S. J. Olivieri, G. P. Henze, C. D. Corbin, and M. J. Brandemuehl, “Evaluationof commercial building demand response potential using optimal short-term cur-tailment of heating, ventilation, and air-conditioning loads,”
Journal of BuildingPerformance Simulation , vol. 7, no. 2, pp. 100–118, mar 2014.[8] G. S. Pavlak, G. P. Henze, and V. J. Cushing, “Optimizing commercial buildingparticipation in energy and ancillary service markets,”
Energy and Buildings , vol. 81,pp. 115–126, oct 2014.[9] T. M. Lawrence, M.-C. Boudreau, L. Helsen, G. Henze, J. Mohammadpour, D. Noo-nan, D. Patteeuw, S. Pless, and R. T. Watson, “Ten questions concerning integratingsmart buildings into the smart grid,”
Building and Environment , vol. 108, pp. 273–283, nov 2016.[10] Y.-J. Kim, D. H. Blum, N. Xu, L. Su, and L. K. Norford, “Technologies and Mag-nitude of Ancillary Services Provided by Commercial Buildings,”
Proceedings of theIEEE , vol. 104, no. 4, pp. 758–779, apr 2016.[11] R. De Coninck and L. Helsen, “Quantification of flexibility in buildings by cost curves– Methodology and application,”
Applied Energy , vol. 162, pp. 653–665, jan 2016.[12] D. H. Blum, T. Zakula, and L. K. Norford, “Opportunity Cost Quantification forAncillary Services Provided by Heating, Ventilating, and Air-Conditioning Systems,”
IEEE Transactions on Smart Grid , vol. 8, no. 3, pp. 1264–1273, may 2017.[13] M. Dayarathna, Y. Wen, and R. Fan, “Data Center Energy Consumption Modeling:A Survey,”
IEEE Communications Surveys & Tutorials
Functional Analysis, Calculus of Variations and Optimal Control , ser.Graduate Texts in Mathematics. London, England: Springer London, 2013, vol.264.[16] K. L. Teo and C. J. Goh,
A unified computational approach to optimal control prob-lems . Essex, England: Longman Scientic & Technical, 1991.1617] V. Rehbockt, K. L. Teo, L. S. Jennings, and H. W. J. Lee, “A Survey of the ControlParametrization and Control Parametrization Enhancing Methods for ConstrainedOptimal Control Problems,” in
Progress in Optimization (Applied Optimization Vol30) . Springer US, 1999, pp. 247–275.[18] R. Deng, Z. Yang, M.-Y. Chow, and J. Chen, “A Survey on Demand Responsein Smart Grids: Mathematical Models and Approaches,”
IEEE Transactions onIndustrial Informatics , vol. 11, no. 3, pp. 570–582, jun 2015.[19] S. Wang and Z. Ma, “Supervisory and Optimal Control of Building HVAC Systems:A Review,”
HVAC&R Research , vol. 14, no. 1, pp. 3–32, jan 2008.[20] T. Reddy, L. Norford, and W. Kempton, “Shaving residential air-conditioner elec-tricity peaks by intelligent use of the building thermal mass,”
Energy , vol. 16, no. 7,pp. 1001–1010, 1991.[21] K. Roth, J. Dieckmann, and J. Brodrick, “Emerging Technologies: Using Off-PeakPrecooling,”
ASHRAE Journal , vol. 51, no. 3, mar 2009.[22] S. H. Tindemans, V. Trovato, and G. Strbac, “Decentralized Control of ThermostaticLoads for Flexible Demand Response,”
IEEE Transactions on Control Systems Tech-nology , vol. 23, no. 5, pp. 1685–1700, 2015.[23] S. Ihara and F. Schweppe, “Physically Based Modeling of Cold Load Pickup,”
IEEETransactions on Power Apparatus and Systems , vol. PAS-100, no. 9, pp. 4142–4150,1981.[24] H. Maurer, “On Optimal Control Problems with Bounded State Variables and Con-trol Appearing Linearly,”
SIAM Journal on Control and Optimization , vol. 15, no. 3,pp. 345–362, may 1977.[25] L. Cesari,
Optimization—Theory and Applications . New York, NY, USA: SpringerNew York, 1983.[26] ASHRAE, “Standard 90.4-2016 – Energy Standard for Data Centers,” 2016.
A Description of the numerical method
We use the control parametrization method [16, 17] to obtain an approximate solutionto the optimal control problem. Let { t k } n p k =0 denote a sequence of time points used topartition the control horizon [0 , T ] into n p > n p is an integer, andlet U n p ⊂ U denote the corresponding subclass of step controls (cf. (11)). Each control u ∈ U n p is parametrized by an n p -dimensional vector with components u k ∈ [0 ,
1] for k = 1 , . . . , n p .The original optimal control problem is approximated by optimizing over the smallersubclass of controls U n p , which can be treated as a constrained nonlinear optimizationproblem over the bounded n p -dimensional parameter space defining controls u ∈ U n p .Standard optimization packages can be used to solve this problem, and we used theSequential Least Squares Programming (SLSQP) routine in Python.17 .1 Loss functions for state and control constraints In order to use the control parametrization method, we define constraints for the stateand control variables in functional form. We only describe the constraints for Problem(16), noting that those for the other optimal control problems are defined analogously.First define ( t, x, u ) ψ ( t, x, u ) and x φ ( x ) by, ( ψ ( t, x, u ) = ( X max − x )( x − X min ) φ ( x ) = ( ˆ X − x )( x − X min ) (A.1)By definition, we say that the integral constraint ψ is satisfied at ( t, x, u ) if and onlyif ψ ( t, x, u ) ≥
0. Similarly, the terminal constraint φ is satisfied at x if and only if φ ( x ) ≥
0. Equation (A.1) corresponds to the time-dependent pure state constraints onthe internal temperature over [0 , T ] (cf. (1)) and at time T (cf. (2)). In a similar way wedefine integral and terminal constraints for DTU, ( ψ ( t, x, u ) = u − P Ni =1 (cid:2) u ref ( t ) + u i,ask (cid:3) [ s i ,e i ) ( t ) φ ( x ) = 0 (A.2)Using these constraints we define loss rate functions ( t, x, u ) Ψ η ( t, x, u ) and termi-nal loss functions x Φ η ( x ), η ∈ { , } , by, ( Ψ η ( t, x, u ) = (min(0 , ψ η ( t, x, u ))) Φ η ( x ) = (min(0 , φ η ( x ))) (A.3)The loss functions Ψ η and Φ η are combined to create a total loss for the constraints, C η ( u ) = Z T Ψ η ( t, x ( t ) , u ( t )) dt + λ η Φ η ( x ( T )) , (A.4)where λ η > η . The total loss C η is non-negative by construction and is equal to zero if, equivalently, the relevant constraints aresatisfied on [0 , T ]. We relax this condition by requiring, C η ( u ) ≤ ε η , η ∈ { , } , (A.5)where ε η ≥ [0 , ∞ ) ( y ) ≈ e θy e θy , [ a,b ] ( y ) ≈ (cid:18) e θ ( y − a ) e θ ( y − a ) (cid:19) (cid:18) e θ ( b − y ) e θ ( b − y ) (cid:19) for a < b, max(0 , y ) ≈ θ log(1 + e θy ) , where θ > Optimal power usage illustrations x ( t )[ ∘ C ] ̄t Internal∘temperat re
Time∘[min] u r e f ( t ) Normalized∘power∘ sage (a) ˆ X = 18 ( ◦ C) x ( t )[ ∘ C ] ̄t Internal∘temperat re
Time∘[min] u r e f ( t ) Normalized∘power∘ sage (b) ˆ X = 20 ( ◦ C) Figure 2: Optimized internal temperature u and reference power profile u ref correspond-ing to two different values for the pre-cooling temperature ˆ X . The profile u ref keeps thetemperature constant at the maximum level for some time, and then quickly ramps upto full power around time ¯ t (vertical dash-dotted line, see (18)) so that the temperaturehits ˆ X exactly at the terminal time. The initial temperature is x (0) = 27 and regular-izer has value α ref = 0 .
01. Control and temperature constraints are shown using dottedhorizontal lines. x ( t )[ ∘ C ] Time∘[min] u ( t ) (a) ˆ X = 18 x ( t )[ ∘ C ] Time∘[min] u ( t ) x del (t)x ref (t)u del (t)u ref (t)u ins (t) (b) ˆ X = 20 Figure 3: Optimized internal temperature x and delivery, reference, and minimum powerprofiles, u del , u ref , and u ins corresponding to two different values for the pre-coolingtemperature ˆ X , shown using the dot-dash line. Reserve service instructions call foran increase in normalized power by 0 . . u del uses minimal power to satisfy the temperatureconstraints and reserve service instructions. The building is also pre-cooled after theservice has been delivered. Parameters values are x (0) = 27 for the initial temperatureand α ref = α del = 0 .
01 for the regularizers. Temperature and control constraints areshown using dotted horizontal lines. 19 .000.250.500.751.00 u ( t ) ̄t Timē[min] x ( t )[ ∘ ∘ ] −1.0−0.50.00.51.0 u c a p ( t ) (a) α alt = 10 u ( t ) ̄t Timē[min] x ( t )[ ∘ ∘ ] −1.0−0.50.00.51.0 u c a p ( t ) (b) α alt = 1 u ( t ) ̄t Timē[min] x ( t )[ ∘ ∘ ] −1.0−0.50.00.51.0 u c a p ( t ) (c) α alt = u ( t ) ̄t Timē[min] x ( t )[ ∘ ∘ ] u alt (t)u ref (t)x alt (t)x ref (t)u cap (t)−1.0−0.50.00.51.0 u c a p ( t ) (d) α alt = Figure 4: Optimized internal temperature x , alternative power usage u alt and reservecapacity u cap corresponding to the benefit-cost ratio RP = 1 and decreasing values for theregularizer α alt ∈ { , , , } . When α alt is large, the alternative profile u alt tries tominimize the overall cost of consumption, similarly to the reference profile u ref . However,as α alt decreases u alt increasingly tries to stay below u ref , leading to rapid bang-bangcontrol behaviour. Parameter values are x (0) = 25 and ˆ X = 18 for the initial and pre-cooling temperatures respectively. Temperature and control constraints are shown usingdotted horizontal lines whilst ¯ t2