Grocery Store Flexibility Management Using Model Predictive Control With Neural Networks
GG ROCERY STORE FLEXIBILITY MANAGEMENT USING MODELPREDICTIVE CONTROL WITH NEURAL NETWORKS
A P
REPRINT
Roope Sarala
VTT Technical Research Centre of FinlandOulu, Finland [email protected]
Jussi Kiljander
VTT Technical Research Centre of FinlandOulu, Finland [email protected]
January 22, 2020 A BSTRACT
As more and more energy is produced from renewable energy sources (RES), the challenge forbalancing production and consumption is being shifted to consumers instead of the power grid. Thisrequires new and intelligent ways of flexibility management at individual building and district levels.To this end, this paper presents a model based optimal control (MPC) algorithm embedded with deepneural network for day-ahead consumption and production forecasting. The algorithm is used tooptimize a medium-sized grocery store energy consumption located in Finland. System was testedin a simulation tool utilising real-life power measurements from the grocery store. We report a . reduction in daily peak loads with flexibility provided by a kWh battery. On the otherhand, a significant benefit was not seen in trying to optimize with respect to the energy spot price.We conclude that our approach is able to significantly reduce peak loads in a grocery store withoutadditional operational costs. Flexibility management is becoming more and more important as increasing amount of electricity is produced fromrenewable sources. In addition, a large portion of renewable production is close to consumers thus requiring localelectricity balancing/management. Large amount of renewable production can cause problems in a traditional gridsystem not designed local energy production in mind. In order to meet the demands of increasing renewable production,local flexibility management is required at local level. Additionally, flexibility management also provides means fordecreasing energy costs, either via shifting load consumption to low spot price periods, or by reducing peak loads thatcontribute to the energy transmission cost via power tariffs.Important flexibility providers in a local power grid are grocery stores with large refrigeration systems offering apotential flexibility storage in the form of heat energy. There are various ways of utilizing flexibility in a grocery store,for example, • Load shifting with respect to spot price. • Storing excess production. • Load shifting with respect to peak demand.In recent years, a lot of work has been put in studying building flexibility management with various methods proposed[1, 2, 3, 4, 5]. Generally, these methods can be split into model-free and model-based strategies. As the names suggest,the key difference between these approaches is that the former requires no building model and latter does.Much of the recent work [6, 7, 8, 9] using model-free strategies has been done employing deep neural networks inreinforcement learning framework, such as Q-learning [10]. While it is a great approach in flexibility management as itis generally scalable and offers large performance potential, it has some downsides which make them not optimal for a r X i v : . [ ee ss . S Y ] J a n PREPRINT - J
ANUARY
22, 2020every scenario. Mainly, reinforcement learning typically takes a lot of data, obtained via interacting with the physicalsystem, which is not always feasible. In addition, given it is a black-box model, thus the reasoning about its decisions isunknown. This compounded with the fact that it is only able to make good decisions on situations it is familiar withusually means that some additional control logic is needed when controlling critical systems.Another, more traditional control strategy is model-based predictive control (MPC). It involves a dynamic model of thesystem, used to predict the future behavior of the system, which is used in optimizing given objective. In addition, aclosed-loop or receding approach is taken, where the trajectory is optimized at each time step. Most of the prior work(such as [11, 12, 13, 14, 15]) use physical models that are robust and data efficient but could suffer from scalabilityissues and slow computation speeds. In addition, physical models usually require a lot of effort to setup.In order to combine scalability and performance of the model-free strategies with robustness and data efficiency ofMPC, this paper presents a MPC agent fitted with deep neural network model used to forecast day-ahead consumptionand photovoltaic production. The agent is used in optimizing supermarket energy costs via reducing peak loads andshifting consumption to low price periods. The performance is validated in a simulation environment using real-worldmeasurements and compared against a rule-based control strategy. In section 2, the optimization problem and theMPC algorithm is described together with the simulation setup. Section 3 is dedicated to presenting the results of bothoptimization goals with emphasis in peak reduction results. In addition, some control examples is visualized. In section3, the results are analyzed and future improvements to the approach is discussed.
In this study, a simulated battery component was used as a flexibility resource. This allowed to focus more onthe feasibility of the approach instead of intrigues of the refrigeration system dynamics. Subsequently, the controlalgorithms were tested in a simulation tool built for this purpose with data from real-world power measurements.For forecasting, we used deep neural network models trained with historical data. Optimization was done usinggradient-based trust-region method using scikit-learn package for python. We defined cost measurement resolution T as24 hours and the market resolution ∆ t as 15 minutes, yielding T = 96 market steps within one cost resolution. The battery can be controlled via three different actions, namely, idle b i , charge b c and discharge b d . In the followingequations, these are used an integers, so that b i = 0 , b c = 1 , b d = − . The battery output is assumed to be stateindependent and have equal and constant charging and discharging power P b . The optimization period T is dividedequally to intervals of time t . The optimization problem is then given by the following cost functions, first for peakreduction C peak = max ( t = T (cid:88) t =1 P st + b i,c,dt · P bt ) , (1)where P s denotes predicted total non-flexible net consumption at time step t . Similarly, for spot price the cost functionis C spot = t = T (cid:88) t =1 ( P st + b i,c,dt · P bt ) · p t , (2)where p t is the spot price at time t . In addition, we have to take into account physical constraints of the battery, i.e.charge has to stay between [0 , S max ] , where S max is the battery capacity. Expressed in terms of battery charge steps,we have the following linear constraints t = T (cid:88) t =1 b i,c,dt ≥ − s t =1 , t = T (cid:88) t =1 b i,c,dt + s t =1 ≤ S max , (3)where s t =1 is the starting battery level. In addition, as the predictions are never exactly accurate, it is beneficial to keepsome charge in the battery in order to respond to a sudden, unexpected changes in the consumption. We can add this tothe optimization problem using following additional set of constraints t (cid:48) = T (cid:88) t (cid:48) =1 t = t (cid:48) (cid:88) t =1 b i,c,dt + s t =1 ≥ (cid:15), (4)2 PREPRINT - J
ANUARY
22, 2020where (cid:15) is a constant expressing minimum charge amount.
The action sequence given by the optimization is used to construct a consumption plan. This is done by applying theselected action to the predicted consumption for each time step t so that P T ARt = P st + b optt · P bt , (5)where P T ARt is the targe power for timestep t . Battery control commands are applied every minute t s , and a closed-loopcontrol algorithm is used to follow the plan by monitoring ∆ P , which is given by ∆ P = ( t s = T s (cid:88) t s =1 P T OTt s ) − P T ART s , (6)where T s = 15 , the number of minutes in the market resolution. The complete algorithm is presented as pseudocode inalgorithm 1. for t s ∈ T s do Forecast consumption and production, i.e. P s Get action sequence b opt from optimizer Recalculate plan P T AR Calculate mean realized consumption P T OTt s Calculate difference ∆ P between the plan and current mean if | ∆ P | < φ then return b i else if ∆ P > then return b d else return b c end end endAlgorithm 1: Control algorithm in pseudocode. φ is a tunable tolerance parameter between [0 , . The data was collected from a new, medium-sized grocery store fitted with solar panels located in Oulu, Finland fromMay 2017 to May 2018. Data was measured in one minute intervals from multiple sub-metering points, visualized infigures 1 and 2. As we can see from the figures, the main consumption drivers are the refridgerator and heating systems.The electricity spot price was acquired from Nordpool. 3
PREPRINT - J
ANUARY
22, 2020Figure 1: Example total consumption of one week. We can see that the consumption and production has a daily cyclethat is fairly stable. In addition, total consumption far exceeds production, which is expected during this time of year.Figure 2: Example total consumption of a typical day. Consumption is significantly lower during night time and thereis large peak just before the store opens at 07:00 AM. Consumption remains high during the day and then decreasesduring evening. 4
PREPRINT - J
ANUARY
22, 2020
Simulation was run on multiple different battery configurations of varying power and capacity. The default batterypower was set as kW with kWh capacity. This is equivalent of approximately one hour of capacity with powerbeing 20 % of total consumption. The algorithm performed well when trying to decrease peak consumption (See figure 3 for example). With defaultbattery power and capacity, the algorithm managed to reduce peak consumption . per day. For comparison, withknowledge of future consumption, which can be viewed close to optimal results, the model-based algorithm was ableto reduce peak load by . per day, as seen in table 1. However, the algorithm did perform better than rule-basedalgorithm by about . In addition, results improved with increasing battery power and capacity, as expected. Forexample, doubling the default battery capacity to kWh resulted in . peak reduction for MPC agent and . with perfect forecasting. Applying a loss of (to efficiency) to the battery efficiency decreased performancearound . Control examples of different static consumption profiles is presented in figure 4.Figure 3: Peak reduction example. Blue curve shows controlled consumption whereas orange is the baseline. In thiscase, the peak was forecasted correctly and algorithm achieved peak reduction of 18%. Control algorithm Optimization objectivePeak reduction Spot priceOptimal .
2% 0 . MPC .
4% 0 . Rule-based .
3% 0 . Table 1: Algorithm performance using kWh / kW battery with efficiency.Optimizing with regards to the spot price turned out to be more chal-lenging than expected. With default battery parameters, neither of thealgorithms were able to produce any significant savings. Moreover,even with knowing the future consumption, the model-based algo-rithm managed to decrease costs only , per day. Performanceincreased somewhat with increasing battery power and capacity, butnot significantly. For example, a cost reduction of only wasachieved using kW / kWh battery. Poor performance can beattributed partly to the relative small differences in prices, thoughit seemed that both large price movements and flexibility storage isneeded in order to achieve significant savings. 5 PREPRINT - J
ANUARY
22, 2020 (a) Example of a day where there was plenty of capacity left throughout the day and thecontrol algorithm was able to reduce the peak load by .(b) Example of a day where consumption during the day remained high and thus the batterycould not be charged. Therefore, control algorithm was not able reduce peak load more than .(c) Example of a day where multiple high peak loads during evening time discharged thebattery and the control algorithm was unable to reduce all peak loads. This resulted in reduction of peak load. Figure 4: Examples of MPC performance with battery level displayed.6
PREPRINT - J
ANUARY
22, 2020
The algorithm was successful in decreasing peak loads, yielding close to reduction. Interestingly, using perfectpredictions, peak load only decreased additional couple of percent. This indicates that MPC is not very sensitiveto forecasting accuracy. Furthermore, it suggests that this approach may well be applicable in complex systems,where forecasting future consumption is difficult. It was observed that the main limiting factor in the peak reductionperformance was the battery capacity. Often the battery would run out of energy in the middle of the day, whenconsumption was generally high and thus could not reduce the peak optimally. If the battery capacity was larger, thuslasting longer, it could have been more optimally utilized.With regards to spot price, the approach could not significantly decrease costs in any practical battery configuration.However, increasing battery power and capacity did improve results but not significantly. Poor performance is alsoattributed to the relatively low variance in energy spot prices.This study was limited by the lack of actual control data and the subsequent lack of testing in the actual supermarket.The dynamics of the refrigeration system of a grocery store are more complicated than that of a battery, thus tofurther evaluate the success of this approach, a real-life control experiments would be preferable. In addition, thealgorithm could improved to include reinforcement learning to control various tunable parameters or more sophisticatedforecasting setup could be used.
References [1] Pervez Hameed Shaikh, Nursyarizal Bin Mohd Nor, Perumal Nallagownden, Irraivan Elamvazuthi, and TaibIbrahim. A review on optimized control systems for building energy and comfort management of smart sustainablebuildings, 2014.[2] A. Chaouachi, R. M. Kamel, R. Andoulsi, and K. Nagasaka. Multiobjective Intelligent Energy Management for aMicrogrid.
IEEE Transactions on Industrial Electronics , 60(4):1688–1699, apr 2013.[3] Hélène Thieblemont, Fariborz Haghighat, Ryozo Ooka, and Alain Moreau. Predictive control strategies based onweather forecast in buildings with energy storage system: A review of the state-of-the art, 2017.[4] John S. Vardakas, Nizar Zorba, and Christos V. Verikoukis. A Survey on Demand Response Programs inSmart Grids: Pricing Methods and Optimization Algorithms.
IEEE Communications Surveys and Tutorials ,17(1):152–178, jan 2015.[5] José R. Vázquez-Canteli and Zoltán Nagy. Reinforcement learning for demand response: A review of algorithmsand modeling techniques, 2019.[6] Yanzhi Wang, Xue Lin, and Massoud Pedram. A near-optimal model-based control algorithm for householdsequipped with residential photovoltaic power generation and energy storage systems.
IEEE Transactions onSustainable Energy , 7(1):77–86, jan 2016.[7] Elena Mocanu, Decebal Constantin Mocanu, Phuong H. Nguyen, Antonio Liotta, Michael E. Webber, MadeleineGibescu, and J. G. Slootweg. On-Line Building Energy Optimization Using Deep Reinforcement Learning.
IEEETransactions on Smart Grid , 10(4):3698–3708, jul 2019.[8] Daniel O’Neill, Marco Levorato, Andrea Goldsmith, and Urbashi Mitra. Residential Demand Response UsingReinforcement Learning. In , pages409–414. IEEE, oct 2010.[9] Karl Mason and Santiago Grijalva. A review of reinforcement learning for autonomous building energy manage-ment.
Computers & Electrical Engineering , 2019.[10] R.S. Sutton and A.G. Barto. Reinforcement Learning: An Introduction.
IEEE Transactions on Neural Networks ,1998.[11] Yudong Ma, Anthony Kelman, Allan Daly, and Francesco Borrelli. Predictive control for energy efficient buildingswith thermal storage: Modeling, stimulation, and experiments.
IEEE Control Systems , 32(1):44–64, 2012.[12] Samuel Prívara, Jiˇrí Cigler, Zdenˇek Váˇna, Frauke Oldewurtel, Carina Sagerschnig, and Eva Žáˇceková. Buildingmodeling as a crucial part for building predictive control.
Energy and Buildings , 56:8–22, jan 2013.[13] Roberto Z. Freire, Gustavo H.C. Oliveira, and Nathan Mendes. Predictive controllers for thermal comfortoptimization and energy savings.
Energy and Buildings , 40(7):1353–1365, 2008.[14] J. A. Candanedo, V. R. Dehkordi, and M. Stylianou. Model-based predictive control of an ice storage device in abuilding cooling system.
Applied Energy , 2013. 7
PREPRINT - J
ANUARY
22, 2020[15] Tobias Gybel Hovgaard, Stephen Boyd, Lars F.S. Larsen, and John Bagterp Jørgensen. Nonconvex modelpredictive control for commercial refrigeration.