Bert Claessens

A Scalable Three-Step Approach for Demand Side Management of Plug-in Hybrid Vehicles

In this paper, we present a scalable approach for DSM (demand side management) of PHEVs (plug-in hybrid electric vehicles). Essentially, our approach consists of three steps: aggregation, optimization, and control. In the aggregation step, individual PHEV charging constraints are aggregated upwards in a tree structure. In the optimization step, the aggregated constraints are used for scalable computation of a collective charging plan, which minimizes costs for electricity supply. In the real-time control step, this charging plan is used to create an incentive signal for all PHEVs, determined by a market-based priority scheme. These three steps are executed iteratively to cope with uncertainty and dynamism. In simulation experiments, the proposed three-step approach is benchmarked against classic, fully centralized approaches. Results show that our approach is able to charge PHEVs with comparable quality to optimal, centrally computed charging plans, while significantly improving scalability.

Self-learning demand side management for a heterogeneous cluster of devices with binary control actions

Finding an optimal planning for a large cluster of devices with binary control actions is a challenging task for both centralized and distributed approaches. This is certainly the case when a significant fraction of devices in the cluster has binary control actions, since the resulting optimization problem belongs to NP-hard integer programming. A distributed approach can be a good solution to address this problem. Good performance however, often relies on the presence of local intelligence, such as planning and prediction at device or household level. In this work we apply a self-learning agent-based demand side management approach to a heterogeneous cluster of devices with binary control actions. The required local intelligence is limited to a state estimation and local comfort and constraint checking. Each device is represented by an individual agent communicating a bid function to a virtual energy market. In the approach the aggregated energy and power constraints of a cluster of devices are learned, independent of the type and number of devices. The aggregated constraints are estimated based upon the aggregated bid functions. These constraints are used to determine an optimal control signal managing the cluster. The approach has been evaluated in two distinct scenarios including devices with binary control actions, showing that the self-learning approach converges within 12 days to obtain 80 % of the maximum optimization potential, with a generic approach that requires limited intelligence at device level.

Demand side management of electric vehicles with uncertainty on arrival and departure times

Uncertainty on arrival and departure times makes the scheduling of plug-in hybrid electric vehicles an intrinsically stochastic optimization problem. To take the stochastic nature of this problem into consideration, a scalable stochastic optimization strategy has been formulated. Generally, stochastic programming methods are computationally demanding and become impractical for large-scale problems. This work reduced the dimensionality of the scheduling problem with techniques from approximate dynamic programming. To illustrate the advantage of the stochastic algorithm a deterministic method has been formulated. Compared to the deterministic method, the proposed stochastic method can help an aggregator to reduce its expensive peak charging or avoid penalties for not fully charging the batteries of its clients.

A Flexible Stochastic Optimization Method for Wind Power Balancing With PHEVs

This paper proposes a flexible optimization method, based on state of the art algorithms, for the smart control of plug-in hybrid electric vehicles (PHEVs) to balance wind power production. The problem is approached from the perspective of a balance responsible party (BRP) with a large share of wind power in its portfolio. The BRP uses controllable PHEVs to minimize the imbalance of its portfolio resulting from wind power forecast errors. A Markov Decision Process (MDP) formulation in combination with dynamic programming is used to solve the multistage stochastic problem. The main difficulty for applying MDPs to this problem is to efficiently include time interdependence of the wind power forecast error. In the presented approach, the probability distribution and time interdependence of the forecast error are represented by a scenario tree. Because of the MDP formulation, the algorithm is adaptable to deal with different transition models and constraints. This feature enables to use the algorithm in a dynamic environment such as the future smart grid. To demonstrate this, a generic charging model for PHEVs is used in the BRP wind balancing case. The flexibility of the algorithm is shown by investigating the solution for different degrees of complexity in the charging model.

Cluster Control of Heterogeneous Thermostatically Controlled Loads Using Tracer Devices

Managing the aggregated demand of large heterogeneous clusters of thermostatically controlled loads (TCLs) is considered a sequential decision-making problem under uncertainty. Recent research indicates that using reduced-order models in combination with a broadcasted control signal offers a viable solution to the tradeoff between computational feasibility, and accurately describing the steady-state and transient cluster response. In this paper, we propose a novel control strategy based on tracer devices, which we define as a limited amount of virtual TCLs that represent the entire cluster of heterogeneous TCLs. These second-order model devices are identified in a nonintrusive manner, and capture both steady-state and transient population dynamics, as well as cluster heterogeneity. Additionally, the dispatch mechanism is included in the optimization, further improving the tracking performance. The parameterizable number of tracer devices enables a covering of the tradeoff domain. Both approaches have been evaluated in two scenarios. In the first small-scale scenario, improvements in price and power deviations are evaluated when using increasing numbers of tracer devices and integrating the dispatch dynamics. Results from the second large-scale scenario show that root mean square dispatch errors can be reduced by more than 10% when integrating the dispatch mechanism in the resulting high-fidelity model.

Reinforcement Learning Applied to an Electric Water Heater: From Theory to Practice

Electric water heaters have the ability to store energy in their water buffer without impacting the comfort of the end user. This feature makes them a prime candidate for residential demand response. However, the stochastic and nonlinear dynamics of electric water heaters, makes it challenging to harness their flexibility. Driven by this challenge, this paper formulates the underlying sequential decision-making problem as a Markov decision process and uses techniques from reinforcement learning. Specifically, we apply an auto-encoder network to find a compact feature representation of the sensor measurements, which helps to mitigate the curse of dimensionality. A well-known batch reinforcement learning technique, fitted <inline-formula> <tex-math notation=LaTeX>

Sequential decision-making strategy for a demand response aggregator in a two-settlement electricity market

{Q}

H2020 STORM project: Self-organising thermal operational resource management presentation of the project and progress update

</tex-math></inline-formula>-iteration, is used to find a control policy, given this feature representation. In a simulation-based experiment using an electric water heater with 50 temperature sensors, the proposed method was able to achieve good policies much faster than when using the full state information. In a laboratory experiment, we apply fitted <inline-formula> <tex-math notation=LaTeX>

Model-free control of thermostatically controlled loads connected to a district heating network

{Q}

Method for distributing an energy flow over a predetermined period of time to a cluster of a plurality of devices taking into account constraints relating to the energy to be delivered to the devices, a computer program for performing such a method and a system therefor

</tex-math></inline-formula>-iteration to an electric water heater with eight temperature sensors. Further reducing the state vector did not improve the results of fitted <inline-formula> <tex-math notation=LaTeX>