Francois Belletti

Optimal Charging of Electric Vehicles for Load Shaping: A Dual-Splitting Framework With Explicit Convergence Bounds

This paper proposes a tailored distributed optimal charging algorithm for plug-in electric vehicles (PEVs). If controlled properly, large PEV populations can enable high penetration of renewables by balancing loads with intermittent generation. The algorithmic challenges include scalability, computation, uncertainty, and constraints on driver mobility and power-system congestion. This paper addresses computation and communication challenges via a scalable distributed optimal charging algorithm. Specifically, we exploit the mathematical structure of the aggregated charging problem to distribute the optimization program, using duality theory. Explicit bounds of convergence are derived to guide computational requirements. Two variations in the dual-splitting algorithm are also presented, which enable privacy-preserving properties. Constraints on both individual mobility requirements and power-system capacity are also incorporated. We demonstrate the proposed dual-splitting framework on a load-shaping case study for the so-called California “Duck Curve” with mobility data generated from the vehicle-to-grid simulator.

Distributed optimal charging of electric vehicles for demand response and load shaping

This paper proposes three novel distributed algorithms to optimally schedule Plug-in Electric Vehicle (PEV) charging. We first define the global optimization problem, where we seek to control large heterogeneous fleets of PEVs to flatten a net Load Curve. We demonstrate that the aggregated objective can be distributed, via a new consensus variable. This leads to a dual maximization problem that can be solved in an iterative and decentralized manner: at each iteration, PEVs solve their optimal problem, communicate their response to the aggregator, which then updates a price signal. We propose three distributed algorithms to compute the optimal solution, namely a gradient ascent and two incremental stochastic gradient methods. We prove their rate of convergence, their precision level and expose their characteristics in terms of communication and privacy. Finally, we use the Vehicle-To-Grid simulator (V2Gsim), and present a set of case studies, with an application to flattening the “Duck Curve” in California.

Expert Level Control of Ramp Metering Based on Multi-Task Deep Reinforcement Learning

This paper shows how the recent breakthroughs in reinforcement learning (RL) that have enabled robots to learn to play arcade video games, walk, or assemble colored bricks, can be used to perform other tasks that are currently at the core of engineering cyberphysical systems. We present the first use of RL for the control of systems modeled by discretized non-linear partial differential equations (PDEs) and devise a novel algorithm to use non-parametric control techniques for large multi-agent systems. Cyberphysical systems (e.g., hydraulic channels, transportation systems, the energy grid, and electromagnetic systems) are commonly modeled by PDEs, which historically have been a reliable way to enable engineering applications in these domains. However, it is known that the control of these PDE models is notoriously difficult. We show how neural network-based RL enables the control of discretized PDEs whose parameters are unknown, random, and time-varying. We introduce an algorithm of mutual weight regularization (MWR), which alleviates the curse of dimensionality of multi-agent control schemes by sharing experience between agents while giving each agent the opportunity to specialize its action policy so as to tailor it to the local parameters of the part of the system it is located in. A discretized PDE, such as the scalar Lighthill–Whitham–Richards PDE can indeed be considered as a macroscopic freeway traffic simulator and which presents the most salient challenges for learning to control large cyberphysical system with multiple agents. We consider two different discretization procedures and show the opportunities offered by applying deep reinforcement for continuous control on both. Using a neural RL PDE controller on a traffic flow simulation based on a Godunov discretization of the San Francisco Bay Bridge, we are able to achieve precise adaptive metering without model calibration thereby beating the state of the art in traffic metering. Furthermore, with the more accurate BeATS simulator, we manage to achieve a control performance on par with ALINEA, a state-of-the-art parametric control scheme, and show how using MWR improves the learning procedure.

Privacy-preserving dual splitting distributed optimization with application to load flattening in California

This article presents a dual splitting technique for a class of strongly convex optimization problems whose constraints are block-wise independent. The average-based input in the objective is the only binding element. A dual splitting strategy enables the design of distributed and privacy preserving algorithms. Theoretical convergence bounds and numerical experiments show this method successfully applies to the problem of charging electric devices so as to even out the daily energy demand in California. The solution we provide is a privacy enforced algorithm readily implementable in a network of smart electric vehicle chargers. It can reach any arbitrary precision for the common optimization goal while relying on randomly perturbed information at the agent level. We show that, provided the community is large enough, an averaging effect enables the group to learn its global optimum faster than individual information is leaked. A limited number of messages are sent out in the distributed implementation which prevents adversary statisticians from having low theoretical Mean Square Errors for their estimates.