Online Charging Scheduling Algorithms of Electric Vehicles in Smart Grid: An Overview
aa r X i v : . [ c s . OH ] A ug Online Charging Scheduling Algorithms ofElectric Vehicles in Smart Grid: An Overview
Wanrong Tang, Suzhi Bi, and Ying Jun (Angela) Zhang A BSTRACT
As an environment-friendly substitute for conventional fuel-powered vehicles, electric vehicles(EVs) and their components have been widely developed and deployed worldwide. The large-scale integration of EVs into power grid brings both challenges and opportunities to the systemperformance. On one hand, the load demand from EV charging imposes large impact on thestability and efficiency of power grid. On the other hand, EVs could potentially act as mobileenergy storage systems to improve the power network performance, such as load flattening,fast frequency control, and facilitating renewable energy integration. Evidently, uncontrolledEV charging could lead to inefficient power network operation or even security issues. Thisspurs enormous research interests in designing charging coordination mechanisms. A key designchallenge here lies in the lack of complete knowledge of events that occur in the future. Indeed,the amount of knowledge of future events significantly impacts the design of efficient chargingcontrol algorithms. This article focuses on introducing online EV charging scheduling techniquesthat deal with different degrees of uncertainty and randomness of future knowledge. Besides,we highlight the promising future research directions for EV charging control.I. I
NTRODUCTION
Electric vehicles (EVs) are referred as the vehicles that are powered fully or partially byelectricity energy. In general, the rechargeable battery of an EV can be charged from an external
This work is supported in part by General Research Funding (Project number 14200315) from the Research Grants Councilof Hong Kong and Theme-Based Research Scheme (Project number T23-407/13-N). The work of S. Bi is supported in part bythe National Natural Science Foundation of China (Project number 61501303) and the Foundation of Shenzhen City (Projectnumber JCYJ20160307153818306).W. Tang is with the Department of Information Engineering, The Chinese University of Hong Kong.S. Bi is with the College of Information Engineering, Shenzhen University. He is the corresponding author of this article.Y. J. Zhang is with the Department of Information Engineering, The Chinese University of Hong Kong. She is also withShenzhen Research Institute, The Chinese University of Hong Kong. (cid:42) (cid:171)(cid:3)(cid:17)(cid:17)(cid:17) (cid:53)(cid:72)(cid:81)(cid:72)(cid:90)(cid:68)(cid:69)(cid:79)(cid:72)(cid:3)(cid:72)(cid:81)(cid:72)(cid:85)(cid:74)(cid:92)(cid:3)(cid:74)(cid:72)(cid:81)(cid:72)(cid:85)(cid:68)(cid:87)(cid:76)(cid:82)(cid:81) (cid:53)(cid:72)(cid:74)(cid:88)(cid:79)(cid:68)(cid:87)(cid:76)(cid:82)(cid:81)(cid:3)(cid:72)(cid:89)(cid:72)(cid:81)(cid:87) (cid:37)(cid:76)(cid:74)(cid:3)(cid:70)(cid:82)(cid:81)(cid:86)(cid:88)(cid:80)(cid:72)(cid:85)(cid:86)(cid:29)(cid:73)(cid:68)(cid:70)(cid:87)(cid:82)(cid:85)(cid:76)(cid:72)(cid:86)(cid:15)(cid:3)(cid:86)(cid:70)(cid:75)(cid:82)(cid:82)(cid:79)(cid:86)(cid:15)(cid:3)(cid:70)(cid:82)(cid:80)(cid:80)(cid:72)(cid:85)(cid:70)(cid:76)(cid:68)(cid:79)(cid:3)(cid:69)(cid:88)(cid:76)(cid:79)(cid:71)(cid:76)(cid:81)(cid:74)(cid:86) (cid:53)(cid:72)(cid:86)(cid:76)(cid:71)(cid:72)(cid:81)(cid:87)(cid:86) (cid:47) (cid:36)(cid:74)(cid:74)(cid:85)(cid:72)(cid:74)(cid:68)(cid:87)(cid:82)(cid:85)(cid:3)(cid:82)(cid:73)(cid:3)(cid:40)(cid:57)(cid:3)(cid:70)(cid:75)(cid:68)(cid:85)(cid:74)(cid:76)(cid:81)(cid:74)(cid:29)(cid:3)(cid:83)(cid:68)(cid:85)(cid:78)(cid:76)(cid:81)(cid:74)(cid:3)(cid:86)(cid:79)(cid:82)(cid:87)(cid:15)(cid:3)(cid:70)(cid:75)(cid:68)(cid:85)(cid:74)(cid:76)(cid:81)(cid:74)(cid:3)(cid:86)(cid:87)(cid:68)(cid:87)(cid:76)(cid:82)(cid:81) (cid:38)(cid:75)(cid:68)(cid:85)(cid:74)(cid:76)(cid:81)(cid:74)(cid:3)(cid:86)(cid:87)(cid:68)(cid:87)(cid:76)(cid:82)(cid:81)(cid:38)(cid:75)(cid:68)(cid:85)(cid:74)(cid:76)(cid:81)(cid:74)(cid:3)(cid:83)(cid:82)(cid:79)(cid:72)(cid:38)(cid:75)(cid:68)(cid:85)(cid:74)(cid:76)(cid:81)(cid:74)(cid:3)(cid:90)(cid:68)(cid:79)(cid:79) (cid:40)(cid:81)(cid:72)(cid:85)(cid:74)(cid:92)(cid:3)(cid:86)(cid:87)(cid:82)(cid:85)(cid:68)(cid:74)(cid:72)(cid:53)(cid:72)(cid:81)(cid:72)(cid:90)(cid:68)(cid:69)(cid:79)(cid:72)(cid:86) (cid:51)(cid:82)(cid:90)(cid:72)(cid:85)(cid:3)(cid:83)(cid:79)(cid:68)(cid:81)(cid:87)(cid:43)(cid:82)(cid:88)(cid:86)(cid:72)(cid:3)(cid:75)(cid:82)(cid:79)(cid:71) (cid:38)(cid:82)(cid:80)(cid:80)(cid:72)(cid:85)(cid:70)(cid:76)(cid:68)(cid:79)(cid:44)(cid:81)(cid:71)(cid:88)(cid:86)(cid:87)(cid:85)(cid:76)(cid:68)(cid:79) (cid:54)(cid:70)(cid:75)(cid:82)(cid:82)(cid:79)(cid:79)(cid:71) (cid:53)(cid:72)(cid:81)(cid:72)(cid:90)(cid:68)(cid:69)(cid:79)(cid:72)(cid:86) (cid:51)(cid:82)(cid:90)(cid:72)(cid:85)(cid:3)(cid:83)(cid:79)(cid:68)(cid:81)(cid:87)(cid:51)(cid:82)(cid:90)(cid:72)(cid:85)(cid:3)(cid:73)(cid:79)(cid:82)(cid:90) (cid:51)(cid:72)(cid:68)(cid:78)(cid:3)(cid:75)(cid:82)(cid:88)(cid:85)(cid:86)(cid:50)(cid:73)(cid:73)(cid:16)(cid:83)(cid:72)(cid:68)(cid:78)(cid:3)(cid:75)(cid:82)(cid:88)(cid:85)(cid:86) (cid:39)(cid:68)(cid:87)(cid:68)(cid:3)(cid:70)(cid:72)(cid:81)(cid:87)(cid:72)(cid:85)(cid:43)(cid:82)(cid:86)(cid:83)(cid:76)(cid:87)(cid:68)(cid:79) (cid:38)(cid:75)(cid:68)(cid:85)(cid:74)(cid:76)(cid:81)(cid:74)(cid:3)(cid:86)(cid:87)(cid:68)(cid:87)(cid:76)(cid:82)(cid:81)(cid:38)(cid:75)(cid:68)(cid:85)(cid:74)(cid:76)(cid:81)(cid:74)(cid:3)(cid:83)(cid:82)(cid:79)(cid:72)(cid:38)(cid:75)(cid:68)(cid:85)(cid:74)(cid:76)(cid:81)(cid:74)(cid:3)(cid:90)(cid:68)(cid:79)(cid:79) (cid:40)(cid:81)(cid:72)(cid:85)(cid:74)(cid:92)(cid:3)(cid:86)(cid:87)(cid:82)(cid:85)(cid:68)(cid:74)(cid:72)(cid:38)(cid:82)(cid:80)(cid:80)(cid:72)(cid:85)(cid:70)(cid:76)(cid:68)(cid:79)(cid:44)(cid:81)(cid:71)(cid:88)(cid:86)(cid:87)(cid:85)(cid:76)(cid:68)(cid:79) (cid:39)(cid:68)(cid:87)(cid:68)(cid:3)(cid:70)(cid:72)(cid:81)(cid:87)(cid:72)(cid:85)(cid:43)(cid:82)(cid:86)(cid:83)(cid:76)(cid:87)(cid:68)(cid:79)(cid:43)(cid:82)(cid:88)(cid:86)(cid:72)(cid:3)(cid:75)(cid:82)(cid:79)(cid:71) (cid:54)(cid:70)(cid:75)(cid:82)(cid:82)(cid:79)(cid:79)(cid:71)(cid:40)(cid:57)(cid:86) (cid:40)(cid:57)(cid:86)(cid:40)(cid:57)(cid:86) (cid:40)(cid:57)(cid:86) (cid:40)(cid:57)(cid:86)(cid:40)(cid:57)(cid:86)
Fig. 1. An illustration of the applications of EVs at the time of peak hours and off-peak hours of base load consumptions. source of electricity through wall sockets, and also discharged to an external energy storageor power grid. Compared with conventional fuel-powered vehicles, EVs produce very little airpollution upon their use. In addition, the environmental benefits of EVs are magnified whenthey are powered by new and clean renewable energy sources, such as solar and wind power.As such, a wide range of countries have pledged billions of dollars to fund the developmentof EVs and their components in an attempt to replace the conventional vehicles. According tothe recent analysis from the Centre for Solar Energy and Hydrogen Research, the demand ofEV accounts for a total global market of more than 740,000 EVs in early 2015 . In the next
50 years, the number of vehicles in operation is expected to increase from 700 million to 2.5billion, where EVs will constitute a major part of them.The fast increasing adoption of EVs brings both challenges and opportunities to the powergrid. On one hand, the massive load caused by the integration of EVs into the power gridraises concerns about the potential impacts to the operating cost, voltage stability and thefrequency excursion at both generation and transmission sides. On the other hand, EVs canbe used as a new type of mobile energy storage systems that can serve many purposes. Withadequate energy stored in the batteries of EVs, the bidirectional charging and discharging controlhas extensive applications in the microgrids/distribution networks, such as load flattening, peakshaving, frequency fluctuation mitigation and improving the integration of renewable sources.For instance, Fig. 1 illustrates the use of EVs for load flattening in a power gird. During theoff-peak hours, EVs can act as loads to withdraw and store electricity from the main grid. Duringthe peak hours, however, the EVs can release the stored energy back to the grid to meet thehigh demand of other electricity consumers. Overall, the use of EVs flattens the power profileover time and improves the stability of the entire power system.In both cases, uncontrolled EV charging/discharging will lead to inefficient system operationor even severe security problems. To mitigate the negative effects and enjoy the benefit ofEV integration, it is critical to develop effective charging/discharging scheduling algorithms forefficient grid operation. In practice, a key design challenge of charging scheduling algorithmslies in the randomness and uncertainty of future events, including the charging profiles of EVsarriving in the future, future load demand in the grid, future renewable energy generation, etc.Therefore, it is necessary to develop online charging/discharging algorithms to cope with differentdegrees of uncertainty when making real-time decisions. Besides, the large-scale charging ofEVs requires low-complexity control mechanisms to reduce the operating delay and the capitalcost of equipment investment. In this article, we introduce various online EV charging controlmechanisms to enhance the efficiency and stability of power networks. We discuss differentonline algorithms under different types of knowledge of future data, including the estimation ofnear-future random data, the mean, variance, and distribution, etc. Specially, we explore someunique features of the charging behaviors of EVs to improve the general online algorithms forbetter performance and lower complexity. We also notice that there are existing surveys on theenergy management strategies of EVs proposed up to 2012 [1]. In comparison, we not only
SetCost k c (cid:229) k k c Pastinformation CurrentinformationPredictionsof future data Statistics offuture dataChargingscheduling += kk k t Decision at time
Fig. 2. The illustration of the online EV charging scheduling process. update the state-of-the-art EV energy management technologies, but also focus on introducingthe design of online charging scheduling algorithms.This article is organized as follows. We first provide the basic model of online stochastic EVcharging control. Then, we introduce the most up-to-date methods to tackle the EV chargingscheduling problems under different degrees of knowledge of future information. At last, wediscuss the future research directions for online stochastic EV charging control in some interestingapplications and conclude the article.II. B
ASIC M ODEL OF R EALTIME S TOCHASTIC
EV C
HARGING C ONTROL
The online EV charging problem assumes that, at any time, the scheduler only knows thecausal information, i.e., the information revealed so far. For instance, a charging facility, e.g., acharging station, only knows the charging profiles of the EVs that have arrived as well as theload demand and renewable energy generation in the grid up to the current time. Based on thecausal information, the scheduler makes a charging decision, i.e., the current charging rates of allarrived EVs. Notice that a past decision that has already been implemented cannot be reversedin the future. In the following, we specify the elements of online EV charging control.
Event driven:
In practice, an EV can arrive or depart at any time instant. As such, the chargingschedule is a function of continuous time, which involves infinite number of control variables in the EV charging problem. In fact, it has been shown in [2] that the charging schedules only needto be updated at the time when an “event” occurs, such that the current system state changes.For instance, an event can be the arrival or departure of an EV, or the change of base load orelectricity price. Specifically, we denote by t , t , · · · , t k the time when events , , · · · , k occur,respectively. In general, the time length between time t k and time t k +1 is a variable, which isdecided by the random events. System time:
The system time horizon can be either finite or infinite. In practice, an EVcharging schedule is optimized over a finite time horizon of from several hours to several days,while the length of a time slot is often in the order of minutes. The system time horizon can beregarded as infinite when it is much longer than the length of a time slot, e.g., several years.
Causal information:
In the realtime scenario, only the past and current information is knownby the charging scheduler. For instance, at any time slot, a charging station only knows thecharging demands and departure deadlines of the EVs that arrive at or before current time, thepast and current base load and renewable energy, etc.
Random data:
Due to the assumption of causality of knowledge, the non-causal informationabout future events appears uncertain and random. The randomness mainly comes from thefollowing aspects: 1) charging profiles of EVs that arrive in the future, including arrival, de-parture, charging demand, and individual charging constraints, 2) the future load demand in thegrid by, for example, residential buildings, factories, schools, hospitals, commercial buildings,data centers, etc. 3) future renewable energy generations from, for example, solar, wind, andhydro-electric plants, 4) future prices including electricity price and regulation service price.
Knowledge of future data:
Based on the historical data, the scheduler may have some predic-tions on the future data, including the near-future predictions or the statistics such as the mean,variance and distributions.
Objective:
The objective EV charging control varies depending on the standpoint we chooseto take. From EV owners’ viewpoint, the objectives could be charging demand satisfaction (i.e.,fulfilling the EVs’ charging demands before their specified deadlines), charging cost minimiza-tion, or profit maximization by selling power to the power grid. On the other hand, the objectiveof a utility owner could be energy cost minimization, load flattening/shaping, peak shaving,frequency regulation, and voltage regulation. In general, the objective of a charging schedulingproblem can be expressed as a cost function to be minimized.
Based on the above definitions, the process of a general online EV charging scheduling can bedescribed as Fig. 2. At time t k , the scheduler makes a decision based on the causal informationand the possible predictions/statistics of future random data, and then induces a cost, denotedby c k . The process repeats until the system time ends. We denoted by T the total number oftimes that the decisions are made. The charging decision and random data revealed at time t k are denoted by x k and ξ k , respectively. The charging decisions and random data revealed fromtime t to t k are denoted by x k and ξ k respectively. Specially, the cost at time t k is a functionof the charging decisions and random data revealed up to time t k , i.e., c k = f ( x k , ξ k ) . Noticethat the charging decisions depend on the the knowledge of the random data in the future. Inthe next section, we will introduce the methodologies of online EV charging scheduling basedon the knowledge of future random data and discuss their performance respectively.III. S TOCHASTIC C ONTROL T ECHNIQUES OF
EV C
HARGING
The knowledge of future random data is rather different in different applications. Fig. 3illustrates the spectrum of future knowledge. As shown in Fig. 3, the most ideal case is whenthe complete knowledge of the future data is known. That is, the charging scheduler knowsall the realizations of the future data before the beginning of system time. Then, the stochasticscheduling problem for EV charging becomes a deterministic problem, which is much easier totackle with deterministic algorithms. Another extreme case is when absolutely no informationabout future data is known by the online charging scheduler. Then, the scheduler makes decisionsbased only on the data that has already revealed. In between, the more general cases are thatthe scheduler has knowledge of some statistical information or short-term predictions of futuredata. For instance, the statistical information of the EV traffic patterns could often be acquiredthrough historic data, while the near-future data of renewable energy generation, e.g., the solarand wind power, can be predicted with high precision.
A. Methodologies with Complete Knowledge of Future Data
We first consider the case that the complete knowledge of data is known beforehand. In thiscase, the random data at all times of making decision ξ T become deterministic. Then, thestochastic EV charging problem is reduced to a deterministic problem, which is often referred toas offline problem . The optimal solution to the offline problem is called optimal offline solution , Achievableperformance Knowledge offuture dataNo knowledge Partialknowledge ofdata/distribution Completeknowledge ofdistribution Completeknowledge ofdataCausalinformation, i.e.,past and currentdata Mean, variance,other high-order moments,types ofdistributions,near-future data Probabilitydensity functionsof future data Realizations offuture data
Fig. 3. The illustration of the spectrum of future knowledge. and the algorithm adopted to solve the offline problem is called offline algorithm . Specifically,the optimal solution, denoted by x ∗ T , is calculated by x ∗ T = arg min x T T X t =1 f ( x t , ξ t ) . (1)Note that offline problem is deterministic and in general easier to handle than the online problem.The optimal offline solution is not achievable in practice due to the unrealistic assumption ofcomplete future information. Instead, it is often used as a benchmark to evaluate other onlinecharging scheduling methods. B. Methodologies with Zero Knowledge of Future Data
When no information about the future data is known, the charging scheduling algorithm makesdecisions based on only the causal information available to the scheduler. A key feature of theonline algorithm is that the performance is generally evaluated in the worst case scenario, as nostatistics of data could be leveraged to evaluate the average cost. A standard metric to evaluatethe worst-case performance of an online algorithm is competitive ratio , defined as the maximum ratio between the cost achieved by an online algorithm and that achieved by the optimal offlinealgorithm over all possible input sequences (e.g., the EV arrival patterns, charging demands,and base load variations). Let Φ be an online algorithm or policy, Π be the set of all feasiblepolicies, and x Φ1: t be the decision at time t , · · · , t i under algorithm/policy Φ . Then, the optimalcompetitive ratio of policy Φ is calculated by min Φ ∈ Π max ξ T P Ti =1 f ( x Φ1: i , ξ i ) P Ti =1 f ( x ∗ i , ξ i ) . (2)To minimize the competitive ratio, there are three main ideas to design competitive onlinealgorithms for EV charging problem. • Classic online scheduling algorithms:
There exist many classic online scheduling algorithmsthat were proposed to solve problems other than EV scheduling, such as computing jobscheduling and industrial process optimization. Some well-known methods include earliestdeadline first (EDF) algorithms, the least laxity first (LLF) algorithm and optimal available(OA) algorithm [3]. When applied to EV charging, the EDF always charges the EV withearliest departure time first, the LLF schedules the EV with least laxity (i.e., the parkingtime length minus the shortest time length of fulfilling charging), and the OA solves theproblem by assuming that no random data (or EVs, base load, etc) will be released in thefuture. In practice, however, the direct extension of these algorithms to EV charging mayyield poor performance due to the special features of EV charging problem, e.g., the burstyand time-varying nature of EV arrivals. These classic algorithms often need modificationsto fit in the structure of EV charging problems. Sometimes, the algorithms are combinedwith pricing and other control schemes, e.g, admission control [4]. • Solution-structure based algorithms:
These algorithms are designed by exploring the struc-tures of the optimal offline solution, given that it is easy to obtain. Indeed, exploring theoffline solution structure is often used as the first step of online algorithm design. Byobserving the optimal offline solution, we try to fathom its solution structure. For example,when the objective function in the offline problem is an increasing convex function of thetotal load from EV charging and other elastic load, an optimal solution to the offline problemalways tends to flatten the total load profile over time as much as possible [2] [5] [11]. Thisleads to the design of online algorithms that charge the EVs neither too fast nor too slowlyto reduce the fluctuation of the total load. • Data-mining/data-driven based algorithms:
The data-mining/data-driven based algorithmsare designed by mining the revealed data and analyzing the statistics. The statistics of theavailable data include the cross-correlation, auto-correlation and partial auto-correlation,etc. Typical data-mining/data-driven based algorithms include genetic algorithms, neuralnetworks and fuzzy rule-based systems. In general, the data-mining/data-driven algorithmsare more suitable for the case where the structure of system model can not be easilydetermined using empirical or analytical approaches [6].An efficient design of online EV charging scheduling is often a combination of the abovemethods. For instance, assuming that the cost function is quadratic with the load, we get theinsight that the optimal offline solution should exhibit a load-flattening structure. Meanwhile,we notice that the classic online algorithm OA only flattens the load demand revealed at currenttime but underestimates the load demand revealed in future. In practice, the pattern of randomEV arrivals often has some peaks. By taking into account the possible peak arrivals of EVs inthe future, an online algorithm named ORCHARD that speeds up the charging rate of OA by aproper factor is proposed in [2], which effectively reduces the possible peak load in the future.As a result, the competitive ratio of online algorithm ORCHARD is shown to be 2.39, which issignificantly better than that achieved by the original OA algorithm, i.e., 4.Notice that most existing online algorithms for EV charging scheduling problem are determin-istic, i.e., fixed decision output as a function of causal information input. A promising methodto improve the worst-case performance of existing deterministic online algorithms is to applyrandomized online algorithm. A randomized online algorithm is a random strategy over a setof deterministic online algorithms based on a probability distribution. For instance, the key ideaof the algorithm designed in [2] is to speed up the processing rate (charging rate) of OA bya factor, where the factor is a fixed constant. A possible randomized online algorithm is to setthe factor as the random variable which follows a certain probability distribution. In general,randomized online algorithms have better worst-case performance than the deterministic onlinealgorithms. However, the difficulty often lies in the setting of the probability distribution of arandom algorithm. C. Methodologies with Partial Knowledge of Future Data
In practice, some partial knowledge of future data, e.g., from the prediction of future data, isavailable in the design of online algorithms. For instance, power generation and load predictionalgorithms are now important components of most modern smart grid. Indeed, the wind speed canbe well-predicted by combining probability and fuzzy systems concepts [7]. For the EV chargingproblem, EV charging profiles can be predicted based on the past data collected and reservationsmade by the EV users in advance. In general, statistical-modeling based algorithms are oftenapplied for data prediction, e.g., artificial neural network (ANN), EV user classification, and otherMachine Learning (ML)-based methods [8]. By incorporating the near future estimation, onlinealgorithms could be designed to neglect some unrealistic worst cases and improve performancebased on the partially-known future.
D. Methodologies with Knowledge of Statistical Information
In this section, we discuss the case where the future data is not known, but its statisticalinformation can be estimated based on the historic data. The estimation of the future randomprocesses mainly includes the estimation of the moments (e.g., mean as the first-order momentand variance as the second-order moment) and the estimation of probability distributions (i.e.,moments of all orders). When the scheduler has the knowledge of probability distributions ofrandom data, i.e., probability density functions (PDF), algorithms based on dynamic program-ming can be applied. When the number of times of making decision is finite, the problem canbe solved by backward induction method or Monte Carlo sampling techniques [9]. When thenumber of times of making decision goes to infinity, the problem can be formulated as aninfinite-time horizon dynamic programming or a Markov Decision Process (MDP). Specifically,we denote by s k the system state at time t k , e.g., the current charging demand of individual EV,the base load, and electricity price, etc. The action is the charging decision at time t k , i.e., x k .Then, the online EV charging problem is that at time t k , the decision maker chooses an action x k that is available in current state s k . The process responds at the next time step by randomlymoving into a new state s k +1 following a known distribution, and then returns a correspondingcost-to-go, denoted by v k ( s k ) . Specifically, the optimal cost-to-go, denote by v ∗ k ( s k ) at time t k , satisfies the following Bellman’s equation [13] v ∗ k ( s k ) = min x k f ( x k , ξ k ) + α X s k +1 P ( s k , s k +1 ) v ∗ k +1 ( s k +1 ) , (3)where α is a discount factor and P ( s k , s k +1 ) is the transition probability from s k to s k +1 . Notethat the EV charging process is featured by the battery memory. When formulating the EVcharging problem as a Markov Decision Process (MDP), the system state could be defined asthe energy levels of the battery stored in the EV or the renewable power supplied in the system.The transition probability could be estimated by the historic data of the renewable power andEV charging demands. There are several standard algorithms to solve the MDP problem, e.g.,value iteration, policy iteration, modified policy iteration, and prioritized sweeping, etc. Whenthe statistic information of the random data is not clear, Q-learning algorithm could be adoptedto solve the MDP problem. Note that the EV charging problem often contains a continuous spaceof system state, e.g., the energy level of battery and the electricity price, and a continuous spaceof action, i.e., the charging rate. The existing research often uses discrete Bellman’s equation tomodel the EV charging problem [10] [13], which can lead to prohibitive computation complexity.On the other hand, as the fast integration of EVs into the power grid, the large scale of EVs couldalso bring the issue about the curse of dimensionality. To reduce the computational complexity,approximate (stochastic) dynamic programming (ADP) methods could be adopted [10].In most cases, it is hard to accurately estimate the complete probability density function ofthe random data based on the historic data. A more practical prediction of data statistics is thelow-order moment, e.g., the mean and the variance, as it requires much fewer data samples thanto accurately characterize the full probability distribution. Then, advanced techniques from robustoptimization could be adopted to tackle the online problems with partial statistic information.Since the first-order moment is the simplest to estimate compared with other statistics, a lot ofworks make use of the mean instead of high-order information. Specifically, Model PredictiveControl (MPC) method is one common approach to handle online problems with the knowledgeof the expected values of random data. To address a wide range of uncertainties and variability,MPC based charging scheduling algorithm replaces all future data, e.g., renewable energy, baseload, arrival rate and charging load demand of EVs, by their expected values and thus reducestochastic problem to a deterministic problem. A well-accepted metric to valuate MPC based charging scheduling algorithm is Value of the Stochastic Solution (VSS), which evaluates theoptimality gap between the optimal solution to (3) by requiring the distributions of ξ and thesolution from MPC based algorithm by replacing ξ with the means [11]. In practice, the statisticsof EV arrival process often exhibit periodicity. For example, the arrival rate of the residential EVcharging demand could have a periodicity, where the period is one day . The daily travel patternsare also likely to exhibit periodicity based on the National Household Travel Survey (NHTS)2009 . Accordingly, the periodicity of EV random arrival process can facilitate the prediction ofEVs’ arrivals to improve the performance. For instance, [11] shows that the MPC based algorithmcould be made more scalable if the random process describing the arrival of charging demandsis first-order periodic. Besides, another scenario is to assume that the random data comes froma population that follows a known probability distribution, where the typical parameters, i.e.,mean, variance, etc, are unknown. These parameters can be estimated by elementary statisticalmethods and made more accurate by sensitivity analysis. For instance, the recent studies on thereal-world data verify the hypothesize that the aggregate arrival rates of EVs follow a Poissondistribution [12].For the ease of reference, we summarize the methodologies to design online EV chargingscheduling algorithms in Table. I. For the case with complete knowledge of distribution, thealgorithms are likely to induce high computational complexity. In this case, exploiting specialsolution structure may lead to a greatly reduced computational cost. For example, a threshold-based charging algorithm is developed in [13]. For the case with partial knowledge of statistics, itis of high interest to improve the performance of sub-optimal scheduling solution. One possiblesolution is to combine online/stochatic learning techniques and robust optimization to improvethe performance of the algorithm.IV. P ERFORMANCE E VALUATION
In this section, we evaluate the performance of the methodologies discussed above. The systemtime is set to be hours, and the length between two adjacent times of making decision is set X. Zhang and S. Grijalva, “An Advanced Data Driven Model for Residential Electric Vehicle Charging Demand,” techniquereport, Georgia Institute of Technology, 2015. The National Household Travel Survey (NHTS) 2009 gathers information about daily travel patterns of different types ofhouseholds in 2009, and shows that the daily travel statistics (e.g., Average Vehicle Trip Length, Average Time Spent Driving,Person Trips, Person Miles of Travel) are very similar for each weekday or weekend. TABLE IS
UMMARY OF KNOWLEDGE OF FUTURE INFORMATION AND COMMON METHODOLOGIES
Knowledge categories Future information Methodologiesknown by Scheduler complete knowledge of data realizations linear programming, convex optimizations,graph algorithms, greedy algorithms,approximation algorithms, heuristic algorithms complete knowledge of distribution probability density dynamic programming, Markov decision process,functions stochastic dynamic programming, Monte Carlo sampling partial knowledge of distribution first-order moments model predictive controlhigh-order moments robust optimizationstypes of distributions parametric methods partial knowledge of data near-future data Markov models, time series,machine-learning based algorithms no knowledge of data zero classic online scheduling algorithms,solution-structure algorithms,data-mining/data-driven based algorithms to be 10 minutes. Suppose that the EV arrivals follow a Poisson distribution and the parking timeof each EV follows an exponential distribution [12]. Their charging demand follows an uniformdistribution. For the traffic patterns, we set two peak periods, i.e.,
12 : 00 to
14 : 00 and
18 : 00 to
20 : 00 , which match with the realistic vehicle trips in National Household Travel Survey (NHTS)2009. We investigate two scenarios where the EVs serve for different purposes. In scenario 1,EVs act only as the consumers that require to satisfy the charging demand. In scenario 2, EVsact as not only consumers but also power suppliers, where EVs could be charged/dischargedfrom/to the grid. For both scenarios, the objective function is to minimize the variance of totalload, which consists of the load from EV charging and the inelastic base load. The minimizationof load variance in effect reduces system power losses and improves voltage regulation [14].Specifically, we choose the following algorithms listed in a decreasing order of the amount offuture data knowledge.1) Optimal offline algorithm: the complete knowledge of the random data is assumed to beknown. Specifically, we adopt interior point method in CVX to compute the optimaloffline solution.2) Online algorithm with PDF: the complete knowledge of distributions of random data areassumed to be known. Specifically, we adopt sample average approximation (SAA) method M. Grant and S. Boyd, CVX: Matlab Software for Disciplined Convex Programming [Online]. Available: http://cvxr.com/cvxMar. 2013, Version 2.0 (beta). as the online algorithm with PDF.3) Online algorithm MPC [11]: the expected values of the random data are assumed to beknown.4) Online algorithm with no knowledge of future information: ORCHARD [2] and OA [3] :no future information is assumed to be known.For both scenarios, we plot the load variance of the five algorithms by increasing the arrival ratesduring the peak hours, as shown in Fig. 4 and Fig. 5. Both figures show that the optimal offlinealgorithm always produces the lowest load variance among the five algorithms. Meanwhile, theonline algorithm with PDF achieves lower cost than the MPC algorithm with prediction of means,and both algorithms follow closely to the optimal offline algorithm. We also notice that onlinealgorithm ORCHARD and OA produce higher load variance than the other three algorithms,since they assume no predictions nor non-causal information of the random data. Between them,ORCHARD significantly outperforms OA, where the OA algorithm performs poorly especiallyunder high peak arrival rate. For all five algorithms, it can be easily observed that the loadvariance of scenario 2 depicted in Fig. 5 is much smaller than that of scenario 1 depicted inFig. 4, which demonstrates the effectiveness of using EVs as mobile energy storage to flattenthe system load profiles. V. F UTURE R ESEARCH D IRECTIONS
The online algorithm design for EV charging scheduling contains rich research problems withdifferent applications of EVs. In this section, we highlight several interesting research topics wedeem particularly worth investigating.
A. Economic Incentive Design
The major challenge of the online charging algorithm design is the uncertainties from thebehavior of EV users. A promising solution is to introduce economic incentive schemes toencourage more users to arrive at the charging station during the off-peak hour of base loadconsumptions and less during the peak hours, so that the total load demand is flattened overtime. Equivalently, pricing method can be used to adjust the EVs’ charging demand over time. Forinstance, distribution locational marginal pricing method could be adopted to alleviate congestioninduced by EV loads [15]. Besides, the scheduler can also offer financial compensation to those
10 20 30 40 50 60 700100200300400500600 Arrival rate (EVs/hour)
Load v a r i an c e ( k W ) Optimal offlineOnline with PDFMPCORCHARDOA
Fig. 4. Load variance of five algorithms over arrival rate at peak hours in scenario 1, where EVs act as load demands.
10 20 30 40 50 60 70050100150200250300 Arrival rate (EVs/hour)
Load v a r i an c e ( k W ) Optimal offlineOnline with PDFMPCORCHARDOA
Fig. 5. Load variance of five algorithms over arrival rate at peak hours in scenario 2, where EVs act as both load demandsand power sources. users who are willing to make reservations day-ahead, park the EV for a longer time, or toleratecharging delay after the specified parking time. Through optimizing the pricing schemes, thescheduler maximizes its overall utility, e.g., its profit defined as the revenue minus the operatingcost and the cost on offering the incentives. The joint design of pricing scheme and online EVscheduling is also a promising yet challenging topic to investigate, considering the complexcorrelations between the pricing and the EV user profiles, including arrival rates, parking timeand charging demand. B. Online/stochastic Learning of Random Data
As shown in Fig. 4 and Fig. 5, the accurate knowledge of future data can lead to significantperformance improvement of online algorithms. Currently, most studies on online schedulingdesign assume perfect knowledge of (partial) future data or statistical information. In practice,however, the actual knowledge could be inaccurate, and the data collected could be noisy,incomplete or out-dated. It is therefore important to incorporate the acquisition of data knowledgein the design of online scheduling algorithm. A promising solution is to use online/stochasticlearning methods to exploit the random data to assist the decisions of EV scheduling in aniterative manner [7] [8]. In this case, however, the learning algorithm efficiency is of paramountimportance, as the EV data size could be enormous and the charging scheduling is a delay-sensitive application.
C. Integration of Renewable Sources
The integration of renewable sources brings both challenges and opportunities to the EVcharging scheduling problem. On one hand, EVs as energy storage can be used to reduce theintermittency of renewable sources, absorb the variability of load caused by renewable sourcesand even as energy carriers to transport energy from remote renewable sources to loads in urgentneed of power supply. On the other hand, renewable source could help reduce the fluctuationof base load and energy generation cost, especially for charging stations that own distributedrenewable generators. Then, the charging scheme should allocate energy from renewable sourcesto EVs in both cost-efficient and system-stability manners. Besides, the integration of renewableenergy introduces another layer of randomness in the system design, such that online algorithms now need to tackle the uncertainties from both the EVs and the renewable sources. Predictionand data mining play even more important role in improving the overall system performance.VI. C ONCLUSIONS
In this article, we have provided an overview of efficient online charging scheduling algorithmsto improve the power grid performance under different assumptions of future data knowledge.Besides, we have also highlighted some promising future research directions. We believe thatthe adoption of advanced online EV charging scheduling algorithms in next-generation powergrids will greatly improve their efficiency, reliability, security, and sustainability.R
EFERENCES [1] A. Panday and H. O. Bansal, “A review of optimal energy management strategies for hybrid electric vehicle,”
International Journal of Vehicular Technology (IJVT) , vol. 2014, pp. 1-19, Nov. 2014.[2] W. Tang, S. Bi, and Y. J. Zhang, “Online coordinated charging decision algorithm for electric vehicles withoutfuture information,”
IEEE Trans. Smart Grid , vol. 5, no. 6, pp. 2810-2824, Nov. 2014.[3] Y. He, B. Venkatesh, and L. Guan, “Optimal scheduling for charging and discharging of electric vehicles,”
IEEE Trans. Smart Grid , vol.3, no.3, pp. 1095-1105, Sept. 2012.[4] S. Chen and L. Tong, “iEMS for large scale charging of electric vehicles architecture and optimal onlinescheduling,” in Proc.
IEEE Int. Conf. Smart Grid Commun. (SmartGridComm) , pp. 629-634, Nov. 2012.[5] L. Gan, U. Topcu, and S. H. Low, “Optimal decentralized protocol for electric vehicle charging,”
IEEE Trans.Power Syst. , vol. 28, no. 2, pp. 940-951, May 2013.[6] E. Xydas, C. Marmaras, L. M. Cipcigan, N. Jenkins, S. Carroll, and M. Barker, “A data-driven approachfor characterising the charging demand of electric vehicles: a UK case study,”
Applied Energy , vol. 162, pp.763-771, Jan. 2016.[7] G. Zhang, H. X. Li, and M. Gan, “Design a wind speed prediction model using probabilistic fuzzy system,”
IEEE Trans. Ind. Informat. , vol. 8, no. 4, pp. 819-827, Nov. 2012.[8] M. Majidpour, C. Qiu, P. Chu, R. Gadh, and H. R. Pota, “Fast prediction for sparse time series: demand forecastof ev charging stations for cell phone applications,”
IEEE Trans. Ind. Informat. , vol. 11, no. 1, pp. 819-827,Feb. 2015.[9] R. Leou, C. Su, and C. Lu, “Stochastic analyses of electric vehicle charging impacts on distribution network,”
IEEE Trans. Power Syst. , vol. 29, no. 3, pp. 1055-1063, May 2014.[10] J. Donadee and M. D. Ilic, “Stochastic optimization of grid to vehicle frequency regulation capacity bids,”
IEEE Trans. Smart Grid , vol. 5, no. 2, pp. 1061-1069, Mar. 2014. [11] W. Tang and Y. J. Zhang, “A model predictive control approach for low-complexity electric vehicle chargingscheduling: optimality and scalability,” accepted by IEEE Trans. Power Syst. , Jun. 2016.[12] M. Alizadeh, A. Scaglione, J. Davies, and K. S. Kurani, “A scalable stochastic model for the electricity demandof electric and plug-in hybrid vehicles,”
IEEE Trans. Smart Grid , vol. 5, no. 2, pp. 848-860, Mar. 2014.[13] Y. J. Zhang, C. Zhao, W. Tang, S. H. Low, “Profit-maximizing planning and control of battery energy storagesystems for primary frequency control,” accepted by
IEEE Trans. Smart Grid , Apr. 2016.[14] E. Sortomme, M. M. Hindi, S. D. J. MacPherson, and S. S. Venkata, “Coordinated charging of plug-in hybridelectric vehicles to minimize distribution system losses,”
IEEE Trans. Smart Grid , vol.2, no.1, pp. 198-205,Mar. 2011.[15] R. Li, Q. Wu, and S. S. Oren, “Distribution locational marginal pricing for optimal electric vehicle chargingmanagement,”