Guang Shi

A Novel Dual Iterative

In this paper, a novel iterative Q-learning method called “dual iterative Q-learning algorithm” is developed to solve the optimal battery management and control problem in smart residential environments. In the developed algorithm, two iterations are introduced, which are internal and external iterations, where internal iteration minimizes the total cost of power loads in each period, and the external iteration makes the iterative Q-function converge to the optimum. Based on the dual iterative Q-learning algorithm, the convergence property of the iterative Q-learning method for the optimal battery management and control problem is proven for the first time, which guarantees that both the iterative Q-function and the iterative control law reach the optimum. Implementing the algorithm by neural networks, numerical results and comparisons are given to illustrate the performance of the developed algorithm.

Q

In this paper, a novel distributed iterative adaptive dynamic programming (ADP) method is developed to solve the multibattery optimal coordination control problems for home energy management systems. According to system transformations, the multi-input optimal control problem is transformed into a single-input optimal control problem, where all the batteries are implemented at their worst performance. Next, based on the worst-performance optimal control law, an effective distributed iterative ADP algorithm is developed, where, in each iteration, only a single-input optimization problem is implemented. Convergence properties of the distributed iterative ADP algorithm are developed to show that the iterative performance index function converges to the optimum. Finally, numerical analysis is given to illustrate the performance of the developed algorithm.

-Learning Method for Optimal Battery Management in Smart Residential Environments

In this paper, a novel optimal energy storage control scheme is investigated in smart grid environments with solar renewable energy. Based on the idea of adaptive dynamic programming (ADP), a self-learning algorithm is constructed to obtain the iterative control law sequence of the battery. Based on the data of the real-time electricity price (electricity rate in brief), the load demand (load in brief), and the solar renewable energy (solar energy in brief), the optimal performance index function, which minimizes the total electricity cost and simultaneously extends the batterys lifetime, is established. A new analysis method of the iterative ADP algorithm is developed to guarantee the convergence of the iterative value function to the optimum under iterative control law sequence for any time index in a period. Numerical results and comparisons are presented to illustrate the effectiveness of the developed algorithm.

Multibattery Optimal Coordination Control for Home Energy Management Systems via Distributed Iterative Adaptive Dynamic Programming

In this paper, an optimization method based on adaptive dynamic programming is developed to improve the electricity consumption of rooms in office buildings through optimal battery management. Rooms in office buildings are generally divided into office rooms, computer rooms, storage rooms, meeting rooms, etc., and each category of rooms have different characteristics of electricity consumption, which is divided into electricity consumption from sockets, lights and air-conditioners in this paper. The developed method based on action-dependent heuristic dynamic programming is explained in detail, and different optimization strategies of electricity consumption in different categories of rooms are proposed in accordance with the developed method. Finally, a detailed case study on an office building is given to demonstrate the practical effect of the developed method.

Adaptive Dynamic Programming-Based Optimal Control Scheme for Energy Storage Systems With Solar Renewable Energy

In this paper, energy consumption of an office building is predicted based on echo state networks (ESNs). Energy consumption of the office building is divided into consumptions from sockets, lights and air-conditioners, which are measured in each room of the office building by three ammeters installed inside, respectively. On the other hand, an office building generally consists of several types of rooms, i.e., office rooms, computer rooms, storage rooms, meeting rooms, etc., the energy consumption of which varies in accordance with different working routines in each type of rooms. In this paper, several novel reservoir topologies of ESNs are developed, the performance of ESNs with different reservoir topologies in predicting the energy consumption of rooms in the office building is compared, and the energy consumption of all the rooms in the office building is predicted with the developed topologies. Moreover, parameter sensitivity of ESNs with different reservoir topologies is analyzed. A case study shows that the developed simplified reservoir topologies are sufficient to achieve outstanding performance of ESNs in the prediction of building energy consumption.

Optimization of electricity consumption in office buildings based on adaptive dynamic programming

In this paper, a novel error-tolerant iterative adaptive dynamic programming (ADP) algorithm is developed to solve optimal battery control and management problems in smart home environments with renewable energy. A main contribution for the iterative ADP algorithm is to implement with the electricity rate, home load demand, and renewable energy as quasi-periodic functions, instead of accurate periodic functions, where the discount factor can adaptively be regulated in each iteration to guarantee the convergence of the iterative value function. A new analysis method is developed to guarantee the iterative value function to converge to a finite neighborhood of the optimal performance index function, in spite of the differences of the electricity rate, the home load demand, and the renewable energy in different periods. Neural networks are employed to approximate the iterative value function and control law, respectively, for facilitating the implementation of the iterative ADP algorithm. Numerical results and comparisons are given to illustrate the performance of the developed algorithm.

Energy consumption prediction of office buildings based on echo state networks

In this paper, based on echo state network (ESN), a data-driven method is developed to solve the room classification problem of office buildings. The developed method is divided into two steps. Given the data of electricity consumption, which are classified into electricity consumption from sockets, lights and air-conditioners for a typical room in an office building, the first step is to reconstruct the behavior of electricity consumption in three types by using three ESNs. The second step is to classify the room into a certain category of office room, computer room, storage room and meeting room by establishing another ESN. The developed method fully utilizes the outstanding performance of ESN in chaotic time-series prediction and classification. Practical study on an office building illustrates the accuracy and effectiveness of the developed method.

Error-Tolerant Iterative Adaptive Dynamic Programming for Optimal Renewable Home Energy Scheduling and Battery Management

In this paper, a novel dual iterative Q-learning algorithm is developed to solve the optimal battery management and control problems in smart residential environments. The main idea is to use adaptive dynamic programming (ADP) technique to obtain the optimal battery management and control scheme iteratively for residential energy systems. In the developed dual iterative Q-learning algorithm, two iterations, including external and internal iterations, are introduced, where internal iteration minimizes the total cost of power loads in each period and the external iteration makes the iterative Q function converge to the optimum. For the first time, the convergence property of iterative Q-learning method is proven to guarantee the convergence property of the iterative Q function. Finally, numerical results are given to illustrate the performance of the developed algorithm.

Data-driven room classification for office buildings based on echo state network

In this paper, a Q-learning based algorithm is developed to optimize energy consumption in an office, where solar energy is introduced as the renewable source and a battery is installed as the control unit. The energy consumption in the office, regarded as the energy demand, is divided into those from sockets, lights and air-conditioners. First, the time series of real-time electricity rate, renewable energy, and energy demand are modeled by echo state networks as periodic functions. Second, given these periodic functions, a Q-learning based algorithm is developed for optimal control of the battery in the office, so that the total cost on energy from the grid is reduced. Finally, numerical analysis is conducted to show the performance of the developed algorithm.

Optimal self-learning battery control in smart residential grids by iterative Q-learning algorithm

This paper investigates a novel fault tolerant control scheme to handle actuator faults in nonlinear systems based on policy iteration algorithm with fault observer. The fault observer is established to estimate the actuator fault, which is used to construct an improved performance index function that reflects the fault, regulation and control simultaneously. With the help of the proper performance index function, the FTC problem is transformed into an optimal control problem. By constructing a critic neural network, the Hamilton-Jacobi-Bellman equation can be solved by using policy iteration algorithm, and then the approximated fault tolerant controller can be obtained directly. The closed-loop system is guaranteed to be uniformly ultimately bounded based on the Lyapunovs direct method. Simulation example is given to verify the effectiveness of the developed FTC scheme.