Reza Yousefian

Design and Real-Time Implementation of Optimal Power System Wide-Area System-Centric Controller Based on Temporal Difference Learning

In this paper, a novel framework for designing and implementing a coordinated wide-area controller architecture for improved power system dynamic stability is presented and tested. The algorithm is an optimal wide-area system-centric controller and observer based on a hybrid reinforcement learning and temporal difference framework. It allows the system to deal with major concerns of wide-area monitoring problem: delays in signal transmission, the uncertainty of the communication network, and data traffic. The main advantage of this design is its ability to learn from the past using eligibility traces and predict the optimal trajectory of cost function through temporal difference method. The control algorithm is evolved from adaptive critic design (ACD) and performed online at a finite horizon through backward and forward view. The ACD controllers training and testing are implemented on the Innovative Integration Picolo card integrated to TMS320C28335 processor. Results on a real experimental test bed using a real power system feeder shows that this architecture provides better stability compared with conventional schemes.

DG-allocation based on reliability indices by means of Monte Carlo simulation and AHP

In this paper, a new method is proposed to determine the best locations of DG units mainly based on reliability indices. Sequential Monte Carlo simulation (sMCs) is utilized to calculate the reliability indices. One of the main advantages of using sMCs in bulk power system reliability analysis is the ability to provide reliability index probability distributions in addition to the expected values of the index. This method can also provide information about the uncertainty related to a definite level of system reliability. Then, the AHP method is used to select the best plans based on reliability indices and some other alternatives including power loss and voltage profile. The results show that the proposed method finds the optimal locations of DG units considering enhancement of criteria with and without uncertainties of reliability and load demand.

Two-tier data-driven intrusion detection for automatic generation control in smart grid

Legacy energy infrastructures are being replaced by modern smart grids. Smart grids provide bi-directional communications for the purpose of efficient energy and load management. In addition, energy generation is adjusted based on the load feedback. However, due to the dependency on the cyber infrastructure for load monitoring and reporting, generation control is inherently vulnerable to attacks. Recent studies have shown that the possibility of data integrity attacks on the generation control can significantly disrupt the energy system. In this work, we present simple yet effective data-driven two-tier intrusion detection system for automatic generation control (AGC). The first tier is a short-term adaptive predictor for system variables, such as load and area control error (ACE). The first tier provides a real-time measurement predictor that adapts to the underlying changing behavior of these system variables, and flags out the abnormal behavior in these variables independently. The second tier provides deep state inspection to investigate the presence of anomalies by incorporating the overall system variable correlation using Markov models. Moreover, we expand our second tier inspection to include multi-AGC environment where a behavior of one AGC is validated against the behavior of the interconnected AGC. The combination of tier-1 light-weight prediction and tier-2 offline deep state inspection offers a great advantage to balance accuracy and real-time requirements of intrusion detection for AGC environment. Our results show high detection accuracy (95%) under different multi-attack scenarios. Second tier successfully verified all the injected intrusions.

Energy Function Inspired Value Priority Based Global Wide-Area Control of Power Grid

Conventional wide-area control strategies that are deployed to damp inter-area oscillations use fixed a priori schemes based on modal analysis of the power system. As these controllers cannot be tuned online, such schemes fail to perform well during severe dynamic system changes. In this paper, a wide-area control architecture designed based on reinforcement learning and optimal adaptive critic network is proposed, that learns and optimizes the system closed-loop performance. Also, a value priority scheme is designed using a derived Lyapunov energy function for prioritization of local and the proposed wide-area global controller which ensures coherent damping of local and inter-area oscillations. The method increases the reliability and allows for automatic tuning of stabilizing controllers especially in the presence of wide-area monitoring constraints. Simulation results on 8-bus 5-machine and 68-bus 16-machine IEEE test systems highlight the efficiency of the proposed method.

Hybrid Transient Energy Function-Based Real-Time Optimal Wide-Area Damping Controller

This paper presents a real-time wide-area damping controller design based on a hybrid intelligent and direct method to improve the power system transient stability. The algorithm applied as a nonlinear optimal wide-area damping controller monitors the oscillations in the system and optimally augments the local excitation system of the synchronous generators. First, energy functions and Prony analysis techniques are used to identify these local or interarea oscillations and develop stability or damping performance index at a given time. Second, artificial neural networks are deployed to learn the dynamics of the system and energy functions based on supervised learning to construct an optimal control design. Then, using online reinforcement learning the quadratic objective function based on the stability index is estimated and optimized forward-in-time. Results on the IEEE 68-bus in Power System Toolbox and HYPERSIM real-time simulator show better transient and damping response when compared to conventional schemes and local power system stabilizers.

Design and real-time implementation of optimal power system wide area system-centric controller based on temporal difference learning

In this paper, a novel framework for designing and implementing a coordinated wide-area controller architecture for improved power system dynamic stability is presented and tested. The algorithm is an optimal wide-area system-centric controller and observer based on a hybrid reinforcement learning and temporal difference framework. It allows the system to deal with major concerns of wide-area monitoring problem: delays in signal transmission, the uncertainty of the communication network, and data traffic. The main advantage of this design is its ability to learn from the past using eligibility traces and predict the optimal trajectory of cost function through temporal difference method. The control algorithm is evolved from adaptive critic design (ACD) and performed online at a finite horizon through backward and forward view. The ACD controllers training and testing are implemented on the Innovative Integration Picolo card integrated to TMS320C28335 processor. Results on a real experimental test bed using a real power system feeder shows that this architecture provides better stability compared with conventional schemes.

System-centric control architecture for wide area monitoring and control of power system

The main objective of this paper is to design and implement secured coordinated control and optimization algorithms and test it for improved power system dynamic stability. The method is to design a brain like control and coordination architecture using optimal control that are modular. Then an intelligent agent knowledge base is developed that can interact with each of the intelligent controllers and optimization algorithms on each bus in the power system providing multi-level and multi-objective capabilities. Such a robust intelligent coordinated controllers with fully functional and fault tolerant Multi-Agent Methodology (MAM) architecture are the important back-bone of next generation smart grid. The main advantage of such a design is its ability to make smart and intelligent control decisions in the presence of drastic system nonlinear movement.

Load curve charactristics improvement by means of optimal utilization of demand response programs

Recently, a massive focus has been made on DR programs, mainly on decreasing electricity price, resolving transmission lines congestion, reliability enhancement and market liquidity improving. Basically, demand response programs are separated into two main groups named, incentive-based programs and time-based programs. The focus of this paper is on Time of Use (TOU) program and Direct Load Control (DLC) program as the common time-based and the most common incentive-based Demand Response (DR) programs, respectively. At first, economic model of programs is developed by using the concept of price elasticity of demand and customer benefit function. The proposed method is appraised by numerical studies based on the selected buses of IEEE 57-bus system for DLC program execution. The impacts of the program on load shape and Expected Power Not Supplied (EPNS) for load specific level before and after program execution are compared for different scenarios.

Hybrid energy function based real-time optimal wide-area transient stability controller for power system stability

This paper presents a hybrid direct and intelligent method of real-time coordinated wide-area controller for improved power system transient stability. The algorithm is applied as an optimal Wide-Area System-Centric Controller and Observer (WASCCO) based on Adaptive Critic Design (ACD). ACD techniques that uses Reinforcement Learning (RL) could be utilized to approximate the transient energy function by dynamic programming and find the solution to nonlinear optimal control problem. However, such technique is highly dependent on the cost function and its dynamics. A Lyupanov-based energy function that is defined offline and updated in real-time through prony analysis is utilized for this purpose. Results on a two area power system and 68-bus New England New York system shows better response compared to conventional schemes and local power system stabilizers.

An approach for real-time tuning of cost functions in optimal System-Centric Wide Area Controller based on Adaptive Critic Design

This paper presents a method for real-time tuning of cost functions in coordinated wide area controller for improved power system transient stability. The algorithm is an optimal Wide Area System-Centric Controller and Observer (WASCCO) based on Adaptive Critic Design (ACD). ACD techniques using reinforcement learning (RL) are able to approximate dynamic programming based nonlinear optimal control but, the cost function for the optimal control requires online tuning. Here cost function defined in ACD are analyzed online with Lyapunov stability function. Further an online tuning method is designed based on eigenvalues extracted based on Prony analysis and linked to performance index generation. Results on a two area system shows better response compared to conventional schemes.