Jennie Si

Handbook of learning and approximate dynamic programming

Foreword. 1. ADP: goals, opportunities and principles. Part I: Overview. 2. Reinforcement learning and its relationship to supervised learning. 3. Model-based adaptive critic designs. 4. Guidance in the use of adaptive critics for control. 5. Direct neural dynamic programming. 6. The linear programming approach to approximate dynamic programming. 7. Reinforcement learning in large, high-dimensional state spaces. 8. Hierarchical decision making. Part II: Technical advances. 9. Improved temporal difference methods with linear function approximation. 10. Approximate dynamic programming for high-dimensional resource allocation problems. 11. Hierarchical approaches to concurrency, multiagency, and partial observability. 12. Learning and optimization - from a system theoretic perspective. 13. Robust reinforcement learning using integral-quadratic constraints. 14. Supervised actor-critic reinforcement learning. 15. BPTT and DAC - a common framework for comparison. Part III: Applications. 16. Near-optimal control via reinforcement learning. 17. Multiobjective control problems by reinforcement learning. 18. Adaptive critic based neural network for control-constrained agile missile. 19. Applications of approximate dynamic programming in power systems control. 20. Robust reinforcement learning for heating, ventilation, and air conditioning control of buildings. 21. Helicopter flight control using direct neural dynamic programming. 22. Toward dynamic stochastic optimal power flow. 23. Control, optimization, security, and self-healing of benchmark power systems.

2007 IEEE international symposium on approximate dynamic programming and reinforcement learning

Welcome to ADPRL 2007 - the very first IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning. The area of approximate dynamic programming and reinforcement learning is a fusion of a number of research areas in engineering, mathematics, artificial intelligence, operations research, and systems and control theory. Fifty years after Richard Bellman’s pioneering work on dynamic programming in the 1950’s, this symposium will provide a remarkable opportunity for the academic and industrial community to address new challenges, share solutions, and define promising future research directions. The main challenge introduced by Bellman which still remains a widely open question is the famous “curse of dimensionality.” The theme of this year’s symposium is “breaking the curse of dimensionality.” A systems approach is required to address new problems of this challenging and promising area, and designing biologically-inspired intelligent systems may be an interesting way to address the problems. You will enjoy an extraordinary technical program thanks to the ADPRL 2007 International Program Committee members who worked very hard to have all papers reviewed before the review deadline. We received a total of 65 submissions from various parts of the world. The final technical program consists of 49 papers among which 40 are oral session papers and 9 are poster session papers. There will be a keynote lecture delivered by Frank L. Lewis entitled “Adaptive Dynamic Programming for Robust Optimal Control Using Nonlinear Network Learning Structures.” We would like to also express our sincere gratitude to all reviewers of ADPRL 2007 for the time and effort they have generously given to the symposium.

NearOptimal Control Through Reinforcement Learning and Hybridization

This chapter focuses on learning to act in a near-optimal manner through reinforcement learning for problems that either have no model or whose model is very complex. The emphasis here is on continuous action space (CAS) methods. Monte-Carlo approaches are employed to estimate function values in an iterative, incremental procedure. Derivative-free line search methods are used to find a near-optimal action in the continuous action space for a discrete subset of the state space. This near-optimal policy is then extended to the entire continuous state space using a fuzzy additive model. To compensate for approximation errors, a modified procedure for perturbing the generated control policy is developed. Convergence results, under moderate assumptions and stopping criteria, are established. References to sucessful applications of the controller are provided.