Kaushik Subramanian

Abstraction from demonstration for efficient reinforcement learning in high-dimensional domains

Reinforcement learning (RL) and learning from demonstration (LfD) are two popular families of algorithms for learning policies for sequential decision problems, but they are often ineffective in high-dimensional domains unless provided with either a great deal of problem-specific domain information or a carefully crafted representation of the state and dynamics of the world. We introduce new approaches inspired by these two techniques, which we broadly call abstraction from demonstration. Our first algorithm, state abstraction from demonstration (AfD), uses a small set of human demonstrations of the task the agent must learn to determine a state-space abstraction. Our second algorithm, abstraction and decomposition from demonstration (ADA), is additionally able to determine a task decomposition from the demonstrations. These abstractions allow RL to scale up to higher-complexity domains, and offer much better performance than LfD with orders of magnitude fewer demonstrations. Using a set of videogame-like domains, we demonstrate that using abstraction from demonstration can obtain up to exponential speed-ups in table-based representations, and polynomial speed-ups when compared with function approximation-based RL algorithms such as fitted Q-learning and LSPI.

HELP---Human assisted Efficient Learning Protocols

OF THE THESIS HELP Human assisted Efficient Learning Protocols by Kaushik Subramanian Thesis Director: Prof. Zoran Gajic In recent years, there has been a growing attention towards the development of artificial agents that can naturally communicate and interact with humans. The focus has primarily been on creating systems that have the ability to unify advanced learning algorithms along with various natural forms of human interaction (like providing advice, guidance, motivation, punishment, etc). However, despite the progress made, interactive systems are still directed towards researchers and scientists and consequently the everyday human is unable to exploit the potential of these systems. Another undesirable component is that in most cases, the interacting human is required to communicate with the artificial agent a large number of times, making the human often fatigued. In order to improve these systems, this thesis extends prior work and introduces novel approaches via Human-assisted Efficient Learning Protocols (HELP). Three case studies are presented that detail distinct aspects of HELP a) representation of the task to be learned and its associated constraints, b) the efficiency of the learning algorithm used by the artificial agent and c) the unexplored “natural” modes of human interaction. The case studies will show how an artificial agent is able to efficiently learn and perform complex tasks using only a limited number of interactions with a human. Each of these studies involves humans subjects interacting with a real robot and/or simulated agent to learn a particular task. The focus of HELP is to show