Majd Hawasly

Bayesian policy reuse

A long-lived autonomous agent should be able to respond online to novel instances of tasks from a familiar domain. Acting online requires ‘fast’ responses, in terms of rapid convergence, especially when the task instance has a short duration such as in applications involving interactions with humans. These requirements can be problematic for many established methods for learning to act. In domains where the agent knows that the task instance is drawn from a family of related tasks, albeit without access to the label of any given instance, it can choose to act through a process of policy reuse from a library in contrast to policy learning. In policy reuse, the agent has prior experience from the class of tasks in the form of a library of policies that were learnt from sample task instances during an offline training phase. We formalise the problem of policy reuse and present an algorithm for efficiently responding to a novel task instance by reusing a policy from this library of existing policies, where the choice is based on observed ‘signals’ which correlate to policy performance. We achieve this by posing the problem as a Bayesian choice problem with a corresponding notion of an optimal response, but the computation of that response is in many cases intractable. Therefore, to reduce the computation cost of the posterior, we follow a Bayesian optimisation approach and define a set of policy selection functions, which balance exploration in the policy library against exploitation of previously tried policies, together with a model of expected performance of the policy library on their corresponding task instances. We validate our method in several simulated domains of interactive, short-duration episodic tasks, showing rapid convergence in unknown task variations.

Multiscale Topological Trajectory Classification with Persistent Homology

Topological approaches to studying equivalence classes of trajectories in a configuration space have recently received attention in robotics since they allow a robot to reason about trajectories at a high level of abstraction. While recent work has approached the problem of topological motion planning under the assumption that the configuration space and obstacles within it are explicitly described in a noise-free manner, we focus on trajectory classification and present a sampling-based approach which can handle noise, which is applicable to general configuration spaces and which relies only on the availability of collision free samples. Unlike previous sampling-based approaches in robotics which use graphs to capture information about the path-connectedness of a configuration space, we construct a multiscale approximation of neighborhoods of the collision free configurations based on filtrations of simplicial complexes. Our approach thereby extracts additional homological information which is essential for a topological trajectory classification. By computing a basis for the first persistent homology groups, we obtain a multiscale classification algorithm for trajectories in configuration spaces of arbitrary dimension. We furthermore show how an augmented filtration of simplicial complexes based on a cost function can be defined to incorporate additional constraints. We present an evaluation of our approach in 2, 3, 4 and 6 dimensional configuration spaces in simulation and using a Baxter robot.

Topological trajectory classification with filtrations of simplicial complexes and persistent homology

In this work, we present a sampling-based approach to trajectory classification which enables automated high-level reasoning about topological classes of trajectories. Our approach is applicable to general configuration spaces and relies only on the availability of collision free samples. Unlike previous sampling-based approaches in robotics which use graphs to capture information about the path-connectedness of a configuration space, we construct a multiscale approximation of neighborhoods of the collision free configurations based on filtrations of simplicial complexes. Our approach thereby extracts additional homological information which is essential for a topological trajectory classification. We propose a multiscale classification algorithm for trajectories in configuration spaces of arbitrary dimension and for sets of trajectories starting and ending in two fixed points. Using a cone construction, we then generalize this approach to classify sets of trajectories even when trajectory start and end points are allowed to vary in path-connected subsets. We furthermore show how an augmented filtration of simplicial complexes based on an arbitrary function on the configuration space, such as a costmap, can be defined to incorporate additional constraints. We present an evaluation of our approach in 2-, 3-, 4- and 6-dimensional configuration spaces in simulation and in real-world experiments using a Baxter robot and motion capture data.

Social networks save energy: optimizing energy consumption in an ecovillage via agent-based simulation

Energy-conscious communities are continually challenged to optimize electricity usage, maximizing the benefits obtainable from generating systems and minimizing the reliance on the national supply. Achieving an ideal balance is complicated by the fluctuating availability of ‘green’ supplies and the varying patterns of domestic usage. Optimizing a communitys net energy balance depends on the degree to which householders can modify their usage patterns. We consider two questions here: given a collection of realistic preferences and constraints on usage patterns of households, what degree of saving is possible by optimizing ‘within’ these preferences and constraints?; and, what amounts of energy saving are possible when the community exploits its social network by sharing electrical appliances? These questions are investigated in the context of Riccarton Ecovillage. A model of the ecovillage was implemented using an agent-based modelling toolkit, then simulated under a range of scenarios, automatically exploring the space of usage patterns to find combinations of usage schedules that minimized dependence on the national supply. Our findings are: evolutionary algorithms perform particularly well at this difficult optimization task; modest savings of 5–10% are achievable under standard assumptions, but savings of 35–40% are achievable when exploiting the underlying social network.