Michail G. Lagoudakis

Auction-Based Multi-Robot Routing.

Recently, auction methods have been investigated as effective, decentralized methods for multi-robot coordination. Experimental research has shown great potential, but has not been complemented yet by theoretical analysis. In this paper we contribute a theoretical analysis of the performance of auction methods for multi-robot routing. We suggest a generic framework for auction-based multi-robot routing and analyze a variety of bidding rules for different team objectives. This is the first time that auction methods are shown to offer theoretical guarantees for such a variety of bidding rules and team objectives.

The Generation of Bidding Rules for Auction-Based Robot Coordination

Robotics researchers have used auction-based coordination systems for robot teams because of their robustness and efficiency. However, there is no research into systematic methods for deriving appropriate bidding rules for given team objectives. In this paper, we propose the first such method and demonstrate it by deriving bidding rules for three possible team objectives of a multi-robot exploration task. We demonstrate experimentally that the resulting bidding rules indeed exhibit good performance for their respective team objectives and compare favorably to the optimal performance. Our research thus allows the designers of auction-based coordination systems to focus on developing appropriate team objectives, for which good bidding rules can then be derived automatically.

Simple auctions with performance guarantees for multi-robot task allocation

We consider the problem of allocating a number of exploration tasks to a team of mobile robots. Each task consists of a target location that needs to be visited by a robot. The objective of the allocation is to minimize the total cost, that is, the sum of the travel costs of all robots for visiting all targets. We show that finding an optimal allocation is an NP-hard problem, even in known environments. The main contribution of this paper is PRIM ALLOCATION, a simple and fast approximate algorithm for allocating targets to robots which provably computes allocations whose total cost is at most twice as large as the optimal total cost. We then cast PRIM ALLOCATION in terms of a multi-round single-item auction where robots bid on targets, which allow for a decentralized implementation. To the best of our knowledge, PRIM ALLOCATION is the first auction-based allocation algorithm that provides a guarantee on the quality of its allocations. Our experimental results in a multi-robot simulator demonstrate that PRIM ALLOCATION is fast and results in close-to-optimal allocations despite its simplicity and decentralized nature. In particular, it needs an order of magnitude fewer bids than a computationally intensive allocation algorithm based on combinatorial auctions, yet its allocations are at least as good.

RoboCup 2009: Robot Soccer world cup XIII

Coordinated Action in a Heterogeneous Rescue Team.- Concept Evaluation of a Reflex Inspired Ball Handling Device for Autonomous Soccer Robots.- Development of a Realistic Simulator for Robotic Intelligent Wheelchairs in a Hospital Environment.- Creating Photo Maps with an Aerial Vehicle in USARsim.- Real-Time Hand Gesture Recognition for Human Robot Interaction.- Combining Key Frame Based Motion Design with Controlled Movement Execution.- Applying Dynamic Walking Control for Biped Robots.- Modeling Human Decision Making Using Extended Behavior Networks.- Motion Synthesis through Randomized Exploration on Submanifolds of Configuration Space.- Robust and Computationally Efficient Navigation in Domestic Environments.- Robust Collision Avoidance in Unknown Domestic Environments.- Real-Time Ball Tracking in a Semi-automated Foosball Table.- Three Humanoid Soccer Platforms: Comparison and Synthesis.- Learning Complementary Multiagent Behaviors: A Case Study.- Rollover as a Gait in Legged Autonomous Robots: A Systems Analysis.- Pareto-Optimal Collaborative Defensive Player Positioning in Simulated Soccer.- A Novel Camera Parameters Auto-adjusting Method Based on Image Entropy.- Object Recognition with Statistically Independent Features: A Model Inspired by the Primate Visual Cortex.- Using Genetic Algorithms for Real-Time Object Detection.- An Approximate Computation of the Dominant Region Diagram for the Real-Time Analysis of Group Behaviors.- A Lua-based Behavior Engine for Controlling the Humanoid Robot Nao.- Stable Mapping Using a Hyper Particle Filter.- A Characterization of 3D Sensors for Response Robots.- Multiple Model Kalman Filters: A Localization Technique for RoboCup Soccer.- Integrated Genetic Algorithmic and Fuzzy Logic Approach for Decision Making of Police Force Agents in Rescue Simulation Environment.- IntellWheels MMI: A Flexible Interface for an Intelligent Wheelchair.- Analyzing the Human-Robot Interaction Abilities of a General-Purpose Social Robot in Different Naturalistic Environments.- Communicating among Robots in the RoboCup Middle-Size League.- Multi-robot Cooperative Object Localization.- Evolution of Biped Walking Using Truncated Fourier Series and Particle Swarm Optimization.- Efficient Behavior Learning by Utilizing Estimated State Value of Self and Teammates.- Sensor and Information Fusion Applied to a Robotic Soccer Team.- Omnidirectional Walking Using ZMP and Preview Control for the NAO Humanoid Robot.- RoboCup@Home: Results in Benchmarking Domestic Service Robots.- Connecting the Real World with the Virtual World - Controlling AIBO through Second Life.- A Hybrid Agent Simulation System of Rescue Simulation and USARSim Simulations from Going to Fire-Escape Doors to Evacuation to Shelters.- SSL-Vision: The Shared Vision System for the RoboCup Small Size League.- Heuristic Formation Control in Multi-robot Systems Using Local Communication and Limited Identification.- Cooperative Multi-robot Map Merging Using Fast-SLAM.

Learning to Select Branching Rules in the DPLL Procedure for Satisfiability

Abstract The DPLL procedure is the most popular complete satisfiability (SAT) solver. While its worst case complexity is exponential, the actual running time is greatly affected by the ordering of branch variables during the search. Several branching rules have been proposed, but none is the best in all cases. This work investigates the use of automated methods for choosing the most appropriate branching rule at each node in the search tree. We consider a reinforcement-learning approach where a value function, which predicts the performance of each branching rule in each case, is learned through trial runs on a typical problem set of the target class of SAT problems. Our results indicate that, provided sufficient training on a given class, the resulting strategy performs as well as (and, in some cases, better than) the best branching rule for that class. Research supported in part by NSF grant IRI-9702576. The first author was also partially supported by the Lilian-Voudouri Foundation in Greece. The authors gratefully acknowledge the influence of Don Loveland, Ron Parr, and Henry Kautz in helping to shape this work.

Rollout sampling approximate policy iteration

Several researchers have recently investigated the connection between reinforcement learning and classification. We are motivated by proposals of approximate policy iteration schemes without value functions, which focus on policy representation using classifiers and address policy learning as a supervised learning problem. This paper proposes variants of an improved policy iteration scheme which addresses the core sampling problem in evaluating a policy through simulation as a multi-armed bandit machine. The resulting algorithm offers comparable performance to the previous algorithm achieved, however, with significantly less computational effort. An order of magnitude improvement is demonstrated experimentally in two standard reinforcement learning domains: inverted pendulum and mountain-car.

Least-Squares Methods in Reinforcement Learning for Control

Least-squares methods have been successfully used for prediction problems in the context of reinforcement learning, but little has been done in extending these methods to control problems. This paper presents an overview of our research efforts in using least-squares techniques for control. In our early attempts, we considered a direct extension of the Least-Squares Temporal Difference (LSTD) algorithm in the spirit of Q-learning. Later, an effort to remedy some limitations of this algorithm (approximation bias, poor sample utilization) led to the Least-Squares Policy Iteration (LSPI) algorithm, which is a form of model-free approximate policy iteration and makes efficient use of training samples collected in any arbitrary manner. The algorithms are demonstrated on a variety of learning domains, including algorithm selection, inverted pendulum balancing, bicycle balancing and riding, multiagent learning in factored domains, and, recently, on two-player zero-sum Markov games and the game of Tetris.

A decision support system to facilitate management of patients with acute gastrointestinal bleeding

OBJECTIVE To develop a model to predict the bleeding source and identify the cohort amongst patients with acute gastrointestinal bleeding (GIB) who require urgent intervention, including endoscopy. Patients with acute GIB, an unpredictable event, are most commonly evaluated and managed by non-gastroenterologists. Rapid and consistently reliable risk stratification of patients with acute GIB for urgent endoscopy may potentially improve outcomes amongst such patients by targeting scarce healthcare resources to those who need it the most. DESIGN AND METHODS Using ICD-9 codes for acute GIB, 189 patients with acute GIB and all available data variables required to develop and test models were identified from a hospital medical records database. Data on 122 patients was utilized for development of the model and on 67 patients utilized to perform comparative analysis of the models. Clinical data such as presenting signs and symptoms, demographic data, presence of co-morbidities, laboratory data and corresponding endoscopic diagnosis and outcomes were collected. Clinical data and endoscopic diagnosis collected for each patient was utilized to retrospectively ascertain optimal management for each patient. Clinical presentations and corresponding treatment was utilized as training examples. Eight mathematical models including artificial neural network (ANN), support vector machine (SVM), k-nearest neighbor, linear discriminant analysis (LDA), shrunken centroid (SC), random forest (RF), logistic regression, and boosting were trained and tested. The performance of these models was compared using standard statistical analysis and ROC curves. RESULTS Overall the random forest model best predicted the source, need for resuscitation, and disposition with accuracies of approximately 80% or higher (accuracy for endoscopy was greater than 75%). The area under ROC curve for RF was greater than 0.85, indicating excellent performance by the random forest model. CONCLUSION While most mathematical models are effective as a decision support system for evaluation and management of patients with acute GIB, in our testing, the RF model consistently demonstrated the best performance. Amongst patients presenting with acute GIB, mathematical models may facilitate the identification of the source of GIB, need for intervention and allow optimization of care and healthcare resource allocation; these however require further validation.

Binary action search for learning continuous-action control policies

Reinforcement Learning methods for controlling stochastic processes typically assume a small and discrete action space. While continuous action spaces are quite common in real-world problems, the most common approach still employed in practice is coarse discretization of the action space. This paper presents a novel method, called Binary Action Search, for realizing continuousaction policies by searching efficiently the entire action range through increment and decrement modifications to the values of the action variables according to an internal binary policy defined over an augmented state space. The proposed approach essentially approximates any continuous action space to arbitrary resolution and can be combined with any discrete-action reinforcement learning algorithm for learning continuous-action policies. Binary Action Search eliminates the restrictive modification steps of Adaptive Action Modification and requires no temporal action locality in the domain. Our approach is coupled with two well-known reinforcement learning algorithms (Least-Squares Policy Iteration and Fitted Q-Iteration) and its use and properties are thoroughly investigated and demonstrated on the continuous state-action Inverted Pendulum, Double Integrator, and Car on the Hill domains.

Coordinated Team Play in the Four-Legged RoboCup League

This paper deals with the multi-agent patrolling problem in unknown environment using two collective approaches exploiting environmental dynamics. After specifying criteria of performances, we define a first algorithm based only on the evaporation of a pheromone dropped by reactive agents (EVAP). Then we present the model CLInG [10] proposed in 2003 which introduces the diffusion of the idleness of areas to visit. We systematically compare by simulations the performances of these two models on growing- complexity environments. The analysis is supplemented by a comparison with the theoretical optimum performances, allowing to identify topologies for which methods are the most adapted.For a decade now, the RoboCup competition promotes research in robotics through soccer games between autonomous robot teams. The ability to coordinate the players within such a team of robots is the key to the success of the team. Team coordination in a human soccer game is achieved through various team formations, tactics, and strategies. Unfortunately, research in the four-legged RoboCup league has focused mostly on single player skills, demonstrating only limited results in coordinated team play. In our work, we adapt and transfer formations, tactics, and strategies used by human soccer teams, such as the popular 4-4-2 scheme, to our four-legged RoboCup team Kouretes. We define roles for each player in all the cases we consider and we implement these roles using Petri Net Plans (PNP). The assignment of appropriate roles to players is performed dynamically during the game depending on the current game state using a simple communication scheme and a finite state machine. Our approach is implemented and tested on our four-legged RoboCup team. The proposed coordination scheme can be generalized and used in various robot team applications beyond robotic soccer, such as planetary exploration and search-and-rescue missions.