Nikolaos Kariotoglou

Approximate dynamic programming via sum of squares programming

We describe an approximate dynamic programming method for stochastic control problems on infinite state and input spaces. The optimal value function is approximated by a linear combination of basis functions with coefficients as decision variables. By relaxing the Bellman equation to an inequality, one obtains a linear program in the basis coefficients with an infinite set of constraints. We show that a recently introduced method, which obtains convex quadratic value function approximations, can be extended to higher order polynomial approximations via sum of squares programming techniques. An approximate value function can then be computed offline by solving a semidefinite program, without having to sample the infinite constraint. The policy is evaluated online by solving a polynomial optimization problem, which also turns out to be convex in some cases. We experimentally validate the method on an autonomous helicopter testbed using a 10-dimensional helicopter model.

A stochastic reachability framework for autonomous surveillance with pan-tilt-zoom cameras

In this work a framework for camera-based autonomous surveillance is introduced based on the theory of stochastic reachability and random sets. We consider set-valued models of a single pan-tilt-zoom (PTZ) camera (pursuer) and multiple targets that need to be tracked (evaders). We define the stochastic pursuer process and the stochastic evader processes and consider the problem of maximizing the probability of satisfying safety (tracking), reachability (acquisition), and reach-avoid (tracking while acquiring) objectives. The solution of the safety, reachability, and reach-avoid tasks are computed via dynamic programming resulting in an optimal control policy for the PTZ camera. Experimental results are given for a single PTZ camera and multiple robotic evaders.

Probabilistic certification of pan-tilt-zoom camera surveillance systems

In this work a method to evaluate the performance of autonomous patrolling systems is introduced based on stochastic reachability with random sets. We consider set-valued models with stochastic dynamics for multiple pan-tilt-zoom (PTZ) cameras acting as pursuers and a single evader. The problem of maximizing the probability that the evader successfully completes an intrusion objective while avoiding capture by the cameras is considered and posed as a stochastic reach-avoid problem. The solution of the stochastic reach-avoid problem is solved via dynamic programming where the optimal value function is used as a quality indicator of each patrolling strategy. A comparison between multiple patrolling strategies is provided via simulation of a realistic patrolling scenario.

Multi-Agent Autonomous Surveillance: A Framework Based on Stochastic Reachability and Hierarchical Task Allocation

We develop and implement a framework to address autonomous surveillance problems with a collection of pan-tilt (PT) cameras. Using tools from stochastic reachability with random sets, we formulate the problems of target acquisition, target tracking, and acquisition while tracking as reach-avoid dynamic programs for Markov decision processes (MDPs). It is well known that solution methods for MDP problems based on dynamic programming (DP), implemented by state space gridding, suffer from the curse of dimensionality. This becomes a major limitation when one considers a network of PT cameras. To deal with larger problems we propose a hierarchical task allocation mechanism that allows cameras to calculate reach-avoid objectives independently while achieving tasks collectively. We evaluate the proposed algorithms experimentally on a setup involving industrial PT cameras and mobile robots as targets.

On the computational complexity and generalization properties of multi-stage and stage-wise coupled scenario programs

Abstract We discuss the computational complexity and feasibility properties of scenario sampling techniques for uncertain optimization programs. We propose an alternative way of dealing with a special class of stage-wise coupled programs and compare it with existing methods in the literature in terms of feasibility and computational complexity. We identify trade-offs between different methods depending on the problem structure and the desired probability of constraint satisfaction. To illustrate our results, an example from the area of approximate dynamic programming is considered.

Design of importance-map based randomized patrolling strategies

We propose a method for designing randomized patrolling strategies that take into account the presence of high value areas. An importance map of the surveillance environment is constructed that explicitly accounts for (and prioritizes) high value areas. The method translates the designed importance map into pan-tilt-zoom camera specific guidance maps. Considering multiple cameras, the mapping between importance and guidance maps involves a distribution of the surveillance coverage objectives, which is achieved in two different ways, a heuristic and a Linear Program (LP). Each camera then monitors the site according to a Markov Chain Monte Carlo (MCMC) algorithm guided by these maps.

The Linear Programming Approach to Reach-Avoid Problems for Markov Decision Processes

One of the most fundamental problems in Markov decision processes is analysis and control synthesis for safety and reachability specifications. We consider the stochastic reach-avoid problem, in which the objective is to synthesize a control policy to maximize the probability of reaching a target set at a given time, while staying in a safe set at all prior times. We characterize the solution to this problem through an infinite dimensional linear program. We then develop a tractable approximation to the infinite dimensional linear program through finite dimensional approximations of the decision space and constraints. For a large class of Markov decision processes modeled by Gaussian mixtures kernels we show that through a proper selection of the finite dimensional space, one can further reduce the computational complexity of the resulting linear program. We validate the proposed method and analyze its potential with a series of numerical case studies.

Model-based current limiting for traction control of an electric four-wheel drive race car

This paper describes a novel traction control method and its application to an electric four-wheel driven race car. The proposed control method is based on a detailed model of tire dynamics and is designed for hardware with limited memory and computational power. We derive a linear parameter-varying model from first principles and validate it against a full nonlinear vehicle model. We then use the model to design a gain-scheduled LQRI controller, parametric on the measured vehicle velocity and lateral acceleration. We show that when incorporating additional information about the tire state, a gain scheduled LQRI controller is capable of minimizing excessive wheel spin by limiting the maximum torque available to the driver. This leads to a performance gain in acceleration while improving the handling characteristics of the race car. The proposed controller is thoroughly tested for its sensitivity to sensor noise and changes in system parameters in simulation and then implemented on a prototype race car competing in Formula Student. Experiments indicate satisfactory experimental performance from the initial control design without additional tuning of the controller parameters. This illustrates the simplicity of the design and ease of implementation.

Experimental Validation of Patrolling Strategies in an Automated Surveillance Environment

Abstract The Autonomous Robotic Patrolling and Surveillance environment (AuRoPaS) is a testbed at the Automatic Control Laboratory of ETH Zurich to experimentally validate tracking, observation, and monitoring strategies for security systems. The setup comprises two high performance closed-circuit television (CCTV) cameras and mobile robots to simulate different types of surveillance scenarios. We propose a velocity based model predictive control scheme for the camera movements, which allows us to generate smooth trajectories and acquire stable images from targets. Experimental results demonstrate the successful reference tracking of the camera controller. We illustrate the integration of high level algorithms into the testbed by applying two stochastic patrolling strategies. The patrolling performances are evaluated on a scenario with moving targets visiting prioritized regions.