Anderson Rocha Tavares

A Multiagent Based Road Pricing Approach for Urban Traffic Management

Traffic is a social system composed by different interacting entities and its optimization is not a trivial task. When drivers and infrastructure co-adapt to deal with the varying demand and infrastructure changes, respectively, centralized traffic optimization approaches face many difficulties. This work presents a multiagent based approach that uses variable road pricing to improve traffic efficiency. While infrastructure updates roads prices to cope with the varying demand, drivers try to adapt themselves to the road network changes in order to minimize their costs. Drivers have different preferences, caring either about their travel time (being hasty) or credit expenditure (being economic). Results show that the proposed road pricing approach benefits the hasty drivers, while more sophisticated pricing update policies need to be developed in order to create better alternatives for economic drivers.

Evolving Swarm Intelligence for Task Allocation in a Real Time Strategy Game

Real time strategy games are complex scenarioswhere multiple agents must be coordinated in a dynamic,partially observable environment. In this work, we model thecoordination of these agents as a task allocation problem, in which specific tasks are given to the agents that are more suited to execute them. We employ a task allocation algorithm based on swarm intelligence and adjust its parameters using a genetic algorithm. To evaluate this approach, we implement this coordination mechanism in the AI of a popular video game: StarCraft: BroodWar. Experiment results show that the genetic algorithm enhances performance of the task allocation algorithm. Besides, performance of the proposed approach in matches against StarCrafts native AI is comparable to that of a tournament-level software-controlled player for StarCraft.

Independent learners in abstract traffic scenarios

Traffic is a phenomena that emerges from individual, uncoordinatedand, most of the times, selfish route choice made by drivers. In general, this leads topoor global and individual performance, regarding travel times and road network loadbalance. This work presents a reinforcement learning based approach for route choicewhich relies solely on drivers experience to guide their decisions. There is no coordinatedlearning mechanism, thus driver agents are independent learners. Our approachis tested on two abstract traffic scenarios and it is compared to other route choice methods.Experimental results show that drivers learn routes in complex scenarios with noprior knowledge. Plus, the approach outperforms the compared route choice methodsregarding drivers’ travel time. Also, satisfactory performance is achieved regardingroad network load balance. The simplicity, realistic assumptions and performance ofthe proposed approach suggests that it is a feasible candidate for implementation innavigation systems for guiding drivers decision regarding route choice.

Algorithms or Actions? A Study in Large-Scale Reinforcement Learning

Large state and action spaces are very challenging to reinforcement learning. However, in many domains there is a set of algorithms available, which estimate the best action given a state. Hence, agents can either directly learn a performance-maximizing mapping from states to actions, or from states to algorithms. We investigate several aspects of this dilemma, showing sufficient conditions for learning over algorithms to outperform over actions for a finite number of training iterations. We present synthetic experiments to further study such systems. Finally, we propose a function approximation approach, demonstrating the effectiveness of learning over algorithms in real-time strategy games.

An agent-based approach for road pricing: system-level performance and implications for drivers

BackgroundRoad pricing is a useful mechanism to align private utility of drivers with a system-level measure of performance. Traffic simulation can be used to predict the impact of road pricing policies. The simulation is not a trivial task because traffic is a social system composed of different interacting entities. To tackle this complexity, agent-based approaches can be employed to model the behavior of the several actors in transportation systems.MethodsWe model traffic as a multiagent system in which link manager agents employ a reinforcement learning scheme to determine road pricing policies in a road network. Drivers who traverse the road network are cost-minimizer agents with local information and different preferences regarding travel time and credits expenditure.ResultsThe vehicular flow achieved by our reinforcement learning approach for road pricing is close to a method where drivers have global information of the road network status to choose their routes. Our approach reaches its peak performance faster than a fixed pricing approach. Moreover, drivers’ welfare is greater when the variability of their preferences regarding minimization of travel time or credits expenditure is higher.ConclusionsOur experiments showed that the adoption of reinforcement learning for determining road pricing policies is a promising approach, even with limitations in the driver agent and link manager models.