Ana L. C. Bazzan

A Distributed Approach for Coordination of Traffic Signal Agents

Innovative control strategies are needed to cope with the increasing urban traffic chaos. In most cases, the currently used strategies are based on a central traffic-responsive control system which can be demanding to implement and maintain. Therefore, a functional and spatial decentralization is desired. For this purpose, distributed artificial intelligence and multi-agent systems have come out with a series of techniques which allow coordination and cooperation. However, in many cases these are reached by means of communication and centrally controlled coordination processes, giving little room for decentralized management. Consequently, there is a lack of decision-support tools at managerial level (traffic control centers) capable of dealing with decentralized policies of control and actually profiting from them. In the present work a coordination concept is used, which overcomes some disadvantages of the existing methods. This concept makes use of techniques of evolutionary game theory: intersections in an arterial are modeled as individually-motivated agents or players taking part in a dynamic process in which not only their own local goals but also a global one has to be taken into account. The role of the traffic manager is facilitated since s/he has to deal only with tactical ones, leaving the operational issues to the agents. Thus the system ultimately provides support for the traffic manager to decide on traffic control policies. Some application in traffic scenarios are discussed in order to evaluate the feasibility of transferring the responsibility of traffic signal coordination to agents. The results show different performances of the decentralized coordination process in different scenarios (e.g. the flow of vehicles is nearly equal in both opposing directions, one direction has a clearly higher flow, etc.). Therefore, the task of the manager is facilitate once s/he recognizes the scenario and acts accordingly.

Decision dynamics in a traffic scenario

Information is a key commodity in many socio-economic systems like stock markets or traffic systems. In this paper the influence of dynamic information on the stability of traffic patterns is investigated using a very simple route choice scenario. The basis of the route decisions is dynamic information generated by traffic flow simulations. A correlation analysis yields that the system can be destabilized by introducing information. It is found that the overall performance of the system is reduced, although the information should help to distribute traffic more efficiently.

The impact of real-time information in a two-route scenario using agent-based simulation

Abstract Since advanced traveler information systems (ATIS) have been introduced, their potential benefits as well as their drawbacks have been discussed controversially. This will continue as long as the drivers’ reactions upon current or even predictive information about the traffic situation are not known. Thus, traffic models that also consider this feedback are necessary. In this paper, we address a basic two-route scenario with different types of information and study the impact of it using simulations. The road users are modeled as agents, a natural and promising approach to describe them. Different ways of generating current information are tested. It is pointed out that the nature of the information very much influences the potential benefits of the ATIS.

Opportunities for multiagent systems and multiagent reinforcement learning in traffic control

The increasing demand for mobility in our society poses various challenges to traffic engineering, computer science in general, and artificial intelligence and multiagent systems in particular. As it is often the case, it is not possible to provide additional capacity, so that a more efficient use of the available transportation infrastructure is necessary. This relates closely to multiagent systems as many problems in traffic management and control are inherently distributed. Also, many actors in a transportation system fit very well the concept of autonomous agents: the driver, the pedestrian, the traffic expert; in some cases, also the intersection and the traffic signal controller can be regarded as an autonomous agent. However, the “agentification” of a transportation system is associated with some challenging issues: the number of agents is high, typically agents are highly adaptive, they react to changes in the environment at individual level but cause an unpredictable collective pattern, and act in a highly coupled environment. Therefore, this domain poses many challenges for standard techniques from multiagent systems such as coordination and learning. This paper has two main objectives: (i) to present problems, methods, approaches and practices in traffic engineering (especially regarding traffic signal control); and (ii) to highlight open problems and challenges so that future research in multiagent systems can address them.

A review on agent-based technology for traffic and transportation

In the last few years, the number of papers devoted to applications of agent-based technologies to traffic and transportation engineering has grown enormously. Thus, it seems to be the appropriate time to shed light over the achievements of the last decade, on the questions that have been successfully addressed, as well as on remaining challenging issues. In the present paper, we review the literature related to the areas of agent-based traffic modelling and simulation, and agent-based traffic control and management. Later we discuss and summarize the main achievements and the challenges.

AgentSpeak(XL): efficient intention selection in BDI agents via decision-theoretic task scheduling

This paper shows how to use a decision-theoretic task scheduler in order to automatically generate efficient intention selection functions for BDI agent-oriented programming languages. We concentrate here on the particular extensions to a known BDI language called AgentSpeak(L) and its interpreter which were necessary so that the integration with a task scheduler was possible. The proposed language, called AgentSpeak(XL), has several other features which increase its usability; some of these are indicated briefly in this paper. We assess the extended language and its interpreter by means of a factory plant scenario where there is one mobile robot that is in charge of packing and storing items, besides other administrative and security tasks. This case study and its simulation results show that, in comparison to AgentSpeak(L), AgentSpeak(XL) provides much easier and efficient implementation of applications that require quantitative reasoning, or require specific control over intentions (e.g., for giving priority to certain tasks once they become intended).

Using BDI agents to improve driver modelling in a commuter scenario

The use of multi-agent systems to model and to simulate real systems consisting of intelligent entities capable of autonomously co-operating with each other has emerged as an important field of research. This has been applied to a variety of areas, such as social sciences, engineering, and mathematical and physical theories. In this work, we address the complex task of modelling drivers’ behaviour through the use of agent-based techniques. Contemporary traffic systems have experienced considerable changes in the last few years, and the rapid growth of urban areas has challenged scientific and technical communities. Influencing drivers’ behaviour appears as an alternative to traditional approaches to cope with the potential problem of traffic congestion, such as the physical modification of road infrastructures and the improvement of control systems. It arises as one of the underlying ideas of intelligent transportation systems. In order to offer a good means to evaluate the impact that exogenous information may exert on drivers’ decision making, we propose an extension to an existing microscopic simulation model called Dynamic Route Assignment Combining User Learning and microsimulAtion (DRACULA). In this extension, the traffic domain is viewed as a multi-agent world and drivers are endowed with mental attitudes, which allow rational decisions about route choice and departure time. This work is divided into two main parts. The first part describes the original DRACULA framework and the extension proposed to support our agent-based traffic model. The second part is concerned with the reasoning mechanism of drivers modelled by means of a Beliefs, Desires, and Intentions (BDI) architecture. In this part, we use AgentSpeak(L) to specify commuter scenarios and special emphasis is given to departure time and route choices. This paper contributes in that respect by showing a practical way of representing and assessing drivers’ behaviour and the adequacy of using AgentSpeak(L) as a modelling language, as it provides clear and elegant specifications of BDI agents.

Diagnosis as an integral part of multi-agent adaptability

Agents working under real world conditions may face an environment capable of changing rapidly from one moment to the next, either through perceived faults, unexpected interactions or adversarial intrusions. The members of a multi-agent system can gracefully and efficiently handle such situations by adapting, either by evolving internal structures and behavior or repairing or isolating those external influences believed to be malfunctioning. The first step in achieving adaptability is diagnosis-being able to accurately detect and determine the cause of a fault based on its symptoms. In this paper we examine how domain independent diagnosis plays a role in multi-agent systems, including the information required to support and produce diagnoses. Particular attention is paid to coordination based diagnosis directed by a causal model. Several examples are described in the context of an Intelligent Home environment, and the issue of diagnostic sensitivity versus efficiency is addressed.

Dealing with non-stationary environments using context detection

In this paper we introduce RL-CD, a method for solving reinforcement learning problems in non-stationary environments. The method is based on a mechanism for creating, updating and selecting one among several partial models of the environment. The partial models are incrementally built according to the systems capability of making predictions regarding a given sequence of observations. We propose, formalize and show the efficiency of this method both in a simple non-stationary environment and in a noisy scenario. We show that RL-CD performs better than two standard reinforcement learning algorithms and that it has advantages over methods specifically designed to cope with non-stationarity. Finally, we present known limitations of the method and future works.

Traffic light control in non-stationary environments based on multi agent Q-learning

In many urban areas where traffic congestion does not have the peak pattern, conventional traffic signal timing methods does not result in an efficient control. One alternative is to let traffic signal controllers learn how to adjust the lights based on the traffic situation. However this creates a classical non-stationary environment since each controller is adapting to the changes caused by other controllers. In multi-agent learning this is likely to be inefficient and computationally challenging, i.e., the efficiency decreases with the increase in the number of agents (controllers). In this paper, we model a relatively large traffic network as a multi-agent system and use techniques from multi-agent reinforcement learning. In particular, Q-learning is employed, where the average queue length in approaching links is used to estimate states. A parametric representation of the action space has made the method extendable to different types of intersection. The simulation results demonstrate that the proposed Q-learning outperformed the fixed time method under different traffic demands.