Brahim Chaib-draa

Online planning algorithms for POMDPs

Partially Observable Markov Decision Processes (POMDPs) provide a rich framework for sequential decision-making under uncertainty in stochastic domains. However, solving a POMDP is often intractable except for small problems due to their complexity. Here, we focus on online approaches that alleviate the computational complexity by computing good local policies at each decision step during the execution. Online algorithms generally consist of a lookahead search to find the best action to execute at each time step in an environment. Our objectives here are to survey the various existing online POMDP methods, analyze their properties and discuss their advantages and disadvantages; and to thoroughly evaluate these online approaches in different environments under various metrics (return, error bound reduction, lower bound improvement). Our experimental results indicate that state-of-the-art online heuristic search methods can handle large POMDP domains efficiently.

TRENDS IN AGENT COMMUNICATION LANGUAGE

Agent technology is an exciting and important new way to create complex software systems. Agents blend many of the traditional properties of AI programs—knowledge–level reasoning, flexibility, proactiveness, goal–directedness, and so forth—with insights gained from distributed software engineering, machine learning, negotiation and teamwork theory, and the social sciences. An important part of the agent approach is the principle that agents (like humans) can function more effectively in groups that are characterized by cooperation and division of labor. Agent programs are designed to autonomously collaborate with each other in order to satisfy both their internal goals and the shared external demands generated by virtue of their participation in agent societies. This type of collaboration depends on a sophisticated system of inter–agent communication. The assumption that inter–agent communication is best handled through the explicit use of an agent communication language (ACL) underlies each of the articles in this special issue. In this introductory article, we will supply a brief background and introduction to the main topics in agent communication.

Trends in distributed artificial intelligence

Distributed artificial intelligence (DAI) is a subfield of artificial intelligence that deals with interactions of intelligent agents. Precisely, DAI attempts to construct intelligent agents that make decisions that allow them to achieve their goals in a world populated by other intelligent agents with their own goals. This paper discusses major concepts used in DAI today. To do this, a taxonomy of DAI is presented, based on the social abilities of an individual agent, the organization of agents, and the dynamics of this organization through time. Social abilities are characterized by the reasoning about other agents and the assessment of a distributed situation. Organization depends on the degree of cooperation and on the paradigm of communication. Finally, the dynamics of organization is characterized by the global coherence of the group and the coordination between agents. A reasonably representative review of recent work done in DAI field is also supplied in order to provide a better appreciation of this vibrant AI field. The paper concludes with important issues in which further research in DAI is needed.

Information Sharing as a Coordination Mechanism for Reducing the Bullwhip Effect in a Supply Chain

The bullwhip effect is an amplification of the variability of the orders placed by companies in a supply chain. This variability reduces the efficiency of supply chains, since it incurs costs due to higher inventory levels and supply chain agility reduction. Eliminating the bullwhip effect is surely simple; every company just has to order following the market demand, i.e., each company should use a lot-for-lot type of ordering policy. However, many reasons, such as inventory management, lot-sizing, and market, supply, or operation uncertainties, motivate companies not to use this strategy. Therefore, the bullwhip effect cannot be totally eliminated. However, it can be reduced by information sharing, which is the form of collaboration considered in this paper. More precisely, we study how to separate demand into original demand and adjustments. We describe two principles explaining how to use the shared information to reduce the amplification of order variability induced by lead times, which we propose as a cause of the effect. Simulations confirm the value of these two principles with regard to costs and customer service levels

Commitment-based and dialogue-game-based protocols: new trends in agent communication languages

This survey introduces existing approaches to Agent Communication Languages (ACLs) and particularly Conversation Policies (CPs) which can be viewed as general constraints on the sequence of semantically coherent messages leading to a goal. Then limitations of these CPs are discussed in detail, particularly limitations on flexibility and specification. Finally, ACLs are viewed from the dialectic point of view, and some approaches are introduced in this context: some focusing on commitment-based protocols and others on dialogue-game-based protocols.

Cooperative Adaptive Cruise Control: A Reinforcement Learning Approach

Recently, improvements in sensing, communicating, and computing technologies have led to the development of driver-assistance systems (DASs). Such systems aim at helping drivers by either providing a warning to reduce crashes or doing some of the control tasks to relieve a driver from repetitive and boring tasks. Thus, for example, adaptive cruise control (ACC) aims at relieving a driver from manually adjusting his/her speed to maintain a constant speed or a safe distance from the vehicle in front of him/her. Currently, ACC can be improved through vehicle-to-vehicle communication, where the current speed and acceleration of a vehicle can be transmitted to the following vehicles by intervehicle communication. This way, vehicle-to-vehicle communication with ACC can be combined in one single system called cooperative adaptive cruise control (CACC). This paper investigates CACC by proposing a novel approach for the design of autonomous vehicle controllers based on modern machine-learning techniques. More specifically, this paper shows how a reinforcement-learning approach can be used to develop controllers for the secure longitudinal following of a front vehicle. This approach uses function approximation techniques along with gradient-descent learning algorithms as a means of directly modifying a control policy to optimize its performance. The experimental results, through simulation, show that this design approach can result in efficient behavior for CACC.

Supply Chain Management and Multiagent Systems: An Overview

This chapter introduces the topic of this book by presenting the fields of supply chain management, multiagent systems, and the merger of these two fields into multiagent-based supply chain management. More precisely, the problems encountered in supply chains and the techniques to address these problems are first presented. Multiagent systems are next broadly presented, before focusing on how agents can contribute to solving problems in supply chains.

An online POMDP algorithm for complex multiagent environments

In this paper, we present an online method for POMDPs, called RTBSS (Real-Time Belief Space Search), which is based on a look-ahead search to find the best action to execute at each cycle in an environment. We thus avoid the overwhelming complexity of computing a policy for each possible situation. By doing so, we show that this method is particularly efficient for large real-time environments where offline approaches are not applicable because of their complexity. We first describe the formalism of our online method, followed by some results on standard POMDPs. Then, we present an adaptation of our method for a complex multiagent environment and results showing its efficiency in such environments.

Multiagent based Supply Chain Management

This book takes a close look at recent progress in the field of supply chain management using agent technology and more specifically multiagent systems. Sixteen chapters are organized in four main parts: Introductory Papers; Multiagent Based Supply Chain Modeling; Collaboration and Coordination Between Agents in a Supply Chain; and Multiagent Based Supply Chain Management: Applications. The result is a comprehensive review of existing literature, and ideas for future research.

Industrial applications of distributed AI

Most work done in distributed artificial intelligence (DAI) had targeted sensory networks, including air traffic control, urban traffic control, and robotic systems. The main reason is that these applications necessitate distributed interpretation and distributed planning by means of intelligent sensors. Planning includes not only the activities to be undertaken, but also the use of material and cognitive resources to accomplish interpretation tasks and planning tasks. These application areas are also characterized by a natural distribution of sensors and receivers in space. In other words, the sensory data-interpretation tasks and action planning are inter-dependent in time and space. For example, in air traffic control, a plan for guiding an aircraft must be coordinated with the plans of other nearby aircraft to avoid collisions.