Nassima Aissani

Dynamic scheduling of maintenance tasks in the petroleum industry: A reinforcement approach

Petroleum industry production systems are highly automatized. Maintenance of such systems is vital, not only to maintain production efficiency but also to insure minimal safety levels. Maintenance task scheduling is difficult since some tasks are already identified because they must be done repeatedly, and other tasks need to be identified dynamically. In this paper, we present a multi-agent approach for the dynamic maintenance task scheduling for a petroleum industry production system. Agents simultaneously insure effective maintenance scheduling and the continuous improvement of the solution quality by means of reinforcement learning, using the SARSA algorithm. Reinforcement learning allows the agents to adapt, learning the best behaviors for their various roles without reducing the performance or reactivity. To demonstrate the innovation of our approach, we include a computer simulation of our model and the results of experimentation applying our model to an Algerian petroleum refinery.

The control of myopic behavior in semi-heterarchical production systems: A holonic framework

Heterarchical control architectures are essentially founded on cooperation and full local autonomy, resulting in high reactivity, no master/slave relationships and local information retention. Consequently, these architectures experience myopic decision-making, bringing entities towards local optimality rather than the systems overall performance. Although this issue has been identified as an important problem within heterarchical control architectures, it has not been formally studied. The aim of this paper is to identify the dimensions of myopic behavior and propose mechanisms to control this behavior. This study focuses on myopic behavior found in manufacturing control. For this particular study, we propose a holonic framework and a holonic organization that integrates specific mechanisms to control the temporal and social myopia. Our proposal was validated using simulations designed for solving the allocation problem in flexible manufacturing systems. These simulations were conducted to show the improvement by integrating the new mechanisms. These simulation results indicate that the myopic control mechanisms achieve better performance than the reactive strategies, because not only they introduce a planning horizon, but also because they balance local and global objectives, seeking a consensus.

Dynamic scheduling for multi-site companies: a decisional approach based on reinforcement multi-agent learning

In recent years, most companies have resorted to multi-site or supply-chain organization in order to improve their competitiveness and adapt to existing real conditions. In this article, a model for adaptive scheduling in multi-site companies is proposed. To do this, a multi-agent approach is adopted in which intelligent agents have reactive learning capabilities based on reinforcement learning. This reactive learning technique allows the agents to make accurate short-term decisions and to adapt these decisions to environmental fluctuations. The proposed model is implemented on a 3-tier architecture that ensures the security of the data exchanged between the various company sites. The proposed approach is compared to a genetic algorithm and a mixed integer linear program algorithm to prove its feasibility and especially, its reactivity. Experimentations on a real case study demonstrate the applicability and the effectiveness of the model in terms of both optimality and reactivity.

Scheduling under uncertainty: Survey and research directions

In real-world scheduling problems, several kinds of hard-to-predict risk must be considered. Scheduling under uncertainty allows these kinds of risks to be taken into account. This paper provides an overview of the state of the art in scheduling under uncertainty, including a survey on modeling techniques of uncertainty and a survey of the existing positioning typologies and contributions. A new classification scheme for the different approaches to scheduling under uncertainty is proposed and discussed. Several areas for future research are suggested.

An approach for temporal myopia reduction in Heterarchical Control Architectures

This paper presents a model that aims at diminishing the impact of temporal myopia in Heterarchical Control Architectures (HeCA) of Flexible Manufacturing System (FMS). This model is based on a “switching architecture”, based upon the concept of intelligent reactive products. A virtual layer hosts Product Agents having the objective of preserving global performance, while a physical layer congregates physical products trying to follow the prescribed behavior, unless they sense a change of conditions. The idea is to empower autonomy while achieving an overall good global performance. For experimental purposes, this approach has been deployed at the AIP-PRIMECA center at University of Valenciennes, France and some results are presented.

A Holonic Approach to Myopic Behavior Correction for the Allocation Process in Flexible-Job Shops Using Recursiveness

This chapter’s main interest is the myopic behaviour inherent to holonic control architectures. Myopic behaviour is the lack of coherence among local decision-making and system’s global goals. So far, holonic architectures use mediator entities to overcome this issue, bringing the holonic paradigms more toward hierarchy than heterarchy. Instead, this chapter explores the recursiveness characteristic of holonic manufacturing systems (HMS) as a possible way to correct myopic behaviour, by distributing decision-making over adjunct entities. The chapter explains our approach and its agent-based implementation for solving the allocation problem in a flexible job-shop. Results from simulations are compared with a mixed-integer linear program to determine its efficiency in terms of makespan and execution time. Preliminary results encourage further research in this area.

Extraction of Priority Rules for Boolean Induction in Distributed Manufacturing Control

In reactive manufacturing control, the allocation of resources for tasks is achieved in real time. When a resource becomes available it chooses one of the tasks in its queue. This choice is made according to priority rules which are designed to optimize costs, time, etc. In this paper, the aim is to exploit a Job Shop scheduling log and simulations in order to extract knowledge enabling one to create rules for the selection of priority rules. These rules are implemented in a CASI cellular automaton. Firstly, symbolic modelling of the scheduling process is exploited to generate a decision tree from the log and simulations. Secondly, decision rules are extracted to select priority rules for execution in a specific situation. Finally, the rules are integrated in CASI which implements the decisional module of agents in a distributed manufacturing control system.

Multiple priority dispatching rules for the job shop scheduling problem

In this paper we focus on the Job Shop Scheduling Problem (JSSP) using Priority Dispatching Rules. Simulation model for makespan optimization is proposed using different Dispatching Rules (DR) for each machine in the shop floor. Collected results are used for learning base construction. This database will be used to develop an inference model able to select the best DR for every new scheduling problem. This preliminary study shows advantages of using different DR and also saving progress JSSP data.

Multi-agent reinforcement learning for adaptive scheduling: application to multi-site company

Abstract In recent years, most companies have resorted to multi-site organization in an effort to improve their competitiveness and to adapt to current conditions. In this article, we propose a model for adaptive scheduling in multi-site companies. We adopt a multi-agent approach in which intelligent agents have reactive learning ability. This allows them to make accurate short-term decisions. Our model is implemented on a 3-tier architecture that ensures the security of the data exchanged between the various company sites. Experimentations on a real case study demonstrate the applicability and the effectiveness of our model concerning both optimality and reactivity.

A Model for Manufacturing Scheduling Optimization Through Learning Intelligent Products

The needs of flexibility, agility and adaptation capabilities for modern manufacturing systems increase constantly. In this paper, we propose an original approach combining active/intelligent product architecture with learning mechanism to assure flexibility and agility to the overall manufacturing system. Using learning approaches as Reinforcement Learning (RL) mechanism, an active product can be able to reuse learned experiences to enhance its decisional performances. A contextualization method is proposed to improve the decision making of the product for scheduling tasks. The approach is then applied to a case study using a multi-agent simulation platform.