Tony Wauters

Learning agents for the multi-mode project scheduling problem

Intelligent optimization refers to the promising technique of integrating learning mechanisms into (meta-)heuristic search. In this paper, we use multi-agent reinforcement learning for building high-quality solutions for the multi-mode resource-constrained project scheduling problem (MRCPSP). We use a network of distributed reinforcement learning agents that cooperate to jointly learn a well-performing constructive heuristic. Each agent, being responsible for one activity, uses two simple learning devices, called learning automata, that learn to select a successor activity order and a mode, respectively. By coupling the reward signals for both learning tasks, we can clearly show the advantage of using reinforcement learning in search. We present some comparative results, to show that our method can compete with the best performing algorithms for the MRCPSP, yet using only simple learning schemes without the burden of complex fine-tuning.

The Multi-Mode Resource-Constrained Multi-Project Scheduling Problem

Scheduling projects is a difficult and time consuming process, and has far-reaching implications for any organization’s operations. By generalizing various aspects of project scheduling, decision makers are enabled to capture reality and act accordingly. In the context of the MISTA 2013 conference, the first MISTA challenge, organized by the authors, introduced such a general problem model: the Multi-Mode Resource-Constrained Multi-Project Scheduling Problem (MRCMPSP). The present paper reports on the competition and provides a discussion on its results. Furthermore, it provides an analysis of the submitted algorithms, and a study of their common elements. By making all benchmark datasets and results publicly available, further research on the MRCMPSP is stimulated.

Real-world production scheduling for the food industry: An integrated approach

The present paper offers an integrated approach to real-world production scheduling for the food processing industries. A manufacturing execution system is very appropriate to monitor and control the activities on the shop floor. Therefore, a specialized scheduler, which is the focus of this paper, has been developed to run at the core of such a system. The scheduler builds on the very general Resource Constrained Project Scheduling Problem with Generalized Precedence Relations. Each local decision step (e.g. choosing a route in the plant layout) is modeled as a separate module interconnected in a feedback loop. The quality of the generated schedules will guide the overall search process to continuously improve the decisions at an intermediate level by using local search strategies. Besides optimization methods, data mining techniques are applied to historical data in order to feed the scheduling process with realistic background knowledge on key performance indicators, such as processing times, setup times, breakdowns, etc. The approach leads to substantial speed and quality improvements of the scheduling process compared to the manual practice common in production companies. Moreover, our modular approach allows for further extending or improving modules separately, without interfering with other modules.

The Concrete Delivery Problem

Abstract From an operational point of view, Ready-Mixed Concrete Suppliers are faced with challenging operational problems such as the acquisition of raw materials, scheduling of production facilities, and the transportation of concrete. This paper is centered around the logistical and distributional part of the operation: the scheduling and routing of concrete, commonly known as the Concrete Delivery Problem (CDP). The problem aims at finding efficient routes for a fleet of (heterogeneous) vehicles, alternating between concrete production centers and construction sites, and adhering to strict scheduling and routing constraints. Thus far, a variety of CDPs and solution approaches have appeared in academic research. However, variations in problem definitions and the lack of publicly available benchmark data inhibit a mutual comparison of these approaches. Therefore, this work presents a more fundamental version of CDP, while preserving the main characteristics of the existing problem variations. Both exact and heuristic algorithms for CDP are proposed. The exact solution approaches include a Mixed Integer Programming (MIP) model and a Constraint Programming model. Similarly, two heuristics are studied: the first heuristic relies on an efficient best-fit scheduling procedure, whereas the second heuristic utilizes the MIP model to improve delivery schedules locally. Computational experiments are conducted on new, publicly accessible, data sets; results are compared against lower bounds on the optimal solutions.

Decomposition and local search based methods for the traveling umpire problem

The Traveling Umpire Problem (TUP) is a challenging combinatorial optimization problem based on scheduling umpires for Major League Baseball. The TUP aims at assigning umpire crews to the games of a fixed tournament, minimizing the travel distance of the umpires. The present paper introduces two complementary heuristic solution approaches for the TUP. A new method called enhanced iterative deepening search with leaf node improvements (IDLI) generates schedules in several stages by subsequently considering parts of the problem. The second approach is a custom iterated local search algorithm (ILS) with a step counting hill climbing acceptance criterion. IDLI generates new best solutions for many small and medium sized benchmark instances. ILS produces significant improvements for the largest benchmark instances. In addition, the article introduces a new decomposition methodology for generating lower bounds, which improves all known lower bounds for the benchmark instances.

A learning-based optimization approach to multi-project scheduling

The present paper introduces a learning-based optimization approach to the resource-constrained multi-project scheduling problem. Multiple projects, each with their own set of activities, need to be scheduled. The objectives dealt with here include minimization of the average project delay and total makespan. The availability of local and global resources, precedence relations between activities, and non-equal project start times have to be considered. The proposed approach relies on a simple sequence learning game played by a group of project managers. The project managers each learn their activity list locally using reinforcement learning, more specifically learning automata. Meanwhile, they learn how to choose a suitable place in the overall sequence of all activity lists. All the projects need to arrive at a unique place in this sequence. A mediator, who usually has to solve a complex optimization problem, now manages a simple dispersion game. Through the mediator, a sequence of feasible activity lists is eventually scheduled by using a serial schedule generation scheme, which is adopted from single project scheduling. It is shown that the sequence learning approach has a large positive effect on minimizing the average project delay. In fact, the combination of local reinforcement learning, the sequence learning game and a forward-backward implementation of the serial scheduler significantly improves the best known results for all the MPSPLIB datasets. In addition, several new best results were obtained for both considered objectives: minimizing the average project delay and minimizing the total makespan.

Branch-and-bound with decomposition-based lower bounds for the Traveling Umpire Problem

The Traveling Umpire Problem (TUP) is an optimization problem in which umpires have to be assigned to games in a double round robin tournament. The objective is to obtain a solution with minimum total travel distance over all umpires, while respecting hard constraints on assignments and sequences. Up till now, no general nor dedicated algorithm was able to solve all instances with 12 and 14 teams. We present a novel branch-and-bound approach to the TUP, in which a decomposition scheme coupled with an efficient propagation technique produces the lower bounds. The algorithm is able to generate optimal solutions for all the 12- and 14-team instances as well as for 11 of the 16-team instances. In addition to the new optimal solutions, some new best solutions are presented and other instances have been proven infeasible.

An effective shaking procedure for 2D and 3D strip packing problems

Strip packing intends to minimise the height required for placing a set of rectangular items into a strip with infinite height. The present paper introduces a fast, yet effective shaking algorithm for the two- and three-dimensional strip packing problems, which are both NP-hard. The proposed heuristic procedure starts from an ordered item list, from which it alternates between forward and backward construction phases. The algorithm builds upon the common (deepest) bottom-left-fill algorithm, but shows significantly better results. Improvements on the solution quality of more than 9% can be observed. Moreover, applying the shaking procedure as a post processing algorithm to existing high performance heuristics also leads to improvements.

A two-dimensional heuristic decomposition approach to a three-dimensional multiple container loading problem

Efficient container loading has the potential to considerably reduce logistics and transportation costs. The container loading problem is computationally complex and, despite extensive academic effort, there remains room for algorithm improvement. Real-world problems are not always addressed satisfactorily primarily due to the large number of complex constraints and objectives. The problem addressed by this work is an unsolved multiple container loading problem introduced by Renault on the occasion of the 2014/2015 ESICUP challenge, organized by the EURO Special Interest Group on Cutting and Packing (ESICUP). Renault’s problem requires a large number of different items to be packed into containers of different types and sizes. Items must be grouped into stacks and respect the ‘this side up’ constraint. The primary objective is to minimize the volume of shipped containers. The smallest volume container may be left behind for the next shipment and is excluded from the main objective. Nevertheless, only a limited percentage of each product may be packed into this container. Additionally, a set of secondary objectives is considered. This work presents a decomposition approach embedded in a multi-phase heuristic for the problem. Feasible solutions are generated quickly, and subsequently improved by local search and post-processing procedures. Experiments revealed that the approach generates optimal solutions for two instances, in addition to good quality solutions for those remaining from the Renault set. The algorithmic contribution is, however, not restricted to the Renault instances. Experiments on less constrained container loading instances from the literature demonstrate the approach’s general applicability and competitiveness.

A Guide-and-Observe Hyper-Heuristic Approach to the Eternity II Puzzle

The present paper considers the optimisation version of the Eternity II puzzle problem and unsigned edge matching puzzles in general. The goal of this optimisation problem is to maximise the number of matching edges in the puzzle. In 2010, the META Eternity II contest awarded the best performing metaheuristic approach to this hard combinatorial optimisation problem. The winning hyper-heuristic of the contest is subject of this paper. Heuristic design decisions are motivated based on the results of extensive experiments. Furthermore, new results for the Eternity II puzzle problem are presented. The main contribution of this paper is the description of a novel guide-and-observe search mechanism combining a set of objectives. The approach significantly outperforms search methods guided by the default objective only.