Multi-node environment strategy for Parallel Deterministic Multi-Objective Fractal Decomposition
MMulti-node environment strategy for ParallelDeterministic Multi-Objective FractalDecomposition
L´eo Souquet, Amir NakibUniversit´e Paris-Est, Laboratoire LISSI,122 Rue Paul Armangot, 94400Vitry sur Seine, FranceAugust 2019
Abstract
This paper presents a new implementation of deterministic multiob-jective (MO) optimization called ”Multiobjective Fractal DecompositionAlgorithm” (Mo-FDA). The original algorithm was designed for mono-objective large-scale continuous optimization problems. It is based on a”divide-and-conquer” strategy and a geometric fractal decomposition ofthe search space using hyperspheres. Then, to deal with MO problemsa scalarization approach is used. In this work, a new approach has beendeveloped on a multi-node environment using containers. The perfor-mance of Mo-FDA was compared to state of the art algorithms from theliterature on classical benchmark of multi-objective optimization.
In multiobjective optimization problems (MOP) the goal is to optimize at leasttwo objective functions. This paper deals with these problems by using a newdecomposition-based algorithm called: ”Fractal geometric decomposition basealgorithm” (FDA). It is a deterministic metaheuristic developed to solve large-scale continuous optimization problems [5]. It can be noticed, that we call largescale problems those having the dimension greater than 1000. In this research,we are interested in using FDA to deal with MOPs because in the literaturedecomposition based algorithms have been with more less success applied tosolve these problems, their main problem is related to their complexity. In thiswork, the goal is to deal with this complexity problem by keeping the samelevel of efficiency. FDA is based on ”divide-and-conquer” paradigm where thesub-regions are hyperspheres rather than hypercubes on classical approaches. Inorder to identify the Pareto optimal solutions, we propose to extend FDA usingthe scalarization approach. We called the proposed algorithm Mo-FDA. This1 a r X i v : . [ c s . D C ] A ug ew approach has been developed to benefit from a multi-node environment toimprove the computational time taken to solved MOPs problems. This chosenarchitecture takes profit from containers, light weight virtual machines thatare design to run a specific task only. A single machine can host many morecontainers than regular virtual instances.The rest of the of paper is organized as follow. The next Section 2 presentsa description of the proposed algorithm. Section 3 presents the chosen archi-tecture. In Section 4 the obtained results and a comparison to the competingmethods are presented. Finally, Section 5 presents the future work. Fractal Decomposition Algorithm uses a ”Divide-and-Conquer” strategy acrossthe search space to find the global optimum, when it exists, or the best localoptimum. The hypercubes are the most used forms in the literature. However,this geometrical form is not adapted to solve large scale problems or high dimen-sional problems because the number of vertices increases exponentially. FDA [5]uses hyperspheres to divide the search domain as this geometrical object scaleswell as the dimension increases allowing FDA to solve large scale problems. Inaddition, the fractal aspect of FDA is a reference to the fact that the searchdomain is decomposed using the same pattern at each level until the maximumfractal depth k (fixed by the user). While searching for the optimum, FDAuses 3 phases: initialization phase; 1) exploration phase, 2) and exploitationphase 3). During the initialization phase, at level 0, the current hypersphere isdecomposed into 2 × D sub-hyperspheres with D being the problem’s dimension.Once the initialization phase is completed, FDA starts the exploration phaseto identify the sub-hypersphere that potentially contains the global optimum orbest local optimum (or global optimum if it is known), to decompose it again in2 × D sub-hyperspheres. This operation is repeated until the maximum fractaldepth, k is reached. k has been experimentally determined and set to 5. Whenthe k − level is reached, FDA enters in the exploitation phase. The aim of thisphase is to find the best local optimum inside the current sub-hyperspheres.This procedure is called Intensification local search (ILS) . Each instance of ILSstarts at the center of the sub-hypersphere being exploited. This local searchis moving along each dimension, evaluating three points on each one and onlythe best is considered for the following dimension (more details are in [5]).Then, the second k-level sub-hypersphere is exploited using ILS. This processwill stop when the maximum number of evaluations is reached. If all k-levelsub-hypersphere have been exploited without FDA stopping, it backtracks todecompose the second best hypersphere at the level k − M inimize max i =1 ,...,k [ ω i ( f i ( x ) − z ∗ i )]Subject to x ∈ X (1)with k the number of objective functions to optimise and z ∗ i the optimumof function f i . In addition, the sum of weights ω i must be equal to 1. InMo-FDA, n different mono-objective problems are solved with different com-bination of weights ω i . As each combination produces one solution, Mo-FDAproduces a Pareto-Front (PF) composed of n points. To improve the speed atwhich Mo-FDA solves a multi-objective problem, the n independent instancesare launched simultaneously using containers. The overhead produced by thedifferent containers’ management can be neglected compared to the benefit fromthe parallel implementation of the n instances. Furthermore, one can see thatthe algorithm is running on a multi-node environment without having to changethe implementation. Using this technique, to obtain n points in the Pareto-Front(PF), the algorithmwill be launched n times with n variation of the weights ω as showed in equa-tion 1 leading a significant increase in computational time. Then, the idea toovercome this problem is to design a multi-node architecture.The goal is to have each node finding one point corresponding to one com-bination of the weights ω and combine all their results to build the full PF.The challenge behind this architecture is that the computing resources neededincrease with the size of the Pareto-Front. For instance, if n = 100 points, itmeans that 100 nodes would be required, hence 100 different computers (or vir-tual machines), which can be seen as an oversized architecture. To tackle thisimportant issue, we proposed a strategy based on using containers . They aresignificantly lighter than virtual machines as they all share the same operatingsystem kernel. This way, a single machine can host more containers than vir-tual machines. This architecture significantly less complex and allows to benefitfrom multi-node approaches while developing it on a limited number of hosts.In parallel to this multiple containers running on a single machine approach,a multi-threaded architecture was also studied. However, the threads are partof the same main process was not compatible with the desired architecture tohave n independent instances of the algorithm. In addition containers can bedeployed on multiple different physical machines seamlessly, without having tochange the structure of our algorithm. This cannot be achieved if Mo-FDA wasdeveloped using multi-threads. 3igure 1: Pareto-Front of DTLZ1 (on the left) and of ZDT3 (on the right) In order to test the performance of Mo-FDA a set of 8 functions, 5 from theZDT family problems [2] and 3 from the DTLZ sets [9] have been used. Theresults obtained were compared to the well known algorithms NSGA-II, NSGA-III and MEOA/D as well as state-of-the-art approaches GWASFGA [8], andCDG [1]. To conduct the different experiments, jMetal 5.0 [6], a popular Java-based framework in the literature has been used. The principal experimentssettings described in [3].In the context of multi-objective problems, many metrics can be used asdiscussed in [7]. In order to have a better overview of the performances of Mo-FDA compared to other algorithms, we have selected four different metrics. Thefirst one, the Hypervolume metric. It measures the size of the portion of theobjective space that is dominated by an approximation set. The GenerationalDistance metric (GD) computes the average distance from a set of solutionsobtained by an algorithm to the true Pareto-Front. The Inverted generationaldistance (IGD), measures both convergence and diversity by computing thedistance from each point known in the true Pareto-Front to each point of a setof solutions found by the executed algorithm. The Spread metric measures theextent of the spread achieved among the obtained solutions. It is important tonotice that the goal is to maximize the first metric and to minimize the others.Moreover, to compare the results obtained by the different algorithms weused the Friedman Rank sum method and the obtained results are presented inTable 1. One can see that Mo-FDA is the most efficient algorithm among threemetrics out of four. Looking at the other algorithms, MEOAD/D is efficient onthe IGD and Spread but not on the GD and the Hypervolume. Consequently,the importance of using multiple criteria highlights strengths and weakness ofeach algorithm. Figure 1 shows the Pareto-Front found by Mo-FDA and thebest algorithms on two functions, DTLZ1 and ZDT3. Functions where Mo-FDAperforms the best and the worst respectively.4 lgorithms Mo-FDA NSGA-II NSGA-III MEOA/D GWASGFA CDG
Hypervolume 1.875 (1) 2.25 (2) 5.625 (6) 2.625 (3) 4.125 (4) 4.25 (5)GD 2.25 (1) 2.875 (2) 4.875 (6) 3.25 (4) 3.125 (3) 4.625 (5)IGD 2.125 (2) 2.25 (3) 5.5 (6) 2 (1) 4.25 (4) 4.875 (5)Spread 1.75 (1) 3.5 (3) 4.25 (4) 1.75 (1) 4.25 (4) 5.5 (6)Table 1: Ranking using Friedman Rank sum (and their global rank) of allalgorithms on the different metrics for all tested functions.In addition to the precision of the algorithm we have also measured the newtime performance of the proposed architecture. Over the 8 different functionstested, on average, the computation time required to solve one function at di-mension with our architecture
D=30 is 0.8 seconds on a single host. Usingtwo separate hosts, the time was reduced to 0.5 seconds. This shows that ourarchitecture is scalable and flexible without having to change the structure ofthe algorithm nor the implementation itself. All experimentations have beenconducted using the following characteristics: Mo-FDA has been developed inPython and the nodes have a processor Intel Xeon Platinum 8000 with 144GBof RAM.
In conclusion, Mo-FDA was tested on 8 different functions and compared to5 other well regarded and state-of-the-art metaheuristics. Its performances tofind good Pareto-front is proved using four popular metrics in the literature. Inaddition, the multi-node architecture using containers improves the flexibilityand scalability of the approach while reducing the computing resources needed.For future work, we aim to adapt Mo-FDA to many-objectives problems andapply it to a real world problems.
References [1] X. Cai, Z. Mei, Z. Fan, and Q. Zhang. A constrained decomposition approachwith grids for evolutionary multiobjective optimization.
IEEE Transactionson Evolutionary Computation , 22(4):564–577, 2018.[2] K. Deb, L. Thiele, M. Laumanns, and E. Zitzler.
Scalable Test Problems forEvolutionary Multiobjective Optimization , pages 105–145. Springer London,London, 2005.[3] S. Jiang, J. Zhang, and Y. Ong. Multiobjective optimization based on rep-utation.
Information Sciences , 286:125 – 146, 2014.54] K. Miettinen, F. Ruiz, and A. P. Wierzbicki.
Introduction to MultiobjectiveOptimization: Interactive Approaches , pages 27–57. Springer Berlin Heidel-berg, Berlin, Heidelberg, 2008.[5] A Nakib, S Ouchraa, N Shvai, L Souquet, and EG Talbi. Deterministicmetaheuristic based on fractal decomposition for large-scale optimization.
Applied Soft Computing , 61:468 – 485, 2017.[6] A. Nebro, J. Durillo, and M. Vergne. Redesigning the jmetal multi-objectiveoptimization framework. In
Proceedings of the Companion Publication ofthe 2015 Annual Conference on Genetic and Evolutionary Computation ,GECCO Companion ’15, pages 1093–1100, New York, NY, USA, 2015.ACM.[7] N. Riquelme, C. Von L¨ucken, and B. Baran. Performance metrics in multi-objective optimization. In , pages 1–11, Oct 2015.[8] R. Saborido, A.B. Ruiz, and M. Luque. Global WASF-GA: An evolutionaryalgorithm in multiobjective optimization to approximate the whole paretooptimal front.
Evolutionary Computation , 25(2):309–349, 2017.[9] E. Zitzler, K. Deb, and L. Thiele. Comparison of multiobjective evolutionaryalgorithms: Empirical results.