AMPSO: Artificial Multi-Swarm Particle Swarm Optimization
AMPSO: Artificial Multi-Swarm Particle Swarm Optimization
Haohao Zhou , Zhi-Hui Zhan , Zhi-Xin Yang , Xiangzhi Wei Institute of Intelligent Manufacturing and Information Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China Guangdong Provincial Key Laboratory of Computational Intelligence and Cyberspace Information, School of Computer Science and Engineering, South China University of Technology, Guangzhou, 510006, PR China State Key Laboratory of Internet of Things for Smart City and Department of Electromechanical Engineering, University of Macau, Macao, China *Corresponding author Email: [email protected]
Abstract:
In this paper we propose a novel artificial multi-swarm PSO which consists of an exploration swarm, an artificial exploitation swarm and an artificial convergence swarm. The exploration swarm is a set of equal-sized sub-swarms randomly distributed around the particles space, the exploitation swarm is artificially generated from a perturbation of the best particle of exploration swarm for a fixed period of iterations, and the convergence swarm is artificially generated from a Gaussian perturbation of the best particle in the exploitation swarm as it is stagnated. The exploration and exploitation operations are alternatively carried out until the evolution rate of the exploitation is smaller than a threshold or the maximum number of iterations is reached. An adaptive inertia weight strategy is applied to different swarms to guarantee their performances of exploration and exploitation. To guarantee the accuracy of the results, a novel diversity scheme based on the positions and fitness values of the particles is proposed to control the exploration, exploitation and convergence processes of the swarms. To mitigate the inefficiency issue due to the use of diversity, two swarm update techniques are proposed to get rid of lousy particles such that nice results can be achieved within a fixed number of iterations. The effectiveness of AMPSO is validated on all the functions in the CEC2015 test suite, by comparing with a set of comprehensive set of 16 algorithms, including the most recently well-performing PSO variants and some other non-PSO optimization algorithms. Keywords: Swarm Intelligence, Particle Swarm Optimization, Multi-swarm
1. Introduction
The PSO algorithm was first proposed by Kennedy and Eberhart [1, 2] for solving optimization problems in 1995. This algorithm is motivated by the emergent motion of the foraging behavior of a flock of birds or a school of fish, where a member learns to move based on its own experience and the experiences of the others. PSO is simple in implementation and has nice convergence properties when compared to other volutionary algorithms [3, 4], which makes PSO one of the most prominent population-based optimization techniques. Nowadays PSO has been successfully extended to many application areas such as communication networks [5, 6], medicine engineering [7], scheduling of tasks in cloud computing [8, 9], airport gate assignment [10] , energy management [11], linguistics studies [12], and supply chain management [13].
Although PSO is considered as a robust algorithm with a fast convergence rate, it suffers from the premature convergence problem: PSO can be easily trapped into some local optima when solving multimodal problems [14, 15, 16, 17, 18, 19, 20, 21]. Therefore, a nice balance between exploration (global investigation of the search place) and exploitation (finer search around a local optimum) is critical to the success of PSO, especially for complex multimodal problems with a large number of local minima. In order to address this issue, a number of PSO variants have been developed. In the following we shall limit our survey to three major types: PSO with adjusted parameters, PSO with various topologies and PSO with hybrid strategies.
PSO with Adjusted Parameters:
Adjusting w , c and c may also change the exploration and exploitation abilities of the PSO. For example, a larger w facilitates the exploration while a smaller one encourages the exploitation. Zhan [14] proposed an adaptive PSO, which adjusted w , c and c based on an estimation of the evolutionary state that relied on the distribution of population and the fitness of particles. Zhang et al. [22] proposed a PSO variant using the Bayesian technique to adaptively change the inertia weight based on the previous positions of the particles such that the exploitation capability of PSO was enhanced. Tanweer et al. [23] proposed a PSO variant with the best particle conducting a self-regulation of its inertia weight to accelerate the searching towards the global optimum. Based on the fact that changing the inertia weight adaptively will change the stability conditions adaptively, Taherkhani et al. [24] proposed a PSO variant with stability-based adaptive inertia weight.
PSO with Various Topologies: I t has been shown that different topologies of the particles significantly influenced the exploration and exploitation performances of PSO [25, 26, 27]. Kennedy [28] proposed a small-world social network and investigated the impact of different topologies of the particles (e.g., circles, wheels, stars and randomly-assigned edges) on the performances of PSO algorithms, and showed that sparse connected networks were suitable for addressing complex functions while dense connected network worked well for simple functions. In 1999, Suganthan [ ] first introduced the concept of dynamic topologies for PSO, where a particle was initially a subswarm by itself and the subswarms merged as the iterations went on, and eventually became a single fully-connected topology. Cooren et al. [27] proposed a particle swarm system called TRIBES, in which the topology evolved over time: the swarm was divided into sub-swarms each had its own structure, good tribes were allowed to get rid of their weakest members and bad tribes were allowed to add in a random number of new members to enhance their success rate. Bonyadi et al. [30] proposed time-varying topologies by growing the sizes of th e subswarms, merging the subswarms, and changing the topologies (e.g., global best topology, ring topology and non-overlap topology) of the subswarms as their sizes ncreased.
PSO with Hybrid Strategies:
PSO has been combined with many other techniques for a better efficiency and accuracy, where the combined techniques are usually used to strengthen the local search of the computation process. For example, Mirjalili et al. [31] hybridized
PSO with gravitational search algorithm for training feedforward neural networks.
Zhang et al. [32] comb ined PSO with a back-propagation algorithm to train the weights of feedforward neural networks . Nagra et al. [33] developed a new hybrid of dynamic multi-swarm PSO with a gravitational search algorithm for improving the exploration and exploitation performance of PSO. Zhan et al. [34] combined PSO with orthogonal experimental d esign to discover the best combination of a particleβs best historical position and its neighborhoodβs best historical position such that an excellent exemplar is constructed for the swarm to learn faster. Bonyadi et al. [30] hybridized PSO with a covariance matrix adaptation evolutionary strategy to improve the solutions at the latter phases of the evolution process. Garg [35] combined PSO with genetic algorithms (GA) such that the local search capability of GA can help speed up the convergence of PSO: for the best individual of the population after certain number of iterations, [35] created a new population by replacing weak points in the current population with excellent points via selection, crossover and mutation operators. Liu et al. [36] also combined PSO with GA to job scheduling issues. Plevris and Papadrakakis [37] combined PSO with a gradient-based quasi-Newton SQP algorithm for the optimization of engineering structures. N. Singh and S.B. Singh [38] combined PSO with Grey Wolf Optimizer for improving the convergence rate of the iterations. Raju et al. [39] combined PSO with a bacterial foraging optimization for FDM 3D printing parameters of complicated structures. Visalakshi and Sivanandam [40] combined PSO and the simulated annealing algorithm for processing dynamic task scheduling with load balance.
For interested readers, a comprehensive survey for hybrid PSO can be available in [41], and a recent survey of PSO with a broader range of techniques is available in [42].
The major innovation of this study is as follows : We proposed an artificial multi-swarm PSO (AMPSO) to improve the exploration and exploitation performances of PSO. The multi-swarm consists of an exploration swarm with a set of equal-sized sub-swarms, an exploitation swarm artificially generated around the neighborhood of the best particles produced by the exploration swarm for a fixed period of iterations, and finally a convergence swarm artificially generated around the neighborhood of the best particles produced by the exploitation swarm. Here by βartificialβ it means that we generate a swarm as we need it instead of mimicking the natural evolutionary process of any social animal. A novel diversity scheme based on the positions and the fitness values of the swarms is proposed, and two update techniques are combined with the diversity scheme to guarantee nice results by using a fixed number of iterations. We validate the effectiveness of AMPSO by comparing it with two recent PSO variants (as well as their corresponding sets of comparison algorithms) on the same set of CEC2015 benchmark functions. The rest of the paper is presented as follows: Section 2 briefly reviews the PSO variants that are related to AMPSO; Section 3 presents our techniques of AMPSO; ection 4 presents the comparison experiments with other PSO variants using the CEC2015 test suite; Section 5 concludes the paper with some discussions.
2. Related work 2.1. Standard PSO
The original PSO did not have an inertia weight [1, 2].
Since the inertia weight was introduced by Shi and Eberhart [43] in 1998, it has shown its power in better control of the exploration and exploitation processes. π£ ππ (π‘ + 1) = ππ£ ππ (π‘) + π π (π ππ (π‘) β π₯ ππ (π‘)) + π π (π ππ (π‘) β π₯ ππ (π‘)) (1) π₯ ππ (π‘ + 1) = π₯ ππ (π‘) + π£ ππ (π‘ + 1) (2) When searching in a D -dimensional hyperspace, each particle i has a velocity vector π£ π (π‘) = [π£ π1 (π‘), π£ π2 (π‘), β― , π£ ππ· (π‘)] and a position vector π₯ π (π‘) =[π₯ π1 (π‘), π₯ π2 (π‘), β― , π₯ ππ· (π‘)] for the t -th iteration. π π is the historically best position of the particle i , and π π is the global best particle of the swarm. Acceleration coefficients π and π are commonly set in the range [0.5, 2.5] (e.g., [
44, 45 ]). π and π are two randomly generated values within range [0, 1], Ο typically decreases linearly from 0.9 to 0.4 [43] . Although competition among individual particles usually improves the computation performance, much greater improvements can be achieved through cooperation among multiple swarms. A large body of literature has been devoted to the cooperation among multiple warms. Due to the limit of space, in the following only a few typical multi-swarms PSO variants are reviewed. Inspired by
Potterβ s idea of splitting the solution vector into smaller vectors for improving the performance of the Genetic Algorithm [46], Bergh and Engelbrecht [47] proposed a cooperative approach for PSO by dividing a d -dimension swarm into k ( k < d ) subswarms and exchanging the information (e.g., best particle) of each subswarm. Blackwell and Branke [48] proposed a multi-swarm PSO for benchmark functions where the peaks of the functions were dynamically changing. Liangβs group [49, 50] proposed a dynamic multi-swarm system that used small-sized sub-swarms, which were randomly regrouped frequently by using various schedules and the information was exchanged among the subswarms; they also exploited the Quasi-Newton method to improve the local search performance of PSO. Xu et al. [3] proposed a dynamic multi-swarm PSO with cooperative learning strategy, where each dimension of the two worst particles learned from two better sub-swarms using tournament selection. Chen et al. [20] proposed a dynamic multi-swarm PSO with a differential learning strategy, where the exploitation capability of PSO was enhanced by combining the differential utation into PSO and employing Quasi-Newton method as a local searcher. Zhang et al. [51] proposed a self-adaptive and cooperative PSO based on four sub-swarms. The cooperative capability was guaranteed by sharing the unique global best. Zhao [52] proposed a dynamic multi-swarm particle swarm optimizer (DMS-PSO) with harmony search (HS), and merged HS into each sub-swarm to make use of exploration capability of the DMS-PSO and stochastic exploitation capability of HS. Ye et al. [53] proposed a novel multi-swarm PSO with two dynamic learning strategies of updating the velocities of the particles, of which one was turning normal particles into communication particles as the iteration number increases and the other was updating the velocities based on the local best of each subswarm; the latter was inspired by the full-informed mechanism of [26] that took the information of all the neighbors (instead of the global best particle) into consideration. Recently, Wang et al. [54] proposed a multi-swarm distributed PSO whose size of sub-swarm was adaptively control by an adaptive granularity learning strategy.
3. Artificial Multi-Swarm Particle Swarm Optimization (AMPSO)
In this section, the technical details of AMPSO are presented in the following manner: the main idea of AMPSO is presented in subsection 3.1, the diversity and parameter adjustment techniques are presented in subsection 3.2, the techniques for the generation of artificial swarms are presented in subsection 3.3, and finally the swarm reconstruction techniques are presented in subsection 3.4.
Our idea of artificial multi-swarm PSO is inspired by the following rule of the game for capturing the commander of the enemy that is widely used in modern wars:
First of all, attackers send out many small searching teams to locate the suspected regions that the commander may stay in; and then a fast-moving air force is sent to attack these regions and determines highly suspected regions; and finally, a ground force with strong fire is sent to sweep each highly suspected region to capture the commander of the enemy.
Based on this idea, we name our three swarms as
Exploration Swarm , Exploitation Swarm and
Convergence Swarm . This naming is consistent with the three evolutionary states proposed in APSO [ ]: exploration state , exploitation state and convergence state. Here the exploration swarm consists of a set of randomly distributed small-sized sub-swarms searching through the space, and each sub-swarm evolves independently (without any communication among sub-swarms) in order to explore as much searching space as possible; the exploitation swarm is generated from a random perturbation of the position of the best particle of the exploration swarm after a number of iterations. It is possible that the selected βbest particleβ is in reality a locally best particle rather than the globally best particle. To increase of chance of achieving the global best, when the exploitation swarm runs into a stagnation state the exploration swarm is reinitialized and a sequence of iterations will output a new best particle, which is then perturbed into a new exploitation swarm; meanwhile, when the exploitation swarm is in a stagnation state its best particle is perturbed into the convergence swarm. Figure 1 illustrates the rocedure. Although we exploit the concept of APSO [14] for different evolutionary states, our approach is quite different in the following aspects: instead of using one swarm, we use multiple swarms; and our swarms are artificially generated around hopeful regions to sweep the targets rapidly. In the experiment section, we shall show that our strategy is fairly promising in achieving better results when compared with the existing popular PSO variants. For ease of description, we present the definition of the parameters used in the rest of the paper in Table 1.
Table 1
Nomenclature π ππ , π ππ , π π the size of exploration swarm, the size of exploitation swarm and the size of convergence swarm, respectively π π‘ππ‘ππ , π π , π , π the total number of iterations, the total number of iterations consumed so far, the maximum number of iterations for the Exploration Swarms, and the maximum number of iterations for the Exploitation Swarm, respectively π π the number of weak particles of the exploitation swarm to be updated π‘ the current index of the iteration π· the dimension of the solution space π£ π (π‘) the velocity of particle π in the π‘ -th iteration π₯ π (π‘) the position of particle π in the π‘ -th iteration π, π , π initial weight and accelerations for PSO, respectively π , π two random real numbers in range [0, 1] πΈπ(π‘) the evolution rate of the swarm in the π‘ -th iteration ππ(π‘) the globally best fitness of the swarm in the π‘ -th iteration πΎ a user-defined constant used for computing πΈπ(π‘) πΌ , πΌ , πΌ ratios used to control π , π and π π respectively and πΌ , πΌ , πΌ β [0,1] π½ a parameter used to control the termination condition of the exploitation swarm: the condition is satisfied if πΈπ(π‘) < π½ πΈ the entropy that represents the swarmβs diversity degree πΈ π (π‘), πΈ π (π‘), πΈ π (π‘) the diversity of the d -th dimension of the swarm, the position diversity of the swarm, and the fitness diversity of the swarm in the π‘ -th iteration, respectively ππ a D -dimension vector ππ[ππ , ππ , β¦ , ππ π· ] , where ππ π is a random real number and ππ π β π(0, (0. 1) ) π ππ the number of non-progress iterations π π the probability of reconstructing the Convergence Swarm π₯ ππ , π₯ π’π the lower bound and upper bound of the search space respectively Usually the evolution rate is calculated as the difference between two consecutive iterations. However, due to the slow evolution of the particles between two consecutive iterations it is possible that the evolution rate is fairly small and cannot be used to measure the progress of the particles. To mitigate this, refer to Eq. (3), we take the evaluation rate as the average evolution value of πΎ iterations divided by the evaluation of the previous iteration. To guarantee that πΎ is chosen reasonably such that the evaluation rate is large enough while the frequency of the evaluations is large enough, assume without loss of generality that the maximum number of iterations (i.e., N er , or N ei , or N c ) is larger than 1000, we set πΎ =50. In general, K can be determined by a simple experiment with some test functions. Formally, we have the expression of the evaluation rate in Eq. (3). πΈπ(π‘) = { ππ(π‘ β πΎ) β ππ(π‘)πΎ β ππ(π‘ β 1) π‘ β₯ πΎππ(1) β ππ(π‘)π‘ β ππ(π‘ β 1) 1 < π‘ < πΎ1 π‘ = 1 (3) Refer to Figure 1, AMPSO works as follows: The AMPSO consists of three modules, the Exploration Swarm, the Exploitation Swarm and the Convergence Swarm. After an initialization of the Exploration Swarm (details are presented in Section 4), it evolves according to Eq. (1-2). As the total number of iterations N i surpasses π π‘ππ‘ππ /3 , the Convergence Swarm starts to work. Before triggering the Convergence Swarm, the Exploration Swarm and the Exploitation Swarm alternatively evolve and search the solution space for all possible local optima. In this process, the Exploration Swarm consists of many equal-sized small sub-swarms searching around the solution space and provides with the Exploitation Swarm small regions around each locally best candidate to focus on; and each sub-swarm iterates N times, evaluate diversity E and updates inertia weight π adaptively. Since the Exploration Swarm requires a stronger capability of searching around the solution space, a large diversity is maintained for it. After the Exploration Swarm runs out of N iterations, its best particle is perturbed into an Exploitation Swarm. To save the effort of function evaluations (FEs) and guarantee the computation efficiency, the evolution of the Exploitation Swarm is forced to stop as the number of iterations exceeds the maximum allowed quota or the evolution rate is smaller than a threshold, i.e., π‘ >π (π = πΌ π π‘ππ‘ππ ) or πΈπ(π‘) < π½ . During the exploitation process of the Exploitation Swarm, a small diversity is required to make the swarm rapidly converge. To guarantee this, at each iteration N s weak particles are updated using the perturbation of the best particleβs position. After this operation, N s operations has been consumed. To maintain the same number of operations for each iteration and therefore a fixed number of iterations for AMPSO, out of the new swarm (of size π ππ ) only (π ππ β π π ) particles are selected randomly to be updated by Eq. (1-2). The exploration and exploitation processes alternate as π π > (π π‘ππ‘ππ /3) . Similar to the generation of the Exploitation Swarm, the Convergence Swarm is also generated from a perturbation of the best particleβs position. Using a fixed total of π π‘ππ‘ππ iterations, the remaining number of iterations for the Convergence Swarm is thus ( π π‘ππ‘ππ β π π ). To guarantee a better result, different from the Exploitation Swarm which emphasizes on rapid convergence, the convergence process balances the exploration and the exploitation states by using a moderate diversity with a new setting of π . In addition, to help the Convergence Swarm jump out of the trap of local optima, a swarm reconstruction strategy is invoked with a probability π π . The βReconstruct the swarmβ and βUpdate the swarmβ in the convergence module of the flowchart (Figure 1) are conducted as follows: Normally, the swarm is βupdatedβ according to Eq. (1-2); when the swarm is trapped in a locally optimal region (evaluated by π π ), to jump out of the trap all particles in the swarm are reconstructed by using the best particleβs position, the details of the reconstruction technique shall be presented in Section 3.4. Fig. 1. AMPSO flowchart Diversity has been known as an especially critical factor for controlling the exploration and exploitation abilities of the swarm [16, 55, 56, 57, 58]. Lack of population diversity is an important factor for premature convergence of the swarm at a local optimum [2]. And enhancing diversity was considered to be a useful approach to escape from the local optima [2, 14]. However, enhancing the swarm diversity may also require a larger number of iterations to achieve the global best solution. In the
StartCalculate E and update Ο by Eq.(10) and Eq.(11)Update all sub-swarms separately by Eq.(1-2) Generate Exploitation Swarm by Eq.(14) and initialize I :=0 I >N or Er(I )< Ξ² N i < ( N total / 3) Generate Convergence Swarm by Eq.(14) and initialize I :=0 I < N total - N i End I < N Calculate E and update Ο , Ο by Eq.(10), (12) and (13) Reconstruct the worst N s particles by Eq.(15) Initialize all sub-swarms of Exploration Swarm and I :=0 Select ( N ei - N s ) particles randomly and update them by Eq.(1-2)Calculate E and update Ο , Ο by Eq.(10), (12) and (13)
Calculate the rebuilding swarm possibility P c by Eq.(16)Update the swarm by Eq.(1-2) rand < P c Reconstruct the swarm by Eq.(15)
Y N
Y N N Y Y NNY
Exploration
Swarm
Convergence SwarmExploitation Swarm
Initialize parameters: N er , N ei , N c , N total , D , Ξ² N := Ξ± * N total , N := Ξ± * N total , N s := Ξ± * N total , N i :=0. best particle I := I +1 N i := I + I best particle I := I +1 I := I +1 ollowing sections, we shall introduce our novel strategy of diversity measurement and the parameter settings for AMPSO in details, which achieves closer solutions to the global optima of a large number of complex functions in CEC2015 test suite by using the same amount of iterations compared with existing peer algorithms. Diversity measures the distribution state of the particles: when the particles are gathering near some local optima (which has the potential to be global best), the difference (e.g., positions and fitness values) and therefore the diversity among the particles are small; on the other hand, when the particles are roaming around the searching space, the diversity is large. Therefore, we can control the exploration and exploitation states of a swarm by adjusting the parameters that affect the diversity of the swarm. To obtain nice diversities, two major metrics have been proposed: the population distribution entropy (denoted as PDE) [59, 60, 61] and average distance amongst points (denoted as ADAP) [62]. Of the two types of methods ADAP is rather straightforward, and it can be further categorized into two methods: one method is based on the average distance from the swarm center [ ] (Eq. (4)) and the other method is based on the average of the average distance in between each pair of particles in the swarm [ ] (Eq. (5)). πππ£πππ ππ‘π¦(π‘) = 1π β πΏ β ββ(π₯ ππ β π₯ π ) (4) πππ£πππ ππ‘π¦(π‘) = 1π β πΏ β ( 1π β ββ(π₯ ππ β π₯ ππ ) ) ππ=1 (5) where π is the number of poarticles in the swarm, and πΏ is the length of the longest diagonal in the search space. In addition, the scheme of population-distribution-entropy was originally motivated by the principle of Shannon entropy in information theory [65]. The basic concept of this method is as follows: the current search space is divided into π parts with equal size; let π π be the number of particles in the i -th part, let π π = π π /π be the probability that the particles are contained in the i -th area. Then, taking population-distribution-entropy as our diversity, the population diversity can be defined by Eq. (6-7). π π = π π π (6) = β β π πππ=1 πππ π π π (7) Based on ADAP and PDE a dozen of algorithms have been proposed [66, 67, 68]. Particularly, Zhan [14] proposed an Evolutionary Factor computed by comprehensively considering the distance between each pair of particles and the distance from the best particle to each of the remaining particles. However, the ADAP and PDE methods are both based on the distance metric, which may lead to misguiding entropy information. For example, for a multimodal function with multiple local optima ADAP might yield a very large entropy when the particles are near a number of local optima, but a small entropy is needed in this case for the swarm to converge rapidly to the global best. PDE may lose the distribution information of the positions and fitness values of the particles. For example, the mutual Euclidean distance metric three points [1,0,0], [0,1,0] and [0,0,1] is 2 , and their Euclidean distance entropy (i.e., the degree of the difference of Euclidean distances between every pair of points) is 0, which cannot reflect the position distribution and fitness distribution of the particles. To avoid the above-mentioned problems due to the distance metric, in this paper we propose a novel hybrid of the position diversity and the fitness diversity as follows: Position Diversity:
First of all, we divide each position dimension d into Q parts, count the number of particles that fall into the i -th part and compute π ππ by Eq. (6), and finally the position diversity of the d -th dimension and the position diversity of the entre swarm can be obtained by Eq. (8) and Eq. (9) respectively. πΈ π (π‘) = β β π ππππ=1 πππ π π ππ (8) πΈ π (π‘) = 1π· β πΈ π (π‘) π·π=1 (9)
Fitness Diversity:
Similar to the position diversity, the distribution of the fitness values of the particles, i.e., the fitness diversity of the swarm, also reflects the exploration or convergence state of the swarm. Particularly, when a swarm gathers around some local optima (which has the potential to be global best) the fitness diversity is fairly small; on the other hand, when the swarm is distributed randomly it can roam around the searching space and the fitness diversity is relatively large. For example, in Figure 2, although the positions of the four points in Figure 2(a) and Figure 2(b) are the same, their fitness values vary significantly. Therefore, to guide the search towards the global best, we take a combination of the position diversity and fitness diversity into consideration. Fig. 2. The difference between position diversity and fitness diversity: (a) points with different fitness, (b) points with the same fitness. Our calculation of the fitness diversity of a swarm is similar to that of [61]. More precisely, we divided [πππ‘πππ π πππ , πππ‘πππ π πππ₯ ] into Q parts and calculate p i for part I , and then obtain πΈ π (π‘) by Eq. (7). Formally, our hybrid of position diversity and fitness diversity is formulated in Eq. (10). πΈ(π‘) = πΈ π (π‘) + πΈ π (π‘)2 (10) The inertia weight plays a critical role in balancing the exploration and exploitation capabilities of the swarm, and it also significantly affects the accuracy of the results [22, 23, 24]. Therefore, we shall adaptively adjust the inertia weight function to guarantee the effective cooperation of the swarms. Meanwhile, for ease of computation, we fix c = c = 1.49445 based on a large set of experimental results of previous PSO variants (e.g., [22, 50, 53]). Since the Exploration Swarm, the Exploitation Swarm and the Convergence Swarm have different functions during the searching process for the global best, different control parameters will be set for the swarms. Specifically, the detailed strategies for the swarms are as follows: (1) Exploration Swarm The role of the swarm is to explore the searching space broadly and provide small searching regions for the Exploitation Swarm. For this purpose, the Exploration Swarm is divided into a set of equal-sized sub-swarms randomly distributed around the solution space (i.e., they are not necessarily neighboring to each other) and each sub-swarm evolves independently (without communication) in order to explore the searching space as much as possible. For the inertia weight w , it non-linearly decreases from 0.9 to 0.6 to maintain a large diversity E ( t ) (with a strong exploration ability). Formally, using a sigmoid-like mapping we have Eq. (11) for the Exploration Swarm.
1( ) [0.6, 0.9] ( ) [0,1]1 0.67
E t t E te ο· β = ο ο’ ο+ (11) ( a ) ( b ) x x x x x x x x
2) Exploitation Swarm
The role of the swarm is rapidly converging to the neighborhood of the local best solutions without emphasizing too much on the precision level. We also design a sigmoid-like update mechanism to change the inertia weight adaptively from 0.8 to 0.5 in order to improve the exploitation ability of Exploitation Swarm. In addition, to further accelerate the convergence process, a partial swarm reconstruction strategy is adopted by replacing the positions of the worst N s particles from a perturbation of the best particleβs position. To control the range of distribution, a Gaussian distribution π(0, π ) is exploited, where π decreases from 0.2 to 0.1. Formally we have Eq. (12-13) for controlling the Exploitation Swarm. π(π‘) = 11 + π βππ4βπΈ(π‘) β [0.5,0.8]ββββπΈ(π‘) β [0,1] (12) π(π‘) = 11 + 9π (ππ4βππ9)βπΈ(π‘) β [0.1,0.2]ββββπΈ(π‘) β [0,1] (13) (3) Convergence Swarm In this scenery, the role of the swarm is converging to a higher accuracy solution at a proper speed and jumping out of the trapes of local optimal regions. In order to obtain a nice solution, a large inertia weight corresponding to the strong exploration ability of the swarm is needed at the beginning and it should decrease to a small value that corresponds to the ability of refining solution as the diversity decreases, therefore we set the range of inertia weight as [0.5, 0.8] by Eq. (12). As for jumping out of local traps, the Convergence Swarm utilizes a swarm reconstruction strategy. Different from the partial swarm reconstruction strategy for Exploitation Swarm that replaces N s worst particles, the positions of all particles are replaced from the perturbation of the best particle. This helps the swarm jump out from a local trap rapidly. Similarly, a Gaussian distribution π(0, π ) is exploited to control the range of distribution, where π is also determined by Eq. (13). In our strategy, the Exploitation Swarm is an artificial swarm generated from the best particle of the Exploration Swarm, and the Convergence Swarm is an artificial swarm generated from the best particle of the Exploitation Swarm. Since the chosen best particle is near some local optimal point, to increase the chance of achieving the local best, we shall perturb the position of the best particle π₯[π₯ , π₯ , β¦ , π₯ π· ] into a set of particles around a small neighborhood of π₯ by Eq. (14). π₯ β² = π₯ + (π₯ π’π β π₯ ππ )ππ (14) where ππ = [ππ , ππ , β¦ , ππ π· ] , and ππ π follows the Gaussion distribution with ππ π β π(0, (0.1) ) , and π₯ β [π₯ ππ , π₯ π’π ] . .4. Swarm Reconstruction Partial Swarm Reconstruction for Exploitation:
To guarantee that the Exploitation Swarm rapidly converges to the neighborhood of a promising local best that has a potential to become the global best, apart from the adjustment of the parameters, in each iteration we also force the lousiest N s to change their positions according to the best particleβs position. Different from the technique of artificial swarm generation by perturbing dr for each dimension, the reconstruction technique here alters merely one dimension randomly. This is based on the consideration that the best particle wins out because quite a few of its dimensions have reached very nice states and changing merely one dimension may preserve its nice property as much as possible. As mentioned in Section 3.1, to guarantee a constant number of update operations in each iteration and therefore a fixed number of total iterations, only (π ππ β π π ) random particles are updated. Formally, the reconstruction technique alters merely one random dimension of the position vector works as follows: ( ) d dnew ub lb x x x x r β= + (15) where d is a randomly chosen dimension, π follows the Gaussian distribution and π β π(0, π ) . Ο can be updated by Eq. (13) to yield nice results. Swarm Reconstruction for Convergence:
For the Convergence Swarm, when the evolution rate of the swarm gets stagnated for a consecutive π ππ iteration, the swarm needs to jump out of a local trap. Using the best particleβs position, we reconstruct the entire swarm based on a probability π π as follows: π π = 11 + π (0.01βπ π‘ππ‘ππ βπ ππ ) (16) where π ππ is increased by one for every new iteration when πΈπ(π‘) < π½ .
4. Experiments and Comparisons
To validate our AMPSO on complex multimodal functions, we choose CEC2015 benchmark functions [69] as our test suite. Refer to Table 2, this test suite consists of the latest collection of 15 unconstrained continuous optimization problems with various difficulty levels. According to their properties, these functions are divided into four groups: unimodal functions, simple multimodal functions, hybrid functions, and composition functions. All the guidelines of CEC2015 have been strictly followed for the experiments. Following the instructions of the test suite, every test of a function is conducted for 30 runs independently. The results for the 10- D case and the 30- D case are calculated. The range of the search space for each dimension is [β100, 100]. Population initialization is generated uniformly in the specified search space by a random number generator with clock time. The iteration process terminates as the maximum function evaluation FEs = 10000 D is reached. The fitness value is Fi ( x ) β Fi ( x* ) after the maximum iteration is reached, where Fi ( x* ) is just a number about the orresponding function for instruction. Table 2
Summary of the CEC2015 benchmark functions
Type No. Function name F i *= F i ( x* ) Unimodal Functions 1 Rotated High Conditioned Elliptic Function 100 2 Rotated Cigar Function 200 Simple Multimodal Functions 3 Shifted and Rotated Ackleyβs Function 300 4 Shifted and Rotated Rastriginβs Function 400 5 Shifted and Rotated Schwefelβs Function 500 Hybrid Functions 6 Hybrid Function ( N = 3) 600 7 Hybrid Function ( N = 4) 700 8 Hybrid Function ( N = 5) 800 Composition Functions 9 Composition Function 1 ( N = 3) 900 10 Composition Function 2 ( N = 3) 1000 11 Composition Function 3 ( N = 5) 1100 12 Composition Function 4 ( N = 5) 1200 13 Composition Function 5 ( N = 5) 1300 14 Composition Function 6 ( N = 7) 1400 15 Composition Function 7 ( N = 10) 1500 To validate the effectiveness of our algorithm, using CEC2015 as their test suite, a set of comparison experiments with 4 recently published PSO variants and the sets of comparison algorithms that they used for comparisons purposes. The information of these algorithms is provided in Table 3.
Table 3
The Algorithms for Comparison 1 A modified particle swarm optimizer
GPSO [43] 2 Population structure and particle swarm performance
LPSO [25] 3 Defining a standard for particle swarm optimization
SPSO [70] 4 Comprehensive learning particle swarm optimizer
CLPSO [ ] 5 The fully informed particle swarm FIPS [26] 6 Dynamic multi-swarm particle swarm optimizer with local search
DMSPSO [49] 7 Modified particle swarm optimization and its applications MPSO [72] 8 Traditional global PSO algorithms with inertia weight GPSO*[1]
9 PSO with multiple subpopulations
MSPSO [73]
10 The fully informed particle swarm FIPS [26]
11 PSO with dynamic tournament topology strategy
DTTPSO [74]
12 Dynamic multi-swarm particle swarm optimizer with local search
DMSPSO [49] 14 Artificial Bee Colony
ABC [75]
15 Teaching-Learning-Based Optimization
TLBO [76]
6 Multi-swarm PSO with dynamic learning strategy
PSODLS [53]
17 Ensemble particle swarm optimizer
EPSO [77]
18 Heterogeneous comprehensive learning PSO
HCLPSO [78]
Note: GPSO* on line 8 and GPSO on line 1 are two distinct PSO variants cited from [1] and [43] respectively. FIPS on line 5 and 10 and DMSPSO on line 6 and 14 are the same but are tested under slightly different criteria
First of all, using the same comparison criterial with MPSO [72], we compare our AMPSO with MPSO and its corresponding comparison algorithms (GPSO, LPSO, SPSO, CLPSO, FIPS, DMSPSO). The same 12 representative functions ( f - f , f , f and f ) that consist of 2 unimodal functions ( f and f ), 3 multimodal functions ( f , f and f ), 3 hybrid functions ( f , f and f ) and 4 composition functions ( f , f , f and f ) from CEC2015 test suite [69] are used tested, and the performance of the algorithms is characterized by statistical means and standard deviations, which is exactly the same criteria as used in [72]. For all the algorithms, the swarm size is set to 40. So, the maximum number of iterations is 2500 for 10 D and 7500 for 30 D . The statistical results for the test functions in 10- D and 30- D are summarized in Table 4 and Table 5 respectively. Note that the lowest mean and standard deviation in each line are the best results and are highlighted in bold. Secondly, using the same criteria with PSODLS [53], we compare our AMPSO with PSODLS and its corresponding comparison algorithms (GPSO*, MSPSO, FIPS, DTTPSO, DMSPSO, ABC, TLBO). The same 15 functions from CEC2015 all are tested and the same statistical criteria (i.e., best, worst, average, and median) are evaluated. For all the algorithms, the swarm size is set to 40. The maximum number of iterations is 2500 for 10 D and 7500 for 30 D . Note that this is also the same setting in [72]. Table 6-9 summarize the statistical results. Finally, we compare our AMPSO with two recent PSO variants, EPSO [77] and HCLPSO [78]. For completeness, we use the statistics of mean, standard deviation, best, worst, and median. Table 10 summarizes the statistical results. To conduct the comparison experiments, some control parameters of our AMPSO are set as follows: N er = 10, N ei = N c = 40 (same as the setting of MPSO [72] and PSODLS [53]), N total = FEs / N c , πΌ = 0.02 , πΌ = 0.2 , πΌ = 0.25 , π½ = 0.001 , π£ πππ₯ = 0.01 β (π₯ π’π β π₯ ππ ) . able 4 Comparison of AMPSO with MPSO and its comparison suite for 10- D case Func Item GPSO LPSO SPSO CLPSO FIPS DMSPSO MPSO AMPSO F1 Mean 5.05E+06 1.17E+05 2.78E+04 5.53E+05 2.51E+05 9.64E+04 2.63E+04
Std 2.31E+07 7.30E+04 1.01E+04 7.42E+04 3.05E+04 2.46E+05 6.61E+03
F2 Mean 6.02E+08 2.94E+07 6.15E+03 4.73E+04 6.49E+03 1.38E+04 6.36E+03
Std 1.05E+09 7.79E+07 2.83E+04 8.12E+04 3.86E+02 5.47E+03
Std 6.41E+03 1.04E+04 5.75E+03 3.96E+02 7.15E+02 1.38E+04 3.17E+02
F7 Mean 5.56E+00 3.06E+00 1.34E+00 1.53E+00 8.87E-01 1.96E+00 8.99E-01
Std 2.12E+00 4.30E-01 1.99E-01 4.24E-01 2.39E-01 4.07E-01
Std 1.79E+01 1.01E+00 7.99E-02 1.39E-02 1.92E-01 5.41E-01
Std 4.04E+06 6.32E+03 8.62E+03 4.51E+02 1.70E+03 2.89E+03 7.25E+02
F15 Mean 1.25E+02 1.06E+02
Std 8.43E+00 4.05E+01 able 5
Comparison of AMPSO with MPSO and its comparison suite for 30- D case As shown in Table 4, when comparing AMPSO with MPSO and its comparison suite for the 10- D case, we can see that the mean values of AMPSO achieves solutions that are closer to the true optimum on 7 functions ( f , f , f , f , f , f and f ), and the accuracy of the solution provided by AMPSO is at least an order of magnitude better than that of other PSO variants on f , f and f . However, compared with MPSO on f , f and f , DMSPSO on f and CLPSO on f , our AMPSO is slightly weaker. As shown in Table 5, for the 30- D case, compared to the counterparts, AMPSO obtains the best mean values in 10 out of 12 benchmark functions (f , f , f , f , f , f , f , f , f , and f ). However, it is slightly weaker than MPSO on f and f . In conclusion, although AMPSO is weak in solving a few test functions, it has obvious advantages over MPSO and its comparison suite of PSO variants on the same 12 test functions of CEC2015 provided in [72]. Particularly, AMPSO has better solutions for high-dimensional complex functions. Table 6, Table 7, Table 8 and Table 9 summarize the statistical values ( Best , Worst , Mean and
Median ) of 8 algorithms on 15 test functions under D = 10 and D = 30 Func Item GPSO LPSO SPSO CLPSO FIPS DMSPSO MPSO AMPSO F1 Mean 2.72E+08 3.52E+07 2.23E+05 5.56E+06 4.00E+06 6.72E+06 1.79E+06
Std 3.54E+08 5.92E+07 2.42E+05 5.80E+06 2.42E+05 1.15E+07 2.38E+05
F2 Mean 2.23E+10 1.91E+09 3.70E+03 4.48E+03 6.29E+03 3.37E+03 1.46E+03
Std 1.96E+09 2.09E+09 2.44E+04 1.70E+03 1.43E+04 2.92E+03
Std 7.54E-03 3.58E-01 8.30E-02 1.90E-02 9.90E-02 1.53E-01
Std 3.22E+02 8.32E+02 1.39E+03 6.03E+02 1.51E+03 5.40E+02
Std 7.16E+07 5.56E+05 3.13E+04 1.71E+05 5.05E+05 8.56E+04
Std 3.55E+01 3.69E+01 1.25E+00 8.60E-01 4.09E+00
Std 2.05E+07 4.11E+05 2.60E+04 9.24E+04 9.13E+03 1.91E+05
Std 3.88E+02 2.09E+02 7.22E+01 4.81E+01 8.05E+01 7.56E+02 1.88E+01
F14 Mean 4.86E+04 3.85E+04 3.36E+04 2.89E+04 2.73E+04 3.07E+04 2.73E+04
Std 4.47E+04 4.23E+03 3.82E+03
Std 3.01E+03 2.77E+01 espectively when comparing AMPSO with PSODLS and its comparison suite, and the best results (the minimum value of each row) are shown in bold. As presented in Table 6 and Table 7, for the 10- D case, the Mean value of AMPSO on the 6 functions f , f , f , f , f , f and f are the best among the 9 algorithms. Note that all these functions are hybrid or composition functions, and they are the most complex functions in the CEC 2015 test suite. In the other hand, PSODLS achieves the better outcomes on f , f and f , DMSPSO finds better results on f , f , and f , while TLBO locates better local optima on f and f . For the 30-D case in Table 8 and Table 9, AMPSO achieved the best results on 11 functions f , f , f , f , f , f , f , f , f , f and f out of 15 benchmark functions . Except f , f and f , all functions are hybrid or composition functions. This again proves the strength of AMPSO in solving complex optimization problems. However, AMPSO is inferior on f , f and f compared with PSODLS and inferior on f with TLBO. In summary, the advantages of AMPSO in high-dimensional situations are still obvious, and 11 optimal solutions are obtained out of 15 functions. In Table 10 and Table 11, statistical results in terms of mean, std, best, worst, and median are chosen as our evaluation criteria to conduct comparison of our algorithm with two recent PSO variants: EPSO and HCLPSO. From the tables, we can see that for the 10- D case, AMPSO obtains 6 best mean results on f , f , f , f , f and f , equal to EPSO and HCLPSO on f and f (EPSO achieves the optimal standard deviation values on these two functions), while weaker than EPSO on f , f , f , f and f and HCLPSO on f , f , f , f . Similarly, for 30-D case, AMPSO achieves the optimal value on 8 functions ( f , f , f , f , f , f , f and f ), while EPSO performs best on 4 functions ( f , f , f and f ) and HCLPSO on 7 functions ( f , f , f , f , f , f and f ). Although the optimal values AMPSO achieved in the 10- D condition is one less than that of EPSO and HCLPSO, in the 30- D AMPSO has four more functions better than EPSO and one more function better than HCLPSO. able 6
Comparison of AMPSO with PSODLS and its comparison suite for 10-D case on f - f Func Item GPSO* MSPSO FIPS DTT-PSO DMS-PSO ABC TLBO PSODLS AMPSO F1 Best 8.37E-10 2.81E+02 4.26E+02 2.72E-07 2.11E-05 6.23E+03
Worst 1.32E+04 5.42E+03 1.53E+04 1.36E+04 1.42E+04 8.32E+02
Average 2.02E+01 2.02E+01 2.00E+01 1.97E+01
F4 Best 2.98E+00 2.01E+00 3.91E+00 1.35E+01 9.90E-01 4.03E+00 3.98E+00
Worst 2.20E+03 4.51E+03 2.29E+03 1.50E+03 1.09E+03 2.74E+05 1.31E+03 1.12E+03
Average 6.64E+02 1.19E+03 8.11E+02 8.17E+02 3.73E+02 6.68E+04 6.73E+02 2.62E+02
Median 5.62E+02 9.19E+02 6.96E+02 8.58E+02 2.77E+02 5.39E+04 5.90E+02
Worst 3.06E+00 2.54E+00 1.81E+00 3.04E+00 2.47E+00 1.62E+00 2.57E+00
Worst 6.14E+03 2.49E+03 3.74E+03 2.03E+03 4.90E+03 5.50E+04 8.98E+02
Median 1.91E+03 7.19E+02 9.09E+02 8.74E+02 1.70E+03 6.87E+03 1.80E+02 2.61E+02 able 7
Comparison of AMPSO with PSODLS and its comparison suite for 10- D case on f - f Func Item GPSO* MSPSO FIPS DTT-PSO DMS-PSO ABC TLBO PSODLS AMPSO F9 Best 1.02E+02 1.06E+02 1.02E+02 1.19E+02 1.00E+02
Average 1.25E+02 1.10E+02 1.14E+02 1.28E+02 1.05E+02 1.01E+02 1.08E+02 1.07E+02
Median 1.12E+02 1.10E+02 1.12E+02 1.25E+02 1.05E+02 1.09E+02 1.08E+02 1.07E+02
F10 Best 7.96E+01 1.50E+02 1.15E+03 3.97E+02 4.88E+02 4.04E+02 2.04E+02
Average 1.12E+03 9.52E+02 1.72E+03 1.29E+03 1.30E+03 2.70E+03 7.73E+02 4.24E+02
Median 1.19E+03 8.46E+02 1.73E+03 1.19E+03 1.29E+03 2.37E+03 8.77E+02
Worst 3.70E+01 3.27E+01 3.26E+01 3.29E+01 3.18E+01 2.98E+01 2.91E+01 2.06E+01
Average 2.67E+01 2.74E+01 2.64E+01 2.40E+01 1.64E+01 2.47E+01 2.41E+01 1.58E+01
Median 2.71E+01 2.79E+01 2.69E+01 2.03E+01 1.56E+01 2.52E+01 2.51E+01 1.63E+01
F14 Best 1.00E+02 1.00E+02 1.00E+02 1.00E+02 1.00E+02 1.00E+02 1.00E+02 1.00E+02
Worst 2.73E+03 1.17E+02 2.02E+02 2.67E+03 2.74E+03
Average 7.85E+02 1.03E+02 1.14E+02 2.72E+02 4.67E+02 1.00E+02 8.18E+02 1.00E+02
Median
F15 Best 2.05E+02 2.05E+02 2.05E+02 2.05E+02 2.05E+02 2.05E+02 2.05E+02 2.05E+02
Worst 2.05E+02 2.05E+02 2.05E+02 2.05E+02 2.05E+02 2.05E+02 2.56E+02 2.05E+02
Average 2.05E+02 2.05E+02 2.05E+02 2.05E+02 2.05E+02 2.05E+02 2.09E+02 2.05E+02
Median 2.05E+02 2.05E+02 2.05E+02 2.05E+02 2.05E+02 2.05E+02 2.05E+02 2.05E+02 able 8
Comparison of AMPSO with PSODLS and its comparison suite for 30- D case on f - f Func Item GPSO* MSPSO FIPS DTT-PSO DMS-PSO ABC TLBO PSODLS AMPSO F1 Best 2.21E+04 4.15E+06 2.57E+06 1.26E+05 1.58E+04 1.61E+06
Average 6.48E+06 1.03E+07 5.83E+06 1.28E+06 3.15E+06 2.92E+06 8.51E+04 1.82E+05
Median 5.81E+06 1.05E+07 5.75E+06 8.71E+05 2.69E+06 3.02E+06
Worst 2.10E+01 2.09E+01 2.10E+01 2.10E+01 2.05E+01 2.04E+01 2.10E+01 2.08E+01
Average 2.07E+01 2.08E+01 2.10E+01 2.10E+01 2.03E+01 2.04E+01 2.09E+01 2.08E+01
Median 2.08E+01 2.08E+01 2.10E+01 2.10E+01 2.03E+01 2.03E+01 2.09E+01 2.08E+01
F4 Best 4.18E+01 5.06E+01 7.13E+01 1.44E+02 1.79E+01 6.10E+01 6.37E+01
Median 3.02E+03 3.44E+03 6.35E+03 6.47E+03 3.22E+03 2.48E+03 3.17E+03 2.48E+03
F6 Best 9.62+e04 1.06E+05 1.03E+05 3.73E+04 1.44E+04 8.15E+05 2.57E+04 8.75E+03
Worst 1.46E+06 3.56E+06 1.05E+06 1.67E+06 1.72E+06 5.73E+06 1.09E+05
Median 2.44E+05 8.39E+05 3.84E+05 2.42E+05 1.88E+05 3.26E+06 5.76E+04 3.78E+04
F7 Best 5.18E+00 6.77E+00 3.80E+00 3.41E+00
Average 4.38E+04 8.28E+04 7.42E+04 7.71E+04 3.61E+04 5.41E+05 2.14E+04 1.76E+04
Median 3.10E+04 6.97E+04 6.16E+04 5.84E+04 3.07E+04 5.18E+05 1.72E+04 1.22E+04 able 9
Comparison of AMPSO with PSODLS and its comparison suite for 30- D case on f - f Func Item GPSO* MSPSO FIPS DTTPSO DMSPSO ABC TLBO PSODLS AMPSO F9 Best 1.53E+02 1.69E+02 1.66E+02 2.01E+02 2.01E+02 1.69E+02 1.84E+02 1.58E+02
Worst 2.13E+02 2.02E+02 2.39E+02 2.94E+02 2.01E+02 2.02E+02 2.22E+02 2.01E+02
Average 1.94E+02 1.99E+02 2.01E+02 2.38E+02 2.01E+02 1.88E+02 2.02E+02 1.99E+02
Median 2.01E+02 2.02E+02 2.01E+02 2.01E+02 2.01E+02 1.87E+02 2.01E+02 2.01E+02
F10 Best 6.64E+03 2.84E+04 1.67E+04 3.42E+03 2.78E+03 6.61E+04 3.03E+03 2.36E+03
Worst 3.67E+05 8.80E+05 2.53E+05 2.16E+05 1.52E+05 1.05E+06 4.40E+04 4.48E+04
Average 8.77E+04 3.81E+05 1.06E+05 4.66E+04 3.58E+04 4.62E+05 1.63E+04 1.36E+04
Median 2.58E+04 3.56E+05 9.83E+04 1.62E+04 2.76E+04 4.37E+05 1.25E+04 1.25E+04
F11 Best 2.08E+02 2.12E+02 2.13E+02 2.03E+02 2.04E+02 2.08E+02 2.05E+02
Worst 1.17E+02 1.15E+02 1.11E+02 1.11E+02 1.10E+02 1.12E+02 1.18E+02 1.10E+02
Average 1.13E+02 1.13E+02 1.10E+02 1.10E+02 1.08E+02 1.11E+02 1.14E+02 1.09E+02
Median 1.14E+02 1.13E+02 1.10E+02 1.10E+02 1.08E+02 1.11E+02 1.14E+02 1.09E+02
F13 Best 9.35E+01 9.59E+01 1.00E+02 1.00E+02 8.73E+01 9.78E+01 1.06E+02 8.27E+01
Worst 1.21E+02 1.19E+02 1.14E+02 1.20E+02 1.05E+02 1.10E+02 1.20E+02 1.00E+02
Average 1.07E+02 1.11E+02 1.08E+02 1.13E+02 9.59E+01 1.05E+02 1.12E+02 9.44E+01
Median 1.06E+02 1.13E+02 1.08E+02 1.14E+02 9.56E+01 1.05E+02 1.11E+02 9.52E+01
F14 Best 2.85E+04 2.98E+04 2.80E+04 2.84E+04 2.80E+04 2.65E+03 2.80E+04 2.80E+04
Worst 3.73E+04 3.25E+04 3.13E+04 3.78E+04 3.57E+04
Median 3.14E+04 3.11E+04 3.10E+04 3.04E+04 3.10E+04 2.74E+04 3.23E+04 2.84E+04
F15 Best 2.74E+02 2.78E+02 2.74E+02 2.73E+02 2.73E+02 2.74E+02 2.75E+02 2.73E+02
Worst 2.78E+02 2.84E+02 2.75E+02 2.74E+02 2.75E+02 2.75E+02 2.82E+02 2.74E+02
Average 2.75E+02 2.81E+02 2.74E+02 2.74E+02 2.74E+02 2.75E+02 2.78E+02 2.74E+02
Median 2.75E+02 2.81E+02 2.74E+02 2.73E+02 2.74E+02 2.75E+02 2.79E+02 2.74E+02 able 10
Comparison of AMPSO with EPSO and HCLPSO for 10-D and 30- D case on f - f Std
Best
Worst
Median
F2 Mean 3.98E+02
Best 4.79E+01 3.32E+00
Worst 2.04E+03
F3 Mean 2.02E+01 2.02E+01
Std
Best 2.00E+01
Worst 2.02E+01 2.02E+01
Worst 2.03E+01 2.03E+01
Median 2.01E+01 2.01E+01
Median 2.02E+01 2.02E+01
F4 Mean 4.52E+00
F6 Mean 2.39E+04 5.13E+04
Std 4.54E+02 3.31E+02
Std
Best
Worst
Median 1.77E+04 4.12E+04
F7 Mean 5.70E-01
Best 3.90E-02
Std
Best
Median able 11
Comparison of AMPSO with EPSO and HCLPSO for 10- D and 30- D case on f - f The Matlab code of our proposed approach is available in [79].
5. Conclusion and Future Work
We proposed an artificial multi-swarm PSO (AMPSO) to improve the performance
F9 Mean
Best
Worst
Median
Std
Best
Worst
Median
F11 Mean 1.21E+02
Std 1.49E+02
Worst
Median
F12 Mean
F13 Mean
Best 2.60E-02
F14 Mean 3.16E+04 3.15E+04
Std 1.49E+03 1.17E+03
Std 4.82E+02
Best 3.11E+04 3.12E+04
Worst 6.68E+03 2.96E+03
Worst 3.30E+04
Median 3.15E+04 3.14E+04
F15 Mean
F15 Mean
Std
Best
Worst
Worst
Median
Median f PSO, the multi-swarm system consists of an exploration swarm with many equal-sized subswarms, an artificial exploitation swarm generated from the best particle of the exploration swarm, and an artificial convergence swarm generated from the best particle of the exploitation swarm. To guarantee the success of the strategy, the Exploration Swarm and the Exploitation Swarm are conducted alternatively, and a novel diversity method is also applied for the swarms; to compensate the drawback of slow convergence of a swarm with large diversity, two swarm update techniques (i.e., partial swarm reconstruction and swarm reconstruction) are applied to the Exploitation Swarm and the Convergence Swarm to guarantee that AMPSO can achieve nice results within a fixed number of iterations. We validate our AMPSO by comparing it with a set of comprehensive 16 algorithms, including the most recently well-performing PSO variants and some other non-PSO optimization algorithms under the same criteria. As presented above, for the 30- D case, AMPSO obtains 10 best average solutions out of 12 benchmark functions when compared with MPSO, 11 out of 15 benchmark functions when compared with PSODLS, and 8 out of 15 benchmark functions when compared with EPSO and HCLPSO. We conclude that AMPSO is excellent in solving complex high-dimensional problems. However, there is still much room for improvement when our AMPSO is dealing with lower dimensions. For example, for the 10- D case, 7 (out of 12), 6 (out of 15) and 6 (out of 15) best mean values can be obtained by AMPSO in the comparison experiments with MPSO, PSODLS and EPSO and HCLPSO respectively. Although the results are nice when compared with other algorithms, but less significant when compared with the performance of AMPSO in the 30- D case. Also, AMPSO has a large std in Table 4 and Table 5, which begs for a future research to reduce it. We have also tested our algorithm on CEC2013 (with simpler benchmark functions than CEC2015) in the case of dimension 30 and found that in terms of accuracy, it is slightly weaker than the state-of-the-art algorithms such as MPSO [72] and MSPSO [80]. Our algorithm still has some room of improvement on high-dimensional but simple functions and this is also worthy of future research. Furthermore, although adjusting the inertia weight has been proved to be effective in balancing the capabilities of the exploration and exploitation, more parameters (e.g., c and c ) can be designed into adaptive settings for the swarms to achieve better results. Finally, our AMPSO may be incorporated with other nice local search algorithms such as GA and SAA to improve its ability in producing best particles for the transmissions from one swarm to another. Acknowledgement:
This work was partially funded by the Shanghai International Science and Technology Cooperation Fund No. 18510745700 and the Science and Technology Development Fund, Macau SAR (Grant no. 0018/2019/AKP, and 0008/2019/AGJ)