Christopher Wesley Cleghorn

A generalized theoretical deterministic particle swarm model

A number of theoretical studies of particle swarm optimization (PSO) have been done to gain a better understanding of the dynamics of the algorithm and the behavior of the particles under different conditions. These theoretical analyses have been performed for both the deterministic PSO model and more recently for the stochastic model. However, all current theoretical analyses of the PSO algorithm were based on the stagnation assumption, in some form or another. The analysis done under the stagnation assumption is one where the personal best and neighborhood best positions are assumed to be non-changing. While analysis under the stagnation assumption is very informative, it could never provide a complete description of a PSO’s behavior. Furthermore, the assumption implicitly removes the notion of a social network structure from the analysis. This paper presents a generalization to the theoretical deterministic PSO model. Under the generalized model, conditions for particle convergence to a point are derived. The model used in this paper greatly weakens the stagnation assumption, by instead assuming that each particle’s personal best and neighborhood best can occupy an arbitrarily large number of unique positions. It was found that the conditions derived in previous theoretical deterministic PSO research could be obtained as a specialization of the new generalized model proposed. Empirical results are presented to support the theoretical findings.

Particle swarm variants: standardized convergence analysis

This paper presents an objective function specially designed for the convergence analysis of a number of particle swarm optimization (PSO) variants. It was found that using a specially designed objective function for convergence analysis is both a simple and valid method for performing assumption free convergence analysis. It was also found that the canonical particle swarm’s topology did not have an impact on the parameter region needed to ensure convergence. The parameter region needed to ensure convergent particle behavior was empirically obtained for the fully informed PSO, the bare bones PSO, and the standard PSO 2011 algorithm. In the case of the bare bones PSO and the standard PSO 2011, the region needed to ensure convergent particle behavior differs from previous theoretical work. The difference in the obtained regions in the bare bones PSO is a direct result of the previous theoretical work relying on simplifying assumptions, specifically the stagnation assumption. A number of possible causes for the discrepancy in the obtained convergent region for the standard PSO 2011 are given.

Particle swarm convergence: An empirical investigation

This paper performs a thorough empirical investigation of the conditions placed on particle swarm optimization control parameters to ensure convergent behavior. At present there exists a large number of theoretically derived parameter regions that will ensure particle convergence, however, selecting which region to utilize in practice is not obvious. The empirical study is carried out over a region slightly larger than that needed to contain all the relevant theoretically derived regions. It was found that there is a very strong correlation between one of the theoretically derived regions and the empirical evidence. It was also found that parameters near the edge of the theoretically derived region converge at a very slow rate, after an initial population explosion. Particle convergence is so slow, that in practice, the edge parameter settings should not really be considered useful as convergent parameter settings.

Particle Swarm Convergence: Standardized Analysis and Topological Influence

This paper has two primary aims. Firstly, to empirically verify the use of a specially designed objective function for particle swarm optimization (PSO) convergence analysis. Secondly, to investigate the impact of PSO’s social topology on the parameter region needed to ensure convergent particle behavior. At present there exists a large number of theoretical PSO studies, however, all stochastic PSO models contain the stagnation assumption, which implicitly removes the social topology from the model, making this empirical study necessary. It was found that using a specially designed objective function for convergence analysis is both a simple and valid method for convergence analysis. It was also found that the derived region needed to ensure convergent particle behavior remains valid regardless of the selected social topology.

Critical considerations on angle modulated particle swarm optimisers

This article investigates various aspects of angle modulated particle swarm optimisers (AMPSO). Previous attempts at improving the algorithm have only been able to produce better results in a handful of test cases. With no clear understanding of when and why the algorithm fails, improving the algorithm’s performance has proved to be a difficult and sometimes blind undertaking. Therefore, the aim of this study is to identify the circumstances under which the algorithm might fail, and to understand and provide evidence for such cases. It is shown that the general assumption that good solutions are grouped together in the search space does not hold for the standard AMPSO algorithm or any of its existing variants. The problem is explained by specific characteristics of the generating function used in AMPSO. Furthermore, it is shown that the generating function also prevents particle velocities from decreasing, hindering the algorithm’s ability to exploit the binary solution space. Methods are proposed to both confirm and potentially solve the problems found in this study. In particular, this study addresses the problem of finding suitable generating functions for the first time. It is shown that the potential of a generating function to solve arbitrary binary optimisation problems can be quantified. It is further shown that a novel generating function with a single coefficient is able to generate solutions to binary optimisation problems with fewer than four dimensions. The use of ensemble generating functions is proposed as a method to solve binary optimisation problems with more than 16 dimensions.

Fully informed particle swarm optimizer: Convergence analysis

At present, the explicit conditions necessary for order-2 stability of the fully informed particle swarm optimizer (FIPS) have not be derived. This paper theoretically derives the criteria for order-2 stability of the FIPS algorithm under the stagnation assumption. The exact relationship between the criteria for order-2 stability and the neighborhood size is presented. The maximum possible convergence region is also presented for an arbitrarily large neighborhood size. Unlike the vast body of theoretical research on particle swarms, this paper validates its conclusions empirically against an assumption free FIPS algorithm. This empirical validation is necessary for a truly accurate representation of FIPSs convergence criteria.

Particle swarm stability: a theoretical extension using the non-stagnate distribution assumption

This paper presents an extension of the state of the art theoretical model utilized for understanding the stability criteria of the particles in particle swarm optimization algorithms. Conditions for order-1 and order-2 stability are derived by modeling, in the simplest case, the expected value and variance of a particle’s personal and neighborhood best positions as convergent sequences of random variables. Furthermore, the condition that the expected value and variance of a particle’s personal and neighborhood best positions are convergent sequences is shown to be a necessary condition for order-1 and order-2 stability. The theoretical analysis presented is applicable to a large class of particle swarm optimization variants.

Particle swarm optimizer: The impact of unstable particles on performance

There exists a wealth of theoretical analysis on particle swarm optimization (PSO), specifically the conditions needed for stable particle behavior are well studied. This paper investigates the effect that the stability of the particle has on the PSOs actually ability to optimize. It is shown empirically that a majority of PSO parameters that are theoretically unstable perform worse than a trivial random search across 28 objective functions, and across various dimensionalities. It is also noted that there exists a number of parameter configurations just outside the stable-2 region which did not exhibit poor performance, implying that a minor violation of the conditions for order-2 stability is still acceptable in terms of overall performance of the PSO.

Unified particle swarm optimizer: Convergence analysis

At present, very little theoretical analysis has been performed on the unified particle swarm optimizer (UPSO). This paper derives the order-1 and order-2 stable regions for the UPSO algorithm, along with the fixed point of particle convergence. The impact that the unification factor has on the stability of UPSO is also analyzed. The theoretical analysis is performed under the stagnation assumption; however, the derived results are shown to be both necessary and sufficient for particle convergence empirically, using a standardized methodology for assumption free convergence region analysis.

Fitness-distance-ratio particle swarm optimization: stability analysis

At present the fitness-distance-ratio particle swarm optimizer (FDR-PSO) has undergone no form of theoretical stability analysis. This paper theoretically derives the conditions necessary for order-1 and order-2 stability under the well known stagnation assumption. Since it has been shown that particle stability has a meaningful impact on PSOs performance, it is important for PSO practitioners to know the actual criteria for particle stability. This paper validates its theoretical findings against an assumption free FDR-PSO algorithm. This empirical validation is necessary for a truly accurate representation of FDR-PSOs stability criteria.