Shu-Kai S. Fan

A hybrid simplex search and particle swarm optimization for unconstrained optimization

This paper proposes the hybrid NM-PSO algorithm based on the Nelder–Mead (NM) simplex search method and particle swarm optimization (PSO) for unconstrained optimization. NM-PSO is very easy to implement in practice since it does not require gradient computation. The modification of both the Nelder–Mead simplex search method and particle swarm optimization intends to produce faster and more accurate convergence. The main purpose of the paper is to demonstrate how the standard particle swarm optimizers can be improved by incorporating a hybridization strategy. In a suite of 20 test function problems taken from the literature, computational results via a comprehensive experimental study, preceded by the investigation of parameter selection, show that the hybrid NM-PSO approach outperforms other three relevant search techniques (i.e., the original NM simplex search method, the original PSO and the guaranteed convergence particle swarm optimization (GCPSO)) in terms of solution quality and convergence rate. In a later part of the comparative experiment, the NM-PSO algorithm is compared to various most up-to-date cooperative PSO (CPSO) procedures appearing in the literature. The comparison report still largely favors the NM-PSO algorithm in the performance of accuracy, robustness and function evaluation. As evidenced by the overall assessment based on two kinds of computational experience, the new algorithm has demonstrated to be extremely effective and efficient at locating best-practice optimal solutions for unconstrained optimization.

Optimal multi-thresholding using a hybrid optimization approach

The Otsus method has been proven as an efficient method in image segmentation for bi-level thresholding. However, this method is computationally intensive when extended to multi-level thresholding. In this paper, we present a hybrid optimization scheme for multiple thresholding by the criteria of (1) Otsus minimum within-group variance and (2) Gaussian function fitting. Four example images are used to test and illustrate the three different methods: the Otsus method; the NM-PSO-Otsu method, which is the Otsus method with Nelder-Mead simplex search and particle swarm optimization; the NM-PSO-curve method, which is Gaussian curve fitting by Nelder-Mead simplex search and particle swarm optimization. The experimental results show that the NM-PSO-Otsu could expedite the Otsus method efficiently to a great extent in the case of multi-level thresholding, and that the NM-PSO-curve method could provide better effectiveness than the Otsus method in the context of visualization, object size and image contrast.

Hybrid simplex search and particle swarm optimization for the global optimization of multimodal functions

This article proposes the hybrid Nelder–Mead (NM)–Particle Swarm Optimization (PSO) algorithm based on the NM simplex search method and PSO for the optimization of multimodal functions. The hybrid NM–PSO algorithm is very easy to implement, in practice, since it does not require gradient computation. This hybrid procedure performed the exploration with PSO and the exploitation with the NM simplex search method. In a suite of 17 multi-optima test functions taken from the literature, the computational results via various experimental studies showed that the hybrid NM–PSO approach is superior to the two original search techniques (i.e. NM and PSO) in terms of solution quality and convergence rate. In addition, the presented algorithm is also compared with eight other published methods, such as hybrid genetic algorithm (GA), continuous GA, simulated annealing (SA), and tabu search (TS) by means of a smaller set of test functions. On the whole, the new algorithm is demonstrated to be extremely effective and efficient at locating best-practice optimal solutions for multimodal functions.

A genetic algorithm and a particle swarm optimizer hybridized with Nelder-Mead simplex search

This paper integrates Nelder-Mead simplex search method (NM) with genetic algorithm (GA) and particle swarm optimization (PSO), respectively, in an attempt to locate the global optimal solutions for the nonlinear continuous variable functions mainly focusing on response surface methodology (RSM). Both the hybrid NM-GA and NM-PSO algorithms incorporate concepts from the NM, GA or PSO, which are readily to implement in practice and the computation of functional derivatives is not necessary. The hybrid methods were first illustrated through four test functions from the RSM literature and were compared with original NM, GA and PSO algorithms. In each test scheme, the effectiveness, efficiency and robustness of these methods were evaluated via associated performance statistics, and the proposed hybrid approaches prove to be very suitable for solving the optimization problems of RSM-type. The hybrid methods were then tested by ten difficult nonlinear continuous functions and were compared with the best known heuristics in the literature. The results show that both hybrid algorithms were able to reach the global optimum in all runs within a comparably computational expense.

A multi-level thresholding approach using a hybrid optimal estimation algorithm

This paper presented a hybrid optimal estimation algorithm for solving multi-level thresholding problems in image segmentation. The distribution of image intensity is modeled as a random variable, which is approximated by a mixture Gaussian model. The Gaussians parameter estimates are iteratively computed by using the proposed PSO+EM algorithm, which consists of two main components: (i) global search by using particle swarm optimization (PSO); (ii) the best particle is updated through expectation maximization (EM) which leads the remaining particles to seek optimal solution in search space. In the PSO+EM algorithm, the parameter estimates fed into EM procedure are obtained from global search performed by PSO, expecting to provide a suitable starting point for EM while fitting the mixture Gaussians model. The preliminary experimental results show that the hybrid PSO+EM algorithm could solve the multi-level thresholding problem quite swiftly, and also provide quality thresholding outputs for complex images.

Image thresholding using a novel estimation method in generalized Gaussian distribution mixture modeling

The generalized Gaussian distribution (GGD) mixture model is a parametric statistical model, which is frequently employed to characterize the statistical behavior of a process signal in industry. This paper considers the GGD mixture model to approximate the empirical distributions, especially for those arising from non-Gaussian sources. A new estimation method is developed for fitting the GGD mixture model. The proposed method integrates Particle Swarm Optimization (PSO) from Computational Intelligence and Entropy Matching Estimator (EME) from Statistical Computation to seek the optimal parameter estimates, particularly when there is at least one large shape parameter in the GGD mixture model. Thus, the method is termed PSO+EME. Applications to multi-level thresholding in image processing are used to illustrate PSO+EME. Image thresholding is a useful technique to separate the interested object from background information. Due to the versatility of the GGD mixture model in characterizing process signals, it is chosen to fit the intensity of image and PSO+EME is used to estimate the parameters. The experimental study shows that the fitted model produced by PSO+EME could depicts quite successfully the non-Gaussian probability density function of image intensity, and therefore present quality effectiveness in the problem of multi-level thresholding.

A decreasing inertia weight particle swarm optimizer

It has been over ten years since the pioneering work of particle swarm optimization (PSO) espoused by Kennedy and Eberhart. Since then, various modifications, well suited to particular application areas, have been reported widely in the literature. The evolutionary concept of PSO is clear-cut in nature, easy to implement in practice, and computationally efficient in comparison to other evolutionary algorithms. The above-mentioned merits are primarily the motivation of this article to investigate PSO when applied to continuous optimization problems. The performance of conventional PSO on the solution quality and convergence speed deteriorates when the function to be optimized is multimodal or with a large problem size. Toward that end, it is of great practical value to develop a modified particle swarm optimizer suitable for solving high-dimensional, multimodal optimization problems. In the first part of the article, the design of experiments (DOE) has been conducted comprehensively to examine the influences of each parameter in PSO. Based upon the DOE results, a modified PSO algorithm, termed Decreasing-Weight Particle Swarm Optimization (DW-PSO), is addressed. Two performance measures, the success rate and number of function evaluations, are used to evaluate the proposed method. The computational comparisons with the existing PSO algorithms show that DW-PSO exhibits a noticeable advantage, especially when it is performed to solve high-dimensional problems.

A parallel particle swarm optimization algorithm for multi-objective optimization problems

In this article, a new proposal of using particle swarm optimization algorithms to solve multi-objective optimization problems is presented. The algorithm is constructed based on the concept of Pareto dominance, as well as a state-of-the-art ‘parallel’ computing technique that intends to improve algorithmic effectiveness and efficiency simultaneously. The proposed parallel particle swarm multi-objective evolutionary algorithm (PPS-MOEA) is tested through a variety of standard test functions taken from the literature; its performance is compared with six noted multi-objective algorithms. The computational experience gained from the first two experiments indicates that the algorithm proposed in this article is extremely competitive when compared with other MOEAs, being able to accurately, reliably and robustly approximate the true Pareto front in almost every tested case. To justify the motivation behind the research of the parallel swarm structure, the computational results of the third experiment confirm the PPS-MOEAs merit in solving really high-dimensional multi-objective optimization problems.

Dynamic multi-swarm particle swarm optimizer using parallel PC cluster systems for global optimization of large-scale multimodal functions

This article presents a novel parallel multi-swarm optimization (PMSO) algorithm with the aim of enhancing the search ability of standard single-swarm PSOs for global optimization of very large-scale multimodal functions. Different from the existing multi-swarm structures, the multiple swarms work in parallel, and the search space is partitioned evenly and dynamically assigned in a weighted manner via the roulette wheel selection (RWS) mechanism. This parallel, distributed framework of the PMSO algorithm is developed based on a master–slave paradigm, which is implemented on a cluster of PCs using message passing interface (MPI) for information interchange among swarms. The PMSO algorithm handles multiple swarms simultaneously and each swarm performs PSO operations of its own independently. In particular, one swarm is designated for global search and the others are for local search. The first part of the experimental comparison is made among the PMSO, standard PSO, and two state-of-the-art algorithms (CTSS and CLPSO) in terms of various un-rotated and rotated benchmark functions taken from the literature. In the second part, the proposed multi-swarm algorithm is tested on large-scale multimodal benchmark functions up to 300 dimensions. The results of the PMSO algorithm show great promise in solving high-dimensional problems.

Automatic detection of Mura defect in TFT-LCD based on regression diagnostics

This paper proposes a computationally efficient Mura defect detection method that is constructed based on regression diagnostics using the prediction error sum of squares (PRESS) residuals and an image estimation procedure for automatic Mura inspection of thin film transistor liquid crystal display (TFT-LCD) devices. The gray-level data of the input image is estimated by a linear model and then the PRESS residuals are calculated to filter Mura regions out. After image dilation, the threshold value is determined for detecting the non-uniform brightness or darkness areas in TFT-LCD by means of examining every pixel in the image. The experimental results of several test images returned by using the proposed method and an existing method in the literature are used to evaluate the performance of effectiveness and efficiency for Mura detection. It has been found that the method proposed in this paper is very swift in processing time and also returns competitive Mura detection performance in comparison to the exiting method.