Wei-Qi Lin

Hybridized particle swarm algorithm for adaptive structure training of multilayer feed-forward neural network: QSAR studies of bioactivity of organic compounds.

The multilayer feed‐forward ANN is an important modeling technique used in QSAR studying. The training of ANN is usually carried out only to optimize the weights of the neural network and without paying attention to the network topology. Some other strategies used to train ANN are, first, to discover an optimum structure of the network, and then to find weights for an already defined structure. These methods tend to converge to local optima, and may also lead to overfitting. In this article, a hybridized particle swarm optimization (PSO) approach was applied to the neural network structure training (HPSONN). The continuous version of PSO was used for the weight training of ANN, and the modified discrete PSO was applied to find appropriate the network architecture. The network structure and connectivity are trained simultaneously. The two versions of PSO can jointly search the global optimal ANN architecture and weights. A new objective function is formulated to determine the appropriate network architecture and optimum value of the weights. The proposed HPSONN algorithm was used to predict carcinogenic potency of aromatic amines and biological activity of a series of distamycin and distamycin‐like derivatives. The results were compared to those obtained by PSO and GA training in which the network architecture was kept fixed. The comparison demonstrated that the HPSONN is a useful tool for training ANN, which converges quickly towards the optimal position, and can avoid overfitting in some extent.

Optimized Block-wise Variable Combination by Particle Swarm Optimization for Partial Least Squares Modeling in Quantitative Structure−Activity Relationship Studies

The use of numerous descriptors that are indicative of molecular structure is becoming common in quantitative structure-activity relationship (QSAR) studies. As all of the descriptors might carry more or less molecular information, it seems more advisable to investigate the possible variable combination rather than variable selection. In this paper, an optimized block-wise variable combination (OBVC) by particle swarm optimization based on partial least squares modeling has been proposed for variable combination. An F statistic is also introduced to determine the dimensionality of the PLS model. The performance is assessed using two QSAR data sets. Experimental results have shown the good performance of this technique compared to those obtained by stepwise regression.

Artificial neural network-based transformation for nonlinear partial least-square regression with application to QSAR studies

In the present study a new version of nonlinear partial least-square method based on artificial neural network transformation (ANN-NLPLS) has been proposed. This algorithm firstly transforms the training descriptors into the hidden layer outputs using the universal nonlinear mapping carried by an artificial neural network, and then utilizes PLS to relate the outputs of the hidden layer to the bioactivities. The weights between the input and hidden layers are optimized by a particle swarm optimization (PSO) method using the criterion of minimized model error via PLS modeling. An F-statistic is introduced to determine automatically the number of PLS components during the weight optimization. The performance is assessed using a simulated data set and two quantitative structure-activity relation (QSAR) data sets. Results of these three data sets demonstrate that ANN-NLPLS offers enhanced capacity in modeling nonlinearity while circumventing the overfitting frequently involved in nonlinear modeling.

Piecewise hypersphere modeling by particle swarm optimization in QSAR studies of bioactivities of chemical compounds.

As the structural diversity in a quantitative structure-activity relationship (QSAR) model increases, constructing a good model becomes increasingly difficult, and simply performing variable selection might not be sufficient to improve the model quality to make it practically usable. To combat this difficulty, an approach based on piecewise hypersphere modeling by particle swarm optimization (PHMPSO) is developed in this paper. It treats the linear models describing the sought-for subsets as hyperspheres which have different radii in the data space. According to the attribute of each hypersphere, all compounds in the training set are allocated to hyperspheres to construct submodels, and particle swarm optimization (PSO) is applied to search the optimal hyperspheres for finding satisfactory piecewise linear models. A new objective function is formulated to determine the appropriate piecewise models. The performance is assessed using three QSAR data sets. Experimental results have shown the good performance of this technique in improving the QSAR modeling.

Optimized sample-weighted partial least squares

In ordinary multivariate calibration methods, when the calibration set is determined to build the model describing the relationship between the dependent variables and the predictor variables, each sample in the calibration set makes the same contribution to the model, where the difference of representativeness between the samples is ignored. In this paper, by introducing the concept of weighted sampling into partial least squares (PLS), a new multivariate regression method, optimized sample-weighted PLS (OSWPLS) is proposed. OSWPLS differs from PLS in that it builds a new calibration set, where each sample in the original calibration set is weighted differently to account for its representativeness to improve the prediction ability of the algorithm. A recently suggested global optimization algorithm, particle swarm optimization (PSO) algorithm is used to search for the best sample weights to optimize the calibration of the original training set and the prediction of an independent validation set. The proposed method is applied to two real data sets and compared with the results of PLS, the most significant improvement is obtained for the meat data, where the root mean squared error of prediction (RMSEP) is reduced from 3.03 to 2.35. For the fuel data, OSWPLS can also perform slightly better or no worse than PLS for the prediction of the four analytes. The stability and efficiency of OSWPLS is also studied, the results demonstrate that the proposed method can obtain desirable results within moderate PSO cycles.

Support vector machine based training of multilayer feedforward neural networks as optimized by particle swarm algorithm: application in QSAR studies of bioactivity of organic compounds.

Multilayer feedforward neural networks (MLFNNs) are important modeling techniques widely used in QSAR studies for their ability to represent nonlinear relationships between descriptors and activity. However, the problems of overfitting and premature convergence to local optima still pose great challenges in the practice of MLFNNs. To circumvent these problems, a support vector machine (SVM) based training algorithm for MLFNNs has been developed with the incorporation of particle swarm optimization (PSO). The introduction of the SVM based training mechanism imparts the developed algorithm with inherent capacity for combating the overfitting problem. Moreover, with the implementation of PSO for searching the optimal network weights, the SVM based learning algorithm shows relatively high efficiency in converging to the optima. The proposed algorithm has been evaluated using the Hansch data set. Application to QSAR studies of the activity of COX‐2 inhibitors is also demonstrated. The results reveal that this technique provides superior performance to backpropagation (BP) and PSO training neural networks.

Adaptive configuring of radial basis function network by hybrid particle swarm algorithm for QSAR studies of organic compounds

The configuring of a radial basis function network (RBFN) consists of selecting the network parameters (centers and widths in RBF units and weights between the hidden and output layers) and network architecture. The issues of suboptimum and overfitting, however, often occur in RBFN configuring. This paper presented a hybrid particle swarm optimization (HPSO) algorithm to simultaneously search the optimal network structure and parameters involved in the RBFN (HPSORBFN) with an ellipsoidal Gaussian function as a basis function. The continuous version of PSO was used for parameter training, while the modified discrete PSO was employed to determine the appropriate network topology. The proposed HPSORBFN algorithm was applied to modeling the inhibitory activities of substituted bis[(acridine-4-carboxamide)propyl]methylamines to murine P388 leukemia cells and the bioactivities of COX-2 inhibitors. The results were compared with those obtained from RBFNs with the parameters optimized by continuous PSO and by conventionally RBFN training the algorithm for a fixed network topology, indicating that the HPSO was competent for RBFN configuring in that it converged quickly toward the optimal solution and avoided overfitting.