Maciej Kusy

Application of Reinforcement Learning Algorithms for the Adaptive Computation of the Smoothing Parameter for Probabilistic Neural Network

In this paper, we propose new methods for the choice and adaptation of the smoothing parameter of the probabilistic neural network (PNN). These methods are based on three reinforcement learning algorithms: Q(0)-learning, Q(λ)-learning, and stateless Q-learning. We regard three types of PNN classifiers: the model that uses single smoothing parameter for the whole network, the model that utilizes single smoothing parameter for each data attribute, and the model that possesses the matrix of smoothing parameters different for each data variable and data class. Reinforcement learning is applied as the method of finding such a value of the smoothing parameter, which ensures the maximization of the prediction ability. PNN models with smoothing parameters computed according to the proposed algorithms are tested on eight databases by calculating the test error with the use of the cross validation procedure. The results are compared with state-of-the-art methods for PNN training published in the literature up to date and, additionally, with PNN whose sigma is determined by means of the conjugate gradient approach. The results demonstrate that the proposed approaches can be used as alternative PNN training procedures.

Probabilistic neural network training procedure based on Q(0)-learning algorithm in medical data classification

In this article, an iterative procedure is proposed for the training process of the probabilistic neural network (PNN). In each stage of this procedure, the Q(0)-learning algorithm is utilized for the adaptation of PNN smoothing parameter (σ). Four classes of PNN models are regarded in this study. In the case of the first, simplest model, the smoothing parameter takes the form of a scalar; for the second model, σ is a vector whose elements are computed with respect to the class index; the third considered model has the smoothing parameter vector for which all components are determined depending on each input attribute; finally, the last and the most complex of the analyzed networks, uses the matrix of smoothing parameters where each element is dependent on both class and input feature index. The main idea of the presented approach is based on the appropriate update of the smoothing parameter values according to the Q(0)-learning algorithm. The proposed procedure is verified on six repository data sets. The prediction ability of the algorithm is assessed by computing the test accuracy on 10 %, 20 %, 30 %, and 40 % of examples drawn randomly from each input data set. The results are compared with the test accuracy obtained by PNN trained using the conjugate gradient procedure, support vector machine algorithm, gene expression programming classifier, k–Means method, multilayer perceptron, radial basis function neural network and learning vector quantization neural network. It is shown that the presented procedure can be applied to the automatic adaptation of the smoothing parameter of each of the considered PNN models and that this is an alternative training method. PNN trained by the Q(0)-learning based approach constitutes a classifier which can be treated as one of the top models in data classification problems.

Probabilistic Neural Network Structure Reduction for Medical Data Classification

Probabilistic neural network (PNN) consists of the number of pattern neurons that equals the cardinality of the data set. The model design is therefore complex for large database classification problems. In this article, two effective PNN reduction procedures are introduced. In the first approach, the PNN’s pattern layer neurons are reduced by means of a k-means clustering procedure. The second method uses a support vector machines algorithm to select pattern layer nodes. Modified PNN networks are compared with the original model in medical data classification problems. The prediction ability expressed in terms of the 20% test set error for the networks is assessed. By means of the experiments, it is shown that the appropriate pruning of the pattern layer neurons in the PNN enhances the performance of the classifier.

Sensitivity Analysis for Probabilistic Neural Network Structure Reduction

In this paper, we propose the use of local sensitivity analysis (LSA) for the structure simplification of the probabilistic neural network (PNN). Three algorithms are introduced. The first algorithm applies LSA to the PNN input layer reduction by selecting significant features of input patterns. The second algorithm utilizes LSA to remove redundant pattern neurons of the network. The third algorithm combines the proposed two and constitutes the solution of how they can work together. PNN with a product kernel estimator is used, where each multiplicand computes a one-dimensional Cauchy function. Therefore, the smoothing parameter is separately calculated for each dimension by means of the plug-in method. The classification qualities of the reduced and full structure PNN are compared. Furthermore, we evaluate the performance of PNN, for which global sensitivity analysis (GSA) and the common reduction methods are applied, both in the input layer and the pattern layer. The models are tested on the classification problems of eight repository data sets. A 10-fold cross validation procedure is used to determine the prediction ability of the networks. Based on the obtained results, it is shown that the LSA can be used as an alternative PNN reduction approach.

CNC Milling Tool Head Imbalance Prediction Using Computational Intelligence Methods

In this paper, a mechanical imbalance prediction problem for a milling tool heads used in Computer Numerical Control (CNC) machines was studied. Four classes of the head imbalance were examined. The data set included 27334 records with 14 features in the time and frequency domains. The feature selection procedure was applied in order to extract the most significant attributes. Only 3 out of 14 attributes were selected and utilized for the representation of each signal. Seven computational intelligence methods were applied in the prediction task: K–Means clustering algorithm, probabilistic neural network, single decision tree, boosted decision trees, multilayer perceptron, radial basis function neural network and support vector machine. The accuracy, sensitivity and specificity were computed in order to asses the performance of the algorithms.

Prediction of radical hysterectomy complications for cervical cancer using computational intelligence methods

In this work, eleven classifiers were tested in the prediction of intra- and post-operative complications in women with cervical cancer. For the real data set the normalization of the input variables was applied, the feature selection was performed and the original data set was binarized. The simulation showed the best model satisfying the quality criteria such as: the average value and the standard deviation of the error, the area under ROC curve, sensitivity and specificity. The results can be useful in clinical practice.

Weighted probabilistic neural network

Abstract In this work, the modification of the probabilistic neural network (PNN) is proposed. The traditional network is adjusted by introducing the weight coefficients between pattern and summation layer. The weights are derived using the sensitivity analysis (SA) procedure. The performance of the weighted PNN (WPNN) is examined in data classification problems on benchmark data sets. The obtained WPNN’s efficiency results are compared with these achieved by a modified PNN model put forward in literature, the original PNN and selected state-of-the-art classification algorithms: support vector machine, multilayer perceptron, radial basis function neural network, k-nearest neighbor method and gene expression programming algorithm. All classifiers are collated by computing the prediction accuracy obtained with the use of a k-fold cross validation procedure. It is shown that in seven out of ten classification cases, WPNN outperforms both the weighted PNN classifier introduced in literature and the original model. Furthermore, according to the ranking statistics, the proposed WPNN takes the first place among all tested algorithms.

Prediction of 5–year overall survival in cervical cancer patients treated with radical hysterectomy using computational intelligence methods

BackgroundComputational intelligence methods, including non-linear classification algorithms, can be used in medical research and practice as a decision making tool. This study aimed to evaluate the usefulness of artificial intelligence models for 5–year overall survival prediction in patients with cervical cancer treated by radical hysterectomy.MethodsThe data set was collected from 102 patients with cervical cancer FIGO stage IA2-IIB, that underwent primary surgical treatment. Twenty-three demographic, tumor-related parameters and selected perioperative data of each patient were collected. The simulations involved six computational intelligence methods: the probabilistic neural network (PNN), multilayer perceptron network, gene expression programming classifier, support vector machines algorithm, radial basis function neural network and k-Means algorithm. The prediction ability of the models was determined based on the accuracy, sensitivity, specificity, as well as the area under the receiver operating characteristic curve. The results of the computational intelligence methods were compared with the results of linear regression analysis as a reference model.ResultsThe best results were obtained by the PNN model. This neural network provided very high prediction ability with an accuracy of 0.892 and sensitivity of 0.975. The area under the receiver operating characteristics curve of PNN was also high, 0.818. The outcomes obtained by other classifiers were markedly worse.ConclusionsThe PNN model is an effective tool for predicting 5–year overall survival in cervical cancer patients treated with radical hysterectomy.

Modification of the probabilistic neural network with the use of sensitivity analysis procedure

In this article, the modified probabilistic neural network (MPNN) is proposed. The network is an extension of conventional PNN with the weight coefficients introduced between pattern and summation layer of the model. These weights are calculated by using the sensitivity analysis (SA) procedure. MPNN is employed to the classification tasks and its performance is assessed on the basis of prediction accuracy. The effectiveness of MPNN is also verified by analyzing its results with these obtained for both the original PNN and commonly known classification algorithms: support vector machine, multilayer perceptron, radial basis function network and k-Means clustering procedure. It is shown that the proposed modification improves the prediction ability of the PNN classifier.

Influence of graphical weights’ interpretation and filtration algorithms on generalization ability of neural networks applied to digit recognition

In this paper, the method of the graphical interpretation of the single-layer network weights is introduced. It is shown that the network parameters can be converted to the image and their particular elements are the pixels. For this purpose, weight-to-pixel conversion formula is used. Moreover, new weights’ modification method is proposed. The weight coefficients are computed on the basis of pixel values for which image filtration algorithms are implemented. The approach is applied to the weights of three types of the models: single-layer network, two-layer backpropagation network and the hybrid network. The performance of the models is then compared on two independent data sets. By means of the experiments, it is presented that the adjustment of the weights to new values decreases test error value compared to the error obtained for initial set of weights.