Jin-Cheng Li

MLPNN Training via a Multiobjective Optimization of Training Error and Stochastic Sensitivity

The training of a multilayer perceptron neural network (MLPNN) concerns the selection of its architecture and the connection weights via the minimization of both the training error and a penalty term. Different penalty terms have been proposed to control the smoothness of the MLPNN for better generalization capability. However, controlling its smoothness using, for instance, the norm of weights or the Vapnik-Chervonenkis dimension cannot distinguish individual MLPNNs with the same number of free parameters or the same norm. In this paper, to enhance generalization capabilities, we propose a stochastic sensitivity measure (ST-SM) to realize a new penalty term for MLPNN training. The ST-SM determines the expectation of the squared output differences between the training samples and the unseen samples located within their Q -neighborhoods for a given MLPNN. It provides a direct measurement of the MLPNNs output fluctuations, i.e., smoothness. We adopt a two-phase Pareto-based multiobjective training algorithm for minimizing both the training error and the ST-SM as biobjective functions. Experiments on 20 UCI data sets show that the MLPNNs trained by the proposed algorithm yield better accuracies on testing data than several recent and classical MLPNN training methods.

Bi-firing deep neural networks

Deep neural networks provide more expressive power in comparison to shallow ones. However, current activation functions can not propagate error using gradient descent efficiently with the increment of the number of hidden layers. Current activation functions, e.g. sigmoid, have large saturation regions which are insensitive to changes of hidden neuron’s input and yield gradient diffusion. To relief these problems, we propose a bi-firing activation function in this work. The bi-firing function is a differentiable function with a very small saturation region. Experimental results show that deep neural networks with the proposed activation functions yield faster training, better error propagation and better testing accuracies on seven image datasets.

RFID access authorization by face recognition

RFID identification has been widely adopted in access control. This kind of card or tag based approaches has a major drawback that anyone could get access with the card. In this work, we propose a neural network based face recognition system as the second access control to make sure the person granted access matches the ID on the RFID card. In this preliminary work, the face of accessing person is detected in video stream and we extract the Scale Invariant Feature Transform (SIFT) features from a face image. To enhance the generalization capability of the face recognition, we introduced the Localized Generalization Error Model (L-GEM) to train the Radial Basis Function Neural Network (RBFNN) for face recognition. Experimental results show that the proposed method could identify person that matches the RFID access card or not in a high probability.

Video shot boundary detection using RBFNN minimizing the L-GEM

Shot boundary detection (SBD) is the key step of key frame extraction for Content-Based Video Retrieval (CBVR). In this paper, we propose a shot boundary detection method by Radial Basis Function Neural Network (RBFNN) trained via a minimization of the Localized Generalization Error (L-GEM). Frame differences are classified as either boundary or non-boundary by the RBFNN. The statistical features of DC image extracted from each frame are used as the input features describing the frame difference. The proposed SBD method is compared with an existing method using News videos. Experimental results show that the proposed method is effective.

L-GEM based MCS aided candlestick pattern investment strategy in the Shenzhen stock market

An integral part of Chinas economic reforms is the privatization of state-owned enterprises (SOEs) and listing the profitable units of the SOEs on the stock market. The two stock exchanges in Shanghai and Shenzhen were opened nearly twenty years ago. The Shenzhen stock exchange market is young and energetic. Moreover, it practices a T+1 settlement rule instead of real time trade as in Hong Kong or other exchange markets. One important research question is whether there are patterns that can be identified in stock prices that can be used to develop profitable investment strategies. If strategies can be found, then this represents a violation of the efficient market hypothesis. In this work, we propose an investment strategy by using Radial Basis Function Neural Networks (RBFNN) trained by Localized Generalization Error Model (L-GEM) and 4 stock price candlestick patterns. Every base RBFNN in the Multiple Classifier System (MCS) recognizes the occurrence of a particular candlestick pattern and the MCS combines opinions from the 4 base RBFNNs by a weighted sum to provide a final prediction. If the MCS predicts an increase for the next day, it will buy the stock and sell it within three days whenever the opening price is higher than the buy-in price or else after three days have passed. Experimental results with stocks in Shenzhen market show that our investment strategy statistically significantly outperforms a random investment, i.e. the EMH is invalid in this case.

A growing architecture selection for Multilayer Perceptron Neural Network by the L-GEM

The number of hidden neurons has a great influence on the generalization capability of Multilayer Perceptron Neural Network (MLPNN). The ultimate goal of building a MLPNN is to recognize (or generalize) future unseen sample correctly based on the training from training samples. Therefore, the Localized Generalization Error Model (L-GEM) is adopted in this work to select the architecture of a MLPNN. The L-GEM has been successfully applied to Radial Basis Function Neural Network (RBFNN) architecture selection, feature selection and other applications. In this work, we propose a new L-GEM for MLPNN and demonstrate its application in architecture selection for MLPNN. Experimental results show that the L-GEM based MLPNN architecture selection method outperforms several off-the-shelf methods.

Image classification using L-GEM based RBFNN with local feature keypoints and MPEG-7 descriptors

Image with MPEG-7 descriptors as features may loss local details. In this work, we combine MPEG-7 descriptors with local feature key points to cover both global and local image characteristics. Images are classified by a Radial Basis Function Neural Network (RBFNN) trained via a minimization of Localized Generalization Error Model (L-GEM). In this paper, we extract local feature key points by the Scale Invariant Feature Transform (SIFT). Four color and three texture MPEG-7 descriptors are extracted. Experimental results show that the introduction of local feature key points effectively improves the testing accuracy of image classification.

An experimental study on the selection of Q-value for the L-GEM

Generalization error is very important in machine learning and pattern classification. However, one can not compute the generalization error for a given problem exactly. Therefore, many research efforts have been put to estimate the generalization error for a given classification problem. The Localized Generalization Error Model (L-GEM) is one of the recently proposed analytical generalization error upper bound models. In the L-GEM, an upper bound of generalization error of unseen samples within a Q-neighborhood of training samples is provided. The L-GEM has been widely adopted in many application areas, e.g. image classification, corporate credit risk prediction and construction productivity enhancement in civil engineering. However, the selection of Q value is vital to the success of L-GEM to application problems. In this work, we provide an experimental study on the selection of the Q value and found that Q value equal to half of average of input variances yield a good generalization capability of RBFNN.

Data Mining of Building Electrical Information Based on Radial Basis Function Neural Network

This paper presents a neural network algorithm for data mining in building LV electrical power information. The power information is recorded by web-based power quality monitoring system. Power information is recorded continuously and stored in a central server system. Presently events were identified by power engineers but in the prototype, an expert system will be used to identify events instead. Neural network approach based on the Radial Basis Function Neural Network (RBFNN) was developed to predict power events in the building LV electrical network. The approach provides useful information for facility managers to conduct planning and operation. The proposed algorithm was tested with power data of a commercial building in Hong Kong. The prediction result by using one week of data achieved 75% accuracy. Further works would be conducted to test the algorithm with more data.

Image segmentation with color and texture using RBFNN minimizing the L-GEM

The Internet provides a huge source of images. Not all of them are professionally edited or well organized. This raises the need of image classification and indexing to enhance the efficiency of using those images. To improve the image classification accuracy, image segmentation is important to remove background and noisy parts in an image. In this paper, we propose an image segmentation method by Radial Basis Function Neural Network (RBFNN) based on the Localized Generalization Error Model (L-GEM). Pixels are classified as target object and background by the RBFNN. Color, gradients and texture are used as features for a pixel. Car images are adopted and we target to separate the car from its background and overlapping objects. Comparison of different neighboring size is conducted. In this pilot study, 11×11 is found to be appropriate size for car segmentation