Daehyon Kim

Normalization methods for input and output vectors in backpropagation neural networks

Neural networks have been increasingly applied to many problems in many areas, and Backpropagation has been the most popular neural network model. Despite its wide application, there are some major issues to be considered before using the model, such as the network topology, learning parameter, and normalization methods for the input and output vectors. Input and output vectors for Backpropagation need to be normalized properly in order to achieve the best performance of the network. In this research, several normalization methods have been studied theoretically and two methods have been compared for performance in terms of prediction accuracy on the test sets through experiments with real world image data

Prediction performance of support vector machines on input vector normalization methods

Support vector machines (SVM) based on the statistical learning theory is currently one of the most popular and efficient approaches for pattern recognition problem, because of their remarkable performance in terms of prediction accuracy. It is, however, required to choose a proper normalization method for input vectors in order to improve the system performance. Various normalization methods for SVMs have been studied in this research and the results showed that the normalization methods could affect the prediction performance. The results could be useful for determining a proper normalization method to achieve the best performance in SVMs.

Improving prediction performance of neural networks in pattern classification

Neural networks that are especially useful for mapping problems requiring tolerance of some errors and deriving the computing power through their massively parallel-distributed structure have been one of the most efficient approaches for many problems. Moreover, backpropagation is one of the most popular neural networks and is widely applied in various problems. Despite of the many successful applications of backpropagation, it has some drawbacks. One of the serious problems of the backpropagation model is that it is sensitive to the initial value of the weights. The performance in terms of prediction accuracy and computing cost highly depends on the initial weights. Nevertheless, until now, there has been no solution to the significant different performance according to the initial weight configuration. In this article, a prediction rule has been proposed to minimize the effect of initial weights and improve the prediction accuracy on the test data sets. According to the experimental results, a significant improvement of the prediction performance has been achieved by using a proposed prediction rule. The proposed rule could be useful for many other applications of backpropagation to achieve the best performance.

Problems Encountered during Implementation of the Backpropagation

Backpropagation is one of the most popular neural networks and is widely applied in various problems. Despite its wide application, there are some major problems encountered during implementation of Backpropagation such as the appropriate decision for the network topology, learning parameter, initial weights, and data sets for training. Although the network performance is sensitive to these elements which should be determined prior to the implementation of Backpropagation, there is no efficient rule for the proper choice of these to achieve the best performance. In this research, the effects of the learning rate, the learning mode, network topology, initial weights, and error goal for stopping the training have been studied through various experiments. Some significant results, which could be efficient for many applications of Backpropagation, were achieved in this research.

STANDARD AND ADVANCED BACKPROPAGATION MODELS FOR IMAGE PROCESSING APPLICATION IN TRAFFIC ENGINEERING

Neural networks have been increasingly applied to many problems in transport planning engineering and the feedforward network with the error backpropagation learning rule, usually called simply “Backpropagation,” has been the most popular neural network. Backpropagation is easy to implement and has been shown to produce relatively good results in many applications. It is capable of approximating arbitrary nonlinear mappings. However, it is noted that one serious disadvantage in the standard Backpropagation is the slow rate of convergence, requiring very long training times. In order to overcome the long training time and susceptibility to trapping at local minima, several enhanced Backpropagation models have been proposed. In this research, the standard Backpropagation and three enhanced Backpropagation models, Backpropagation with Momentum, Quickprop, and Backpropagation with Momentum & Prime-offset (BPMP), have been studied to compare their performance in terms of computing cost and predictive accuracy.

Performance comparison of neural network models: backpropagation vs. fuzzy artmap

Neural networks have been increasingly applied to many problems in civil engineering. Even though there are currently many different types of neural network models, Backpropagation is the most popular neural network model. It is also known that Fuzzy ARTMAP, which is a combination of fuzzy logic and Adaptive Resonance Theory (ART), is superior to any other neural network models in terms of computing cost and predictive accuracy. In this research, two neural network paradigms, Backpropagation and Fuzzy ARTMAP have been studied to compare their performance in terms of computing cost and predictive accuracy through the experiment with real world image data of traffic scenes, as well as biological and theoretical aspects. In addition, three enhanced Backpropagation models, Backpropagation with Momentum, Quickprop, BPMP (Backpropagation with Momentum and Prime-offset) have been considered to compare the network performance of each model.

Normalization methods on backpropagation for the estimation of driver's route choice

The artificial neural network has recently been applied in many areas including transport engineering and planning. Even though its successful application for wide transportation areas, there are some major issues to be considered before using the neural network models, such as the network topology, learning parameter, and normalization methods for the input vectors. In this research, se veral normalization methods for input vectors were studied and the experimental results showed that the performance of the driers ro ute choice model using the neural networks was dependent on the normalization methods. For the estimation of drivers route choice, the best normalization method in the Backpropagation neural network model was suggested in this study.

Parameter Decision for Efficient Automatic Incident Detection System

Incidents on the freeway disrupt traffic flow and the cost of delay caused by incidents is significant. To reduce the impact of an incident a traffic management center needs to quickly detect and remove it from the freeway. Quick and efficient automatic incident detection has been a main goal of the transportation research for many years. Also many algorithms based on loop detector data have been developed tested for the Automatic Incident Detection (AID). However, many of them have a limited success in their overall performance in terms of detection rate, false alarm rate, and the mean time to detect an incident. Recently, the neural network models are known as the one of the popular and efficient approach for real-time automatic incident detection and many researches have shown that the neural network models were much more efficient than various other previous models. More importantly, Support Vector Machine (SVM) which is based on the statistical learning theory, has been shown that it is more efficient than the most popular neural network model, Backpropagation. The important element in the performance of SVM and the neural network is input vectors. However, there has been little research on the performance according to the attribute of input vectors. Normally, the three parameters of volume, speed, occupancy from the field detectors are available for the application of the SVM. The performance of automatic incident detection system is effected by the training data on the number of detectors and time slices. The purpose of this research is to determine a proper number of time slices in order to provide the best performance in automatic incident detection system. The experiments have been done with real world freeway data and the results showed that the 8 time slices could provide the best prediction performance in terms of DR (Detection Rate) and FAR (False Alarm Rate).