Eric B. Bartlett

Dynamic node architecture learning: an information theoretic approach

Abstract Typically, artificial neural network (ANN) training schemes require network size to be set before learning is initiated. The learning speed and generalization characteristics of ANNs are, however, dependent on this pretraining selection of the network architecture. The training and generalization viability of a specific network can, therefore, only be evaluated posttraining. This work presents an information theoretic method that alleviates this predicament by building the appropriate network architecture dynamically during the training process. The method, called dynamic node architecture learning (DNAL), eliminates the need to select network size before training. Examples illustrate the use and advantages of the information theoretic DNAL approach over static architecture learning (SAL).

Nuclear power plant fault diagnosis using neural networks with error estimation by series association

The accuracy of the diagnosis obtained from a nuclear power plant fault-diagnostic advisor using neural networks is addressed in this paper in order to ensure the credibility of the diagnosis. A new error estimation scheme called error estimation by series association provides a measure of the accuracy associated with the advisors diagnoses. This error estimation is performed by a secondary neural network that is fed both the input features for and the outputs of the advisor. Our error estimation by series association outperforms previous error estimation techniques in providing more accurate confidence information with considerably reduced computational requirements. We demonstrate the extensive usability of our method by applying it to a complicated transient recognition problem of 33 transient scenarios. The simulated transient data at different severities consists of 25 distinct transients for the Duane Arnold Energy Center nuclear power station ranging from a main steam line break to anticipated transient without scram (ATWS) conditions. The fault-diagnostic advisor system with the secondary error prediction network is tested on the transients at various severity levels and degraded noise conditions. The results show that our error estimation scheme provides a useful measure of the validity of the advisors output or diagnosis with considerable reduction in computational requirements over previous error estimation schemes.

Information theoretic subset selection for neural network models

In this work, an information theoretic input variable subset selection (ITSS) scheme for neural network based modeling of chemical processes is proposed. In recent years, artificial neural network models have been shown to be useful empirical models for modeling complex nonlinear chemical processes. ITSS selects an informative subset to be used as input data for constructing a neural network model. ITSS can select appropriate subsets for neural network model development, regardless of the dependencies between the process outputs and inputs. The power of the ITSS method is illustrated through its application to three example problems. Results obtained show that ITSS is capable of identifying subsets for developing viable artificial neural network models. As it uses a smaller set of input variables, ITSS can help identify simpler neural models with better generalization and ease of interpretability.

Detecting faults in a nuclear power plant by using dynamic node architecture artificial neural networks

An artificial neural network (ANN)-based diagnostic adviser capable of identifying the operating status of a nuclear power plant is described. A dynamic node architecture scheme is used to optimize the architectures of the two backpropagation ANNs that embody the advisor. The first or root network is used to determine whether or not the plant is in a normal operating condition. If the plant is not in a normal condition, the second or classifier network is used to recognize the particular off-normal condition or transient taking place. These networks are developed using simulated plant behavior during both normal and abnormal conditions. The adviser is effective at diagnosing 27 distinct transients based on 43 scenarios simulated at various severities that contain up to 3% noise.

A statistically tailored neural network approach to tomographic image reconstruction

In previous work it has been shown that a standard backpropagation neural network can be trained to reconstruct sections of single photon emission computed tomography (SPECT) images based on the planar image projections as inputs. In this study, it is demonstrated that an artificial neural network (ANN) trained on a series of simulated SPECT images or trained on a set of rudimentary geometric images can learn the planar data-to-tomographic image relationship for 64 x 64 tomograms. As a result, a properly trained ANN can produce accurate, novel image reconstructions but without the high computational cost inherent in some traditional reconstruction techniques. We also present a method of deriving activation functions for a backpropagation ANN that make it readily trainable for cardiac SPECT image reconstruction. The activation functions are derived from the estimated probability density functions (p.d.f.s) of the ANN training set data. The performance of the statistically tailored ANNs are compared with the performance of standard sigmoidal back-propagation ANNs, both in terms of their trainability and generalization ability. The results presented demonstrate that statistically tailored ANNs are significantly better than standard sigmoidal ANNs at reconstructing novel tomographic images based on a simulated SPECT image training set or a rudimentary geometric image training set. Neural network based image reconstruction has two potential advantages over conventional reconstruction methods. The first advantage is that ANNs can rapidly reconstruction tomograms. Secondly, the quality of the reconstructions produced are directly correlated to the quality of the images used to train the ANN.

Error Prediction for a Nuclear Power Plant Fault-Diagnostic Advisor Using Neural Networks

AbstractThe objective of this research is to develop a fault-diagnostic advisor for nuclear power plant transients that is based on artificial neural networks. A method is described that provides an error bound and therefore a figure of merit for the diagnosis provided by this advisor. The data used in the development of the advisor contain ten simulated anomalies for the San Onofre Nuclear Power Generating Station. The stacked generalization approach is used with two different partitioning schemes. The results of these partitioning schemes are compared. It is shown that the advisor is capable of recognizing all ten anomalies while providing estimated error bounds on each of its diagnoses.

Error estimation by series association for neural network systems

Estimation of confidence intervals for neural network outputs is important when the uncertainty of a neural network system must be addressed for safety or reliability. This paper presents a new approach for estimating confidence intervals, which can help users validate neural network outputs. The estimation of confidence intervals, called error estimation by series association, is performed by a supplementary neural network trained to predict the error of the main neural network using input features and the output of the main network. The accuracy of this approach is shown using a simple nonlinear mapping and more complicated, realistic nuclear power plant fault diagnosis problems. The results demonstrate that the approach performs confidence estimation successfully.

Neural network reconstruction of single-photon emission computed tomography images.

An artificial neural network (ANN) trained on high-quality medical tomograms or phantom images may be able to learn the planar data-to-tomographic image relationship with very high precision. As a result, a properly trained ANN can produce comparably accurate image reconstruction without the high computational cost inherent in some traditional reconstruction techniques. We have previously shown that a standard backpropagation neural network can be trained to reconstruct sections of single photon emission computed tomography (SPECT) images based on the planar image projections as inputs. In this study, we present a method of deriving activation functions for a backpropagation ANN that make it readily trainable for full SPECT image reconstruction. The activation functions used for this work are based on the estimated probability density functions (PDFs) of the ANN training set data. The statistically tailored ANN and the standard sigmoidal backpropagation ANN methods are compared both in terms of their trainability and generalization ability. The results presented show that a statistically tailored ANN can reconstruct novel tomographic images of a quality comparable with that of the images used to train the network. Ultimately, an adequately trained ANN should be able to properly compensate for physical photon transport effects, background noise, and artifacts while reconstructing the tomographic image.

An artificial neural network for wavelet steganalysis

Hiding messages in image data, called steganography, is used for both legal and illicit purposes. The detection of hidden messages in image data stored on websites and computers, called steganalysis, is of prime importance to cyber forensics personnel. Automating the detection of hidden messages is a requirement, since the shear amount of image data stored on computers or websites makes it impossible for a person to investigate each image separately. This paper describes research on a prototype software system that automatically classifies an image as having hidden information or not, using a sophisticated artificial neural network (ANN) system. An ANN software package, the ISU ACL NetWorks Toolkit, is trained on a selection of image features that distinguish between stego and nonstego images. The novelty of this ANN is that it is a blind classifier that gives more accurate results than previous systems. It can detect messages hidden using a variety of different types of embedding algorithms. A Graphical User Interface (GUI) combines the ANN, feature selection, and embedding algorithms into a prototype software package that is not currently available to the cyber forensics community.

A stochastic training algorithm for artificial neural networks

Abstract Random optimization methods typically use Gaussian probability density functions (PDFs) to generate random search vectors. In this paper the random search technique is modified to dynamically seek out the optimal probability density function (OPDF) from which to select the search vector. To this end, the optimization problem is posed as an integral, and the theory of Monte Carlo importance function biasing is applied to determine the theoretical OPDF from which to select changes in the parameters to be optimized. The approach updates its estimate of the OPDF as more information is gained during each training cycle. The dynamic OPDF search process, combined with an adaptive stratified sampling technique, completes the modifications of the basic method. The approach is applied to layered artificial neural networks of generalized, fully interconnected, continuous perceptions and is benchmarked against the backpropagation training method.