Francisco J. Maldonado

An integrated growing-pruning method for feedforward network training

In order to facilitate complexity optimization in feedforward networks, several algorithms are developed that combine growing and pruning. First, a growing scheme is presented which iteratively adds new hidden units to full-trained networks. Then, a non-heuristic one-pass pruning technique is presented, which utilizes orthogonal least squares. Based upon pruning, a one-pass approach is developed for generating the validation error versus network size curve. A combined approach is described in which networks are continually pruned during the growing process. As a result, the hidden units are ordered according to their usefulness, and the least useful units are eliminated. Examples show that networks designed using the combined method have less training and validation error than growing or pruning alone. The combined method exhibits reduced sensitivity to the initial weights and generates an almost monotonic error versus network size curve. It is shown to perform better than two well-known growing methods-constructive backpropagation and cascade correlation.

Finding optimal neural network basis function subsets using the Schmidt procedure

In designing feedforward neural networks, one often trains a large network and then prunes less useful hidden units. In this paper, two non-heuristic pruning algorithms are derived from the Schmidt procedure. In both, orthonormal systems of basis functions are found, ordered, pruned, and mapped back to the original network. In the first algorithm, the orthonormal basis functions are found and ordered one at a time. In optimal pruning, the best subset of orthonormal basis functions is found for each size network. Linear dependency of basis functions is considered and computational cost is analyzed. Simulation results are given.

Distributed intelligent health monitoring with the coremicro Reconfigurable Embedded Smart Sensor Node

Condition monitoring systems capable of efficiently and accurately diagnosing and identifying faults is a current need for ensuring the proper operation of critical systems. Distributed health monitoring leveraging large sensor networks that provide validated data ensures the proper operation and performance of systems. A key consideration is to have non-intrusive embedded sensors that can be easily added or removed. These needs have motivated the realization of a distributed intelligent health monitoring framework described in this paper based on standardized methods, advanced health monitoring functions at the sensor and system levels, and a state-of-the-art low-power miniature smart sensor (termed the coremicro Reconfigurable Embedded Smart Sensor Node). Major involved technologies consist of: (a) miniature embedded hardware; (b) embedded sensor health monitoring functions (e.g. sensor self-diagnostics, self-healing, and calibration); (c) distributed and intelligent health monitoring at the various system levels; (d) standardized design and communications leveraging the IEEE 1451 standards; and (e) an efficient anomaly awareness mechanism that merges the health monitoring and standardized design aspects.

Upper Bound on Pattern Storage in Feedforward Networks

Starting from the strict interpolation equations for multivariate polynomials, an upper bound is developed for the number of patterns that can be memorized by a nonlinear feedforward network. A straightforward proof by contradiction is presented for the upper bound. It is shown that the hidden activations do not have to be analytic. Networks, trained by conjugate gradient, are used to demonstrate the tightness of the bound for random patterns. Based upon the upper bound, small multilayer perceptron models are successfully demonstrated for large support vector machines.

Predictive Fault Diagnosis System for Intelligent and Robust Health Monitoring

This paper describes work related to the Predictive Fault Diagnosis System for Intelligent and Robust Health Monitoring, which exists as a solution to complete failure detection, identification, and prognostics (FDI&P) in health monitoring applications. Several advanced FDI analytical redundancy techniques have been applied for such a purpose, with the most notable being a compound method comprised of optimal filtering, statistical analysis, and neuro-fuzzy algorithms that is able to detect and diagnose both known and unknown failures. Although this scheme has been proven to be quite successful for systems that can be well described in a state space representation, current research has shown the viability in extending the process to highly complex systems with considerable nonlinearities while still maintaining the FDI capabilities. This paper highlights the utility of these algorithms for determining failures in (1) a known reusable liquid rocket engine model and (2) an unknown input-output relation in a fluid flow testbed. Other research has focused on prognostic capabilities provided by a neural architecture enhanced with fuzzy logic using rule-based knowledge. An example of using data to construct fuzzy rules for determining the remaining useful life of components is provided to give insight into the process.

Letters: Upper bound on pattern storage in feedforward networks

An upper bound on pattern storage is stated for nonlinear feedforward networks with analytic activation functions, like the multilayer perceptron and radial basis function network. The bound is given in terms of the number of network weights, and applies to networks having any number of output nodes and arbitrary connectivity. Starting from the strict interpolation equations and exact finite degree polynomial models for the hidden units, a straightforward proof by contradiction is developed for the upper bound. Several networks, trained by conjugate gradient, are used to demonstrate the tightness of the bound for random patterns.

Self-learning and neural network adaptation by embedded collaborative learning engine (eCLE) — An overview

This paper provides an overview of a novel scheme for constructing machine evolutionary behavior within systems. Specifically, evolving learning for the autonomous recognition of both known as well as newly emerging behaviors is provided. The paper is related with several open research problems such as cognition, incremental learning, and self-learning within the context of health monitoring systems (fault diagnosis and prognosis). Also, it is addressed the need for a formal methodology and its implementation for adding new knowledge, thus enabling the automated recognition of new patterns (e.g. behaviors) within systems. A key feature of the resulting embedded Collaborative Learning Engine (eCLE) when generating machine evolutionary behavior consists of operating with an ensemble of learning paradigms, which when instantiated work in a collaborative way. The resulting framework not only compiles the inherent advantages of the involved methods, but also enables synergistic behavior by working in a collaborative fashion.

Optimized neuro genetic fast estimator (ONGFE) for efficient distributed intelligence instantiation within embedded systems

The Optimized Neuro Genetic Fast Estimator (ONGFE) is a software tool that allows for embedding system, subsystem, and component failure detection, identification, and prognostics (FDI&P) capability by using Intelligent Software Elements (ISE) based upon Artificial Neural Networks (ANN). With an Application Programming Interface (API), highly innovative algorithms are compiled for efficient distributed intelligence instantiation within embedded systems. The original design had the purpose of providing a real time kernel to deploy health monitoring functions for Condition Based Maintenance (CBM) and Real Time Monitoring (RTM) systems in a broad variety of applications (such as aerospace, structural, and widely distributed support systems). The ONGFE contains embedded fast and on-line training for designing ANNs to perform several high performance FDI&P functions. A key advantage of this technology is an optimization block based upon pseudogenetic algorithms which compensate for effects due to initial weight values and local minimums without the computational burden of genetic algorithms. The ONGFE also provides a synchronization block for communication with secondary diagnostic modules. The algorithms are designed for a distributed, scalar, and modular deployment. Based on this technology, a scheme for conducting sensor data validation has been embedded in Smart Sensors.

Enhancing vibration analysis by embedded sensor data validation technologies

The ability to predict and understand the responses of an airframe with simultaneous external influences acting on its elements is a challenging effort. Moreover, although considerable research has been devoted to monitoring structures within the aerospace industry (commercial, military, and space), successful field implementations have not been widely achieved. Breakthroughs for in-flight measurement techniques and processing tools are thus required for enhancing flight research and to ultimately boost operations, increase safety, and reduce costs. This article presents an embedded approach based on a high performance vibration-based diagnostic framework using validated data from low power miniature smart sensors. The architecture is divided into two levels, with the low level built on smart sensors capable of self-diagnostics, robust data acquisition, and vibration analysis, and the high level comprising a computation system with a graphical user interface, feature extraction toolset, and artificial neural network diagnostics. The goal is a system consisting of smart sensors and intelligent processing to be deployed in aircraft for the detection and isolation of global and incipient failures.

Intelligent Distributed and Ubiquitous Health Management System: Data Storage and Processing

We present here recent developments and results in data storage and processing for an Intelligent Distributed and Ubiquitous Health Management System. The system is designed to store and present data related to faults that may occur in distributed sensor and actuator networked systems. Such systems are common at NASA centers, industrial plants, and other distributed electro-mechanical systems. We show how sensor and actuator fault information can be stored and processed. We describe database development, communication with a Network Capable Application Processor (NCAP), and interfacing with a smart phone including smart phone application developed specifically for this purpose. An experimental setup is described which is used to test, debug, and evaluate the overall system design.