Jin-Song Pei

Neural Network Initialization with Prototypes - Function Approximation in Engineering Mechanics Applications

A prototype-based initialization methodology is proposed to approximate functions that are used to characterize nonlinear stress-strain, moment-curvature, and load-displacement relationships, as well as restoring forces and time histories in engineering mechanics applications. Three prototypes are defined by exploiting the capabilities of linear sums of sigmoidal functions. By using the proposed prototypes either individually or combinatorially, successful training can take place for ten specific types of nonlinear functions and far beyond when the required number of hidden nodes and initial values of weights and biases can always be derived before the training starts. Some mathematical insights to this initialization methodology and a few prototypes are offered, while training examples are provided to demonstrate a clear procedure that is used to implement this methodology. With the derived numbers of hidden nodes in each approximation, applying the Nguyen-Widrow algorithm is enabled and the training performance is compared between the existing and the proposed initialization options.

Neural network initialization with prototypes - a case study in function approximation

The initialization of neural networks in function approximation has been studied by many researchers yet remains a challenging problem. Another important yet open issue in the neural network community is to incorporate knowledge and hints with regard to training for a meaningful neural network. This study makes an attempt to address these two issues in handling a specific type of engineering problems, namely, modeling nonlinear hysteretic restoring forces of a dynamic system under a specific formulation. The paper showcases a heuristic idea on using a growing technique through a prototype-based initialization where the insights to the governing mathematics/physics are related to the features of the activation functions.

Solving Dynamical Systems Involving Piecewise Restoring Force Using State Event Location

Abstract Many theoretical and experimental studies of complex path-dependent dynamic systems lead to restoring forces expressed as piecewise nonlinear algebraic equations. Examples include, but are not limited to, bilinear hysteretic, Ramberg-Osgood, Masing, generalized Masing, Clough, and Takeda models, which are popular in engineering mechanics applications. These models relate restoring force to displacement and velocity by means of piecewise relations having only C0 continuity, which leads to two sorts of challenges in numerical simulation. First, the equations of motion may not simply be a set of ordinary differential equations, rather they may fall within the framework of differential-algebraic equations (DAEs). Second, there are unknown locations of discontinuities of low-order derivatives of the solution. This study seeks accurate and efficient numerical solutions of the DAEs with C0 continuity, enabling robust simulation of these complex nonlinear dynamic systems. This study focuses on explicit t...

A heuristic neural network initialization scheme for modeling nonlinear functions in engineering mechanics

This paper introduces a heuristic methodology for designing multilayer feedforward neural networks to model the types of nonlinear functions common to many engineering mechanics applications. It is well known that a perfect way to determine the ideal architecture to initialize neural network training has not yet been established. This could be because this challenging issue can only be properly addressed by looking into the features of the function to be approximated and thus might be hard to tackle in a general sense. In this study, the authors do not presume to provide a universal method approximate an arbitrary function, rather the focus is given to modeling nonlinear hysteretic restoring forces, a significant domain function approximation problem. The governing physics and mathematics of nonlinear hysteretic dynamics as well as the strength of the sigmoidal basis function are exploited to determine both an efficient neural network architecture (e.g., the number of hidden nodes) as well as effective initial weight and bias values for those nodes. Training examples are presented to demonstrate and validate the proposed initial design methodology. Comparisons are made between the proposed methodology and the widely used Nguyen-Widrow Initialization. Future work is also identified.

Development of an off-the-shelf field programmable gate array-based wireless sensing unit for structural health monitoring

This paper presents the preliminary results of an investigation on the application of Field Programmable Gate Arrays (FPGAs) to civil infrastructure health monitoring. An off-the-shelf FPGA development board available at a comparable price to microprocessor development boards is adopted in this study. Advantages, disadvantages, feasibility and design concerns when using such a reconfigurable hardware architecture for implementing algorithms for structural health monitoring in a wireless sensor unit are studied in a showcase of implementing Fast Fourier Transform (FFT) in a wireless data transmitting setting.

Embedded EMD algorithm within an FPGA-based design to classify nonlinear SDOF systems

Compared with traditional microprocessor-based systems, rapidly advancing field-programmable gate array (FPGA) technology offers a more powerful, efficient and flexible hardware platform. An FPGA and microprocessor (i.e., hardware and software) co-design is developed to classify three types of nonlinearities (including linear, hardening and softening) of a single-degree-of-freedom (SDOF) system subjected to free vibration. This significantly advances the teams previous work on using FPGAs for wireless structural health monitoring. The classification is achieved by embedding two important algorithms - empirical mode decomposition (EMD) and backbone curve analysis. Design considerations to embed EMD in FPGA and microprocessor are discussed. In particular, the implementation of cubic spline fitting and the challenges encountered using both hardware and software environments are discussed. The backbone curve technique is fully implemented within the FPGA hardware and used to extract instantaneous characteristics from the uniformly distributed data sets produced by the EMD algorithm as presented in a previous SPIE conference by the team. An off-the-shelf high-level abstraction tool along with the MATLAB/Simulink environment is utilized to manage the overall FPGA and microprocessor co-design. Given the limited computational resources of an embedded system, we strive for a balance between the maximization of computational efficiency and minimization of resource utilization. The value of this study lies well beyond merely programming existing algorithms in hardware and software. Among others, extensive and intensive judgment is exercised involving experiences and insights with these algorithms, which renders processed instantaneous characteristics of the signals that are well-suited for wireless transmission.

Embedded algorithms within an FPGA-based system to process nonlinear time series data

This paper presents some preliminary results of an ongoing project. A pattern classification algorithm is being developed and embedded into a Field-Programmable Gate Array (FPGA) and microprocessor-based data processing core in this project. The goal is to enable and optimize the functionality of onboard data processing of nonlinear, nonstationary data for smart wireless sensing in structural health monitoring. Compared with traditional microprocessor-based systems, fast growing FPGA technology offers a more powerful, efficient, and flexible hardware platform including on-site (field-programmable) reconfiguration capability of hardware. An existing nonlinear identification algorithm is used as the baseline in this study. The implementation within a hardware-based system is presented in this paper, detailing the design requirements, validation, tradeoffs, optimization, and challenges in embedding this algorithm. An off-the-shelf high-level abstraction tool along with the Matlab/Simulink environment is utilized to program the FPGA, rather than coding the hardware description language (HDL) manually. The implementation is validated by comparing the simulation results with those from Matlab. In particular, the Hilbert Transform is embedded into the FPGA hardware and applied to the baseline algorithm as the centerpiece in processing nonlinear time histories and extracting instantaneous features of nonstationary dynamic data. The selection of proper numerical methods for the hardware execution of the selected identification algorithm and consideration of the fixed-point representation are elaborated. Other challenges include the issues of the timing in the hardware execution cycle of the design, resource consumption, approximation accuracy, and user flexibility of input data types limited by the simplicity of this preliminary design. Future work includes making an FPGA and microprocessor operate together to embed a further developed algorithm that yields better computational and power efficiency.

Investigation of data quality in a wireless sensing unit composed of off-the-shelf components

This paper presents the preliminary findings of a study on data and system identification results (derived from collected data) in a wireless sensing environment. The goal of this study is to understand how various hardware design choices and operational conditions affect the quality of the data and accuracy of the identified results; the focus of this paper is packet and data loss. A series of experimental investigations are carried out using a laboratory shaking table instrumented with off-the-shelf Micro-Electro-Mechanical Systems (MEMS) accelerometers. A wireless sensing unit is developed to interface with these wired analog accelerometers to enable wireless data transmission. To reduce the overall design variance and aid convenient application in civil infrastructure health monitoring, this wireless unit is built with off-the-shelf microcontroller and radio development boards. The anti-aliasing filter and analog-to-digital convectors (ADC) are the only customized components in the hardware. By varying critical hardware configurations, including using analog accelerometers of different commercial brands, taking various designs for the anti-aliasing filter, and adopting ADCs with different resolutions, shaking table tests are repeated, the collected data are processed, and the results are compared. Operational conditions such as sampling rate and wireless data transmitting range are also altered separately in the repeated testing. In all of the cases tested, data is also collected using a wire-based data acquisition system to serve as a performance baseline for evaluation of the wireless data transmission performance. Based on this study, the challenges in the hardware design of wireless sensing units and data processing are identified.

More Transparent Neural Network Approach for Modeling Nonlinear Hysteretic Systems

A powerful Volterra/Wiener Neural Network (VWNN) is designed to reflect the underlying dynamics of hysteretic systems. The nonlinear response of multi-degree-of-freedom systems subjected to force excitation can be tracked using this neural network. More importantly, the inner-workings of the network, such as the design parameters as well as the weights and biases, can be loosely related to physical properties of dynamic systems. This effort differs markedly from what is typically done for neural networks as well as the original version of the VWNN in Ref. 1. An adaptive training algorithm and improved formulation of high-order nodes are adopted to enable fast training and stable convergence. A training example is provided to demonstrate that the VWNN is able to yield a unique set of solutions (i.e., the weights) when the values of the controlling design parameters are fixed a priori. The selection of these design parameters in practical applications is discussed. The advantages of the VWNN illustrate the potential of applying highly flexible nonparametric identification techniques in a parametric fashion to suit the needs of structural health monitoring and damage detections.

Mapping some functions and four arithmetic operations to multilayer feedforward neural networks

This paper continues the development of a heuristic initialization methodology for designing multilayer feedforward neural networks aimed at modeling nonlinear functions for engineering mechanics applications as presented previously at SPIE 2003, and 2005 to 2007. Seeking a transparent and domain knowledge-based approach for neural network initialization and result interpretation, the authors examine the efficiency of linear sums of sigmoidal functions while offering constructive methods to approximate functions in engineering mechanics applications. This study provides details and results of mapping the four arithmetic operations (summation, subtraction, multiplication, division) as well as other functions including reciprocal, Gaussian and Mexican hat functions into multilayer feedforward neural networks with one hidden layer. The approximation and training examples demonstrate the efficiency and accuracy of the proposed mapping techniques and details. Future work is also identified. This effort directly contributes to the further extension of the proposed initialization procedure in that it opens the door for the approximation of a wider range of nonlinear functions.