Ramiro Rico-Martínez

Order reduction for nonlinear dynamic models of distributed reacting systems

Abstract Detailed first-principles models of transport and reaction (based on partial differential equations) lead, after discretization, to dynamical systems of very high order. Systematic methodologies for model order reduction are vital in exploiting such fundamental models in the analysis, design and real-time control of distributed reacting systems. We briefly review some approaches to model order reduction we have successfully used in recent years, and illustrate their capabilities through (a) the design of an observer and stabilizing controller of a reaction-diffusion problem and (b) two-dimensional simulations of the transient behavior of a horizontal MOVPE reactor.

Identification of distributed parameter systems: A neural net based approach

Abstract Advances in scientific computation and developments in spatially resolved sensor technology have, in recent years, critically enhanced our ability to develop modeling strategies and experimental techniques for the study of the spatiotemporal response of distributed nonlinear systems. The usual alternatives for the modeling of these systems, simplifying techniques that seek to capture the distributed system dynamics through lumped parameter models, can be drastically underresolved, and miss important features of the true system response. Robust implementations of distributed system identification algorithms based on detailed spatiotemporal experimental data have, therefore, an important role to play. In this contribution we present a methodology for the identification of distributed parameter systems, based on artificial neural network architectures, motivated by standard numerical discretization techniques used for the solution of partial differential equations.

Noninvertibility in neural networks

We present and discuss an inherent shortcoming of neural networks used as discrete-time models in system identification, time series processing, and prediction. Trajectories of nonlinear ordinary differential equations (ODEs) can, under reasonable assumptions, be integrated uniquely backward in time. Discrete-time neural network mappings derived from time series, on the other hand, can give rise to multiple trajectories when followed backward in time: they are in principle noninvertible. This fundamental difference can lead to model predictions that are not only slightly quantitatively different, but qualitatively inconsistent with continuous time series. We discuss how noninvertibility arises, present key analytical concepts and some of its phenomenology. Using two illustrative examples (one experimental and one computational), we demonstrate when noninvertibility becomes an important factor in the validity of artificial neural network (ANN) predictions, and show some of the overall complexity of the predicted pathological dynamical behavior. These concepts can be used to probe the validity of ANN time series models, as well as provide guidelines for the acquisition of additional training data.

A systems-based approach to multiscale computation: Equation-free detection of coarse-grained bifurcations

We discuss certain basic features of the equation-free (EF) approach to modeling and computation for complex/multiscale systems. We focus on links between the equation-free approach and tools from systems and control theory (design of experiments, data analysis, estimation, identification and feedback). As our illustrative example, we choose a specific numerical task (the detection of stability boundaries in parameter space) for stochastic models of two simplified heterogeneous catalytic reaction mechanisms. In the equation-free framework the stochastic simulator is treated as an experiment (albeit a computational one). Short bursts of fine scale simulation (short computational experiments) are designed, executed, and their outputs processed and fed back to the process, in integrated protocols aimed at performing the particular coarse-grained task (the detection of a macroscopic instability). Two distinct approaches are presented; one is a direct translation of our previous protocol for adaptive detection of instabilities in laboratory experiments [Rico-Martinez, R., Krisher, K., Flatgen, G., Anderson, J. S., & Kevrekidis, I. G. (2003). Adaptive detection of instabilities: An experimental feasibility study. Physica D, 176, 1–18]; the second approach is motivated from numerical bifurcation algorithms for critical point detection. A comparison of the two approaches brings forth a key feature of equation-free computation: computational experiments can be easily initialized at will, in contrast to laboratory ones.

Adaptive detection of instabilities: an experimental feasibility study

Abstract We implement a practical protocol for the active, on-line detection of bifurcations in experimental systems, based on real-time identification and feedback control ideas. Current experimental practice for the detection of bifurcations typically requires long observation times in the vicinity of marginally stable solutions, as well as frequent re-settings of the experiment for the detection of turning point or subcritical bifurcations. The approach exemplified here addresses these issues drawing from numerical bifurcation detection procedures. The main idea is to create an augmented experiment, using the experimental bifurcation parameter(s) as additional state variables. We implement deterministic laws for the evolution of these new variables by coupling the experiment with an on-line, computer-assisted identification/feedback protocol. The “augmented” experiment (the closed-loop system) thus actively converges to what, for the original experiment (the open-loop system), is a bifurcation point. We apply this method to the real-time, computer-assisted detection of period-doubling bifurcations in an electronic circuit. The method succeeds in actively driving the circuit to the bifurcation points, even in the presence of modest experimental uncertainties, noise, and limited resolution. The active experimental tracing of a codimension-1 bifurcation boundary in two-parameter space is also demonstrated.

Self-consistency in neural network-based NLPC analysis with applications to time-series processing

Abstract The success of nonlinear system identification and characterization from experimental time-series often depends on the appropriate pre-processing of the data. This pre-processing can, in many instances, be achieved through linear and/or “nonlinear” principal component analysis. In recent years, neural network based techniques as a means to perform the nonlinear principal component (NLPC) analysis have gained increased attention. In this contribution, we address an inherent shortcoming of these techniques: the self-consistency problem. We present a modification to the usual feedforward neural network architecture used to extract the NLPCs that results in attenuation of this problem. The proposed modification may significantly reduce representation errors in many other applications for which NLPC analysis is a powerful tool, such as feature extraction and image processing.

A comparison of recurrent training algorithms for time series analysis and system identification

Abstract Artificial Neural Networks (ANNs) can be used for grey-box or black-box modeling of continuous-time systems by placing them in a framework based on numerical integration techniques. When an implicit integration scheme is used as a template, it imposes a recurrent structure on the overall network. Here we present three algorithms suitable for the training of such “network-plus-integrator” assemblies and compare their relative computational efficiencies. Pinedas Recurrent Back-Propagation (RBP) training method is recast to exploit the structure of the assembly. The second approach is RBP modified to evaluate partial derivatives of network outputs with respect to parameters exactly, while the third is a Newton-Raphson based algorithm in which outputs of the network and partial derivatives are computed at each step instead of approximated. We compare the methods via an illustrative example and discuss aspects of training in a parallel computing environment.

Order Reduction of Nonlinear Dynamic Models for Distributed Reacting Systems

Abstract Detailed first-principles models of transport and reaction (based on partial differential equations) lead, after discretization, to dynamical systems of very high order. Systematic methodologies for model order reduction are vital in exploiting such fundamental models in the analysis, design and real-time control of distributed reacting systems. We briefly review some approaches to model order reduction we have successfully used in recent years, and illustrate their capabilities through (a) the design of an observer and stabilization of a reaction-diffusion problem and (b) two-dimensional simulations of the transient behavior of a horizontal MOVPE reactor.

Characterization of a two-parameter mixed-mode electrochemical behavior regime using neural networks

Abstract We use nonlinear signal processing techniques, based on artificial neural networks, to construct a continuous-time model (set of ordinary differential equations, ODEs) from experimental observations of mixed-mode oscillations during the galvanostatic oxidation of hydrogen on platinum in the presence of bismuth and chloride ions. The data was in the form of time-series of the potential for different values of the applied current and chloride ion concentration. We use the model to reconstruct the experimental dynamics and to explore the associated bifurcation structures in phase-space. Using numerical bifurcation techniques we locate stable and unstable periodic solutions, calculate eigenvalues, and identify bifurcation points. This approach constitutes a promising data post-processing procedure for investigating phase-space and parameter space of real experimental systems; it allows us to infer phase-space structures which the experiments can only probe with limited measurement precision and only at a discrete number of operating parameter settings. For example, the fitted model suggests the existence of a sub-critical Hopf bifurcation near the range of parameters probed in the experiments; this might explain the experimental difficulty in locating small amplitude simple oscillations.