J. Peres

Hybrid semi-parametric modeling in process systems engineering: Past, present and future

Abstract Hybrid semi-parametric models consist of model structures that combine parametric and nonparametric submodels based on different knowledge sources. The development of a hybrid semi-parametric model can offer several advantages over traditional mechanistic or data-driven modeling, as reviewed in this paper. These advantages, such as broader knowledge base, transparency of the modeling approach and cost-effective model development, have been widely recognized, not only in academia but also in the industry. In this paper, the most common hybrid semi-parametric modeling and parameter identification techniques are revisited. Applications in the areas of (bio)chemical engineering for process monitoring, control, optimization, scale-up and model-reduction are reviewed. It is outlined that the application of hybrid semi-parametric techniques does not automatically lead into better results but that rational knowledge integration has potential to significantly improve model-based process operation and design.

Hybrid modeling framework for process analytical technology: application to Bordetella pertussis cultures.

In the process analytical technology (PAT) initiative, the application of sensors technology and modeling methods is promoted. The emphasis is on Quality by Design, online monitoring, and closed‐loop control with the general aim of building in product quality into manufacturing operations. As a result, online high‐throughput process analyzers find increasing application and therewith high amounts of highly correlated data become available online. In this study, an hybrid chemometric/mathematical modeling method is adopted for data analysis, which is shown to be advantageous over the commonly used chemometric techniques in PAT applications. This methodology was applied to the analysis of process data of Bordetella pertussis cultivations, namely online data of near‐infrared, (NIR), pH, temperature and dissolved oxygen, and off‐line data of biomass, glutamate, and lactate concentrations. The hybrid model structure consisted of macroscopic material balance equations in which the specific reactions rates are modeled by nonlinear partial least square (PLS). This methodology revealed a significant higher statistical confidence in comparison to PLSs, translated in a reduction of mean squared prediction errors (e.g., individual root mean squared prediction errors calibration/validation obtained through the hybrid model for the concentrations of lactate: 0.8699/0.7190 mmol/L; glutamate: 0.6057/0.2917 mmol/L; and biomass: 0.0520/0.0283 OD; and obtained through the PLS model for the concentrations of lactate: 1.3549/1.0087 mmol/L; glutamate: 0.7628/0.3504 mmol/L; and biomass: 0.0949/0.0412 OD). Moreover, the analysis of loadings and scores in the hybrid approach revealed that process features can, as for PLS, be extracted by the hybrid method.

A novel identification method for hybrid (N)PLS dynamical systems with application to bioprocesses

This paper presents a method for the identification of nonlinear partial least square (NPLS) models embedded in macroscopic material balance equations with application to bioprocess modeling. The proposed model belongs to the class of hybrid models and consists of a NPLS submodel, which mimics the cellular system, coupled to a set of material balance equations defining the reactor dynamics. The method presented is an analog to the non-iterative partial least square (NIPALS) algorithm where the PLS inner model is trained using the sensitivity method. This strategy avoids the estimation of the target fluxes from measurements of metabolite concentrations, which is rather unrealistic in the case of sparse and noisy off-line measurements. The method is evaluated with a simulation case study on the fed-batch production of a recombinant protein, and an experimental case study of Bordetella pertussis batch cultivations. The results show that the proposed method leads to more consistent models with higher statistical confidence, better calibration properties and reinforced prediction power when compared to other dynamic (N)PLS structures.

Modelling biochemical networks with intrinsic time delays: a hybrid semi-parametric approach

BackgroundThis paper presents a method for modelling dynamical biochemical networks with intrinsic time delays. Since the fundamental mechanisms leading to such delays are many times unknown, non conventional modelling approaches become necessary. Herein, a hybrid semi-parametric identification methodology is proposed in which discrete time series are incorporated into fundamental material balance models. This integration results in hybrid delay differential equations which can be applied to identify unknown cellular dynamics.ResultsThe proposed hybrid modelling methodology was evaluated using two case studies. The first of these deals with dynamic modelling of transcriptional factor A in mammalian cells. The protein transport from the cytosol to the nucleus introduced a delay that was accounted for by discrete time series formulation. The second case study focused on a simple network with distributed time delays that demonstrated that the discrete time delay formalism has broad applicability to both discrete and distributed delay problems.ConclusionsSignificantly better prediction qualities of the novel hybrid model were obtained when compared to dynamical structures without time delays, being the more distinctive the more significant the underlying system delay is. The identification of the system delays by studies of different discrete modelling delays was enabled by the proposed structure. Further, it was shown that the hybrid discrete delay methodology is not limited to discrete delay systems. The proposed method is a powerful tool to identify time delays in ill-defined biochemical networks.

Hybrid Modelling of a PHA Production Process Using Modular Neural Networks

Abstract A novel method for bioreactor hybrid modeling is presented that combines first principles models and modular artificial neural networks trained with the Expectation Maximization (EM) algorithm. The use of modular networks was motivated by the nature of the ‘cells system’ that may be viewed as a highly complex network of metabolic reactions organised in modular pathways. The proposed hybrid modelling technique is validated experimentally with a laboratory scale Polyhydroxyalkanoates (PHAs) production process. The main results show that the embedded modular network, if trained with the EM algorithm, is able to organise itself in modules that have correspondence to the underlying biological pathways. In the particular case of the PHA process discussed, the network learned to discriminate between acetate and internal reserves respiration, with the smaller network modules developing expertise in describing the reaction kinetics of the one or other metabolic state.

Hybrid Modelling of Fermentation Processes: A Study on the Use of Modular Neural Networks for Modelling Cells Reaction Kinetics

Abstract Hybrid models of fermentation processes usually employ Artificial Neural Networks (ANNs) for modelling cells reaction kinetics. The most employed network types are the Multi Layer Perceptron (MLP) and Radial Basis Function (RBF) networks. The main objective in this work was to investigate the applicability of modular neural networks for modelling cell reaction kinetics in bioreactors. The study was supported with simulations of a wastewater treatment process using the ASM model number 2d. The main results show that modular networks if trained with the expectation maximisation algorithm are able to discriminate between pathways and to develop expertise in describing the different pathways. However no advantage was observed in terms of the ratio modelling accuracy/number of parameters

Modelling cells reaction kinetics with artificial neural networks: A comparison of three network architectures

Abstract The present work compares three neural network architectures for modelling reaction kinetics in biological systems: the Mixture of Experts (ME) network, the Backpropagation (BP) network and the Radial Basis Function (RBF) network. The methods are outlined for the case of the growth kinetics of the Saccharomyces cerevisae yeast. The S. cerevisae yeast is able to grow through 3 different pathways. The main results show that a ME network with 3 linear expert modules was able to discriminate between the 3 pathways. The network was trained with the Expectation Maximisation method. A Gaussian gating system produced three input space partitions, one for each of the pathways. The 3 expert modules developed expertise in describing the kinetics of each of the pathways.

Hybrid Modelling of Fermentation Processes Using Artificial Neural Networks: A Study on Identification and Stability

Abstract When processes are complex and poorly understood in a mechanistic sense, hybrid modelling through knowledge integration can be employed with advantage because the model accuracy can be increased by the incorporation of alternative and complementary sources of knowledge. In this work a bioreactor hybrid model structure is studied that combines first principles modelling with artificial neural networks: the bioreactor system is described by a set of mass balance equations, and the cell population system is represented by an adjustable mixture of neural network and mechanistic representations. Two strategies for the identification of embedded neural networks are compared. The sensitivities equations are derived enabling the analytical calculation of the Jacobian Matrix. The application of the theory is illustrated with a simulation case study

An efficient method for the numerical integration of measured variable dependent ordinary differential equations

Abstract The Ordinary Differential Equations (ODEs) of dynamic models that are used in process monitoring, control or optimization, are not only functions of time and states, but also of measured variables. So far two possibilities for the numerical integration of such ODEs were given: (i) a fixed step size integration schema could be applied, matching the step size to the time instances of the measurements; or (ii) using an adaptive step size method while interpolating the measurements. While fixed step size methods are computationally expensive, the repetitive interpolation of measurements for the application of adaptive step size methods is prone to errors and time prohibitive, especially for great numbers of measured variables. In this paper, an adaptive step size numerical integration method is proposed and evaluated for dynamic neural network/hybrid semi-parametric models. The method evaluates the ODEs only at time instances at which online measurements are available and adapts the step size according to those time instances. The numerical solution of the ODEs is provided at time instances which are specified by the user, i.e. at time instances of offline measured states. The rationale behind the proposed method is carefully analyzed, and it is demonstrated that its application along with a hybrid model/dynamic neural network model can result into a significant reduction of number of function evaluations, in the studied cases about 50%, while adhering user specified error tolerances for the numerical integration. In addition, the mutual interference between step-size adaption, parameter identification, coping of the neural network and model performance is studied, a fact that other studies have paid little to no attention.

Hybrid semi-parametric modeling of biological systems: Application to spectroscopic data for the estimation of concentrations

In this work, bioprocess monitoring based on spectral data is improved when compared to commonly applied chemometric tools, by merging nonparametric modeling, biological and process a priori knowledge into a hybrid semi-parametric model. This particular semi-parametric structure comprises a nonparametric submodel inspired by a NPLS structure, as NPLS has been reported to be successful for dealing with massive numbers of highly correlated spectral data. The method was applied to Bordetella pertussis cultivations equipped with a Near-InfraRed (NIR) probe, showing that estimates of metabolite concentrations are improved when compared to those obtained through classical chemometric modeling, as expressed by lower mean square errors, better calibration properties and a higher statistical confidence.