Ne-Zheng Sun

Development of Data-Driven Models

In previous chapters, we have mainly dealt with physics-based models represented by a general mapping \(\mathbf{u}=\mathcal{M}(\mathrm{}{\boldsymbol{\uptheta}}{}),\), where \(\mathbf{u}\) denotes state variable(s), \(\mathcal{M}(\mathrm{}{\boldsymbol{\uptheta}}{})\) is obtained from mathematical equation(s) describing the underlying physical processes, and \(\mathrm{}{\boldsymbol{\uptheta}}{}\) is a set of unknown physical parameters that need to be estimated from measurements. As mentioned in Chap. 1, various data-driven models that connect model inputs and model outputs directly are also developed and extensively used in environmental and water resource (EWR) fields. They are most useful when there is no or little a priori knowledge about the form of the actual physical processes, or when it is desired to replace a physics-based model with a surrogate model for improving computational efficiency in optimization problems. The latter usage is referred to as reduced-order modeling or metamodeling.

Model Uncertainty Quantification

Uncertainty quantification (UQ) is the analytic process of determining the effect of input uncertainties (both their magnitudes and sources) on system outcomes. Traditionally applied in engineering reliability analysis, UQ now plays a significant role in environmental and water resource (EWR) applications as environmental engineers and modelers are increasingly involved in designing or permitting complex systems, including examining their long-term environmental impacts and providing decision support under risks. Risk analysis is the process of determining the consequence of uncertain and often undesirable outcomes. UQ provides inputs to risk analysis which, in turn, provides bases for decision making and for improving data collection design. UQ and risk analysis are integral and often required components of EWR applications nowadays.

Goal-Oriented Modeling

Two criteria are traditionally used for model parameter estimation and model structure identification: (C-1) fitting observation data; and (C-2) honoring prior information. As we have shown in Chaps. 3 and 7, these two criteria cannot determine a model uniquely by solving either classical inverse problem (CIP) or extended inverse problem (EIP) when observation error and model error exist. Models that cannot be rejected by prior information and observed data are called data-acceptable models. In environmental and water resource (EWR) modeling, because the real system structure is complex and unknown, there may be infinite combinations of model structures and model parameters that can fit the existing data equally well. Different modelers may construct different models for the same system based on the same data. As we explained in Chap. 10, this type of model nonuniqueness is called equifinality by Beven and coworkers.

The Classical Inverse Problem

The classical inverse problem (CIP) assumes no model structure error and, thus, only model parameters need to be identified. In this chapter, we consider the solution of CIP for single state variable models. Using the “fitting data” criterion of inverse problem formulation, we can obtain a quasi-solution by solving an optimization problem. We will show that when the inverse problem is extended well-posed and the conditions of quasi-identifiability are satisfied, thequasi-solution will approach the accurate inverse solution when the observation error reaches zero. The singular value decomposition (SVD) is introduced for linear model inversion, followed by a general procedure of linearization for mildly nonlinear model inversion. Optimization problems resulting from nonlinear inversion can be difficult to solve and the solutions are generally not unique. In this chapter, we shall restrict ourselves to cases in which the CIP is extended well-posed in a known region and the objective function for optimization is convex in the region. Consequently, the inverse solution can be found by using a local optimization algorithm. Common numerical methods for local optimization and norm selection problems will be briefly discussed.

Data Assimilation for Inversion

Many EWR applications are data centric and naturally call for the ability to fuse spatial and temporal information from multiple sources and in different formats and scales. With the rapid advance of in situ and remote sensing technologies and cyberinfrastructures, a major EWR research front in recent years has focused on the development of effective algorithms for integrating real-time data into prediction models. Oftentimes, prior knowledge and historic training data are limited in both quality and quantity, leading to uncertainties in calibrated models and estimated parameters. Such issues are relevant not only to distributed EWR models, but also to data-driven models, as we have seen in Chap. 8. A fundamental need in EWR is thus related to systematic and continuous extraction of useful information from new observations to provide updated estimates of model states and parameters. In this chapter, mathematical tools and methods that can be applied to automate such information fusion process will be introduced.

Statistical Methods for Parameter Estimation

In this chapter, the CIP is formulated in a statistical framework. Byusing the Bayesian inference theory, we cast the CIP asaninformation transfer problem, in which the prior information andtheinformation transferred from state observations are combinedtoreduce uncertainty in the estimated parameters. The priorinformationis modeled using a probability density function (PDF)called the priorPDF and the inverse solution is also a PDF known asthe posteriorPDF. Because it is a PDF, the inverse solution is alwaysexistent andunique but with uncertainty. When the posterior PDF is ina relativelysimple form, point estimates of the unknown parameterscan bereadily obtained by solving an optimization problem, just as wehavedone in the deterministic framework. When the posterior PDFhas acomplex multimodal shape, however, the non-uniquenessandinstability issues associated with the inverse solution arise again.Forsuch cases, Monte Carlo sampling methods provide powerfultoolsfor learning the posterior PDFs without requiring theiractualfunctional forms be known. Two popular Markov Chain MonteCarlo(MCMC) algorithms are introduced. The application of MCMCforinverse solution and global optimization is also discussed.

Model Structure Identification

In practice, determination of the model structure is the first problemthat a modeler has to deal with when modeling a complicatedphysical system. Model structure error is often the major cause ofmodel failure. Even a carefully calibrated model may produceunreliable results when it is used for prediction and decision-makingpurposes. In this chapter, an extended inverse problem (EIP) isformulated for identifying both the model structure and modelparameters using observation data and prior information. After themodel structure is parameterized, the solution of EIP becomes a minminoptimization problem. In the statistical framework, structureidentification requires estimation of both shape parameters andstatistical parameters. Statistical parameters are hyperparametersthat do not appear directly in the model but must be estimated duringthe inversion process. Model parameters and hyperparameters canbe estimated simultaneously or iteratively by hierarchical Bayesianinversion. Geostatistical inversion uses kriging and cokriging asparameterization for estimating a distributed parameter. The pilotpoint method is a flexible parameterization method for inversion thatcan decrease the model structure error of a geostatistical modeleffectively.

Optimal Experimental Design

In environmental and water resource (EWR) engineering, different types of design problems exist, such as operation design, monitoring design, detection design, and remediation design. This chapter is devoted to the subject of experimental design for model calibration and parameter estimation. Experimental design plays a critical role in model construction because the reliability of a model is mainly dependent on the quantity and quality of data used for its calibration. In other words, the experimental design dictates how data should be collected in field campaigns and how many observations are needed. An optimal experimental design (OED), when it is executed, should provide the maximum amount of information with the minimum cost. Basic concepts, theories, and methods of OED are well established in statistics and have been applied to various scientific and engineering disciplines.

Multiobjective Inversion and Regularization

After incorporating prior information, the inverse problem becomes a bi-criterion optimization problem. The use of prior information in such a way can be seen as a special case of the regularization method introduced in this chapter. Regularization provides a generalframework for increasing the stability of inverse solutions at the priceof possible loss of accuracy. EWR models are often multistatemodels characterized by a set of coupled equations. The statevariable of one equation may depend on states or parameters inother equations. This implies that measurements of one statevariable may provide information for identifying the states and/orparameters in other equations. The inversion of a multistate model iscalled a coupled inverse problem and can be solved by multiobjectiveoptimization (MOO). Various algorithms for solving MOO, includingrecently developed multiobjective evolutionary algorithms, areintroduced. These algorithms can also be used to solvemultiobjective inverse problems.

Model Dimension Reduction

A real system, especially a distributed parameter system, may havehigh or even infinite dimensions of freedom (DOF). When the DOF ofa model is too high, all inversion methods that we have learnedbecome inefficient and the inverse problem becomes unsolvablebecause of data and computational limitations. On the other hand,when the structure of a model is overly simplified, the model maybecome useless because of its large structure error. An appropriatemodel complexity depends not only on the complexity of the modeledsystem, but also on data availability, data format, and the intendeduse of the model. The main purpose of this chapter is to give acomprehensive survey of various parameterization techniques, suchas Voronoi diagram, radial basis functions, and local polynomialapproximation for representing deterministic functions, and Gaussianrandom field, Markov random field, variogram analysis, andmultipoint statistics for representing stochastic fields. Linear andnonlinear model dimension reduction methods, such as POD, FA,and kernel PCA, are considered and applied to model inversion.