Alexey Zaytsev

Regression on the basis of nonstationary Gaussian processes with Bayesian regularization

We consider the regression problem, i.e. prediction of a real valued function. A Gaussian process prior is imposed on the function, and is combined with the training data to obtain predictions for new points. We introduce a Bayesian regularization on parameters of a covariance function of the process, which increases quality of approximation and robustness of the estimation. Also an approach to modeling nonstationary covariance function of a Gaussian process on basis of linear expansion in parametric functional dictionary is proposed. Introducing such a covariance function allows to model functions, which have non-homogeneous behaviour. Combining above features with careful optimization of covariance function parameters results in unified approach, which can be easily implemented and applied. The resulting algorithm is an out of the box solution to regression problems, with no need to tune parameters manually. The effectiveness of the method is demonstrated on various datasets.

Surrogate modeling of multifidelity data for large samples

The problem of construction of a surrogate model based on available lowand high-fidelity data is considered. The low-fidelity data can be obtained, e.g., by performing the computer simulation and the high-fidelity data can be obtained by performing experiments in a wind tunnel. A regression model based on Gaussian processes proves to be convenient for modeling variable-fidelity data. Using this model, one can efficiently reconstruct nonlinear dependences and estimate the prediction accuracy at a specified point. However, if the sample size exceeds several thousand points, direct use of the Gaussian process regression becomes impossible due to a high computational complexity of the algorithm. We develop new algorithms for processing multifidelity data based on Gaussian process model, which are efficient even for large samples. We illustrate application of the developed algorithms by constructing surrogate models of a complex engineering system.

Large scale variable fidelity surrogate modeling

Engineers widely use Gaussian process regression framework to construct surrogate models aimed to replace computationally expensive physical models while exploring design space. Thanks to Gaussian process properties we can use both samples generated by a high fidelity function (an expensive and accurate representation of a physical phenomenon) and a low fidelity function (a cheap and coarse approximation of the same physical phenomenon) while constructing a surrogate model. However, if samples sizes are more than few thousands of points, computational costs of the Gaussian process regression become prohibitive both in case of learning and in case of prediction calculation. We propose two approaches to circumvent this computational burden: one approach is based on the Nyström approximation of sample covariance matrices and another is based on an intelligent usage of a blackbox that can evaluate a low fidelity function on the fly at any point of a design space. We examine performance of the proposed approaches using a number of artificial and real problems, including engineering optimization of a rotating disk shape.

Reliable surrogate modeling of engineering data with more than two levels of fidelity

Surrogate modeling problems often include variable fidelity data. Most approaches consider the case of two available levels of fidelity, while engineers can have data with more than two samples sorted by fidelity. We consider Gaussian process regression framework that can construct surrogate models with arbitrary number of fidelity levels. While straightforward implementation struggles from numerical instability and numerical problems, our approach adopts Bayesian paradigm and provides direct control of numerical properties of surrogate model construction problems. Benchmark of the presented approach consists of various artificial and real data problems with the focus on surrogate modeling of an airfoil and a C-shape press.

Variable Fidelity Regression Using Low Fidelity Function Blackbox and Sparsification

We consider construction of surrogate models based on variable fidelity samples generated by a high fidelity function an exact representation of some physical phenomenon and by a low fidelity function a coarse approximation of the exact representation. A surrogate model is constructed to replace the computationally expensive high fidelity function. For such tasks Gaussian processes are generally used. However, if the sample size reaches a few thousands points, a direct application of Gaussian process regression becomes impractical due to high computational costs. We propose two approaches to circumvent this difficulty. The first approach uses approximation of sample covariance matrices based on the Nystrom method. The second approach relies on the fact that engineers often can evaluate a low fidelity function on the fly at any point using some blackbox; thus each time calculating prediction of a high fidelity function at some point, we can update the surrogate model with the low fidelity function value at this point. So, we avoid issues related to the inversion of large covariance matrices -- as we can construct model using only a moderate low fidelity sample size. We applied developed methods to a real problem, dealing with an optimization of the shape of a rotating disk.