Nikolay Y. Nikolaev

Learning polynomial feedforward neural networks by genetic programming and backpropagation

This paper presents an approach to learning polynomial feedforward neural networks (PFNNs). The approach suggests, first, finding the polynomial network structure by means of a population-based search technique relying on the genetic programming paradigm, and second, further adjustment of the best discovered network weights by an especially derived backpropagation algorithm for higher order networks with polynomial activation functions. These two stages of the PFNN learning process enable us to identify networks with good training as well as generalization performance. Empirical results show that this approach finds PFNN which outperform considerably some previous constructive polynomial network algorithms on processing benchmark time series.

Polynomial harmonic GMDH learning networks for time series modeling

This paper presents a constructive approach to neural network modeling of polynomial harmonic functions. This is an approach to growing higher-order networks like these build by the multilayer GMDH algorithm using activation polynomials. Two contributions for enhancement of the neural network learning are offered: (1) extending the expressive power of the network representation with another compositional scheme for combining polynomial terms and harmonics obtained analytically from the data; (2) space improving the higher-order network performance with a backpropagation algorithm for further gradient descent learning of the weights, initialized by least squares fitting during the growing phase. Empirical results show that the polynomial harmonic version phGMDH outperforms the previous GMDH, a Neurofuzzy GMDH and traditional MLP neural networks on time series modeling tasks. Applying next backpropagation training helps to achieve superior polynomial network performances.

Sequential Bayesian kernel modelling with non-Gaussian noise

This paper presents a sequential Bayesian approach to kernel modelling of data, which contain unusual observations and outliers. The noise is heavy tailed described as a one-dimensional mixture distribution of Gaussians. The development uses a factorised variational approximation to the posterior of all unknowns, that helps to perform tractable Bayesian inference at two levels: (1) sequential estimation of the weights distribution (including its mean vector and covariance matrix); and (2) recursive updating of the noise distribution and batch evaluation of the weights prior distribution. These steps are repeated, and the free parameters of the non-Gaussian error distribution are adapted at the end of each cycle. The reported results show that this is a robust approach that can outperform standard methods in regression and time-series forecasting.

Nonlinear filtering of asymmetric stochastic volatility models and Value-at-Risk estimation

This paper develops an efficient approach to analytical learning of Asymmetric Stochastic Volatility (ASV) models through nonlinear filtering, and shows that they are useful for practical risk management. This involves the derivation of a Nonlinear Quadrature Filter (NQF) that operates directly on the nonlinear ASV model. The NQF filter makes Gaussian approximations to the prior and posterior density of the latent volatility, but not in the observation space which makes possible easy handling of heavy-tailed data. Experiments in Value-at-Risk (VaR) assessment via an original bootsrtapping methodology are conducted with NQF and several ASV learning algorithms. The results indicate that our approach yields models with better statistical characteristics than the considered competitors, and slightly improved VaR forecasts.

Heavy-tailed mixture GARCH volatility modeling and Value-at-Risk estimation

This paper presents a heavy-tailed mixture model for describing time-varying conditional distributions in time series of returns on prices. Student-t component distributions are taken to capture the heavy tails typically encountered in such financial data. We design a mixture MT(m)-GARCH(p,q) volatility model for returns, and develop an EM algorithm for maximum likelihood estimation of its parameters. This includes formulation of proper temporal derivatives for the volatility parameters. The experiments with a low order MT(2)-GARCH(1,1) show that it yields results with improved statistical characteristics and economic performance compared to linear and nonlinear heavy-tail GARCH, as well as normal mixture GARCH. We demonstrate that our model leads to reliable Value-at-Risk performance in short and long trading positions across different confidence levels.

Genetic programming of polynomial harmonic models using the discrete Fourier transform

This paper presents a Genetic Programming (GP) system that evolves polynomial harmonic networks. The hybrid tree-structured network representation suggests that terminal harmonics with non-multiple frequencies may enter polynomial function nodes as variables. The harmonics with non-multiple, irregular frequencies are derived analytically using the discrete Fourier transform. The development of polynomial harmonic GP includes also design of a regularized statistical fitness function for improved search control and overfitting avoidance. Empirical results show that this hybrid version outperforms the previous GP system manipulating polynomials STROGANOFF, the traditional Koza-style GP, and the harmonic GMDH network algorithm on processing time series.

A one-step unscented particle filter for nonlinear dynamical systems

This paper proposes a one-step unscented particle filter for accurate nonlinear estimation. Its design involves the elaboration of a reliable one-step unscented filter that draws state samples deterministically for doing both the time and measurement updates, without linearization of the observation model. Empirical investigations show that the onestep unscented particle filter compares favourably to relevant filters on nonlinear dynamic systems modelling.

Instance-Based Decompositions of Error Correcting Output Codes

This paper proposes instance decomposition schemes (IDSs) for mapping multi-class classification tasks into a series of binary classification tasks. It demonstrates theoretically that IDSs can handle two main problems of the class decomposition schemes: the problem of difficult binary classification tasks and the problem of positive error correlation of the binary classifiers. The experiments show that IDSs can outperform standard ECOC class decompositions.

Analytical factor stochastic volatility modeling for portfolio allocation

This paper proposes a computationally efficient approach to estimation of factor stochastic volatility models using analytical formulae. Following the maximum likelihood principle there are obtained formulae for the evaluation of the parameters of the dynamic factor model, as well as for the parameters of the ingredient stochastic volatility processes. The approach uses the Hull-White stochastic volatility model to represent the common factors and the idiosyncratic factors. Empirical investigations show that this analytical factor stochastic volatility modeling generates plausible forecasts which are useful for portfolio selection.

Unscented grid filtering and elman recurrent networks

This paper develops an unscented grid-based filter for improved recurrent neural network modeling of time series. The filter approximates directly the weight posterior distribution as a linear mixture using deterministic unscented sampling. The weight posterior is obtained in one step, without linearisation through derivatives. An expectation maximisation algorithm is formulated for evaluation of the complete data likelihood and finding the state noise and observation noise hyperparemeters. Empirical investigations show that the proposed unscented grid filter compares favourably to other similar filters on recurrent network modeling of two real-world time series of environmental importance.