Jean-Pierre Vila

Bayesian nonlinear model selection and neural networks: a conjugate prior approach

In order to select the best predictive neural-network architecture in a set of several candidate networks, we propose a general Bayesian nonlinear regression model comparison procedure, based on the maximization of an expected utility criterion. This criterion selects the model under which the training set achieves the highest level of internal consistency, through the predictive probability distribution of each model. The density of this distribution is computed as the model posterior predictive density and is asymptotically approximated from the assumed Gaussian likelihood of the data set and the related conjugate prior density of the parameters. The use of such a conjugate prior allows the analytic calculation of the parameter posterior and predictive posterior densities, in an empirical-Bayes-like approach. This Bayesian selection procedure allows us to compare general nonlinear regression models and in particular feedforward neural networks, in addition to embedded models as usual with asymptotic comparison tests.

Nonparametric identification and adaptive control of an anaerobic fluidized bed digester

Abstract In this paper, a new adaptive and robust control algorithm that is able to successfully deal with unpredictable internal changes (unmodeled dynamics) and external disturbances (changes in input) of the processes in an anaerobic digestion bioreactor is presented. The adaptive controller is based on a nonparametric statistical approach of the process identification. The regulation is done by optimally adapting the input liquid flow rate of the wastewater to designated changes in the output flow rate of the biogas (methane and carbon dioxide) resulting from the biological reaction. The fundamental advantage of this approach is its freedom from any a priori modeling assumptions about uncertain dynamic components. Experimental results, obtained using a pilot-scale 150 l fluidized bed reactor for the treatment of industrial wine distillery liquid wastes, demonstrates the usefulness of this approach in controlling biological processes.

Immune response gene(s) controlling the humoral antilysozyme response (Ir-Lys) linked to the major histocompatibility complexSL-A in the pig.

A genetic control of humoral immune response in pigs against hen egg-white lysozyme was shown to be linked to the major histocompatibility complexSLA. This control was detected when high antigen doses were used for immunization. It was more prominent with small immunizing doses of lysozyme. Under these latter conditions,SL- A heterozygous individuals exhibited a higher response than correspondingSL- A homozygous animals, suggesting a complementation phenomenon between several genes, at least one of which is linked to the porcine MHC,SL- A.

Predictive neuro-control of uncertain systems: design and use of a neuro-optimizer

We consider the problem of predictive control of uncertain stochastic discrete I/O systems. Given a model identification procedure able to give accurate output system estimates, e.g. a neural network approximation, we use another feedforward neural network to generate at each time step a constrained optimal control. Dynamic backpropagation is used to improve when necessary the controller network parameters. Both system and controller neural structures are first selected off-line by a statistical Bayesian procedure in order to make the predictive control minimizing process more efficient. The issue of stochastic stability of the closed-loop is considered. We developed this approach for the tracking control of such uncertain systems as biotechnological processes. Actual and simulated predictive neuro-control case studies in this field of application are proposed as illustrations. A comparison with a more classic quasi-Newton-based approach is also proposed, showing the interest of this neuro-control approach.

Optimality of CUSUM Rule Approximations in Change-Point Detection Problems: Application to Nonlinear State–Space Systems

The well-known cumulative sum (CUSUM) sequential rule for abrupt model change detection in stochastic dynamic systems relies on the knowledge of the probability density functions of the system output variables conditional on their past values and on the system functioning mode at each time step. This paper shows how to build an asymptotically optimal detection rule under the common average run length (ARL) constraint when these densities are not available but can be consistently estimated. This is the case for nonlinear state-space systems observed through output variables: for such systems, a new class of particle filters based on convolution kernels allows to get consistent estimates of the conditional densities, leading to an optimal CUSUM-like filter detection rule (FDR).

Local optimality of replications from a minimal D-optimal design in regression: A sufficient and quasi-necessary condition

Abstract Conditionally to a priori parameter values, exact D-optimal designs for a general regression model, are frequently found as equitable replications of the points of the minimal exact D-optimal design, i.e. the exact D-optimal design whose size equals the number of model parameters. Using Constrained Optimization Theory, a sufficient condition for strong local optimality of such replicated design is given. This condition is valid whatever the number of independent variables, and whatever the size n of the design, which has not to be necessarily a multiple of the number p of model parameters. This condition takes explicit account of the constraint system defining the experimental region, which can be general. A necessary condition is obtained by slightly weakening this sufficient condition. These two conditions are close enough for the first one to be considered necessary and sufficient for most practical purposes. Several examples show its practicability.

Neural network architecture selection: new Bayesian perspectives in predictive modelling: Application to a soil hydrology problem

The aim of this paper is to present to the community of ecologists concerned with predictive modelling by feedforward neural network, a new statistical approach to select the best neural network architecture (number of layers, number of neurons per layer and connectivity) in a set of several candidate networks. The interest of this approach is demonstrated on a soil hydrology problem.

Bayesian multioutput feedforward neural networks comparison: a conjugate prior approach

A Bayesian method for the comparison and selection of multioutput feedforward neural network topology, based on the predictive capability, is proposed. As a measure of the prediction fitness potential, an expected utility criterion is considered which is consistently estimated by a sample-reuse computation. As opposed to classic point-prediction-based cross-validation methods, this expected utility is defined from the logarithmic score of the neural model predictive probability density. It is shown how the advocated choice of a conjugate probability distribution as prior for the parameters of a competing network, allows a consistent approximation of the network posterior predictive density. A comparison of the performances of the proposed method with the performances of usual selection procedures based on classic cross-validation and information-theoretic criteria, is performed first on a simulated case study, and then on a well known food analysis dataset.

Nonparametric Identification of Controlled Nonlinear Time Varying Processes

We are interested in the identification of an unknown time varying additive component of a controlled nonlinear autoregressive model, a problem of interest in the modeling and control of uncertain systems, such as those met in biotechnological processes. A kernel-based nonparametric estimator is proposed whose almost sure convergence is studied by means of a Lyapunov stabilizability assumption and laws of large numbers for martingales. We then adapt the general result to several classes of deterministic or random functional model uncertainties.

New prediction interval and band in the nonlinear regression model: application to predictive modeling in foods.

This article is concerned with the proposal of a new prediction interval and band for the nonlinear regression model. The construction principle of this interval and band is based on an exact (the meaning of the term “exact” will be given later) confidence region for parameters of the nonlinear regression model. This region, fully described in Vila and Gauchi (2007), provides a rigorous justification for the new prediction interval and band that we propose. This new band is then compared to the classical bands (which are asymptotic and thus approximate for small n), and also to the band based on the bootstrap resampling method. The comparison of these bands is undertaken with simulated and real data from predictive modeling in food science.