Bernard Delyon

Nonlinear black-box modeling in system identification: a unified overview

A nonlinear black-box structure for a dynamical system is a model structure that is prepared to describe virtually any nonlinear dynamics. There has been considerable recent interest in this area, with structures based on neural networks, radial basis networks, wavelet networks and hinging hyperplanes, as well as wavelet-transform-based methods and models based on fuzzy sets and fuzzy rules. This paper describes all these approaches in a common framework, from a users perspective. It focuses on what are the common features in the different approaches, the choices that have to be made and what considerations are relevant for a successful system-identification application of these techniques. It is pointed out that the nonlinear structures can be seen as a concatenation of a mapping form observed data to a regression vector and a nonlinear mapping from the regressor space to the output space. These mappings are discussed separately. The latter mapping is usually formed as a basis function expansion. The basis functions are typically formed from one simple scalar function, which is modified in terms of scale and location. The expansion from the scalar argument to the regressor space is achieved by a radial- or a ridge-type approach. Basic techniques for estimating the parameters in the structures are criterion minimization, as well as two-step procedures, where first the relevant basis functions are determined, using data, and then a linear least-squares step to determine the coordinates of the function approximation. A particular problem is to deal with the large number of potentially necessary parameters. This is handled by making the number of ‘used’ parameters considerably less than the number of ‘offered’ parameters, by regularization, shrinking, pruning or regressor selection.

Nonlinear black-box models in system identification: mathematical foundations

We discuss several aspects of the mathematical foundations of the nonlinear black-box identification problem. We shall see that the quality of the identification procedure is always a result of a certain trade-off between the expressive power of the model we try to identify (the larger the number of parameters used to describe the model, the more flexible is the approximation), and the stochastic error (which is proportional to the number of parameters). A consequence of this trade-off is the simple fact that a good approximation technique can be the basis of a good identification algorithm. From this point of view, we consider different approximation methods, and pay special attention to spatially adaptive approximants. We introduce wavelet and ‘neuron’ approximations, and show that they are spatially adaptive. Then we apply the acquired approximation experience to estimation problems. Finally, we consider some implications of these theoretical developments for the practically implemented versions of the ‘spatially adaptive’ algorithms.

Lp solutions of backward stochastic differential equations

In this paper, we are interested in solving backward stochastic differential equations (BSDEs for short) under weak assumptions on the data. The first part of the paper is devoted to the development of some new technical aspects of stochastic calculus related to BSDEs. Then we derive a priori estimates and prove existence and uniqueness of solutions in Lp p>1, extending the results of El Karoui et al. (Math. Finance 7(1) (1997) 1) to the case where the monotonicity conditions of Pardoux (Nonlinear Analysis; Differential Equations and Control (Montreal, QC, 1998), Kluwer Academic Publishers, Dordrecht, pp. 503-549) are satisfied. We consider both a fixed and a random time interval. In the last section, we obtain, under an additional assumption, an existence and uniqueness result for BSDEs on a fixed time interval, when the data are only in L1.

General results on the convergence of stochastic algorithms

A deterministic approach is proposed for proving the convergence of stochastic algorithms of the most general form under necessary conditions on the input noise and reasonable conditions on the (nonnecessarily continuous) mean field. Emphasis is placed on the case where more than one stationary point exists. We also use this approach to prove the convergence of a stochastic algorithm with Markovian dynamics.

Accelerated Stochastic Approximation

A technique to accelerate convergence of stochastic approximation algorithms is studied. It is based on Kesten’s idea of equalization of the gain coefficient for the Robbins–Monro algorithm. Convergence with probability 1 is proved for the multidimensional analog of the Kesten accelerated stochastic approximation algorithm. Asymptotic normality of the delivered estimates is also shown. Results of numerical simulations are presented that demonstrate the efficiency of the acceleration procedure.

Minimal L/sub 1/-norm reconstruction function for oversampled signals: applications to time-delay estimation

We consider the problem of the reconstruction of an oversampled band-limited signal and obtain an explicit expression for the reconstruction function with minimal L/sub 1/-norm. It has good sparseness and localization properties that recommends its use in time-delay estimation but the result may be of interest in other domains. Compared to the standard sine cardinal reconstruction function, its rate of decrease is almost one order of magnitude higher.

Determination of singular points in 2D deformable flow fields

Digital image analysis appears to be more and more relevant to the study of physical phenomena involving fluid motion, and of their evolution over time. In that context, 2D deformable motion analysis is one of the important issues to be investigated. The interpretation of such deformable 2D flow fields can generally be stated as the characterization of linear models provided that first order approximations are considered in an adequate neighborhood of so-called singular points, where the velocity becomes null. This paper describes an efficient method, based on a statistical approach, which explicitly addresses these problems, and allows us to locate, characterize and track such singular points in an image sequence. It does not require the prior computation of the velocity field. The method has been validated by experiments carried out with synthetic and real examples corresponding to meteorological image sequences. In fact, the described approach can be of interest in different applications dealing with the characterization of vector fields.

Estimating Wavelet Coefficients

We consider fast algorithms of wavelet decomposition of functionf when discrete observations of \( f(\operatorname{supp} f \subseteq [0,1]) \) are available. The properties of the algorithms are studied for three types of observation design: the regular design, when the observationsf(x i) are taken on the regular grid \( {{\chi }_{i}} = i/N,i = 1, \ldots ,N; \) the case of jittered regular grid, when it is only known that for all \( 1{\underset{\raise0.3em\hbox{

Stochastic optimization with averaging of trajectories

\smash{\scriptscriptstyle-}

On the relationship between identification and local tests

}}{ < }}i{\underset{\raise0.3em\hbox{