Vincent Wertz

The Concentration of Fractional Distances

Nearest neighbor search and many other numerical data analysis tools most often rely on the use of the euclidean distance. When data are high dimensional, however, the euclidean distances seem to concentrate; all distances between pairs of data elements seem to be very similar. Therefore, the relevance of the euclidean distance has been questioned in the past, and fractional norms (Minkowski-like norms with an exponent less than one) were introduced to fight the concentration phenomenon. This paper justifies the use of alternative distances to fight concentration by showing that the concentration is indeed an intrinsic property of the distances and not an artifact from a finite sample. Furthermore, an estimation of the concentration as a function of the exponent of the distance and of the distribution of the data is given. It leads to the conclusion that, contrary to what is generally admitted, fractional norms are not always less concentrated than the euclidean norm; a counterexample is given to prove this claim. Theoretical arguments are presented, which show that the concentration phenomenon can appear for real data that do not match the hypotheses of the theorems, in particular, the assumption of independent and identically distributed variables. Finally, some insights about how to choose an optimal metric are given.

Mutual information for the selection of relevant variables in spectrometric nonlinear modelling

Data from spectrophotometers form vectors of a large number of exploitable variables. Building quantitative models using these variables most often requires using a smaller set of variables than the initial one. Indeed, a too large number of input variables to a model results in a too large number of parameters, leading to overfitting and poor generalization abilities. In this paper, we suggest the use of the mutual information measure to select variables from the initial set. The mutual information measures the information content in input variables with respect to the model output, without making any assumption on the model that will be used; it is thus suitable for nonlinear modelling. In addition, it leads to the selection of variables among the initial set, and not to linear or nonlinear combinations of them. Without decreasing the model performances compared to other variable projection methods, it allows therefore a greater interpretability of the results.

Resampling methods for parameter-free and robust feature selection with mutual information

Combining the mutual information criterion with a forward feature selection strategy offers a good trade-off between optimality of the selected feature subset and computation time. However, it requires to set the parameter(s) of the mutual information estimator and to determine when to halt the forward procedure. These two choices are difficult to make because, as the dimensionality of the subset increases, the estimation of the mutual information becomes less and less reliable. This paper proposes to use resampling methods, a K-fold cross-validation and the permutation test, to address both issues. The resampling methods bring information about the variance of the estimator, information which can then be used to automatically set the parameter and to calculate a threshold to stop the forward procedure. The procedure is illustrated on a synthetic data set as well as on the real-world examples.

Paper: Uniquely identifiable state-space and ARMA parametrizations for multivariable linear systems

Multivariable systems can be represented, in a uniquely identifiable way, either by canonical forms or by so-called overlapping forms. The advantage of the latter is that they do not require the a priori estimation of a set of structural invariants (e.g. Kronecker invariants). We show here how to define uniquely identifiable overlapping parametrizations for state-space and ARMA models. We show that these parametrizations are all related to a set of intrinsic invariants, which are obtained from the Markov parameters of the system. Different forms of overlapping ARMA parametrizations are derived and their properties discussed.

Model selection with cross-validations and bootstraps: application to time series prediction with RBFN models

This paper compares several model selection methods, based on experimental estimates of their generalization errors. Experiments in the context of nonlinear time series prediction by Radial-Basis Function Networks show the superiority of the bootstrap methodology over classical cross-validations and leave-one-out.

Forecasting electricity consumption using nonlinear projection and self-organizing maps

A general-purpose useful parameter in time series forecasting is the regressor size, corresponding to the minimum number of variables necessary to forecast the future values of the time series. If the models are nonlinear, the choice of this regressor becomes very difficult. We present a quasi-automatic method using a nonlinear projection named curvilinear component analysis to build this regressor. The size of this regressor will be determined by the estimation of the intrinsic dimension of an over-sized regressor. This method will be applied to electric consumption of Poland using systematic cross-validation. The nonlinear model used for the prediction is a Kohonen map (self-organizing map)

Takagi-Sugeno fuzzy modeling incorporating input variables selection

Fuzzy models, especially Takagi-Sugeno (T-S) fuzzy models, have received particular attention in the area of nonlinear modeling due to their capability to approximate any nonlinear behavior. Based only on measured data without any prior knowledge, there is no systematic way to obtain a T-S fuzzy model with a simple structure and sufficient accuracy. The main idea discussed in this paper is to reduce the complexity of T-S fuzzy models by estimating an optimal number of fuzzy rules and selecting relevant inputs as antecedent variables independently of the selection of consequent regressors. A systematic procedure is proposed here and illustrated on static and dynamical nonlinear systems.

Multiobjective fuzzy linear programming problems with fuzzy decision variables

In this paper, a multiobjective decision-making process is modeled by a multiobjective fuzzy linear programming problem with fuzzy coefficients for the objectives and the constraints. Moreover, the decision variables are linked together because they have to sum up to a constant. Most of the time, the solutions of a multiobjective fuzzy linear programming problem are compelled to be crisp values. Thus the fuzzy aspect of the decision is partly lost and the decision-making process is constrained to crisp decisions. We propose a method that uses fuzzy decision variables with a joint membership function instead of crisp decision variables. First, we consider lower-bounded fuzzy decision variables that set up the lower bounds of the decision variables. Then, the method is generalized to lower-upper-bounded fuzzy decision variables that set up also the upper bounds of the decision variables. The results are closely related to the special type of problem we are coping with, since we embed a sum constraint in the joint membership function of the fuzzy decision variables. Numerical examples are presented in order to illustrate our method

Multivariable nonlinear predictive control of cement mills

A multivariable controller for cement milling circuits is presented, which is based on a nonlinear model of the circuit and on a nonlinear predictive control strategy. Comparisons with previous LQ control strategies show improved performances with respect to an important source of perturbations of the circuit: a change of hardness of the raw material.

A fuzzy clustering method for the identification of fuzzy models for dynamic systems

Fuzzy modeling is an important topic in fuzzy sets theory and applications. One particular fuzzy model structure, which can be used effectively to describe the behaviour of complex nonlinear systems, has been given by Takagi and Sugeno (1985). By means of a fuzzy clustering method, a new approach to the identification of this kind of fuzzy model is proposed, which integrates the structure and parameter identification steps, and/or the premise and consequence identification.<<ETX>>