Cédric Richard

Online Prediction of Time Series Data With Kernels

Kernel-based algorithms have been a topic of considerable interest in the machine learning community over the last ten years. Their attractiveness resides in their elegant treatment of nonlinear problems. They have been successfully applied to pattern recognition, regression and density estimation. A common characteristic of kernel-based methods is that they deal with kernel expansions whose number of terms equals the number of input data, making them unsuitable for online applications. Recently, several solutions have been proposed to circumvent this computational burden in time series prediction problems. Nevertheless, most of them require excessively elaborate and costly operations. In this paper, we investigate a new model reduction criterion that makes computationally demanding sparsification procedures unnecessary. The increase in the number of variables is controlled by the coherence parameter, a fundamental quantity that characterizes the behavior of dictionaries in sparse approximation problems. We incorporate the coherence criterion into a new kernel-based affine projection algorithm for time series prediction. We also derive the kernel-based normalized LMS algorithm as a particular case. Finally, experiments are conducted to compare our approach to existing methods.

Nonlinear Unmixing of Hyperspectral Images: Models and Algorithms

When considering the problem of unmixing hyperspectral images, most of the literature in the geoscience and image processing areas relies on the widely used linear mixing model (LMM). However, the LMM may be not valid, and other nonlinear models need to be considered, for instance, when there are multiscattering effects or intimate interactions. Consequently, over the last few years, several significant contributions have been proposed to overcome the limitations inherent in the LMM. In this article, we present an overview of recent advances in nonlinear unmixing modeling.

Multitask Diffusion Adaptation Over Networks

Adaptive networks are suitable for decentralized inference tasks. Recent works have intensively studied distributed optimization problems in the case where the nodes have to estimate a single optimum parameter vector collaboratively. However, there are many important applications that are multitask-oriented in the sense that there are multiple optimum parameter vectors to be inferred simultaneously, in a collaborative manner, over the area covered by the network. In this paper, we employ diffusion strategies to develop distributed algorithms that address multitask problems by minimizing an appropriate mean-square error criterion with l2-regularization. The stability and performance of the algorithm in the mean and mean-square error sense are analyzed. Simulations are conducted to verify the theoretical findings, and to illustrate how the distributed strategy can be used in several useful applications related to target localization and hyperspectral data unmixing.

Nonlinear Unmixing of Hyperspectral Data Based on a Linear-Mixture/Nonlinear-Fluctuation Model

Spectral unmixing is an important issue to analyze remotely sensed hyperspectral data. Although the linear mixture model has obvious practical advantages, there are many situations in which it may not be appropriate and could be advantageously replaced by a nonlinear one. In this paper, we formulate a new kernel-based paradigm that relies on the assumption that the mixing mechanism can be described by a linear mixture of endmember spectra, with additive nonlinear fluctuations defined in a reproducing kernel Hilbert space. This family of models has clear interpretation, and allows to take complex interactions of endmembers into account. Extensive experiment results, with both synthetic and real images, illustrate the generality and effectiveness of this scheme compared with state-of-the-art methods.

Diffusion LMS Over Multitask Networks

The diffusion LMS algorithm has been extensively studied in recent years. This efficient strategy allows to address distributed optimization problems over networks in the case where nodes have to collaboratively estimate a single parameter vector. Nevertheless, there are several problems in practice that are multitask-oriented in the sense that the optimum parameter vector may not be the same for every node. This brings up the issue of studying the performance of the diffusion LMS algorithm when it is run, either intentionally or unintentionally, in a multitask environment. In this paper, we conduct a theoretical analysis on the stochastic behavior of diffusion LMS in the case where the single-task hypothesis is violated. We analyze the competing factors that influence the performance of diffusion LMS in the multitask environment, and which allow the algorithm to continue to deliver performance superior to non-cooperative strategies in some useful circumstances. We also propose an unsupervised clustering strategy that allows each node to select, via adaptive adjustments of combination weights, the neighboring nodes with which it can collaborate to estimate a common parameter vector. Simulations are presented to illustrate the theoretical results, and to demonstrate the efficiency of the proposed clustering strategy.

Testing Stationarity With Surrogates: A Time-Frequency Approach

An operational framework is developed for testing stationarity relatively to an observation scale, in both stochastic and deterministic contexts. The proposed method is based on a comparison between global and local time-frequency features. The originality is to make use of a family of stationary surrogates for defining the null hypothesis of stationarity and to base on them two different statistical tests. The first one makes use of suitably chosen distances between local and global spectra, whereas the second one is implemented as a one-class classifier, the time- frequency features extracted from the surrogates being interpreted as a learning set for stationarity. The principle of the method and of its two variations is presented, and some results are shown on typical models of signals that can be thought of as stationary or nonstationary, depending on the observation scale used.

Decentralized Variational Filtering for Target Tracking in Binary Sensor Networks

The prime motivation of our work is to balance the inherent trade-off between the resource consumption and the accuracy of the target tracking in wireless sensor networks. Toward this objective, the study goes through three phases. First, a cluster-based scheme is exploited. At every sampling instant, only one cluster of sensors that located in the proximity of the target is activated, whereas the other sensors are inactive. To activate the most appropriate cluster, we propose a nonmyopic rule, which is based on not only the target state prediction but also its future tendency. Second, the variational filtering algorithm is capable of precise tracking even in the highly nonlinear case. Furthermore, since the measurement incorporation and the approximation of the filtering distribution are jointly performed by variational calculus, an effective and lossless compression is achieved. The intercluster information exchange is thus reduced to one single Gaussian statistic, dramatically cutting down the resource consumption. Third, a binary proximity observation model is employed by the activated slave sensors to reduce the energy consumption and to minimize the intracluster communication. Finally, the effectiveness of the proposed approach is evaluated and compared with the state-of-the-art algorithms in terms of tracking accuracy, internode communication, and computation complexity.

Preimage Problem in Kernel-Based Machine Learning

While the nonlinear mapping from the input space to the feature space is central in kernel methods, the reverse mapping from the feature space back to the input space is also of primary interest. This is the case in many applications, including kernel principal component analysis (PCA) for signal and image denoising. Unfortunately, it turns out that the reverse mapping generally does not exist and only a few elements in the feature space have a valid preimage in the input space. The preimage problem consists of finding an approximate solution by identifying data in the input space based on their corresponding features in the high dimensional feature space. It is essentially a dimensionality-reduction problem, and both have been intimately connected in their historical evolution, as studied in this article.

Distributed Variational Filtering for Simultaneous Sensor Localization and Target Tracking in Wireless Sensor Networks

The tracking of a moving target in a wireless sensor network (WSN) requires exact knowledge of sensor positions. However, precise information about sensor locations is not always available. Given the observation that a series of measurements are generated in the sensors when the target moves through the network field, we propose an algorithm that exploits these measurements to simultaneously localize the detecting sensors and track the target (SLAT). The main difficulties that are encountered in this problem are the ambiguity of sensor locations, the unrestricted target moving manner, and the extremely constrained resources in WSNs. Therefore, a general state evolution model is employed to describe the dynamical system with neither prior knowledge of the target moving manner nor precise location information of the sensors. The joint posterior distribution of the parameters of interest is updated online by incorporating the incomplete and inaccurate measurements between the target and each of the sensors into a Bayesian filtering framework. A variational approach is adopted in the framework to approximate the filtering distribution, thus minimizing the intercluster communication and the error propagation. By executing the algorithm on a fully distributed cluster scheme, energy and bandwidth consumption in the network are dramatically reduced, compared with a centralized approach. Experiments on an SLAT problem validate the effectiveness of the proposed algorithm in terms of tracking accuracy, localization precision, energy consumption, and execution time.

Geometric Unmixing of Large Hyperspectral Images: A Barycentric Coordinate Approach

In hyperspectral imaging, spectral unmixing is one of the most challenging and fundamental problems. It consists of breaking down the spectrum of a mixed pixel into a set of pure spectra, called endmembers, and their contributions, called abundances. Many endmember extraction techniques have been proposed in literature, based on either a statistical or a geometrical formulation. However, most, if not all, of these techniques for estimating abundances use a least-squares solution. In this paper, we show that abundances can be estimated using a geometric formulation. To this end, we express abundances with the barycentric coordinates in the simplex defined by endmembers. We propose to write them in terms of a ratio of volumes or a ratio of distances, which are quantities that are often computed to identify endmembers. This property allows us to easily incorporate abundance estimation within conventional endmember extraction techniques, without incurring additional computational complexity. We use this key property with various endmember extraction techniques, such as N-Findr, vertex component analysis, simplex growing algorithm, and iterated constrained endmembers. The relevance of the method is illustrated with experimental results on real hyperspectral images.