Clement Magnant

On Computing Jeffrey’s Divergence Between Time-Varying Autoregressive Models

Autoregressive (AR) and time-varying AR (TVAR) models are widely used in various applications, from speech processing to biomedical signal analysis. Various dissimilarity measures such as the Itakura divergence have been proposed to compare two AR models. However, they do not take into account the variances of the driving processes and only apply to stationary processes. More generally, the comparison between Gaussian processes is based on the Kullback-Leibler (KL) divergence but only asymptotic expressions are classically used. In this letter, we suggest analyzing the similarities of two TVAR models, sample after sample, by recursively computing the Jeffreys divergence between the joint distributions of the successive values of each TVAR model. Then, we show that, under some assumptions, this divergence tends to the Itakura divergence in the stationary case.

Jeffrey's divergence for state-space model comparison

Optimal filters such as Kalman filters are used in a wide range of applications from speech enhancement to biomedical applications. They are based on an a priori state-space model describing a dynamic system. If this model is not well-suited, the accuracy of the state vector may be poor. Therefore, several estimators based on different models can be combined. However, state-space models that are dissimilar enough must be chosen. To our knowledge, there are no guidelines to select them, we hence address this issue in this paper. Given an initial model set, our aim is to determine subsets of similar models by using Jeffrey?s divergence between the distributions of the state-vector time paths based on the different models. Our approach operates with the following steps: the so-called dissimilarity matrix composed of Jeffrey?s divergences between model pairs is created. Then, this matrix is transformed to get the same properties as a correlation matrix and an eigenvalue decomposition is performed. Subsequently, we propose an interpretation of the predominant eigenvalues which is then used to deduce the number of model subsets and their cardinals. A classification algorithm can then be considered to determine which models belong to which subsets. HighlightsWe propose to compare two state models by using the Jeffreys divergence (JD).We generalize the above approach to determine subsets in an initial model set.The eigenvalues decomposition is then interpreted to deduce the number of model subsets.Then, a classification algorithm can be used to determine which model belongs to which subset.The method is more particularly applied to a toy example and to tracking.

Classifying several state models using Jeffrey's divergence: Application to target tracking

One of the most important challenges when applying multiple-model approaches is model-set design. However, to our knowledge, there are not general rules to choose models. For this purpose, we recently proposed an approach based on the Jeffreys divergence to characterize dissimilarities between two state models. In this paper, our contribution is to use the Jeffreys divergence to classify at least two models into subsets. Our approach consists in creating a dissimilarity matrix composed of Jeffreys divergences between model pairs. Then, we transform this matrix to get a correlation-like matrix and an eigenvalue decomposition is computed. We propose an interpretation of the predominant eigenvalues and use it to deduce the number of model subsets and their cardinals. Finally, a classification algorithm can be considered to determine which models belong to which subsets. Among the applications, we focus on target tracking.

Compensating power amplifier distortion in cognitive radio systems with adaptive interacting multiple model

This work aims at improving the power amplifier (PA) efficiency in uplink OFDM-based cognitive radio (CR) communications. Unlike the traditional approaches, we suggest transmitting a non-linearily ampliied signal without any il-tering and addressing the OFDM sample estimation from the distorted signal at the receiver. The proposed post-distortion and detection technique is based on a Volterra model for the PA and the channel. As the transmission can switch from one sub-band to another, the CR-PA behavior varies over time and the Volterra kernels can be constant or suddenly change. Therefore, an interactive multiple model (IMM) combining extended Kalman filters is considered. The transition probability matrix, which plays a key role in the IMM, is also sequentially estimated. The resulting uplink system has various advantages: it learns from the observations and a part of the computational load is exported to the receiver, which is not battery driven unlike the mobile terminal.

Classifying autoregressive models using dissimilarity measures: A comparative study

Autoregressive (AR) models are used in various applications, from speech processing to radar signal analysis. In this paper, our purpose is to extract different model subsets from a set of two or more AR models. The approach operates with the following steps: firstly the matrix composed of dissimilarity measures between AR-model pairs are created. This can be based on the symmetric Itakura divergence, the symmetric Itakura-Saito divergence, the log-spectral distance or Jeffreys divergence (JD), which corresponds to the symmetric version of the Kullback-Leibler divergence. These matrices are then transformed to get the same properties as correlation matrices. Eigenvalue decompositions are performed to get the number of AR-model subsets and the estimations of their cardinals. Finally, K-means are used for classification. A comparative study points out the relevance of the JD-based method. Illustrations with sea radar clutter are also provided.

Multi-target tracking using a PHD-based joint tracking and classification algorithm

When using Bayesian estimation techniques for target tracking, the algorithm accuracy is induced by the choice of the system evolution model. Information on the type of target and its maneuver capability can then be helpful to choose relevant motion models. Joint tracking and classification (JTC) methods based on target features have thus been introduced. Among them, we recently proposed to take into account the target extent measurements for single-target tracking. In this paper, we extend this work to multi-target tracking (MTT) by using probability hypothesis density (PHD) filters. More precisely, assuming that each target class is characterized by its own kinematic-model set, a multiple-model (MM) PHD filter is used for each class. State estimates from each class are then combined by using class probabilities. Finally, the proposed approach, namely a multiclass MM-GMPHD, is applied to maritime-target tracking and simulation results show the relevance of the proposed approach regarding the tracking of various types of targets.

Bayesian non-parametric methods for dynamic state-noise covariance matrix estimation

When using Bayesian estimation techniques, the algorithm is strongly sensitive to the system evolution model and more particularly to the setting of the state-noise covariance matrix. Recently, Bayesian non-parametric models and in particular Dirichlet processes (DPs) have been proposed as a scalable solution to this issue. They assume that the system can switch between an infinite number of state-space representations corresponding to different values of the state-noise covariance matrix. In this framework, jointly estimating the state vector and the covariance matrix is a non-linear non-Gaussian problem. The inference is thus classically carried out using particle filtering techniques. In this case, the choice of the proposal distribution for the particles is of paramount importance regarding the estimation accuracy. A first contribution of this paper is to derive an approximation of the optimal proposal distribution of the particle filter when a DP prior is placed on the distribution of the state-noise covariance matrix. Then, an alternative DP-based formulation of the inference problem is proposed to reduce its dimensionality. It takes advantage that the possible functional forms of the state-noise covariance matrices are known up to a reduced number of time-switching hyperparameters in many applications. An approximation of the optimal proposal distribution is also derived. Finally, the relevance of both proposed approaches is analyzed in the framework of target tracking and a comparative study with existing methods is carried out. HighlightsWe address the joint estimations of the state vector and the state-noise covariance matrix.We use Bayesian non-parametric methods implemented by particle filtering.Two approaches are presented.In both cases, the optimal proposal distribution is derived.Both proposed algorithms are applied to target tracking.

Dirichlet-process-mixture-based Bayesian nonparametric method for Markov switching process estimation

Dirichlet process (DP) mixtures were recently introduced to deal with switching linear dynamical models (SLDM). They assume the system can switch between an a priori infinite number of state-space representations (SSR) whose parameters are on-line inferred. The estimation problem can thus be of high dimension when the SSR matrices are unknown. Nevertheless, in many applications, the SSRs can be categorized in different classes. In each class, the SSRs are characterized by a known functional form but differ by a reduced set of unknown hyperparameters. To use this information, we thus propose a new hierarchical model for the SLDM wherein a discrete variable indicates the SSR class. Conditionally to this class, the distributions of the hyperparameters are modeled by DPs. The estimation problem is solved by using a Rao-Blackwellized particle filter. Simulation results show that our model outperforms existing methods in the field of target tracking.