Patrice Aknin

A hidden process regression model for functional data description. Application to curve discrimination

A new approach for functional data description is proposed in this paper. It consists of a regression model with a discrete hidden logistic process which is adapted for modeling curves with abrupt or smooth regime changes. The model parameters are estimated in a maximum likelihood framework through a dedicated expectation maximization (EM) algorithm. From the proposed generative model, a curve discrimination rule is derived using the maximum a posteriori rule. The proposed model is evaluated using simulated curves and real world curves acquired during railway switch operations, by performing comparisons with the piecewise regression approach in terms of curve modeling and classification.

2009 Special Issue: Time series modeling by a regression approach based on a latent process

Time series are used in many domains including finance, engineering, economics and bioinformatics generally to represent the change of a measurement over time. Modeling techniques may then be used to give a synthetic representation of such data. A new approach for time series modeling is proposed in this paper. It consists of a regression model incorporating a discrete hidden logistic process allowing for activating smoothly or abruptly different polynomial regression models. The model parameters are estimated by the maximum likelihood method performed by a dedicated Expectation Maximization (EM) algorithm. The M step of the EM algorithm uses a multi-class Iterative Reweighted Least-Squares (IRLS) algorithm to estimate the hidden process parameters. To evaluate the proposed approach, an experimental study on simulated data and real world data was performed using two alternative approaches: a heteroskedastic piecewise regression model using a global optimization algorithm based on dynamic programming, and a Hidden Markov Regression Model whose parameters are estimated by the Baum-Welch algorithm. Finally, in the context of the remote monitoring of components of the French railway infrastructure, and more particularly the switch mechanism, the proposed approach has been applied to modeling and classifying time series representing the condition measurements acquired during switch operations.

Dedicated sensor and classifier of rail head defects

Abstract This paper presents an original system based on a specialized eddy-current sensor for the inspection of railway tracks. The device aims at the detection of broken rails and large head spalls of the rail, and it has been designed to be mounted underneath automatic driving railway vehicles. In fact, less important types of defects can also be detected. Another use of the sensor would be for predictive management of the rail. All these applications need a procedure for parametrization of the output signals. Then a classification procedure is performed by a set of neural networks which is able to assign each ‘defect’ into one particular class.

Mixture Model Estimation with Soft Labels

This paper addresses classification problems in which the class membership of training data is only partially known. Each learning sample is assumed to consist in a feature vector and an imprecise and/or uncertain “soft” label m i defined as a Dempster-Shafer basic belief assignment over the set of classes. This framework thus generalizes many kinds of learning problems including supervised, unsupervised and semi-supervised learning. Here, it is assumed that the feature vectors are generated from a mixture model. Using the General Bayesian Theorem, we derive a criterion generalizing the likelihood function. A variant of the EM algorithm dedicated to the optimization of this criterion is proposed, allowing us to compute estimates of model parameters. Experimental results demonstrate the ability of this approach to exploit partial information about class labels.

Comparison of Supervised Self-Organizing Maps Using Euclidian or Mahalanobis Distance in Classification Context

The supervised self-organizing map consists in associating output vectors to input vectors through a map, after self-organizing it on the basis of both input and desired output given altogether. This paper compares the use of Euclidian distance and Mahalanobis distance for this model. The distance comparison is made on a data classification application with either global approach or partitioning approach. The Mahalanobis distance in conjunction with the partitioning approach leads to interesting classification results.

Combined use of partial least squares regression and neural network for diagnosis tasks

This work deals with a diagnosis system, based on a combined use of partial least squares regression (PLS) and neural network (NN). An application concerning the French railway track/vehicle transmission system illustrates this approach. It is shown that a reliable selection of a reduced set of relevant descriptors is made by the PLS regression. Moreover, the projection of the data on the first PLS plane allows to highlight trajectories of the evolution of the system state between different classes. The modeling of the process state is performed by a multilayer NN. In this case, the PLS algorithm provides also a suitable approach to initialize the NN weights and to determine the optimal number of hidden nodes.

A regression model with a hidden logistic process for feature extraction from time series

A new approach for feature extraction from time series is proposed in this paper. This approach consists of a specific regression model incorporating a discrete hidden logistic process. The model parameters are estimated by the maximum likelihood method performed by a dedicated Expectation Maximization (EM) algorithm. The parameters of the hidden logistic process, in the inner loop of the EM algorithm, are estimated using a multi-class Iterative Reweighted Least-Squares (IRLS) algorithm. A piecewise regression algorithm and its iterative variant have also been considered for comparisons. An experimental study using simulated and real data reveals good performances of the proposed approach.

HYBRID TRAINING OF RADIAL BASIS FUNCTION NETWORKS IN A PARTITIONING CONTEXT OF CLASSIFICATION

Abstract The design of radial basis function networks is rather complex because of the great number of parameters that must be adjusted: positioning and number of kernels, choice of the distance type and centre widths, weight values. This article details these points in the framework of classification tasks with a partitioning approach. An adaptation of the orthogonal least-squares method is presented in order to select the centres of each sub-classifier in connection with a particular stopping criterion based on the addition of a random centre. Moreover, different choices of distance and centre widths are compared and illustrated by a 4-class problem in the non-destructive evaluation domain.

Dedicated Sensor and Classifier of Rail Head Defects for Railway Systems

Abstract This article presents an original system based on specialized eddy current sensor for the inspection of the railway tracks. The device aims at the detection of broken rails and large head spalls of the rail and it has been designed in order to be mounted underneath automatic driving railway vehicles. In fact, less important types of defects can also be detected. Other uses of the sensor are predictive management of the rail. All the applications need a parametrization procedure of the output signals of the sensor. Then a classification is performed by a set of Neural Networks which is able to split the “defects” into several classes.

Model-based clustering with Hidden Markov Model regression for time series with regime changes

This paper introduces a novel model-based clustering approach for clustering time series which present changes in regime. It consists of a mixture of polynomial regressions governed by hidden Markov chains. The underlying hidden process for each cluster activates successively several polynomial regimes during time. The parameter estimation is performed by the maximum likelihood method through a dedicated Expectation-Maximization (EM) algorithm. The proposed approach is evaluated using simulated time series and real-world time series issued from a railway diagnosis application. Comparisons with existing approaches for time series clustering, including the stand EM for Gaussian mixtures, K-means clustering, the standard mixture of regression models and mixture of Hidden Markov Models, demonstrate the effectiveness of the proposed approach.