Pierre Beauseroy

Mutual information-based feature extraction on the time-frequency plane

A method is proposed for automatic extraction of effective features for class separability. It applies to nonstationary processes described only by sample sets of stochastic signals. The extraction is based on time-frequency representations (TFRs) that are potentially suited to the characterization of nonstationarities. The features are defined by parameterized mappings applied to a TFR. These mappings select a region of the time-frequency plane by using a two-dimensional (2-D) parameterized weighting function and provide a standard characteristic in the restricted representation obtained. The features are automatically drawn from the TFR by tuning the weighting function parameters. The extraction is driven to maximize the information brought by the features about the class membership. It uses a mutual information criterion, based on estimated probability distributions. The framework is developed for the extraction of a single feature and extended to several features. A classification scheme adapted to the extracted features is proposed. Finally, some experimental results are given to demonstrate the efficacy of the method.

l p -norms in One-Class Classification for Intrusion Detection in SCADA Systems.

The massive use of information and communication technologies in supervisory control and data acquisition (SCADA) systems opens new ways for carrying out cyberattacks against critical infrastructures relying on SCADA networks. The various vulnerabilities in these systems and the heterogeneity of cyberattacks make the task extremely difficult for traditional intrusion detection systems (IDS). Modeling cyberattacks has become nearly impossible and their potential consequences may be very severe. The primary objective of this work is to detect malicious intrusions once they have already bypassed traditional IDS and firewalls. This paper investigates the use of machine learning for intrusion detection in SCADA systems using one-class classification algorithms. Two approaches of one-class classification are investigated: 1) the support vector data description (SVDD); and 2) the kernel principle component analysis. The impact of the considered metric is examined in detail with the study of lp-norms in radial basis function (RBF) kernels. A heuristic is proposed to find an optimal choice of the bandwidth parameter in these kernels. Tests are conducted on real data with several types of cyberattacks.

Multi-task learning with one-class SVM

Multi-task learning technologies have been developed to be an effective way to improve the generalization performance by training multiple related tasks simultaneously. The determination of the relatedness between tasks is usually the key to the formulation of a multi-task learning method. In this paper, we make the assumption that when tasks are related to each other, usually their models are close enough, that is, their models or their model parameters are close to a certain mean function. Following this task relatedness assumption, two multi-task learning formulations based on one-class support vector machines (one-class SVM) are presented. With the help of new kernel design, both multi-task learning methods can be solved by the optimization program of a single one-class SVM. Experiments conducted on both low-dimensional nonlinear toy dataset and high-dimensional textured images show that our approaches lead to very encouraging results.

Optimal Decision Rule with Class-Selective Rejection and Performance Constraints

The problem of defining a decision rule which takes into account performance constraints and class-selective rejection is formalized in a general framework. In the proposed formulation, the problem is defined using three kinds of criteria. The first is the cost to be minimized, which defines the objective function, the second are the decision options, determined by the admissible assignment classes or subsets of classes, and the third are the performance constraints. The optimal decision rule within the statistical decision theory framework is obtained by solving the stated optimization problem. Two examples are provided to illustrate the formulation and the decision rule is obtained.

Multilabel Classification Rule with Performance Constraints

A formulation for multilabel and performance constraints classification problems is presented within the framework of statistical decision theory. The definition of the problem takes into account three concerns. The first is the cost function which defines the criterion to minimize; the second is the decision options which are defined by the admissible assignment classes or subsets of classes and the third one is the constraints of performance. Assuming that the conditional probability density functions are known, the classification rule that is solution of the stated problem is expounded. Two examples are provided to illustrate the formulation and the decision rule obtained

General solution and learning method for binary classification with performance constraints

In this paper, the problem of binary classification is studied with one or two performance constraints. When the constraints cannot be satisfied, the initial problem has no solution and an alternative problem is solved by introducing a rejection option. The optimal solution for such problems in the framework of statistical hypothesis testing is shown to be based on likelihood ratio with one or two thresholds depending on whether it is necessary to introduce a rejection option or not. These problems are then addressed when classes are only defined by labelled samples. To illustrate the resolution of cases with and without rejection option, the problem of Neyman-Pearson and the one of minimizing reject probability subject to a constraint on error probability are studied. Solutions based on SVMs and on a kernel based classifier are experimentally compared and discussed.

Mahalanobis-based one-class classification

Machine learning techniques have become very popular in the past decade for detecting nonlinear relations in large volumes of data. In particular, one-class classification algorithms have gained the interest of the researchers when the available samples in the training set refer to a unique/single class. In this paper, we propose a simple one-class classification approach based on the Mahalanobis distance. We make use of the advantages of kernel whitening and KPCA in order to compute the Mahalanobis distance in the feature space, by projecting the data into the subspace spanned by the most relevant eigenvectors of the covariance matrix. We also propose a sparse formulation of this approach. The tests are conducted on simulated data as well as on real data.

Detection of cyberattacks in a water distribution system using machine learning techniques

Cyberattacks threatening the industrial processes and the critical infrastructures have become more and more complex, sophisticated, and hard to detect. These cyberattacks may cause serious economic losses and may impact the health and safety of employees and citizens. Traditional Intrusion Detection Systems (IDS) cannot detect new types of cyberattacks not existing in their databases. Therefore, IDS need a complementary help to provide a maximum protection to industrial systems against cyberattacks. In this paper, we propose to use machine learning techniques, in particular one-class classification, in order to bring the necessary and complementary help to IDS in detecting cyberattacks and intrusions. One-class classification algorithms have been used in many data mining applications, where the available samples in the training dataset refer to a unique/single class.We propose a simple one-class classification approach based on a new novelty measure, namely the truncated Mahalanobis distance in the feature space. The tests are conducted on a real dataset from the primary water distribution system in France, and the proposed approach is compared with other well-known one-class approaches.

New feature selection method based on neural network and machine learning

Feature selection becomes the focus of much research in many areas of applications for which datasets with large number of features are available. Feature selection is a problem of choosing a subset of relevant features to increase the execution speed of the algorithm and the classification accuracy. It also removes inappropriate features to increase the precision and improve the performances. There has been much effort for solving the feature selection problem up to now and many researchers have proposed and developed many feature selection algorithms in this purpose. In this paper, we propose a new feature selection method based on neural network and machine learning. This new algorithm tends to highlight the best features among existing ones: new weighting-based method of the input features is used in the neural network to choose the best features. Performances show that this method selects the best features on simulated data.

Gene-based multiclass cancer diagnosis with class-selective rejections.

Supervised learning of microarray data is receiving much attention in recent years. Multiclass cancer diagnosis, based on selected gene profiles, are used as adjunct of clinical diagnosis. However, supervised diagnosis may hinder patient care, add expense or confound a result. To avoid this misleading, a multiclass cancer diagnosis with class-selective rejection is proposed. It rejects some patients from one, some, or all classes in order to ensure a higher reliability while reducing time and expense costs. Moreover, this classifier takes into account asymmetric penalties dependant on each class and on each wrong or partially correct decision. It is based on ν-1-SVM coupled with its regularization path and minimizes a general loss function defined in the class-selective rejection scheme. The state of art multiclass algorithms can be considered as a particular case of the proposed algorithm where the number of decisions is given by the classes and the loss function is defined by the Bayesian risk. Two experiments are carried out in the Bayesian and the class selective rejection frameworks. Five genes selected datasets are used to assess the performance of the proposed method. Results are discussed and accuracies are compared with those computed by the Naive Bayes, Nearest Neighbor, Linear Perceptron, Multilayer Perceptron, and Support Vector Machines classifiers.