George G. Cabral

Combining nearest neighbor data description and structural risk minimization for one-class classification

One-class classification is an important problem with applications in several different areas such as novelty detection, anomaly detection, outlier detection and machine monitoring. In this paper, we propose two novel methods for one-class classification, referred to as NNDDSRM and kNNDDSRM. The methods are based on the principle of structural risk minimization and the nearest neighbor data description (NNDD) one-class classifier. Experiments carried out using both artificial and real-world datasets show that the proposed methods are able to significantly reduce the number of stored prototypes in comparison to NNDD. The experimental results also show that the proposed methods outperformed NNDD—in terms of the area under the receiver operating characteristic (ROC) curve—on four of the five datasets considered in the experiments and had a similar performance on the remaining one.

One-Class Classification based on searching for the problem features limits

Novelty detection is a problem with a large number of relevant applications. For some applications, the main interest is the prevention or detection of undesired states. In some cases, these undesired states are not known in advance; in others, such as machine monitoring, for instance, machine breakdown may be very rare and examples of this event may be unavailable. In such cases, the most widely accepted approach is to model the normal behavior of the system in order to subsequently detect unknown events. This is the basic concept of One-Class Classification (OCC). In some preliminary works we have proposed the Feature Boundaries Detector for One-Class Classification FBDOCC method, which operates by examining each problem feature at a time. In this paper we propose an extensive study of the behavior of the FBDOCC and introduce another version of the method, namely FBDOCC2. This work, also considers the use of the Particle Swarm Optimization (PSO) algorithm to find the best parameter configuration of the proposed method. Furthermore, this work also introduces a procedure to improve the training time performance without degrading the quality of the classification as well as some other contributions hereafter described. A number of experiments were carried out with synthetic and real datasets aiming at comparing both versions of the FBDOCC method with the most recent and effective OCC methods, namely: SVDD, One-class SVM, Least Squares One-class SVM, Kernel PCA, Gaussian Process Prior OCC, Condensed Nearest Neighbor Data Description and One-class Random Forests. The evaluation metrics considered in the experiments are: (i) the Area Under the ROC Curve (AUC); (ii) the Matthews Correlation Coefficient (MCC); (iii) the training time; and (iv) the prototype reduction rate. Regarding the metrics AUC and MCC, the first FBDOCC version has presented the best overall results among all the methods whereas the FBDOCC2 obtained results comparable to the best methods in some experiments where the standard FBDOCC yielded a poorer performance. FBDOCC was the faster method to train in comparison to the other methods in all but one dataset. In addition, FBDOCC was much faster than all methods based on support vector machines.

A novel one-class classification method based on feature analysis and prototype reduction

One-class classification is an important problem with applications in several different areas such as outlier detection and machine monitoring. In this paper we propose a novel method for one-class classification which also implements prototype reduction. The main feature of the proposed method is to analyze every limit of all the feature dimensions to find the true border which describes the normal class. To this end, the proposed method simulates the novelty class by creating artificial prototypes outside the normal description. The method is able to describe data distributions with complex shapes. Aiming to assess the proposed method, we carried out experiments with synthetic and real datasets to compare it with the Support Vector Domain Description (SVDD), kMeansDD, ParzenDD and kNNDD methods. The experimental results show that our one-class classification approach outperformed the other methods in terms of the area under the receiver operating characteristic (ROC) curve in three out of six data sets. The results also show that the proposed method remarkably outperformed the SVDD regarding training time and reduction of prototypes.

One-class Classification for heart disease diagnosis

As has been shown by the recent literature, machine learning techniques are important tools for diagnosing a number of diseases. Hospitals and medical clinics store a large amount of data with respect to the treatment of their patients. However, rarely an analysis of these data is conducted in order to extract intrinsic information for modeling a specific problem. This work presents an analysis of medical data aimed at determining whether or not patients are cardiac. To this end, raw data was collected and preprocessed at a Brazilian local hospital in order to build a new dataset containing only non-invasive information of children with heart murmur symptoms. The gathered data contain information, such as height, weight, gender and birthday date. The collected data was shown to be very imbalanced. Due to this imbalance, we employ the One-class Classification (OCC) paradigm to solve the problem by experimenting five methods; including the FBDOCC, that we proposed in a previous paper. Furthermore, two additional datasets were experimented in order to assess effectiveness of One-Class classifiers on the domain of heart disease detection. The overall results show that the FBDOCC succeeded in this task, yielding, statistically, the best performance for the gathered dataset as well as the other two heart disease datasets.

Preprocessing unbalanced data using weighted support vector machines for prediction of heart disease in children

Machine learning techniques are an important tool for diagnosing a number of diseases, as has been shown by the recent literature. Hospitals and medical clinics have a huge amount of data about the treatment of their patients, however, rarely analysis of these data is performed in order to extract intrinsic information aimed at modeling a specific problem. This work presents an analysis of medical data aimed at determining whether children patients are cardiac or not. To this end, raw data was collected at a Brazilian local hospital to be preprocessed in order to build the classification models. Only non invasive information were used, such as height, weight, gender and birthday date to create another set of derived variables such as BMI (Body Mass Index) to support the classification phase. However, the collected data was shown to be very imbalanced. Aimed at treat this problem, many tecniques were employed and one new approach was proposed. The results shown that the proposed approach outperforms the other methods in three out of four evaluation metrics.

A hybrid method for novelty detection in time series based on states transitions and swarm intelligence

This paper introduces a novel instance-based one-class classification method for novelty detection in time series based on its states transition. The main feature of our work is to generate an efficient method which automatically finds the parameters (whose yields the best model) according with the quality of the discovered time series states and the validation error. This method involves clustering and reducing the number of samples in a training dataset which does not contain novelty samples. Experiments carried out using three real-world time series show that the proposed method is able to build models with a reduced number of stored prototypes. The results obtained by our method were compared with the results of the SAX and both methods have successfully detected the novelties, however, the parameters which resulted in the best SAX model were achieved without validation phase (i.e. analyzing the results obtained for the test set).

A Novel Method for One-Class Classification Based on the Nearest Neighbor Data Description and Structural Risk Minimization

One-class classification is an important problem with applications in several different areas such as novelty detection, outlier detection and machine monitoring. In this paper we propose a novel method for one-class classification, referred to as NNDDSRM. It is based on the principle of structural risk minimization and the nearest neighbor data description (NNDD) method. Experiments carried out using both artificial and real-world datasets show that the proposed method is able to significantly reduce the number of stored prototypes in comparison to NNDD. The experimental results also show that the proposed method outperformed NNDD - in terms of the area under the receiver operating characteristic (ROC) curve - on four of the five datasets considered in the experiments and had a similar performance on the remaining one.

A Comparative Analysis of One-class Structural Risk Minimization by Support Vector Machines and Nearest Neighbor Rule

One-class classification is an important problem with applications in several diAEerent areas such as outlier detection and machine monitoring. In this paper we propose a novel method for one-class classification, referred to as kernel k NNDDSRM. This is a modification of an earlier algorithm, the kNNDDSRM, which aims to make the method able to build more flexible descriptions with the use of the kernel trick. This modification does not aAEect the algorithm’s main feature which is the significant reduction in the number of stored prototypes in comparison to NNDD. Aiming to assess the results, we carried out experiments with synthetic and real data to compare the method with the support vector data description (SVDD) method. The experimental results show that our oneclass classification approach outperformed SVDD in terms of the area under the receiver operating characteristic (ROC) curve in six out of eight data sets. The results also show that the kernel kNNDDSRM remarkably outperformed kNNDDSRM.

Novelty Detection Based on Genuine Normal and Artificially Generated Novelty Examples

One-class classification (OCC) is an important problem with applications in several different areas such as outlier detection and machine monitoring. Since in OCC there are no examples of the novelty class, the description generated may be a tight or a bulky description. Both cases are undesirable. In order to create a proper description, the presence of examples of the novelty class is very important. However, such examples may be rare or absent during the modeling phase. In these cases, the artificial generation of novelty samples may overcome this limitation. In this work it is proposed a two steps approach for generating artificial novelty examples in order to guide the parameter optimization process. The results show that the adopted approach has shown to be competitive with the results achieved when using real (genuine) novelty samples.

One-Class classification through optimized feature boundaries detection and prototype reduction

In this paper we propose a novel method for one-class classification. The proposed method analyses the limit of all feature dimensions to find the true border which describes the normal class. To this end, it simulates the novelty class by creating artificial prototypes outside the normal description. The parameters involved in the definition of the border are optimized via particle swarm optimization (PSO), which enables the method to describe data distributions with complex shapes. An experimental analysis is conducted with the proposed method using twelve data sets and considering the performance measures (i) Area Under the ROC Curve (AUC), (ii) training time, and (iii) prototype reduction. A comparison with One-Class SVM (OCSVM), kMeansDD, ParzenDD and kNNDD is carried out. The results show that performance of the proposed method is equivalent to OCSVM regarding the AUC, yet the proposed method outperforms OCSVM regarding the number of stored prototypes and training time.