Paweł Ksieniewicz

Active Learning Classification of Drifted Streaming Data

Abstract Objects being recognized may arrive continuously to a classifier in the form of data stream, therefore contemporary classification systems have to make a decision not only on the basis of the static data, but on the data in motion as well. Additionally, we would like to start a classifier exploitation as soon as possible, then the models which can improve their models during exportation are very desirable. Basically, we may produce the model on the basis a few learning objects only and then we use and improve the classifier when new data comes. This concept is still vibrant and may be used in the plethora of practical cases. Nevertheless, constructing such a system we should realize, that we have the limited resources (as memory and computational power) at our disposal. Additionally, during the exploitation of a classifier system the chosen characteristic of the classifier model may change within a time. This phenomena is called concept drift and may lead the deep deterioration of the classification performance. This work deals with the data stream classification with the presence of concept drift . We propose a novel classifier training algorithm based on the sliding windows approach, which allows us to implement forgetting mechanism, i.e., that old objects come from outdated model will not be taken into consideration during the classifier updating and on the other hand we assume that only part of arriving examples can be labeled, because we assume that we have a limited budget for labeling. We will employ active learning paradigm to choose an “interesting” objects to be be labeled. The proposed approach has been evaluated on the basis of the computer experiments carried out on the data streams. Obtained results confirmed the usability of proposed method to the smoothly drifted data stream classification.

Ensembles of Heterogeneous Concept Drift Detectors - Experimental Study

For the contemporary enterprises, possibility of appropriate business decision making on the basis of the knowledge hidden in stored data is the critical success factor. Therefore, the decision support software should take into consideration that data usually comes continuously in the form of so-called data stream, but most of the traditional data analysis methods are not ready to efficiently analyze fast growing amount of the stored records. Additionally, one should also consider phenomenon appearing in data stream called concept drift, which means that the parameters of an using model are changing, what could dramatically decrease the analytical model quality. This work is focusing on the classification task, which is very popular in many practical cases as fraud detection, network security, or medical diagnosis. We propose how to detect the changes in the data stream using combined concept drift detection model. The experimental evaluations confirm its pretty good quality, what encourage us to use it in practical applications.

Ensemble of Extreme Learning Machines with trained classifier combination and statistical features for hyperspectral data

Abstract Remote sensing and hyperspectral data analysis are areas offering wide range of valuable practical applications. However, they generate massive and complex data that is very difficult to be analyzed by a human being. Therefore, methods for efficient data representation and data mining are of high interest to these fields. In this paper, we introduce a novel pipeline for feature extraction and classification of hyperspectral images. To obtain a compressed representation we propose to extract a set of statistical-based properties from these images. This allows for embedding feature space into fourteen channels, obtaining a significant dimensionality reduction. These features are used as an input for the ensemble learning based on randomized neural networks. We introduce a novel method for forming ensembles of Extreme Learning Machines based on randomized feature subspaces and a trained combiner. It is based on continuous outputs and uses a perceptron-based learning scheme to calculate weights assigned to each classifier and class independently. Extensive experiments carried on a number of benchmarks images prove that using proposed feature extraction and extreme learning ensemble leads to a significant gain in classification accuracy.

Active Learning Classifier for Streaming Data

This work reports the research on active learning approach applied to the data stream classification. The chosen characteristics of the proposed frameworks were evaluated on the basis of the wide range of computer experiments carried out on the three benchmark data streams. Obtained results confirmed the usability of proposed method to the data stream classification with the presence of incremental concept drift.

Hyperspectral Image Analysis Based on Color Channels and Ensemble Classifier

Hyperspectral image analysis is a dynamically developing branch of computer vision due to the numerous practical applications and high complexity of data. There exist a need for introducing novel machine learning methods, that can tackle high dimensionality and large number of classes in these images. In this paper, we introduce a novel ensemble method for classification of hyperspectral data. The pool of classifiers is built on the basis of color decomposition of the given image. Each base classifier corresponds to a single color channel that is extracted. We propose a new method for decomposing hyperspectral image into 11 different color channels. As not all of the channels may bear as useful information as other, we need to promote the most relevant ones. For this, our ensemble uses a weighted trained fuser, which uses a neural methods for establishing weights. We show, that the proposed ensemble can outperform other state-of-the-art classifiers in the given task.

Ensemble of One-Dimensional Classifiers for Hyperspectral Image Analysis

Remote sensing and hyperspectral data analysis are areas offering wide range of valuable practical applications. However, they generate massive and complex data that is very difficult to be analyzed by a human being. Therefore, methods for efficient data representation and data mining are of high interest to these fields. In this paper we introduce a novel pipeline for feature extraction and classification of hyperspectral images. To obtain a compressed representation we propose to extract a set of statistical-based properties from these images. This allows for embedding feature space into fourteen channels, obtaining a significant dimensionality reduction. These features are used as an input for the ensemble learning based on minimal-distance classifiers. We introduce a novel method for forming ensembles simple one dimensional classifiers. They are constructed independently on a low-dimensional representation - a single classifier for each extracted feature. Then a voting procedure is being used to obtain the final decision. Extensive experiments carried on a number of benchmarks images prove that using proposed feature extraction and ensemble of simple classifiers can offer a significant improvement in terms of classification accuracy when compared to state-of-the-art methods.

Artificial Photoreceptors for Ensemble Classification of Hyperspectral Images

Data obtained by hyperspectral imaging gives us enough information to recreate the human vision, and also to extend it by a new methods to extract features coded in a light spectra. This work proposes a set of functions, based on abstraction of natural photoreceptors. The proposed method was employed as the feature extraction for the classification system based on combined approach and compared with other state-of-art methods on the basis of the selected benchmark images.

Blurred Labeling Segmentation Algorithm for Hyperspectral Images

This work is focusing on the hyperspectral imaging classification, which is nowadays a focus of intense research. The hyperspectral imaging is widely used in agriculture, mineralogy, or food processing to enumerate only a few important domains. The main problem of such image classification is access to the ground truth, because it needs the experienced experts. This work proposed a novel three-stage image segmentation method, which prepares the data for the classification and employs the active learning paradigm which reduces the expert works on image. The proposed approach was evaluated on the basis of the computer experiments carried out on the benchmark hyperspectral datasets.

Hyperspectral Image Analysis Based on Quad Tree Decomposition

Hyperspectral image analysis is among one of the current trends in computer vision and machine learning. Due to the high dimensionality, large number of classes, presence of noise and complex structure, this is not a trivial task. There exists a need for more precise and computationally efficient algorithms for hyperspectral image segmentation and classification. In this paper, we introduce a novel algorithm for segmentation of hyperspectral images and selecting valuable pixels for classifier training procedure. Our approach is based on Quad Tree Decomposition method, which detects homogeneous region in the input image. This allows to precisely detect groups with similar structure and recognize different possible classes on the image. We discuss the computational complexity of our algorithm and show that it can be easily used in real-life applications. Further, this algorithm is extended by active learning approach, which allows to select a single representative pixel for each detected homogeneous region. With this, the classifier is trained on a significantly reduced dataset without sacrificing its quality. We examine the correlation between the number of folds taken by our segmentation algorithm and used classifiers. We show, that the segmentation procedure can be stopped earlier without drop of the accuracy.

Ensemble Classifier Systems for Headache Diagnosis

Headache, medically known as cephalalgia, may have a wide range of symptoms and its types may be related and mixed. Its proper diagnosis is difficult and automatic diagnosis is usually rather imprecise, therefore, the problem is still the focus of intensive research. In the paper we propose headache diagnosis method which makes the decision on the basis of questionnaire only. It distinguished among 11 headache classes, which taxonomy is provided. The paper presents results of experiments which aim at selecting the best classification algorithm including several classical machine learning methods as well as ensemble approach. Results of experiments carried on dataset collected in University of Novi Sad confirm that the automatic classification system can gain high accuracy of classification for the problem under consideration.