Antanas Verikas

Mining data with random forests: A survey and results of new tests

Random forests (RF) has become a popular technique for classification, prediction, studying variable importance, variable selection, and outlier detection. There are numerous application examples of RF in a variety of fields. Several large scale comparisons including RF have been performed. There are numerous articles, where variable importance evaluations based on the variable importance measures available from RF are used for data exploration and understanding. Apart from the literature survey in RF area, this paper also presents results of new tests regarding variable rankings based on RF variable importance measures. We studied experimentally the consistency and generality of such rankings. Results of the studies indicate that there is no evidence supporting the belief in generality of such rankings. A high variance of variable importance evaluations was observed in the case of small number of trees and small data sets.

Soft combination of neural classifiers: a comparative study

This paper presents four schemes for soft fusion of the outputs of multiple classifiers. In the first three approaches, the weights assigned to the classifiers or groups of them are data dependent. The first approach involves the calculation of fuzzy integrals. The second scheme performs weighted averaging with data-dependent weights. The third approach performs linear combination of the outputs of classifiers via the BADD defuzzification strategy. In the last scheme, the outputs of multiple classifiers are combined using Zimmermanns compensatory operator. An empirical evaluation using widely accessible data sets substantiates the validity of the approaches with data-dependent weights, compared to various existing combination schemes of multiple classifiers.

Feature selection with neural networks

We present a neural network based approach for identifying salient features for classification in feedforward neural networks. Our approach involves neural network training with an augmented cross-entropy error function. The augmented error function forces the neural network to keep low derivatives of the transfer functions of neurons when learning a classification task. Such an approach reduces output sensitivity to the input changes. Feature selection is based on the reaction of the cross-validation data set classification error due to the removal of the individual features. We demonstrate the usefulness of the proposed approach on one artificial and three real-world classification problems. We compared the approach with five other feature selection methods, each of which banks on a different concept. The algorithm developed outperformed the other methods by achieving higher classification accuracy on all the problems tested.

Hybrid and ensemble-based soft computing techniques in bankruptcy prediction: a survey

This paper presents a comprehensive review of hybrid and ensemble-based soft computing techniques applied to bankruptcy prediction. A variety of soft computing techniques are being applied to bankruptcy prediction. Our focus is on techniques, namely how different techniques are combined, but not on obtained results. Almost all authors demonstrate that the technique they propose outperforms some other methods chosen for the comparison. However, due to different data sets used by different authors and bearing in mind the fact that confidence intervals for the prediction accuracies are seldom provided, fair comparison of results obtained by different authors is hardly possible. Simulations covering a large variety of techniques and data sets are needed for a fair comparison. We call a technique hybrid if several soft computing approaches are applied in the analysis and only one predictor is used to make the final prediction. In contrast, outputs of several predictors are combined, to obtain an ensemble-based prediction.

Automated speech analysis applied to laryngeal disease categorization

The long-term goal of the work is a decision support system for diagnostics of laryngeal diseases. Colour images of vocal folds, a voice signal, and questionnaire data are the information sources to be used in the analysis. This paper is concerned with automated analysis of a voice signal applied to screening of laryngeal diseases. The effectiveness of 11 different feature sets in classification of voice recordings of the sustained phonation of the vowel sound /a/ into a healthy and two pathological classes, diffuse and nodular, is investigated. A k-NN classifier, SVM, and a committee build using various aggregation options are used for the classification. The study was made using the mixed gender database containing 312 voice recordings. The correct classification rate of 84.6% was achieved when using an SVM committee consisting of four members. The pitch and amplitude perturbation measures, cepstral energy features, autocorrelation features as well as linear prediction cosine transform coefficients were amongst the feature sets providing the best performance. In the case of two class classification, using recordings from 79 subjects representing the pathological and 69 the healthy class, the correct classification rate of 95.5% was obtained from a five member committee. Again the pitch and amplitude perturbation measures provided the best performance.

The mass appraisal of the real estate by computational intelligence

Mass appraisal is the systematic appraisal of groups of properties as of a given date using standardized procedures and statistical testing. Mass appraisal is commonly used to compute real estate tax. There are three traditional real estate valuation methods: the sales comparison approach, income approach, and the cost approach. Mass appraisal models are commonly based on the sales comparison approach. The ordinary least squares (OLS) linear regression is the classical method used to build models in this approach. The method is compared with computational intelligence approaches - support vector machine (SVM) regression, multilayer perceptron (MLP), and a committee of predictors in this paper. All the three predictors are used to build a weighted data-depended committee. A self-organizing map (SOM) generating clusters of value zones is used to obtain the data-dependent aggregation weights. The experimental investigations performed using data cordially provided by the Register center of Lithuania have shown very promising results. The performance of the computational intelligence-based techniques was considerably higher than that obtained using the official real estate models of the Register center. The performance of the committee using the weights based on zones obtained from the SOM was also higher than of that exploiting the real estate value zones provided by the Register center.

Increasing the discrimination power of the co-occurrence matrix-based features

This paper is concerned with an approach to exploiting information available from the co-occurrence matrices computed for different distance parameter values. A polynomial of degree n is fitted to each of 14 Haralicks coefficients computed from the average co-occurrence matrices evaluated for several distance parameter values. Parameters of the polynomials constitute a set of new features. The experimental investigations performed substantiated the usefulness of the approach.

A feature selection technique for generation of classification committees and its application to categorization of laryngeal images

This paper is concerned with a two phase procedure to select salient features (variables) for classification committees. Both filter and wrapper approaches to feature selection are combined in this work. In the first phase, definitely redundant features are eliminated based on the paired t-test. The test compares the saliency of the candidate and the noise features. In the second phase, the genetic search is employed. The search integrates the steps of training, aggregation of committee members, selection of hyper-parameters, and selection of salient features into the same learning process. A small number of genetic iterations needed to find a solution is the characteristic feature of the genetic search procedure developed. The experimental tests performed on five real-world problems have shown that significant improvements in classification accuracy can be obtained in a small number of iterations if compared to the case of using all the features available.

Colour image segmentation by modular neural network

In this paper segmentation of colour images is treated as a problem of classification of colour pixels. A hierarchical modular neural network for classification of colour pixels is presented. The n ...

Towards a computer-aided diagnosis system for vocal cord diseases

OBJECTIVE The objective of this work is to investigate a possibility of creating a computer-aided decision support system for an automated analysis of vocal cord images aiming to categorize diseases of vocal cords. METHODOLOGY The problem is treated as a pattern recognition task. To obtain a concise and informative representation of a vocal cord image, colour, texture, and geometrical features are used. The representation is further analyzed by a pattern classifier categorizing the image into healthy, diffuse, and nodular classes. RESULTS The approach developed was tested on 785 vocal cord images collected at the Department of Otolaryngology, Kaunas University of Medicine, Lithuania. A correct classification rate of over 87% was obtained when categorizing a set of unseen images into the aforementioned three classes. CONCLUSION Bearing in mind the high similarity of the decision classes, the results obtained are rather encouraging and the developed tools could be very helpful for assuring objective analysis of the images of laryngeal diseases.