Shounak Datta

Near-Bayesian Support Vector Machines for imbalanced data classification with equal or unequal misclassification costs

Support Vector Machines (SVMs) form a family of popular classifier algorithms originally developed to solve two-class classification problems. However, SVMs are likely to perform poorly in situations with data imbalance between the classes, particularly when the target class is under-represented. This paper proposes a Near-Bayesian Support Vector Machine (NBSVM) for such imbalanced classification problems, by combining the philosophies of decision boundary shift and unequal regularization costs. Based on certain assumptions which hold true for most real-world datasets, we use the fractions of representation from each of the classes, to achieve the boundary shift as well as the asymmetric regularization costs. The proposed approach is extended to the multi-class scenario and also adapted for cases with unequal misclassification costs for the different classes. Extensive comparison with standard SVM and some state-of-the-art methods is furnished as a proof of the ability of the proposed approach to perform competitively on imbalanced datasets. A modified Sequential Minimal Optimization (SMO) algorithm is also presented to solve the NBSVM optimization problem in a computationally efficient manner.

Handling data irregularities in classification: Foundations, trends, and future challenges

Abstract Most of the traditional pattern classifiers assume their input data to be well-behaved in terms of similar underlying class distributions, balanced size of classes, the presence of a full set of observed features in all data instances, etc. Practical datasets, however, show up with various forms of irregularities that are, very often, sufficient to confuse a classifier, thus degrading its ability to learn from the data. In this article, we provide a bird’s eye view of such data irregularities, beginning with a taxonomy and characterization of various distribution-based and feature-based irregularities. Subsequently, we discuss the notable and recent approaches that have been taken to make the existing stand-alone as well as ensemble classifiers robust against such irregularities. We also discuss the interrelation and co-occurrences of the data irregularities including class imbalance, small disjuncts, class skew, missing features, and absent (non-existing or undefined) features. Finally, we uncover a number of interesting future research avenues that are equally contextual with respect to the regular as well as deep machine learning paradigms.

A feature weighted penalty based dissimilarity measure for k-nearest neighbor classification with missing features

kNN-FWPD classifier is proposed with FWPD as the underlying dissimilarity measure.kNN-FWPD classifier can be directly applied to datasets having missing features.The proposed classifier has similar time complexity compared to the kNN classifier.Experiments are conducted on 4 types of missingness: MCAR, MAR, MNAR1, and MNAR2.kNN-FWPD is found to outperform ZI, AI, and kNNI in terms of classification accuracy. The k-Nearest Neighbor (kNN) classifier is an elegant learning algorithm widely used because of its simple and non-parametric nature. However, like most learning algorithms, kNN cannot be directly applied to data plagued by missing features. We make use of the philosophy of a Penalized Dissimilarity Measure (PDM) and incorporate a PDM called the Feature Weighted Penalty based Dissimilarity (FWPD) into kNN, forming the kNN-FWPD classifier which can be directly applied to datasets with missing features, without any preprocessing (like marginalization or imputation). Extensive experimentation on simulations of four different missing feature mechanisms (using various datasets) suggests that the proposed method can handle the missing feature problem much more effectively compared to some of the popular imputation mechanisms (used in conjunction with kNN).

Clustering with missing features: a penalized dissimilarity measure based approach

Many real-world clustering problems are plagued by incomplete data characterized by missing or absent features for some or all of the data instances. Traditional clustering methods cannot be directly applied to such data without preprocessing by imputation or marginalization techniques. In this article, we overcome this drawback by utilizing a penalized dissimilarity measure which we refer to as the feature weighted penalty based dissimilarity (FWPD). Using the FWPD measure, we modify the traditional k-means clustering algorithm and the standard hierarchical agglomerative clustering algorithms so as to make them directly applicable to datasets with missing features. We present time complexity analyses for these new techniques and also undertake a detailed theoretical analysis showing that the new FWPD based k-means algorithm converges to a local optimum within a finite number of iterations. We also present a detailed method for simulating random as well as feature dependent missingness. We report extensive experiments on various benchmark datasets for different types of missingness showing that the proposed clustering techniques have generally better results compared to some of the most well-known imputation methods which are commonly used to handle such incomplete data. We append a possible extension of the proposed dissimilarity measure to the case of absent features (where the unobserved features are known to be undefined).

A Radial Boundary Intersection aided interior point method for multi-objective optimization

Abstract We propose a novel multi-objective optimization technique combining non-convex Radial Boundary Intersection based decomposition with an Interior Point method (which utilizes both line search and trust region steps) suitable for non-convex nonlinear optimization. Radial Boundary Intersection decomposes the multi-objective optimization problem into subproblems which are concerned with finding the solutions closest to a reference point along equally spaced lines emanating radially outwards from the latter point. The proposed approach is found to be able to generate good approximations of the Pareto front (including the periphery) by generating a sufficiently diverse set of Pareto optimal solutions. The proposed method is extensively tested on a large number of recent benchmark problems and real world problems and the performance is found to be favorable in comparison to those of some of the cutting-edge stochastic/evolutionary optimization algorithms that are commonly used to solve non-convex multi-objective optimization problems.

Fast automatic estimation of the number of clusters from the minimum inter-center distance for k-means clustering

Abstract Center-based clustering methods like k-Means intend to identify closely packed clusters of data points by respectively finding the centers of each cluster. However, k-Means requires the user to guess the number of clusters, instead of estimating the same on the run. Hence, the incorporation of accurate automatic estimation of the natural number of clusters present in a data set is important to make a clustering method truly unsupervised. For k-Means, the minimum of the pairwise distance between cluster centers decreases as the user-defined number of clusters increases. In this paper, we observe that the last significant reduction occurs just as the user-defined number surpasses the natural number of clusters. Based on this insight, we propose two techniques: the Last Leap (LL) and the Last Major Leap (LML) to estimate the number of clusters for k-Means. Over a number of challenging situations, we show that LL accurately identifies the number of well-separated clusters, whereas LML identifies the number of equal-sized clusters. Any disparity between the values of LL and LML can thus inform a user about the underlying cluster structures present in the data set. The proposed techniques are independent of the size of the data set, making them especially suitable for large data sets. Experiments show that LL and LML perform competitively with the best cluster number estimation techniques while imposing drastically lower computational burden.

Generalized mean based back-propagation of errors for ambiguity resolution

The Ambiguity Resolution problem is formally defined.A new multi-layer ambiguity resolving perceptron is proposed.A continuous and differentiable generalized mean based error function is introduced.Back-propagation algorithm for the proposed error function is formulated.The new method is compared with 4 alternatives to show its usefulness. Ambiguity in a dataset, characterized by data points having multiple target labels, may occur in many supervised learning applications. Such ambiguity originates naturally or from misinterpretation, faulty encoding, and/or incompleteness of data. However, most applications demand that a data point be assigned a single label. In such cases, the supervised learner must resolve the ambiguity. To effectively perform ambiguity resolution, we propose a new variant of the popular Multi-Layer Perceptron model, called the Generalized Mean Multi-Layer Perceptron (GMMLP). In GMMLP, a novel differentiable error function guides the back-propagation algorithm towards the minimum distant target for each data point. We evaluate the performance of the proposed algorithm against three alternative ambiguity resolvers on 20 new artificial datasets containing ambiguous data points. To further test for scalability and comparison with multi-label classifiers, 18 real datasets are also used to evaluate the new approach.

Rough-Fuzzy Collaborative Multi-level Image Thresholding: A Differential Evolution Approach

In this article, a granular computing based multi-level gray image thresholding algorithm is presented. An image is divided into spatial blocks called granules, and the classes of gray levels are represented using a fuzzy-rough collaborative approach, where the measure of roughness of a rough set is also modified from the classical definition of rough sets. This measure for each rough set is minimized simultaneously to obtain the optimal thresholds. Tchebycheff decomposition approach is employed to transform this multi-objective optimization problem to a single objective optimization problem. Differential Evolution (DE), one of the most efficient evolutionary optimizers of current interest, is used to optimize this single objective function, thus reducing the execution time. Superiority of the proposed method is presented by comparing it with some popular image thresholding techniques. MSSIM index and Probabilistic Rand Index (PRI) are used for quantitative comparison on the Berkley Image Segmentation Data Set (BSDS300).