Jin-Mao Wei

Feature Selection by Maximizing Independent Classification Information

Feature selection approaches based on mutual information can be roughly categorized into two groups. The first group minimizes the redundancy of features between each other. The second group maximizes the new classification information of features providing for the selected subset. A critical issue is that large new information does not signify little redundancy, and vice versa. Features with large new information but with high redundancy may be selected by the second group, and features with low redundancy but with little relevance with classes may be highly scored by the first group. Existing approaches fail to balance the importance of both terms. As such, a new information term denoted as Independent Classification Information is proposed in this paper. It assembles the newly provided information and the preserved information negatively correlated with the redundant information. Redundancy and new information are properly unified and equally treated in the new term. This strategy helps find the predictive features providing large new information and little redundancy. Moreover, independent classification information is proved as a loose upper bound of the total classification information of feature subset. Its maximization is conducive to achieve a high global discriminative performance. Comprehensive experiments demonstrate the effectiveness of the new approach.

Probabilistic Confusion Entropy for Evaluating Classifiers

For evaluating the classification model of an information system, a proper measure is usually needed to determine if the model is appropriate for dealing with the specific domain task. Though many performance measures have been proposed, few measures were specially defined for multi-class problems, which tend to be more complicated than two-class problems, especially in addressing the issue of class discrimination power. Confusion entropy was proposed for evaluating classifiers in the multi-class case. Nevertheless, it makes no use of the probabilities of samples classified into different classes. In this paper, we propose to calculate confusion entropy based on a probabilistic confusion matrix. Besides inheriting the merit of measuring if a classifier can classify with high accuracy and class discrimination power, probabilistic confusion entropy also tends to measure if samples are classified into true classes and separated from others with high probabilities. Analysis and experimental comparisons show the feasibility of the simply improved measure and demonstrate that the measure does not stand or fall over the classifiers on different datasets in comparison with the compared measures.

A novel method for pruning decision trees

Pruning decision trees is deemed an effective way of solving over-fitting in practice. Pruned decision trees usually have simpler structure and are expected to have higher generalization ability at the expense of classification accuracy. Nowadays, various pruning methods are available. However, the problem of how to make a trade-off between structural simplicity and classification accuracy has not been well solved. In this paper, we firstly propose a method to evaluate structural complexities of decision trees in pruning process. Based upon the method, we introduce a new measure for post-pruning decision trees, which takes into account both classification accuracy and structural complexity. The experimental results on 20 benchmark data sets from the UCI machine learning data repository show that the proposed method is competitively feasible for pruning decision trees.

An Enhanced Convolutional Neural Network Model for Answer Selection

Answer selection is an important task in question answering (QA) from the Web. To address the intrinsic difficulty in encoding sentences with semantic meanings, we introduce a general framework, i.e., Lexical Semantic Feature based Skip Convolution Neural Network (LSF-SCNN), with several optimization strategies. The intuitive idea is that the granular representations with more semantic features of sentences are deliberately designed and estimated to capture the similarity between question-answer pairwise sentences. The experimental results demonstrate the effectiveness of the proposed strategies and our model outperforms the state-of-the-art ones by up to 3.5% on the metrics of MAP and MRR.

Supervised Feature Selection by Preserving Class Correlation

Feature selection is an effective technique for dimension reduction, which assesses the importance of features and constructs an optimal feature subspace suitable for recognition task. Two recognition scenarios, i.e., single-label learning and multi-label learning, pose different challenges for feature selection. For the single-label task, how to accurately measure and reduce feature redundancy is crucial. For the multi-label task, how to effectively exploit class correlation information during selection is critical. However, both issues cannot be simultaneously resolved by any existing selection methods. In this paper, we propose effective supervised feature selection techniques to address the problems. The original class correlation information in the reduced feature space is preserved, and meanwhile the feature redundancy for classification is alleviated. To the best of our knowledge, this study is the first attempt to accomplish both recognition tasks in a unified framework. Comprehensive experimental evaluations on artificial, single-label, and multi-label data sets demonstrate the effectiveness of the new approach.

Learning decision trees using confusion entropy

Confusion entropy is a new measure for evaluating performance of classifiers. For each class in a classification problem, the CEN metric considers not only the misclassification information about how the true samples in this class have been misclassified to the other classes, but also the misclassification information about how the other samples have been misclassified to this class. In this paper we propose a novel splitting criterion named CENsplit based on CEN for learning decision trees with higher performance, especially with regard to class discrimination power of the induced trees. Experimental results on some data sets show that CENsplit criterion leads to trees with better CEN value without reducing accuracy.

Probabilistic Topic and Role Model for Information Diffusion in Social Network

Information diffusion, which addresses the issue of how a piece of information spreads and reaches individuals in or between networks, has attracted considerable research attention due to its widespread applications, such as viral marketing and rumor control. However, the process of information diffusion is complex and its underlying mechanism remains unclear. An important reason is that social influence takes many forms and each form may be determined by various factors. One of the major challenges is how to capture all the crucial factors of a social network such as users’ interests (which can be represented as topics), users’ attributes (which can be summarized as roles), and users’ reposting behaviors in a unified manner to model the information diffusion process. To address the problem, we propose the joint information diffusion model (TRM) that integrates user topical interest extraction, role recognition, and information diffusion modeling into a unified framework. TRM seamlessly unifies the user topic role extraction, role recognition, and modeling of information diffusion, and then translates the calculations of individual level influence to the role-topic pairwise influence, which can provide a coarse-grained diffusion representation. Extensive experiments on two real-world datasets validate the effectiveness of our approach under various evaluation indices, which performs superior than the state-of-the-art models by a large margin.

HAVAE: Learning Prosodic-Enhanced Representations of Rap Lyrics

Learning and analyzing rap lyrics is a significant basis for many applications, such as music recommendation, automatic music categorization, and music information retrieval. Although numerous studies have explored the topic, knowledge in this field is far from satisfactory, because critical issues, such as prosodic information and its effective representation, as well as appropriate integration of various features are usually ignored. In this paper, we propose a hierarchical attention variational autoencoder framework (HAVAE), which simultaneously consider semantic and prosodic features for rap lyrics representation learning. Specifically, the representation of the prosodic features is encoded by phonetic transcriptions with a novel and effective strategy (i.e., rhyme2vec). Moreover, a feature aggregation strategy is proposed to appropriately integrate various features and generate prosodic-enhanced representation. A comprehensive empirical evaluation demonstrates that the proposed framework outperforms the state-of-the-art approaches under various metrics in both NextLine prediction task and rap genre classification task.

ANNC: AUC-Based Feature Selection by Maximizing Nearest Neighbor Complementarity.

Feature selection is crucial for dimension reduction. Dozens of approaches employ the area under ROC curve, i.e., AUC, to evaluate features, and have shown their attractiveness in finding discriminative targets. However, feature complementarity for jointly discriminating classes is generally improperly handled by these approaches. In a recent approach to deal with such issues, feature complementarity was evaluated by computing the difference between the neighbors of each instance in different feature dimensions. This local-learning based approach introduces a distinctive way to determine how a feature is complementarily discriminative given another. Nevertheless, neighbor information is usually sensitive to noises. Furthermore, evaluating merely one-side information of nearest misses will definitely neglect the impacts of nearest hits on feature complementarity. In this paper, we propose to integrate all-side local-learning based complementarity into an AUC-based approach, dubbed ANNC, to evaluate pairwise features by scrutinizing their comprehensive misclassification information in terms of both k-nearest misses and k-nearest hits. This strategy contributes to capture complementary features that collaborate with each other to achieve remarkable recognition performance. Extensive experiments on openly available benchmarks demonstrate the effectiveness of the new approach under various metrics.

Semi-Supervised Multi-Label Feature Selection by Preserving Feature-Label Space Consistency

Semi-supervised learning and multi-label learning pose different challenges for feature selection, which is one of the core techniques for dimension reduction, and the exploration of reducing feature space for multi-label learning with incomplete label information is far from satisfactory. Existing feature selection approaches devote attention to either of two issues, namely, alleviating negative effects of imperfectly predicted labels and quantitatively evaluating label correlations, exclusively for semi-supervised or multi-label scenarios. A unified framework to extract label correlation information with incomplete prior knowledge and embed this information in feature selection however, is rarely touched. In this paper, we propose a space consistency-based feature selection model to address this issue. Specifically, correlation information in feature space is learned based on the probabilistic neighborhood similarities, and correlation information in label space is optimized by preserving feature-label space consistency. This mechanism contributes to appropriately extracting label information in semi-supervised multi-label learning scenario and effectively employing this information to select discriminative features. An extensive experimental evaluation on real-world data shows the superiority of the proposed approach under various evaluation metrics.