Patrick P. K. Chan

Content-based image retrieval using color moment and Gabor texture feature

Aim to currently content-based image retrieval method having high computational complexity and low retrieval accuracy problem, this paper proposes a content-based image retrieval method based on color and texture features. As its color features, color moments of the Hue, Saturation and Value (HSV) component images in HSV color space are used. As its texture features, Gabor texture descriptors are adopted. Users assign the weights to each feature respectively and calculate the similarity with combined features of color and texture according to normalized Euclidean distance. Experiment results show that the proposed method has higher retrieval accuracy than conventional methods using color and texture features even though its feature vector dimension results in a lower rate than the conventional method.

Adversarial Feature Selection Against Evasion Attacks

Pattern recognition and machine learning techniques have been increasingly adopted in adversarial settings such as spam, intrusion, and malware detection, although their security against well-crafted attacks that aim to evade detection by manipulating data at test time has not yet been thoroughly assessed. While previous work has been mainly focused on devising adversary-aware classification algorithms to counter evasion attempts, only few authors have considered the impact of using reduced feature sets on classifier security against the same attacks. An interesting, preliminary result is that classifier security to evasion may be even worsened by the application of feature selection. In this paper, we provide a more detailed investigation of this aspect, shedding some light on the security properties of feature selection against evasion attacks. Inspired by previous work on adversary-aware classifiers, we propose a novel adversary-aware feature selection model that can improve classifier security against evasion attacks, by incorporating specific assumptions on the adversarys data manipulation strategy. We focus on an efficient, wrapper-based implementation of our approach, and experimentally validate its soundness on different application examples, including spam and malware detection.

Radial Basis Function network learning using localized generalization error bound

Training a classifier with good generalization capability is a major issue for pattern classification problems. A novel training objective function for Radial Basis Function (RBF) network using a localized generalization error model (L-GEM) is proposed in this paper. The localized generalization error model provides a generalization error bound for unseen samples located within a neighborhood that contains all training samples. The assumption of the same width for all dimensions of a hidden neuron in L-GEM is relaxed in this work. The parameters of RBF network are selected via minimization of the proposed objective function to minimize its localized generalization error bound. The characteristics of the proposed objective function are compared with those for regularization methods. For weight selection, RBF networks trained by minimizing the proposed objective function consistently outperform RBF networks trained by minimizing the training error, Tikhonov Regularization, Weight Decay or Locality Regularization. The proposed objective function is also applied to select center, width and weight in RBF network simultaneously. RBF networks trained by minimizing the proposed objective function yield better testing accuracies when compared to those that minimizes training error only.

Machine Learning and Cybernetics

Combining multiple classifiers to achieve better performance than any single classifier is one of the most important research areas in machine learning. In this paper, we focus on combining different classifiers to form an effective ensemble system. By introducing a novel framework operated on outputs of different classifiers, our aim is to build a powerful model which is competitive to other well-known combining algorithms such as Decision Template, Multiple Response Linear Regression (MLR), SCANN and fixed combining rules. Our approach is difference from the traditional approaches in that we use Gaussian Mixture Model (GMM) to model distribution of Level1 data and to predict the label of an observation based on maximizing the posterior probability realized through Bayes model. We also apply Principle Component Analysis (PCA) to output of base classifiers to reduce its dimension of what before GMM modeling. Experiments were evaluated on 21 datasets coming from University of California Irvine (UCI) Machine Learning Repository to demonstrate the benefits of our framework compared with several benchmark algorithms.

Fuzzy support vector machines for solving two-class problems

A support vector machine (SVM) was originally developed to solve two-class non-fuzzy problems. An SVM can act as a linear learning machine when handling data in a high dimensional feature space for non-linear separable and non-separable problems. A few methods have been proposed to solve two-class and multi-class classification problems by including fuzzy concepts. In this paper, we propose a new fuzzy support vector machine which improves the traditional SVM by adding fuzzy memberships to each training sample to indicate degree of membership of this sample to different classes. This fuzzy SVM is more complete and meaningful, and could generalize the traditional non-fuzzy SVM to a fuzzy one, i.e., the traditional non-fuzzy SVM is an extreme case of our fuzzy SVM when the degrees of membership of a sample to two different classes are the same.

Static detection of Android malware by using permissions and API calls

Android smart phones have become more and more popular due to its increasing functionalities, compatibility and convenience. More and more Android applications have been developed and can be downloaded easily from app markets. However, Android malwares have increased significantly in recent years. In this paper, we proposed a feature set containing the permissions and API calls for Android malware static detection. Classifiers that used the proposed feature set outperform those only with the permissions experimentally. It showed that the information of API calls is helpful in recognizing Android malware.

Spam filtering for short messages in adversarial environment

The unsolicited bulk messages are widespread in the applications of short messages. Although the existing spam filters have satisfying performance, they are facing the challenge of an adversary who misleads the spam filters by manipulating samples. Until now, the vulnerability of spam filtering technique for short messages has not been investigated. Different from the other spam applications, a short message only has a few words and its length usually has an upper limit. The current adversarial learning algorithms may not work efficiently in short message spam filtering. In this paper, we investigate the existing good word attack and its counterattack method, i.e. the feature reweighting, in short message spam filtering in an effort to understand whether, and to what extent, they can work efficiently when the length of a message is limited. This paper proposes a good word attack strategy which maximizes the influence to a classifier with the least number of inserted characters based on the weight values and also the length of words. On the other hand, we also proposes the feature reweighting method with a new rescaling function which minimizes the importance of the feature representing a short word in order to require more inserted characters for a successful evasion. The methods are evaluated experimentally by using the SMS and the comment spam dataset. The results confirm that the length of words is a critical factor of the robustness of short message spam filtering to good word attack.

Dynamic fusion method using Localized Generalization Error Model

Multiple Classifier Systems (MCSs), which combine the outputs of a set of base classifiers, were proposed as a method to develop a more accurate classification system. One fundamental issue is how to combine the base classifiers. In this paper, a new dynamic fusion method named Localized Generalization Error Model Fusion Method (LFM) for MCSs is proposed. The Localized Generalization Error Model (L-GEM) has been used to estimate the local competence of base classifiers in MCSs. L-GEM provides a generalization error bound for unseen samples located within neighborhoods of testing samples. Base classifiers with lower generalization error bounds are assigned higher weights. In contrast to the current dynamic fusion methods, LFM estimates the local competence of base classifiers not only using the information of training error but also the sensitivity of classifier outputs. The additional effect of the sensitivity on the performance of model and the time complexity of the LFM are discussed and analyzed. Experimental results show that the MCSs using the LFM as a combination method outperform those using the other 21 dynamic fusion methods in terms of testing accuracy and time.

Asymmetric Cyclical Hashing for Large Scale Image Retrieval

This paper addresses a problem in the hashing technique for large scale image retrieval: learn a compact hash code to reduce the storage cost with performance comparable to that of the long hash code. A longer hash code yields a better precision rate of retrieved images. However, it also requires a larger storage, which limits the number of stored images. Current hashing methods employ the same code length for both queries and stored images. We propose a new hashing scheme using two hash codes with different lengths for queries and stored images, i.e., the asymmetric cyclical hashing. A compact hash code is used to reduce the storage requirement, while a long hash code is used for the query image. The image retrieval is performed by computing the Hamming distance of the long hash code of the query and the cyclically concatenated compact hash code of the stored image to yield a high precision and recall rate. Experiments on benchmarking databases consisting up to one million images show the effectiveness of the proposed method.

Active learning using localized generalization error of candidate sample as criterion

In classification problem, the learning process can be more efficient if the informative samples can be selected actively based on the knowledge of the classifier. This problem is called active learning. Most of the existing active learning methods did not directly relate to the generalization error of classifiers. Also, some of them need high computational time or are based on strict assumptions. This paper describes a new active learning strategy using the concept of localized generalization error of the candidate samples. The sample which yields the largest generalization error will be chosen for query. This method can be applied to different kinds of classifiers and its complexity is low. Experimental results demonstrate that the prediction accuracy of the classifier can be improved by using this selecting method and fewer training samples are possible for the same prediction accuracy.