Mineichi Kudo

Comparison of algorithms that select features for pattern classifiers

Abstract A comparative study of algorithms for large-scale feature selection (where the number of features is over 50) is carried out. In the study, the goodness of a feature subset is measured by leave-one-out correct-classification rate of a nearest-neighbor (1-NN) classifier and many practical problems are used. A unified way is given to compare algorithms having dissimilar objectives. Based on the results of many experiments, we give guidelines for the use of feature selection algorithms. Especially, it is shown that sequential floating search methods are suitable for small- and medium-scale problems and genetic algorithms are suitable for large-scale problems.

Structural, Syntactic, and Statistical Pattern Recognition

Peer-to-Peer (P2P) lending is an online platform to facilitate borrowing and investment transactions. A central problem for these P2P platforms is how to identify the most influential factors that are closely related to the credit risks. This problem is inherently complex due to the various forms of risks and the numerous influencing factors involved. Moreover, raw data of P2P lending are often high-dimension, highly correlated and unstable, making the problem more untractable by traditional statistical and machine learning approaches. To address these problems, we develop a novel filter-based feature selection method for P2P lending analysis. Unlike most traditional feature selection methods that use vectorial features, the proposed method is based on graphbased features and thus incorporates the relationships between pairwise feature samples into the feature selection process. Since the graph-based features are by nature completed weighted graphs, we use the steady state random walk to encapsulate the main characteristics of the graphbased features. Specifically, we compute a probability distribution of the walk visiting the vertices. Furthermore, we measure the discriminant power of each graph-based feature with respect to the target feature, through the Jensen-Shannon divergence measure between the probability distributions from the random walks. We select an optimal subset of features based on the most relevant graph-based features, through the Jensen-Shannon divergence measure. Unlike most existing state-of-theart feature selection methods, the proposed method can accommodate both continuous and discrete target features. Experiments demonstrate the effectiveness and usefulness of the proposed feature selection algorithm on the problem of P2P lending platforms in China.

Partitioning of Web graphs by community topology

We introduce a stricter Web community definition to overcome boundary ambiguity of a Web community defined by Flake, Lawrence and Giles [2], and consider the problem of finding communities that satisfy our definition. We discuss how to find such communities and hardness of this problem.We also propose Web page partitioning by equivalence relation defined using the class of communities of our definition. Though the problem of efficiently finding all communities of our definition is NP-complete, we propose an efficient method of finding a subclass of communities among the sets partitioned by each of n-1 cuts represented by a Gomory-Hu tree [10], and partitioning a Web graph by equivalence relation defined using the subclass.According to our preliminary experiments, partitioning by our method divided the pages retrieved by keyword search into several different categories to some extent.

Multidimensional curve classification using passing—through regions

Abstract A new method is proposed for classifying sets of a variable number of points and curves in a multidimensional space as time series. Almost all classifiers proposed so far assume that there is a constant number of features and they cannot treat a variable number of features. To cope with this difficulty, we examine a fixed number of questions like “how many points are in a certain range of a certain dimension”, and we convert the corresponding answers into a binary vector with a fixed length. These converted binary vectors are used as the basis for our classification. With respect to curve classification, many conventional methods are based on a frequency analysis such as Fourier analysis, a predictive analysis such as auto-regression, or a hidden Markov model. However, their resulting classification rules are difficult to interpret. In addition, they also rely on the global shape of curves and cannot treat cases in which only one part of a curve is important for classification. We propose some methods that are especially effective for such cases and the obtained rule is visualized.

Performance analysis of minimum /spl lscr//sub 1/-norm solutions for underdetermined source separation

Results of the analysis of the performance of minimum /spl lscr//sub 1/-norm solutions in underdetermined blind source separation, that is, separation of n sources from m(<n) linearly mixed observations, are presented in this paper. The minimum /spl lscr//sub 1/-norm solutions are known to be justified as maximum a posteriori probability (MAP) solutions under a Laplacian prior. Previous works have not given much attention to the performance of minimum /spl lscr//sub 1/-norm solutions, despite the need to know about its properties in order to investigate its practical effectiveness. We first derive a probability density of minimum /spl lscr//sub 1/-norm solutions and some properties. We then show that the minimum /spl lscr//sub 1/-norm solutions work best in a case in which the number of simultaneous nonzero source time samples is less than the number of sensors at each time point or in a case in which the source signals have a highly peaked distribution. We also show that when neither of these conditions is satisfied, the performance of minimum /spl lscr//sub 1/-norm solutions is almost the same as that of linear solutions obtained by the Moore-Penrose inverse. Our results show when the minimum /spl lscr//sub 1/-norm solutions are reliable.

Non-parametric classifier-independent feature selection

Feature selection is used for finding a feature subset that has the most discriminative information from the original feature set. In practice, since we do not know the classifier to be used after feature selection, it is desirable to find a feature subset that is universally effective for any classifier. Such a trial is called classifier-independent feature selection. In this study, we propose a novel classifier-independent feature selection method on the basis of the estimation of Bayes discrimination boundary. The experimental results on 12 real-world datasets showed the fundamental effectiveness of the proposed method.

Construction of class regions by a randomized algorithm: a randomized subclass method

A randomized algorithm is proposed for solving the problem of finding hyper-rectangles, sufficiently approximating the true region in each class. This method yields a suboptimal solution, but is more efficient than previous methods. The performance is analysed based on a criterion of PAC (Probably Approximately Correct) learning. Experimental results show that the proposed method can solve large problems which were not able to be solved previously.

Algorithms for adversarial bandit problems with multiple plays

Adversarial bandit problems studied by Auer et al. [4] are multi-armed bandit problems in which no stochastic assumption is made on the nature of the process generating the rewards for actions. In this paper, we extend their theories to the case where k(≥1) distinct actions are selected at each time step. As algorithms to solve our problem, we analyze an extension of Exp3 [4] and an application of a bandit online linear optimization algorithm [1] in addition to two existing algorithms (Exp3, ComBand [5]) in terms of time and space efficiency and the regret for the best fixed action set. The extension of Exp3, called Exp3.M, performs best with respect to all the measures: it runs in O(K(log k+1)) time and O(K) space, and suffers at most O(√kTK log(K/k)) regret, where K is the number of possible actions and T is the number of iterations. The upper bound of the regret we proved for Exp3.M is an extension of that proved by Auer et al. for Exp3.

Privacy-preserved behavior analysis and fall detection by an infrared ceiling sensor network

An infrared ceiling sensor network system is reported in this study to realize behavior analysis and fall detection of a single person in the home environment. The sensors output multiple binary sequences from which we know the existence/non-existence of persons under the sensors. The short duration averages of the binary responses are shown to be able to be regarded as pixel values of a top-view camera, but more advantageous in the sense of preserving privacy. Using the “pixel values” as features, support vector machine classifiers succeeded in recognizing eight activities (walking, reading, etc.) performed by five subjects at an average recognition rate of 80.65%. In addition, we proposed a martingale framework for detecting falls in this system. The experimental results showed that we attained the best performance of 95.14% (F1 value), the FAR of 7.5% and the FRR of 2.0%. This accuracy is not sufficient in general but surprisingly high with such low-level information. In summary, it is shown that this system has the potential to be used in the home environment to provide personalized services and to detect abnormalities of elders who live alone.

Feature selection based on the structural indices of categories

Abstract A new technique is proposed to select features out of all available ones on the basis of structural indices of categories. In terms of hyper-rectangles including as many training samples of a category as possible, two characteristic indices are calculated which summarize its underlying distribution of samples. The hyper-rectangles and the indices are available in evaluating the degree of importance of features, and are used to increase the discrimination rates of discrimination rules by removing redundant features. The running time of the algorithm is linear order in the number of features. Experiments on artificial and real data attests its effectiveness.