Luis Talavera

An evaluation of filter and wrapper methods for feature selection in categorical clustering

Feature selection for clustering is a problem rarely addressed in the literature. Although recently there has been some work on the area, there is a lack of extensive empirical evaluation to assess the potential of each method. In this paper, we propose a new implementation of a wrapper and adapt an existing filter method to perform experiments over several data sets and compare both approaches. Results confirm the utility of feature selection for clustering and the theoretical superiority of wrapper methods. However, it raises some problems that arise from using greedy search procedures and also suggest evidence that filters are a reasonably alternative with limited computational cost.

Generality-based conceptual clustering with probabilistic concepts

Statistical research in clustering has almost universally focused on data sets described by continuous features and its methods are difficult to apply to tasks involving symbolic features. In addition, these methods are seldom concerned with helping the user in interpreting the results obtained. Machine learning researchers have developed conceptual clustering methods aimed at solving these problems. Following a long term tradition in AI, early conceptual clustering implementations employed logic as the mechanism of concept representation. However, logical representations have been criticized for constraining the resulting cluster structures to be described by necessary and sufficient conditions. An alternative are probabilistic concepts which associate a probability or weight with each property of the concept definition. In this paper, we propose a symbolic hierarchical clustering model that makes use of probabilistic representations and extends the traditional ideas of specificity-generality typically found in machine learning. We propose a parameterized measure that allows users to specify both the number of levels and the degree of generality of each level. By providing some feedback to the user about the balance of the generality of the concepts created at each level and given the intuitive behavior of the user parameter, the system improves user interaction in the clustering process.

Integrating Declarative Knowledge in Hierarchical Clustering Tasks

The capability of making use of existing prior knowledge is an important challenge for Knowledge Discovery tasks. As an unsupervised learning task, clustering appears to be one of the tasks that more benefits might obtain from prior knowledge. In this paper, we propose a method for providing declarative prior knowledge to a hierarchical clustering system stressing the interactive component. Preliminary results suggest that declarative knowledge is a powerful bias in order to improve the quality of clustering in domains were the internal biases of the system are inappropriate or there is not enough evidence in data and that it can lead the system to build more comprehensible clusterings.

Dynamic Feature Selection in Incremental Hierarchical Clustering

Feature selection has received a lot of attention in the machine learning community, but mainly under the supervised paradigm. In this work we study the potential benefits of feature selection in hierarchical clustering tasks. Particularly we address this problem in the context of incremental clustering, following the basic ideas of Gennari [8]. By using a simple implementation, we show that a feature selection scheme running in parallel with the learning process can improve the clustering task under the dimensions of accuracy, efficiency in learning, efficiency in prediction and comprehensibility.

Robust Incremental Clustering with Bad Instance Orderings: A New Strategy

It is widely reported in the literature that incremental clustering systems suffer from instance ordering effects and that under some orderings, extremely poor clusterings may be obtained. In this paper we present a new general strategy aimed to mitigate these effects, the Not-Yet strategy which has a general and open formulation and it is not coupled to any particular system. Unlike other proposals, this strategy maintains the incremental nature of learning process. In addition, we propose a classification of strategies to avoid ordering effects which clarifies the benefits and disadvantages we can expect from the proposal made in the paper as well from existing ones. A particular implementation of the Not-Yet strategy is used to conduct several experiments. Results suggest that the strategy improves the clustering quality. We also show that, when combined with other local strategies, the Not-Yet strategy allows the clustering system to get high quality clusterings.

A buffering strategy to avoid ordering effects in clustering

It is widely reported in the literature that incremental clustering systems suffer from instance ordering effects and that under some orderings, extremely poor clusterings may be obtained. In this paper we present a new strategy aimed to mitigate these effects, the Not-Yet strategy which has a general and open formulation and it is not coupled to any particular system. Results suggest that the strategy improves the clustering quality and also that performance is limited by its limited foresight. We also show that, when combined with other strategies, the Not-Yet strategy may help the system to get high quality clusterings.

Feature Selection as Retrospective Pruning in Hierarchical Clustering

Although feature selection is a central problem in inductive learning as suggested by the growing amount of research in this area, most of the work has been carried out under the supervised learning paradigm, paying little attention to unsupervised learning tasks and, particularly, clustering tasks. In this paper, we analyze the particular benefits that feature selection may provide in hierarchical clustering. We propose a view of feature selection as a tree pruning process similar to those used in decision tree learning. Under this framework, we perform several experiments using different pruning strategies and considering a multiple prediction task. Results suggest that hierarchical clusterings can be greatly simplified without diminishing accuracy.

Efficient Construction of Comprehensible Hierarchical Clusterings

Clustering is an important data mining task which helps in finding useful patterns to summarize the data. In the KDD context, data mining is often used for description purposes rather than for prediction. However, it turns out difficult to find clustering systems that help to ease the interpretation task to the user in both, statistics and Machine Learning fields. In this paper we present Isaac, a hierarchical clustering system which employs traditional clustering ideas combined with a feature selection mechanism and heuristics in order to provide comprehensible results. At the same time, it allows to efficiently deal with large datasets by means of a preprocessing step. Results suggest that these aims are achieved and encourage further research.

Exploiting Inductive Bias Shift in Knowledge Acquisition from Ill-Structured Domains

Machine Learning (ML) methods are very powerful tools to automate the knowledge acquisition (KA) task. Particularly, in illstructured domains where there is no clear idea about which concepts exist, inductive unsupervised learning systems appear to be a promising approach to help experts in the early stages of the acquisition process. In this paper we examine the concept of inductive bias, which have received great attention from the ML community, and discuss the importance of bias shift when using ML algorithms to help experts in constructing a knowledge base (KB) A simple framework for the interaction of the expert with the inductive system exploiting bias shift is shown. Also, it is suggested that under some assumptions, bias selection in unsupervised learning may be performed via parameter setting, thus allowing the user to shift the system bias externally.