Bojan Cestnik

On estimating probabilities in tree pruning

In this paper we introduce a new method for decision tree pruning, based on the minimisation of the expected classification error method by Niblett and Bratko. The original Niblett-Bratko pruning algorithm uses Laplace probability estimates. Here we introduce a new, more general Bayesian approach to estimating probabilities which we call m-probability-estimation. By varying a parameter m in this method, tree pruning can be adjusted to particular properties of the learning domain, such as level of noise. The resulting pruning method improves on the original Niblett-Bratko pruning in the following respects: apriori probabilities can be incorporated into error estimation, several trees pruned to various degrees can be generated, and the degree of pruning is not affected by the number of classes. These improvements are supported by experimental findings. m-probability-estimation also enables the combination of learning data obtained from various sources.

Decision Support Through Subgroup Discovery: Three Case Studies and the Lessons Learned

This paper presents ways to use subgroup discovery to generate actionable knowledge for decision support. Actionable knowledge is explicit symbolic knowledge, typically presented in the form of rules, that allows the decision maker to recognize some important relations and to perform an appropriate action, such as targeting a direct marketing campaign, or planning a population screening campaign aimed at detecting individuals with high disease risk. Different subgroup discovery approaches are outlined, and their advantages over using standard classification rule learning are discussed. Three case studies, a medical and two marketing ones, are used to present the lessons learned in solving problems requiring actionable knowledge generation for decision support.

Literature mining method RaJoLink for uncovering relations between biomedical concepts

To support biomedical experts in their knowledge discovery process, we have developed a literature mining method called RaJoLink for identification of relations between biomedical concepts in disconnected sets of articles. The method implements Swansons ABC model approach for generating hypotheses in a new way. The main novelty is a semi-automated suggestion of candidates for agents a that might be logically connected with a given phenomenon c under investigation. The choice of candidates for a is based on rare terms identified in the literature on c. As rare terms are not part of the typical range of information, which describe the phenomenon under investigation, such information might be considered as unusual observations about the phenomenon c. If literatures on these rare terms have an interesting term in common, this joint term is declared as a candidate for a. Linking terms b between literature on a and literature on c are then searched for in the closed discovery to provide additional supportive evidence for uncovered connections. We have applied the method to the literature on autism and have used MEDLINE as a source of data. Expert evaluation has confirmed that the discovered relations might contribute to a better understanding of autism.

Data mining and visualization for decision support and modeling of public health-care resources

This paper proposes an innovative use of data mining and visualization techniques for decision support in planning and regional-level management of Slovenian public health-care. Data mining and statistical techniques were used to analyze databases collected by a regional Public Heath Institute. We also studied organizational aspects of public health resources in the selected Celje region with the objective to identify the areas that are atypical in terms of availability and accessibility of public health services for the population. The most important step was the detection of outliers and the analysis of availability and accessibility deviations. The results are applicable to health-care planning and support in decision making by local and regional health-care authorities. In addition to the practical results, which are directly useful for decision making in planning of the regional health-care system, the main methodological contribution of the paper are the developed visualization methods that can be used to facilitate knowledge management and decision making processes.

Literature Mining: Towards Better Understanding of Autism

In this article we present a literature mining method RaJoLink that upgrades Swansons ABC model approach to uncovering hidden relations from a set of articles in a given domain. When these relations are interesting from medical point of view and can be verified by medical experts, they represent new pieces of knowledge and can contribute to better understanding of diseases. In our study we analyzed biomedical literature about autism, which is a very complex and not yet sufficiently understood domain. On the basis of word frequency statistics several rare terms were identified with the aim of generating potentially new explanations for the impairments that are observed in the affected population. Calcineurin was discovered as a joint term in the intersection of their corresponding literature. Similarly, NF-kappaB was recognized as a joint term. Pairs of documents that point to potential relations between the identified joint terms and autism were also automatically detected. Expert evaluation confirmed the relevance of these relations.

Outlier Detection in Cross-Context Link Discovery for Creative Literature Mining

This paper investigates the role of outliers in literature-based knowledge discovery. It shows that detecting interesting outliers which appear in the literature on a given phenomenon can help the expert to find implicit relationships among concepts of different domains. The underlying assumption is that while the majority of articles in the given scientific domain describe matters related to a common understanding of the domain, the exploration of outliers may lead to the detection of scientifically interesting bridging concepts among disjoint sets of scientific articles. The proposed approach contributes to cross-context link discovery by proving the utility of outlier detection for finding bisociative links in the process of autism literature exploration, as well as by uncovering implicit relationships in the articles from the migraine domain.

RaJoLink: A Method for Finding Seeds of Future Discoveries in Nowadays Literature

In this article we present a study which demonstrates the ability of the method RaJoLink to uncover candidate hypotheses for future discoveries from rare terms in existing literature. The method is inspired by Swansons ABC model approach to finding hidden relations from a set of articles in a given domain. The main novelty is in a semi-automated way of suggesting which relations might have more potential for new discoveries and are therefore good candidates for further investigations. In our previous articles we reported on a successful application of the method RaJoLink in the autism domain. To support the evaluation of the method with a well-known example from the literature, we applied it to the migraine domain, aiming at reproducing Swansons finding of magnesium deficiency as a possible cause of migraine. Only literature which was available at the time of the Swansons experiment was used in our test. As described in this study, in addition to actually uncovering magnesium as a candidate for formulating the hypothesis, RaJoLink pointed also to interferon, interleukin and tumor necrosis factor as candidates for potential discoveries connecting them with migraine. These connections were not published in the titles contemporary to the ones used in the experiment, but have been recently reported in several scientific articles. This confirms the ability of the RaJoLink method to uncover seeds of future discoveries in existing literature by using rare terms as a beacon.

Exploring the power of outliers for cross-domain literature mining

In bisociative cross-domain literature mining the goal is to identify interesting terms or concepts which relate different domains. This chapter reveals that a majority of these domain bridging concepts can be found in outlier documents which are not in the mainstream domain literature. We have detected outlier documents by combining three classification-based outlier detection methods and explored the power of these outlier documents in terms of their potential for supporting the bridging concept discovery process. The experimental evaluation was performed on the classical migraine-magnesium and the recently explored autism-calcineurin domain pairs.

Bridging concept identification for constructing information networks from text documents

A major challenge for next generation data mining systems is creative knowledge discovery from diverse and distributed data sources. In this task an important challenge is information fusion of diverse mainly unstructured representations into a unique knowledge format. This chapter focuses on merging information available in text documents into an information network --- a graph representation of knowledge. The problem addressed is how to efficiently and effectively produce an information network from large text corpora from at least two diverse, seemingly unrelated, domains. The goal is to produce a network that has the highest potential for providing yet unexplored cross domain links which could lead to new scientific discoveries. The focus of this work is better identification of important domain bridging concepts that are promoted as core nodes around which the rest of the network is formed. The evaluation is performed by repeating a discovery made on medical articles in the migraine magnesium domain.

Attribute-based learning

1 Main approaches The following main approaches to attribute-based learning are considered in this paper: • Rule induction • Decision-tree induction • Regression-tree induction • Naive Bayes classifiers We have included Naive Bayes Classifiers, that is Bayes classifiers which assume that the attributes are independent, although they are not typically considered as an AI approach to Machine Learning. They are however of fundamental importance in principle, and provide a straightforward standard reference for accuracy comparisons with other more complicated approaches. In fact, despite its simplicity, naive Bayes usually outperforms other AI approaches. It is sometimes considered as inferior with respect to the classi-fiers comprehensibility, but that too is often not the case. On the other hand, there are some related approaches that we have omitted from this discussion: Instance-Based Learning and Feedforward Neural Nets. These are usually treated separately. The following can be considered as reasonable representative list of sys-tems/algorithms of the approaches considered, although this list is by no means exhaustive: