Aleks Jakulin

Analyzing Attribute Dependencies

Many effective and efficient learning algorithms assume independence of attributes. They often perform well even in domains where this assumption is not really true. However, they may fail badly when the degree of attribute dependencies becomes critical. In this paper, we examine methods for detecting deviations from independence. These dependencies give rise to “interactions” between attributes which affect the performance of learning algorithms. We first formally define the degree of interaction between attributes through the deviation of the best possible “voting” classifier from the true relation between the class and the attributes in a domain. Then we propose a practical heuristic for detecting attribute interactions, called interaction gain. We experimentally investigate the suitability of interaction gain for handling attribute interactions in machine learning. We also propose visualization methods for graphical exploration of interactions in a domain.

Testing the significance of attribute interactions

Attribute interactions are the irreducible dependencies between attributes. Interactions underlie feature relevance and selection, the structure of joint probability and classification models: if and only if the attributes interact, they should be connected. While the issue of 2-way interactions, especially of those between an attribute and the label, has already been addressed, we introduce an operational definition of a generalized n-way interaction by highlighting two models: the reductionistic part-to-whole approximation, where the model of the whole is reconstructed from models of the parts, and the holistic reference model, where the whole is modelled directly. An interaction is deemed significant if these two models are significantly different. In this paper, we propose the Kirkwood superposition approximation for constructing part-to-whole approximations. To model data, we do not assume a particular structure of interactions, but instead construct the model by testing for the presence of interactions. The resulting map of significant interactions is a graphical model learned from the data. We confirm that the P-values computed with the assumption of the asymptotic X2 distribution closely match those obtained with the boot-strap.

Nomograms for visualizing support vector machines

We propose a simple yet potentially very effective way of visualizing trained support vector machines. Nomograms are an established model visualization technique that can graphically encode the complete model on a single page. The dimensionality of the visualization does not depend on the number of attributes, but merely on the properties of the kernel. To represent the effect of each predictive feature on the log odds ratio scale as required for the nomograms, we employ logistic regression to convert the distance from the separating hyperplane into a probability. Case studies on selected data sets show that for a technique thought to be a black-box, nomograms can clearly expose its internal structure. By providing an easy-to-interpret visualization the analysts can gain insight and study the effects of predictive factors.

Attribute Interactions in Medical Data Analysis

There is much empirical evidence about the success of naive Bayesian classification (NBC) in medical applications of attribute-based machine learning. NBC assumes conditional independence between attributes. In classification, such classifiers sum up the pieces of class-related evidence from individual attributes, independently of other attributes. The performance, however, deteriorates significantly when the “interactions” between attributes become critical. We propose an approach to handling attribute interactions within the framework of “voting” classifiers, such as NBC. We propose an operational test for detecting interactions in learning data and a procedure that takes the detected interactions into account while learning. This approach induces a structuring of the domain of attributes, it may lead to improved classifier’s performance and may provide useful novel information for the domain expert when interpreting the results of learning. We report on its application in data analysis and model construction for the prediction of clinical outcome in hip arthroplasty.

Acquiring an ontology from the text a legal case study

A topic ontology applies the usual ontological constructs to the task of annotating the topic of a document. The topic is the highly summarized essence of the document. The topics are usually chosen intuitively and rarely questioned. However, we have studied several ways of allocating frequently asked questions from a legal domain into a set of topical sub-domains. Our criteria were: 1) The sub-domains should not overlap. 2) The sub-domain should be objectively identifiable from the words of the text. 3) Which words and grammatical categories can serve as keywords? 4) Can the structure of sub-domains be induced semi-automatically from the text itself?

Analysis of Legal and Political Data

With the abundance of publicly available data registering the judgements at supreme courts and parliamentary votes, we can employ various data mining techniques to identify interesting patterns. For example, we can identify explicit and implicit voting blocs, which may or may not agree with official party affiliations. We can assess the political strength of those blocs. We can examine the vote of which particular senators is the most representative of the final outcome of the vote. We can employ text mining and visualization tools to cope with a large number of issues discussed in parliaments. While the paper primarily acts as a survey, it demonstrates the utility of several techniques that have not yet been used in the context of law.