Luc De Raedt

Inductive Logic Programming: Theory and methods

Abstract Inductive Logic Programming (ILP) is a new discipline which investigates the inductive construction of first-order clausal theories from examples and background knowledge. We survey the most important theories and methods of this new field. First, various problem specifications of ILP are formalized in semantic settings for ILP, yielding a “model-theory” for ILP. Second, a generic ILP algorithm is presented. Third, the inference rules and corresponding operators used in ILP are presented, resulting in a “proof-theory” for ILP. Fourth, since inductive inference does not produce statements which are assured to follow from what is given, inductive inferences require an alternative form of justification. This can take the form of either probabilistic support or logical constraints on the hypothesis language. Information compression techniques used within ILP are presented within a unifying Bayesian approach to confirmation and corroboration of hypotheses. Also, different ways to constrain the hypothesis language or specify the declarative bias are presented. Fifth, some advanced topics in ILP are addressed. These include aspects of computational learning theory as applied to ILP, and the issue of predicate invention. Finally, we survey some applications and implementations of ILP. ILP applications fall under two different categories: first, scientific discovery and knowledge acquisition, and second, programming assistants.

Top-down induction of first-order logical decision trees

Although topddown induction of decision trees is a very popular induction method, up till now it has mainly been used for propositional learnings relational decision tree learners are scarce. This dissertation discusses the application domain of decision tree learning and extends it towards the first order logic context of Inductive Logic Programming.

Clausal Discovery

The clausal discovery engine claudien is presented. CLAUDIEN is an inductive logic programming engine that fits in the descriptive data mining paradigm. CLAUDIEN addresses characteristic induction from interpretations, a task which is related to existing formalisations of induction in logic. In characteristic induction from interpretations, the regularities are represented by clausal theories, and the data using Herbrand interpretations. Because CLAUDIEN uses clausal logic to represent hypotheses, the regularities induced typically involve multiple relations or predicates. CLAUDIEN also employs a novel declarative bias mechanism to define the set of clauses that may appear in a hypothesis.

Molecular feature mining in HIV data

We present the application of Feature Mining techniques to the Developmental Therapeutics Programs AIDS antiviral screen database. The database consists of 43576 compounds, which were measured for their capability to protect human cells from HIV-1 infection. According to these measurements, the compounds were classified as either active, moderately active or inactive. The distribution of classes is extremely skewed: Only 1.3 % of the molecules is known to be active, and 2.7 % is known to be moderately active.Given this database, we were interested in molecular substructures (i.e., features) that are frequent in the active molecules, and infrequent in the inactives. In data mining terms, we focused on features with a minimum support in active compounds and a maximum support in inactive compounds. We analyzed the database using the levelwise version space algorithm that forms the basis of the inductive query and database system MOLFEA (Molecular Feature Miner). Within this framework, it is possible to declaratively specify the features of interest, such as the frequency of features on (possibly different) datasets as well as on the generality and syntax of them. Assuming that the detected substructures are causally related to biochemical mechanisms, it should be possible to facilitate the development of new pharmaceuticals with improved activities.

Mining Association Rules in Multiple Relations

The application of algorithms for efficiently generating association rules is so far restricted to cases where information is put together in a single relation. We describe how this restriction can be overcome through the combination of the available algorithms with standard techniques from the field of inductive logic programming. We present the system Warmr, which extends Apriori [2] to mine association rules in multiple relations. We apply Warmr to the natural language processing task of mining part-of-speech tagging rules in a large corpus of English. be applied to further constrain the space of interesting ARMRs.

Machine Learning: ECML 2001

This paper presents a missing link between Plotkin’s least general generalization formalism and generalization on the Order Sorted Feature (OSF) foundation. A feature term (or ψ-term) is an extended logic term based on ordered sorts and is a normal form of an OSF-term. An axiomatic definition of ψ-term generalization is given as a set of OSF clause generalization rules and the least generality of the axiomatic definition is proven in the sense of Plotkin’s least general generalization (lgg). The correctness of the definition is given on the basis of the axiomatic foundation. An operational definition of the least general generalization of clauses based on ψ-terms is also shown as a realization of the axiomatic definition.

Probabilistic logic learning

The past few years have witnessed an significant interest in probabilistic logic learning, i.e. in research lying at the intersection of probabilistic reasoning, logical representations, and machine learning. A rich variety of different formalisms and learning techniques have been developed. This paper provides an introductory survey and overview of the state-of-the-art in probabilistic logic learning through the identification of a number of important probabilistic, logical and learning concepts.

Logical settings for concept-learning

Abstract Three different formalizations of concept-learning in logic (as well as some variants) are analyzed and related. It is shown that learning from interpretations reduces to learning from entailment, which in turn reduces to learning from satisfiability. The implications of this result for inductive logic programming and computational learning theory are then discussed, and guidelines for choosing a problem-setting are formulated.

Towards Combining Inductive Logic Programming with Bayesian Networks

Recently, new representation languages that integrate first order logic with Bayesian networks have been developed. Bayesian logic programs are one of these languages. In this paper, we present results on combining Inductive Logic Programming (ILP) with Bayesian networks to learn both the qualitative and the quantitative components of Bayesian logic programs. More precisely, we show how to combine the ILP setting learning from interpretations with score-based techniques for learning Bayesian networks. Thus, the paper positively answers Koller and Pfeffers question, whether techniques from ILP could help to learn the logical component of first order probabilistic models.

A perspective on inductive databases

Inductive databases tightly integrate databases with data mining. The key ideas are that data and patterns (or models) are handled in the same way and that an inductive query language allows the user to query and manipulate the patterns (or models) of interest.This paper proposes a simple and abstract model for inductive databases. We describe the basic formalism, a simple but fairly powerful inductive query language, some basics of reasoning for query optimization, and discuss some memory organization and implementation issues.