Marenglen Biba

Discriminative Structure Learning of Markov Logic Networks

Markov Logic Networks (MLNs) combine Markov networks and first-order logic by attaching weights to first-order formulas and viewing these as templates for features of Markov networks. Learning the structure of MLNs is performed by state-of-the-art methods by maximizing the likelihood of a relational database. This can lead to suboptimal results given prediction tasks. On the other hand better results in prediction problems have been achieved by discriminative learning of MLNs weights given a certain structure. In this paper we propose an algorithm for learning the structure of MLNs discriminatively by maximimizing the conditional likelihood of the query predicates instead of the joint likelihood of all predicates. The algorithm chooses the structures by maximizing conditional likelihood and sets the parameters by maximum likelihood. Experiments in two real-world domains show that the proposed algorithm improves over the state-of-the-art discriminative weight learning algorithm for MLNs in terms of conditional likelihood. We also compare the proposed algorithm with the state-of-the-art generative structure learning algorithm for MLNs and confirm the results in [22] showing that for small datasets the generative algorithm is competitive, while for larger datasets the discriminative algorithm outperfoms the generative one.

A General Similarity Framework for Horn Clause Logic

First-Order Logic formulae are a powerful representation formalism characterized by the use of relations, that cause serious computational problems due to the phenomenon of indeterminacy (various portions of one description are possibly mapped in different ways onto another description). Being able to identify the correct corresponding parts of two descriptions would help to tackle the problem: hence, the need for a framework for the comparison and similarity assessment. This could have many applications in Artificial Intelligence: guiding subsumption procedures and theory revision systems, implementing flexible matching, supporting instance-based learning and conceptual clustering. Unfortunately, few works on this subject are available in the literature. This paper focuses on Horn clauses, which are the basis for the Logic Programming paradigm, and proposes a novel similarity formula and evaluation criteria for identifying the descriptions components that are more similar and hence more likely to correspond to each other, based only on their syntactic structure. Experiments on real-world datasets prove the effectiveness of the proposal, and the efficiency of the corresponding implementation in the above tasks.

Automatic topics identification for reviewer assignment

Scientific conference management involves many complex and multi-faceted activities, which would make highly desirable for the organizing people to have a Web-based management system that makes some of them a little easier to carry out. One of such activities is the assignment of submitted papers to suitable reviewers, involving the authors, the reviewers and the conference chair. Authors that submit the papers usually must fill a form with paper title, abstract and a set of conference topics that fit their submission subject. Reviewers are required to register and declare their expertise on the conference topics (among other things). Finally, the conference chair has to carry out the review assignment taking into account the information provided by both the authors (about their paper) and the reviewers (about their competencies). Thus, all this subtasks needed for the assignment are currently carried out manually by the actors. While this can be just boring in the case of authors and reviewers, in case of conference chair the task is also very complex and time-consuming. n nIn this paper we propose the exploitation of intelligent techniques to automatically extract paper topics from their title and abstract, and the expertise of the reviewers from the titles of their publications available on the Internet. Successively, such a knowledge is exploited by an expert system able to automatically perform the assignments. The proposed methods were evaluated on a real conference dataset obtaining good results when compared to handmade ones, both in terms of quality and user-satisfaction of the assignments, and for reduction in execution time with respect to the case of humans performing the same process.

Similarity-Guided Clause Generalization

Few works are available in the literature to define similarity criteria between First-Order Logic formulae, where the presence of relations causes various portions of one description to be possibly mapped in different ways onto another description, which poses serious computational problems. Hence, the need for a set of general criteria that are able to support the comparison between formulae. This could have many applications; this paper tackles the case of two descriptions (e.g., a definition and an observation) to be generalized, where the similarity criteria could help in focussing on the subparts of the descriptions that are more similar and hence more likely to correspond to each other, based only on their syntactic structure. Experiments on real-world datasets prove the effectiveness of the proposal, and the efficiency of the corresponding implementation in a generalization procedure.

k-Nearest Neighbor Classification on First-Order Logic Descriptions

Classical attribute-value descriptions induce a multi-dimensional geometric space. One way for computing the distance between descriptions in such a space consists in evaluating an Euclidean distance between tuples of coordinates. This is the ground on which a large part of the Machine Learning literature has built its methods and techniques. However, the complexity of some domains require the use of First-Order Logic as a representation language. Unfortunately, when First-Order Logic is considered, descriptions can have different length and multiple instance of predicates, and the problem of indeterminacy arises. This makes computation of the distance between descriptions much less straight forward, and hence prevents the use of traditional distance-based techniques. This paper proposes the exploitation of a novel framework for computing the similarity between relational descriptions in a classical instance-based learning technique, k-Nearest Neighbor classification. Experimental results on real-world datasets show good performance, comparable to that of state-of-the-art conceptual learning systems, which supports the viability of the proposal.

Boosting learning and inference in Markov logic through metaheuristics

Markov Logic (ML) combines Markov networks (MNs) and first-order logic by attaching weights to first-order formulas and using these as templates for features of MNs. State-of-the-art structure learning algorithms in ML maximize the likelihood of a database by performing a greedy search in the space of structures. This can lead to suboptimal results because of the incapability of these approaches to escape local optima. Moreover, due to the combinatorially explosive space of potential candidates these methods are computationally prohibitive. We propose a novel algorithm for structure learning in ML, based on the Iterated Local Search (ILS) metaheuristic that explores the space of structures through a biased sampling of the set of local optima. We show through real-world experiments that the algorithm improves accuracy and learning time over the state-of-the-art algorithms. On the other side MAP and conditional inference for ML are hard computational tasks. This paper presents two algorithms for these tasks based on the Iterated Robust Tabu Search (IRoTS) metaheuristic. The first algorithm performs MAP inference and we show through extensive experiments that it improves over the state-of-the-art algorithm in terms of solution quality and inference time. The second algorithm combines IRoTS steps with simulated annealing steps for conditional inference and we show through experiments that it is faster than the current state-of-the-art algorithm maintaining the same inference quality.

Generalization-based similarity for conceptual clustering

Knowledge extraction represents an important issue that concerns the ability to identify valid, potentially useful and understandable patterns from large data collections. Such a task becomes more difficult if the domain of application cannot be represented by means of an attribute-value representation. Thus, a more powerful representation language, such as First-Order Logic, is necessary. Due to the complexity of handling First-Order Logic formulae, where the presence of relations causes various portions of one description to be possibly mapped in different ways onto another description, few works presenting techniques for comparing descriptions are available in the literature for this kind of representations. Nevertheless, the ability to assess similarity between first-order descriptions has many applications, ranging from description selection to flexible matching, from instance-based learning to clustering. n nThis paper tackles the case of Conceptual Clustering, where a new approach to similarity evaluation, based on both syntactic and semantic features, is exploited to support the task of grouping together similar items according to their relational description. After presenting a framework for Horn Clauses (including criteria, a function and composition techniques for similarity assessment), classical clustering algorithms are exploited to carry out the grouping task. Experimental results on real-world datasets prove the effectiveness of the proposal.

Multi-class Protein Fold Recognition Through a Symbolic-Statistical Framework

Protein fold recognition is an important problem in molecular biology. Machine learning symbolic approaches have been applied to automatically discover local structural signatures and relate these to the concept of fold in SCOP. However, most of these methods cannot handle uncertainty being therefore not able to solve multiple prediction problems. In this paper we present an application of the symbolic-statistical framework PRISM to a multi-class protein fold recognition problem. We compare the proposed approach to a symbolic-only technique and show that the hybrid framework outperforms the symbolic-only one in terms of predictive accuracy in the multiple prediction problem.

A hybrid symbolic-statistical approach to modeling metabolic networks

Biological systems consist of many components and interactions between them. In Systems Biology the principal problem is modeling complex biological systems and reconstructing interactions between their building blocks. Symbolic machine learning approaches have the power to model structured domains and relations among objects. However biological domains require uncertainty handling due to their hidden complex nature. Statistical machine learning approaches have the potential to model uncertainty in a robust manner. In this paper we apply a hybrid symbolic-statistical framework to modeling metabolic pathways and show through experiments that complex phenomenon such as biochemical reactions in cells metabolic networks can be modeled and simulated in the proposed framework.

A Novel Structure Refining Algorithm for Statistical-Logical Models

Statistical Relational Learning (SRL) is a growing field in Machine Learning that aims at the integration of logic-based learning approaches with probabilistic graphical models. Markov Logic Networks (MLNs) are one of the state-of-the-art SRL models that combine first-order logic and Markov networks (MNs) by attaching weights to first-order formulas and viewing these as templates for features of MNs. Learning models in SRL consists in learning the structure (logical clauses in MLNs) and the parameters (weights for each clause in MLNs). Structure learning of MLNs is performed by maximizing a likelihood function over relational databases and MLNs have been successfully applied to problems in relational and uncertain domains. Theory revision is the process of refining an existing theory by generalizing or specializing it depending on the nature of the new evidence. If the positive evidence is not explained then the theory must be generalized, whereas if the negative evidence is explained the theory must be specialized in order to exclude the negative example. Current SRL systems do not revise an existing model but learn structure and parameters from scratch. In this paper we propose a novel refining algorithm for theory revision under the statistical logical framework of MLNs. The novelty of the proposed approach consists in a tight integration of structure and parameter learning of an SRL model in a single step inside which a specialization or generalization step is performed for theory refinement.