Milen Kouylekov

Towards Extensible Textual Entailment Engines: The EDITS Package

This paper presents the first release of EDITS, an open-source software package for recognizing Textual Entailment developed by FBK-irst. The main contributions of EDITS consist in: i) providing a basic framework for a distance-based approach to the task, ii) providing a highly customizable environment to experiment with different algorithms, iii) allowing for easy extensions and integrations with new algorithms and resources. Systems main features are described, together with experiments over different datasets showing its potential in terms of tuning and adaptation capabilities.

Exploiting linguistic indices and syntactic structures for multilingual question answering: ITC-irst at CLEF 2005

We participated at four Question Answering tasks at CLEF 2005: the Italian monolingual (I), Italian-English (I/E), Bulgarian monolingual (B), and Bulgarian-English (B/E) bilingual task. While we did not change the approach in the Italian task (I), we experimented with several new approaches based on linguistic structures and statistics in the B, I/E, and B/E tasks.

The DIOGENE question answering system at CLEF-2004

This paper presents the ITC-irst Multilingual Question Answering system Diogene. The system was used successfully on the CLEF-2003, TREC-2003, TREC-2002 and TREC-2001 QA tracks. Diogene relies on a classical three-layer architecture: question processing, document retrieval, answer extraction and validation. Diogene uses MultiWordNet [8] (http:// multiwordnet.itc.it) which facilitates the transfer of knowledge between languages. For answer validation we used the Web. This year we also used a set of linguistic templates for answering specific questions like definition questions, location questions, and a subset of who-is and what-is questions. Diogene participated in both the monolingual Italian-Italian task and in the cross-language Italian-English task. We also collaborated with the Bulgarian Academy of Sciences in the cross-language Bulgarian-English QA task.

Towards entailment-based question answering: ITC-irst at CLEF 2006

This year, besides providing support to other groups participating in cross-language Question Answering (QA) tasks, and submitting runs both for the monolingual Italian and for the cross-language Italian/English tasks, the ITC-irst participation in the CLEF campaign concentrated on the Answer Validation Exercise (AVE). The participation in the AVE task, with an answer validation module based on textual entailment recognition, is motivated by our objectives of (i) creating a modular framework for an entailment-based approach to QA, and (ii) developing, in compliance with this framework, a stand-alone component for answer validation which implements different approaches to the problem.

Development of corpora within the CLaRK system: the BulTreeBank project experience

CLaRK is an XML-based software system for corpora development. It incorporates several technologies: XML technology; Unicode; Regular Cascaded Grammars; Constraints over XML Documents. The basic components of the system are: a tagger, a concordancer, an extractor, a grammar processor, a constraint engine.

Semantic Parsing for Textual Entailment

In this paper we gauge the utility of general-purpose, open-domain semantic parsing for textual entailment recognition by combining graph-structured meaning representations with semantic technologies and formal reasoning tools. Our approach achieves high precision, and in two case studies we show that when reasoning over n-best analyses from the parser the performance of our system reaches stateof-the-art for rule-based textual entailment systems. 1 Background and Motivation There is a growing interest in recent years in general-purpose semantic parsing into graphbased meaning representations, which provide greater expressive power than tree-based structures. Recent efforts in this spirit include, for example, Abstract Meaning Representation (Banarescu et al., 2013), and Semantic Dependency Parsing (SDP) (Oepen et al., 2014; Oepen et al., 2015). Simultaneously, in the Semantic Web community, a range of generic semantic technologies for storing and processing graph-structured data has been made available, but these have not been much used for natural language processing tasks. We propose a flexible, generic framework for precision-oriented Textual Entailment (TE) recognition that combines semantic parsing, graph-based representations of sentence meaning, and semantic technologies. During the decade since the TE task was defined, (logical) inference-based approaches have made some important contributions to the field. Systems such as Bos and Markert (2006) and Tatu and Moldovan (2006) employ automated proof search over logical representations of the input sentences. Other systems, such as Bar-Haim et al. (2007), apply transformational rules to linguistic representations of the sentence pairs, and determine entailment through graph subsumption. Because inference-based systems are vulnerable to incomplete knowledge in the rule set and errors in the mapping from natural language sentences to logical forms or linguistics representations, and because the definition of the TE task encourages a more relaxed, non-logical notion of entailment, the majority of TE systems have used more robust approaches, however. Our work supports a notion of logical inference for TE by reasoning with formal rules over graph-structured meaning representations, while achieving results that are comparable with robust approaches. We use a freely available, grammar-driven semantic parser and a well-defined reduction of underspecified logical-form meaning representations into variable-free semantic graphs called Elementary Dependency Structures (EDS) (Oepen and Lonning, 2006). We capitalize on a pre-existing storage and search infrastructure for EDSs using generic semantic technologies. For entailment classification, we create inference rules that enrich the EDS graphs, apply the rules with a generic reasoner, and use graph alignment as a decision tool. To test our generic setup, we perform two case studies where we replicate well-performing TE systems, one from the Parser Evaluation using Textual Entailments (PETE) task (Yuret et al., 2010), and one from SemEval 2014 Task 1 (Marelli et al., 2014). The best published results for the PETE task, Lien (2014), were obtained through heuristic rules that align meaning representations based on structural similarity. Lien and Kouylekov (2014) extend the same basic approach for SemEval 2014 by including lexical relations and negation handling. We recast the handwritten heuristic rules from these systems as formal Semantic Web Rule Language (SWRL) rules, and run them with a generic reasoning tool over EDS

Who are we talking about? tracking the referent in a question answering series

The capability of handling anaphora is becoming a key feature for Question Answering systems, as it can play a crucial role at different stages of the QA loop. At the question processing stage, on which this paper is focused, it enhances the treatment of follow-up questions, allowing for a more natural interaction with the user. As much as the QA task evolves towards a realistic dialogue-based scenario, one of the concrete problems raised by follow-up questions is tracking their actual referent. Each question may in fact refer to the topic of the session, to an answer given to an earlier question, or to a new entity it introduces in the dialogue. Focusing on the referent traceability problem, we present and experiment with a possible data-driven solution which exploits simple features of the input question and its surrounding context (the target of the session, and the previous questions) to inform the next phases of the QA process.

Combining lexical resources with tree edit distance for recognizing textual entailment

This paper addresses Textual Entailment (i.e. recognizing that the meaning of a text entails the meaning of another text) using a Tree Edit Distance algorithm between the syntactic trees of the two texts. A key aspect of the approach is the estimation of the cost for the editing operations (i.e. Insertion, Deletion, Substitution) among words. The aim of the paper is to compare the contribution of two different lexical resources for recognizing textual entailment: WordNet and a word-similarity database. In both cases we derive entailment rules that are used by the Tree Edit Distance Algorithm. We carried out a number of experiments over the PASCAL-RTE dataset in order to estimate the contribution of different combinations of the available resources.

Bulgarian-english question answering: adaptation of language resources

This paper describes the Bulgarian part of a Bulgarian–English question answering system. The Bulgarian modules are implemented as a question analysis procedure within a Bulgarian question answering system — BulQA. The paper presents the available language resources and corresponding technology which is used for the analysis of the questions in Bulgarian and their translation into English format, which is necessary for answer extraction. CLaRK System is used as an implementation platform.

Unsupervised acquisition of entailment relations from the Web

Entailment recognition is a primary generic task in natural language inference, whose focus is to detect whether the meaning of one expression can be inferred from the meaning of the other. Accordingly, many NLP applications would benefit from high coverage knowledgebases of paraphrases and entailment rules. To this end, learning such knowledgebases from the Web is especially appealing due to its huge size as well as its highly heterogeneous content, allowing for a more scalable rule extraction of various domains. However, the scalability of state-of-the-art entailment rule acquisition approaches from the Web is still limited. We present a fully unsupervised learning algorithm for Webbased extraction of entailment relations. We focus on increased scalability and generality with respect to prior work, with the potential of a large-scale Web-based knowledgebase. Our algorithm takes as its input a lexical–syntactic template and searches the Web for syntactic templates that participate in an entailment relation with the input template. Experiments show promising results, achieving performance similar to a state-of-the-art unsupervised algorithm, operating over an offline corpus, but with the benefit of learning rules for different domains with no additional effort.