Hector Llorens

Enhancing QA systems with complex temporal question processing capabilities

This paper presents a multilayered architecture that enhances the capabilities of current QA systems and allows different types of complex questions or queries to be processed. The answers to these questions need to be gathered from factual information scattered throughout different documents. Specifically, we designed a specialized layer to process the different types of temporal questions. Complex temporal questions are first decomposed into simple questions, according to the temporal relations expressed in the original question. In the same way, the answers to the resulting simple questions are recomposed, fulfilling the temporal restrictions of the original complex question. A novel aspect of this approach resides in the decomposition which uses a minimal quantity of resources, with the final aim of obtaining a portable platform that is easily extensible to other languages. In this paper we also present a methodology for evaluation of the decomposition of the questions as well as the ability of the implemented temporal layer to perform at a multilingual level. The temporal layer was first performed for English, then evaluated and compared with: a) a general purpose QA system (F-measure 65.47% for QA plus English temporal layer vs. 38.01% for the general QA system), and b) a well-known QA system. Much better results were obtained for temporal questions with the multilayered system. This system was therefore extended to Spanish and very good results were again obtained in the evaluation (F-measure 40.36% for QA plus Spanish temporal layer vs. 22.94% for the general QA system).

Applying semantic knowledge to the automatic processing of temporal expressions and events in natural language

This paper addresses the problem of the automatic recognition and classification of temporal expressions and events in human language. Efficacy in these tasks is crucial if the broader task of temporal information processing is to be successfully performed. We analyze whether the application of semantic knowledge to these tasks improves the performance of current approaches. We therefore present and evaluate a data-driven approach as part of a system: TIPSem. Our approach uses lexical semantics and semantic roles as additional information to extend classical approaches which are principally based on morphosyntax. The results obtained for English show that semantic knowledge aids in temporal expression and event recognition, achieving an error reduction of 59% and 21%, while in classification the contribution is limited. From the analysis of the results it may be concluded that the application of semantic knowledge leads to more general models and aids in the recognition of temporal entities that are ambiguous at shallower language analysis levels. We also discovered that lexical semantics and semantic roles have complementary advantages, and that it is useful to combine them. Finally, we carried out the same analysis for Spanish. The results obtained show comparable advantages. This supports the hypothesis that applying the proposed semantic knowledge may be useful for different languages.

SemEval-2015 Task 5: QA TempEval - Evaluating Temporal Information Understanding with Question Answering

QA TempEval shifts the goal of previous TempEvals away from an intrinsic evaluation methodology toward a more extrinsic goal of question answering. This evaluation requires systems to capture temporal information relevant to perform an end-user task, as opposed to corpus-based evaluation where all temporal information is equally important. Evaluation results show that the best automated TimeML annotations reach over 30% recall on questions with ‘yes’ answer and about 50% on easier questions with ‘no’ answers. Features that helped achieve better results are event coreference and a time expression reasoner.

Automatic Generalization of a QA Answer Extraction Module Based on Semantic Roles

In recent years, improvements on automatic semantic role labeling have grown the interest of researchers in its application to different NLP fields, specially to QA systems. We present a proposal of automatic generalization of the use of SR in QA systems to extract answers for different types of questions. Firstly, we have implemented two different versions of an answer extraction module using SR: a) rules-based, and b) patterns-based. These modules work as part of a QA system to extract answers for location questions. Secondly, these approaches have been automatically generalized to any type of factoid questions using generalization rules. The whole system has been evaluated using both location and temporal questions from TREC datasets. Results indicate that an automatic generalization is feasible, obtaining same quality results for both original type of questions and new auto-generalized one (Precision: 88.20% LOC and 95.08% TMP). Furthermore, results show that patterns-based approach works better in both types of questions (F1 improvement + 40.88% LOCand + 15.41% TMP).

Merging Temporal Annotations

In corpus linguistics obtaining high-quality semantically-annotated corpora is a fundamental goal. Various annotations of the same text can be obtained from automated systems, human annotators, or a combination of both. Obtaining, by manual means, a merged annotation from these, which improves the correctness of each individual annotation, is costly. We present automatic algorithms specifically for merging temporal annotations. These have been evaluated merging the annotations of three state-of-the-art systems on the gold standard corpora and the correctness of the merged annotation improved over that of individual annotations and baseline merging algorithms.

An integrated multidimensional modeling approach to access big data in business intelligence platforms

The huge amount of information available and its heterogeneity has surpassed the capacity of current data management technologies. Dealing with that huge amounts of structured and unstructured data, often referred as Big Data, is a hot research topic and a technological challenge. In this paper, we present an approach aimed to allow OLAP queries over different, heterogeneous, data sources. The modeling approach proposed is based on a MapReduce paradigm, which integrates different formats into the recent RDF Data Cube format. The benefits of our approach are that it allows a user to make queries that need data from different sources while maintaining, at the same time, an integrated, comprehensive view of the data available. The paper discusses the advantages and disadvantages, as well as the implementation challenges that such approach presents. Furthermore, the approach is illustrated in an example of application.

Automatic system for identifying and categorizing temporal relations in natural language

Nowadays, the automatic processing of digitalized documents is crucial to cope with the increasing amount of information available. This issue is addressed from the natural language processing (NLP) research field. One of the tasks required for many NLP applications is temporal information processing. It involves the automatic extraction and interpretation of temporal expressions, events, and their relations. Specifically, the identification and the categorization of temporal relations are the most complex subtasks yet to solve, judging from the results reported in the latest international evaluation exercise. Temporal relation identification has been addressed by very few approaches, and the current categorization approaches are still not a definitive solution. This paper presents a system that approaches temporal relation identification and categorization. The former is approached with a knowledge‐driven strategy and the later with data‐driven strategy based on different machine‐learning techniques. Our proposal has been empirically evaluated over the currently available English data sets annotated with temporal information (TimeBank and AQUAINT) in a 10‐fold cross‐validated experiment. The results obtained support that the presented approach achieves a high performance. It improves the baseline F1 by 46% and outperforms the state of the art.

Text summarization contribution to semantic question answering: New approaches for finding answers on the web

As the Internet grows, it becomes essential to find efficient tools to deal with all the available information. Question answering (QA) and text summarization (TS) research fields focus on presenting the information requested by users in a more concise way. In this paper, the appropriateness and benefits of using summaries in semantic QA are analyzed. For this purpose, a combined approach where a TS component is integrated into a Web‐based semantic QA system is developed. The main goal of this paper is to determine to what extent TS can help semantic QA approaches, when using summaries instead of search engine snippets as the corpus for answering questions. In particular, three issues are analyzed: (i) the appropriateness of query‐focused (QF) summarization rather than generic summarization for the QA task, (ii) the suitable length comparing short and long summaries, and (iii) the benefits of using TS instead of snippets for finding the answers, tested within two semantic QA approaches (named entities and semantic roles). The results obtained show that QF summarization is better than generic (58% improvement), short summaries are better than long (6.3% improvement), and the use of TS within semantic QA improves the performance for both named‐entity‐based (10%) and, especially, semantic‐role‐based QA (47.5%).

Syntax-motivated context windows of morpho-lexical features for recognizing time and event expressions in natural language

We present an analysis of morpho-lexical features to learn SVM models for recognizing TimeML time and event expressions. We evaluate over the TempEval-2 data, the features: word, lemma, and PoS in isolation, in different size static-context windows, and in a syntax-motivated dynamic-context windows defined in this paper. The results show that word, lemma, and PoS introduce complementary advantages and their combination achieves the best performance; this performance is improved using context, and, with dynamic-context, timex recognition is improved to reach state-of-art performance. Although more complex approaches improve the efficacy, the morpho-lexical features can be obtained more efficiently and show a reasonable efficacy.

Two Proposals of a QA Answer Extraction Module Based on Semantic Roles

The contribution of semantic roles to question answering is considered to be very valuable. Due to this fact, the aim of this paper is to analyze the influence of semantic roles in this area. In order to achieve this goal a web QA system has been implemented using two different proposals for the answer extraction module based on semantic roles, and both implementations have been evaluated for location type questions. For the first proposal, a simple set of semantic rules was created, whereas, for the second proposal, a database of possible answer semantic patterns was automatically developed. This DB is created in a first step and it will be reused each time the answer extraction module is used. Results of both approaches have been analyzed and compared showing that the patterns-based approach improves the rules-based one in precision (+ 34.40%) and recall (+ 42.80%).