Diego Mollá

Question Answering in Restricted Domains: An Overview

Automated question answering has been a topic of research and development since the earliest AI applications. Computing power has increased since the first such systems were developed, and the general methodology has changed from the use of hand-encoded knowledge bases about simple domains to the use of text collections as the main knowledge source over more complex domains. Still, many research issues remain. The focus of this article is on the use of restricted domains for automated question answering. The article contains a historical perspective on question answering over restricted domains and an overview of the current methods and applications used in restricted domains. A main characteristic of question answering in restricted domains is the integration of domain-specific information that is either developed for question answering or that has been developed for other purposes. We explore the main methods developed to leverage this domain-specific information.Automated question answering has been a topic of research and development since the earliest AI applications. Computing power has increased since the first such systems were developed, and the general methodology has changed from the use of hand-encoded knowledge bases about simple domains to the use of text collections as the main knowledge source over more complex domains. Still, many research issues remain. The focus of this article is on the use of restricted domains for automated question answering. The article contains a historical perspective on question answering over restricted domains and an overview of the current methods and applications used in restricted domains. A main characteristic of question answering in restricted domains is the integration of domain-specific information that is either developed for question answering or that has been developed for other purposes. We explore the main methods developed to leverage this domain-specific information.

Exploiting Paraphrases in a Question Answering System

We present a Question Answering system for technical domains which makes an intelligent use of paraphrases to increase the likelihood of finding the answer to the users question. The system implements a simple and efficient logic representation of questions and answers that maps paraphrases to the same underlying semantic representation. Further, paraphrases of technical terminology are dealt with by a separate process that detects surface variants.

Learning of Graph-based Question Answering Rules

In this paper we present a graph-based approach to question answering. The method assumes a graph representation of question sentences and text sentences. Question answering rules are automatically learnt from a training corpus of questions and answer sentences with the answer annotated. The method is independent from the graph representation formalism chosen. A particular example is presented that uses a specific graph representation of the logical contents of sentences.

Indexing on Semantic Roles for Question Answering

Semantic Role Labeling (SRL) has been used successfully in several stages of automated Question Answering (QA) systems but its inherent slow procedures make it difficult to use at the indexing stage of the document retrieval component. In this paper we confirm the intuition that SRL at indexing stage improves the performance of QA and propose a simplified technique named the Question Prediction Language Model (QPLM), which provides similar information with a much lower cost. The methods were tested on four different QA systems and the results suggest that QPLM can be used as a good compromise between speed and accuracy.

ExtrAns: Extracting answers from technical texts

Describes the ExtrAns answer-extraction system which uses logical forms and lexical relations for semantic representation, to delve into and leverage the meaning of sentences, phrases, and words.

Recognizing textual entailment via atomic propositions

This paper describes Macquarie Universitys Centre for Language Technology contribution to the PASCAL 2005 Recognizing Textual Entailment challenge. Our main aim was to test the practicability of a purely logical approach. For this, atomic propositions were extracted from both the text and the entailment hypothesis and they were expressed in a custom logical notation. The text entails the hypothesis if every proposition of the hypothesis is entailed by some proposition in the text. To extract the propositions and encode them into a logical notation the system uses the output of Link Parser. To detect the independent entailment relations the system relies on the use of Otter and WordNet.

An Approach for Query-Focused Text Summarisation for Evidence Based Medicine

We present an approach for extractive, query-focused, single-document summarisation of medical text. Our approach utilises a combination of target-sentence-specific and target-sentence-independent statistics derived from a corpus specialised for summarisation in the medical domain. We incorporate domain knowledge via the application of multiple domain-specific features, and we customise the answer extraction process for different question types. The use of carefully selected domain-specific features enables our summariser to generate content-rich extractive summaries, and an automatic evaluation of our system reveals that it outperforms other baseline and benchmark summarisation systems with a percentile rank of 96.8%.

Towards reconciling use cases via controlled language and graphical models

In requirements engineering use cases are employed to describe the flow of events and the occurrence of states in a future information system. Use cases consist of a set of scenarios each of them describing an exemplary behaviour of the system to be developed. Different stakeholders describe the steps in varying ways since they perceive the state of affairs in the application domain from different viewpoints. This results in ambiguous use cases written in natural language that use different terminology and are therefore difficult to reconcile. To solve this problem, we have developed a set of simple guidelines to rewrite use cases and scenarios in a controlled language. The sentences are translated into flat logical forms by the Prolog module of our RECOCASE system. These resulting flat logical forms are used by RECOCASE to generate graphical models for the elaboration and refinement of functional requirements between project stakeholders. As an experiment we have chosen Formal Concept Analysis to automatically represent the viewpoints of different stakeholders graphically in a concept lattice.

Automatic evidence quality prediction to support evidence-based decision making

BACKGROUND Evidence-based medicine practice requires practitioners to obtain the best available medical evidence, and appraise the quality of the evidence when making clinical decisions. Primarily due to the plethora of electronically available data from the medical literature, the manual appraisal of the quality of evidence is a time-consuming process. We present a fully automatic approach for predicting the quality of medical evidence in order to aid practitioners at point-of-care. METHODS Our approach extracts relevant information from medical article abstracts and utilises data from a specialised corpus to apply supervised machine learning for the prediction of the quality grades. Following an in-depth analysis of the usefulness of features (e.g., publication types of articles), they are extracted from the text via rule-based approaches and from the meta-data associated with the articles, and then applied in the supervised classification model. We propose the use of a highly scalable and portable approach using a sequence of high precision classifiers, and introduce a simple evaluation metric called average error distance (AED) that simplifies the comparison of systems. We also perform elaborate human evaluations to compare the performance of our system against human judgments. RESULTS We test and evaluate our approaches on a publicly available, specialised, annotated corpus containing 1132 evidence-based recommendations. Our rule-based approach performs exceptionally well at the automatic extraction of publication types of articles, with F-scores of up to 0.99 for high-quality publication types. For evidence quality classification, our approach obtains an accuracy of 63.84% and an AED of 0.271. The human evaluations show that the performance of our system, in terms of AED and accuracy, is comparable to the performance of humans on the same data. CONCLUSIONS The experiments suggest that our structured text classification framework achieves evaluation results comparable to those of human performance. Our overall classification approach and evaluation technique are also highly portable and can be used for various evidence grading scales.

Extractive summarisation of medical documents using domain knowledge and corpus statistics

BACKGROUND Evidence Based Medicine (EBM) practice requires practitioners to extract evidence from published medical research when answering clinical queries. Due to the time- consuming nature of this practice, there is a strong motivation for systems that can automatically summarise medical documents and help practitioners find relevant information. AIM The aim of this work is to propose an automatic query- focused, extractive summarisation approach that selects informative sentences from medical documents. METHOD We use a corpus that is specifically designed for summarisation in the EBM domain. We use approximately half the corpus for deriving important statistics associated with the best possible extractive summaries. We take into account factors such as sentence position, length, sentence content, and the type of the query posed. Using the statistics from the first set, we evaluate our approach on a separate set. Evaluation of the qualities of the generated summaries is performed automatically using ROUGE, which is a popular tool for evaluating automatic summaries. RESULTS Our summarisation approach outperforms all baselines (best baseline score: 0.1594; our score 0.1653). Further improvements are achieved when query types are taken into account. CONCLUSION The quality of extractive summarisation in the medical domain can be significantly improved by incorporating domain knowledge and statistics derived from a specialised corpus. Such techniques can therefore be applied for content selection in end-to-end summarisation systems.