Ioannis P. Klapaftis

Reviewing and Evaluating Automatic Term Recognition Techniques

Automatic Term Recognition (ATR) is defined as the task of identifying domain specific terms from technical corpora. Termhood-basedapproaches measure the degree that a candidate term refers to a domain specific concept. Unithood-basedapproaches measure the attachment strength of a candidate term constituents. These methods have been evaluated using different, often incompatible evaluation schemes and datasets. This paper provides an overview and a thorough evaluation of state-of-the-art ATRmethods, under a common evaluation framework, i.e. corpora and evaluation method. Our contributions are two-fold: (1) We compare a number of different ATRmethods, showing that termhood-basedmethods achieve in general superior performance. (2) We show that the number of independent occurrences of a candidate term is the most effective source for estimating term nestedness, improving ATRperformance.

SemEval-2010 Task 14: Evaluation Setting for Word Sense Induction & Disambiguation Systems

This paper presents the evaluation setting for the SemEval-2010 Word Sense Induction (WSI) task. The setting of the SemEval-2007 WSI task consists of two evaluation schemes, i.e. unsupervised evaluation and supervised evaluation. The first one evaluates WSI methods in a similar fashion to Information Retrieval exercises using F-Score. However, F-Score suffers from the matching problem which does not allow: (1) the assessment of the entire membership of clusters, and (2) the evaluation of all clusters in a given solution. In this paper, we present the use of V-measure as a measure of objectively assessing WSI methods in an unsupervised setting, and we also suggest a small modification on the supervised evaluation.

Evaluating Word Sense Induction and Disambiguation Methods

Word Sense Induction (WSI) is the task of identifying the different uses (senses) of a target word in a given text in an unsupervised manner, i.e. without relying on any external resources such as dictionaries or sense-tagged data. This paper presents a thorough description of the SemEval-2010 WSI task and a new evaluation setting for sense induction methods. Our contributions are two-fold: firstly, we provide a detailed analysis of the Semeval-2010 WSI task evaluation results and identify the shortcomings of current evaluation measures. Secondly, we present a new evaluation setting by assessing participating systems’ performance according to the skewness of target words’ distribution of senses showing that there are methods able to perform well above the Most Frequent Sense (MFS) baseline in highly skewed distributions.

Graph Connectivity Measures for Unsupervised Parameter Tuning of Graph-Based Sense Induction Systems.

Word Sense Induction (WSI) is the task of identifying the different senses (uses) of a target word in a given text. This paper focuses on the unsupervised estimation of the free parameters of a graph-based WSI method, and explores the use of eight Graph Connectivity Measures (GCM) that assess the degree of connectivity in a graph. Given a target word and a set of parameters, GCM evaluate the connectivity of the produced clusters, which correspond to subgraphs of the initial (unclustered) graph. Each parameter setting is assigned a score according to one of the GCM and the highest scoring setting is then selected. Our evaluation on the nouns of SemEval-2007 WSI task (SWSI) shows that: (1) all GCM estimate a set of parameters which significantly outperform the worst performing parameter setting in both SWSI evaluation schemes, (2) all GCM estimate a set of parameters which outperform the Most Frequent Sense (MFS) baseline by a statistically significant amount in the supervised evaluation scheme, and (3) two of the measures estimate a set of parameters that performs closely to a set of parameters estimated in supervised manner.

UOY: A Hypergraph Model For Word Sense Induction & Disambiguation

This paper is an outcome of ongoing research and presents an unsupervised method for automatic word sense induction (WSI) and disambiguation (WSD). The induction algorithm is based on modeling the cooccurrences of two or more words using hypergraphs. WSI takes place by detecting high-density components in the cooccurrence hypergraphs. WSD assigns to each induced cluster a score equal to the sum of weights of its hyperedges found in the local context of the target word. Our system participates in SemEval-2007 word sense induction and discrimination task.

Detecting Predatory Behaviour from Online Textual Chats

This paper presents a novel methodology for learning the behavioural profiles of sexual predators by using state-of-the-art machine learning and computational linguistics methods. The presented methodology targets at distinguishing between predatory and non-predatory conversations and is evaluated in real-world data. All the text fragments within a malicious chat is not of predatory nature. Thus it is necessary to distinguish the predatory fragments from non-predatory ones. This distinction is made by implementing the notion of n-grams which captures predatory sequences from conversations. The paper uses as features both content words and stylistic features within conversations. The content words are weighed using tf-idf measure. Experiments show that content words alone are not enough to make distinction between predatory and non-predatory chats. The implementation of various stylistic features however improves the performance of the system.

Term Sense Disambiguation for Ontology Learning

An important issue in the construction of domain ontologies is the task of identifying terms and their corresponding definitions. Though many methods exist for automatic extraction of terminology from plain text, the semantic interpretation of these terms is either manual or semi-automatic. In this paper we present an unsupervised method for automatic term sense disambiguation (TSD), based on the identification of relevant contextual information from the Web. The proposed TSD can be applied to ontology learning (OL)

Graph-Based Relation Mining

Relationship mining or Relation Extraction (RE) is the task of identifying the different relations that might exist between two or more named entities. Relation extraction can be exploited in order to enhance the usability of a variety of applications, including web search, information retrieval, question answering and others. This paper presents a novel unsupervised method for relation extraction which casts the problem of RE into a graph-based framework. In this framework, entities are represented as vertices in a graph, while edges between vertices are drawn according to the distributional similarity of the corresponding entities. The RE problem is then formulated in a bootstrapping manner as an edge prediction problem, where in each iteration the target is to identify pairs of disconnected vertices (entities) most likely to share a relation.