Alessandro Cucchiarelli

A Taxonomy Learning Method and Its Application to Characterize a Scientific Web Community

The need to extract and manage domain-specific taxonomies has become increasingly relevant in recent years. A taxonomy is a form of business intelligence used to integrate information, reduce semantic heterogeneity, describe emergent communities and interest groups, and facilitate communication between information systems. We present a semiautomated strategy to extract domain-specific taxonomies from Web documents and its application to model a network of excellence in the emerging research field of enterprise interoperability

Quantitative and qualitative evaluation of the OntoLearn ontology learning system

Ontology evaluation is a critical task, even more so when the ontology is the output of an automatic system, rather than the result of a conceptualisation effort produced by a team of domain specialists and knowledge engineers. This paper provides an evaluation of the OntoLearn ontology learning system. The proposed evaluation strategy is twofold: first, we provide a detailed quantitative analysis of the ontology learning algorithms, in order to compute the accuracy of OntoLearn under different learning circumstances. Second, we automatically generate natural language descriptions of formal concept specifications, in order to facilitate per-concept qualitative analysis by domain specialists.

Unsupervised named entity recognition using syntactic and semantic contextual evidence

Proper nouns form an open class, making the incompleteness of manually or automatically learned classification rules an obvious problem. The purpose of this paper is twofold: first, to suggest the use of a complementary backup method to increase the robustness of any hand-crafted or machine-learning-based NE tagger; and second, to explore the effectiveness of using more fine-grained evidencenamely, syntactic and semantic contextual knowledgein classifying NEs.

Computer Based Assessment Systems Evaluation via the ISO9126 Quality Model

Many commercial products, as well as freeware and shareware tools, are the result of studies and research in this field made by companies and public institutions. This noteworthy growth in the market raises the problem of identifying a set of criteria that may be useful to an educational team wishing to select the most appropriate tool for their assessment needs. The scientific literature is very poor in respect of this issue. An important help is provided in this direction, by a number of research studies in the field of Software Engineering providing general criteria that may be used to evaluate software systems. Furthermore, a relevant effort has been made in this field by the International Standard Organization that in 1991 defined the ISO9126 standard for “Information Technology – Software Quality Characteristics and Sub-characteristics” (ISO, 1991). It is important to note that a typical CBA system is composed by: • A Test Management System (TMS) - i.e. a tool providing the instructor with an easy to use interface, the ability to create questions and to assemble them into tests, the possibility of grading the tests and making some statistical evaluations of the results. • A Test Delivery System (TDS) - i.e. a tool for the delivery of tests to the students. The tool may be used to deliver tests using paper and pencil, a stand-alone computer, on a LAN, or over the web. The TDS may be augmented with a web-enabler used to deliver the tests over the Web. In many cases producers distribute two different versions of the same TDS, one to deliver tests either on single computers or on LAN, and the other to deliver tests over the web. The TMS and TDS modules may be integrated in a single application or may be delivered as separate applications. Thus, it is of fundamental importance to devise a set of quality factors that can be used to evaluate both the modules belonging to this general structure of a CBA system.

Automatic adaptation of proper noun dictionaries through cooperation of machine learning and probabilistic methods

The recognition of Proper Nouns (PNs) is considered an important task in the area of Information Retrieval and Extraction. However the high performance of most existing PN classifiers heavily depends upon the availability of large dictionaries of domain-specific Proper Nouns, and a certain amount of manual work for rule writing or manual tagging. Though it is not a heavy requirement to rely on some existing PN dictionary (often these resources are available on the web), its coverage of a domain corpus may be rather low, in absence of manual updating. In this paper we propose a technique for the automatic updating of an PN Dictionary through the cooperation of an inductive and a probabilistic classifier. In our experiments we show that, whenever an existing PN Dictionary allows the identification of 50% of the proper nouns within a corpus, our technique allows, without additional manual effort, the successful recognition of about 90% of the remaining 50%.

Finding a domain-appropriate sense inventory for semantically tagging a corpus

Semantically tagging a corpus is useful for many intermediate NLP tasks such as: acquisition of word argument structures in sublanguages; acquisition of syntactic disambiguation cues; terminology learning; etc. The general idea is that semantic tags allow the generalization of observed word patterns, and facilitate the discovery of recurrent sublanguage phenomena and selectional rules of various types. Yet, as opposed to POS tags in morphology, there is no consensus in the literature about the type and granularity of the semantic tags to be used. In this paper, we argue that an appropriate selection of semantic tags should be domain-dependent. We propose a method by which we select from WordNet an inventory of semantic tags that are ‘optimal’ for a given corpus, according to a scoring function defined as a linear combination of general and corpus-dependent performance factors. We believe that an optimal selection of a category inventory is a necessary premise for obtaining better results in all lexically learning algorithms that are based on, or concerned with, semantic categorization of words. Furthermore, an adequate inventory (one which intuitively ‘fits’ with the semantics of a domain, e.g. phenomenon for Natural Science, or part , piece for a technical handbook) may facilitate the manual annotation of large corpora.

Semantically interconnected social networks

Social network analysis aims to identify collaborations and helps people organize themselves through community participation and information sharing. The primary sources for social network modelling are explicit relationships such as co-authoring, citations, friendship, etc. However, to enable the integration of on-line community information and to fully describe the content and structure of community sites, secondary sources of information, such as documents, e-mails, blogs and discussions, can be exploited. In this paper we describe a methodology and a battery of tools to automatically extract from documents the relevant topics shared among community members and to analyse the evolution of the network also in terms of emergence and decay of collaboration themes. Experiments are conducted on a scientific network funded by the European Community, the INTEROP network of excellence, and on the United Kingdom research community in medical image understanding and analysis.

Overcoming communication obstacles in user-analyst interaction for functional requirements elicitation

The importance of requirement engineering in the software development process has been widely recognised by the scientific community. One of the major error sources that arise in this phase is represented by ineffectual communication between users and analysts.Valusek and Fryback in [32] identify three classes of communication obstacles to a successful elicitation of requirements. Purpose of this paper is to discuss these obstacles and to identify the structure of a CASE tool that may allow to overcome them.

Semantic tagging of unknown proper nouns

In this paper, we describe a context-based method to semantically tag unknown proper nouns (U-PNs) in corpora. Like many others, our system relies on a gazetteer and a set of context-dependent heuristics to classify proper nouns. However, proper nouns are an open-end class: when parsing new fragments of a corpus, even in the same language domain, we can expect that several proper nouns cannot be semantically tagged. The algorithm that we propose assigns to an unknown PN an entity type based on the analysis of syntactically and semantically similar contexts already seen in the application corpus. The performance of the algorithm is evaluated not only in terms of precision, following the tradition of MUC conferences, but also in terms of information gain, an information theoretic measure that takes into account the complexity of the classification task.

Automatic Selection of Class Labels from a Thesaurus for an Effective Semantic Tagging of Corpora.

It is widely accepted that tagging text with semantic information would improve the quality of lexical learning in corpus-based NLP methods. However available on-line taxonomies are rather entangled and introduce an unnecessary level of ambiguity. The noise produced by the redundant number of tags often overrides the advantage of semantic tagging. In this paper we propose an automatic method to select from WordNet a subset of domain-appropriate categories that effectively reduce the overambiguity of WordNet, and help at identifying and categorise relevant language patterns in a more compact way. The method is evaluated against a manually tagged corpus, SEMCOR.