Rachele Sprugnoli

Comparing two methods for crowdsourcing speech transcription

This paper presents the results of an experimental study conducted with the aim of comparing two methods for crowdsourcing speech transcription that incorporate two different quality control mechanisms (i.e. explicit versus implicit) and that are based on two different processes (i.e. parallel versus iterative). In the Gold Standard method the same speech segment is transcribed in parallel by multiple contributors whose reliability is checked with respect to some reference transcriptions provided by experts. On the other hand, in the Dual Pathway method two independent groups of contributors work on the same set of transcriptions refining them in an iterative way until they converge, and thus eliminating the need to have reference transcriptions and to check transcription quality in a separate phase. These two methods were tested on about half an hour of broadcast news speech and for two different European languages, namely German and Italian. Both methods obtained good results in terms of Word Error Rate (WER) and compare well with the word disagreement rate of experts on the same data.

Annotating Causality in the TempEval-3 Corpus

While there is a wide consensus in the NLP community over the modeling of temporal relations between events, mainly based on Allen’s temporal logic, the question on how to annotate other types of event relations, in particular causal ones, is still open. In this work, we present some annotation guidelines to capture causality between event pairs, partly inspired by TimeML. We then implement a rule-based algorithm to automatically identify explicit causal relations in the TempEval-3 corpus. Based on this annotation, we report some statistics on the behavior of causal cues in text and perform a preliminary investigation on the interaction between causal and temporal relations.

Towards sentiment analysis for historical texts

This article presents the integration of sentiment analysis in ALCIDE, an online platform for historical content analysis. A prior polarity approach has been applied to a corpus of Italian historical texts, and a new lexical resource has been developed with a semi-automatic mapping starting from two English lexica. This article also reports on a first experiment on contextual polarity using both expert annotators and crowdsourced contributors. The long-term goal of our research is to create a system to support historical studies, which is able to analyse the sentiment in historical texts and to discover the opinion about a topic and its change over time.

The New Release of CORPS: A Corpus of Political Speeches Annotated with Audience Reactions

In this paper we present the new release of CORPS CORpus of tagged Political Speeches that contains transcripts of political speeches tagged with audience reactions, such as APPLAUSE or LAUGHTER. The corpus has been built with the goal of allowing automatic processing of the stored data. These tags signal hot-spots about persuasive communication and can be usefully employed in many theoretical and applied fields, providing insights well beyond those of traditional word-count approaches. After introducing the main characteristics of the corpus and some quantitative descriptions, we discuss possible uses of this resource.

The LME project: legislative metadata based on semantic formal models

The Law Making Environment (LME) system for planning, drafting and managing legislative sources is made up of editing and search support tools. To handle the semantics of legislative sources, two interacting models have been created: a rule model, which can describe the illocutionary profile of legislative texts through metadata, and a lightweight ontology. The search support tool, LMEmetaSearch, based on both models, is able to search for rules in legislative sources, and additionally to find implicit or complementary rules providing the user with a wide account on the subject.

One, no one and one hundred thousand events: Defining and processing events in an inter-disciplinary perspective *

We present an overview of event definition and processing spanning 25 years of research in NLP. We first provide linguistic background to the notion of event, and then present past attempts to formalize this concept in annotation standards to foster the development of benchmarks for event extraction systems. This ranges from MUC-3 in 1991 to the Time and Space Track challenge at SemEval 2015. Besides, we shed light on other disciplines in which the notion of event plays a crucial role, with a focus on the historical domain. Our goal is to provide a comprehensive study on event definitions and investigate which potential past efforts in the NLP community may have in a different research domain. We present the results of a questionnaire, where the notion of event for historians is put in relation to the NLP perspective.

Named Entity Recognition on Transcribed Broadcast News at EVALITA 2011

This paper describes features and outcomes of the Named Entity Recognition on Transcribed Broadcast News task at EVALITA 2011. This task represented a change with respect to previous editions of the NER task within the EVALITA evaluation campaign because it was based on automatic transcription of spoken broadcast news. In this paper, we present the training and test data used, the evaluation procedure and participants’ results. In particular, three participating systems are described and the results they obtained are discussed; special attention is given to the analysis of the impact of transcription errors on NER performance.

Legislative meta-data based on semantic formal models

The Law Making Environment system for planning, drafting and managing legislative sources is made up of editing and search support tools. In order to handle the semantics of legislative sources, two interacting models have been developed: a rule model and a lightweight ontology. Semantic draft and search support tools are based on both models, in a way that makes their interaction possible. A rule theory has been developed, which can describe the illocutionary profile of legislative texts through metadata. As far as concepts are concerned, a conceptual dictionary has also been created. Through the search tool, called LMEmetaSearch, the user is enabled, for instance, to search for obligations related to particular actions or subjects, described in the query through the domain ontology. The LMEmetaSearch shows the related rules and additionally, if requested, finds implicit information providing the user with a wide account on the subject and playing to some extent the role of an advice system.