Debanjan Ghosh

Analyzing Argumentative Discourse Units in Online Interactions

Argument mining of online interactions is in its infancy. One reason is the lack of annotated corpora in this genre. To make progress, we need to develop a principled and scalable way of determining which portions of texts are argumentative and what is the nature of argumentation. We propose a two-tiered approach to achieve this goal and report on several initial studies to assess its potential.

Sarcastic or Not: Word Embeddings to Predict the Literal or Sarcastic Meaning of Words

Sarcasm is generally characterized as a figure of speech that involves the substitution of a literal by a figurative meaning, which is usually the opposite of the original literal meaning. We re-frame the sarcasm detection task as a type of word sense disambiguation problem, where the sense of a word is either literal or sarcastic. We call this the Literal/Sarcastic Sense Disambiguation (LSSD) task. We address two issues: 1) how to collect a set of target words that can have either literal or sarcastic meanings depending on context; and 2) given an utterance and a target word, how to automatically detect whether the target word is used in the literal or the sarcastic sense. For the latter, we investigate several distributional semantics methods and show that a Support Vector Machines (SVM) classifier with a modified kernel using word embeddings achieves a 7-10% F1 improvement over a strong lexical baseline.

Identification of nonliteral language in social media: A case study on sarcasm

With the rapid development of social media, spontaneously user‐generated content such as tweets and forum posts have become important materials for tracking peoples opinions and sentiments online. A major hurdle for current state‐of‐the‐art automatic methods for sentiment analysis is the fact that human communication often involves the use of sarcasm or irony, where the author means the opposite of what she/he says. Sarcasm transforms the polarity of an apparently positive or negative utterance into its opposite. Lack of naturally occurring utterances labeled for sarcasm is one of the key problems for the development of machine‐learning methods for sarcasm detection. We report on a method for constructing a corpus of sarcastic Twitter messages in which determination of the sarcasm of each message has been made by its author. We use this reliable corpus to compare sarcastic utterances in Twitter to utterances that express positive or negative attitudes without sarcasm. We investigate the impact of lexical and pragmatic factors on machine‐learning effectiveness for identifying sarcastic utterances and we compare the performance of machine‐learning techniques and human judges on this task.

Coarse-grained Argumentation Features for Scoring Persuasive Essays.

Scoring the quality of persuasive essays is an important goal of discourse analysis, addressed most recently with highlevel persuasion-related features such as thesis clarity, or opinions and their targets. We investigate whether argumentation features derived from a coarse-grained argumentative structure of essays can help predict essays scores. We introduce a set of argumentation features related to argument components (e.g., the number of claims and premises), argument relations (e.g., the number of supported claims) and typology of argumentative structure (chains, trees). We show that these features are good predictors of human scores for TOEFL essays, both when the coarsegrained argumentative structure is manually annotated and automatically predicted.

Towards Feasible Guidelines for the Annotation of Argument Schemes

The annotation of argument schemes represents an important step for argumentation mining. General guidelines for the annotation of argument schemes, applicable to any topic, are still missing due to the lack of a suitable taxonomy in Argumentation Theory and the need for highly trained expert annotators. We present a set of guidelines for the annotation of argument schemes, taking as a framework the Argumentum Model of Topics (Rigotti and Morasso, 2010; Rigotti, 2009). We show that this approach can contribute to solving the theoretical problems, since it offers a hierarchical and finite taxonomy of argument schemes as well as systematic, linguistically-informed criteria to distinguish various types of argument schemes. We describe a pilot annotation study of 30 persuasive essays using multiple minimally trained non-expert annotators .Our findings from the confusion matrixes pinpoint problematic parts of the guidelines and the underlying annotation of claims and premises. We conduct a second annotation with refined guidelines and trained annotators on the 10 essays which received the lowest agreement initially. A significant improvement of the inter-annotator agreement shows that the annotation of argument schemes requires highly trained annotators and an accurate annotation of argumentative components (premises and claims).

Annotating Multiparty Discourse: Challenges for Agreement Metrics

To computationally model discourse phenomena such as argumentation we need corpora with reliable annotation of the phenomena under study. Annotating complex discourse phenomena poses two challenges: fuzziness of unit boundaries and the need for multiple annotators. We show that current metrics for inter-annotator agreement (IAA) such as P/R/F1 and Krippendorff’s provide inconsistent results for the same text. In addition, IAA metrics do not tell us what parts of a text are easier or harder for human judges to annotate and so do not provide sufficiently specific information for evaluating systems that automatically identify discourse units. We propose a hierarchical clustering approach that aggregates overlapping text segments of text identified by multiple annotators; the more annotators who identify a text segment, the easier we assume that the text segment is to annotate. The clusters make it possible to quantify the extent of agreement judges show about text segments; this information can be used to assess the output of systems that automatically identify discourse units.

Effects of Topic Familiarity on Query Reformulation Strategies

Users employ various Query Reformulation (QR) Strategies to improve their search experiences as well as to retrieve the better results. In this exploratory research, we investigate the association of topic familiarities with such QR strategies. We design a user study where 38 participants with different levels of Topic Familiarity (TF) took part in the TREC Genomics Search Track (a total of five search tasks). We conducted two experiments: 1) we evaluate a set of predefined QR strategies to discover if participants with a specific level of TF tends to use a specific QR strategy. 2) we propose a supervised classification system that classifies consecutive queries made by the participants with different levels (low, medium, and high) of topic familiarity. Various lexical and distributional semantic features are implemented and feature analysis shows that participants with low familiarity tend to alter the spelling and use stemming for QR whereas participants with medium or higher familiarity are inclined to add new terms and phrases to reformulate queries.

#Criming and #alive: network and content analysis of two sides of a story on Twitter

On December 3, 2014, after a grand jury decided not to indict the white police officer in the death of Eric Garner, the social networking platform Twitter was flooded with tweets sharing stances on racial profiling and police brutality. To examine how issues concerning race were communicated and exchanged during this time, this study compares differences between tweets using two trending hashtags #CrimingWhileWhite (#cww) and #AliveWhileBlack (#awb) from December 3 through December 11, 2014. To this end, network and content analysis are used on a large dataset of tweets containing the hashtags #awb and #cww. Findings indicate that there are clear differences, both structurally and in linguistic style, between how individuals express themselves based on which hashtag they used. Specifically, we found that #cww users disproportionately shared informational content, which may have led to the hashtag gaining more network volume and attention as a trending topic than #awb. In contrast, #awb tweets tended to be more subjective, expressing a sense of community and strong negative sentiment toward persistent structural racism.

Sarcasm Analysis using Conversation Context

Computational models for sarcasm detection have often relied on the content of utterances in isolation. However, the speaker’s sarcastic intent is not always apparent without additional context. Focusing on social media discussions, we investigate three issues: (1) does modeling conversation context help in sarcasm detection? (2) can we identify what part of conversation context triggered the sarcastic reply? and (3) given a sarcastic post that contains multiple sentences, can we identify the specific sentence that is sarcastic? To address the first issue, we investigate several types of Long Short-Term Memory (LSTM) networks that can model both the conversation context and the current turn. We show that LSTM networks with sentence-level attention on context and current turn, as well as the conditional LSTM network, outperform the LSTM model that reads only the current turn. As conversation context, we consider the prior turn, the succeeding turn, or both. Our computational models are tested on two types of social media platforms: Twitter and discussion forums. We discuss several differences between these data sets, ranging from their size to the nature of the gold-label annotations. To address the latter two issues, we present a qualitative analysis of the attention weights produced by the LSTM models (with attention) and discuss the results compared with human performance on the two tasks.