Anirban Sen

Extracting situational awareness from microblogs during disaster events

Microblogging sites such as Twitter and Weibo are increasingly being used to enhance situational awareness during various natural and man-made disaster events such as floods, earthquakes, and bomb blasts. During any such event, thousands of microblogs (tweets) are posted in short intervals of time. Typically, only a small fraction of these tweets contribute to situational awareness, while the majority merely reflect the sentiment or opinion of people. Real-time extraction of tweets that contribute to situational awareness is especially important for relief operations when time is critical. However, automatically differentiating such tweets from those that reflect opinion / sentiment is a non-trivial challenge, mainly because of the very small size of tweets and the informal way in which tweets are written (frequent use of emoticons, abbreviations, and so on). This study applies Natural Language Processing (NLP) techniques to address this challenge. We extract low-level syntactic features from the text of tweets, such as the presence of specific types of words and parts-of-speech, to develop a classifier to distinguish between tweets which contribute to situational awareness and tweets which do not. Experiments over tweets related to four diverse disaster events show that the proposed features identify situational awareness tweets with significantly higher accuracy than classifiers based on standard bag-of-words models.

On the role of conductance, geography and topology in predicting hashtag virality

We focus on three aspects of the early spread of a hashtag in order to predict whether it will go viral: the network properties of the subset of users tweeting the hashtag, its geographical properties, and, most importantly, its conductance-related properties. One of our significant contributions is to discover the critical role played by the conductance-based features for the successful prediction of virality. More specifically, we show that the second derivative of the conductance gives an early indication of whether the hashtag is going to go viral or not. We present a detailed experimental evaluation of the effect of our various categories of features on the virality prediction task. When compared to the baselines and the state-of-the-art techniques proposed in the literature our feature set is able to achieve significantly better accuracy on a large dataset of 7.7 million users and all their tweets over a period of month, as well as on existing datasets.

TEXT-TO-DIAGRAM CONVERSION: A METHOD FOR FORMAL REPRESENTATION OF NATURAL LANGUAGE GEOMETRY PROBLEMS

Natural language geometry problems are translated into formal representation. This is done as an essential step involved in text to diagram conversion. A parser is designed that analyzes a problem statement in order to describe it as a language independent, unambiguous formal representation. Natural language processing tools and a lexical knowledge base are used to assist the parser that finally generates a graph as the parsing output. The parse graph is the formal representation of the input natural language problem. This graph is later translated into another intermediate representation consisting of a set of graphics-friendly statements. High school level geometry problems are used to develop and test the proposed methods. Experimental results show high accuracy of the approach in translating a natural language problem into a formal description.

Multi-task Representation Learning for Enhanced Emotion Categorization in Short Text

Embedding based dense contextual representations of data have proven to be efficient in various NLP tasks as they alleviate the burden of heavy feature engineering. However, generalized representation learning approaches do not capture the task specific subtleties. In addition, often the computational model for each task is developed in isolation, overlooking the interrelation among certain NLP tasks. Given that representation learning typically requires a good amount of labeled annotated data which is scarce, it is essential to explore learning embedding under supervision of multiple related tasks jointly and at the same time, incorporating the task specific attributes too. Inspired by the basic premise of multi-task learning, which supposes that correlation between related tasks can be used to improve classification, we propose a novel technique for building jointly learnt task specific embeddings for emotion and sentiment prediction tasks. Here, a sentiment prediction task acts as an auxiliary input to enhance the primary emotion prediction task. Our experimental results demonstrate that embeddings learnt under supervised signals of two related tasks, outperform embeddings learnt in a uni-tasked setup for the downstream task of emotion prediction.

Empirical Analysis of the Presence of Power Elite in Media

Politicians, politically connected business persons, bureaucrats, celebrities, and highly placed government officials (collectively termed as the power elite in sociology literature) can influence national and regional policy for personal and organizational benefit, which may not always be in the best interests of the people. Media is a crucial tool to shape public opinion, and is used heavily by the power elite to bring legitimacy to their policy decisions. In this paper, we empirically analyze the coverage given to the power elite in mainstream media on Demonetization, a significant, recent policy event in India. We compare the extent of coverage given to the elite and non-elite, the policy slant expressed by them, and differences in coverage between seven of the largest news media organizations in India. We find that among the power elite, powerful politicians and political parties are given the maximum coverage in the media, with conspicuous negligence in coverage given to expert opinions. Sentiment analysis clearly reveals that opposing political factions express opposing views towards the policy, and there is variation across different news sources as well. We are applying our methods on other contentious policy events to be able to do a more systematic analysis of how media can aid the power elite to shape public opinion by giving them disproportionately large coverage and visibility.

Leveraging Web Data to Monitor Changes in Corporate-Government Interlocks in India

Corporate executives who are linked to politicians or administrative officials, or family members of public officials with links to corporate organizations, are known to build an interlocking social network that becomes a power structure of highly influential entities. Such power structures often lead to an inequitable distribution of resources and manipulation of policies. A deeper look at this power structure and the constituent interlocks can provide users with valuable insights on these influential connections, and eventually, on the shaping of socio-economic outcomes by the interlocked political economy. In this paper, we describe the design of a platform to empirically monitor the degree of corporate-government interlock in India over time, by making use of publicly available data on the web. We find that the interlock has strengthened over the last decade, and we report the kind of interconnections and structural changes that have happened during this time. We also describe the design of an application to present news articles about an event or topic alongside the interconnection network of entities referred to in the news articles, to help users get a quick view of the main actors involved in the event. We find that this news search application is able to highlight several interconnections between prominent entities in an event, which had not been reported in the media. Overall, we find it relevant to build a technology platform which can help researchers and journalists to monitor the extent of interlocks between powerful stakeholders in the corporate and government spheres.

Fine-Grained Emotion Detection in Contact Center Chat Utterances

Contact center chats are textual conversations involving customers and agents on queries, issues, grievances etc. about products and services. Contact centers conduct periodic analysis of these chats to measure customer satisfaction, of which the chat emotion forms one crucial component. Typically, these measures are performed at chat level. However, retrospective chat-level analysis is not sufficiently actionable for agents as it does not capture the variation in the emotion distribution across the chat. Towards that, we propose two novel weakly supervised approaches for detecting fine-grained emotions in contact center chat utterances in real time. In our first approach, we identify novel contextual and meta features and treat the task of emotion prediction as a sequence labeling problem. In second approach, we propose a neural net based method for emotion prediction in call center chats that does not require extensive feature engineering. We establish the effectiveness of the proposed methods by empirically evaluating them on a real-life contact center chat dataset. We achieve average accuracy of the order 72.6% with our first approach and 74.38% with our second approach respectively.

Embedding Learning of Figurative Phrases for Emotion Classification in Micro-Blog Texts

Figurative phrases such as idioms are a type of Multi-Word Expressions (MWE) that possess a specialized meaning, which is independent and different from the literal meaning of the constituent words. Figurative language is widely used to express emotions and are very predominant in micro-blog data.Therefore, an efficient model of emotion categorization for micro-blogs should be able to correctly represent the instances of figurative phrases in the data. However, due to their non-compositional nature, the phrasal representation of figurative language cannot be directly obtained from the constituent words and hence this requires novel approaches for addressing the problem of modeling figurative phrases in micro-blogs. Most of the existing methods of modeling figurative idiomatic phrases in traditional text data use the broader textual context available for better results. However, in case of micro-blog data, such large context is not available due to very short length of text, which poses an additional challenge. Given the need to model figurative language for emotion classification, this paper develops the novel idea of Emotion Sensitive Figurative Phrase Embedding (ESFPE) to model idiomatic phrases in micro-blog data and show upto 14% improvement in emotion classification performance over baseline. To the best of our knowledge, this is the first work towards figurative phrase modeling for emotion classification in micro-blog text.