Matheus Araújo

Comparing and combining sentiment analysis methods

Several messages express opinions about events, products, and services, political views or even their authors emotional state and mood. Sentiment analysis has been used in several applications including analysis of the repercussions of events in social networks, analysis of opinions about products and services, and simply to better understand aspects of social communication in Online Social Networks (OSNs). There are multiple methods for measuring sentiments, including lexical-based approaches and supervised machine learning methods. Despite the wide use and popularity of some methods, it is unclear which method is better for identifying the polarity (i.e., positive or negative) of a message as the current literature does not provide a method of comparison among existing methods. Such a comparison is crucial for understanding the potential limitations, advantages, and disadvantages of popular methods in analyzing the content of OSNs messages. Our study aims at filling this gap by presenting comparisons of eight popular sentiment analysis methods in terms of coverage (i.e., the fraction of messages whose sentiment is identified) and agreement (i.e., the fraction of identified sentiments that are in tune with ground truth). We develop a new method that combines existing approaches, providing the best coverage results and competitive agreement. We also present a free Web service called iFeel, which provides an open API for accessing and comparing results across different sentiment methods for a given text.

SentiBench - a benchmark comparison of state-of-the-practice sentiment analysis methods

In the last few years thousands of scientific papers have investigated sentiment analysis, several startups that measure opinions on real data have emerged and a number of innovative products related to this theme have been developed. There are multiple methods for measuring sentiments, including lexical-based and supervised machine learning methods. Despite the vast interest on the theme and wide popularity of some methods, it is unclear which one is better for identifying the polarity (i.e., positive or negative) of a message. Accordingly, there is a strong need to conduct a thorough apple-to-apple comparison of sentiment analysis methods, as they are used in practice, across multiple datasets originated from different data sources. Such a comparison is key for understanding the potential limitations, advantages, and disadvantages of popular methods. This article aims at filling this gap by presenting a benchmark comparison of twenty-four popular sentiment analysis methods (which we call the state-of-the-practice methods). Our evaluation is based on a benchmark of eighteen labeled datasets, covering messages posted on social networks, movie and product reviews, as well as opinions and comments in news articles. Our results highlight the extent to which the prediction performance of these methods varies considerably across datasets. Aiming at boosting the development of this research area, we open the methods’ codes and datasets used in this article, deploying them in a benchmark system, which provides an open API for accessing and comparing sentence-level sentiment analysis methods.

iFeel: a system that compares and combines sentiment analysis methods

Sentiment analysis methods are used to detect polarity in thoughts and opinions of users in online social media. As businesses and companies are interested in knowing how social media users perceive their brands, sentiment analysis can help better evaluate their product and advertisement campaigns. In this paper, we present iFeel, a Web application that allows one to detect sentiments in any form of text including unstructured social media data. iFeel is free and gives access to seven existing sentiment analysis methods: SentiWordNet, Emoticons, PANAS-t, SASA, Happiness Index, SenticNet, and SentiStrength. With iFeel, users can also combine these methods and create a new Combined-Method that achieves high coverage and F-measure. iFeel provides a single platform to compare the strengths and weaknesses of various sentiment analysis methods with a user friendly interface such as file uploading, graphical visualizing, and weight tuning.

An evaluation of machine translation for multilingual sentence-level sentiment analysis

Sentiment analysis has become a key tool for several social media applications, including analysis of users opinions about products and services, support to politics during campaigns and even for market trending. There are multiple existing sentiment analysis methods that explore different techniques, usually relying on lexical resources or learning approaches. Despite the large interest on this theme and amount of research efforts in the field, almost all existing methods are designed to work with only English content. Most existing strategies in specific languages consist of adapting existing lexical resources, without presenting proper validations and basic baseline comparisons. In this paper, we take a different step into this field. We focus on evaluating existing efforts proposed to do language specific sentiment analysis. To do it, we evaluated twenty-one methods for sentence-level sentiment analysis proposed for English, comparing them with two language-specific methods. Based on nine language-specific datasets, we provide an extensive quantitative analysis of existing multi-language approaches. Our main result suggests that simply translating the input text on a specific language to English and then using one of the existing English methods can be better than the existing language specific efforts evaluated. We also rank those implementations comparing their prediction performance and identifying the methods that acquired the best results using machine translation across different languages. As a final contribution to the research community, we release our codes and datasets. We hope our effort can help sentiment analysis to become English independent.

Using Facebook Ads Audiences for Global Lifestyle Disease Surveillance: Promises and Limitations

Every day, millions of users reveal their interests on Facebook, which are then monetized via targeted advertisement marketing campaigns. In this paper, we explore the use of demographically rich Facebook Ads audience estimates for tracking non-communicable diseases around the world. Across 47 countries, we compute the audiences of marker interests, and evaluate their potential in tracking health conditions associated with tobacco use, obesity, and diabetes, compared to the performance of placebo interests. Despite its huge potential, we find that, for modeling prevalence of health conditions across countries, differences in these interest audiences are only weakly indicative of the corresponding prevalence rates. Within the countries, however, our approach provides interesting insights on trends of health awareness across demographic groups. Finally, we provide a temporal error analysis to expose the potential pitfalls of using Facebooks Marketing API as a black box.

Measuring sentiments in online social networks

Sentiment analysis has being used in several applications including the analysis of the repercussion of events in online social networks (OSNs), as well as to summarize public perception about products and brands on discussions on those systems. There are multiple methods to measure sentiments, varying from lexical-based approaches to machine learning methods. Despite the wide use and popularity of some those methods, it is unclear which method is better for identifying the polarity (i.e. positive or negative) of a message, as the current literature does not provide a comparison among existing methods. This comparison is crucial to allow us to understand the potential limitations, advantages, and disadvantages of popular methods in the context of OSNs messages. This work aims at filling this gap by presenting a comparison between 8 popular sentiment analysis methods. Our analysis compares these methods in terms of coverage and in terms of correct sentiment identification. We also develop a new method that combines existing approaches in order to provide the best coverage results with competitive accuracy. Finally, we present iFeel, a Web service which provides an open API for accessing and comparing results across different sentiment methods for a given text.

Sentiment Analysis Methods for Social Media

Sentiment Analysis became an important topic in the Web recently, especially regarding to Online Social Networks. Many applications are monitoring products and brands, and even important social events like political campaigns. A large number of Sentiment Analysis methods and techniques were proposed in the literature. This short course offers an introduction to explore this theme. First, we present an overview about sentiment analysis and its popular applications. Next, we discuss main methods from literature. Finally, we compare these methods with each other highlighting advantages and limitations.

Towards sentiment analysis for mobile devices

The increasing use of smartphones to access social media platforms opens a new wave of applications that explore sentiment analysis in the mobile environment. However, there are various existing sentiment analysis methods and it is unclear which of them are deployable in the mobile environment. This paper provides the first of a kind study in which we compare the performance of 17 sentence-level sentiment analysis methods in the mobile environment. To do that, we adapted these sentence-level methods to run on Android OS and then we measure their performance in terms of memory usage, CPU usage, and battery consumption. Our findings unveil sentence-level methods that require almost no adaptations and run relatively fast as well as methods that could not be deployed due to excessive use of memory. We hope our effort provides a guide to developers and researchers interested in exploring sentiment analysis as part of a mobile application and can help new applications to be executed without the dependency of a server-side API.

An evaluation of sentiment analysis for mobile devices

Sentiment analysis has become a key tool to extract knowledge from data containing opinions and sentiments, particularly, data from online social systems. With the increasing use of smartphones to access social media platforms, a new wave of applications that explore sentiment analysis in the mobile environment is beginning to emerge. However, there are various existing sentiment analysis methods and it is unclear which of them are deployable in the mobile environment. In this paper, we provide the first of a kind study in which we compare the performance of 14 sentence-level sentiment analysis methods in the mobile environment. To do that, we adapted these methods to run on Android OS and then, we measure their performance in terms of memory, CPU, and battery consumption. Our findings unveil methods that require almost no adaptations and run relatively fast as well as methods that could not be deployed due to excessive use of memory. We hope our effort provides a guide to developers and researchers interested in exploring sentiment analysis as part of a mobile application and can help new applications to be executed without the dependency of a server-side API. We also share the Android API that implements all the 14 sentiment analysis methods used in this paper.

Emotional Fingerprint from Authors in Classical Literature

The Internet deeply changed the way people share their knowledge. Almost all content that people produces is now available in digital formats, like e-books, apps, newspapers, and magazines. That content has commonly some metadata available that can be used to generate complex recommendation systems that track content similarity. Since there is some effort in the literature to explore this direction, almost all use classical recommendation approaches, like collaborative filter data and information present on websites that sells books. While most efforts in the literature use features derived from the text syntax to create a recommendation model, our approach aims to trace an emotional fingerprint of authors extracted from their texts. This approach, known as psychometry, consists of the study of behavioral characteristics like positivity, negativity, sadness, fear, religiosity, sexuality, which are able to disguise individuals. Using two sentiment analysis lexicons and a collection of 641 books from the English literature written by 56 authors, we show the effectiveness of these psychometric features in order to trace those authors emotional fingerprint.