Real Time Monitoring of Social Media and Digital Press
aa r X i v : . [ c s . C L ] J a n Real Time Monitoring of Social Media and Digital Press
Iñaki San Vicente*, Xabier Saralegi a , Rodrigo Agerri b a Elhuyar Foundation b IXA NLP Group, University of the Basque Country UPV/EHU
Abstract
Talaia is a platform for monitoring social media and digital press. A configurablecrawler gathers content with respect to user defined domains or topics. Crawleddata is processed by means of the EliXa Sentiment Analysis system. A Djangopowered interface provides data visualization for a user-based analysis of the data.This paper presents the architecture of the system and describes in detail its differentcomponents. To prove the validity of the approach, two real use cases are accountedfor: one in the cultural domain and one in the political domain. Evaluation for thesentiment analysis task in both scenarios is also provided, showing the capacity fordomain adaptation.
Keywords:
Sentiment Analysis, Social Media Analysis, Crawling, NaturalLanguage Processing, Digital Media Monitoring
1. Introduction
The Internet is a very rich source of user-generated information. As knowledgemanagement technologies have evolved, many organizations have turned their eyes tosuch information, as a way of obtaining global feedback on their activities (Chen et al.,2012). Some studies (O’Connor et al., 2010; Ceron et al., 2015) have pointed outthat such systems could perform as well as traditional polling systems, but at amuch lower cost.Talaia is a platform for monitoring the impact of topics specified by the userin social media and digital press. The process starts when the user configures thesystem to find information related to a domain or topic. Talaia provides real timeinformation on the topic and graphic visualizations to help users interpreting thedata. Such technology has various applications areas, such as: • Monitoring events: Following public events in real time harvesting people’sopinions and media news (Sutton, 2009; Yu & Wang, 2015).
Preprint submitted to Engineering Applications of Artificial Intelligence January 16, 2019
Analyze citizen or electors voice: Tracking the opinions citizens convey withrespect to public services or trends during electoral campaigns (Ceron et al.,2015). • Marketing and brand management: Measuring the impact of marketing campaignsin a digital environment (Ahmed et al., 2018). • Business Intelligence: Fast and efficient visualization of the information extractedfrom social media offers companies the possibility to analyse opinions abouttheir products or services (He et al., 2013; Mostafa, 2013). • Security: Detection of social conflicts, crimes, and cyberbullying (Xu et al.,2012; Dadvar et al., 2013).Talaia consists of three main modules: (i) a crawler collecting the data; (ii) a dataanalysis module; and (iii) a Graphical User Interface (GUI) providing interpretationof the data analyzed. Figure 1 describes the architecture. Its main features are thefollowing: • Monitoring and automatic analysis: Definition of the domain/topic by means ofterm taxonomies. Continuous monitoring of various mention sources includingsocial media and digital press. • Multilingual extraction of mentions and opinions relevant to the topics monitored,by means of Natural Language Processing (NLP) techniques. • Result exploration: Intuitive GUI to visualize and analyse the results. Advancedstatistics and filters, such as per language results, impact of the topics or authorstatistics. • Control of the monitoring process through the user interface: update searchterms or review and correct gathered mentions.This paper focuses on those processes monitoring user satisfaction with respectto a topic, and that is why we pay special attention to the Sentiment Analysis(SA) module. Nevertheless, Talaia is capable of performing further data analysistasks involving user profiling, in order to get the most out of the data. Specifically,geolocalization, user community identification and gender detection have been implemented.Section 4.4 provides more details.The rest of the paper is organized as follows. Section 2 discusses previous work,focusing on social media and SA. Both the academic and industrial points of view2 igure 1: Diagram Talaia’s components. are taken into account. The third section describes in detail the modules composingTalaia. Section 6 presents two success cases where the platform has been used formonitoring different events. Section 7 provides evaluation and results on the SA taskfor both scenarios. The last section draws some conclusions and future directions.
2. Background
Social media are becoming the primary environment for producing, spreading andconsuming information. Enormous quantities of user generated content are produced3onstantly. Even traditional media spread their news and get a large amount of traffictrough social media. Monitoring events or topics in such an environment is howevera challenging task. That is where data mining and Natural Language Processing(NLP) become essential. We have to be able to collect large scale data, but alsoto extract the relevant information. Tracking a topic over an extended time periodmeans that the information flow grows and fades over time. Also a topic may evolvein terms of the vocabulary used, and thus “topic detection and tracking” (TDT)(Allan et al., 1998) techniques become relevant to maintain a successful monitoring.Several systems have been proposed in the literature to explore events. TrendMiner (Preoţiuc-Pietro & Cohn, 2013) extracts multilingual terms from social media,groups and visualizes them in temporal series. Social Sensor (Aiello et al., 2013) andTwitcident (Abel et al., 2012) may be the most similar systems to ours. The first onefocuses on tracking topic or events predefined by the user. The second makes userdefined searches related to crisis management. LRA aims to discovering and trackingcrisis situations based on crowdsourced information. ReDites (Osborne et al., 2014)detects an tracks topics in a fully automated way.Detecting terms that represent a domain or topic semantically has been traditionallyaddressed by statistical models such as Latent Dirichlet Association (LDA) (Blei et al.,2003). Classical LDA models are applied over static document collections. In orderto extract terms from dynamic collections, the most common approach is to follow atwo step strategy (Shamma et al., 2011) consisting of detecting emerging terms andgrouping them in clusters thereby defining a domain.Nguyen et al. (2016) predict emerging terms by means of word co-occurrencedistributional models, comparing the terms in an specific time window against thewhole collection. Abilhoa & De Castro (2014) use a graph-based representation ofthe document collection. Aiello et al. (2013) propose df - idf t (Document Frequency- Inverse Document Frequency), a variation of tf - idf that includes the temporalfactor. Kim et al. (2016) combine neural networks and sequence labelling in order toextract relevant terms from conversations. Miao et al. (2017) propose to reduce thecost of predicting emerging topics by finding a small group of representative usersand predict the emerging topics from their social media activity.There is also the problem of the scope of the event or topic to be tracked. Anevent might be tracked at global level (e.g. Football World Cup), but most eventsare local or regional at most. Two issues arise at this point. Firstly, how to restrictthe data gathered to a specific region, and, secondly, how to cope with multilingualdata. Some authors tackle the problem by automatically geolocating tweets while In the last years microblogging sites such as Twitter have attracted the attentionof many researchers with diverse objectives such as stock market prediction (Bollen et al.,2010; Oliveira et al., 2017), polling estimation (O’Connor et al., 2010; Ceron et al.,2015) or analysis of crisis situations (Pope & Griffith, 2016; Shaikh et al., 2017;Öztürk & Ayvaz, 2018). The growing number of SA related shared tasks (e.g.,SemEval Aspect based SA and Twitter SA shared tasks) or the commercial platformsfor reputation management (see section 2.3) are proof of the interest from bothacademic and market worlds.The particularities of its language make it hard to analyze tweets. User mentions,hashtags, the growing presence of emojis, ungrammatical sentences, vocabulary variationsand other phenomena pose a great challenge for traditional NLP tools (Foster et al.,2011; Liu et al., 2011). Brody & Diakopoulos (2011) deal with the word lengtheningphenomenon, which is especially important for sentiment analysis because it usuallyexpresses emphasis of the message. Hashtag decomposition (e.g., ) (Brun & Roux, 2014; Belainine et al., 2016) or matchingOut Of Vocabulary (OOV) forms and acronyms to their standard vocabulary forms(e.g., ‘imo = in my opinion’ ) (Han & Baldwin, 2011; Liu et al., 2012; Alegria et al.,2014) are other addressed issues. International benchmarking initiatives such asthe TweetNorm shared task(Alegria et al., 2015) or the WNUT workshop series areproof of the interest to solve this task.Once texts are normalized, sentiment analysis can be performed. Several ruled-basedsystems to polarity classification have been proposed (Hu & Liu, 2004; Thelwall,2017; Taboada et al., 2011). Nevertheless, we will focus on Machine Learning (ML)based approaches which are the most widespread. Support Vector Machines (SVM)and Logistic Regression algorithms have been the very popular for polarity classificationas various international shared tasks (Román et al., 2015; Pontiki et al., 2014; Rosenthal et al.,2014) show. Typical features of those systems include sentiment word/lemma ngramfeatures, POS tags (Barbosa & Feng, 2010), Sentiment Lexicons (Kouloumpis et al.,2011), emoticons (O’Connor et al., 2010), discourse information (Somasundaran et al., http://noisy-text.github.io/2018/ (Go et al.,2009). This is feasible for major languages, but it is a very difficult (if possible atall) and time costly task for non major languages such as Basque.Our system is closest to Barbosa & Feng (2010) and Kouloumpis et al. (2011)because it combines polarity lexicons with machine learning for labelling sentimentof tweets. This strategy has proven to be a successful approach in previous sharedtasks (Saralegi & San Vicente, 2012; Mohammad et al., 2013). We can find various commercial solutions in the market. We are particularlyinterested in systems that provide an integral solution of the monitoring process,leaving out tools that only approach specific phases of the surveillance process, orsolutions that offer bare NLP processing chains which require further developmentto achieve a working social media monitor. Table A.6 in Annex I offers a detailedcomparative of the tools analyzed. We focus our analysis on the sources where Collect tweets containing the “:)” emoticon and regard them as positive, and likewise for the“:(” emoticon.
Iconoce is a system oriented to reputation management, offering various featuressuch as measuring impact of campaigns, or reputation monitoring. Although it alsocan monitor social media (Twitter and Facebook) its strength lies on the analysis ofdigital press. Multilingual information can be gathered but no linguistic processingis performed (lemmatization or crosslingual searches). It has 3 separated searchengines for authors, mentions and comments. A customizable dashboard offersvarious visualizations and data aggregations (e.g., salient term and topics, influencer,sentiment or trends). Periodical reports and alerts in the face of tendency changesare provided. As a distinctive feature, it offers a personalized press archive based onthe customer configuration.In a similar way, INNGUMA is a tool providing business intelligence services.They put their main effort in the crawling step. Rather than offering to the userresults over analysed data, the tool is designed for a group of customers to analyzethe data collaboratively. Customers are provided with a search engine (more or lesspowerful depending on the pricing plan), and interface where they can store andshare their findings. Lexalitycs and Meaning Cloud are text analytics enterprises. Their strengthis the data analysis part rather than the monitoring of many sources. Both systemsare built upon robust NLP chains. Document classification, entity extraction andaspect based sentiment analysis are performed. Sentiment Analysis is approached bymeans of rule-based systems based on lexicons and deep linguistic analysis, offeringthe possibility of custom domain adaptations. Both Lexalitics and Meaning Cloudlack a result visualization interface, limiting their outputs to Excel plugins, leavingthe full analysis of the data into the user’s hands. Websays monitors a wide range of sources including news, Blogs/RSS, Forums,Facebook, Twitter, LinkedIn, Instagram, Foursquare, Pinterest, Youtube, Vimeo,Reviews (Tripadvisor, Booking,...). The user is able to configure the crawling usingkeywords. Negative words are also allowed in order to effectively restrict the searchto the desired domain. The system is able to process data in several languages,but they report to be most effective with European languages (Spanish, English, http://info.iconoce.com/ https://websays.com/ Keyhole is a monitoring and analyticstool that provides trends, insights, and analysis (including sentiment) of hashtags,keywords, or accounts on Twitter and Instagram. It reports supporting data processingin a number of languages, but no details are given on the technology. User can alsotrack web mentions, but two separate monitoring processes must be setup. Lynguo is also in the same group of Websays and Keyhole. It claims to providesupport in 24 languages, although it reports full processing chain for Spanish andEnglish . NLP is done by means of “a range of linguistic tools to cover and combinein real time the different lexical, morphological and semantic processing layers,with machine learning and deep learning models, and software architectures” . SAincludes lexicons, customizable by the user. Monitoring is configured specifyingkeywords and users, allowing for negative ones as well. Lynguo is also able togeolocate comments. Ubermetrics is one of the few platforms that monitors multimedia sourcesincluding Youtube and Vimeo, but also TV and Radio sources. According to theirreports, it processes data in 40 languages. Its visualization dashboard offers customizablegraphs based on multiple search criteria. Ubermetrics main objective is analyzingvirality (impact) of the mentions and author profiling. Snaptrends monitors social media (Twitter, Facebook, Instagram, and Pinterest).Multilingual data is handled by means of MT (80 languages to English). It uses aproprietary NLP chain for processing English data, including sentiment analysis andrelevant term extraction. The main feature for filtering large volumes of informationis a geolocation-based search engine, combined with keyword based searches andother filters such as data sources. With respect to visualization, it has variousdata aggregations, such as influencer rankings or sentiment evolution across time https://keyhole.co/ http://lynguo.iic.uam.es/ http://snaptrends.com/
8y geographical area. Snaptrend makes an special effort in visualizing specific data,generating mention mosaics and timelines in real time.Talaia shares features with many of the aforementioned commercial solutions,yet it also possess its own characteristics. With a more robust text analysis thanIconoce and INNGUMA, and a more advanced interpretation of the data thanMeaningCloud and Lexalitics, Talaia is closer to tools such as Websays and Lynguo.Having keywords organized in a taxonomy allows us to provide deeper data analysisand aggregations. Moreover, Talaia is built using open source software with astrong academical background and tested against well known benchmarks. Talaia’sperformance is thus, verifiable.
3. Data Collection
The first step of a monitoring system such as Talaia is the collection of information.The Multi Source Monitor (MSM) system is currently able to monitor Twitter,syndication feeds and also multimedia sources such as television or radio programs.Support for other social media such as Youtube, Facebook, etc. is under development.MSM is a keyword based crawler, which works on a set of keywords defined bythe user. Rather than a list of unconnected terms, Talaia is designed to work over ahierarchy, which allows a better organization of the data for the analysis step. In thisway, keywords are defined as belonging to a specific category in the taxonomy. Onehandicap of crawling using a keyword-based strategy is that it is often difficult todefine unambiguous terms that do not capture noisy messages. In order to minimizethis situation, MSM implements a number of features:1. Regular expressions are used to define keywords. This allows to differentiatebetween common words and proper names, or full words and affixes (e.g., podemos ‘we can’ vs.
Podemos political party). These phenomena are speciallyfrequent in social media, where language rules are often ignored.2.
Language specific keywords . A word that is a very good keyword in alanguage can be a source of noise in another, e.g. mendia , ‘mountain’ inSpanish, is unambiguously referring to ‘Idoia Mendia’, a Basque politician,in our context, while in Basque it is clearly ambiguous.3.
Anchor terms usually define the general topic (e.g. election campaign) tomonitor. If the user specifies that a keyword requires an anchor, then in order http://github.com/Elhuyar/MSM
9o accept a message containing that keyword the message must also contain atleast one anchor term. Anchor terms may be keywords or not.4.
Long paragraphs are split before looking for keywords in the case of messagescoming from news sites. First, it looks if any keyword appears in a candidatearticle. If so, it looks for keywords sentence by sentence, and those sentencesare considered as the message unit.
Language Identification (LID)
LID is indispensable in order to apply the corresponding NLP analysis. LID isintegrated into the crawling process as part of the MSM system. There are twomain reasons for that. First, it allows us implement the aforementioned “languagespecific keyword” feature. Second, having the language identified in the first placegives us flexibility for applying the subsequent NLP tools. At the moment languageidentification is implemented using the library Optimaize , combined with sourcespecific optimizations (social media vs. feeds).
4. Data Analysis
The data analysis is mainly performed by EliXa (San Vicente et al., 2015) whichintegrates the following processes, each of them further detailed in the next sections.EliXa is a supervised Sentiment Analysis system. It was developed as a modularplatform which allows to easily conduct experiments by replacing the modules oradding new features. It was first tested in the ABSA 2015 shared task at SemEvalworkshop (Pontiki et al., 2015). EliXa currently offers resources and models for4 languages: Basque, Spanish, English and French. Its implementation is easilyadaptable to other languages, requiring a polarity lexicon and/or a training datasetfor each new language. To address the particular characteristics of tweets, EliXa integrates a microtextnormalization module which is applied to social media messages, based on Saralegi & San Vicente(2013b). The normalizer is based on heuristic rules, such as standardizing URLs,normalizing character repetitions or dividing long words (e.g. → ‘a very long day’). Also Out Of Vocabulary (OOV) term normalization is addressedby means of language specific frequency lists based on Twitter corpora. https://github.com/optimaize/language-detector https://github.com/Elhuyar/Elixa • Emoticons are normalized into a 7 sentiment scale: smiley, crying, shock, mute,angry, kiss, sadness . • Expressions that are meaningful for detecting SA such as interjections andonomatopoeia are marked.Those normalized terms must be included in the polarity lexicons in order tohave a greater impact in the sentiment analysis classification. Table 1 presents theresources provided for normalization according to their use. Word form dictionariesare composed of word forms extracted from corpora. When applying microtextnormalization, candidates are compared to forms in the dict in order to discardnoisy candidates. For example, if we were to normalize “happppy”, we would knowthe that the correct normalization is “happy” by looking at theses dictionaries.OOV dictionaries are composed of “OOV - standard form” pairs. These resourcesare valuable to normalize slang and commonly used abbreviations. In order toproduce such dictionaries word form frequency lists were generated from Twittercorpora, and after pruning standard dictionary forms, the most frequent n formswere manually reviewed and manually translated . When available, dictionarieswere completed using precompiled lists existing in the Web.Emoticon lexicon is a dictionary of regular expression matching a number ofemoticons to their corresponding sentiment in the aforementioned scale.Lastly, stopword lemma lists are used to discard most frequent lemmas whenextracting n-gram features from texts. We adapted this lists to SA requirements byremoving some lemmas, because of their relevance to polarity classification (e.g., no,good, ...). EliXa currently performs tokenization, lemmatization and POS tagging priorto sentiment analysis classification. No entity recognition is applied; entities arematched only if they are defined as keywords. Although EliXa is able to work withcorpora preprocessed with other taggers, its default NLP processing is made bymeans of IXA pipes (Agerri et al., 2014) which is integrated as a library. n varies depending on the size of the input corpus. We reviewed up to 1,500 candidates. esource Use Language eu es en frWord formdictionaries text normalization(e.g. 4ever → forever) 122,085 556,501 67,811 453,037OOVdictionaries text normalization(e.g. 4ever → forever) 63 7,823 223 279Emoticonlexicon Polarity tagging 60 (regexes matching emoji groups)Stopwordlemma lists Polarity taggingfeature extraction 56 46 75 100 Table 1: Resources for text normalization included in EliXa.
EliXa’s core feature is its polarity classifier, which implements a multiclass SupportVector Machine (SVM) algorithm (Hall et al., 2009) combining the information extractedfrom polarity lexicons with linguistic features obtained from the NLP pre-processingstep. Main features include polarity values from general and domain specific polaritylexicons, lemma and POS tag ngrams and positivity and negativity counts based onpolarity lexicons. Features representing other linguistic phenomena such as treatmentof negation, locutions or punctuation marks are also included. Finally, there are somesocial media specific features, such as the proportion of capitalized symbols (whichoften is used to increase the intensity of the message) or emoticon information.EliXa currently provides ready to use polarity classification models, although oneof its strengths is that new models can easily be trained if training data is availablefor a new domain.
Talaia is also capable of providing deeper analysis of the data, by means of userprofiling. Specifically, geolocation, gender detection and user community identificationare implemented.Opinions gathered are geolocated. Geolocation may be done using two differentapproaches: (i) building a census of twitter users on a region or (ii) trying to geolocatethe origin of the users detected. Approach (i) is most suitable when monitoring isdone for a specific region, and high precision is required from geolocation. Detailson this approach are given in section 6.2.Approach (ii) allows Talaia to analyze the differences in opinions with respectto a topic that may arise between regions or countries. Geolocation is done by12xploiting social media information from both messages and authors. If a messageis geolocated, its information is used straightforwardly. Otherwise, user profileinformation is analyzed. The task is challenging, because users do not provide suchinformation always (40%-45% of the users in our datasets have location information),or they define fictitious locations (e.g., ‘Middle earth’, ‘In a galaxy far, far away...’;14%). Several geocoding APIs are queried, and results are then weighted, becauseAPIs show divergent results when feeding fictitious locations. The weighted systemobtains 82% accuracy for those users containing location information in their profile.Roughly, we are able to geolocate correctly around 32% of the users that appear ina monitoring process.Gender detection is another important factor in many social science studies. Asupervised gender classifier is implemented to infer user gender, based on featuresextracted from academic papers (Kokkos & Tzouramanis, 2014; Rangel et al., 2017).User gender detection is based on classifying messages, no user profile information isused.
5. Data Visualization
The GUI has been developed using the Django Web Application framework .This interface provides data analysis visualizations and manages the communicationwith both the crawler and EliXa.Talaia implements a number of visualizations which may be customized dependingon the needs of the specific use case at hand . The main visualizations includepopularity, sympathy and antipathy comparison, evolution of mentions across time,most recent mentions, most widespread mentions, most active users in social mediaand news sources, and most frequent topics. All those visualizations include interactionsthat provide further analysis such as looking at the specific data regarding an specificparty or candidate, or filtering the data according to various criteria such as language,time period, data source or author influence. All graph visualizations are implementedusing d3.js javascript library. OpenStreetMap Nominatim ( https://wiki.openstreetmap.org/wiki/Nominatim ) andGoogle Geocoding API https://developers.google.com/maps/documentation/geocoding/intro ). Existing demos and installations implement different visualizations. Visualizations for the usecases described in this paper can be seen at http://talaia.elhuyar.eus/demo_eae2016 and http://behagune.elhuyar.eus/ https://d3js.org/
6. Success Cases
In this section we present two real use cases where Talaia has been applied,and use them for evaluation purposes. The first one focuses on tracking culturalevents. The second one analyses citizen opinions with respect to political partiesand candidates during an electoral campaign.Both monitoring processes presented here ran on their own dedicated servers.We provide details on hardware specifications and volumes of data processed in thefollowing subsections. As a measure of the performance capabilities of our system,the largest monitoring process we have carried out until now gathered 24M tweets permonth, with an average of 700K tweets processed per day and a maximum of 1.35Mtweets in a single day. Talaia ran on a server with two Intel Xeon 4 core processors(E5530) at 2.4 GHz and 16GB RAM. MySQL databases are locally stored in theserver. The crawler and the text analyzers all ran locally, but no interface wasimplemented in this case.
Talaia was first applied in the Behagunea project. The objective of the projectinvolved tracking the social media impact of cultural events and projects carriedout (more than 500) in the framework of the Donostia European Capital of Culture(DSS2016) year during 2016. The project included monitoring opinions in press andsocial media in four languages: Basque, French and Spanish as coexisting languagesin the different Basque speaking territories and English as international language. Queries retrieving a million results could take up to 7 minutes (depending on the visualizationsrequired)), while using a joint view takes 40 seconds for the same configuration. http://behagune.elhuyar.eus . The crawler, thetext analyzers and the interface all run locally. A total amount of 166K tweets andpress mentions were gathered, with a maximum of 6.6K mentions in a single day.The interface was public and offered real-time results refreshed each 15 minutes. Wecan see from the volume of the data, that this was a low latency monitoring. Evenif there were a lot of events to track, the local nature of most of them explains thelittle impact they have in social media. Talaia was used to track citizen opinions during the electoral Basque electoralcampaign of September 2016. The crawling was carried out during the electioncampaign period, starting on September 8th (23:59pm) and finishing on September23th (23:59pm). It offers useful insights for political analysis such as sympathyrankings, the evolution of the opinions over time, most relevant messages, etc.The system ran on a server with a Intel Xeon 4 core processor (E5530) at 2.4 GHzand 16GB RAM. MySQL databases are locally stored in the server. The crawler,text analyzers and the interface all run locally. A total amount of 4.25M tweetsand press mentions were gathered, with an average of 125K mentions per day, anda maximum of 433K mentions in a single day. The interface was public and offeredreal-time results.The crawler was configured to find mentions talking about the main politicalparties present on the campaign and their respective candidates (only main candidatesmonitored, i.e., those opting to be
Lehendakari , ‘head of the government’).Regarding social media, Twitter was monitored. Since we are talking aboutmonitoring an event happening on a regional scope, two main restrictions wereapplied: only mentions written in Basque and Spanish were crawled, because thoseare the two official languages in the region. The second restriction was to constrainmentions to users from the specific geographical area of the Basque Country. The Especications are 2 vCPUs, 8GB RAM, 100GB EBS storage disk. More information at https://aws.amazon.com/ec2/instance-types/ D geo .(iii) Taking users tagged as Basque citizens from the previous step, we extended ourdataset by retrieving up to the first 5,000 followers and friends from each userusing the twitter following API . We compute the frequency of coocurrencefor each of the candidates , and manually label the most frequent 10,000candidates. Let’s call this dataset D geo + ff − manual .(iv) We train a binary SVM classifier with a linear kernel over D geo + ff − manual .Features of the classifier are the number of followers and friends a user has andthe relative number of followers and friends (with respect to the total numberof follower and friends). The classifier obtains 96% accuracy in a 4-fold crossvalidation.(v) Repeat step (ii) with the users labelled as Basque in D geo + ff − manual , but thistime label the most frequent 20,000 candidates with the classifier trained instep (iv). Our final census consists of 23,195 user ids.As for the news sources, a list of 30 sources was manually compiled, includingTV, printed media and radio stations, all of them with working within the regionalscope.
7. Evaluation
For the evaluation of Talaia, we evaluate the performance of Elixa’s polarityclassifier for the two aforementioned domains. In all cases the L2-loss SVM implementationof the LIBLINEAR (Fan et al., 2008) toolkit was used as classification algorithm https://developer.twitter.com/en/docs/accounts-and-users/follow-search-get-users/api-reference/get-users-show The number of times a candidate appears as a follower or friend of another candidate. C = 0 . ).For the sake of comparison, all the systems presented from here onwards havebeen trained using the following set of features: • > = 2 and document frequency (df) > = 2 . • POS tag 1-gram features. • Polarity lemmas included in language dependent polarity lexicons. Defaultlexicons provided with EliXa were used (see Table 2 for details). • Sentence length. • Upper case ratio: percentage of the capital letters with respect to the totalnumber of characters in a sentence.Microtext normalization features (URL standardization, OOV normalization andemoticon mapping) are applied before extracting the features of each sentence.
Language Lexicon eu ElhP olar eu (San Vicente & Saralegi, 2016) 742 499 1,241es ElhP olar es (Saralegi & San Vicente, 2013a) 3,314 1,903 5.217en EliXa en (San Vicente et al.,2015) 6,123 3,992 10,115fr Feel(Abdaoui et al., 2017) 5.717 8,430 14,147 Table 2: Polarity lexicons used in our experiments.
Table 3 presents the statistics and class distributions of the datasets gathered andannotated in order to build the polarity classifiers for each language in the culturaldomain. All annotations were done manually. Polarity was annotated at mention17evel. Because of the level of specificity reached when defining the keyword taxonomy,we rarely find a mention referring to more than one entity or event. Statistics showthat corpora in all languages have a similar distribution, with a high number ofneutral mentions, and a larger presence of positive opinions than negative ones.
Language Total size eu 2937 931 408 1598es 4754 1487 1303 1964en 12,273 4,654 1,837 5,782fr 11,071 3,459 2,618 4,994
Table 3: Multilingual dataset statistics for the cultural domain.
Table 4 shows the characteristics of the political domain datasets. In this case,each tweet was annotated with respect to a number of entities appearing in the tweet.Annotators were asked to annotate the polarity of a tweet from the perspective ofeach of the entities detected in a tweet, that is, a tweet may contain more thanone polarity annotation. Example 1 shows a real case where a tweet was given twodifferent annotations, one for each entity (negative expressions underlined, positiveones in bold). In fact, the numbers in table 4 give 1.3 and 1.24 average annotationsper tweet for Basque and Spanish, respectively.
Example 1. @pnvgasteiz erabat ados, lotsagarria. Aukera ona aurrera begiratu ta @ehbildu |ren euskara arloko proposamena martxan jartzeko Annotating tweets in the political domain proved to be a rather challenging task.Sarcasm is often present, interpellations to a person are frequent even if they are notthe target of the opinion, an opinion may be present but in an implicit manner, ora third party negative opinion may be expressed towards an entity but the authormay defend it against the expressed opinion. The full annotation guidelines can beconsulted in Annex II.Annotation was carried out in real time during the period of the electoral campaign.We established three shifts a day to annotate messages gathered until then. Threeannotators took part in the process. Because of the limited resources and the volumeof messages crawled daily, each tweet was annotated by a single annotator. English translation: @pnvgasteiz totally agrees, shameful. Good chance to look forward andapply the proposal of @ehbildu in the field of Basque igure 2: Distribution of mentions in Basquewith respect to the political parties. Fromleft to right, parties are sorted accordingthe percentage of negative opinions received,with respect to the total amount of mentionsreceived. Figure 3: Distribution of mentions in Spanishwith respect to the political parties. Fromleft to right, parties are sorted accordingthe percentage of negative opinions received,with respect to the total amount of mentionsreceived. Political domain datasets show very different distributions across languages. Whilethe Basque dataset seems to follow the same pattern seen in the cultural domain,the Spanish dataset has a very high number of negative opinions. Analyzing someresult samples, we realized that there are various phenomena that could explain thisbehaviour. First, much more debate and criticism takes place in Spanish comparedto Basque where the tendency is to write a lot more supportive messages. Thosenot associated with Basque nationalist ideologies mainly communicate in Spanish.A clear example is that the left party EH Bildu has very few negative mentions inBasque (See figure 2). Also the fact that the right wing parties such as
PartidoPopular (PP) and
Ciudadanos (Cs) receive almost no attention in Basque is asymptom of the little engagement they show in this language.Second, left wing people were more active in Twitter (in this specific campaign),with right wing parties concentrating the largest amount of negative mentions (seefigures 3).Lastly, there is the effect of negative campaigning (Skaperdas & Grofman, 1995),which is more pronounced in Spanish, because as we already said there is much moredebate than in Basque. Figure 3 also aligns with studies of negative campaigning inmulti-party scenarios (Walter, 2014; Haselmayer & Jenny, 2017), being the front-runner
Partido Nacionalista Vasco (PNV) and its previous government partner
PartidoSocialista de Euskadi (PSE) those who receive the greatest amount of negativementions.
Table 5 shows the performance of the various multilingual classifiers trained.19 anguage eu 9,418 11,692 3,974 3,185 4,533es 15,550 20,278 3,788 7,601 8,889
Table 4: Multilingual dataset statistics for the political domain.
Reported results are in general higher for the cultural domain, even if the datasetsare smaller in comparison. Basque and Spanish classifier obtain results above 70%.English and French achieve lower results. Positive mentions present the greatestchallenge for English. The main reason for this is the lack of positive trainingexamples. Neutral mentions perform very good in all languages. After analyzinga random sample, we conclude that neutral mentions are of an homogeneous nature,mainly containing agenda events or promotion messages. If we add this to the factthat neutral is the class with the highest number of examples, it seems logical thatour classifiers find highly representative features for this class.
Language
Cultural Domain eu 4,777 74.02 0.658 0.635 0.803es 10,037 73.03 0.683 0.756 0.744en 24,183 70.43 0.715 0.530 0.743fr 23,779 66.17 0.600 0.617 0.721
Political Domain eu 9,394 69.88 0.714 0.702 0.683es 15,751 67.05 0.545 0.693 0.700
Table 5: EliXa polarity classification results.
Regarding the political domain, if we compare Basque and Spanish classifiers,their performance drops around 4% with respect to the results in the cultural domain.Results are not directly comparable, because political domain classifiers are evaluatedover entity level tags. Also, political data is more challenging in terms of the linguisticphenomena used. We have detected a fair amount of messages containing sarcasmor opinion ambiguity towards targets.In the case of Basque, the most sensible drop happens with neutral mentions.After analyzing a random sample we found that they are more heterogeneous thanthose in the cultural domain. They do contain agenda and promotion messages,20ut also many third party statements (candidate x says "...") or messages thatinterpellate parties and candidates over hot topics in the campaign (e.g. “@DanielMaeztu@ehbildu @PodemosEuskadi_ obra gelditzea onuragarria liteke ekonimia arloan 4.000miloi gastatu eta gero?” ). Many neutral messages contain personal opinions notinvolving any of our predefined target entities, even if they are interpellated. Thesephenomena make neutral class harder to represent.Regarding Spanish, performance for positive mentions is significantly lower. Erroranalysis shows that incorrectly classified instances do not fall into a single category(42% negative, 58% neutral). Analysing the errors, we find two main reasons. First,many positive mentions are incorrectly classified as negative because their contentis mainly negative (e.g. “ @AgirreGarita La diferencia es clara, PNV apoyando eldesahucio y EHBILDU al desahuciado. NO SEAS COMPLICE, no votes a quiendesahucia.” ). Our strategy for assigning message level polarity to all entitiesinvolved in a mention is prone to this type of errors. Second, as we saw for Basque,neutral mentions also contain polar expressions or opinions, making it harder todistinguish them from actual positive or negative messages.
8. Conclusion and Future Work
We have presented Talaia, a real time monitor of social media and digital press.Talaia is able to extract information related to an specific topic and analyse itby means of natural language processing technologies. Two success cases and theresources generated from those cases have been described. In that sense, we haveshown the ability to adapt our system to different domains and languages.Talaia is still under development. The short term objectives include work onoptimizing the information extraction process. Specifically, extracting keywords fromthe data downloaded up to a certain point would allow us automatically adapt thesystem to new terms, without losing information because the keyword hierarchy isoutdated or the topic is poorly defined.Another important point is the adaptation of our sentiment analysis model to newdomains. In that sense experiments are being carried out in order to minimize thedomain adaptation process, both in terms of data collection and annotation effort. English translation: @DanielMaeztu @ehbildu @PodemosEuskadi_ stopping the constructionwould be beneficial after spending 4,000 millions? English translation: @AgirreGarita The difference is that PNV is in favour of evictions andEHBILDU is with the evicted ones. DO NOT BE AND ACCOMPLICE, don’t vote to those whopractice evictions.
Acknowledgements
This work has been supported by the following projects: Elkarola project (Elkartekgrant No. IE-14-382), and Tuner project (MINECO/FEDER grant No. TIN2015-65308-C5-1-R).22 eferences
Abdaoui, A., Azé, J., Bringay, S., & Poncelet, P. (2017). Feel: a french expandedemotion lexicon.
Language Resources and Evaluation , , 833–855.Abel, F., Hauff, C., Houben, G.-J., Stronkman, R., & Tao, K. (2012). Twitcident:fighting fire with information from social web streams. In Proceedings of the 21stInternational Conference on World Wide Web (pp. 305–308). ACM.Abilhoa, W. D., & De Castro, L. N. (2014). A keyword extraction method fromtwitter messages represented as graphs.
Applied Mathematics and Computation , , 308–325.Agerri, R., Bermudez, J., & Rigau, G. (2014). Ixa pipeline: Efficient and readyto use multilingual nlp tools. In Proceedings of the 9th Language Resources andEvaluation Conference (LREC2014) (pp. 26–31). Reykjavik, Iceland.Ahmed, W., Bath, P. A., Sbaffi, L., & Demartini, G. (2018). Measuring the effect ofpublic health campaigns on twitter: The case of world autism awareness day. In
International Conference on Information (pp. 10–16). Springer.Aiello, L. M., Petkos, G., Martin, C., Corney, D., Papadopoulos, S., Skraba, R.,Göker, A., Kompatsiaris, I., & Jaimes, A. (2013). Sensing trending topics intwitter.
IEEE Transactions on Multimedia , , 1268–1282.Alegria, I., Aranberri, N., Comas, P. R., Fresno, V., Gamallo, P., Padró, L.,San Vicente, I., Turmo, J., & Zubiaga, A. (2014). Tweetnorm_es corpus: anannotated corpus for spanish microtext normalization. In Proceedings of theLanguage Resources and Evaluation Conference .Alegria, I., Aranberri, N., Comas, P. R., Fresno, V., Gamallo, P., Padró, L.,San Vicente, I., Turmo, J., & Zubiaga, A. (2015). Tweetnorm: a benchmarkfor lexical normalization of spanish tweets.
Language Resources and Evaluation , , 883–905.Allan, J., Carbonell, J., Doddington, G., Yamron, J., Yang, Y. et al. (1998). Topicdetection and tracking pilot study: Final report. In Proceedings of the DARPAbroadcast news transcription and understanding workshop (pp. 194–218). Citeseervolume 1998. 23arbosa, L., & Feng, J. (2010). Robust sentiment detection on twitter frombiased and noisy data. In
Proceedings of the 23rd International Conference onComputational Linguistics: Posters
COLING ’10 (pp. 36–44). Stroudsburg, PA,USA.Belainine, B., Fonseca, A., & Sadat, F. (2016). Named entity recognition and hashtagdecomposition to improve the classification of tweets. In
Proceedings of the 2ndWorkshop on Noisy User-generated Text (WNUT) (pp. 102–111).Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation.
Journalof machine Learning research , , 993–1022.Bollen, J., Mao, H., & Zeng, X.-J. (2010). Twitter mood predicts the stock market. , .Brody, S., & Diakopoulos, N. (2011). Cooooooooooooooollllllllllllll!!!!!!!!!!!: usingword lengthening to detect sentiment in microblogs. In Proceedings of theConference on Empirical Methods in Natural Language Processing
EMNLP ’11(pp. 562–570).Brun, C., & Roux, C. (2014). Decomposing hashtags to improve tweetpolarity classification (décomposition des «hash tags» pour l’amélioration de laclassification en polarité des «tweets»)[in french]. In
Proceedings of TALN 2014(Volume 2: Short Papers) (pp. 473–478).Ceron, A., Curini, L., & Iacus, S. M. (2015). Using sentiment analysis to monitorelectoral campaigns: Method matters—evidence from the united states and italy.
Social Science Computer Review , , 3–20.Chen, H., Chiang, R. H., & Storey, V. C. (2012). Business intelligence and analytics:from big data to big impact. MIS quarterly , (pp. 1165–1188).Cliche, M. (2017). Bb_twtr at semeval-2017 task 4: Twitter sentiment analysis withcnns and lstms. In
Proceedings of the 11th International Workshop on SemanticEvaluation (SemEval-2017) (pp. 573–580).Dadvar, M., Trieschnigg, D., Ordelman, R., & de Jong, F. (2013). Improvingcyberbullying detection with user context. In
European Conference on InformationRetrieval (pp. 693–696). Springer. 24ai, A. M., & Le, Q. V. (2015). Semi-supervised sequence learning. In C. Cortes,N. D. Lawrence, D. D. Lee, M. Sugiyama, & R. Garnett (Eds.),
Advances in NeuralInformation Processing Systems 28 (pp. 3079–3087).Deriu, J., Lucchi, A., De Luca, V., Severyn, A., Müller, S., Cieliebak, M., Hofmann,T., & Jaggi, M. (2017). Leveraging large amounts of weakly supervised data formulti-language sentiment classification. In
Proceedings of the 26th InternationalConference on World Wide Web
WWW ’17 (pp. 1045–1052). Republic and Cantonof Geneva, Switzerland.Fan, R.-E., Chang, K.-W., Hsieh, C.-J., Wang, X.-R., & Lin, C.-J. (2008). Liblinear:A library for large linear classification.
Journal of machine learning research , ,1871–1874.Foster, J., Cetinoglu, O., Wagner, J., Le Roux, J., Hogan, S., Nivre, J., Hogan, D., &van Genabith, J. (2011). Workshops at the Twenty-Fifth AAAI Conference on Artificial Intelligence .Go, A., Bhayani, R., & Huang, L. (2009). Twitter sentiment classification usingdistant supervision.
CS224N Project Report, Stanford , (pp. 1–12).Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H.(2009). The WEKA data mining software: an update.
SIGKDD Explor. Newsl. , , 10–18.Han, B., & Baldwin, T. (2011). Lexical normalisation of short text messages: Maknsens a Proceedings of the 49th Annual Meeting of the Associationfor Computational Linguistics: Human Language Technologies (pp. 368–378).volume 1.Haselmayer, M., & Jenny, M. (2017). Sentiment analysis of political communication:combining a dictionary approach with crowdcoding.
Quality & quantity , ,2623–2646.He, W., Zha, S., & Li, L. (2013). Social media competitive analysis and textmining: A case study in the pizza industry. International Journal of InformationManagement , , 464–472.Howard, J., & Ruder, S. (2018). Universal language model fine-tuning for textclassification. In Proceedings of the 56th Annual Meeting of the Association forComputational Linguistics (Volume 1: Long Papers) (pp. 328–339).25u, M., & Liu, B. (2004). Mining and summarizing customer reviews. In
Proceedingsof the tenth ACM SIGKDD international conference on Knowledge discovery anddata mining (pp. 168–177).Johnson, R., & Zhang, T. (2016). Supervised and semi-supervised text categorizationusing lstm for region embeddings. In
International Conference on MachineLearning (pp. 526–534).Jurgens, D., Finethy, T., McCorriston, J., Xu, Y. T., & Ruths, D. (2015).Geolocation prediction in twitter using social networks: A critical analysis andreview of current practice.
ICWSM , , 188–197.Kim, S., Banchs, R., & Li, H. (2016). Exploring convolutional and recurrent neuralnetworks in sequential labelling for dialogue topic tracking. In Proceedings of the54th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers) (pp. 963–973). volume 1.Kokkos, A., & Tzouramanis, T. (2014). A robust gender inference model for onlinesocial networks and its application to linkedin and twitter.
First Monday , .Kouloumpis, E., Wilson, T., & Moore, J. D. (2011). Twitter sentiment analysis: Thegood the bad and the OMG! In Fifth International AAAI Conference on Weblogsand Social Media .Liu, F., Weng, F., & Jiang, X. (2012). A broad-coverage normalization systemfor social media language. In
Proceedings of the 50th Annual Meeting ofthe Association for Computational Linguistics (Volume 1: Long Papers) (pp.1035–1044). Jeju Island, Korea.Liu, X., Zhang, S., Wei, F., & Zhou, M. (2011). Recognizing named entities in tweets.In
Proceedings of the 49th Annual Meeting of the Association for ComputationalLinguistics (ACL) (pp. 359–367).Miao, Z., Chen, K., Fang, Y., He, J., Zhou, Y., Zhang, W., & Zha, H.(2017). Cost-effective online trending topic detection and popularity predictionin microblogging.
ACM Transactions on Information Systems (TOIS) , , 18.Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributedrepresentations of words and phrases and their compositionality. In Advances inNeural Information Processing Systems (pp. 3111–3119).26ohammad, S., Kiritchenko, S., & Zhu, X. (2013). NRC-canada: Building thestate-of-the-art in sentiment analysis of tweets. In
Second Joint Conference onLexical and Computational Semantics (*SEM), Volume 2: Proceedings of theSeventh International Workshop on Semantic Evaluation (SemEval 2013) (pp.321–327).Mostafa, M. M. (2013). More than words: Social networks’ text mining for consumerbrand sentiments.
Expert Systems with Applications , , 4241–4251.Nakov, P., Ritter, A., Rosenthal, S., Sebastiani, F., & Stoyanov, V. (2016).Semeval-2016 task 4: Sentiment analysis in twitter. In Proceedings of the 10thInternational Workshop on Semantic Evaluation (SemEval-2016) (pp. 1–18).Nguyen, K.-L., Shin, B.-J., & Yoo, S. J. (2016). Hot topic detection andtechnology trend tracking for patents utilizing term frequency and proportionaldocument frequency and semantic information. In
Big Data and Smart Computing(BigComp), 2016 International Conference on (pp. 223–230). IEEE.O’Connor, B., Balasubramanyan, R., Routledge, B. R., & Smith, N. A. (2010). Fromtweets to polls: Linking text sentiment to public opinion time series. In
FourthInternational AAAI Conference on Weblogs and Social Media .Oliveira, N., Cortez, P., & Areal, N. (2017). The impact of microblogging data forstock market prediction: using twitter to predict returns, volatility, trading volumeand survey sentiment indices.
Expert Systems with Applications , , 125–144.Osborne, M., Moran, S., McCreadie, R., Von Lunen, A., Sykora, M., Cano, E.,Ireson, N., Macdonald, C., Ounis, I., He, Y. et al. (2014). Real-time detection,tracking, and monitoring of automatically discovered events in social media. ACL2014 , (p. 37).Öztürk, N., & Ayvaz, S. (2018). Sentiment analysis on twitter: A text miningapproach to the syrian refugee crisis.
Telematics and Informatics , , 136–147.Pontiki, M., Galanis, D., Papageorgiou, H., Manandhar, S., & Androutsopoulos, I.(2015). Semeval-2015 task 12: Aspect based sentiment analysis. In Proceedingsof the 9th International Workshop on Semantic Evaluation (SemEval 2015) (pp.486–495).Pontiki, M., Galanis, D., Pavlopoulos, J., Papageorgiou, H., Androutsopoulos, I., &Manandhar, S. (2014). Semeval-2014 task 4: Aspect based sentiment analysis. In
Proceedings of the International Workshop on Semantic Evaluation (SemEval) .27ope, D., & Griffith, J. (2016). An analysis of online twitter sentiment surroundingthe european refugee crisis. In
Proceedings of the International Joint Conference onKnowledge Discovery, Knowledge Engineering and Knowledge Management
IC3K2016 (pp. 299–306). Portugal.Preoţiuc-Pietro, D., & Cohn, T. (2013). A temporal model of text periodicities usinggaussian processes. In
Proceedings of the 2013 Conference on Empirical Methodsin Natural Language Processing (pp. 977–988).Rangel, F., Rosso, P., Potthast, M., & Stein, B. (2017). Overview of the 5th authorprofiling task at pan 2017: Gender and language variety identification in twitter.
Working Notes Papers of the CLEF , .Román, J. V., Cámara, E. M., Morera, J. G., & Zafra, S. M. J. (2015). Tass 2014-thechallenge of aspect-based sentiment analysis.
Procesamiento del Lenguaje Natural , , 61–68.Rosenthal, S., Farra, N., & Nakov, P. (2017). Semeval-2017 task 4: Sentimentanalysis in twitter. In Proceedings of the 11th International Workshop on SemanticEvaluation (SemEval-2017) (pp. 502–518).Rosenthal, S., Nakov, P., Ritter, A., & Stoyanov, V. (2014). Semeval-2014 task 9:Sentiment analysis in twitter. In
Proceedings of the 8th International Workshopon Semantic Evaluation, SemEval . volume 14.San Vicente, I., & Saralegi, X. (2016). Polarity lexicon building: to what extent isthe manual effort worth? In
Proceedings of the Tenth International Conference onLanguage Resources and Evaluation (LREC 2016) .San Vicente, I., Saralegi, X., & Agerri, R. (2015). Elixa: A modular and flexibleabsa platform. In
Proceedings of the 9th International Workshop on SemanticEvaluation (SemEval 2015) (pp. 748–752).Saralegi, X., & San Vicente, I. (2012). Tass: Detecting sentiments in spanish tweets.In
Proceedings of the TASS Workshop at SEPLN .Saralegi, X., & San Vicente, I. (2013a). Elhuyar at tass 2013. In
Proceedings of theTASS 2013 Workshop at SEPLN .Saralegi, X., & San Vicente, I. (2013b). Elhuyar at tweetnorm 2013. In
Proceedingsof the TweetNorm Workshop at SEPLN .28everyn, A., & Moschitti, A. (2015). Unitn: Training deep convolutional neuralnetwork for twitter sentiment classification. In
Proceedings of the 9th InternationalWorkshop on Semantic Evaluation (SemEval 2015) (pp. 464–469).Shaikh, S., Feldman, L. B., Barach, E., & Marzouki, Y. (2017). Tweet sentimentanalysis with pronoun choice reveals online community dynamics in response tocrisis events. In
Advances in cross-cultural decision making (pp. 345–356).Shamma, D. A., Kennedy, L., & Churchill, E. F. (2011). Peaks and persistence:modeling the shape of microblog conversations. In
Proceedings of the ACM 2011conference on Computer supported cooperative work (pp. 355–358). ACM.Skaperdas, S., & Grofman, B. (1995). Modeling negative campaigning.
AmericanPolitical Science Review , , 49–61.Somasundaran, S., Namata, G., Wiebe, J., & Getoor, L. (2009). Supervised andunsupervised methods in employing discourse relations for improving opinionpolarity classification. In Proceedings of the 2009 Conference on EmpiricalMethods in Natural Language Processing: Volume 1 -
EMNLP ’09 (pp. 170–179).Stroudsburg, PA, USA.Sutton, J. N. (2009). Social media monitoring and the democratic nationalconvention: New tasks and emergent processes.
Journal of Homeland Securityand Emergency Management , .Taboada, M., Brooke, J., Tofiloski, M., Voll, K., & Stede, M. (2011). Lexicon-basedmethods for sentiment analysis. Computational linguistics , , 267–307.Thelwall, M. (2017). The heart and soul of the web? sentiment strength detectionin the social web with sentistrength. In Cyberemotions (pp. 119–134).Walter, A. S. (2014). Choosing the enemy: Attack behaviour in a multiparty system.
Party Politics , , 311–323.Xu, J.-M., Zhu, X., & Bellmore, A. (2012). Fast learning for sentiment analysison bullying. In Proceedings of the First International Workshop on Issues ofSentiment Discovery and Opinion Mining
WISDOM ’12 (pp. 10:1–10:6). NewYork, NY, USA: ACM. URL: http://doi.acm.org/10.1145/2346676.2346686 .doi: . 29u, Y., & Wang, X. (2015). World cup 2014 in the twitter world: A big dataanalysis of sentiments in us sports fans’ tweets.
Computers in Human Behavior , , 392–400.Zubiaga, A., Voss, A., Procter, R., Liakata, M., Wang, B., & Tsakalidis, A. (2017).Towards real-time, country-level location classification of worldwide tweets. IEEETransactions on Knowledge and Data Engineering , , 2053–2066.30 ppendix A. Comparative of commercial Social Media MonitorsPlatform Data Sources Crawling Data Processing Search Navigation Iconoce Digital press, blogs,videos, social media(Facebook, Twitter,Linked-in?) Personalized,subject toagreement no Personalized archive3 separate searchengines (mentions,comment, authors)no lemmatization -no crosslingual. Graphs(aggregations?),salient terms,salient topics,Influencers,alerts, reportsINNGUMA Rss multimedia, DeepWeb, Twitter, Facebook,Linkedin, possibilityto include externaldocumentation manually Custom filters,not clear iffiltering isdone at apost-crawlingstage MT, No mention of textprocessing. No SA Semantic search(techniquesnot especified).Index cardsand documents.Information istagged manually. Reports, contentcreation,social mediamanagement.MultilingualGUI.MeaningCloud Digital news, blogs,Twitter, satisfactionsurveys (customerprovided), phone surveytranscriptions, 5 languages (Es, En,Fr, Pt, It). Languageidentification, Clusteringfor topic detection.Lemmatization, postagging, parsing, NERC,GATE APISA: Ruled-based.Sentiment Lexicons+ rules. Irony andsubjectivity detection.Entity polarity detectedusing manually compileddictionaries. no No Dashboard,visualizations ordata aggregatios.Excel plugin orAPI accessnap-trends Social Media (Twitter,Facebook, Instagram,Google+,...) MT from 80 languages.Propietary linguisticprocessing. Topic (trends)detection.Propietary sentimentanalysis. Geolocation basedsearch engine,mutiple criteria:social network,search terms,geolocation.Previous searchfeature. Agreggations,interactivevisualization,temporal trends.Websays News, Blogs/RSS,Forums, Facebook,Twitter, Google+,LinkedIn, Instagram,Foursquare, Pinterest,Youtube, Vimeo, Reviews(Tripadvisor, Booking,...) Keyword based,accepts alsonegativekeywords. Multilingual dataprocessing, no specificdata about the coverageSA: AI (ML) + humanvalidation Multiple searchcriteria, filter-based. Graphs,salient terms,trending topics,influencers,sentiment,trends. Alertsand reports.Lynguo Facebook, Twitter,Instagram, YouTube,online media, blogs andforums. Keyword based,accepts alsonegativekeywords andaccounts. Es,En.SA: ML + Lexicons+ rules. Polarity andemotions. Aspect basedSA Customizabledashboard.Several defaultaggregations andpossibility togenerate customvisualizations.Alerts andperiodicalreportsKeyhole Twitter, Instagram, websources. Social mediaand web sourcesare configuredand monitoredseparately.Keywords,users. 13 languages.SA: Polarity Influencers,timeline, trends,sentiment,aggregations.ber-metrics Blogs, forums,academic/scientificjournals, digital press,Instagram, Tumblr,Google+, Facebook,Twitter, YouTube, Vimeo,Flickr, and Foursquare.With Ubermetrics you caneven capture commentsfrom YouTube, Facebook,and major online newssources. TV/Radio Customizable"searchagents".Keyword based 40 languages. Propietarydata processing. Detailed searchbased on multiplecriteria includedin visualizationdashboard Dashboard,alerts, reports.Table A.6: Comparison of commercial social mediamonitoring platforms. ppendix B. Polarity annotation guidelines