ALONE: A Dataset for Toxic Behavior among Adolescents on Twitter
Thilini Wijesiriwardene, Hale Inan, Ugur Kursuncu, Manas Gaur, Valerie L. Shalin, Krishnaprasad Thirunarayan, Amit Sheth, I. Budak Arpinar
AALONE: A Dataset for Toxic Behavior amongAdolescents on Twitter
Thilini Wijesiriwardene Hale Inan Ugur Kursuncu Manas Gaur Valerie L. Shalin Krishnaprasad Thirunarayan AmitSheth I. Budak Arpinar AI Institute, University of South Carolina [email protected], [email protected], [email protected], [email protected] Department of Psychology, Wright State University [email protected] Department of Computer Science and Engineering, Wright State University [email protected] Department of Computer Science, University of Georgia [email protected], [email protected] * Equally contributed.
Abstract.
The convenience of social media has also enabled its mis-use, potentially resulting in toxic behavior. Nearly 66% of internet usershave observed online harassment, and 41% claim personal experience,with 18% facing severe forms of online harassment. This toxic commu-nication has a significant impact on the well-being of young individuals,affecting mental health and, in some cases, resulting in suicide. Thesecommunications exhibit complex linguistic and contextual characteris-tics, making recognition of such narratives challenging. In this paper, weprovide a multimodal dataset of toxic social media interactions betweenconfirmed high school students, called ALONE (AdoLescents ON twit-tEr), along with descriptive explanation. Each instance of interactionincludes tweets, images, emoji and related metadata. Our observationsshow that individual tweets do not provide sufficient evidence for toxicbehavior, and meaningful use of context in interactions can enable high-lighting or exonerating tweets with purported toxicity.
Keywords:
Toxicity · Harassment · Social Media · Resource · Dataset
The language of social media is a socio-cultural product, reflecting issues ofrelevance to the sample population and evolving norms in the exchange of coarselanguage and acceptable sarcasm, employing toxic, questionable language, andsometimes constituting actual harassment. According to a 2017 Pew ResearchCenter survey, 41% of U.S. adults claim to have experienced some type of onlineharassment, offensive name-calling, purposeful embarrassment, physical threats,harassment over a sustained period of time, sexual harassment or stalking . a r X i v : . [ c s . S I] A ug Wijesiriwardene, Inan and Kursuncu et al.
Toxic behavior is prevalent among adolescents, sometimes leading to aggres-sion [26,27]. Adolescents exemplify a population that is particularly vulnerableto disturbing social media interactions [47], and this behavior is observable ina network of high school students [5]. Further, a toxic online environment maycause mental health problems for this population [2,40,20,48]. While a victimmay experience a negative reaction from a toxic environment of offensive lan-guage, this differs from targeted toxicity which is usually directed whose contentcollected and confirmed with a unique method towards one individual. The anal-ysis of single tweets or individual users is potentially misleading as the contextof interactions between the two people (e.g., source and target) dictates the de-termination of toxicity. In other words, two individuals who are friends may usecoarse keywords or language that is seemingly toxic, but it may be sarcastic,exonerating them from toxicity.In this paper, we provide a dataset and its details, specific to toxic behaviorin social media communications. This dataset has two particular contributions:(i) the population is high school students whose content was collected and con-firmed with a unique method, and (ii) it was designed based on the interactions between participants. The detection of true toxic behavior against a persistingbackground of coarse language poses a challenging task. Moreover, the scope ofthe original crawl has great bearing on the prevalence of toxicity features andthe criteria for toxic behavior itself. To address these issues, we have assembleda social media corpus from Twitter for a sample of midwestern American HighSchool Students. We assert a dyadic, directed interaction, between a source anda target. Existing related datasets (see Related Work section) focus mainly onthe user or tweet level for the task of detecting toxic content. Such datasets failto capture adequately the fundamental and contextual nuances in the languageof these conversations. Thus, our corpus preserves and aggregates the social me-dia interaction history between participants. This enables the determination ofexisting friendship and hence possible sarcasm. Because individuals can commu-nicate with multiple partners, we have the potential of detecting unique toxicperson-victim pairings that would be otherwise undetectable in the raw originalcrawl.Each entry in our dataset consists of 12 fields: Interaction Id, Count, SourceUser Id, Target User Id, Emoji, Emoji Keywords, Tweets, Image Keywords,created at, favorite count, in reply to screenname and label where the
Tweets fieldcontains an aggregation of the tweets between a specific pair of source and target.For preliminary analysis, we define a single dimension of toxic language , peggedat one end by benign content and the other by harassment. This dimensioncan be partitioned into several, partially overlapping classes, determined by adecision rule. We have identified and experimented with three levels of toxicinteractions between source and target:
Toxic (T), Non-Toxic (N), or Unclear (U) . However, the boundaries between levels are discretionary, accommodatingconstruct definitions that are, at best, debatable.We include examples across the continuum of toxic language, with sufficientcontext to determine the nature of toxicity. We detect true toxicity on Twitterby analyzing interactions among a collection of tweets, in contrast with priorapproaches where the main focus is performing user or tweet level analysis.Further, we assert that detecting a user as a toxic person with respect to onevictim does not provide evidence of being a universal toxic person because theycan be friendly to a majority of others. We reviewed prior work for the variety of overlapping constructs related to toxicexchanges. The social media literature related to toxic behavior lacks crisp dis-tinctions between: offensive language [14,19,37], hate speech [14,10,4,50], abusivelanguage [14,33,31] and cyberbullying [8,18,11]. For example, the following defi-nition of offensive language substantially overlaps with the subsequent definitionof hate speech. According to [14], offensive language is profanity, strongly im-polite, rude or vulgar language expressed with fighting or hurtful words in orderto insult a targeted individual or group. Hate speech is language used to expresshatred towards a targeted individual or group, or is intended to be derogatory, tohumiliate, or to insult the members of a group, on the basis of attributes such asrace, religion, ethnic origin, sexual orientation, disability, or gender. [42] clas-sifies swearing, aggressive comments, or mentioning the past political or ethnicconflicts in a non-constructive and harmful way as hateful: @user name nope youjust a stupid hoe who wouldn’t know their place comprises both offensiveand hate speech. Specifically, the challenge lies in operationalizing the contextualdifferences between offensiveness, hate speech and harassment. As the existingwork on offensive content, harassment and hate speech fails to take into accountthe nature of the relationship between participants, we focus our attention onthe context-aware analyses of targeted exchanges.
Offensive [4] annotated 16K tweets from [52] with the labels, racist, sexist orneither. 3383 and 1972 tweets were sexist and racist respectively, and others werelabeled as neither. In [31], their aim was to detect abusive language on onlineuser comments posted on Yahoo. 56,280 comments were labeled as Abusive and895,546 comments as Clean.
Hate Speech [44] developed a dataset to identify the main targets of on-line hate speech including the nine categories such as race, behavior, physical,sexual orientation, class, gender, ethnicity, disability, religion, and other for non-classified hate targets. 178 most popular targets from Whisper and Twitter weremanually labeled, unveiling new forms of online hate that can be harmful topeople. [10] focused on distinguishing hate speech from other forms of offensivelanguage. They extracted 85.4 million tweets from 33,458 users, and randomlysampled 25K tweets containing words from a hate speech lexicon. Individualtweets were labeled as hate speech, offensive or neither. [43] presented an anno-tated corpus of tweets classified by different levels of hate to provide an onto-
Wijesiriwardene, Inan and Kursuncu et al. logical classification model to identify harmful speech. They randomly sampled14,906 tweets and developed a supervised system used for detection of the classof harmful speech. In [52], tweets were sampled from the 130K tweets, and inaddition to racism, sexism, and neither, the label both was added. A character n-gram based approach provided better performance for hate speech detection. [51]examined the influence of annotators’ knowledge for hate speech on classifica-tion models, labeling individual tweets. Considering only cases of full agreementamong amateur annotators, they found that amateur annotators can producerelatively good annotations as compared to expert annotators.
Harassment
A number of researchers have attempted to identify dimensionsor factors underpinning harassment. [29] drew on the model [6] that conceptual-ized aggression on four dimensions: verbal, physical, direct-indirect, and active-passive . [38] analyses the linguistics aspects of harassment based on differentharassment types. Consistent with our interest in interaction history betweenparticipants, cyberbullying emphasizes the repetitiveness of aggressive acts [35].The harasser may target a victim over a period of time, or a group of harassersmay target a victim about the same demeaning characteristic or incident. Apartfrom repetitiveness, the difference of power between the harasser and victimsuggests cyberbullying. However, this work [35] is not computationally oriented.Golbeck [16] introduced a large, human labeled corpus of online harassment dataincluding 35,000 tweets with 5495 non-harassing and 29505 harassing examples.In contrast to this literature, our approach to the problem is to focus on in-teractions between participants to capture the context of the relationship ratherthan solely tweets or users. As online toxic behavior is a complex issue thatinvolves different contexts and dimensions [22,21,1], tweet-level or user-level ap-proaches do not adequately capture the context with important nuances due tothe fluidity in the language. Our interaction-based dataset will enable researchersto uncover critical patterns for gaining a better understanding of toxic behavioron social media. Additionally, our dataset is unique in its focus on high schoolstudent demographic.
For the dataset ALONE, we retrieved 469,786 tweets from our raw Twitter data,and used a harassment lexicon provided by [39] to filter tweets that are likely tocontain toxic behavior, obtaining a collection of 688 interactions with aggregated16,901 tweets.
We focused on tweets as the source for our dataset because of its public access.Besides text, tweets can contain images, emoji and URLs as additional content.To create a ground truth dataset, we reviewed public lists of students, suchas the list of National Merit Scholars published in newspapers, identifying 143names of the attendees of a high school. Using the list of identified individuals,we searched Twitter for the profiles associated with these students using Twitter
LONE: A Dataset for Toxic Behavior among Adolescents on Twitter 5
APIs. Then, with the guidance of our cognitive scientist co-author, we confirmedthat the users that we retrieved were high school students, through their profilesand tweets conversing on their school mascot, clubs or faculty members. The 143user profiles with their tweets constituted the seed corpus.
Dataset Expansion:
As a typical network of high school students is larger than143 users, we expanded the network using the friend and follower relationships.We followed the following procedure: – Collect friends and followers lists for each seed profile. – Exclude non-student accounts: We identified the accounts follow-ing each other considering them as candidate students, and removedaccounts that are not both following and being followed by the ac-counts in the friends and followers lists of seed accounts (not commonprofiles). As the adults, such as teachers, would notice any toxic be-havior, such as harassment, bullying or aggression, which may haveconsequences, students with potentially toxic behavior would avoidfollowing their social media accounts [46,30] to sequester social net-work behavior [28,30]. We obtained 8805 accounts that follow andare being followed by at least one seed account, as candidates for stu-dent accounts in the high school. We removed 80 accounts as theywere suspended or deleted or otherwise protected by account owners. – Retain only the peer profiles that follow and are being followed bymore than 10% of the seed profiles, yielding 320 likely peers. Toconfirm the absence of false positives, 50 accounts out of the 320likely peers were randomly selected and manually validated that allthe 50 were confirmed student accounts. When tweets of the newlyadded 320 accounts were crawled, seven accounts were deleted orrestricted. Hence, we removed them from the dataset, resulting in456 accounts (143 seed and 313 added).After we finalized the 456 accounts, tweets (up to 3200 if available) were collectedfor each, starting from the most recent (May 2018), along with their accountmetadata, using the Twitter API.
Interaction-based Dataset:
As our toxic behavior construct requires interac-tions between participants, we pruned the tweet corpus to retain a dataset thatconsists of interactions. We define an interaction as a collection of tweets ex-changed between the two participants (e.g., source and target) in one direction,and on Twitter, we consider mentions (including replies) and retweets as interac-tions. For instance, one user may mention another user in a tweet for harassing,bullying or insulting. Moreover, retweeting a harassing tweet potentially boostspopularity, which creates the role of bystander for the source, suggesting that theretweeting user (source) is actually supporting or helping the harasser (target).We have left retweet indicators (e.g., RT @username:) in the data. Further, sometweets are included in multiple interactions; hence, these communications are apart of a group communication that is not dyadic. For some instances, source
Wijesiriwardene, Inan and Kursuncu et al. and target are the same users, and we left these conversations in the dataset asthey may be likely a part of group aggression.We aggregated tweets that qualify as interactions between users, potentiallyreducing the false alarm rate of an analysis solely based on the presence of char-acteristics of offensive language [3]. This allows for the detection of a particularlyintriguing combination of positive and negative sentiment lexical items, sugges-tive of sarcasm, e.g., happy birthday @user name love you but hate your feet and
Happy birthday ugly!! . The presence of Happy Birthday or posi-tive emoji (see above) alters the interpretation of content that would otherwisebe regarded as potentially suggestive of toxic behavior and the phenomenon ofconflicting valence exoneration content, assuming that the toxic content is sar-castic, e.g., the source does not really believe the recipient has unattractive feetor is generally ugly. Moreover, contextual analysis reveals that some of these arenot truly toxic. Prior tweets in an interaction provides exonerating context, byindicating the presence of friendship, thus correcting the false positive. Design-ing the dataset based on interactions captures the context of the relationshipbetween the two user; thus, enabling one to employ computational techniquesto retrieve meaningful information concerning true toxicity.A portion of the tweets does not include any interaction indicator, but theyrefer to a person indirectly without mentioning or writing the name with mali-cious intent, to avoid the authority figures. This is called
Subtweeting [36,13,9].Adolescents have specifically developed such practice due to their own privacyconcerns and parental intrusion. For each user, we aggregated the tweets thatdo not mention the target explicitly, and indicated the target as “None .Then, a harassment lexicon [39] was utilized to filter the interactions thatpotentially contain toxic content. For online harassment, source and target dyadscan be considered as harasser-victim or bystander-harasser . Further, as capturingcontext to determine the toxicity in the content is critical, an interaction shouldinclude a sufficient number of tweets. Therefore, we set an empirical thresholdfor one interaction as having at least three tweets, to capture context.We have fully de-identified the interactions by replacing; (i) Twitter user-names and mentions in tweets with a numeric user id, (ii) URLs with the tokenof < url > , and (iii) person names with the token of < name > . We have alsoincluded the following metadata for each tweet in the interactions: timestamp,favorite counts, and the de-identified user id of the replied user (if the tweets is areply). Thus, researchers will have the ability to study a variety of aspects of thisproblem such as time series analysis. The finalized dataset includes 688 interac-tions with 16,901 tweets. The fields in an instance are as follows: Interaction Id,Count, Source User Id, Target User Id, Label, Emoji, Emoji Keywords, Tweets,Image Keywords, Timestamp, in reply to and favorite count. “Count field holdsinformation for the number of tweets in an interaction. “Source and Target UserId fields hold numeric identification (after de-identification) information. A “La- http://bolobhi.org/abuse-subtweeting-tweet-school-cyber-bullying/LONE: A Dataset for Toxic Behavior among Adolescents on Twitter 7 bel field holds the assigned label (T,N,U) for the interaction. While the “Emojifield holds the emoji being used in the tweets, “Emoji Keywords field providesthe keywords that explain the meaning of the emoji, retrieved from EmojiNet[53]. The “Tweets field has the tweets, and the following fields holds the meta-data for each tweet: (i) Timestamp: time information of a tweet, (ii) in replyto: (non-real) user id of the target if the tweet is a reply, (iii) favorite count:number of favorites. See Table 1 for example interactions from the dataset withfour fields. Label Tweets
T if you gon say n. this much, the LEAST you could do is hit the tanning bed < url > *** you’re f... the most hideous and racist piece of s... *** YOU ARELITERALLY F... RACIST SHUT THE F UP *** yeah you’re not racistat all !!!!!!!! *** are you in f... politics no, you’re like 17 s... the f... up and stopputting your ¨facts¨onT ight f... you again *** nah f... all of you frfr bunch of f... f... *** f... you < url > you have no room to be talking s... shut your bum a.. up frfr **you’re halarious, f... you and everyone that favorited that and retweeted thatN “Kix is the handjob of cereals”- John Doe < imageurl > *** Explain tothat i.. that doing it spreads the word and the chance of someone donatingXD fedora wearing as *** get the f... off my twitter b. BOI *** guys follow bche’s an i. and forgot his password.U This tweet was dumb I agree with u this time *** hahaha I’m so dumb ***that’s my mom f.... *** boob *** never seen a bigger lie on the Internet thenthis one right hereTable 1: Examples from the dataset with labels Toxic (T), Non-Toxic (N) and Unclear(U). The expletives were replaced with the first letter followed by as many dots asthere are remaining letters.
Multimodality:
As it will be described in Section Descriptive Statistics, dif-ferent modalities of data, such as text, image, emoji, appear in Toxic and Non-Toxic interactions with different proportions. Therefore, we provided explana-tions of potentially valuable emoji and images. Each image name was createdby combining “source user id”, “target user id”, and “tweet number” in an in-teraction that each image pertains to. For example: the image 0023.0230.5.jpgis from a tweet between “user 0023” and “user 0230” and the 5th tweet intheir interaction. We processed these images utilizing a state-of-the-art imagerecognition tool, ResNet [17], providing the objects recognized in images withtheir probabilities (top-5 accuracy= 0.921). We kept the top 20 (empirically set)recognized object names. For example, an image has the following set of recog-nized objects: “television”, “cash machine”, “screen”, “monitor”, “neck brace”,“toyshop”, “medicine chest”, “library”, “home theater”, “wardrobe”, “score- https://github.com/onnx/models/tree/master/vision/classification/resnet Wijesiriwardene, Inan and Kursuncu et al. board”, “moving van”,“entertainment center”, “barbershop”, “desk”, “web site”.We utilized EmojiNet [53] to retrieve the meanings of the emoji in the inter-actions, and provided in the dataset. For instance, for the emoji , EmojiNetprovides the following set of keywords: “face”, “tear”, “joy”, “laugh”, “happy”,“cute”, “funny”, “joyful”, “hilarious”, “teary”, “laughing”, “person”, “smiley”,“lol”, “emoji”, “wtf”, “cry”, “crying”, “tears”, “lmao”. Specifically, the signif-icant difference in the use of image, video and emoji between the content ofToxic and Non-Toxic interactions, suggests that the contribution of multimodalelements would likely be critical. Three Label Two Label
Table 2: For three and two la-bels, agreement scores between thethree annotators using
Krippen-dorffs alpha . Kappa A BB C Table 3: Pairwise agreement forthe three label scheme, agreementscores between the three annotators(A,B,C) using Cohen Kappa
Privacy and Ethics Disclosure:
We useonly public Twitter data, and our study doesnot involve any direct interaction with any in-dividuals or their personally identifiable pri-vate data. This study was reviewed by thehost institution’s IRB and received an ex-emption determination. As noted above, wefollow standard practices for anonymizationduring data collection and processing by re-moving any identifiable information includ-ing names, usernames, URLs. We do not pro-vide any Twitter user or tweet id, or geolo-cation information. Due to privacy concernsand terms of use by Twitter, we make thisdataset available upon request to the authors,and researchers will be required to sign anagreement to use it only for research purposesand without public dissemination. C Table 4: Pairwise agreement for two labels, agreement scores be-tween the three annotators (A,B,C)using Cohen Kappa
Capturing truly toxic content on social me-dia for humans requires reliable annotationguidelines for training annotators. Our an-notators have completed a rigorous train-ing process including literature reviews anddiscussions on online toxic behavior andits socio-cultural context among adolescents.Three annotators labeled the interactions us-ing three labels:
Toxic (T), Non-Toxic (N) and
Unclear (U) . The annotators were trained by our co-author cognitive scien-tist to consider the context of the interaction rather than individual tweets whiledetermining the label of an interaction. We developed a guideline for annotatorsto follow that comprises intent-oriented criteria for labeling interactions as Toxic(T). That is, a tweet is toxic if the interactions contain: (i) Threat to harm a http://wiki.aiisc.ai/index.php/EmojiNetLONE: A Dataset for Toxic Behavior among Adolescents on Twitter 9 person, (ii) Effort to degrade or belittle a person, (iii) Express dislike towardsa person or a group of people, (iv) Promote hate/violence/offensive languagetowards a person or a group of people, (v) Negatively stereotype a person or aminority, (vi) Support and defend xenophobia, sexism or racism. Number ofTweets Mean Min Max
Toxic 13.28 3.0 304.0Non-Toxic 7.15 3.0 99.0(a)
Number ofEmoji Mean Min Max
Toxic 6.72 0.0 290.0Non-Toxic 3.51 0.0 60.0(b)
Number ofURLs Mean Min Max
Toxic 2.70 0.0 73.0Non-Toxic 1.63 0.0 26.0(c)
Number ofImages Mean Min Max
Toxic 1.18 0.0 20.0Non-Toxic 0.86 0.0 12.0(d)Table 5: (a)Descriptive statistics of tweets per interaction. (b) Descriptive statistics ofemoji per interaction. (c) Descriptive statistics of URLs per interaction. (d) Descriptivestatistics of images per interaction. There were 140 images showing Toxic Behavior and471 images showing Non-Toxic Behavior.
If an annotator could not arrive at a conclusion after assessing the interac-tion following this guideline, it was labeled as
Unclear . After the annotationswere completed by the three annotators , the labels were finalized by majorityvote. Then, agreement scores were computed utilizing Krippendorffs alpha ( α )and Cohens Kappa ( κ ). Note that the instances labelled Unclear (U) can beincluded in the training to exercise the robustness of a learned model, or theycan be removed as they add noise (as per the consensus of the annotators). Toaccommodate both scenarios, we create two schemes: (i) three label (T, N, U),(ii) two label (T, N) removing Unclear (U) instances [15]. We perform two anno-tation analysis for both schemes: (i) A group-wise annotator agreement to findthe robustness of the annotation by the three annotators using Krippendorffsalpha ( α ) [45], (ii) A pair-wise annotator agreement using Cohens Kappa ( κ )to identify the annotator with highest agreement with others. In the three-labelscheme, α was computed as 0.63, and for the two label scheme, ( α ) was 0.65.The agreement scores reported in Table 2 imply substantial agreement [7]. Wealso computed the agreement between annotators using κ and provided in Table3 and Table 4, for three label and the two label, respectively. While the annota-tors A and B have substantial and near perfect agreement, C has moderate andsubstantial agreement with A and B, both for the three and two label schemesrespectively [7]. http://homepages.inf.ed.ac.uk/jeanc/maptask-coding-html/node23.html0 Wijesiriwardene, Inan and Kursuncu et al.
118 (17.15%) 547 (79.51%) 23 (3.34%)
Table 6: Overall distribution of the data in-stances over the three labels.
Type of URLs Number of URLs
Image URLs 140 (43.88%)Video URLs 44 (13.79%)Text URLs 48 (15.04%)
Table 7: Different types of URLs in toxic interactions.
In this section, we provide descrip-tive statistics of the dataset concern-ing the distribution of tweets, images,emoji and URLs with respect to la-bels. Table 6 shows the overall distri-bution of the instances as Toxic in-teractions constitute the 17.15% ofthe dataset, while 79.51% remains asNon-Toxic. A minority group of in-teractions with 3.34% comprises theUnclear instances where annotatorsagreed that no conclusion could bederived. While the imbalance in thedataset provides challenges in themodeling of toxic behavior, it is re-flective of the nature of occurrencein real life. On the other hand, although the number of toxic interactions issmaller, they are richer in content as well as multimodal elements, comparedto non-toxic interactions [23] (see Tables 5a, 5b, 5c, 5d, and 7). Prior researchshows that appropriate incorporation of multimodal elements in modeling withsocial media data would improve performance [23,24,12,32]. In Table 5a, we seemean and maximum number of tweets per interaction for Toxic ones being signif-icantly higher than Non-toxic ones, suggesting the intensity of the toxic content.Further, according to Tables 5a, 5b, 5c, 5d, and 7, in the Toxic content, theuse of multimodal elements such as image, video, and emoji, is clearly higher,suggesting that the incorporation of these different modalities in the analysis ofthis dataset will be critical for a reliable outcome [24,23,12,32].
We created and examined the multimodal ALONE dataset for adolescent partic-ipants utilizing a lexicon [39] that divides offensive language into different typesconcerning appearance, intellectual, political, race, religion, and sexual prefer-ence. Given its unique characteristics concerning (i) adolescent population and(ii) interaction-based design, this dataset is an important contribution to the re-search community, as ground truth to provide a better understanding of onlinetoxic behavior as well as training machine learning models [42,25] and perform-ing time-series analysis. Specifically, quantitative as well as qualitative analysisof this dataset will reveal patterns with respect to social, cultural and behav-ioral dimensions [34,49,41] and shed light on etiology of toxicity in relationships.Further, researchers can develop guidelines for different kinds of toxic behav-ior such as harassment and hate speech, and annotate the dataset accordingly.Lastly, we reiterate that the ALONE dataset will be available upon request tothe authors, and the researchers will be required to sign an agreement to use itonly for research purposes and without public dissemination.
LONE: A Dataset for Toxic Behavior among Adolescents on Twitter 11
Acknowledgement
We acknowledge partial support from the National Science Foundation (NSF)award CNS-1513721: Context-Aware Harassment Detection on Social Media”.Any opinions, conclusions or recommendations expressed in this material arethose of the authors and do not necessarily reflect the views of the NSF.
References
1. Arpinar, I.B., Kursuncu, U., Achilov, D.: Social media analytics to identify andcounter islamist extremism: Systematic detection, evaluation, and challenging ofextremist narratives online. In: 2016 International Conference on CollaborationTechnologies and Systems (CTS). pp. 611–612. IEEE (2016)2. Arseneault, L., Bowes, L., Shakoor, S.: Bullying victimization in youths and mentalhealth problems:much ado about nothing? Psychological medicine (2010)3. Badjatiya, P., Gupta, M., Varma, V.: Stereotypical bias removal for hate speechdetection task using knowledge-based generalizations. In: The World Wide WebConference. pp. 49–59 (2019)4. Badjatiya, P., Gupta, S., Gupta, M., Varma, V.: Deep learning for hate speechdetection in tweets. In: WWW (2017)5. Brener, N.D., Simon, T.R., Krug, E.G., Lowry, R.: Recent trends in violence-relatedbehaviors among high school students in the united states. JAMA (1999)6. Buss, A.H.: The psychology of aggression (1961)7. Carletta, J., Isard, A., Isard, S., Kowtko, J.C., Doherty-Sneddon, G., Anderson,A.H.: The reliability of a dialogue structure coding scheme. (1997)8. Chatzakou, D., Kourtellis, N., Blackburn, J., De Cristofaro, E., Stringhini, G.,Vakali, A.: Mean birds: Detecting aggression and bullying on twitter. In: ACMWeb Science (2017)9. Crumback, D.: Subtweets: The new online harassment (2017)10. Davidson, T., Warmsley, D., Macy, M., Weber, I.: Automated hate speech detectionand the problem of offensive language. In: AAAI-ICWSM (2017)11. Dinakar, K., Reichart, R., Lieberman, H.: Modeling the detection of textual cy-berbullying. In: AAAI-ICWSM (2011)12. Duong, C.T., Lebret, R., Aberer, K.: Multimodal classification for analysing socialmedia. arXiv preprint arXiv:1708.02099 (2017)13. Edwards, A., Harris, C.J.: To tweet or subtweet?: Impacts of social networkingpost directness and valence on interpersonal impressions. Computers in HumanBehavior , 304–310 (2016)14. Founta, A., Djouvas, C., Chatzakou, D., Leontiadis, I., Blackburn, J., Stringh-ini, G., Vakali, A., Sirivianos, M., Kourtellis, N.: Large scale crowdsourcing andcharacterization of twitter abusive behavior (2018)15. Gaur, M., Alambo, A., Sain, J.P., Kursuncu, U., Thirunarayan, K., Kavuluru,R., Sheth, A., Welton, R., Pathak, J.: Knowledge-aware assessment of severityof suicide risk for early intervention. In: The World Wide Web Conference. pp.514–525. ACM (2019)16. Golbeck, J., Ashktorab, Z., Banjo, R.O., Berlinger, A., Bhagwan, S., Buntain,C., Cheakalos, P., Geller, A.A., Gergory, Q., Gnanasekaran, R.K., et al.: A largelabeled corpus for online harassment research. In: ACM Web Science (2017)17. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition.In: CVPR (2016)2 Wijesiriwardene, Inan and Kursuncu et al.18. Hosseinmardi, H., Mattson, S.A., Rafiq, R.I., Han, R., Lv, Q., Mishra, S.: Analyzinglabeled cyberbullying incidents on the instagram social network. In: SocInfo (2015)19. Jay, T., Janschewitz, K.: The pragmatics of swearing. Journal of Politeness Re-search. Language, Behaviour, Culture (2008)20. Kumpulainen, K., R¨as¨anen, E., Puura, K.: Psychiatric disorders and the use ofmental health services among children involved in bullying. Aggressive BehaviorJournal (2001)21. Kursuncu, U.: Modeling the Persona in Persuasive Discourse on Social Media UsingContext-aware and Knowledge-driven Learning. Ph.D. thesis, University of Georgia(2018)22. Kursuncu, U., Gaur, M., Castillo, C., Alambo, A., Thirunarayan, K., Shalin, V.,Achilov, D., Arpinar, I.B., Sheth, A.: Modeling islamist extremist communicationson social media using contextual dimensions: Religion, ideology, and hate. Pro-ceedings of the ACM on Human-Computer Interaction (CSCW), 1–22 (2019)23. Kursuncu, U., Gaur, M., Lokala, U., Illendula, A., Thirunarayan, K., Daniulaityte,R., Sheth, A., Arpinar, I.B.: What’s ur type? contextualized classification of usertypes in marijuana-related communications using compositional multiview embed-ding. In: 2018 IEEE/WIC/ACM International Conference on Web Intelligence(WI). pp. 474–479. IEEE (2018)24. Kursuncu, U., Gaur, M., Lokala, U., Thirunarayan, K., Sheth, A., Arpinar, I.B.:Predictive analysis on twitter: Techniques and applications. In: Emerging ResearchChallenges and Opportunities in Computational Social Network Analysis and Min-ing, pp. 67–104. Springer (2019)25. Kursuncu, U., Gaur, M., Sheth, A.: Knowledge infused learning (k-il): Towardsdeep incorporation of knowledge in deep learning. In: Proceedings of the AAAI 2020Spring Symposium on Combining Machine Learning and Knowledge Engineeringin Practice. Stanford University, Palo Alto, California, USA. AAAI-MAKE (2020)26. Liu, J., Lewis, G., Evans, L.: Understanding aggressive behaviour across the lifes-pan. Journal of psychiatric and mental health nursing (2013)27. Lowry, R., Powell, K.E., Kann, L., Collins, J.L., Kolbe, L.J.: Weapon-carrying,physical fighting, and fight-related injury among us adolescents. American journalof preventive medicine (1998)28. Mishna, F., Schwan, K.J., Lefebvre, R., Bhole, P., Johnston, D.: Students in dis-tress: Unanticipated findings in a cyber bullying study. Children and youth servicesreview (2014)29. Namie, G., Namie, R.: Bully at work: What you can do to stop the hurt and reclaimyour dignity on the job (2009)30. Nilan, P., Burgess, H., Hobbs, M., Threadgold, S., Alexander, W.: Youth, socialmedia, and cyberbullying among australian youth:sick friends. Social Media+ So-ciety (2015)31. Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y., Chang, Y.: Abusive languagedetection in online user content. In: WWW (2016)32. O’Halloran, K., Chua, A., Podlasov, A.: The role of images in social media analyt-ics: A multimodal digital humanities approach. In: Visual communication (2014)33. Papegnies, E., Labatut, V., Dufour, R., Linar`es, G.: Detection of abusive messagesin an on-line community. In: CORIA (2017)34. Parent, M.C., Gobble, T.D., Rochlen, A.: Social media behavior, toxic masculinity,and depression. Psychology of Men & Masculinities (3), 277 (2019)35. Patchin, J.W., Hinduja, S.: Bullies move beyond the schoolyard: A preliminarylook at cyberbullying. Youth violence and juvenile justice (2006)LONE: A Dataset for Toxic Behavior among Adolescents on Twitter 1336. Rafla, M., Carson, N.J., DeJong, S.M.: Adolescents and the internet: what mentalhealth clinicians need to know. Current psychiatry reports (9), 472 (2014)37. Razavi, A.H., Inkpen, D., Uritsky, S., Matwin, S.: Offensive language detectionusing multi-level classification. In: CCAI (2010)38. Rezvan, M., Shekarpour, S., Alshargi, F., Thirunarayan, K., Shalin, V.L., Sheth,A.: Analyzing and learning the language for different types of harassment. Plosone (3), e0227330 (2020)39. Rezvan, M., Shekarpour, S., Balasuriya, L., Thirunarayan, K., Shalin, V.L., Sheth,A.: A quality type-aware annotated corpus and lexicon for harassment research.In: ACM Web Science (2018)40. Rivers, I., Poteat, V.P., Noret, N., Ashurst, N.: Observing bullying at school: Themental health implications of witness status. School Psychology Quarterly (2009)41. Safadi, H., Li, W., Rahmati, P., Soleymani, S., Kursuncu, U., Kochut, K., Sheth,A.: Curtailing fake news propagation with psychographics. Available at SSRN3558236 (2020)42. Salminen, J., Almerekhi, H., Milenkovic, M., Jung, S.g., An, J., Kwak, H., Jansen,B.J.: Anatomy of online hate: Developing a taxonomy and machine learning modelsfor identifying and classifying hate in online news media. In: ICWSM. pp. 330–339(2018)43. Sharma, S., Agrawal, S., Shrivastava, M.: Degree based classification of harmfulspeech using twitter data. arXiv preprint arXiv:1806.04197 (2018)44. Silva, L., Mondal, M., Correa, D., Benevenuto, F., Weber, I.: Analyzing the targetsof hate in online social media. In: AAAI-ICWSM (2016)45. Sober´on, G., Aroyo, L., Welty, C., Inel, O., Lin, H., Overmeen, M.: Measuring crowdtruth: Disagreement metrics combined with worker behavior filters. In: CrowdSem2013 Workshop (2013)46. Søndergaard, D.M.: Bullying and social exclusion anxiety in schools. British Jour-nal of Sociology of Education (2012)47. Unicef, et al.: An Everyday Lesson: End Violence in Schools (2018)48. Viner, R.M., Aswathikutty-Gireesh, A., Stiglic, N., Hudson, L.D., Goddings, A.L.,Ward, J.L., Nicholls, D.E.: Roles of cyberbullying, sleep, and physical activity inmediating the effects of social media use on mental health and wellbeing amongyoung people in england: a secondary analysis of longitudinal data. The LancetChild & Adolescent Health (2019)49. Wandersman, A., Nation, M.: Urban neighborhoods and mental health: Psycholog-ical contributions to understanding toxicity, resilience, and interventions. AmericanPsychologist53