Optimism/Pessimism Prediction of Twitter Messages and Users Using BERT with Soft Label Assignment

Abstract

Being able to accurately predict users outlooks on social media platforms is important for developing educational public health interventions. In this paper, utilizing the contextualized representations provided by BERT, we propose new models to predict optimism/pessimism by fine-tuning BERT. By paying attention to the negation and other syntactic patterns, the self-attention mechanism via the transformer in BERT leads to more accurate models. For example, using the commonly used dataset for optimism/pessimism prediction with the proposed Soft Label Assignment (SLA), we have achieved 100% prediction accuracy at the user level and 97.10% at the tweet message level on the test set excluding the neutral ones. Furthermore, utilizing available training sample scores, we assign labels softly, which improves the generalization of the models and therefore their performance further. We additionally analyze the attention heads to illustrate the mechanisms of our models to classify different messages and demonstrate the connections between emotions and optimism/pessimism prediction.