Single-Pass On-Line Event Detection in Twitter Streams

Abstract

Intensive information is emerged in social media every second. Many breaking news often appear first in social media, much earlier than they appear in traditional news media. Through the technology of event detection on social media data streams, scatter information can be gathered together to inform us the popular events discussing online. An event is often modeled as a cluster of documents which discuss the same subject. Traditional event detection methods perform poorly on social media because of their huge amount of data and irregular expressions. In this paper, we propose a simple yet efficient event detection method towards social media. An event is represented by a sequence of keywords extracted from social media. We use a single-pass incremental clustering method with a trained encoder mapping documents and events into the same semantic space, which is helpful for the similarity calculation between them. We consider the similarity calculation between a tweet and an event as a matching process and construct a relevance matching dataset with tweet-event pairs. We finetune BERT (Bidirectional Encoder Representations from Transformers) model in the matching dataset to get an appropriate semantic encoder. Keywords are dynamically changed to represent an event for capturing the development of the event. Our proposed method achieves 0.86 on NMI (Normed Mutual Information), 0.69 on ARI (Adjusted Rand Index) and 0.70 on F1-score on a public twitter dataset, which shows the superiority of our method compared with baseline methods.