Credibility Evaluation of Twitter-Based Event Detection by a Mixing Analysis of Heterogeneous Data

Abstract

Twitter has been recognized as an important data resource for real-time event detection. However, Twitter-based event detection systems cannot guarantee credibility in terms of their detection results. Rumor detection has been studied recently to enable credible event detection. Nevertheless, this problem has not yet been solved because most of the existing studies only focus on the information on Twitter with Twitter-based event detection systems. More specifically, the existing studies detect rumors by identifying and checking special features of incredible information on Twitter. However, values of the identified features can be faked easily and so it is important to conduct a mixing analysis of both Twitter and external credible data resources to solve this problem. The problem is how to harmoniously analyze heterogeneous data since they have different data formats, generation times, and so on. To solve this problem, this paper proposes a method to evaluate the credibility of Twitter-based event detection, which considers the two kinds of data resources for credibility evaluation to exclude influence by falsification. In particular, our method utilizes an event detection result and the number of articles related to the event. We performed comprehensive experiments to evaluate the proposed method. The experiments show that the proposed method gives the detected events high credibility and other events low credibility, correctly. More specifically, event detection accuracy increases by an average of 26.8% by reviewing the detection results according to their credibility evaluated with the proposed method. Additionally, to measure the appropriateness of the credibility evaluation, we filtered out incorrectly detected events from the event detection results referencing their evaluated credibility based on the proposed method. We calculated the F-measure, precision, and recall of the experimental results, and through the experiments, we present the effectiveness of the method.