Effectively Unified optimization for Large-scale Graph Community Detection

Abstract

In this paper, we present a unified graph clustering framework based on an asynchronous approach. We study the similarities among the Louvain algorithm and the Infomap algorithm. Based on their common features, we build an end-to-end optimized distributed framework for implementing both algorithms. By extending the existing asynchronous distributed framework for large-scale graphs traversal, we ensure both workload and communication balanced. Our extensive experiments show that our framework is correct and effective with different large real-world and synthetic datasets using up to 32,768 processors for the Louvain algorithm and 16,384 processors for the Infomap algorithm. The quality and the scalability of our framework are superior to the existing work.