Improvement of Apriori-Pro Algorithm Based on MapReduce

Abstract

The improved Apriori-Pro algorithm solves the problem that the traditional Apriori algorithm generates a large number of candidate sets, but in the case of a large amount of data, the time complexity is significantly improved. In order to solve the problem of time consumption of Apriori-Pro algorithm under big data, the Apriori-Pro algorithm based on MapReduce is proposed. Based on the Apriori-Pro algorithm to solve the candidate set, this method introduces MapReduce parallelization technology to reduce the time consumed by a large number of comparison TID (Transaction ID) columns. Through time complexity analysis between different methods, the improved algorithm reduces the time consumption of comparisons and connections when processing large-scale data. Experiments on the Hadoop platform show that the proposed method has higher efficiency under big data.