Weiming Shen

A distributed frequent itemset mining algorithm using Spark for Big Data analytics

Frequent itemset mining is an essential step in the process of association rule mining. Conventional approaches for mining frequent itemsets in big data era encounter significant challenges when computing power and memory space are limited. This paper proposes an efficient distributed frequent itemset mining algorithm (DFIMA) which can significantly reduce the amount of candidate itemsets by applying a matrix-based pruning approach. The proposed algorithm has been implemented using Spark to further improve the efficiency of iterative computation. Numeric experiment results using standard benchmark datasets by comparing the proposed algorithm with the existing algorithm, parallel FP-growth, show that DFIMA has better efficiency and scalability. In addition, a case study has been carried out to validate the feasibility of DFIMA.

Evacuation path optimization based on quantum ant colony algorithm

Abstract Evacuation planning contains more than a few decisions which have to be made in a very short period of time and in the most appropriate way. Evacuation path optimization has vital importance in reducing the human and social harm and saving the aid time. Significant research efforts have been made in the literature to deal with evacuation optimization on the basis of deterministic optimization model, nevertheless the stochastic aspects or uncertainty of real-world evacuation have not been taken into account comprehensively. Inspired by the promising performance of heuristic algorithms to solve combinatorial problems, this paper proposes an improved quantum ant colony algorithm (QACA) for exhaustive optimization of the evacuation path that people can evacuate from hazardous areas to safe areas. In comparison with ACO (ant colony optimization) based method, QACA has the capability of finding a good solution faster using fewer individuals and possesses strong robustness, as a result of the quantum representation and updating of pheromone. Experiment results show that the proposed approach executes more effectively during evacuation.

Incremental FP-Growth mining strategy for dynamic threshold value and database based on MapReduce

With the coming of the Big Data era, data mining has been confronted with new opportunities and challenges. Some limitations are exposed when traditional association rule mining algorithms are used to deal with large-scale data. In the Apriori algorithm, scanning the external storage repeatedly leads to high I/O load and brings about low performance. As for FP-Growth algorithm, the effectiveness is limited by internal memory size because mining process is on the base of large tree-form data structure. Whats more, although remarkable achievements have been scored, there are still problems in dynamic scenarios. The paper presents a parallelized incremental FP-Growth mining strategy based on MapReduce, which aims to process large-scale data. The proposed incremental algorithm realizes effective data mining when threshold value and original database change at the same time. This novel algorithm is implemented on Hadoop and shows great advantages according to the experimental results.

An IoT-Based Online Monitoring System for Continuous Steel Casting

Monitoring solutions using the Internet of Things (IoT) techniques, can continuously gather sensory data, such as temperature and pressure, and provide abundant information for a monitoring center. Nevertheless, the heterogeneous and massive data bring significant challenges to real-time monitoring and decision making, particularly in time-sensitive industrial environments. This paper presents an online monitoring system based on an IoT system architecture which is composed of four layers: 1) sensing; 2) network; 3) service resource; and 4) application layers. It integrates various data processing techniques including protocol conversion, data filtering, and data conversion. The proposed system has been implemented and demonstrated through a real continuous steel casting production line, and integrated with the TeamCenter platform. Results indicate that the proposed solution well addresses the challenge of heterogeneous data and multiple communication protocols in real-world industrial environments.

A multi-agent based failure prediction method using neural network algorithm

A continuous monitoring system with high reliability is significantly important for complex equipment which is usually expensive, large-scale and sophisticated. Once a failure happens, it brings about not only serious economic losses, but also potential security hazards. In order to overcome outage damage caused by temporary failure and ensure excellent operation of the equipment, this paper presented an effective prediction model which combined the back propagation neural network (BPNN) with multi-agent cooperation grouping algorithm. The values of weights and thresholds of BPNN were obtained through optimization results of the multi-agent cooperation grouping algorithm. Based on above initialization parameters which met corresponding demands, repeated BPNN training was utilized to forecast fault. Case study on continuous casting equipment validated that the proposed model is valid for failure prognosis with forecasting accuracy elevated, compared with classical BPNN prediction method. Another comparison, function approximation experiment on the basis of a benchmark function, also showed that the suggested method is superior to BPNN in convergence speed.

Data mining for privacy preserving association rules based on improved MASK algorithm

With the arrival of the big data era, information privacy and security issues become even more crucial. The Mining Associations with Secrecy Konstraints (MASK) algorithm and its improved versions were proposed as data mining approaches for privacy preserving association rules. The MASK algorithm only adopts a data perturbation strategy, which leads to a low privacy-preserving degree. Moreover, it is difficult to apply the MASK algorithm into practices because of its long execution time. This paper proposes a new algorithm based on data perturbation and query restriction (DPQR) to improve the privacy-preserving degree by multi-parameters perturbation. In order to improve the time-efficiency, the calculation to obtain an inverse matrix is simplified by dividing the matrix into blocks; meanwhile, a further optimization is provided to reduce the number of scanning database by set theory. Both theoretical analyses and experiment results prove that the proposed DPQR algorithm has better performance.

Quantum ant colony algorithm-based emergency evacuation path choice algorithm

The evacuation path optimization in the disaster area plays an important role in reducing the human and social harm and saving aid time. In this paper, a novel algorithm for emergency evacuation path choice based on quantum ant colony algorithm (QACA) is proposed, and it avoids premature convergence and speeds up the convergence to the global optimal solution. In the proposed algorithm, Q-bit is used to represent the pheromone, and the rotation gate is used to update the pheromone. Simulation results show that the proposed algorithm is feasible and effective.

An expert knowledge-based dynamic maintenance task assignment model using discrete stress–strength interference theory

Abstract Expert knowledge has become an important factor in optimization decision-making for complex equipment maintenance. Motivated by the challenges of quantifying expert knowledge as a decision basis, we presented an expert knowledge-based dynamic maintenance task assignment model by using discrete stress–strength interference (DSSI) theory. We constructed the task assignment framework consisting of three parts: building expert database, selecting experts for tasks, and implementing the tasks, in which selecting experts for tasks based on expert knowledge is the key part of the model. To quantify tacit knowledge (experience) in optimization decision for expert recommendation, experience was defined as a probability, which is relevant to two random variables: quantity of task successfully implemented (strength) and quantity of task failed (stress), and experience is defined as the probability that the former (strength) is larger than the latter (stress). Further, universal generating function (UGF) method was used to calculate the experience, and decision rule was designed for the dynamic maintenance task assignment. The model can help collaborative maintenance platform periodically review experts’ performances and assign the corresponding task to the most suitable expert at different periods. A case study shows that the proposed model helps not only to achieve rational allocation of expert resources, but to promote positive competition among experts.

A user behavior prediction model based on parallel neural network and k-nearest neighbor algorithms

In the last decade, we have witnessed the dramatic development of the smart home industry. Smart home systems are currently facing an explosive growth of data. Making good use of this vast amount of data has become an attractive research topic in recent years. In order to develop smart home systems’ abilities for learning users’ behaviors autonomously and offering services spontaneously, a user behavior prediction model based on parallel back propagation neural network (BPNN) and k-nearest neighbor (KNN) algorithms is introduced in this paper. Based on MapReduce, a parallel BPNN algorithm is proposed to improve the prediction accuracy and speed, and a parallel KNN algorithm is developed for user decision-making rule selection. The experimental results indicate that the proposed model is significantly better than traditional user behavior prediction models in term of prediction accuracy and speed. A case study on smart home also illustrates the effectiveness of the proposed model.

Key Nodes Discovery in Large-Scale Logistics Network Based on MapReduce

In recent years, the study of social network is raising more and more attentions of researchers, locating the key nodes in social network is a hot research point. Lots of papers about how to discover the key nodes in social network such as mail network, micro log network was published. However, few people study on key nodes discovery in logistics network. In addition, most of methods of key nodes discovery only take relationship strength between nodes into account, few take the weight of node into account. In this paper, a node activity degree based on users behavior features was defined, As a result, the logistics networks can be considered as a double-weighted networks by taking relationship strength as edges weight and node activity as node weight. Based on Page Rank algorithm, an improved algorithms was proposed in this paper. The nodes weights were used as damping coefficient, and weight of the edges was used to compute importance of nodes during iterative process. At last, we implemented the improved Page Rank algorithm using MapReduce. One dataset from a logistics company were selected and comprehensive experiments were conducted. The experimental results show that proposed algorithms can effectively and efficiently discover key nodes in real logistics network.