Xinjie Lu

Missing Data Imputation: A Fuzzy K-means Clustering Algorithm over Sliding Window

Fuzzy set theory is motivated by the practical needs to manage and process uncertainty inherent in real world problem solving. It is useful in applications to data mining, conflict analysis, and so on. Although ignored by much of the related work, the high rate and unbounded nature of data make the sliding window indispensable. In this paper, we present a fuzzy k-means clustering algorithm over sliding window for the missing value imputation of incomplete data to improve the data quality. The experiments show that our missing data imputation algorithm tends to be more tolerant of imprecision and uncertainty and can lead to a better performance with accuracy guarantees.

QoS-Aware Publish-Subscribe Service for Real-Time Data Acquisition

Many complex distributed real-time applications need complicated processing and sharing of an extensive amount of data under critical timing constraints. In this paper, we present a comprehensive overview of the Data Distribution Service standard (DDS) and describe its QoS features for developing real-time applications. An overview of an active real-time database (ARTDB) named Agilor is also provided. For efficient expressing QoS policy in Agilor, a Real-time ECA (RECA) rule model is presented based on common ECA rule. And then we propose a novel QoS-aware Real-Time Publish-Subscribe (QRTPS) service compatible to DDS for distributed real-time data acquisition. Furthermore, QRTPS is implemented on Agilor by using objects and RECA rules in Agilor. To illustrate the benefits of QRTPS for real-time data acquisition, an example application is presented.

Incremental outlier detection in data streams using local correlation integral

In this paper, an incremental outlier detection technique capable of dealing with a large amount of data is presented and evaluated in the context of intrusion detection. The proposed method is based on the LOcal Correlation Integral (LOCI for short). The detection technique consists of two parts. The first part named insertion receives the sequence of input point and updates Multi-granularity DEviation Factor (MDEF) of the point at intervals. The second part named deletion deletes one or a batch of points. This technique is able to process streaming data in a single scan. Moreover, the number of updates in the incremental LOCI algorithm per insertion/deletion of a single data record does not depend on the total number of data records. Experimental results with real life data sets show that the technique is capable of dealing with data streams, successfully detecting outlier.

RRPS: A Ranked Real-Time Publish/Subscribe Using Adaptive QoS

Publish-Subscribe paradigm has been widely employed in Real-Time applications. However, the existing technologies and models only support a simple binary concept of matching: an event either matches a subscription or it does not; for instance, a production monitoring event will either match or not match a subscription for production anomaly. Based on adaptive Quality of Service (QoS) management, we propose a novel publish/subscribe model, which is implemented as a critical service in a real-time database Agilor . We argue that publications have different relevance to a subscription. On the premise of guaranteeing deadline d , a subscriber approximately receives k most relevant publications, where k and d are parameters defined by each subscription. After the architecture of our model is described, we present negotiations between components and scalable strategies for adaptive QoS management. Then, we propose an efficient algorithm to select different strategies adaptively depending on estimation of current QoS. Furthermore, we experimentally evaluate our model on real production data collected from manufacture industry to demonstrate its applicability in practice.

A Novel QoS-Enable Real-Time Publish-Subscribe Service

Complex distributed real-time applications require complicated processing and sharing of an extensive amount of data under critical timing constraints. In this paper, we present a comprehensive overview of the Data Distribution Service standard (DDS) and describe its QoS (Quality of Service) features for developing real-time applications. Real-time ECA (RECA) rules are introduced to efficiently describe QoS policy in an active real-time database (ARTDB) named Agilor. And then we propose a novel QoS-Enable Real-Time Publish-Subscribe (QERTPS) service compatible to DDS for distributed real-time data acquisition. QERTPS could support several different QoS levels for various applications at the same time. Furthermore, QERTPS is implemented by object models and RECA rules in Agilor. To illustrate the benefits of QERTPS for real-time data acquisition, an example application is presented. Experimental evaluation shows that the proposed service provides a stable and timely service for providing different QoS levels.

Data Prediction in Manufacturing: An Improved Approach Using Least Squares Support Vector Machines

Support vector machine (SVM) is a set of related supervised learning methods used for classification and regression based on statistical learning theory. In this paper, we present a least squares support vector machines (LSSVM) regression method based on relative error for manufacturing industries to estimate the true value of imprecise measured data during production logistics process. Our method has already been successfully applied in Manufacturing Execution System (MES) of some petrochemical enterprises in China.

Rule-Based Publish-Subscribe Mechanism for Real-time Applications

A Trusted Publish-Subscribe mechanism (TPS) has been developed for complex real-time applications by combining real-time database and reactive behavior. At first, a novel real-time object model is proposed which extends object-oriented data models by incorporating temporal consistency into objects. And then the Event-Condition-Action (ECA)rules with deadlines and values are defined with special emphasis on timing constraints. All transactions in TPS are classified into three types according to their features, and, their deadlines and priorities are discussed. In rule processing, a rule execution graph with couple modes and weights has been proposed in order to handle rules efficiently. Furthermore, to determine execution order of multiple rules triggered at the same time, a value-based conflict resolution is established on the highest value first principle. At last, the full-fledged structure of TPS server is implemented in detail.

An approximate approach to constraint solving in soft sensing

Soft sensing is usually presented as a constraint solving problem. In a manufacturing context, traditional methods of soft sensing have to face challenges in robustness and efficiency. In this paper, we proposed a granular-based approach to constraint solving for soft sensing. In our method, we first construct a granular-based soft sensing model, then estimate bounds of each granule, and finally solve this granulated problem with a smaller size. According to our analysis, this method is robust and efficient.

Quality Skyline in Sensor Database

Data quality of sensor database is very important in industrial applications. An appropriate time period can improve the quality of result of high-level applications. In this paper we present a quality assessment and then give an algorithm to detect a quality skyline in a sensor dataset with stationary assumption. This algorithm returns an appropriate length of time period that query with a longer time period will get a result with stationary data quality.

An Algorithm for Uncertain Data Reconciliation in Process Industry

This paper proposes an uncertain data reconciliation algorithm for Process Industry. First of all, the dynamic Event Dependency Graph is defined to abstract the problem. Taking into account the scale of the industry, a granularity partition algorithm relied on event detection is presented. In the following for the purpose of data prediction to improve the precision of the predicted value of the measured data, an improved Least Squares Support Vector Machine (LSSVM) model based on relative error is proposed. On the basis of the above, we present our data reconciliation algorithm by constructing a constraint model to achieve the goal of on-line/off-line data reconciliation. The practical industrial applications proved the efficiency and performance of the algorithm.