Zhiqiang Ge

HMM-Driven Robust Probabilistic Principal Component Analyzer for Dynamic Process Fault Classification

In this paper, a novel hidden Markov model (HMM)-driven robust latent variable model (LVM) is proposed for fault classification in dynamic industrial processes. A robust probabilistic model with Students t mixture output is designed for tolerating outliers. Based on the robust LVM, the probabilistic structure is further developed into a classifier form so as to incorporate various types of process information during model acquisition. After that, the robust probabilistic classifier is extended within the HMM framework so as to characterize the time-domain stochastic uncertainties. The model parameters are derived through the expectation-maximization algorithm. For performance validation, the developed model is tested on the Tennessee Eastman benchmark process.

Data Mining and Analytics in the Process Industry: The Role of Machine Learning

Data mining and analytics have played an important role in knowledge discovery and decision making/supports in the process industry over the past several decades. As a computational engine to data mining and analytics, machine learning serves as basic tools for information extraction, data pattern recognition and predictions. From the perspective of machine learning, this paper provides a review on existing data mining and analytics applications in the process industry over the past several decades. The state-of-the-art of data mining and analytics are reviewed through eight unsupervised learning and ten supervised learning algorithms, as well as the application status of semi-supervised learning algorithms. Several perspectives are highlighted and discussed for future researches on data mining and analytics in the process industry.

Plant-Wide Industrial Process Monitoring: A Distributed Modeling Framework

With the growing complexity of the modern industrial process, monitoring large-scale plant-wide processes has become quite popular. Unlike traditional processes, the measured data in the plant-wide process pose great challenges to information capture, data management, and storage. More importantly, it is difficult to efficiently interpret the information hidden within those data. In this paper, the road map of a distributed modeling framework for plant-wide process monitoring is introduced. Based on this framework, the whole plant-wide process is decomposed into different blocks, and statistical data models are constructed in those blocks. For online monitoring, the results obtained from different blocks are integrated through the decision fusion algorithm. A detailed case study is carried out for performance evaluation of the plant-wide monitoring method. Research challenges and perspectives are discussed and highlighted for future work.

Distributed Parallel PCA for Modeling and Monitoring of Large-Scale Plant-Wide Processes With Big Data

In order to deal with the modeling and monitoring issue of large-scale industrial processes with big data, a distributed and parallel designed principal component analysis approach is proposed. To handle the high-dimensional process variables, the large-scale process is first decomposed into distributed blocks with a priori process knowledge. Afterward, in order to solve the modeling issue with large-scale data chunks in each block, a distributed and parallel data processing strategy is proposed based on the framework of MapReduce and then principal components are further extracted for each distributed block. With all these steps, statistical modeling of large-scale processes with big data can be established. Finally, a systematic fault detection and isolation scheme is designed so that the whole large-scale process can be hierarchically monitored from the plant-wide level, unit block level, and variable level. The effectiveness of the proposed method is evaluated through the Tennessee Eastman benchmark process.

Mixture Bayesian Regularization of PCR Model and Soft Sensing Application

In this paper, a Bayesian regularization mechanism is provided for automatically determining the number of latent variables in the probabilistic principal component regression (PPCR) model. Different from the unsupervised principal-component-analysis model, the response variable is incorporated for the supervision of selecting latent variables in the PPCR model. By introducing two hyperparameter vectors, the effectiveness of each latent variable can be well measured and controlled. For the mixture form of the PPCR model, a corresponding mixture Bayesian regularization strategy is further developed to control the dimensionality of latent variables. The expectation-maximization algorithm is employed for the parameter learning of both single and mixture Bayesian regularization models. Two probabilistic soft sensors are then developed for the online estimation of key variables in industrial processes, the performances of which are evaluated through two case studies. Compared to the single Bayesian regularization model, the mixture model shows stronger soft sensing abilities in nonlinear and multimode processes.

A Probabilistic Just-in-Time Learning Framework for Soft Sensor Development With Missing Data

Just-in-time learning (JITL) is one of the most widely used strategies for soft sensor modeling in nonlinear processes. However, traditional JITL methods have difficulty in dealing with data samples that contain missing values. Meanwhile, data noises and uncertainties have not been taken into consideration for relevant sample selection in existing JITL approaches. To overcome these problems, a new probabilistic JITL (P-JITL) framework is proposed in this brief. In P-JITL, variational Bayesian principal component analysis is first utilized to handle missing values and extract Gaussian posterior distributions of latent variables. Then, symmetric Kullback–Leibler divergence is creatively employed to measure the dissimilarity of two distributions for relevant sample selection in the JITL framework. Finally, a nonlinear regression model, Gaussian process regression, is carried out to model the nonlinear relationship between the output and the extracted latent variables. In this way, the proposed probabilistic JITL (P-JITL) is able to deal with missing data and select relevant samples more accurately. To evaluate the effectiveness and flexibility of P-JITL, comparative studies between P-JITL and traditional deterministic JITL (D-JITL) are carried out on a numerical example and an industrial application example, in which missing data are simulated with percentages from 0% to 50%. The results show that P-JITL can provide more accurate prediction accuracy than D-JITL in each scenario considered.

Supervised Latent Factor Analysis for Process Data Regression Modeling and Soft Sensor Application

This brief proposed a new supervised latent factor analysis (FA) method for process data regression modeling. Different from the traditional principal component analysis/regression model, the new model can successfully estimate heterogeneous variances from different process variables, which is more practical. Under the same probabilistic modeling framework, the single supervised latent FA model is further extended to the mixture form. Efficient expectation-maximization algorithms are developed for parameter learning in both single and mixture supervised latent FA models. Based on the regression modeling between easy-to-measure and difficult-to-measure process variables, two soft sensors are built for quality prediction in the process. Two case studies are provided to evaluate the modeling and performances of the new methods.

Locally Weighted Prediction Methods for Latent Factor Analysis With Supervised and Semisupervised Process Data

Through calculating the similarity between the historical and the new query data samples, a probabilistic locally weighted prediction method based on supervised latent factor analysis (SLFA) model is proposed. In this method, the contributions of different historical samples are expressed through incorporating the similarity index into the noise variance of the process variables, which renders strong adaptability of the method for describing nonlinear relationships and abrupt changes of the process. Additionally, the proposed locally weighted method is extended to the semisupervised form, which is apparently more practical in real industrial processes, since the sampling rates of quality variables are much lower than those of ordinary process variables. Efficient expectation maximization algorithms are designed for parameter learning in both SLFA and semisupervised locally weighted LFA methods. Two real industrial processes are provided to evaluate the feasibility and the effectiveness of the newly developed soft sensors.

Soft Sensor Modeling of Nonlinear Industrial Processes Based on Weighted Probabilistic Projection Regression

Probabilistic principal component regression (PPCR) has been introduced for soft sensor modeling as a probabilistic projection regression method, which is effective in handling data collinearity and random noises. However, the linear limitation of data relationships may cause its performance deterioration when applied to nonlinear processes. Therefore, a novel weighted PPCR (WPPCR) algorithm is proposed in this paper for soft sensing of nonlinear processes. In WPPCR, by including the most relevant samples for local modeling, different weights will be assigned to these samples according to their similarities with the testing sample. Then, a weighted log-likelihood function is constructed, and expectation-maximization algorithm can be carried out iteratively to obtain the optimal model parameters. In this way, the nonlinear data relationship can be locally approximated by WPPCR. The effectiveness and flexibility of the proposed method are validated on a numerical example and an industrial process.

Semisupervised Kernel Learning for FDA Model and its Application for Fault Classification in Industrial Processes

For fault classification in industrial processes, the performance of the classification model highly depends on the size of labeled dataset. Unfortunately, labeling the fault types of data samples need expert experiences and prior knowledge of the process, which is costly and time consuming. As a result, semisupervised modeling with both labeled and unlabeled data have recently become an interest in industrial processes. In this paper, a kernel-driven semisupervised fisher discriminant analysis (FDA) model is proposed for nonlinear fault classification. Two discriminant analytical strategies are introduced for online fault assignment, namely k-nearest neighborhood and Bayesian inference. Detailed comparative studies are carried out through two industrial benchmark processes between the linear and kernel-driven semisupervised FDA models, in which the best fault classification performance is obtained by the kernel semisupervised model with Bayesian inference as its discriminant strategy.