Soft sensor modeling with a selective updating strategy for Gaussian process regression based on probabilistic principle component analysis

In recent studies, the Probabilistic Principal Component Analysis (PPCA) for imputing missing data was shown to be a good tool for traffic flow data processing. The PPCA method has two major benefits: it results in significantly smaller reconstruction errors and much less computation time, which make it outperform the conventional historical and regression imputing methods. In this paper, the possibility of applying more complex PPCA methods, e.g. Kernel Probabilistic Principal Component Analysis (KPPCA) or Mixed Probabilistic Principal Component Analysis (MPPCA), to impute missing data is explored. Considering reconstruction errors and computation time cost, test results show that the basic PPCA method is still our first choice in missing data imputing for traffic flow for online systems.

In standard Principal Component Analysis (PCA) comes one problem in addressing the set of incomplete data. The standard PCA procedure on incomplete data is to eliminate (listwise deletion procedure) or using the mean of the variable, this procedure may result in loss information from these observations. Another method used is to integrate Expectation Maximization (EM) to the method of Probabilistic Principal Component Analysis (PPCA). But PPCA can produce overfitting response prediction. In this study discussed the Variational Bayesian Principal Component Analysis (VBPCA) which is a method of development of PPCA method by incorporating prior information from the distribution of the principal components of the model parameters. From the simulation studies by eliminating the data through the concept of missing at random (MAR), obtained results that the value of the correlation scores principal components complete data with the principal component score predicted results PPCA method is superior when compared with VBPCA, as well as to the value of the correlation scores for the various percentages are generally incomplete data. However, judging from the size of a match between the response to predictions by the size normalized root mean square error of prediction (NRMSEP) VBPCA method produces better than PPCA.

In order to evaluate the perceptual quality of images, a full-reference quality index, which is called principal component deviation (PCD), is presented. This research is motivated by the discovery that the filters learned by the probabilistic principal component analysis are closely resemble the neurons of the human visual system. These filters are used as a model of the visual system, which are trained on 100,000 color image patches of size 4×4 by probabilistic principal component analysis. The PCD relates the image quality with the deviation between two sets of features that extracted by the learned filters. Experimental results show that PCD has relatively low computational complexity and high correlation with subjective quality evaluations.

Gaussian mixture model (GMM) techniques are popular for speaker identification. Theoretically, each Gaussian function should have a full covariance matrix. However, the diagonal covariance matrix is usually used because the inverse of diagonal covariance matrix can be easily calculated via expectation maximization (EM) algorithm. This paper proposes a new probabilistic principal component analysis (PPCA) model for speaker identification. The full covariance of speaker’s data is considered. This model is originated from factor analysis theory. The probability distributions using PPCA are well defined. In particular, GMM and PPCA are found to be equivalent when using diagonal covariance matrix. In this study, we derive a novel PPCA model selection and establish models for different speakers. Applying PPCA model selection, we can dynamically determine the numbers of speech features and mixture components. Experiments show that PPCA achieves desirable speaker recognition performance with proper model regularization.

Component Analysis (CA) comprises of statistical techniques that decompose signals into appropriate latent components, relevant to a task-at-hand (e.g., clustering, segmentation, classification). Recently, an explosion of research in CA has been witnessed, with several novel probabilistic models proposed (e.g., Probabilistic Principal CA, Probabilistic Linear Discriminant Analysis (PLDA), Probabilistic Canonical Correlation Analysis). PLDA is a popular generative probabilistic CA method, that incorporates knowledge regarding class-labels and furthermore introduces class-specific and sample-specific latent spaces. While PLDA has been shown to outperform several state-of-the-art methods, it is nevertheless a static model; any feature-level temporal dependencies that arise in the data are ignored. As has been repeatedly shown, appropriate modelling of temporal dynamics is crucial for the analysis of temporal data (e.g., videos). In this light, we propose the first, to the best of our knowledge, probabilistic LDA formulation that models dynamics, the so-called Dynamic-PLDA (DPLDA). DPLDA is a generative model suitable for video classification and is able to jointly model the label information (e.g., face identity, consistent over videos of the same subject), as well as dynamic variations of each individual video. Experiments on video classification tasks such as face and facial expression recognition show the efficacy of the proposed method.

This paper proposes a network travel time prediction methodology based on probe data. The model is intended as a tool for traffic management, trip planning, and online vehicle routing, and is designed to be efficient and scalable in calibration and real-time prediction; flexible to changes in network, data, and model extensions; and robust against noisy and missing data. A multivariate probabilistic principal component analysis (PPCA) model is proposed. Spatio-temporal correlations are inferred from historical data based on MLE and an efficient EM algorithm for handling missing data. Prediction is performed in real time by computing the expected distribution of link travel times in future time intervals, conditional on recent current-day observations. A generalization of the methodology partitions the network and applies a distinct PPCA model to each subnetwork. The methodology is applied to the network of downtown Shenzhen, China, using taxi probe data. The model captures variability over months and weekdays as well as other factors. Prediction with PPCA outperforms $k$ -nearest neighbors prediction for horizons from 15 to 45 min, and a hybrid method of PPCA and local smoothing provides the highest accuracy.

This study proposes a probabilistic principal component analysis- (PPCA-) based method for online monitoring of water-quality contaminant events by UV-Vis (ultraviolet-visible) spectroscopy. The purpose of this method is to achieve fast and sound protection against accidental and intentional contaminate injection into the water distribution system. The method is achieved first by properly imposing a sliding window onto simultaneously updated online monitoring data collected by the automated spectrometer. The PPCA algorithm is then executed to simplify the large amount of spectrum data while maintaining the necessary spectral information to the largest extent. Finally, a monitoring chart extensively employed in fault diagnosis field methods is used here to search for potential anomaly events and to determine whether the current water-quality is normal or abnormal. A small-scale water-pipe distribution network is tested to detect water contamination events. The tests demonstrate that the PPCA-based online monitoring model can achieve satisfactory results under the ROC curve, which denotes a low false alarm rate and high probability of detecting water contamination events.

In aerospace engineering, various problems such as restoring impaired experimental flow data can be handled by gappy proper orthogonal decomposition (POD). Similarly to gappy POD, probabilistic principal component analysis (PPCA) can approximate missing data with the help of an expectation-maximization (EM) algorithm, yielding an EM algorithm applied to PPCA (EM-PCA). Although both gappy POD and EM-PCA address the same missing date estimation problem, their antithetical formulation perspectives hinder their direct comparison; the development of the former is deterministic whereas that of the latter is probabilistic. In order to effectively differentiate both methods, this research provides a unifying least squares perspective to qualitatively dissect them within a unified least squares framework. By virtue of the unifying least squares perspective, gappy POD and the EM-PCA turn out to be similar in that they are twofold: basis and least squares coefficient evaluations. On the other hand, they are dissimilar because the EM-PCA, unlike gappy POD, dispenses with either a gappy norm or a POD basis. To illustrate the theoretical analysis of both methods, numerical experiments using simple and complex data sets quantitatively examine their performance in terms of convergence rates and computational cost. Finally, comprehensive comparisons, including theoretical and numerical aspects, establish that the EM-PCA is simpler and thereby more efficient than gappy POD.

The missing data problem exists in a lot of transportation systems while many traffic applications require accurate traffic data to work well. So, various methods were proposed for traffic data imputation in recent years. However, most of these approaches only consider the information of a single detector. In this paper, we use the temporal and spatial information collected from multiple detectors to improve the traffic data imputation accuracy. Specially, we consider Probabilistic Principal Component Analysis (PPCA) and Mixed Probabilistic Principal Component Analysis (MPPCA), which had been proven to be useful methods to handle data imputation. We examine the performance of these two methods using the temporal and spatial information. Tests show that when only using the spatial information, the errors of these two methods are reduced and MPPCA works even better than PPCA. Moreover, when both of the temporal and spatial information from multiple detectors are used, more imputation accuracy is obtained than only using the spatial information for MPPCA.

Principal Component Analysis (PCA), also known as Proper Orthogonal Decomposition (POD), is one of the prevailing Reduced Order Modeling (ROM) techniques for aerodynamic data analysis. Along with the PCA, its variant for missing data, gappy POD, has recently found its application by transforming problems into missing data problems. In this paper, an alternative based on probability theory, i.e., Probabilistic Principal Component Analysis (PPCA), is employed for aerodynamic data reconstruction problems in lieu of the PCA and the gappy POD. Unlike the PCA, the PPCA generates spanning vectors of low-dimensional space iteratively with an Expectation Maximization (EM) algorithm. The combined algorithm, termed as Expectation-Maximization Probabilistic Principal Component Analysis (EM-PCA), can deal with not only complete data but also incomplete data sets, since the EM algorithm naturally incorporates missing data in its formulation. Therefore, the EM-PCA has potential to be a substitute for both PCA and gappy POD. For illustration, the EM-PCA is applied to complete as well as incomplete snapshot ensembles compiled from aerodynamic simulations, and acquired modes and restored data are verified with those obtained by the PCA and the gappy POD.

In the complex industrial processes, the process data have the characteristics of imbalance and are incomplete due to the difficult-to-measure key variables, leading to the performance degradation of soft sensors. In order to deal with this problem, a novel noise injection virtual sample generation approach based on target-relevant autoencoder (NITAE-VSG) is proposed in this article. Different from the conventional VSG methods, the presented NITAE-VSG model can generate useful input and output virtual samples directly. To increase the diversity of virtual samples, the Gaussian noise (G_noise) is added into the features extracted from the target-relevant autoencoder (TAE) hidden layer. The generated virtual samples are combined with the original small samples to enhance the soft sensor model performance. To verify the reliable feasibility and superior effectiveness of the presented NITAE-VSG, two actual industrial cases: an industrial process of purified terephthalic acid shortened as PTA and a production system of ethylene are carried out. The results of simulations demonstrate that the presented NITAE-VSG can generate superior virtual samples to some other virtual sample generation (VSG) methods and can effectively improve the soft sensor modeling accuracy.

Recently, the development of multilevel inverters has great progress in many Industrial applications because of its high efficiency and low switching frequency control methods. To improve the fault diagnosis accuracy, A k-Nearest Neighbors (k-NN) algorithm based on the different feature extractions is used. In this paper, the Principle Component Analysis (PCA) and Probabilistic Principle Component Analysis (PPCA) are used for the feature extraction. Firstly, the data from the output voltage signals under different fault conditions of the Cascaded H-Bridge Multilevel Inverter (CHMI) is optimized by using different feature extractions. Then, the k-NN classifier is used to identify the accurate fault location to diagnosis the fault. Finally, the FFT analysis also applied to evaluate the proposed k-NN technique. To validate the proposed technique the experimental setup has built in the laboratory and verify the simulation results. Based on the experimental and simulation results, the proposed k-NN technique has better performance when the PPCA feature extraction is used.

Online quality prediction for cobalt oxalate synthesis process using least squares support vector regression approach with dual updating

Ultrasound (US) imaging is the initial phase in the preliminary diagnosis for the treatment of kidney diseases, particularly to estimate kidney size, shape and position, to give information about kidney function, and to help in diagnosis of abnormalities like cysts, stones, junctional parenchyma and tumors which is shown in Figs. 7–9. This study proposes Grey Level Co-occurrence Matrix (GLCM)-based Probabilistic Principal Component Analysis (PPCA) and Artificial Neural Network (ANN) method for the classification of kidney images. Grey Wolf Optimization (GWO) is used to update the current positions of abnormal kidney images in the discrete searching space, thus getting the optimal feature subset for better classification purposes based on Feed Forward Neural Network (FFNN). The scanned image is pre-processed and the required features are extracted by GLCM, among those, some features are selected by PPCA. Feed Forward Back propagation Neural Network (FFBN) is used to classify the normalities and abnormalities in the part of kidney images. The proposed methodology is implemented in MATLAB platform and the analyzed result produces 98% accuracy using GWO-FFBN technique.

Rolling element bearing fault detection using PPCA and spectral kurtosis