Robust probabilistic principal component analysis for process modeling subject to scaled mixture Gaussian noise

In recent studies, the Probabilistic Principal Component Analysis (PPCA) for imputing missing data was shown to be a good tool for traffic flow data processing. The PPCA method has two major benefits: it results in significantly smaller reconstruction errors and much less computation time, which make it outperform the conventional historical and regression imputing methods. In this paper, the possibility of applying more complex PPCA methods, e.g. Kernel Probabilistic Principal Component Analysis (KPPCA) or Mixed Probabilistic Principal Component Analysis (MPPCA), to impute missing data is explored. Considering reconstruction errors and computation time cost, test results show that the basic PPCA method is still our first choice in missing data imputing for traffic flow for online systems.

Principal component analysis (PCA) is widely used to extract features and reduce dimensionality in various computer vision and image/video processing tasks. Conventional approaches either lack robustness to outliers and corrupted data or are designed for one-dimensional signals. To address this problem, we propose a robust PCA model for two-dimensional images incorporating structured sparse priors, referred to as structured sparse 2D-PCA. This robust model considers the prior of structured and grouped pixel values in two dimensions. As the proposed formulation is jointly nonconvex and nonsmooth, which is difficult to tackle by joint optimization, we develop a two-stage alternating minimization approach to solve the problem. This approach iteratively learns the projection matrices by bidirectional decomposition and utilizes the proximal method to obtain the structured sparse outliers. By considering the structured sparsity prior, the proposed model becomes less sensitive to noisy data and outliers in two dimensions. Moreover, the computational cost indicates that the robust two-dimensional model is capable of processing quarter common intermediate format video in real time, as well as handling large-size images and videos, which is often intractable with other robust PCA approaches that involve image-to-vector conversion. Experimental results on robust face reconstruction, video background subtraction data set, and real-world videos show the effectiveness of the proposed model compared with conventional 2D-PCA and other robust PCA algorithms.

In standard Principal Component Analysis (PCA) comes one problem in addressing the set of incomplete data. The standard PCA procedure on incomplete data is to eliminate (listwise deletion procedure) or using the mean of the variable, this procedure may result in loss information from these observations. Another method used is to integrate Expectation Maximization (EM) to the method of Probabilistic Principal Component Analysis (PPCA). But PPCA can produce overfitting response prediction. In this study discussed the Variational Bayesian Principal Component Analysis (VBPCA) which is a method of development of PPCA method by incorporating prior information from the distribution of the principal components of the model parameters. From the simulation studies by eliminating the data through the concept of missing at random (MAR), obtained results that the value of the correlation scores principal components complete data with the principal component score predicted results PPCA method is superior when compared with VBPCA, as well as to the value of the correlation scores for the various percentages are generally incomplete data. However, judging from the size of a match between the response to predictions by the size normalized root mean square error of prediction (NRMSEP) VBPCA method produces better than PPCA.

In order to evaluate the perceptual quality of images, a full-reference quality index, which is called principal component deviation (PCD), is presented. This research is motivated by the discovery that the filters learned by the probabilistic principal component analysis are closely resemble the neurons of the human visual system. These filters are used as a model of the visual system, which are trained on 100,000 color image patches of size 4×4 by probabilistic principal component analysis. The PCD relates the image quality with the deviation between two sets of features that extracted by the learned filters. Experimental results show that PCD has relatively low computational complexity and high correlation with subjective quality evaluations.

Gaussian mixture model (GMM) techniques are popular for speaker identification. Theoretically, each Gaussian function should have a full covariance matrix. However, the diagonal covariance matrix is usually used because the inverse of diagonal covariance matrix can be easily calculated via expectation maximization (EM) algorithm. This paper proposes a new probabilistic principal component analysis (PPCA) model for speaker identification. The full covariance of speaker’s data is considered. This model is originated from factor analysis theory. The probability distributions using PPCA are well defined. In particular, GMM and PPCA are found to be equivalent when using diagonal covariance matrix. In this study, we derive a novel PPCA model selection and establish models for different speakers. Applying PPCA model selection, we can dynamically determine the numbers of speech features and mixture components. Experiments show that PPCA achieves desirable speaker recognition performance with proper model regularization.

This paper proposes a network travel time prediction methodology based on probe data. The model is intended as a tool for traffic management, trip planning, and online vehicle routing, and is designed to be efficient and scalable in calibration and real-time prediction; flexible to changes in network, data, and model extensions; and robust against noisy and missing data. A multivariate probabilistic principal component analysis (PPCA) model is proposed. Spatio-temporal correlations are inferred from historical data based on MLE and an efficient EM algorithm for handling missing data. Prediction is performed in real time by computing the expected distribution of link travel times in future time intervals, conditional on recent current-day observations. A generalization of the methodology partitions the network and applies a distinct PPCA model to each subnetwork. The methodology is applied to the network of downtown Shenzhen, China, using taxi probe data. The model captures variability over months and weekdays as well as other factors. Prediction with PPCA outperforms $k$ -nearest neighbors prediction for horizons from 15 to 45 min, and a hybrid method of PPCA and local smoothing provides the highest accuracy.

Soft sensor modeling with a selective updating strategy for Gaussian process regression based on probabilistic principle component analysis

This study proposes a probabilistic principal component analysis- (PPCA-) based method for online monitoring of water-quality contaminant events by UV-Vis (ultraviolet-visible) spectroscopy. The purpose of this method is to achieve fast and sound protection against accidental and intentional contaminate injection into the water distribution system. The method is achieved first by properly imposing a sliding window onto simultaneously updated online monitoring data collected by the automated spectrometer. The PPCA algorithm is then executed to simplify the large amount of spectrum data while maintaining the necessary spectral information to the largest extent. Finally, a monitoring chart extensively employed in fault diagnosis field methods is used here to search for potential anomaly events and to determine whether the current water-quality is normal or abnormal. A small-scale water-pipe distribution network is tested to detect water contamination events. The tests demonstrate that the PPCA-based online monitoring model can achieve satisfactory results under the ROC curve, which denotes a low false alarm rate and high probability of detecting water contamination events.

In aerospace engineering, various problems such as restoring impaired experimental flow data can be handled by gappy proper orthogonal decomposition (POD). Similarly to gappy POD, probabilistic principal component analysis (PPCA) can approximate missing data with the help of an expectation-maximization (EM) algorithm, yielding an EM algorithm applied to PPCA (EM-PCA). Although both gappy POD and EM-PCA address the same missing date estimation problem, their antithetical formulation perspectives hinder their direct comparison; the development of the former is deterministic whereas that of the latter is probabilistic. In order to effectively differentiate both methods, this research provides a unifying least squares perspective to qualitatively dissect them within a unified least squares framework. By virtue of the unifying least squares perspective, gappy POD and the EM-PCA turn out to be similar in that they are twofold: basis and least squares coefficient evaluations. On the other hand, they are dissimilar because the EM-PCA, unlike gappy POD, dispenses with either a gappy norm or a POD basis. To illustrate the theoretical analysis of both methods, numerical experiments using simple and complex data sets quantitatively examine their performance in terms of convergence rates and computational cost. Finally, comprehensive comparisons, including theoretical and numerical aspects, establish that the EM-PCA is simpler and thereby more efficient than gappy POD.

Summary We develop new statistical theory for probabilistic principal component analysis models in high dimensions. The focus is the estimation of the noise variance, which is an important and unresolved issue when the number of variables is large in comparison with the sample size. We first unveil the reasons for an observed downward bias of the maximum likelihood estimator of the noise variance when the data dimension is high. We then propose a bias-corrected estimator by using random-matrix theory and establish its asymptotic normality. The superiority of the new and bias-corrected estimator over existing alternatives is checked by Monte Carlo experiments with various combinations of (p, n) (the dimension and sample size). Next, we construct a new criterion based on the bias-corrected estimator to determine the number of the principal components, and a consistent estimator is obtained. Its good performance is confirmed by a simulation study and real data analysis. The bias-corrected estimator is also used to derive new asymptotics for the related goodness-of-fit statistic under the high dimensional scheme.

The missing data problem exists in a lot of transportation systems while many traffic applications require accurate traffic data to work well. So, various methods were proposed for traffic data imputation in recent years. However, most of these approaches only consider the information of a single detector. In this paper, we use the temporal and spatial information collected from multiple detectors to improve the traffic data imputation accuracy. Specially, we consider Probabilistic Principal Component Analysis (PPCA) and Mixed Probabilistic Principal Component Analysis (MPPCA), which had been proven to be useful methods to handle data imputation. We examine the performance of these two methods using the temporal and spatial information. Tests show that when only using the spatial information, the errors of these two methods are reduced and MPPCA works even better than PPCA. Moreover, when both of the temporal and spatial information from multiple detectors are used, more imputation accuracy is obtained than only using the spatial information for MPPCA.

Principal Component Analysis (PCA), also known as Proper Orthogonal Decomposition (POD), is one of the prevailing Reduced Order Modeling (ROM) techniques for aerodynamic data analysis. Along with the PCA, its variant for missing data, gappy POD, has recently found its application by transforming problems into missing data problems. In this paper, an alternative based on probability theory, i.e., Probabilistic Principal Component Analysis (PPCA), is employed for aerodynamic data reconstruction problems in lieu of the PCA and the gappy POD. Unlike the PCA, the PPCA generates spanning vectors of low-dimensional space iteratively with an Expectation Maximization (EM) algorithm. The combined algorithm, termed as Expectation-Maximization Probabilistic Principal Component Analysis (EM-PCA), can deal with not only complete data but also incomplete data sets, since the EM algorithm naturally incorporates missing data in its formulation. Therefore, the EM-PCA has potential to be a substitute for both PCA and gappy POD. For illustration, the EM-PCA is applied to complete as well as incomplete snapshot ensembles compiled from aerodynamic simulations, and acquired modes and restored data are verified with those obtained by the PCA and the gappy POD.

A probabilistic model with spike-and-slab regularization for inferential fault detection and isolation of industrial processes

Efficient dimensionality reduction can involve generative latent variable models such as probabilistic principal component analysis (PPCA) or independent component analysis (ICA). Such models aim to extract a reduced set of variables (latent variables) from the original ones. In most cases, the learning of these models occur within an unsupervised framework where only unlabeled samples are used. In this paper, we investigate the possibility of estimating an independent factor analysis model (IFA), and thus projecting original data onto a lower dimensional space, when prior knowledge on the cluster membership of some training samples is incorporated. We propose to allow this model to learn within a semi-supervised framework in which likelihood of both labeled and unlabeled samples is maximized by a generalized expectation-maximization (GEM) algorithm. Experimental results with real data sets are provided to demonstrate the ability of our approach to find a low dimensional manifold with good explanatory power.

Improved Sparse Representation based Robust Hybrid Feature Extraction Models with Transfer and Deep Learning for EEG Classification