A Unifying Least Squares Perspective for Gappy Proper Orthogonal Decomposition and Probabilistic Principal Component Analysis

Principal Component Analysis (PCA), also known as Proper Orthogonal Decomposition (POD), is one of the prevailing Reduced Order Modeling (ROM) techniques for aerodynamic data analysis. Along with the PCA, its variant for missing data, gappy POD, has recently found its application by transforming problems into missing data problems. In this paper, an alternative based on probability theory, i.e., Probabilistic Principal Component Analysis (PPCA), is employed for aerodynamic data reconstruction problems in lieu of the PCA and the gappy POD. Unlike the PCA, the PPCA generates spanning vectors of low-dimensional space iteratively with an Expectation Maximization (EM) algorithm. The combined algorithm, termed as Expectation-Maximization Probabilistic Principal Component Analysis (EM-PCA), can deal with not only complete data but also incomplete data sets, since the EM algorithm naturally incorporates missing data in its formulation. Therefore, the EM-PCA has potential to be a substitute for both PCA and gappy POD. For illustration, the EM-PCA is applied to complete as well as incomplete snapshot ensembles compiled from aerodynamic simulations, and acquired modes and restored data are verified with those obtained by the PCA and the gappy POD.

In a power delivery system, excessive loads may lead to high temperature of power transmission lines. The extent of aging is proportional to the severity of the thermal overload. How to evaluate aging is an important issue for electric power systems. A critical span refers to a span where the highest conductor temperature occurs in transmission lines and thereby limits how much power can be delivered. In order to monitor real-time temperature of an overhead transmission network, this paper develops a method to simulate a dynamic model for conductor temperature distribution of transmission lines. Proper orthogonal decomposition (POD) is a widely used technique to simulate a spatio-temporal model. For high-dimensional data, POD is an efficient method to reduce the complexity of modeling. Furthermore, an extension of POD — the gappy POD — is used in this study to determine the critical spans so as to reconstruct the conductor temperature distributions of all spans. The simulation results show that the proposed method provides an efficient way of estimating transmission conductor temperature with high accuracy. The average mean square error is less than 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">−4</sup> . Through building a transmission conductor temperature model, the critical span of transmission lines are determined so that effective management and scheduling in real time for power delivery systems can be achieved.

In standard Principal Component Analysis (PCA) comes one problem in addressing the set of incomplete data. The standard PCA procedure on incomplete data is to eliminate (listwise deletion procedure) or using the mean of the variable, this procedure may result in loss information from these observations. Another method used is to integrate Expectation Maximization (EM) to the method of Probabilistic Principal Component Analysis (PPCA). But PPCA can produce overfitting response prediction. In this study discussed the Variational Bayesian Principal Component Analysis (VBPCA) which is a method of development of PPCA method by incorporating prior information from the distribution of the principal components of the model parameters. From the simulation studies by eliminating the data through the concept of missing at random (MAR), obtained results that the value of the correlation scores principal components complete data with the principal component score predicted results PPCA method is superior when compared with VBPCA, as well as to the value of the correlation scores for the various percentages are generally incomplete data. However, judging from the size of a match between the response to predictions by the size normalized root mean square error of prediction (NRMSEP) VBPCA method produces better than PPCA.

The proper orthogonal decomposition (POD) has been widely used in fluid dynamic applications for extracting dominant flow features. The “gappy” POD is an extension to this method that allows the consideration of incomplete data sets. In this paper, the gappy POD is extended to handle unsteady flow reconstruction problems, such as those encountered when limited flow measurement data is available. In addition, a systematic approach for effective sensor placement is formulated within the gappy framework using a condition number criterion. This criterion allows for accurate flow reconstruction results and yields sensor configurations that are robust to sensor noise. Two applications are considered. The first aims to reconstruct the unsteady flow field using a small number of surface pressure measurements for a subsonic airfoil undergoing plunging motion. The second considers estimation of POD modal content of a cylinder wake flow for active control purposes. In both cases, using the dominant POD basis vectors and a small number of sensor signals, the gappy approach is found to yield accurate flow reconstruction results.

Gappy POD, based on proper orthogonal decomposition (POD), is a method to repair missing data. A gappy POD-based interpolation method for the separation trajectory of the external store is proposed to address the challenge of low computational efficiency of the current numerical simulation based on computational fluid dynamics. Firstly, the numerical simulation method based on quasi-steady computational fluid dynamics was employed to calculate the separation trajectory of the external store in various drop conditions. Then, the data features of the separation trajectory, used as a known data set, were extracted using the gappy POD method for the reconstruction of the missing separation trajectory in the target drop condition. The effect of known sample size and data missing rate on reconstruction error was studied. The results show that the gappy POD method can quickly and accurately reconstruct the missing separation trajectory of the store in the target condition based on the known trajectory data set, even when only a small amount of data sample is known.

Fusing sensor data with CFD results using gappy POD

Sparsity can be leveraged with dimensionality-reduction techniques to characterize and model parametrized nonlinear dynamical systems. Sparsity is used for both sparse representation, via proper orthogonal decomposition (POD) modes in different dynamical regimes, and by compressive sensing, which provides the mathematical architecture for robust classification of POD subspaces. The method relies on constructing POD libraries in order to characterize the dominant, low-rank coherent structures. Using a greedy sampling algorithm, such as gappy POD and one of its many variants, an accurate Galerkin-POD projection approximating the nonlinear terms from a sparse number of grid points can be constructed. The selected grid points for sampling, if chosen well, can be shown to be effective sensing/measurement locations for classifying the underlying dynamics and reconstruction of the nonlinear dynamical system. The use of sparse sampling for interpolating nonlinearities and classification of appropriate POD modes facilitates a family of local reduced-order models for each physical regime, rather than a higher-order global model. We demonstrate the sparse sampling and classification method on the canonical problem of flow around a cylinder. The method allows for a robust mathematical framework for robustly selecting POD modes from a library, accurately constructing the full state space, and generating a Galerkin-POD projection for simulating the nonlinear dynamical system.

Gappy proper orthogonal decomposition (GPOD) is assessed here as a data reconstruction technique for particle image velocimetry (PIV) measurements, specifically those applicable to gas turbine combustors (GTC). At practical operating conditions, PIV measurements are plagued with issues, such as low signal-to-noise ratios, that result in significant gaps in data. Four GPOD methods are studied here, including a new method that makes use of a median filter (MF) outlier detection technique to adaptively update reconstructions between iterations. The analysis of the performance of the GPOD methods is done by implementing them on a non-gappy PIV data set. Artificial gaps of varying amounts are added to this non-gappy data set in a manner similar to the gaps found in real experimental data. Additionally, two criteria to check for GPOD convergence are investigated. It was found that the MF method produced the lowest reconstruction error regardless of the amount of gappiness. Furthermore, the MF method was found to be relatively insensitive to the accuracy of the convergence criterion. The MF GPOD therefore is an effective method for filling in missing data in PIV measurements of gas turbine combustor flows.

In order to evaluate the perceptual quality of images, a full-reference quality index, which is called principal component deviation (PCD), is presented. This research is motivated by the discovery that the filters learned by the probabilistic principal component analysis are closely resemble the neurons of the human visual system. These filters are used as a model of the visual system, which are trained on 100,000 color image patches of size 4×4 by probabilistic principal component analysis. The PCD relates the image quality with the deviation between two sets of features that extracted by the learned filters. Experimental results show that PCD has relatively low computational complexity and high correlation with subjective quality evaluations.

Gaussian mixture model (GMM) techniques are popular for speaker identification. Theoretically, each Gaussian function should have a full covariance matrix. However, the diagonal covariance matrix is usually used because the inverse of diagonal covariance matrix can be easily calculated via expectation maximization (EM) algorithm. This paper proposes a new probabilistic principal component analysis (PPCA) model for speaker identification. The full covariance of speaker’s data is considered. This model is originated from factor analysis theory. The probability distributions using PPCA are well defined. In particular, GMM and PPCA are found to be equivalent when using diagonal covariance matrix. In this study, we derive a novel PPCA model selection and establish models for different speakers. Applying PPCA model selection, we can dynamically determine the numbers of speech features and mixture components. Experiments show that PPCA achieves desirable speaker recognition performance with proper model regularization.

In recent studies, the Probabilistic Principal Component Analysis (PPCA) for imputing missing data was shown to be a good tool for traffic flow data processing. The PPCA method has two major benefits: it results in significantly smaller reconstruction errors and much less computation time, which make it outperform the conventional historical and regression imputing methods. In this paper, the possibility of applying more complex PPCA methods, e.g. Kernel Probabilistic Principal Component Analysis (KPPCA) or Mixed Probabilistic Principal Component Analysis (MPPCA), to impute missing data is explored. Considering reconstruction errors and computation time cost, test results show that the basic PPCA method is still our first choice in missing data imputing for traffic flow for online systems.

An iterative algorithm for POD basis adaptation in solving parametric convection–diffusion equations

In this paper, a new approach is proposed based on data fusing with vibration signals using time-frequency parameters, probabilistic principal component analysis (PPCA) and statistical inference, for improving the accuracy and visibility of damage identification for numerical control (NC) machine tools. Time-frequency feature principal components are put forward, which extracted from eight dimensionless parameters statistically in the time and frequency domains by PPCA. The Chi-2 statistic is established according to statistical inference principle, and the feature figure of principal components is built that can acquire damage distribution of tools by measured data. An empirical analysis in NC milling machine tools is developed, and the result shows high accuracy and visibility of the proposed approach.

Three-dimensional aerodynamic shape inverse design based on ISOMAP

The proper orthogonal decomposition (POD) is an efficient model order reduction method, which is frequently coupled with the discrete empirical interpolation method (DEIM) to solve nonlinear electromagnetic problems. A drawback of this method is that instabilities can occur related to the reduction operator of the nonlinear part. In this contribution, different DEIMs and the Gappy POD are employed and analyzed. Consecutively, the methods are employed to efficiently estimate the behavior of a permanent magnet synchronous machine in terms of global quantities, such as torque and iron losses.