An Application of Robust Principal Component Analysis Methods for Anomaly Detection

The robust principal component analysis (PCA) method has shown very promising results in seismic ambient noise attenuation when dealing with outliers in the data. However, the model assumes a general Gaussian distribution plus sparse outliers for the noise. In seismic data however, the noise standard variation could vary from one place to another leading to a more heavy-tailed noise distribution. In this paper, we present a new method which solves a convex minimisation problem of the robust PCA method with an M-estimate penalty function. Our empirical results show that the proposed method can outperform the robust PCA method.

Robust principal component analysis via optimal mean by joint ℓ2,1 and Schatten p-norms minimization

Principal component analysis (PCA) is one of the most popular dimension reduction methods. The usual PCA is known to be sensitive to the presence of outliers, and thus many robust PCA methods have been developed. Among them, the Tyler's M-estimator is shown to be the most robust scatter estimator under the elliptical distribution. However, when the underlying distribution is contaminated and deviates from ellipticity, Tyler's M-estimator might not work well. In this article, we apply the semiparametric theory to propose a robust semiparametric PCA. The merits of our proposal are twofold. First, it is robust to heavy-tailed elliptical distributions as well as robust to non-elliptical outliers. Second, it pairs well with a data-driven tuning procedure, which is based on active ratio and can adapt to different degrees of data outlyingness. Theoretical properties are derived, including the influence functions for various statistical functionals and asymptotic normality. Simulation studies and a data analysis demonstrate the superiority of our method.

Numerous studies have demonstrated the potential of robust principal component analysis (RPCA) in image recovery. However, conventional RPCA methods for color image recovery apply RPCA independently to each color channel, which ignores the correlation information of the red, green and blue channels. To improve the performance of RPCA-based methods and draw inspiration from the success of quaternion representation in color image processing, we propose an improved quaternion RPCA (IQRPCA) method for color image recovery. The IQRPCA method treats all color channels holistically and considers the correlation information of different color channels naturally. In addition, we have developed a quaternion nuclear norm known as improved quaternion Cauchy nuclear norm, which produces an even more effective and robust approach to color image recovery task. Compared to the RPCA method in the quaternion setting, the IQRPCA method treats the singular values differently, which shows better recovery performance than competing methods. Furthermore, we provide a convergence analysis of the proposed method. Our experiments confirm the effectiveness of IQRPCA in the application of color image recovery.

In this letter, we first present a rank-revealing matrix factorization algorithm by using randomization called randomized truncated pivoted QLP (RTp-QLP) to approximate an input matrix. For a dense and large $n_1\times n_2$ matrix with numerical rank $k$ , RTp-QLP needs only a few passes over the matrix (regardless of $k$ ) and $\mathcal {O}(n_1n_2d)$ floating-point operations, where $d$ is much smaller than both $n_1$ and $n_2$ . Next, we develop a robust principal component analysis (RPCA) method by utilizing RTp-QLP. In addition, we propose a rank estimation technique that efficiently solves the RPCA task. RTp-QLP is highly accurate and numerically stable. Our proposed RTp-QLP-based RPCA method yields the optimal solution, and it is faster than existing methods. Our simulation results support our claims.

Recent advances in optical coherence tomography (OCT) lead to the development of OCT angiography to provide additional helpful information for diagnosis of diseases like basal cell carcinoma. In this paper, we investigate how to extract blood vessels of human skin from full-field OCT (FF-OCT) data using the robust principal component analysis (RPCA) technique. Specifically, we propose a short-time RPCA method that divides the FF-OCT data into segments and decomposes each segment into a low-rank structure representing the relatively static tissues of human skin and a sparse matrix representing the blood vessels. The method mitigates the problem associated with the slow-varying background and is free of the detection error that RPCA may have when dealing with FF-OCT data. Both short-time RPCA and RPCA methods can extract blood vessels from FF-OCT data with heavy speckle noise, but the former takes only half the computation time of the latter. We evaluate the performance of the proposed method by comparing the extracted blood vessels with the ground truth vessels labeled by a dermatologist and show that the proposed method works equally well for FF-OCT volumes of different quality. The average F-measure improvements over the correlation-mapping OCT method, the modified amplitude-decorrelation OCT angiography method, and the RPCA method, respectively, are 0.1835, 0.1032, and 0.0458.

Principal Component Analysis (PCA) is a widely used dimensional reduction method that aims to find a low dimension sub space of highly correlated data for its major information to be used in further analysis. Machine learning methods based on PCA are popular in high dimensional data analysis, such as video and image processing. In video processing, the Robust PCA (RPCA), which is a modified method of the traditional PCA, has good properties in separating moving objects from the background, but it may have difficulties in separating those when light intensity of the background varies significantly in time. To overcome the difficulties, a modified PCA method, called Regression PCA (RegPCA), is proposed. The method is developed by combining the traditional PCA and regression approaches together, and it can be easily combined with RPCA for video processing. We focus the presentation of RegPCA with the combination of RPCA on video processing and find that it is more reliable than RPCA only. We use RegPCA to separate moving object from the background in a color video and get a better result than that given by RPCA. In the implementation, we first derive the explanatory variables by the background information. we then process a number of frames of the video and use those as a set of response variables. We remove the impact of the background by regressing the response against the explanatory variables by a regression model. The regression model provides a set of residuals, which can be further analyzed by RPCA. We compare the results of RegRPCA against those of RPCA only. It is evident that the moving objects can be completely removed from the background using our method but not in RPCA. Note that our result is based on a combination of RegPCA with RPCA. Our proposed method provides a new implementation of RPCA under the framework of regression approaches, which can be used to account for the impact of risk factors. This problem cannot be addressed by the application of RPCA only.

Low-rank matrix recovery algorithms aim to recover a corrupted low-rank matrix with sparse errors. However, corrupted errors may not be sparse in real-world problems and the relationship between ℓ1 regularizer on noise and robust M-estimators is still unknown. This paper proposes a general robust framework for low-rank matrix recovery via implicit regularizers of robust M-estimators, which are derived from convex conjugacy and can be used to model arbitrarily corrupted errors. Based on the additive form of half-quadratic optimization, proximity operators of implicit regularizers are developed such that both low-rank structure and corrupted errors can be alternately recovered. In particular, the dual relationship between the absolute function in ℓ1 regularizer and Huber M-estimator is studied, which establishes a connection between robust low-rank matrix recovery methods and M-estimators based robust principal component analysis methods. Extensive experiments on synthetic and real-world data sets corroborate our claims and verify the robustness of the proposed framework.

Highlight removal for endoscopic images based on accelerated adaptive non-convex RPCA decomposition

In this letter, we propose an unsupervised framework for speech noise reduction based on the recent development of low-rank and sparse matrix decomposition. The proposed framework directly separates the speech signal from noisy speech by decomposing the noisy speech spectrogram into three submatrices: the noise structure matrix, the clean speech structure matrix, and the residual noise matrix. Evaluations on the Noisex-92 dataset show that the proposed method achieves a signal-to-distortion ratio approximately 2.48 dB and 3.23 dB higher than that of the robust principal component analysis method and the non-negative matrix factorization method, respectively, when the input SNR is -5 dB.

Matrix low rank approximation including the classical PCA and the robust PCA (RPCA) method have been applied to solve the background modeling problem in video analysis. Recently, it has been demonstrated that a special weighted low rank approximation of matrices can be made robust to the outliers similar to the $\ell_1$-norm in RPCA method. In this work, we propose a new algorithm that can speed up the existing algorithm for solving the special weighted low rank approximation and demonstrate its use in background estimation problem.

Background. Modern methods for determining the coordinates of elements of wireless sensor networks allow solving the problems of determining the mutual distances between the elements of a wireless network under the assumption that the errors in measuring the mutual distances between network elements are distributed according to the normal law. With increasing requirements for the accuracy of determining coordinates, these methods do not allow solving the problem.
 Objective. The purpose of the paper is to improve the accuracy of determining the coordinates of elements of wireless sensor networks by using robust estimation methods.
 Methods. Determination of coordinates of elements of wireless sensor networks is implemented on the basis of two robust methods.
 The first is the use of a median estimate based on multiple measurements of the mutual distances between elements to determine their coordinates.
 The second is based on multiple measurements of the mutual distances between elements, determining their coordinates based on the Huber influence function and comparing two robust methods.
 Results. The use of a robust method based on the Huber influence function makes it possible to increase the accuracy of determining the coordinates of elements of wireless sensor networks by 5-10% compared to classical estimation methods.
 Conclusions. The proposed robust approach to determining the coordinates of elements of wireless sensor networks can be implemented in modern ground-based sensor networks for various purposes.
 Keywords: wireless sensor network; determination of coordinates of elements of wireless sensor networks; robust estimation methods.

Principal component analysis (PCA) is a typical unsupervised dimensionality reduction algorithm, and one of its important weaknesses is that the squared l2 -norm cannot overcome the influence of outliers. Existing robust PCA methods based on paradigm have the following two drawbacks. First, the objective function of PCA based on the l1 -norm has no rotational invariance and limited robustness to outliers, and its solution mostly uses a greedy search strategy, which is expensive. Second, the robust PCA based on the l2,1 -norm and the l2,p -norm is essential to learn probability weights for data, which only weakens the influence of outliers on the learning projection matrix and cannot be completely eliminated. Moreover, the ability to detect anomalies is also very poor. To solve these problems, we propose a novel discrete robust principal component analysis (DRPCA). Through self-learning binary weights, the influence of outliers on the projection matrix and data center estimation can be completely eliminated, and anomaly detection can be directly performed. In addition, an alternating iterative optimization algorithm is designed to solve the proposed problem and realize the automatic update of binary weights. Finally, our proposed model is successfully applied to anomaly detection applications, and experimental results demonstrate that the superiority of our proposed method compared with the state-of-the-art methods.

Principal Component Analysis (PCA) is one of the most important unsupervised methods to handle high-dimensional data. However, due to the high computational complexity of its eigen-decomposition solution, it is hard to apply PCA to the large-scale data with high dimensionality, e.g., millions of data points with millions of variables. Meanwhile, the squared L2-norm based objective makes it sensitive to data outliers. In recent research, the L1-norm maximization based PCA method was proposed for efficient computation and being robust to outliers. However, this work used a greedy strategy to solve the eigenvectors. Moreover, the L1-norm maximization based objective may not be the correct robust PCA formulation, because it loses the theoretical connection to the minimization of data reconstruction error, which is one of the most important intuitions and goals of PCA. In this paper, we propose to maximize the L21-norm based robust PCA objective, which is theoretically connected to the minimization of reconstruction error. More importantly, we propose the efficient non-greedy optimization algorithms to solve our objective and the more general L21-norm maximization problem with theoretically guaranteed convergence. Experimental results on real world data sets show the effectiveness of the proposed method for principal component analysis.

BackgroundHigh throughput RNA sequencing is a powerful approach to study gene expression. Due to the complex multiple-steps protocols in data acquisition, extreme deviation of a sample from samples of the same treatment group may occur due to technical variation or true biological differences. The high-dimensionality of the data with few biological replicates make it challenging to accurately detect those samples, and this issue is not well studied in the literature currently. Robust statistics is a family of theories and techniques aim to detect the outliers by first fitting the majority of the data and then flagging data points that deviate from it. Robust statistics have been widely used in multivariate data analysis for outlier detection in chemometrics and engineering. Here we apply robust statistics on RNA-seq data analysis.ResultsWe report the use of two robust principal component analysis (rPCA) methods, PcaHubert and PcaGrid, to detect outlier samples in multiple simulated and real biological RNA-seq data sets with positive control outlier samples. PcaGrid achieved 100% sensitivity and 100% specificity in all the tests using positive control outliers with varying degrees of divergence. We applied rPCA methods and classical principal component analysis (cPCA) on an RNA-Seq data set profiling gene expression of the external granule layer in the cerebellum of control and conditional SnoN knockout mice. Both rPCA methods detected the same two outlier samples but cPCA failed to detect any. We performed differentially expressed gene detection before and after outlier removal as well as with and without batch effect modeling. We validated gene expression changes using quantitative reverse transcription PCR and used the result as reference to compare the performance of eight different data analysis strategies. Removing outliers without batch effect modeling performed the best in term of detecting biologically relevant differentially expressed genes.ConclusionsrPCA implemented in the PcaGrid function is an accurate and objective method to detect outlier samples. It is well suited for high-dimensional data with small sample sizes like RNA-seq data. Outlier removal can significantly improve the performance of differential gene detection and downstream functional analysis.