Discrete Robust Principal Component Analysis via Binary Weights Self-Learning.

Principal component analysis (PCA) is widely used to extract features and reduce dimensionality in various computer vision and image/video processing tasks. Conventional approaches either lack robustness to outliers and corrupted data or are designed for one-dimensional signals. To address this problem, we propose a robust PCA model for two-dimensional images incorporating structured sparse priors, referred to as structured sparse 2D-PCA. This robust model considers the prior of structured and grouped pixel values in two dimensions. As the proposed formulation is jointly nonconvex and nonsmooth, which is difficult to tackle by joint optimization, we develop a two-stage alternating minimization approach to solve the problem. This approach iteratively learns the projection matrices by bidirectional decomposition and utilizes the proximal method to obtain the structured sparse outliers. By considering the structured sparsity prior, the proposed model becomes less sensitive to noisy data and outliers in two dimensions. Moreover, the computational cost indicates that the robust two-dimensional model is capable of processing quarter common intermediate format video in real time, as well as handling large-size images and videos, which is often intractable with other robust PCA approaches that involve image-to-vector conversion. Experimental results on robust face reconstruction, video background subtraction data set, and real-world videos show the effectiveness of the proposed model compared with conventional 2D-PCA and other robust PCA algorithms.

Principal Component Analysis (PCA) is a widely used dimensional reduction method that aims to find a low dimension sub space of highly correlated data for its major information to be used in further analysis. Machine learning methods based on PCA are popular in high dimensional data analysis, such as video and image processing. In video processing, the Robust PCA (RPCA), which is a modified method of the traditional PCA, has good properties in separating moving objects from the background, but it may have difficulties in separating those when light intensity of the background varies significantly in time. To overcome the difficulties, a modified PCA method, called Regression PCA (RegPCA), is proposed. The method is developed by combining the traditional PCA and regression approaches together, and it can be easily combined with RPCA for video processing. We focus the presentation of RegPCA with the combination of RPCA on video processing and find that it is more reliable than RPCA only. We use RegPCA to separate moving object from the background in a color video and get a better result than that given by RPCA. In the implementation, we first derive the explanatory variables by the background information. we then process a number of frames of the video and use those as a set of response variables. We remove the impact of the background by regressing the response against the explanatory variables by a regression model. The regression model provides a set of residuals, which can be further analyzed by RPCA. We compare the results of RegRPCA against those of RPCA only. It is evident that the moving objects can be completely removed from the background using our method but not in RPCA. Note that our result is based on a combination of RegPCA with RPCA. Our proposed method provides a new implementation of RPCA under the framework of regression approaches, which can be used to account for the impact of risk factors. This problem cannot be addressed by the application of RPCA only.

This paper presents a novel algorithm - robust kernel principal component analysis (robust KPCA), on the basis of the research of kernel principal component analysis (KPCA) and robust principal component analysis (RPCA). First, this algorithm sets the radius of the images of the training samples in the feature space using kernel tricks, then determines whether the samples are outliers or not, and finally analyzes the training samples which have eliminated the outliers using KPCA algorithm. The improved KPCA algorithm not only retains the non-linearity property of KPCA algorithm but also gets better robustness. Because the effects of outliers are eliminated, robust KPCA algorithm gets higher accuracy than KPCA algorithm for data analysis. The simulation experiments show that the robust KPCA algorithm developed is better than the KPCA algorithm.

Principal component analysis (PCA) is one of the most widely used unsupervised dimensionality reduction algorithms, but it is very sensitive to outliers because the squared l2 -norm is used as distance metric. Recently, many scholars have devoted themselves to solving this difficulty. They learn the projection matrix from minimum reconstruction error or maximum projection variance as the starting point, which leads them to ignore a serious problem, that is, the original PCA learns the projection matrix by minimizing the reconstruction error and maximizing the projection variance simultaneously, but they only consider one of them, which imposes various limitations on the performance of model. To solve this problem, we propose a novel robust principal component analysis via joint reconstruction and projection, namely, RPCA-RP, which combines reconstruction error and projection variance to fully mine the potential information of data. Furthermore, we carefully design a discrete weight for model to implicitly distinguish between normal data and outliers, so as to easily remove outliers and improve the robustness of method. In addition, we also unexpectedly discovered that our method has anomaly detection capabilities. Subsequently, an effective iterative algorithm is explored to solve this problem and perform related theoretical analysis. Extensive experimental results on several real-world datasets and RGB large-scale dataset demonstrate the superiority of our method.

Principal component analysis (PCA) is a favorite tool in chemometrics for data compression and information extraction. PCA finds linear combinations of the original measurement variables that describe the significant variations in the data. However, it is well-known that PCA, as with any other multivariate statistical method, is sensitive to outliers, missing data, and poor linear correlation between variables due to poorly distributed variables. As a result data transformations have a large impact upon PCA. In this regard one of the most powerful approaches to improve PCA appears to be the fuzzification of the matrix data, thus diminishing the influence of outliers. In this paper we discuss a robust fuzzy PCA algorithm (FPCA). The new algorithm is illustrated on a data set concerning interaction of carbon-hydrogen bonds with transition metal-oxo bonds in molybdenum complexes. Considering, for example, a two component model, FPCA accounts for 97.20% of the total variance and PCA accounts only for 69.75%.

RPCA (Robust Principal Component Analysis) recovers sparse and low rank components from the original observation data, RPCA commonly uses ADM (Alternate Direction Method) for solving, the efficiency of algorithm depends on the nuclear norm optimization solution, that is SVD. And the application of RPCA in computer vision, image and video and so on, large amounts of data, such as images and video, make it difficult for large-scale data SVD. In this paper, we used the random matrix algorithm to improve the SVD, respectively the Count Sketch algorithm, the Prototype Randomized k-SVD algorithm and the Faster Randomized k-SVD algorithm, The main idea is to reduce the size of the original large-scale data matrix and sample randomly. Using the random projection algorithm to obtain an approximation of the original matrix, and operate QR decomposition of the approximate matrix, get the unitary matrix corresponding to the approximate matrix, and do the corresponding SVD, finally we can get the results which is similar to the original matrix calculation. Obtaining approximation of the original data matrix. But the cost time and space have been greatly optimized. Simulation experiments based on single image and video foreground detection show that the proposed methods can greatly improve the efficiency of RPCA iterative optimization.

The analysis and understanding of video sequences is currently quite an active research field. Many applications such as video surveillance, optical motion capture or those of multimedia need to first be able to detect the objects moving in a scene filmed by a static camera. This requires the basic operation that consists of separating the moving objects called from the static information called background. Many background subtraction methods have been developed (Bouwmans et al. (2010); Bouwmans et al. (2008)). A recent survey (Bouwmans (2009)) shows that subspace learning models are well suited for background subtraction. Principal Component Analysis (PCA) has been used to model the background by significantly reducing the data's dimension. To perform PCA, different Robust Principal Components Analysis (RPCA) models have been recently developed in the literature. The background sequence is then modeled by a low rank subspace that can gradually change over time, while the moving foreground objects constitute the correlated sparse outliers. However, authors compare their algorithm only with the PCA (Oliver et al. (1999)) or another RPCA model. Furthermore, the evaluation is not made with the datasets and the measures currently used in the field of background subtraction. Considering all of this, we propose to evaluate RPCA models in the field of video-surveillance. Contributions of this chapter can be summarized as follows: 1) A survey regarding robust principal component analysis and 2) An evaluation and comparison on different video surveillance datasets

Robust principal component analysis (PCA) has been proved to be an efficient multivariate statistical method for extraction of a multivariate structure of geochemical data containing outlier values. Furthermore, because of the compositional characteristics of geochemical data, logratio transformation approaches also have to be implemented to transform the data from the simplex space into real space. In order to model the extracted principal component of transformed data, we used sequential Gaussian simulation (SGS) to overcome the drawbacks associated with the traditional kriging method, e.g. bias conditionally relevant to underestimation and overestimation, the smoothing effect and so on. The results of implementing robust PCA on both untransformed and logratio transformed data, which have been acquired from an orogenic gold deposit, indicate that robust PCA analysis of untransformed data is not capable of certifying element association of gold occurrences. However, in robust PCA analysis of transformed data, both varieties of gold mineralization, associated with pyrite mineral and quartz veins, have been revealed. The first variety of gold occurrence is detected in an integrated PC obtained from PC1, PC2 and PC3, which are associated with Pb, As, Hg, Mo, S and Sb elements in quartz–sulphide veins. The second one, PC4, could be correlated with quartz veins. Furthermore, the results of the geostatistical SGS approach are compatible with the detected anomaly in the Alut gold deposit.

Chemical exchange saturation transfer (CEST) MRI requires the acquisition of multiple saturation-weighted images and can last several minutes. Misalignments among these images, which are often due to the inevitable motion of the subject, will corrupt CEST contrast maps and result in large quantification errors. Therefore, the registration of the CEST series is critical. However, registration is challenging since common intensity-based registration algorithms may fail to differentiate CEST signals from motion artifacts. Herein, we studied how different patterns of motion affect CEST quantification and proposed a cascaded two-step registration scheme by utilizing features extracted from the entire Z-spectral image series instead of direct registration to a single image. The proposed approach is conducted in two stages: during the first coarse registration, the Z-spectral image series is decomposed by robust principal component analysis (RPCA) to separate CEST contrast from motion. The recomposed image series using only the low-rank component, which contains minimized motion, are averaged to generate a reference for the alignment of the image series. To further remove residual misalignments, the coarse registration is followed by a refinement stage, which uses PCA iteratively to generate motionless synthetic reference series with the first few principal components (PCs) that correspond to CEST contrast. In the end, the quality check is performed to exclude the images with unsuccessful registration. The proposed registration scheme (RPCA + PCA_R) was assessed by both phantom experiments and in vivo data of tumor-bearing mouse brain, with simulated random rigid motion in different patterns applied to the acquired static Z-spectral image series. For comparison, previous correction schemes using an explicit image [either S0 or Ssat(∆ω)] as registration reference were also performed, named as S0_R and Ssat_R respectively. To illustrate the advantage of combination of RPCA and PCA, registration was also exploited using either only the RPCA-based method (RPCA_R) or only the PCA-based one (PCA_R). Compared with the above four methods, RPCA + PCA_R allowed for more accurate correction of the corrupted Z-spectral images, exhibiting smaller MTRasym(∆ω) error maps and lower residual Z-spectra referring to the static data. Among all the five correction methods, the corrected Z-spectral image series by RPCA + PCA_R and the resulting MTRasym(∆ω) maps achieved the highest correlation coefficients (CC) with respect to the static ones. The registration scheme of RPCA + PCA_R provides robust motion correction between two specific Z-spectral images and among an entire image series, through extraction of the static component from the entire Z-spectra set and inclusion of a PCA-based refinement step. Therefore, this method can help improve CEST acquisition and quantification.

This paper presents a novel method for handling the effects of shape outliers in statistical shape analysis. Usually performed by a variant of classical principal component analysis (PCA), variability analysis may be highly affected by erroneous shapes. Principal components may thus imply aberrant modes, while eigenshapes may not accurately describe variability in a given set of shapes. Our robust analysis is performed using an elastic metric associated with the square-root velocity representation of shapes. This elastic shape analysis allows shape variability to be described with natural and intuitive deformations. The proposed method based on shape outlier detection applies the shape restoration procedure to rectify aberrant shapes. The resultant components are thus obtained from a tangent PCA on the restored database. By performing experiments based on MPEG-7 and HAND databases, we demonstrate that the proposed scheme is effective for shape variability analysis in the presence of outlying shapes. Our method is then compared with two existing schemes for robust data variability analysis: minimum covariance determinant-based PCA and projection pursuit-based PCA.

Solving optimization problems with sparse or low-rank optimal solutions has been an important topic since the recent emergence of compressed sensing and its matrix extensions such as the matrix rank minimization and robust principal component analysis problems. Compressed sensing enables one to recover a signal or image with fewer observations than the “length” of the signal or image, and thus provides potential breakthroughs in applications where data acquisition is costly. However, the potential impact of compressed sensing cannot be realized without efficient optimization algorithms that can handle extremely large-scale and dense data from real applications. Although the convex relaxations of these problems can be reformulated as either linear programming, second-order cone programming or semidefinite programming problems, the standard methods for solving these relaxations are not applicable because the problems are usually of huge size and contain dense data. In this dissertation, we give efficient algorithms for solving these “sparse” optimization problems and analyze the convergence and iteration complexity properties of these algorithms. Chapter 2 presents algorithms for solving the linearly constrained matrix rank minimization problem. The tightest convex relaxation of this problem is the linearly constrained nuclear norm minimization. Although the latter can be cast and solved as a semidefinite programming problem, such an approach is computationally expensive when the matrices are large. In Chapter 2, we propose fixed-point and Bregman iterative algorithms for solving the nuclear norm minimization problem and prove convergence of the first of these algorithms. By using a homotopy approach together with an approximate singular value decomposition procedure, we get a very fast, robust and powerful algorithm, which we call FPCA (Fixed Point Continuation with Approximate SVD), that can solve very large matrix rank minimization problems. Our numerical results on randomly generated and real matrix completion problems demonstrate that this algorithm is much faster and provides much better recoverability than semidefinite programming solvers such as SDPT3. For example, our algorithm can recover 1000 × 1000 matrices of rank 50 with a relative error of 10−5 in about 3 minutes by sampling only 20 percent of the elements. We know of no other method that achieves as good recoverability. Numerical experiments on online recommendation, DNA microarray data set and image inpainting problems demonstrate the effectiveness of our algorithms. In Chapter 3, we study the convergence/recoverability properties of the fixed-point continuation algorithm and its variants for matrix rank minimization. Heuristics for determining the rank of the matrix when its true rank is not known are also proposed. Some of these algorithms are closely related to greedy algorithms in compressed sensing. Numerical results for these algorithms for solving linearly constrained matrix rank minimization problems are reported. Chapters 4 and 5 considers alternating direction type methods for solving composite convex optimization problems. We present in Chapter 4 alternating linearization algorithms that are based on an alternating direction augmented Lagrangian approach for minimizing the sum of two convex functions. Our basic methods require at most O(1/e) iterations to obtain an e-optimal solution, while our accelerated (i.e., fast) versions require at most O(1/ 3 ) iterations, with little change in the computational effort required at each iteration. For more general problem, i.e., minimizing the sum of K convex functions, we propose multiple-splitting algorithms for solving them. We propose both basic and accelerated algorithms with O(1/e) and O(1/ 3 ) iteration complexity bounds for obtaining an e-optimal solution. To the best of our knowledge, the complexity results presented in these two chapters are the first ones of this type that have been given for splitting and alternating direction type methods. Numerical results on various applications in sparse and low-rank optimization, including compressed sensing, matrix completion, image deblurring, robust principal component analysis, are reported to demonstrate the efficiency of our methods.

We propose an adaptive sampling and reconstruction method based on the robust principal component analysis (PCA) to denoise Monte Carlo renderings. Addressing spike noise is a challenging problem in adaptive rendering methods. We adopt the robust PCA as a pre-processing step to efficiently decompose spike noise from rendered image after the image space is sampled. Then we leverage patch-based propagation filter for feature prefiltering and apply the robust PCA to reduce dimensionality in high-dimensional feature space. After that, we estimate a per-pixel pilot bandwidth derived from kernel density estimation and construct the multivariate local linear estimator in the reduced feature space to estimate the value of each pixel. Finally, we distribute additional ray samples in the regions with higher estimated mean squared error if sampling budget remains. We demonstrate that our method makes significant improvement in terms of both numerical error and visual quality compared to the state-of-the-art.

Principal Component Analysis (PCA) has been widely used for the representation of shape, appearance and motion. One drawback of typical PCA methods is that they are least squares estimation techniques and hence fail to account for "outliers" which are common in realistic training sets. In computer vision applications, outliers typically occur within a sample (image) due to pixels that are corrupted by noise, alignment errors, or occlusion. We review previous approaches for making PCA robust to outliers and present a new method that uses an intra-sample outlier process to account for pixel outliers. We develop the theory of Robust Principal Component Analysis (RPCA) and describe a robust M-estimation algorithm for learning linear multi-variate representations of high dimensional data such as images. Quantitative comparisons with traditional PCA and previous robust algorithms illustrate the benefits of RPCA when outliers are present. Details of the algorithm are described and a software implementation is being made publically available.

Active infrared thermography is one of the favourable non-destructive testing and evaluation methods popularly being used for remote inspection of various materials/products/components/structures. It captures the temperature distribution over the test material for predefined thermal stimulus onto the surface, which is further processed to detect the sub-surface anomalies/defects hidden inside the test object. Various attempts have been made by several research groups to reveal the hidden finer subsurface features with improved sensitivity and resolution. Present work highlights a principal component analysis and its extension, robust principal component analysis to inspect for sub-surface flat-bottomed hole defects inside a mild steel sample. Further, the proposed data analysis approaches and their capabilities have been compared on the temporal thermal experimental sequence for a frequency modulated incident thermal stimulus. It is clear from the obtained results that principal component analysis outperforms the robust principal component analysis in providing the information regarding the hidden defect details lying deep inside the material with enhanced signal to noise ratio leading to increased temperature contrast over the detected sub-surface defects.

Traditional robust principle component analysis (RPCA) has a high computational cost because RPCA needs to calculate the singular value decomposition of large matrices. To address this issue, this paper proposes a matrix-factorization-based RPCA (MFRPCA) model. MFRPCA has high computation efficiency while improving the robustness and flexibility of traditional RPCA using a non-convex low-rank approximation. Experiment results on challenging datasets demonstrate superior performance of MFRPCA compared with several advanced low-rank reconstruction methods.