Non-Greedy L21-Norm Maximization for Principal Component Analysis.

Principal Component Analysis (PCA) is a widely used dimensional reduction method that aims to find a low dimension sub space of highly correlated data for its major information to be used in further analysis. Machine learning methods based on PCA are popular in high dimensional data analysis, such as video and image processing. In video processing, the Robust PCA (RPCA), which is a modified method of the traditional PCA, has good properties in separating moving objects from the background, but it may have difficulties in separating those when light intensity of the background varies significantly in time. To overcome the difficulties, a modified PCA method, called Regression PCA (RegPCA), is proposed. The method is developed by combining the traditional PCA and regression approaches together, and it can be easily combined with RPCA for video processing. We focus the presentation of RegPCA with the combination of RPCA on video processing and find that it is more reliable than RPCA only. We use RegPCA to separate moving object from the background in a color video and get a better result than that given by RPCA. In the implementation, we first derive the explanatory variables by the background information. we then process a number of frames of the video and use those as a set of response variables. We remove the impact of the background by regressing the response against the explanatory variables by a regression model. The regression model provides a set of residuals, which can be further analyzed by RPCA. We compare the results of RegRPCA against those of RPCA only. It is evident that the moving objects can be completely removed from the background using our method but not in RPCA. Note that our result is based on a combination of RegPCA with RPCA. Our proposed method provides a new implementation of RPCA under the framework of regression approaches, which can be used to account for the impact of risk factors. This problem cannot be addressed by the application of RPCA only.

Principal component analysis (PCA) is one of the most widely used unsupervised dimensionality reduction algorithms, but it is very sensitive to outliers because the squared l2 -norm is used as distance metric. Recently, many scholars have devoted themselves to solving this difficulty. They learn the projection matrix from minimum reconstruction error or maximum projection variance as the starting point, which leads them to ignore a serious problem, that is, the original PCA learns the projection matrix by minimizing the reconstruction error and maximizing the projection variance simultaneously, but they only consider one of them, which imposes various limitations on the performance of model. To solve this problem, we propose a novel robust principal component analysis via joint reconstruction and projection, namely, RPCA-RP, which combines reconstruction error and projection variance to fully mine the potential information of data. Furthermore, we carefully design a discrete weight for model to implicitly distinguish between normal data and outliers, so as to easily remove outliers and improve the robustness of method. In addition, we also unexpectedly discovered that our method has anomaly detection capabilities. Subsequently, an effective iterative algorithm is explored to solve this problem and perform related theoretical analysis. Extensive experimental results on several real-world datasets and RGB large-scale dataset demonstrate the superiority of our method.

Principal component analysis (PCA) is widely used to extract features and reduce dimensionality in various computer vision and image/video processing tasks. Conventional approaches either lack robustness to outliers and corrupted data or are designed for one-dimensional signals. To address this problem, we propose a robust PCA model for two-dimensional images incorporating structured sparse priors, referred to as structured sparse 2D-PCA. This robust model considers the prior of structured and grouped pixel values in two dimensions. As the proposed formulation is jointly nonconvex and nonsmooth, which is difficult to tackle by joint optimization, we develop a two-stage alternating minimization approach to solve the problem. This approach iteratively learns the projection matrices by bidirectional decomposition and utilizes the proximal method to obtain the structured sparse outliers. By considering the structured sparsity prior, the proposed model becomes less sensitive to noisy data and outliers in two dimensions. Moreover, the computational cost indicates that the robust two-dimensional model is capable of processing quarter common intermediate format video in real time, as well as handling large-size images and videos, which is often intractable with other robust PCA approaches that involve image-to-vector conversion. Experimental results on robust face reconstruction, video background subtraction data set, and real-world videos show the effectiveness of the proposed model compared with conventional 2D-PCA and other robust PCA algorithms.

Introduction: Saffron is one of the most important economic plants in the Khorasan province. Awareness of soil quality in agricultural lands is essential for the best management of lands and for obtaining maximum economic benefit. In general, plant growth is a function of environmental factors especially chemical and physical properties of soil (20). It has been demonstrated that there was a positive and high correlation between soil organic matter and saffron yield. Increasing the yield of saffron due to organic matter is probably due to soil nutrient, especially phosphorous and nitrogen and also improvement of soil physical quality (6, 28, 29). The yield of saffron in soils with high nitrogen as a result of vegetative growth is high (8). Shahandeh (6) found that most of the variation of saffron yield depends on soil properties. Due to the economic importance of saffron and the role of soil properties on saffron yield, this research was conducted to find the relationship between saffron yield and some soil physical and chemical properties, and to determine the contribution of soil properties that have the greatest impact on saffron yield in the Ghayenat area. Materials and Methods: This research was performed in 30 saffron fields (30 soil samples) of the Ghayenat area (longitude 59° 10΄ 10.37˝ - 59° 11΄ 38.41˝ and latitude 33° 43΄ 35.08˝ - 33΄ 44΄ 02.78˝), which is located in the Khrasan province of Iran. In this research, 21 soil properties were regarded as the total data set (TDS). Then the principal component analysis (PCA) was used to determine the most important soil properties affecting saffron yield as a minimum data set (MDS) and the stepwise regression to estimate saffron yield. To estimate the yield of saffron in stepwise regression method, saffron yield was considered as a dependent variable and soil physical and chemical properties were considered to be independent variables. Results and Discussion: According to the PCA method, among the 21 studied properties, 7 out of them including calcium, iron, zinc contents, sand, calcium carbonate equivalent percent, mean weight diameter of aggregates (MWD) and manganese (Mn) had the higher Eigenvalues. Therefore, the above properties were introduced as the most important soil properties in saffron fields. Calcium carbonate had the negative effect on the availability of micro-nutrients (26). Christensen et al. (15) found that by increasing the calcium carbonate in soil due to high pH and formation of insoluble components, the uptake of micro-nutrients is especially limited. The results of stepwise regression method (equation 1) showed that soil acidity (pH), zinc content, bulk density, MWD, iron content, salinity (EC), organic carbon and available potassium in soil were the most important properties that affect the yield of saffron, so that the determination coefficient (R2) of the regression model was high (Table 2) and it can explain 74% of the variation of saffron yield. Y = 6924.51 – 1187.31 pH – 89.65 EC + 71.6 Fe – 826.02 Zn + 471.55 OC, + 5490.96 K + 1353.56 BD + 752.82 MWD (1) where Y: saffron yield (kgha-1), pH: soil acidity, EC: electoral conductivity (dSm-1), Fe: iron concentration (mgkg-1), Zn: zinc concentration (mgkg-1), OC: organic carbon (%), K: soil potassium (%), BD: soil bulk density (Mgm-3), and MWD: mean weight diameter of aggregates (MM). Based on the absolute values of standard s in the regression model (Table 3), pH value and then after Zn concentration had the most effect on saffron yield. In general, responses of different plants to soil pH is varied, and saffron grows satisfactory in pH = 7.8 (5). Soil pH influences the uptake of soil nutrients by plants (15), so that this parameter had the most effect on saffron yield and by increasing the soil pH, the yield of saffron decreases. According to the regression model, Zn concentration was the second parameter in saffron yield. Zn has the important role in structure of plant enzymes (30). After these 2 parameters, Bd, MWD, Fe concentration, EC, Organic carbon and K concentration in soil had more effect on saffron yield (Table 3). Conclusion: According to both PCA and regression methods, the concentration of iron and zinc and MWD were determined as the important and effective soil properties on saffron yield in the Ghayenat area. In addition, soil pH in stepwise regression method and calcium carbonate in PCA method were determined as the effective properties on saffron yield. Therefore, it is suggested that the parameters of Zn, Fe, and MWD along with soil pH and calcium carbonate which were regarded individually in two methods, were considered as the most soil properties in saffron yield.

Mangroves are coastal plants influenced by tidal cycles. One of the regions in South Sumatra Province with a mangrove ecosystem is Sembilang National Park. As a national park, this location is strategically positioned for research related to mangrove species. The utilization of the Principal Component Analysis (PCA) method is considered to enhance the capabilities of remote sensing data in mangrove mapping. However, its application has been limited to the mangrove level. The research objective is to identify mangrove species in Sembilang National Park using the PCA method by leveraging Landsat-8 image data acquired on September 9, 2019. Mangrove distribution is obtained through the Mangrove Vegetation Index (MVI) and vector data from Global Mangrove Watch. The image is then overlaid with field species data to obtain species spectral patterns. Additionally, the correlation between spectral band values and eigenvalues (PC) is analyzed to detect parameters correlated with eigenvalues. The research results show that PC values correlate with mangrove species and can be used for mangrove species identification. This is demonstrated by Bruguiera Gymnorrhiza species with canopy coverage of 60.8% and 62.46% at ST7 and ST8 having the same PC values, while mangroves with canopy coverage of 62.5% in different species have different PC values. Moreover, the PCA method can indicate two crucial factors in identifying mangrove species, namely vegetation and soil factors. Mangrove merupakan tumbuhan pesisir yang dipengaruhi oleh pasang surut. Salah satu wilayah di Provinsi Sumatera Selatan yang memiliki ekosistem mangrove adalah Taman Nasional Sembilang. Sebagai taman nasional, lokasi ini sangat strategis untuk dilakukan penelitian terkait spesies mangrove. Pemanfaatan metode principal component analysis (PCA) dinilai mampu meningkatkan kemampuan data penginderaan jauh dalam pemetaan mangrove. Namun selama ini, pemanfaatan terbatas pada level mangrove. Adapun tujuan penelitian adalah mengidentifikasi spesies mangrove di Taman Nasional Sembilang menggunakan metode PCA dengan memanfaatkan data citra Landsat-8 yang diakusisi pada 09 September 2019. Sebaran mangrove diperoleh melalui indeks vegetasi mangrove (MVI) dan data vektor dari Global Mangrove Watch. Citra tersebut kemudian di overlay dengan data spesies lapangan untuk mendapatkan pola spektral species. Selain itu, korelasi nilai spektral band dan eigen (PC) dianalisis untuk mendeteksi parameter yang berkorelasi dengan nilai eigen. Hasil penelitian menunjukkan nilai PC memiliki korelasi dengan spesies mangrove dan dapat digunakan untuk mengidentifikasi spesies mangrove. Hal ini ditunjukkan spesies Bruguiera Gymnorrhiza dengan tutupan kanopi 60,8% dan 62,46% pada ST7 dan ST8 memiliki nilai PC yang sama serta mangrove dengan besaran tutupan kanopi 62,5% pada spesies nilai PC berbeda. Selain itu, metode PCA mampu menunjukkan dua faktor penting dalam mengidentifikasi spesies mangrove, yaitu faktor vegetasi dan tanah.

The development of artificial neural networks is related to statistical and biometric analysis which is one of the applications that can require artificial neural network models. Recognition of facial patterns is an important part of identifying a person. The face can be divided into areas such as the nose, eyes and mouth. Face pattern recognition is a research area that can be applied to the principal component analysis (PCA) method. The training process carried out by the eigenface calculation uses PCA and the results of this study show that facial pattern recognition based on the proportion of memorization and generalization for the use of the method without PCA is better than facial pattern recognition using PCA. Pattern recognition without using the PCA method, the level of memorization and generalization reaches 100% at the 40th iteration and 0.0099 error with a learning rate and momentum of 0.8, while facial pattern recognition using the PCA method, the memorization and generalization level reaches 100% in the iteration. to -1000 and error 0.00103 with learning rate and momentum 0.9.

Systematic principal component analysis (PCA) methods are presented in this paper for reliable islanding detection for power systems with significant penetration of distributed generations (DGs), where synchrophasors, recorded by phasor measurement units, are used for system monitoring. Existing islanding detection methods, such as rate-of-change-of-frequency and vector shift are fast for processing local information; however, with the growth in installed capacity of DGs, they suffer from several drawbacks. Incumbent genset islanding detection cannot distinguish a system-wide disturbance from an islanding event, leading to maloperation. The problem is even more significant when the grid does not have sufficient inertia to limit frequency divergences in the system fault/stress due to the high penetration of DGs. To tackle such problems, this paper introduces PCA methods for islanding detection. A simple control chart is established for intuitive visualization of the transients. A recursive PCA scheme is proposed as a reliable extension of the PCA method to reduce the false alarms for time-varying process. To further reduce the computational burden, the approximate linear dependence condition errors are calculated to update the associated PCA model. The proposed PCA and RPCA methods are verified by detecting abnormal transients occurring in the U.K. utility network.

This paper presents a novel Gabor-based kernel Principal Component Analysis (PCA) method by integrating the Gabor wavelet representation of face images and the kernel PCA method for face recognition. Gabor wavelets first derive desirable facial features characterized by spatial frequency, spatial locality, and orientation selectivity to cope with the variations due to illumination and facial expression changes. The kernel PCA method is then extended to include fractional power polynomial models for enhanced face recognition performance. A fractional power polynomial, however, does not necessarily define a kernel function, as it might not define a positive semidefinite Gram matrix. Note that the sigmoid kernels, one of the three classes of widely used kernel functions (polynomial kernels, Gaussian kernels, and sigmoid kernels), do not actually define a positive semidefinite Gram matrix either. Nevertheless, the sigmoid kernels have been successfully used in practice, such as in building support vector machines. In order to derive real kernel PCA features, we apply only those kernel PCA eigenvectors that are associated with positive eigenvalues. The feasibility of the Gabor-based kernel PCA method with fractional power polynomial models has been successfully tested on both frontal and pose-angled face recognition, using two data sets from the FERET database and the CMU PIE database, respectively. The FERET data set contains 600 frontal face images of 200 subjects, while the PIE data set consists of 680 images across five poses (left and right profiles, left and right half profiles, and frontal view) with two different facial expressions (neutral and smiling) of 68 subjects. The effectiveness of the Gabor-based kernel PCA method with fractional power polynomial models is shown in terms of both absolute performance indices and comparative performance against the PCA method, the kernel PCA method with polynomial kernels, the kernel PCA method with fractional power polynomial models, the Gabor wavelet-based PCA method, and the Gabor wavelet-based kernel PCA method with polynomial kernels.

Robust principal component analysis (RPCA) is a technique that aims to make principal component analysis (PCA) robust to noise samples. The current modeling approaches of RPCA were proposed by analyzing the prior distribution of the reconstruction error terms. However, these methods ignore the influence of samples with large reconstruction errors, as well as the valid information of these samples in principal component space, which will degrade the ability of PCA to extract the principal component of data. In order to solve this problem, Fuzzy sparse deviation regularized robust principal component Analysis (FSD-PCA) is proposed in this paper. First, FSD-PCA learns the principal components by minimizing the square of l2 -norm-based reconstruction error. Then, FSD-PCA introduces sparse deviation on reconstruction error term to relax the samples with large bias, thus FSD-PCA can process noise and principal components of samples separately as well as improve the ability of FSD-PCA for retaining the principal component information. Finally, FSD-PCA estimates the prior probability of each sample by fuzzy weighting based on the relaxed reconstruction error, which can improve the robustness of the model. The experimental results indicate that the proposed model performs excellent robustness against different types of noise than the state-of-art algorithms, and the sparse deviation term enables FSD-PCA to process noise information and principal component information separately, so FSD-PCA can filter the noise information of an image and restore the corrupted image.

Solving optimization problems with sparse or low-rank optimal solutions has been an important topic since the recent emergence of compressed sensing and its matrix extensions such as the matrix rank minimization and robust principal component analysis problems. Compressed sensing enables one to recover a signal or image with fewer observations than the “length” of the signal or image, and thus provides potential breakthroughs in applications where data acquisition is costly. However, the potential impact of compressed sensing cannot be realized without efficient optimization algorithms that can handle extremely large-scale and dense data from real applications. Although the convex relaxations of these problems can be reformulated as either linear programming, second-order cone programming or semidefinite programming problems, the standard methods for solving these relaxations are not applicable because the problems are usually of huge size and contain dense data. In this dissertation, we give efficient algorithms for solving these “sparse” optimization problems and analyze the convergence and iteration complexity properties of these algorithms. Chapter 2 presents algorithms for solving the linearly constrained matrix rank minimization problem. The tightest convex relaxation of this problem is the linearly constrained nuclear norm minimization. Although the latter can be cast and solved as a semidefinite programming problem, such an approach is computationally expensive when the matrices are large. In Chapter 2, we propose fixed-point and Bregman iterative algorithms for solving the nuclear norm minimization problem and prove convergence of the first of these algorithms. By using a homotopy approach together with an approximate singular value decomposition procedure, we get a very fast, robust and powerful algorithm, which we call FPCA (Fixed Point Continuation with Approximate SVD), that can solve very large matrix rank minimization problems. Our numerical results on randomly generated and real matrix completion problems demonstrate that this algorithm is much faster and provides much better recoverability than semidefinite programming solvers such as SDPT3. For example, our algorithm can recover 1000 × 1000 matrices of rank 50 with a relative error of 10−5 in about 3 minutes by sampling only 20 percent of the elements. We know of no other method that achieves as good recoverability. Numerical experiments on online recommendation, DNA microarray data set and image inpainting problems demonstrate the effectiveness of our algorithms. In Chapter 3, we study the convergence/recoverability properties of the fixed-point continuation algorithm and its variants for matrix rank minimization. Heuristics for determining the rank of the matrix when its true rank is not known are also proposed. Some of these algorithms are closely related to greedy algorithms in compressed sensing. Numerical results for these algorithms for solving linearly constrained matrix rank minimization problems are reported. Chapters 4 and 5 considers alternating direction type methods for solving composite convex optimization problems. We present in Chapter 4 alternating linearization algorithms that are based on an alternating direction augmented Lagrangian approach for minimizing the sum of two convex functions. Our basic methods require at most O(1/e) iterations to obtain an e-optimal solution, while our accelerated (i.e., fast) versions require at most O(1/ 3 ) iterations, with little change in the computational effort required at each iteration. For more general problem, i.e., minimizing the sum of K convex functions, we propose multiple-splitting algorithms for solving them. We propose both basic and accelerated algorithms with O(1/e) and O(1/ 3 ) iteration complexity bounds for obtaining an e-optimal solution. To the best of our knowledge, the complexity results presented in these two chapters are the first ones of this type that have been given for splitting and alternating direction type methods. Numerical results on various applications in sparse and low-rank optimization, including compressed sensing, matrix completion, image deblurring, robust principal component analysis, are reported to demonstrate the efficiency of our methods.

Dimensionality reduction methods play an important role in face recognition. Principal component analysis (PCA) and two-dimensional principal component analysis (2DPCA) are two kinds of important methods in this field. Recent research seems like that 2DPCA method is superior to PCA method. To prove if this conclusion is always true, a comprehensive comparison study between PCA and 2DPCA methods was carried out. A novel concept, called column-image difference (CID), was proposed to analyze the difference between PCA and 2DPCA methods in theory. It is found that there exist some restrictive conditions when 2DPCA outperforms PCA. After theoretical analysis, the experiments were conducted on four famous face image databases. The experiment results confirm the validity of theoretical claim.

Chapter 19 - Dimensionality Reduction and Latent Variable Modeling

Biometric recognition systems or human identification are very important in security access for identification and verification systems. The biometric recognition system can be used as an identification system based on the characteristics possessed by the body part of each individual. The soles of the feet can be used for identification because the soles of the feet have certain and unique characteristics which include major lines, protrusions, small dots, single points, and textures. The introduction of biometrics in babies is still conventional, which is a standard operating procedure such as attaching bracelets on baby's feet and imprinting or inking on the soles of baby's feet which are affixed to paper and are very vulnerable to the risk of damage or loss of data, there is a need for a system that can store data automatically digital and able to do the baby identification process. The Principal Component Analysis method is used for the extraction process of the characteristics of the baby's feet. The classification uses the K-Nearest Neighbor (K-NN) method with the euclidean distance approach. Tests were carried using 120 images of baby feet, there are 20 classes, each class contains 3 images of the right foot and 3 images of the foot of the left foot, and a dataset of 280 training images. The highest accuracy result obtained in system testing is 91% with a computation time of 5.63 seconds using the Principal Component Analysis method with the K-Nearest Neighbor (K-NN) classification.Keywords: Footprint, Feature Extraction, Principal Component Analysis, K-Nearest Neighbor.

Satellite microwave emission mixed with signals from active sensors is referred to as radio-frequency interference (RFI). RFI affects greatly the quality of data and retrieval products from space-borne microwave radiometry. An accurate RFI detection will not only enhance geophysical retrievals over land but also provide evidence of the much-needed protection of the microwave frequency band for satellite remote sensing technologies. It is difficult to detect RFI from space-borne microwave radiometer data over winter land, because RFI signals are usually mixed with snow in mid-high latitudes. A modified principal component analysis (PCA) method is proposed in this paper for detecting microwave low frequency RFI signals. Only three original variables, one RFI index (sensitive to RFI signal) and two scattering indices (sensitive to snow scattering), are included in the vector for principal component analysis in this modified method instead of the nine or seven RFI index original variables used in a normal PCA algorithm. The principal component with higher correlation and contribution to the original RFI index is the RFI-related principal component. In the absence of a reliable validation data set of the “true” RFI, the consistency in the identified RFI distribution obtained from this method compared to other independent methods, such as the spectral difference method, the normalized PCA method, and the double PCA method, give confidence to the RFI signals’ identification over land. The simple and reliable modified PCA method could successfully detect RFI not only in summer but also in winter AMSR-E data.

Head-related transfer function (HRTF) plays an important role in the construction of 3D auditory display. This paper presents an individual HRTF modeling method using deep neural networks based on spatial principal component analysis. The HRTFs are represented by a small set of spatial principal components combined with frequency and individual-dependent weights. By estimating the spatial principal components using deep neural networks and mapping the corresponding weights to a quantity of anthropometric parameters, we predict individual HRTFs in arbitrary spatial directions. The objective and subjective experiments evaluate the HRTFs generated by the proposed method, the principal component analysis (PCA) method, and the generic method. The results show that the HRTFs generated by the proposed method and PCA method perform better than the generic method. For most frequencies the spectral distortion of the proposed method is significantly smaller than the PCA method in the high frequencies but significantly larger in the low frequencies. The evaluation of the localization model shows the PCA method is better than the proposed method. The subjective localization experiments show that the PCA and the proposed methods have similar performances in most conditions. Both the objective and subjective experiments show that the proposed method can predict HRTFs in arbitrary spatial directions.