Application of Principal Component Analysis with High-Order Truncated Gradient Ascent Method in Data Dimension Reduction

Principal Component Analysis (PCA) is a widely used dimensional reduction method that aims to find a low dimension sub space of highly correlated data for its major information to be used in further analysis. Machine learning methods based on PCA are popular in high dimensional data analysis, such as video and image processing. In video processing, the Robust PCA (RPCA), which is a modified method of the traditional PCA, has good properties in separating moving objects from the background, but it may have difficulties in separating those when light intensity of the background varies significantly in time. To overcome the difficulties, a modified PCA method, called Regression PCA (RegPCA), is proposed. The method is developed by combining the traditional PCA and regression approaches together, and it can be easily combined with RPCA for video processing. We focus the presentation of RegPCA with the combination of RPCA on video processing and find that it is more reliable than RPCA only. We use RegPCA to separate moving object from the background in a color video and get a better result than that given by RPCA. In the implementation, we first derive the explanatory variables by the background information. we then process a number of frames of the video and use those as a set of response variables. We remove the impact of the background by regressing the response against the explanatory variables by a regression model. The regression model provides a set of residuals, which can be further analyzed by RPCA. We compare the results of RegRPCA against those of RPCA only. It is evident that the moving objects can be completely removed from the background using our method but not in RPCA. Note that our result is based on a combination of RegPCA with RPCA. Our proposed method provides a new implementation of RPCA under the framework of regression approaches, which can be used to account for the impact of risk factors. This problem cannot be addressed by the application of RPCA only.

Recently, many dimensionality reduction algorithms, including local methods and global methods, have been presented. The representative local linear methods are locally linear embedding (LLE) and linear preserving projections (LPP), which seek to find an embedding space that preserves local information to explore the intrinsic characteristics of high dimensional data. However, both of them still fail to nicely deal with the sparsely sampled or noise contaminated datasets, where the local neighborhood structure is critically distorted. On the contrary, principal component analysis (PCA), the most frequently used global method, preserves the total variance by maximizing the trace of feature variance matrix. But PCA cannot preserve local information due to pursuing maximal variance. In order to integrate the locality and globality together and avoid the drawback in LLE and PCA, in this paper, inspired by the dimensionality reduction methods of LLE and PCA, we propose a new dimensionality reduction method for face recognition, namely, unsupervised linear difference projection (ULDP). This approach can be regarded as the integration of a local approach (LLE) and a global approach (PCA), so that it has better performance and robustness in applications. Experimental results on the ORL, YALE and AR face databases show the effectiveness of the proposed method on face recognition.

Background Recently, the need for artificial intelligence (AI) and machine learning (ML) methods in drug development and research is gaining high concern and more grounds. Moreover, providing pharmaceutical and related schools with non-commercial, free-to-use programming languages, software and tools is becoming an unavoidable need. The R programming language can be easily used, through the correct and simplified codes and packages, in conducting unsupervised ML methods, such as principal component analysis (PCA) and hierarchical clustering analysis (HCA), after calculating relevant descriptors of drugs and molecules. Objective The objective of this study was to assess the enhancement of non-computer sciences-based students’ perception of the use of machine learning methods such as PCA and HCA using R-programming in drug formulation. Results Undergraduate students were taught to use R program to derive PCA distinguishable plots such as score, loading and scree, in addition to HCA dendrograms, in the context of developing new pharmaceutical formulations. Surveys conducted pre- and post-teaching the course proved that implementation of such ML methods can help in better understanding and exploring the data, in order to derive meaningful conclusions, and make informed decisions that help develop pharmaceutical formulations of premium quality, with minimal resources consumption. Conclusion We hereby report the easy use of R-programming in applications and activities that introduce undergraduate Pharmaceutical Engineering and Biotechnology students to ML methods. Student surveys showed better student satisfaction and understanding of AI applications in solving pharmaceutical problems. We claim that these students and early_career researchers, who are non-specialists in computer science, can utilize R-programming to perform important pharmaceutical applications through the step-by-step guide and codes provided in this article. Graphical Abstract

Introduction: Saffron is one of the most important economic plants in the Khorasan province. Awareness of soil quality in agricultural lands is essential for the best management of lands and for obtaining maximum economic benefit. In general, plant growth is a function of environmental factors especially chemical and physical properties of soil (20). It has been demonstrated that there was a positive and high correlation between soil organic matter and saffron yield. Increasing the yield of saffron due to organic matter is probably due to soil nutrient, especially phosphorous and nitrogen and also improvement of soil physical quality (6, 28, 29). The yield of saffron in soils with high nitrogen as a result of vegetative growth is high (8). Shahandeh (6) found that most of the variation of saffron yield depends on soil properties. Due to the economic importance of saffron and the role of soil properties on saffron yield, this research was conducted to find the relationship between saffron yield and some soil physical and chemical properties, and to determine the contribution of soil properties that have the greatest impact on saffron yield in the Ghayenat area. Materials and Methods: This research was performed in 30 saffron fields (30 soil samples) of the Ghayenat area (longitude 59° 10΄ 10.37˝ - 59° 11΄ 38.41˝ and latitude 33° 43΄ 35.08˝ - 33΄ 44΄ 02.78˝), which is located in the Khrasan province of Iran. In this research, 21 soil properties were regarded as the total data set (TDS). Then the principal component analysis (PCA) was used to determine the most important soil properties affecting saffron yield as a minimum data set (MDS) and the stepwise regression to estimate saffron yield. To estimate the yield of saffron in stepwise regression method, saffron yield was considered as a dependent variable and soil physical and chemical properties were considered to be independent variables. Results and Discussion: According to the PCA method, among the 21 studied properties, 7 out of them including calcium, iron, zinc contents, sand, calcium carbonate equivalent percent, mean weight diameter of aggregates (MWD) and manganese (Mn) had the higher Eigenvalues. Therefore, the above properties were introduced as the most important soil properties in saffron fields. Calcium carbonate had the negative effect on the availability of micro-nutrients (26). Christensen et al. (15) found that by increasing the calcium carbonate in soil due to high pH and formation of insoluble components, the uptake of micro-nutrients is especially limited. The results of stepwise regression method (equation 1) showed that soil acidity (pH), zinc content, bulk density, MWD, iron content, salinity (EC), organic carbon and available potassium in soil were the most important properties that affect the yield of saffron, so that the determination coefficient (R2) of the regression model was high (Table 2) and it can explain 74% of the variation of saffron yield. Y = 6924.51 – 1187.31 pH – 89.65 EC + 71.6 Fe – 826.02 Zn + 471.55 OC, + 5490.96 K + 1353.56 BD + 752.82 MWD (1) where Y: saffron yield (kgha-1), pH: soil acidity, EC: electoral conductivity (dSm-1), Fe: iron concentration (mgkg-1), Zn: zinc concentration (mgkg-1), OC: organic carbon (%), K: soil potassium (%), BD: soil bulk density (Mgm-3), and MWD: mean weight diameter of aggregates (MM). Based on the absolute values of standard s in the regression model (Table 3), pH value and then after Zn concentration had the most effect on saffron yield. In general, responses of different plants to soil pH is varied, and saffron grows satisfactory in pH = 7.8 (5). Soil pH influences the uptake of soil nutrients by plants (15), so that this parameter had the most effect on saffron yield and by increasing the soil pH, the yield of saffron decreases. According to the regression model, Zn concentration was the second parameter in saffron yield. Zn has the important role in structure of plant enzymes (30). After these 2 parameters, Bd, MWD, Fe concentration, EC, Organic carbon and K concentration in soil had more effect on saffron yield (Table 3). Conclusion: According to both PCA and regression methods, the concentration of iron and zinc and MWD were determined as the important and effective soil properties on saffron yield in the Ghayenat area. In addition, soil pH in stepwise regression method and calcium carbonate in PCA method were determined as the effective properties on saffron yield. Therefore, it is suggested that the parameters of Zn, Fe, and MWD along with soil pH and calcium carbonate which were regarded individually in two methods, were considered as the most soil properties in saffron yield.

Mangroves are coastal plants influenced by tidal cycles. One of the regions in South Sumatra Province with a mangrove ecosystem is Sembilang National Park. As a national park, this location is strategically positioned for research related to mangrove species. The utilization of the Principal Component Analysis (PCA) method is considered to enhance the capabilities of remote sensing data in mangrove mapping. However, its application has been limited to the mangrove level. The research objective is to identify mangrove species in Sembilang National Park using the PCA method by leveraging Landsat-8 image data acquired on September 9, 2019. Mangrove distribution is obtained through the Mangrove Vegetation Index (MVI) and vector data from Global Mangrove Watch. The image is then overlaid with field species data to obtain species spectral patterns. Additionally, the correlation between spectral band values and eigenvalues (PC) is analyzed to detect parameters correlated with eigenvalues. The research results show that PC values correlate with mangrove species and can be used for mangrove species identification. This is demonstrated by Bruguiera Gymnorrhiza species with canopy coverage of 60.8% and 62.46% at ST7 and ST8 having the same PC values, while mangroves with canopy coverage of 62.5% in different species have different PC values. Moreover, the PCA method can indicate two crucial factors in identifying mangrove species, namely vegetation and soil factors. Mangrove merupakan tumbuhan pesisir yang dipengaruhi oleh pasang surut. Salah satu wilayah di Provinsi Sumatera Selatan yang memiliki ekosistem mangrove adalah Taman Nasional Sembilang. Sebagai taman nasional, lokasi ini sangat strategis untuk dilakukan penelitian terkait spesies mangrove. Pemanfaatan metode principal component analysis (PCA) dinilai mampu meningkatkan kemampuan data penginderaan jauh dalam pemetaan mangrove. Namun selama ini, pemanfaatan terbatas pada level mangrove. Adapun tujuan penelitian adalah mengidentifikasi spesies mangrove di Taman Nasional Sembilang menggunakan metode PCA dengan memanfaatkan data citra Landsat-8 yang diakusisi pada 09 September 2019. Sebaran mangrove diperoleh melalui indeks vegetasi mangrove (MVI) dan data vektor dari Global Mangrove Watch. Citra tersebut kemudian di overlay dengan data spesies lapangan untuk mendapatkan pola spektral species. Selain itu, korelasi nilai spektral band dan eigen (PC) dianalisis untuk mendeteksi parameter yang berkorelasi dengan nilai eigen. Hasil penelitian menunjukkan nilai PC memiliki korelasi dengan spesies mangrove dan dapat digunakan untuk mengidentifikasi spesies mangrove. Hal ini ditunjukkan spesies Bruguiera Gymnorrhiza dengan tutupan kanopi 60,8% dan 62,46% pada ST7 dan ST8 memiliki nilai PC yang sama serta mangrove dengan besaran tutupan kanopi 62,5% pada spesies nilai PC berbeda. Selain itu, metode PCA mampu menunjukkan dua faktor penting dalam mengidentifikasi spesies mangrove, yaitu faktor vegetasi dan tanah.

Relevance. The need for accurate forecasting of electricity consumption to improve efficiency and reduce costs at industrial enterprises, which leads to increased competitiveness of goods manufactured by the enterprise. Traditional forecasting methods often do not take into account complex interactions between various factors affecting energy consumption and do not provide the necessary forecast accuracy. The principal component analysis method offers a promising prospect – reducing the volume of processed data (dimensionality) without significant loss of information, which simplifies forecast models while maintaining their accuracy. Aim. To develop an accurate and efficient model for forecasting electricity consumption at industrial enterprises using the principal component analysis method. This model is aimed at eliminating the limitations of traditional forecasting approaches by reducing data dimensionality and increasing the accuracy of predictions, which ultimately improves the efficiency of electricity consumption and reduces financial costs, including those due to forecasting errors. Methods. The principal component analysis method, which allows us to reduce the volume of processed data (dimensionality) by transforming a large set of correlated variables into a smaller set of uncorrelated principal components. The study included the following stages: data import and factor analysis, correlation matrix construction, analysis of selected and accumulated variances for each factor, factor loading matrix construction, dimension reduction, development of a mathematical model using linear regression, and forecast installation and validation. Results. The application of the principal component analysis method allowed us to create a model for forecasting electricity consumption. Its application showed that the first principal component explains 69.65% of the total variance, the second component – ​​17.28%, i. e. together they explain almost 87% of the variance. The developed model provides good agreement between the actual and forecast values ​​of electricity consumption in several time intervals with an average error level within the range of +3 to –5%. This indicates the suitability of the model for forecasting electricity consumption, although some discrepancies indicate the need for its further improvement.

The development of artificial neural networks is related to statistical and biometric analysis which is one of the applications that can require artificial neural network models. Recognition of facial patterns is an important part of identifying a person. The face can be divided into areas such as the nose, eyes and mouth. Face pattern recognition is a research area that can be applied to the principal component analysis (PCA) method. The training process carried out by the eigenface calculation uses PCA and the results of this study show that facial pattern recognition based on the proportion of memorization and generalization for the use of the method without PCA is better than facial pattern recognition using PCA. Pattern recognition without using the PCA method, the level of memorization and generalization reaches 100% at the 40th iteration and 0.0099 error with a learning rate and momentum of 0.8, while facial pattern recognition using the PCA method, the memorization and generalization level reaches 100% in the iteration. to -1000 and error 0.00103 with learning rate and momentum 0.9.

Systematic principal component analysis (PCA) methods are presented in this paper for reliable islanding detection for power systems with significant penetration of distributed generations (DGs), where synchrophasors, recorded by phasor measurement units, are used for system monitoring. Existing islanding detection methods, such as rate-of-change-of-frequency and vector shift are fast for processing local information; however, with the growth in installed capacity of DGs, they suffer from several drawbacks. Incumbent genset islanding detection cannot distinguish a system-wide disturbance from an islanding event, leading to maloperation. The problem is even more significant when the grid does not have sufficient inertia to limit frequency divergences in the system fault/stress due to the high penetration of DGs. To tackle such problems, this paper introduces PCA methods for islanding detection. A simple control chart is established for intuitive visualization of the transients. A recursive PCA scheme is proposed as a reliable extension of the PCA method to reduce the false alarms for time-varying process. To further reduce the computational burden, the approximate linear dependence condition errors are calculated to update the associated PCA model. The proposed PCA and RPCA methods are verified by detecting abnormal transients occurring in the U.K. utility network.

This paper carries on research surrounding the influences produced by dimensionality reduction on machine learning classification effect. Firstly, paper constructs the analysis architecture of data dimension reduction classification, combines the two different unsupervised dimension reduction methods, locally linear embedding (LLE) and principal component analysis (PCA) with the five machine learning classification methods: Gradient Boosting Decision Tree (GBDT), Random Forest, Support Vector Machine (SVM), K-Nearest Neighbor (KNN) and Logistic Regression. And then uses the handwritten digital identification dataset to analyze the classification performance of these five classification methods on different dimension datasets by different dimensionality reduction methods. The analysis shows that using the appropriate dimensionality reduction method for dimensionality reduction classification can effectively improve the classification accuracy; the dimensionality reduction classification effect of non-linear dimensionality reduction method is generally better than the linear dimensionality reduction method; different machine learning classification algorithms have significant differences in the sensitivity of dimensions.

There are limitations for the existing methods to model multivariate time series because that defining the input components is highly difficult. The main purpose of this paper is to expand the principal components analysis (PCA) method to extract the joint information of multiple variables. First, both the linear correlations and the nonlinear correlations are detected to initialize an embedding delay window, which contains enough information for prediction. Then, the PCA method is expanded to extract the joint information of multiple variables in a complex system. Finally, neural network makes predictions on the basis of approximating both the functional relationship between different variables and the map between current state and future state.

This paper presents a novel Gabor-based kernel Principal Component Analysis (PCA) method by integrating the Gabor wavelet representation of face images and the kernel PCA method for face recognition. Gabor wavelets first derive desirable facial features characterized by spatial frequency, spatial locality, and orientation selectivity to cope with the variations due to illumination and facial expression changes. The kernel PCA method is then extended to include fractional power polynomial models for enhanced face recognition performance. A fractional power polynomial, however, does not necessarily define a kernel function, as it might not define a positive semidefinite Gram matrix. Note that the sigmoid kernels, one of the three classes of widely used kernel functions (polynomial kernels, Gaussian kernels, and sigmoid kernels), do not actually define a positive semidefinite Gram matrix either. Nevertheless, the sigmoid kernels have been successfully used in practice, such as in building support vector machines. In order to derive real kernel PCA features, we apply only those kernel PCA eigenvectors that are associated with positive eigenvalues. The feasibility of the Gabor-based kernel PCA method with fractional power polynomial models has been successfully tested on both frontal and pose-angled face recognition, using two data sets from the FERET database and the CMU PIE database, respectively. The FERET data set contains 600 frontal face images of 200 subjects, while the PIE data set consists of 680 images across five poses (left and right profiles, left and right half profiles, and frontal view) with two different facial expressions (neutral and smiling) of 68 subjects. The effectiveness of the Gabor-based kernel PCA method with fractional power polynomial models is shown in terms of both absolute performance indices and comparative performance against the PCA method, the kernel PCA method with polynomial kernels, the kernel PCA method with fractional power polynomial models, the Gabor wavelet-based PCA method, and the Gabor wavelet-based kernel PCA method with polynomial kernels.

Background Availability of numerous oil deposits with a wide variety of the conditions of occurrence, reservoir properties and physical-chemical properties of fluids makes important the need of grouping of development objects and identification of relatively uniform groups by certain criteria. Grouping allows to define similarities and differences between deposits in determining development objects, and to substantiate systems of development. Analysis of large data sets on objects characterized by the measured or estimated parameters is labor-intensive, requiring a lot of time and effort, so there is a question of dimension reduction, i.e. of reducing the original data set to a smaller number of parameters, and grouping of objects into relatively uniform groups. Here the parameters can be selected from the original ones or obtained by calculation and conversion (with the least loss of information about the objects under study). An effective tool of analysis that can help in this matter is the method of principal component analysis. Aims and Objectives Apply the method of principal component analysis to reduce the number of parameters which define objects of study, so that to find effective ways to develop new oil and gas deposits, and fast and correct solution of practical problems through the use of gained summary experience and assessment of similarities and differences of oil and gas reservoirs by combinations of geological-field attributes. Methods The problem of ranking oil deposits in Western Siberia, is solved by the method of principal component analysis, which is a linear transformation of features in a new set of «independent» random values, which are arranged in descending order of their variances, and the main components are determined by the eigenvectors of the correlation matrix obtained based on input parameters characterizing the objects under investigation. Having selected the components that comprise the bulk of the variances, one can calculate them for different types of objects and classify by clusterable points. Grouping of objects of research was carried out using built-in functions of computer statistics programs (a system of statistical analysis Statistica, analytical applications for Microsoft Excel XLStat). The same calculation was performed on the basis of the theory of mathematical statistics. The calculation results coincided. Results The method of principal component analysis allowed selection from the original data set comprising 50 research objects characterized by 12 parameters of six relatively uniform groups using only 4 factors (principal components) which, however, contain much more information than the individual parameters. As a result of the carried out analysis data were systematized and the dimension of the initial population (transition from multidimensional space to the four-dimensional) was reduced, with no significant loss of information about research objects. Conclusion Based on the proposed algorithm, and relatively uniform groups obtained there is a possibility for inclusion of new deposits into one of these groups and selection of the nearest to it analogous object, that will simplify the design of development system and identify ways and means of improving system’s effectiveness.

Brillouin optical correlation-domain sensing enables high-speed Brillouin gain spectrum (BGS) measurement at random positions along the optical fiber. To extract the Brillouin frequency shift (BFS) that reflects the real-time strain information, machine learning methods of principal components analysis (PCA) and support vector machine (SVM) are used in the signal processing for the BGSs. The performances of dimensionality reduction by PCA and SVM based on classification and regression are analyzed and compared. The experiment demonstrates an 8 kHz BGS acquisition repetition rate and an average BFS extraction time of 0.0104 ms, which is 27.3 times faster than the conventional method with no PCA. The proposed methods realize a real-time dynamic strain measurement at the frequency of 40 Hz.

The article examines the application of the Principal Component Analysis (PCA) and machine learning method, the Long Short-Term Memory (LSTM), in the prediction of the yield curve for Poland. The PCA was applied to decompose the yield curve, forecast its components using the LSTM, and obtain the yield curve predictions upon recomposition. The results from the PCA-LSTM model were compared to predictions generated directly by the LSTM model, simple autoregression and random walk, which serves as a benchmark. Overall, LSTM predictions are the most accurate with PCA-LSTM being a close second, nonetheless PCA-LSTM is more accurate in short-term forecasting of interest rates at long maturities. Both methods outperform the benchmark, while autoregression usually underperforms. For these reasons, the PCA-LSTM as well as the LSTM can be useful in interest rate management or building trading strategies. The PCA-LSTM has the advantage that it can focus on particular components of the yield curve, such as variability of the yield curve’s level or steepness.

The aim of data transformation is to transform the original feature space of data into another space with better properties. This is typically combined with dimensionality reduction, so that the dimensionality of the transformed space is smaller. A widely used method for data transformation and dimensionality reduction is Principal Component Analysis (PCA). PCA finds a subspace that explains most of the data variance. While the new PCA feature space has interesting properties, such as removing linear correlation, PCA is an unsupervised method. Therefore, there is no guarantee that the PCA feature space will be the most appropriate for supervised tasks, such as classification or regression. On the other hand, 3-layer Multi Layer Perceptrons (MLP), which are supervised methods, can also be understood as a data transformation carried out by the hidden layer, followed by a classification/regression operation performed by the output layer. Given that the hidden layer is obtained after a supervised training process, it can be considered that it is performing a supervised data transformation. And if the number of hidden neurons is smaller than the input, also dimensionality reduction. Despite this kind of transformation being widely available (any neural network package that allows access to the hidden layer weights can be used), no extensive experimentation on the quality of 3-layer MLP data transformation has been carried out. The aim of this article is to carry out this research for classification problems. Results show that, overall, this transformation offers better results than the PCA unsupervised transformation method.