Adaptive dimensionality reduction for neural network-based online principal component analysis.

We investigated three classifiers for the task of distinguishing between benign and malignant breast lesions. Classification performance was measured in terms of area under the ROC curve (AUC value). We compared linear discriminant analysis (LDA), quadratic discriminant analysis (QDA) and a Bayesian neural net (BNN) with 5 hidden units. For each lesion, 46 image features were extracted and principal component analysis (PCA) of these features was used as classifier input. For each classifier, the optimal number of principal components was determined by performing PCA within each step of a leave-one-case-out protocol for the training dataset (1125 lesions, 14% cancer prevalence) and determining which number of components maximized the AUC value. Subsequently, each classifier was trained on the training dataset and applied ‘cold turkey’ to an independent test set from a different population (341 lesions, 30% cancer prevalence). The optimal number of principal components for LDA was 24, accounting for 97% of the variance in the image features. For QDA and BNN, these numbers were 5 (70%) and 15 (93%), respectively. The LDA, QDA and BNN obtained AUC values of 0.88, 0.85, and 0.91, respectively, in the leave-one-case-out analysis. In the independent test – with AUCs of 0.88, 0.76, and 0.82 – only LDA achieved performance identical to that for the training set (lower bound of 95% non-inferiority interval -.0067), while the others performed significantly worse (p-values << 0.05). While the more complex BNN classifier outperformed the others in leave-one-case-out of a large dataset, LDA was the robust best-performer in an independent test.

Principal component analysis is a multivariate technique widely used in dimensionality reduction. The ideal number of principal components retained should be defined when one is dealing with high-dimensional data. Some criteria for this choice were proposed in the literature. Most of them have serious limitations, such as normality assumptions, subjective analysis, and asymptotic properties. This study aims to propose two new tests using the parametric bootstrap for determining the optimal number of principal components (PC) retained for subsequent analysis, based on the amount of the total variation accounted for by the k first principal components. The performances of these tests were compared among themselves and with those of Fujikoshi (1980) and Gebert and Ferreira (2010) through Monte Carlo simulations. Under multivariate normality the two proposed parametric bootstrap tests are recommended. Under nonnormality the test of Gebert and Ferreira (2010) is recommended. The three bootstrap tests surpass t...

The development of large wind turbines that enable to harvest energy more efficiently is a consequence of the increasing demand for renewables in the world. To optimize the potential energy output, light and flexible wind turbine blades (WTBs) are designed. However, the higher flexibilities and lower buckling capacities adversely affect the long-term safety and reliability of WTBs, and thus the increased operation and maintenance costs reduce the expected revenue. Effective structural health monitoring techniques can help to counteract this by limiting inspection efforts and avoiding unplanned maintenance actions. Vibration-based methods deserve high attention due to the moderate instrumentation efforts and the applicability for in-service measurements. The present paper proposes the use of cross-correlations (CCs) of acceleration responses between sensors at different locations for structural damage detection in WTBs. CCs were in the past successfully applied for damage detection in numerical and experimental beam structures while utilizing only single lags between the signals. The present approach uses vectors of CC coefficients for multiple lags between measurements of two selected sensors taken from multiple possible combinations of sensors. To reduce the dimensionality of the damage sensitive feature (DSF) vectors, principal component analysis is performed. The optimal number of principal components (PCs) is chosen with respect to a statistical threshold. Finally, the detection phase uses the selected PCs of the healthy structure to calculate scores from a current DSF vector, where statistical hypothesis testing is performed for making a decision about the current structural state. The method is applied to laboratory experiments conducted on a small WTB with non-destructive damage scenarios.

Face recognition efficiency is extremely challenged by image corruption, noise, shadowing and variant face expressions. In this paper, we propose a reliable incremental face recognition algorithm to address this problem. The algorithm is robust to face image misalignment, occlusion, corruption and different style variations. To apply this for large face data streams, the proposed algorithm uses incremental robust principal component analysis (PCA) to regain the intrinsic data of a bunch of images regarding one subject. A novel similarity metric is defined for face recognition and classification. Five different databases and a base of four different criteria are used in the experiments to illustrate the reliability of the proposed method. Experiments point that it outperforms other existing incremental PCA approaches namely incremental singular value decomposition, add block singular value decomposition and candid covariance-free incremental PCA in terms of recognition rate under occlusions, facial expressions and image perspectives.

Rapid and Early Detection of Salmonella Serotypes with Hyperspectral Microscopy and Multivariate Data Analysis

In this paper, we propose a robust incremental principal component analysis (IPCA) for stream data that can handle missing values on an ongoing basis. In the proposed IPCA, a missing value is substituted with the value estimated from a conditional probability density function. The conditional probability density functions are incrementally updated when new data are given. In the experiments, we evaluate the performance for both artificial and real data sets through the comparison with the two conventional approaches to handing missing values. We first investigate the estimation errors of missing values. The experimental results demonstrate that the proposed IPCA gives lower estimation errors compared to the other approaches. Next, we investigate the approximation accuracy of eigenvectors. The results show that the proposed IPCA has relatively good accuracy of eigenvectors not only for major components but also for minor components.

Classification Of Acute Myelogenous Leukemia (AML) Based On Functionally Related Proteins Groups

In the oil and gas industry, an extensive volume of logging data is routinely generated and acquired during operations. This data is accumulated across various phases of exploration, production, and subsequent operations. The challenge lies in extracting meaningful insights from this high-dimensional dataset. Constraints in data visualization (such as the three spatial axes, x, y, and z) and manipulation further complicate the process of drawing valuable conclusions. As a method to tackle this challenge, a technique has been devised to streamline data visualization by reducing high-dimensional data to a lower dimensionality, while retaining vital features crucial for big data analysis. Principal component analysis (PCA) emerges as a prominent approach employed for data dimensionality reduction. PCA simplifies multivariate datasets encompassing well-logging tools (e.g., gamma ray, density, neutron porosity, and resistivity) and drilling parameters such as rate of penetration, weight on bit, and vibration data, condensing them into a reduced number of factors known as principal components (PCs). The results of the dimensionality reduction technique show that various logging data and drilling parameters were condensed into a set of principal components (PC1, PC2…PCx), where x corresponds to the number of utilized variables. The analysis indicates that the first two components (PC1, PC2) capture most of data patterns. The graphical representation of these components (PCA biplot) reveals distinct clusters with clear patterns, facilitating the identification of separate electro-facies. Moreover, PC1 exhibits a strong correlation with lithology variations, enabling its utilization in well-to-well correlation. Additionally, regression analysis demonstrates a significant predictive relationship between PCA components and well logging variables, allowing the use of the R-squared regression technique to forecast a result curve based on PCA input. Higher principal components are found to be more associated with formation fluid, thus complementing the standard petrophysical analysis. The industry's current shift is from data collection to practical applications. By employing diverse dimensionality reduction techniques, the workflow enables the analysis of big data in a more comprehensive manner, unveiling concealed trends and insights. This approach not only facilitates machine-learning and artificial-intelligence applications but also enables a deeper understanding of the data through descriptive forms, thereby supporting industry advancements.

Abstract. In remote sensing community, Principal Component Analysis (PCA) is widely utilized for dimensionality reduction in order to deal with high spectral-dimension data. However, dimensionality reduction through PCA results in loss of some spectral information. Analysis of an Earth-scene, based on first few principal component bands/channels, introduces error in classification, particularly since the dimensionality reduction in PCA does not consider accuracy of classification as a requirement. The present research work explores a different approach called Multi-Classifier System (MCS)/Ensemble classification to analyse high spectral-dimension satellite remote sensing data of WorldView-2 sensor. It examines the utility of MCS in landuse-landcover (LULC) classification without compromising any channel i.e. avoiding loss of information by utilizing all of the available spectral channels. It also presents a comparative study of classification results obtained by using only principal components by a single classifier and using all the original spectral channels in MCS. Comparative study of the classification results in the present work, demonstrates that utilizing all channels in MCS of five Artificial Neural Network classifiers outperforms a single Artificial Neural Network classifier that uses only first three principal components for classification process.

Genome-wide association studies (GWAS) are susceptible to bias due to population stratification (PS). The most widely used method to correct bias due to PS is principal components (PCs) analysis (PCA), but there is no objective method to guide which PCs to include as covariates. Often, the ten PCs with the highest eigenvalues are included to adjust for PS. This selection is arbitrary, and patterns of local linkage disequilibrium may affect PCA corrections. To address these limitations, we estimate genomic propensity scores based on all statistically significant PCs selected by the Tracy-Widom (TW) statistic. We compare a principal components and propensity scores (PCAPS) approach to PCA and EMMAX using simulated GWAS data under no, moderate, and severe PS. PCAPS reduced spurious genetic associations regardless of the degree of PS, resulting in odds ratio (OR) estimates closer to the true OR. We illustrate our PCAPS method using GWAS data from a study of testicular germ cell tumors. PCAPS provided a more conservative adjustment than PCA. Advantages of the PCAPS approach include reduction of bias compared to PCA, consistent selection of propensity scores to adjust for PS, the potential ability to handle outliers, and ease of implementation using existing software packages.

Principal Component Analysis (PCA), also known as Proper Orthogonal Decomposition (POD), is one of the prevailing Reduced Order Modeling (ROM) techniques for aerodynamic data analysis. Along with the PCA, its variant for missing data, gappy POD, has recently found its application by transforming problems into missing data problems. In this paper, an alternative based on probability theory, i.e., Probabilistic Principal Component Analysis (PPCA), is employed for aerodynamic data reconstruction problems in lieu of the PCA and the gappy POD. Unlike the PCA, the PPCA generates spanning vectors of low-dimensional space iteratively with an Expectation Maximization (EM) algorithm. The combined algorithm, termed as Expectation-Maximization Probabilistic Principal Component Analysis (EM-PCA), can deal with not only complete data but also incomplete data sets, since the EM algorithm naturally incorporates missing data in its formulation. Therefore, the EM-PCA has potential to be a substitute for both PCA and gappy POD. For illustration, the EM-PCA is applied to complete as well as incomplete snapshot ensembles compiled from aerodynamic simulations, and acquired modes and restored data are verified with those obtained by the PCA and the gappy POD.

This paper carries on research surrounding the influences produced by dimensionality reduction on machine learning classification effect. Firstly, paper constructs the analysis architecture of data dimension reduction classification, combines the two different unsupervised dimension reduction methods, locally linear embedding (LLE) and principal component analysis (PCA) with the five machine learning classification methods: Gradient Boosting Decision Tree (GBDT), Random Forest, Support Vector Machine (SVM), K-Nearest Neighbor (KNN) and Logistic Regression. And then uses the handwritten digital identification dataset to analyze the classification performance of these five classification methods on different dimension datasets by different dimensionality reduction methods. The analysis shows that using the appropriate dimensionality reduction method for dimensionality reduction classification can effectively improve the classification accuracy; the dimensionality reduction classification effect of non-linear dimensionality reduction method is generally better than the linear dimensionality reduction method; different machine learning classification algorithms have significant differences in the sensitivity of dimensions.

Principal Component Analysis (PCA) is a widely used dimensional reduction method that aims to find a low dimension sub space of highly correlated data for its major information to be used in further analysis. Machine learning methods based on PCA are popular in high dimensional data analysis, such as video and image processing. In video processing, the Robust PCA (RPCA), which is a modified method of the traditional PCA, has good properties in separating moving objects from the background, but it may have difficulties in separating those when light intensity of the background varies significantly in time. To overcome the difficulties, a modified PCA method, called Regression PCA (RegPCA), is proposed. The method is developed by combining the traditional PCA and regression approaches together, and it can be easily combined with RPCA for video processing. We focus the presentation of RegPCA with the combination of RPCA on video processing and find that it is more reliable than RPCA only. We use RegPCA to separate moving object from the background in a color video and get a better result than that given by RPCA. In the implementation, we first derive the explanatory variables by the background information. we then process a number of frames of the video and use those as a set of response variables. We remove the impact of the background by regressing the response against the explanatory variables by a regression model. The regression model provides a set of residuals, which can be further analyzed by RPCA. We compare the results of RegRPCA against those of RPCA only. It is evident that the moving objects can be completely removed from the background using our method but not in RPCA. Note that our result is based on a combination of RegPCA with RPCA. Our proposed method provides a new implementation of RPCA under the framework of regression approaches, which can be used to account for the impact of risk factors. This problem cannot be addressed by the application of RPCA only.

Appearance-based image analysis techniques require fast computation of principal components of high-dimensional image vectors. We introduce a fast incremental principal component analysis (IPCA) algorithm, called candid covariance-free IPCA (CCIPCA), used to compute the principal components of a sequence of samples incrementally without estimating the covariance matrix (so covariance-free). The new method is motivated by the concept of statistical efficiency (the estimate has the smallest variance given the observed data). To do this, it keeps the scale of observations and computes the mean of observations incrementally, which is an efficient estimate for some well known distributions (e.g., Gaussian), although the highest possible efficiency is not guaranteed in our case because of unknown sample distribution. The method is for real-time applications and, thus, it does not allow iterations. It converges very fast for high-dimensional image vectors. Some links between IPCA and the development of the cerebral cortex are also discussed.

Groundwater is considered as one of the most important sources for water supply in Iran. The Fasa Plain in Fars Province, Southern Iran is one of the major areas of wheat production using groundwater for irrigation. A large population also uses local groundwater for drinking purposes. Therefore, in this study, this plain was selected to assess the spatial variability of groundwater quality and also to identify main parameters affecting the water quality using multivariate statistical techniques such as Cluster Analysis (CA), Discriminant Analysis (DA), and Principal Component Analysis (PCA). Water quality data was monitored at 22 different wells, for five years (2009-2014) with 10 water quality parameters. By using cluster analysis, the sampling wells were grouped into two clusters with distinct water qualities at different locations. The Lasso Discriminant Analysis (LDA) technique was used to assess the spatial variability of water quality. Based on the results, all of the variables except sodium absorption ratio (SAR) are effective in the LDA model with all variables affording 92.80% correct assignation to discriminate between the clusters from the primary 10 variables. Principal component (PC) analysis and factor analysis reduced the complex data matrix into two main components, accounting for more than 95.93% of the total variance. The first PC contained the parameters of TH, Ca2+, and Mg2+. Therefore, the first dominant factor was hardness. In the second PC, Cl-, SAR, and Na+ were the dominant parameters, which may indicate salinity. The originally acquired factors illustrate natural (existence of geological formations) and anthropogenic (improper disposal of domestic and agricultural wastes) factors which affect the groundwater quality.