Sparse and Functional Principal Components Analysis

Sparse Principal Component Analysis (SPCA) is a method that can get the sparse loadings of the principal components (PCs), and it may formulate PCA as a regression-type optimization problem by using the elastic net. But the selected features are different with each PC and generally independent. A new method named SPCA has been proposed for removing these detect, which replaces the elastic net with L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -norm penalty. The results of the method on gene expression data are still unknown. Therefore, we will take a test to prove this point in this paper. Firstly, this method is applied to the simulated data for obtaining an optimal parameter. Secondly, the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> SPCA method is applied to the gene expression data, that is the head and neck squamous carcinoma data (HNSC). Thirdly, the characteristic genes are selected according the PCs. The results consist of very lower P-value and very higher hit count, which shows the method of L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> SPCA can obtain higher recognition accuracy and higher relevancy to the genes. Finally, the experimental results demonstrate that the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> SPCA works well and has good performances in the gene expression data.

Previous versions of sparse principal component analysis (PCA) have presumed that the eigen-basis (a p × k matrix) is approximately sparse. We propose a method that presumes the p × k matrix becomes approximately sparse after a k × k rotation. The simplest version of the algorithm initializes with the leading k principal components. Then, the principal components are rotated with an k × k orthogonal rotation to make them approximately sparse. Finally, soft-thresholding is applied to the rotated principal components. This approach differs from prior approaches because it uses an orthogonal rotation to approximate a sparse basis. One consequence is that a sparse component need not to be a leading eigenvector, but rather a mixture of them. In this way, we propose a new (rotated) basis for sparse PCA. In addition, our approach avoids “deflation” and multiple tuning parameters required for that. Our sparse PCA framework is versatile; for example, it extends naturally to a two-way analysis of a data matrix for simultaneous dimensionality reduction of rows and columns. We provide evidence showing that for the same level of sparsity, the proposed sparse PCA method is more stable and can explain more variance compared to alternative methods. Through three applications—sparse coding of images, analysis of transcriptome sequencing data, and large-scale clustering of social networks, we demonstrate the modern usefulness of sparse PCA in exploring multivariate data. An R package, epca, and the supplementary materials for this article are available online.

Most of the existing procedures for sparse principal component analysis (PCA) use a penalty function to obtain a sparse matrix of weights by which a data matrix is post-multiplied to produce PC scores. In this paper, we propose a new sparse PCA procedure which differs from the existing ones in two ways. First, the new procedure does not sparsify the weight matrix. Instead, the so-called loadings matrix is sparsified by which the score matrix is post-multiplied to approximate the data matrix. Second, the cardinality of the loading matrix i.e., the total number of nonzero loadings, is pre-specified to be an integer without using penalty functions. The procedure is called unpenalized sparse loading PCA (USLPCA). A desirable property of USLPCA is that the indices for the percentages of explained variances can be defined in the same form as in the standard PCA. We develop an alternate least squares algorithm for USLPCA which uses the fact that the PCA loss function can be decomposed as a sum of a term irrelevant to the loadings, and another one being easily minimized under cardinality constraints. A procedure is also presented for selecting the best cardinality using information criteria. The procedures are assessed in a simulation study and illustrated with real data examples.

Hyperspectral imaging (HSI) is increasingly being used in a wide range of practical applications. High correlations between and within classes, the curse of dimensionality, and overfitting have all piqued the interest of researchers. Because of the excellent feature extraction and good overall performance, deep learning-based algorithms with dimensionality reduction methods have been recommended as a viable alternative for hyperspectral image analysis. Principal component analysis (PCA) and Sparse PCA (SPCA) are commonly used in dimensionality reduction. Because each principal component is a linear mixture of the original variables, principal component analysis (PCA) is difficult to interpret, and SPCA cannot choose the most relevant features. To overcome the issue, a mutual information-based greedy feature selection approach called minimum redundancy maximum relevancy (mRMR) is combined with SPCA (SPCA-mRMR). Many deep learning combination techniques rely on the 2D and 3D Convolutional Neural Network (CNN) methodologies. A fused 3D-2D CNN model is less difficult than a 3D-CNN alone, yet it can also operate effectively in noisy settings with few training examples. Therefore, SPCA-mRMR and fused 3D-2D CNN are used to create a hybrid model called HMC-NET for HSI classification. Several optimization strategies are used to address overfitting and improve classification accuracy. To assess the efficacy of this proposed technique, performance results were compared to highly regarded models with fewer variables.

Abnormal glucose variability (GV) is a risk factor for diabetes complications, and tens of indices for its quantification from continuous glucose monitoring (CGM) time series have been proposed. However, the information carried by these indices is redundant, and a parsimonious description of GV can be obtained through sparse principal component analysis (SPCA). We have recently shown that a set of 10 metrics selected by SPCA is able to describe more than 60% of the variance of 25 GV indicators in type 1 diabetes (T1D). Here, we want to extend the application of SPCA to type 2 diabetes (T2D). A data set of CGM time series collected in 13 T2D subjects was considered. The 25 GV indices considered for T1D were evaluated. SPCA was used to select a subset of indices able to describe the majority of the original variance. A subset of 10 indicators was selected and allowed to describe 83% of the variance of the original pool of 25 indices. Four metrics sufficient to describe 67% of the original variance turned out to be shared by the parsimonious sets of indices in T1D and T2D. Starting from a pool of 25 indices assessed from CGM time series in T2D subjects, reduced subsets of metrics virtually providing the same information content can be determined by SPCA. The fact that these indices also appear in the parsimonious description of GV in T1D may indicate that they could be particularly informative of GV in diabetes, regardless of the specific type of disease.

Principal Component Analysis (PCA) is a convenient tool used in aggregating the indicators of sustainable development and providing indices where different weights are assigned to the various indicators. There are, however, problems in interpreting of indices, especially if time series data are used. This study explores the feasibility of applying recent developments in PCA of time series using Philippine data. We present the comparative advantages of SPCA (Sparse Principal Component Analysis) relative to averaging of an adequacy/inadequacy index and PCA in index construction from various indicators of sustainable development in the Philippines in terms of usefulness and validity of indices being developed. SPCA can attain sparse and non-overlapping loadings without losing a large amount of explained variance compared to PCA. Because of the non-overlapping contribution of variables in SPCA components, indices can have clear and mutually exclusive meanings, facilitating interpretation. Even with a more complicated algorithm, reduced dimensions and simpler interpretation of indices justify the advantages of SPCA over PCA in index construction. The indices are interpreted in terms of the milestone of sustainability in the Philippines. The resulting indices provide an adequate summary of the sustainable indicators and evidence of the importance of leadership and political will in sustainable development.

Stability of a learning algorithm with respect to small input perturbations is an important property, as it implies the derived models to be robust with respect to the presence of noisy features and/or data sample fluctuations. In this paper we explore the effect of stability optimization in the standard feature selection process for the continuous (PCA-based) k-means clustering problem. Interestingly, we derive that stability maximization naturally introduces a tradeoff between cluster separation and variance, leading to the selection of features that have a high cluster separation index that is not artificially inflated by the feature’s variance. The proposed algorithmic setup is based on a Sparse PCA approach, that selects the features that maximize stability in a greedy fashion. In our study, we also analyze several properties of Sparse PCA relevant to stability that promote Sparse PCA as a viable feature selection mechanism for clustering. The practical relevance of the proposed method is demonstrated in the context of cancer research, where we consider the problem of detecting potential tumor biomarkers using microarray gene expression data. The application of our method to a leukemia dataset shows that the tradeoff between cluster separation and variance leads to the selection of features corresponding to important biomarker genes. Some of them have relative low variance and are not detected without the direct optimization of stability in Sparse PCA based k-means.KeywordsFeature SelectionFeature SubsetNormalize Mutual InformationFeature Selection AlgorithmCluster SeparationThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Abnormal glucose variability (GV) is considered to be a risk factor for the development of diabetes complications. For its quantification from continuous glucose monitoring (CGM) data, tens of different indices have been proposed in the literature, but the information carried by them is highly redundant. In the present work, the Sparse Principal Component Analysis (SPCA) technique is used to select, from a wide pool of GV metrics, a smaller subset of indices that preserves the majority of the total original variance, providing a parsimonious but still comprehensive description of GV. In detail, SPCA is applied to a set of 25 literature GV indices evaluated on CGM time-series collected in 17 type 1 (T1D) and 13 type 2 (T2D) diabetic subjects. Results show that the 10 GV indices selected by SPCA preserve more than the 75% of the variance of the original set of 25 indices, both in T1D and T2D. Moreover, 6 indices of the parsimonious set are shared by T1D and T2D.

Sparse Principal Component Analysis (S-PCA) is a novel framework for learning a linear, orthonormal basis representation for structure intrinsic to an ensemble of images. S-PCA is based on the discovery that natural images exhibit structure in a low-dimensional subspace in a sparse, scale-dependent form. The S-PCA basis optimizes an objective function which trades off correlations among output coefficients for sparsity in the description of basis vector elements. This objective function is minimized by a simple, robust and highly scalable adaptation algorithm, consisting of successive planar rotations of pairs of basis vectors. The formulation of S-PCA is novel in that multi-scale representations emerge for a variety of ensembles including face images, images from outdoor scenes and a database of optical flow vectors representing a motion class.

Feature selection is an efficient pre-processing method for high-dimensional data. It can promote the accuracy of classifiers by removing redundancy and outliers. However, the traditional feature selection method pay more attention on the correlation between samples and features while ignoring the correlation between samples and samples. When dealing with high-dimensional data with small sample sizes, those methods have poor performance. In this paper, we propose a new feature selection method based on sparse effective distance, which consider the local and global similarities among all the samples. Meanwhile, the proposed feature selection method combine with sparse principal component analysis (sPCA) to improve the sparsity and structure of loadings. The experiments were performed on eight gene expression data sets, and the results show that the proposed method can simultaneously improve the accuracy of classification and the stability.

The extension of internet facility profoundly eases the dissemination of all digital data such as audio, images and videos to the general public. This paper proposes a blind scheme for digital video watermarking. In this paper, a discrete wavelet domain watermarking is adopted for hiding a large amount of data with high security, good invisibility and no loss to the secret message. At first, dual tree complex wavelet transform (DTCWT) is applied on each frame decomposing it into a number of sub-bands. Then, holo entropy of each sub-band is calculated and the maximum entropy blocks are selected. The selected blocks are transformed using sparse principal component analysis (SPCA). The maximum coefficient of the SPCA blocks of each sub-band is quantised using quantisation index modulation (QIM). The proposed video watermarking scheme is analysed through various constraints such as the normalised correlation (NC) and the peak signal to noise ratio (PSNR).

Sparse principal component analysis (SPCA) imposes extra constraints or penalty terms to the standard PCA to achieve sparsity. In this paper, we introduce an efficient algorithm for finding an effective sparse feature principal component (PC) of multiple physiological signals. The algorithm consists of two stages. In the first stage, it identifies an active index set with a desired cardinality corresponding to the nonzero entries of the PC. In the second one, it uses the power iteration method to find the best direction with respect to the active index set. Experiments on randomly generated data and multiple physiological signals datasets show that our algorithm is very fast, especially on large and sparse data sets, while the numerical quality of the solution is comparable to the state-of-art algorithm.

Wind power is a popular renewable energy source, and the accurate prediction of wind speed plays an important role in improving the power generation efficiency of wind turbines and ensuring the normal operation of wind power equipment. Due to the instability and randomness of wind speed, it is difficult to achieve accurate prediction by traditional prediction methods. To improve the power generation efficiency of wind turbines and realize the predictability of wind speed, a hybrid wind speed prediction model based on GRUs (gated recurrent units) was constructed in this paper based on a deep neural network and feature extraction method. The hybrid model feature extraction module was implemented based on a combination of Tsfresh (a python package for time series feature extraction) and sparse PCA (sparse principal component analysis), and the network structure and other hyperparameters of the GRU module were determined through experiments. The model was validated using actual wind measurement data from a wind farm on the west coast of the United States. The results showed that the proposed model had less computational time and higher computational accuracy than the SARIMAX (seasonal auto-regressive integrated moving average with exogenous factors) and LSTM (long short-term memory) models.

ABSTRACTThe implementation of conventional sparse principal component analysis (SPCA) on high-dimensional data sets has become a time consuming work. In this paper, a series of subspace projections are constructed efficiently by using Householder QR factorization. With the aid of these subspace projections, a fast deflation method, called SPCA-SP, is developed for SPCA. This method keeps a good tradeoff between various criteria, including sparsity, orthogonality, explained variance, balance of sparsity, and computational cost. Comparative experiments on the benchmark data sets confirm the effectiveness of the proposed method.

This paper investigates the problem of sparse principal component analysis (SPCA), which is an extension of principal component analysis (PCA) for a sparser subspace of the original data. We propose a fully decentralized algorithm based on the primal-dual technique to solve SPCA in a distributed manner. The proposed algorithm has the ability to handle massively large datasets stored in multiple machines. The proposed method is shown to converge to stationary solutions of SPCA. Numerical experiments are provided to demonstrate the efficacy of our distributed primal-dual approach.