Unsupervised feature extraction based on uncorrelated approach

Identifying differentially expressed genes is difficult because of the small number of available samples compared with the large number of genes. Conventional gene selection methods employing statistical tests have the critical problem of heavy dependence of P-values on sample size. Although the recently proposed principal component analysis (PCA) and tensor decomposition (TD)-based unsupervised feature extraction (FE) has often outperformed these statistical test-based methods, the reason why they worked so well is unclear. In this study, we aim to understand this reason in the context of projection pursuit (PP) that was proposed a long time ago to solve the problem of dimensions; we can relate the space spanned by singular value vectors with that spanned by the optimal cluster centroids obtained from K-means. Thus, the success of PCA- and TD-based unsupervised FE can be understood by this equivalence. In addition to this, empirical threshold adjusted P-values of 0.01 assuming the null hypothesis that singular value vectors attributed to genes obey the Gaussian distribution empirically corresponds to threshold-adjusted P-values of 0.1 when the null distribution is generated by gene order shuffling. For this purpose, we newly applied PP to the three data sets to which PCA and TD based unsupervised FE were previously applied; these data sets treated two topics, biomarker identification for kidney cancers (the first two) and the drug discovery for COVID-19 (the thrid one). Then we found the coincidence between PP and PCA or TD based unsupervised FE is pretty well. Shuffling procedures described above are also successfully applied to these three data sets. These findings thus rationalize the success of PCA- and TD-based unsupervised FE for the first time.

Unsupervised multi-view feature extraction with dynamic graph learning

In this paper, a deep-learning based architecture is proposed to estimate gender which includes both supervised and unsupervised facial feature extraction techniques and a deep network to fuse features and to classify them. As unsupervised techniques we have benefited deep-learning feature extraction method called convolutional neural network, and for supervised facial feature extraction we have used facial Action Units and Histogram of Oriented Gradients. Our proposed architecture has been tested on multiple datasets and for most of them the accuracy is considerably high.

Graph convolutional networks (GCN) excel in rapidly mining semantic information from data with superior capabilities, enabling deep feature extraction and classification for hyperspectral images (HSI). However, supervised classification methods heavily rely on many high-quality labeled samples, which is often a significant limitation in real-world applications. Unsupervised learning, with its unique ability to automatically discover potential structures in data without labeled samples, provides a new approach to address this issue. Thus, this paper proposes an unsupervised dual-masked graph autoencoder feature extraction network (UDMG) for HSI classification. This method first utilizes a masked autoencoder to mask and reconstruct the edges and node features of the graph structure. Meanwhile, the encoder combines GCN and Transformer to capture local and global features in HSI. To validate the performance of UDMG, we employ various evaluation methods and conduct experiments on three publicly available hyperspectral datasets. The results demonstrate that UDMG exhibits superior classification performance compared to existing state-of-the-art unsupervised feature extraction methods.

An automated classification system based on a Deep Learning (DL) technique for Cardiac Disease (CD) monitoring and detection is proposed in this paper. The proposed DL architecture is divided into Deep Auto-Encoders (DAEs) as an unsupervised form of feature learning and Deep Neural Networks (DNNs) as a classifier. The objective of this study is to improve on the previous machine learning technique that consists of several data processing steps such as feature extraction and feature selection or feature reduction. It is also noticed that the previously used machine learning technique required human interference and expertise in determining robust features, yet was time-consuming in the labeling and data processing steps. In contrast, DL enables an embedded feature extraction and feature selection in DAEs pre-training and DNNs fine-tuning process directly from raw data. Hence, DAEs is able to extract high-level of features not only from the training data but also from unseen data. The proposed model uses 10 classes of imbalanced data from ECG signals. Since it is related to the cardiac region, abnormality is usually considered for an early diagnosis of CD. In order to validate the result, the proposed model is compared with the shallow models and DL approaches. Results found that the proposed method achieved a promising performance with 99.73% accuracy, 91.20% sensitivity, 93.60% precision, 99.80% specificity, and a 91.80% F1-Score. Moreover, both the Receiver Operating Characteristic (ROC) curve and the Precision-Recall (PR) curve from the confusion matrix showed that the developed model is a good classifier. The developed model based on unsupervised feature extraction and deep neural network is ready to be used on a large population before its installation for clinical usage.

We applied principal component analysis (PCA)-based unsupervised feature extraction (FE) to amyotrophic lateral sclerosis (ALS) gene expression profiles. ALS is a debilitating neurodegenerative disorder with no effective therapy. The relevant gene expression profiles contained a small number of samples (from a few to tens) with a large number of features (several tens of thousands). Although it is important to recognize critical genes from gene expression profiles, a small-sample-large-feature situation makes FE difficult. In PCA-based unsupervised FE, features rather than samples are embedded into a low dimensional space, and critical genes are identified as outliers that are supposed to obey group-oriented behavior. The 29 candidate genes identified as critical for ALS by this methodology turned out to be biologically feasible based on comparisons with numerous previous studies. Together, they formed a collected gene regulation/protein binding network within which the known, but not explicitly identified in this study, three ALS-causing genes, SOD1, TDP-43, and SETX, could be naturally placed/embedded. Among the 29 genes, the translated chemokine receptor CCR6 protein was considered to be a potential therapy target and its antagonists/agonists were identified using the in silico drug discovery software ChooseLD. The ten top-ranked antagonists/agonists shared structures with many compounds that were previously known to bind to various proteins.

Feature extraction (FE) is a crucial research area in hyperspectral image (HSI) processing. Recently, due to the powerful ability of deep learning (DL) to extract spatial and spectral features, DL-based FE methods have shown great potentials for HSI processing. However, most of the DL-based FE methods are supervised, and the training of them suffers from the absence of labeled samples in HSIs severely. The training issue of supervised DL-based FE methods limits their application on HSI processing. To address this issue, in this paper, a novel modified generative adversarial network (GAN) is proposed to train a DL-based feature extractor without supervision. The designed GAN consists of two components, which are a generator and a discriminator. The generator can focus on the learning of real probability distributions of data sets and the discriminator can extract spatial-spectral features with superior invariance effectively. In order to learn upsampling and downsampling strategies adaptively during FE, the proposed generator and discriminator are designed based on a fully deconvolutional subnetwork and a fully convolutional subnetwork, respectively. Moreover, a novel min-max cost function is designed for training the proposed GAN in an end-to-end fashion without supervision, by utilizing the zero-sum game relationship between the generator and discriminator. Besides, the proposed modified GAN replaces the original Jensen-Shannon divergence with the Wasserstein distance, aiming to mitigate the unstability and difficulty of the training of GAN frameworks. Experimental results on three real data sets validate the effectiveness of the proposed method.

Generating pixel-level annotations for semantic segmentation tasks of high-resolution remote sensing images is both time-consuming and labor-intensive, which has led to increased interest in unsupervised methods. Therefore, in this paper, we propose an unsupervised multi-scale hybrid feature extraction network based on the CNN-Transformer architecture, referred to as MSHFE-Net. The MSHFE-Net consists of three main modules: a Multi-Scale Pixel-Guided CNN Encoder, a Multi-Scale Aggregation Transformer Encoder, and a Parallel Attention Fusion Module. The Multi-Scale Pixel-Guided CNN Encoder is designed for multi-scale, fine-grained feature extraction in unsupervised tasks, efficiently recovering local spatial information in images. Meanwhile, the Multi-Scale Aggregation Transformer Encoder introduces a multi-scale aggregation module, which further enhances the unsupervised acquisition of multi-scale contextual information, obtaining global features with stronger feature representation. The Parallel Attention Fusion Module employs an attention mechanism to fuse global and local features in both channel and spatial dimensions in parallel, enriching the semantic relations extracted during unsupervised training and improving the performance of unsupervised semantic segmentation. K-means clustering is then performed on the fused features to achieve high-precision unsupervised semantic segmentation. Experiments with MSHFE-Net on the Potsdam and Vaihingen datasets demonstrate its effectiveness in significantly improving the accuracy of unsupervised semantic segmentation.

ABSTRACTDeep learning models have been widely applied to extract the high-level features of hyperspectral images (HSIs) due to their strong data mining ability, and some encouraging results have been obtained. Sufficient labelled samples are the guarantee of the network performance. However, the collection of labelled hyperspectral data is time consuming and labour consuming. To get rid of the limitation of labelled samples, an unsupervised feature extractor is proposed based on transfer learning and an improved Wasserstein generative adversarial network (WGAN-GP) in this paper. In addition, considering the spectral and spatial characteristics of HSIs, the framework is designed in three-dimensional (3D) form which helps fully exploit the information in hyperspectral data. Thanks to transfer learning, the well-trained discriminator of WGAN-GP is used as a feature extractor for learning the spectral-spatial features without involving labelled samples. Moreover, in order to facilitate the optimization of WGAN-GP, we propose a novel dimensionality reduction (DR) method using convolutions to get lower-dimensional target data. Experimental results on real hyperspectral datasets demonstrate the efficiency of the proposed method combining the proposed DR and WGAN-GP(DR+WGAN-GP), which has great potential prospects in HSI classification.

The spike sorting, including neuronal spike waveform acquisition and classification, is one of the important procedures in neuronal information processing, and its feature extraction and recognition are the basis of the above issues. Based on Locality Preserving Projection ( LPP) algorithm, an unsupervised feature extraction and classification algorithm was proposed. The neighbor parameter was selected automatically, the distribution dispersion standard was obtained according to the original data set, and the features of extraction results in spikes were separated effectively. The application in simulation and real experimental data show that, the proposed method based on the LPP can more effectively extract and separate features of spikes in comparison of the traditional Principle Component Analysis ( PCA) algorithm.

In the multimedia processing, the extraction and the representation of characteristics are considered as an important step. The extraction of the ideal characteristics having the ability to reflect the intrinsic content of the images as complete as possible is still a difficult problem in computer vision. Little research has focused on this problem. This paper presents a new unsupervised method based on the self-organizing map (SOM) for features extraction. Our method consists of two main steps: Extracting the sub-regions of an image according to their points of interest and using the SOM for the different views of an image such as color, texture and shape. Then, combine them to have finally a > vector characteristic. The proposed method is evaluated on three image classification datasets Cloud, Coil100 and CIFAR-10. The classification accuracies of the proposed method for the three datasets are much higher in comparison with the other methods cited in the literature.

Due to missed sample labeling, unsupervised feature selection during single-cell (sc) RNA-seq can identify critical genes under the experimental conditions considered. In this paper, we applied principal component analysis (PCA)-based unsupervised feature extraction (FE) to identify biologically relevant genes from mouse and human embryonic brain development expression profiles retrieved by scRNA-seq. When evaluating the biological relevance of selected genes by various enrichment analyses, the PCA-based unsupervised FE outperformed conventional unsupervised approaches that select highly variable genes as well as bimodal genes in addition to the recently proposed dpFeature.

As a non-destructive sensing technique, Raman spectroscopy is often combined with regression models for real-time detection of key components in microbial cultivation processes. However, achieving accurate model predictions often requires a large amount of offline measurement data for training, which is both time-consuming and labor-intensive. In order to overcome the limitations of traditional models that rely on large datasets and complex spectral preprocessing, in addition to the difficulty of training models with limited samples, we have explored a genetic algorithm-based semi-supervised convolutional neural network (GA-SCNN). GA-SCNN integrates unsupervised process spectral labeling, feature extraction, regression prediction, and transfer learning. Using only an extremely small number of offline samples of the target protein, this framework can accurately predict protein concentration, which represents a significant challenge for other models. The effectiveness of the framework has been validated in a system of Escherichia coli expressing recombinant ProA5M protein. By utilizing the labeling technique of this framework, the available dataset for glucose, lactate, ammonium ions, and optical density at 600 nm (OD600) has been expanded from 52 samples to 1302 samples. Furthermore, by introducing a small component of offline detection data for recombinant proteins into the OD600 model through transfer learning, a model for target protein detection has been retrained, providing a new direction for the development of associated models. Comparative analysis with traditional algorithms demonstrates that the GA-SCNN framework exhibits good adaptability when there is no complex spectral preprocessing. Cross-validation results confirm the robustness and high accuracy of the framework, with the predicted values of the model highly consistent with the offline measurement results.

In many fields such as data mining, machine learning, pattern recognition and signal processing, data sets containing huge number of features are often involved. Feature selection is an essential data preprocessing technique for such high-dimensional data classification tasks. Traditional dimensionality reduction approach falls into two categories: Feature Extraction (FE) and Feature Selection (FS). Principal component analysis is an unsupervised linear FE method for projecting high-dimensional data into a low-dimensional space with minimum loss of information. It discovers the directions of maximal variances in the data. The Rough set approach to feature selection is used to discover the data dependencies and reduction in the number of attributes contained in a data set using the data alone, requiring no additional information. For selecting discriminative features from principal components, the Rough set theory can be applied jointly with PCA, which guarantees that the selected principal components will be the most adequate for classification. We call this method Rough PCA. The proposed method is successfully applied for choosing the principal features and then applying the Upper and Lower Approximations to find the reduced set of features from a gene expressiondata.

The rapid development of information and communication technologies has facilitated machining condition monitoring toward a data-driven paradigm, of which the Industrial Internet of Things (IIoT) serves as the fundamental basis to acquire data from physical equipment with sensing technologies as well as to learn the relationship between the system condition and the collected condition monitoring data. However, most data-driven methods suffer from using a single-domain space, ignoring the importance of the learned features, and failing to incorporate the handcrafted features assisted by domain knowledge. To solve these limitations, a novel deep learning approach is proposed for machining condition monitoring in the IIoT environment, which consists of three phases, including: 1) the unsupervised parallel feature extraction; 2) adaptive feature importance weighting; and 3) hybrid feature fusion. First, separate sparse autoencoders are utilized to conduct the unsupervised parallel feature extraction, which enables to learn abstract feature representation from multiple domain spaces simultaneously. Then, an attention module is designed for the adaptive feature importance weighting, which can assign higher weights to those critical features accordingly. Moreover, a hybrid feature fusion is deployed to complement the automatic feature learning and further yield better model performance by fusing the handcrafted features assisted by domain knowledge. Finally, a real-life case study and extensive experiments have been conducted to show the effectiveness and superiority of the proposed approach.