An Adaptive Unsupervised Feature Selection Algorithm Based on MDS for Tumor Gene Data Classification.

Over the years, many dimensionality reduction algorithms have been proposed for learning the structure of high dimensional data by linearly or non-linearly transforming it into a low-dimensional space. Some techniques can keep the local structure of data, while the others try to preserve the global structure. In this paper, we propose a linear dimensionality reduction technique that characterizes the local and global properties of data by firstly applying k-means algorithm on original data, and then finding the projection by simultaneously globally maximizing the between-cluster scatter matrix and locally minimizing the within-cluster scatter matrix, which actually keeps both local and global structure of data. Low complexity and structure preserving are two main advantages of the proposed technique. The experiments on both artificial and real data sets show the effectiveness and novelty of proposed algorithm in visualization and classification tasks.

Dimensionality reduction is an important operation in information visualization, feature extraction, clustering, regression, and classification, especially for processing noisy high dimensional data. However, most existing approaches preserve either the global or the local structure of the data, but not both. Approaches that preserve only the global data structure, such as principal component analysis (PCA), are usually sensitive to outliers. Approaches that preserve only the local data structure, such as locality preserving projections, are usually unsupervised (and hence cannot use label information) and uses a fixed similarity graph. We propose a novel linear dimensionality reduction approach, supervised discriminative sparse PCA with adaptive neighbors (SDSPCAAN), to integrate neighborhood-free supervised discriminative sparse PCA and projected clustering with adaptive neighbors. As a result, both global and local data structures, as well as the label information, are used for better dimensionality reduction. Classification experiments on nine high-dimensional datasets validated the effectiveness and robustness of our proposed SDSPCAAN.

Unsupervised feature selection via discrete spectral clustering and feature weights

The dimension reduction methods have been proved powerful and practical to extract latent features hidden in the signal for process monitoring. A novel nonlinear dimension reduction method called kernel orthogonal global-local preserving projections (KOGLPP) is proposed and applied for condition monitoring and fault detection. To overcome the shortcomings of kernel locality preserving projections (KLPP) and kernel principal component analysis (KPCA), the KOGLPP model aims at preserving the global and local data structures simultaneously by constructing a dual-objective optimization function, and a tuning parameter is introduced to adjust the trade-off between the global and local data structures. For the purpose of condition monitoring and fault detection, monitoring statistics are constructed in low dimensional feature space. As KOGLPP combines the advantages of both KPCA and KLPP, KOGLPP is also more powerful in extracting potential useful data characteristics. Finally, the effectiveness of the proposed nonlinear dimension reduction method is evaluated experimentally on a numerical example and a bearing test-rig. The results indicate its potential applications as an effective and reliable tool for condition monitoring and fault detection.

The problem of feature selection has been an area of considerable research in machine learning. Feature selection is known to be particularly difficult in unsupervised learning because different subgroups of features can yield useful insights into the same dataset. In other words, many theoretically-right answers may exist for the same problem. Furthermore, designing algorithms for unsupervised feature selection is technically harder than designing algorithms for supervised feature selection because unsupervised feature selection algorithms cannot be guided by class labels. As a result, previous work attempts to discover intrinsic structures of data with heavy computation such as matrix decomposition, and require significant time to find even a single solution. This paper proposes a novel algorithm, named Explainability-based Unsupervised Feature Value Selection (EUFVS), which enables a paradigm shift in feature selection, and solves all of these problems. EUFVS requires only a few tens of milliseconds for datasets with thousands of features and instances, allowing the generation of a large number of possible solutions and select the solution with the best fit. Another important advantage of EUFVS is that it selects feature values instead of features, which can better explain phenomena in data than features. EUFVS enables a paradigm shift in feature selection. This paper explains its theoretical advantage, and also shows its applications in real experiments. In our experiments with labeled datasets, EUFVS found feature value sets that explain labels, and also detected useful relationships between feature value sets not detectable from given class labels.KeywordsFeature selectionUnsupervised learningClustering

Feature selection plays an important role in preprocessing in pattern recognition and data mining, especially in large scale image, digital text, and biological data. Specifically, class label information is unavailable for conducting the selection of a minimal feature subset in unsupervised learning, which is full of challenging and interesting problems. In this paper, we present an unsupervised REPresentative Feature Selection (REPFS) algorithm based on information entropy and relevance analysis. The proposed method seeks to find a high-quality feature subset through feature clustering without using any learning algorithms. More importantly, the features’ relevance will be computed based on an information metric of the relevance gain, which provides an information theoretical foundation for finding a minimum of the redundancy between features. Our results on nine benchmark data sets demonstrate that REPFS can significantly improve upon state-of-the-art unsupervised algorithms.

The linear discriminant analysis (LDA) is a very popular linear feature extraction approach. The algorithms of LDA usually perform well under the following two assumptions. The first assumption is that the global data structure is consistent with the local data structure. The second assumption is that the input data classes are Gaussian distributions. However, in real-world applications, these assumptions are not always satisfied. In this paper, we propose an improved LDA framework, the local LDA (LLDA), which can perform well without needing to satisfy the above two assumptions. Our LLDA framework can effectively capture the local structure of samples. According to different types of local data structure, our LLDA framework incorporates several different forms of linear feature extraction approaches, such as the classical LDA and principal component analysis. The proposed framework includes two LLDA algorithms: a vector-based LLDA algorithm and a matrix-based LLDA (MLLDA) algorithm. MLLDA is directly applicable to image recognition, such as face recognition. Our algorithms need to train only a small portion of the whole training set before testing a sample. They are suitable for learning large-scale databases especially when the input data dimensions are very high and can achieve high classification accuracy. Extensive experiments show that the proposed algorithms can obtain good classification results.

Object recognition accuracy can be improved when information from multiple views is integrated, but information in each view can often be highly redundant. We consider the problem of distributed object recognition or indexing from multiple cameras, where the computational power available at each camera sensor is limited and communication between cameras is prohibitively expensive. In this scenario, it is desirable to avoid sending redundant visual features from multiple views. Traditional supervised feature selection approaches are inapplicable as the class label is unknown at each camera. In this paper we propose an unsupervised multi-view feature selection algorithm based on a distributed coding approach. With our method, a Gaussian process model of the joint view statistics is used at the receiver to obtain a joint encoding of the views without directly sharing information across encoders. We demonstrate our approach on recognition and indexing tasks with multi-view image databases and show that our method compares favorably to an independent encoding of the features from each camera.

Feature selection plays a crucial role in scientific research and practical applications. In the real world applications, labeling data is time and labor consuming. Thus, unsupervised feature selection methods are desired for many practical applications. Linear discriminant analysis (LDA) with trace ratio criterion is a supervised dimensionality reduction method that has shown good performance to improve classifications. In this paper, we first propose a unified objective to seamlessly accommodate trace ratio formulation and K-means clustering procedure, such that the trace ratio criterion is extended to unsupervised model. After that, we propose a novel unsupervised feature selection method by integrating unsupervised trace ratio formulation and structured sparsity-inducing norms regularization. The proposed method can harness the discriminant power of trace ratio criterion, thus it tends to select discriminative features. Meanwhile, we also provide two important theorems to guarantee the unsupervised feature selection process. Empirical results on four benchmark data sets show that the proposed method outperforms other sate-of-the-art unsupervised feature selection algorithms in all three clustering evaluation metrics.

A new nonlinear dimensionality reduction method called data-dependent kernel global–local preserving projections (DDKGLPP) is proposed and used for process monitoring. To achieve performance improvements, DDKGLPP uses a data-dependent kernel rather than a conventional kernel. A unified kernel optimization framework is developed to optimize the data-dependent kernel by minimizing a data structure preserving index. The optimized kernel can unfold both global and local data structures in the feature space. The data-dependent kernel principal component (DDKPCA) and data-dependent kernel locality preserving projections (DDKLPP) also can be developed under the unified kernel optimization framework. However, unlike DDKPCA and DDKLPP, DDKGLPP is able to preserve both global and local structures of the data set when performing dimensionality reduction. Consequently, DDKGLPP is more powerful in capturing useful data characteristics. A DDKGLPP-based monitoring method is then proposed for nonlinear processes. Its per...

Existing unsupervised feature selection algorithms are designed to extract the most relevant subset of features which can facilitate clustering and interpretation of the obtained results. However, these techniques are not directly applicable in many real-world situations in which one has access to datasets consisting of multiple views or dissimilar representations. By leveraging information from these different views one can obtain more robust and accurate results compared with single-view methods. In this paper Unsupervised Multi-View Feature Selection algorithm is used to simultaneously extract a relevant subset of features and to perform clustering that is consistent across different views. In the empirical evaluation of synthetic multi-view data we demonstrate the advantage of using the proposed algorithm over the use of its single-view analogue. We used the National Cancer Institute's NCI-60 panel dataset, exploiting information from various -omics representations to identify tumor subtypes based on very few requisite, biologically relevant features.

Unsupervised feature selection always occupies a key position as a preprocessing in the tasks of classification or clustering due to the existence of extra essential features within high-dimensional data. Although lots of efforts have been made, the existing methods neglect to consider the redundancy of features, and thus select redundant features. In this brief, by virtue of a generalized uncorrelated constraint, we present an improved sparse regression model [generalized uncorrelated regression model (GURM)] for seeking the uncorrelated yet discriminative features. Benefited from this, the structure of data is kept in the Stiefel manifold, which avoids the potential trivial solution triggered by a conventional ridge regression model. Besides that, the uncorrelated constraint equips the model with the closed-form solution. In addition, we also incorporate a graph regularization term based on the principle of maximum entropy into the GURM model (URAFS), so as to embed the local geometric structure of data into the manifold learning. An efficient algorithm is designed to perform URAFS by virtue of the existing generalized powered iteration method. Extensive experiments on eight benchmark data sets among seven state-of-the-art methods on the task of clustering are conducted to verify the effectiveness and superiority of the proposed method.

Symmetric low-rank representation with adaptive distance penalty for semi-supervised learning

Technological advancements have generated huge amount of data in recent times. The objective is to analyze this data to get meaningful insights from them. One of the techniques is to perform clustering, an unsupervised learning technique. The performance of clustering is dependent on the size of data (number of features, number of records), a well-known fact. Feature selection/extraction algorithms not only eliminate irrelevant features from the data but also improve the performance of the clustering algorithm applied. Knowing your data sets (type, nature, and size), the appropriate algorithm to use and the appropriate evaluation metrics that will be insightful to improve the quality of clusters is important. No such correlation has been established as per author’s knowledge. In this paper, we report the results of various internal and external evaluation metrics on application of distinct unsupervised feature selection/extraction algorithms on a variety of data sets. The output of various algorithms indicates that retaining 100 features give best results. The experiments prove that Principal Component Analysis (PCA) algorithm is suitable for linear and low-dimensional data sets. Laplacian Score, Non-negative Discriminative Feature Selection (NDFS), and Spectral Feature Selection (SPEC) work well with high dimensional, binary data sets. Low variance algorithm and Unsupervised Discriminative Feature Selection (UDFS) do not perform well on any type of data sets.

Recently, low-rank and sparse model-based dimensionality reduction (DR) methods have aroused lots of interest. In this paper, we propose an effective supervised DR technique named block-diagonal constrained low-rank and sparse-based embedding (BLSE). BLSE has two steps, i.e., block-diagonal constrained low-rank and sparse representation (BLSR) and block-diagonal constrained low-rank and sparse graph embedding (BLSGE). Firstly, the BLSR model is developed to reveal the intrinsic intra-class and inter-class adjacent relationships as well as the local neighborhood relations and global structure of data. Particularly, there are mainly three items considered in BLSR. First, a sparse constraint is required to discover the local data structure. Second, a low-rank criterion is incorporated to capture the global structure in data. Third, a block-diagonal regularization is imposed on the representation to promote discrimination between different classes. Based on BLSR, informative and discriminative intra-class and inter-class graphs are constructed. With the graphs, BLSGE seeks a low-dimensional embedding subspace by simultaneously minimizing the intra-class scatter and maximizing the inter-class scatter. Experiments on public benchmark face and object image datasets demonstrate the effectiveness of the proposed approach.