Unsupervised dimensionality reduction by jointing dynamic hypergraph and low-rank embedding for classification and clustering

The majority of the classical dimensionality reduction methods can be unified into a graph-embedding-based framework. A fixed graph constructed in a high-dimensional space has been extensively employed in the graph-embedding-based dimensionality reduction methods. However, a fixed graph often cannot characterize the structure of high-dimensional data owing to the curse of dimensionality. To solve this problem, we combine graph construction and dimensionality reduction into a coherent framework. Thus, the constructed graph can be updated dynamically in dimensionality reduction. In the existing methods based on the coherent framework, graphs are usually constructed by a type of neighborhood relationship and single clustering result. This study proposes an unsupervised dimensionality reduction method guided by fusing multiple clustering results. In the proposed method, multiple clustering results are first obtained by the k-means algorithm, and then a graph is constructed using a weighted co-association matrix of fusing the clustering results to capture data distribution information. Based on the graph, we present an objective function of combining graph construction and dimensionality reduction to implement mutual guidance between them. Numerical experiments on real data sets illustrate that the proposed method achieves significant improvement over some representative and state-of-the-art unsupervised dimensionality reduction methods.

This paper carries on research surrounding the influences produced by dimensionality reduction on machine learning classification effect. Firstly, paper constructs the analysis architecture of data dimension reduction classification, combines the two different unsupervised dimension reduction methods, locally linear embedding (LLE) and principal component analysis (PCA) with the five machine learning classification methods: Gradient Boosting Decision Tree (GBDT), Random Forest, Support Vector Machine (SVM), K-Nearest Neighbor (KNN) and Logistic Regression. And then uses the handwritten digital identification dataset to analyze the classification performance of these five classification methods on different dimension datasets by different dimensionality reduction methods. The analysis shows that using the appropriate dimensionality reduction method for dimensionality reduction classification can effectively improve the classification accuracy; the dimensionality reduction classification effect of non-linear dimensionality reduction method is generally better than the linear dimensionality reduction method; different machine learning classification algorithms have significant differences in the sensitivity of dimensions.

Unsupervised dimensionality reduction of medical hyperspectral imagery in tensor space

Surveys of neurosurgical outcome data have suggested suitable extents (aggressiveness) of resection to improve survival for many brain cancer types. Here, personalization of the extent of resection relies on highly subjective intraoperative diagnoses wherein the depth of the diagnostic information depends on the level of experience of the pathologist. Picosecond InfraRed Laser Mass Spectrometry (PIRL-MS) uses picosecond bursts of mid- infrared laser radiation to extract, in a non-thermal manner, tissue lipids (1) present in cellular membranes, giving cells and nuclei their unique shapes (utilized in morphometric pathology with staining & microscopy) and (2) altered in tumorigenesis due to cross-talk with cellular signalling in metabolism (potentially also revealing cancer molecular subtypes in 10 seconds as shown) for real-time profiling of molecular content unique to each tumour type. In our laboratory, close to 1, 300 frozen brain cancer specimens over 30 different classes of adult and pediatric cancers are being subjected PIRL-MS to build a comprehensive molecular signature library using various supervised and unsupervised dimensionality reduction and data analysis methods. The sensitivity and specificity of the integrated morphometric and molecular diagnosis for pediatric brain cancers with 10-second PIRL-MS was > 96%. This classification could use as little as only 18 tissue lipids whose identities were determined using chromatography and tandem mass spectrometry. Greater than 90% sensitivity and specificity have been obtained for adult brain cancer classifications with 10-second PIRL-MS analysis utilizing 40 lipids spanning fatty acids, phospholipids and ceramides. PIRL-MS can provide non-subjective differentiation between various brain cancer types with as little as 10 seconds of sampling and analysis time. Citation Format: Alexa Fiorante, Michael Woolman, David Munoz, Lan Anna Ye, Taira Kiyota, Sunit Das, Ahmed Aman, Howard Ginsberg, Gelareh Zadeh, Arash Zarrine-Afsar. Molecular determinants of 10-second diagnosis of brain cancer types with laser mass spectrometry [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 5932.

One of the central problems in machine learning and pattern recognition is how to deal with high-dimensional data either for visualization or for classification and clustering. Most of dimensionality reduction technologies, designed to cope with the curse of dimensionality, are based on Euclidean distance metric. In this work, we propose an unsupervised nonlinear dimensionality reduction method which attempt to preserve the distribution of input data, called distribution preserving embedding (DPE). It is done by minimizing the dissimilarity between the densities estimated in the original and embedded spaces. In theory, patterns in data can effectively be described by the distribution of the data. Therefore, DPE is able to discover the intrinsic pattern (structure) of data, including the global structures and the local structures. Additionally, DPE can be extended to cope with out-of-sample problem naturally. Extensive experiments on different data sets compared with other competing methods are reported to demonstrate the effectiveness of the proposed approach.

Single-cell Raman Spectroscopy (SCRS) emerges as a promising tool for single-cell phenotyping in environmental ecological studies, offering non-intrusive, high-resolution, and high-throughput capabilities. In this study, we obtained a large and the first comprehensive SCRS dataset that captured phenotypic variations with cell growth status for 36 microbial strains, and we compared and optimized analysis techniques and classifiers for SCRS-based taxonomy identification. First, we benchmarked five dimensionality reduction (DR) methods, 10 classifiers, and the impact of cell growth variances using a SCRS dataset with both taxonomy and cellular growth stage labels. Unsupervised DR methods and non-neural network classifiers are recommended for at a balance between accuracy and time efficiency, achieved up to 96.1% taxonomy classification accuracy. Second, accuracy variances caused by cellular growth variance (<2.9% difference) was found less than the influence from model selection (up to 41.4% difference). Remarkably, simultaneous high accuracy in growth stage classification (93.3%) and taxonomy classification (94%) were achievable using an innovative two-step classifier model. Third, this study is the first to successfully apply models trained on pure culture SCRS data to achieve taxonomic identification of microbes in environmental samples at an accuracy of 79%, and with validation via Raman-FISH (fluorescence in situ hybridization). This study paves the groundwork for standardizing SCRS-based biotechnologies in single-cell phenotyping and taxonomic classification beyond laboratory pure culture to real environmental microorganisms and promises advances in SCRS applications for elucidating organismal functions, ecological adaptability, and environmental interactions.

Riemannian geometry has been found accurate and robust for classifying multidimensional data, for instance, in brain-computer interfaces based on electroencephalography. Given a number of data points on the manifold of symmetric positive-definite matrices, it is often of interest to embed these points in a manifold of smaller dimension. This is necessary for large dimensions in order to preserve accuracy and useful in general to speed up computations. Geometry-aware methods try to accomplish this task while respecting as much as possible the geometry of the original data points. We provide a closed-form solution for this problem in a fully unsupervised setting. Through the analysis of three brain-computer interface data bases we show that our method allows substantial dimensionality reduction without affecting the classification accuracy.

Hyperspectral images (HSIs) possess a large number of spectral bands, which easily lead to the curse of dimensionality. To improve the classification performance, a huge challenge is how to reduce the number of spectral bands and preserve the valuable intrinsic information in the HSI. In this letter, we propose a novel unsupervised dimensionality reduction method called local neighborhood structure preserving embedding (LNSPE) for HSI classification. At first, LNSPE reconstructs each sample with its spectral neighbors and obtains the optimal weights for constructing the adjacency graph by modifying its loss function. Then, to discover the scatter information of the training samples, LNSPE minimizes the scatter between the pixels and the corresponding neighbors and maximizes the total scatter of the HSI data. Finally, it incorporates the scatter information and the dual graph structure to enhance the aggregation of the HSI. As a result, LNSPE can effectively reveal the intrinsic structure and improve the classification performance of the HSI data. The experimental results on two real hyperspectral data sets exhibit the efficiency and superiority of LNSPE to some state-of-the-art methods.

Kernel ridge regression for out-of-sample mapping in supervised manifold learning

Face recognition technology has evolved as an enchanting solution to address the contemporary needs in order to perform identification and verification of identity claims. By advancing the feature extraction methods and dimensionality reduction techniques in the application of pattern recognition, a number of face recognition systems has been developed with distinct degrees of success. Locality preserving projection (LPP) is a recently proposed method for unsupervised linear dimensionality reduction. LPP preserve the local structure of face image space which is usually more significant than the global structure preserved by principal component analysis (PCA) and linear discriminant analysis (LDA). This paper focuses on a systematic analysis of locality-preserving projections and the application of LPP in combination with an existing technique This combined approach of LPP through MPCA can preserve the global and the local structure of the face image which is proved very effective. Proposed approach is tested using the AT & T face database. Experimental results show the significant improvements in the face recognition performance in comparison with some previous methods.

Recent years have witnessed a surge of interest in graph-based transductive image classification. Existing simple graph-based transductive learning methods only model the pairwise relationship of images, however, and they are sensitive to the radius parameter used in similarity calculation. Hypergraph learning has been investigated to solve both difficulties. It models the high-order relationship of samples by using a hyperedge to link multiple samples. Nevertheless, the existing hypergraph learning methods face two problems, i.e., how to generate hyperedges and how to handle a large set of hyperedges. This paper proposes an adaptive hypergraph learning method for transductive image classification. In our method, we generate hyperedges by linking images and their nearest neighbors. By varying the size of the neighborhood, we are able to generate a set of hyperedges for each image and its visual neighbors. Our method simultaneously learns the labels of unlabeled images and the weights of hyperedges. In this way, we can automatically modulate the effects of different hyperedges. Thorough empirical studies show the effectiveness of our approach when compared with representative baselines.

Local appearance-based methods have been proposed recently for face recognition. We analyze the effects of different dimension reduction and normalization methods on local appearance-based face recognition in this paper. Each image is divided into equal sized blocks and six different dimension reduction methods are implemented for each block separately to create local visual feature vectors. On these local features, several normalization methods are applied in an attempt to eliminate the changes in lighting conditions and contrast differences among blocks of different face images. The experimental results show the improvements in recognition rates due to the effects of dimension reduction and normalization for three different classifiers. Usage of trainable dimension reduction methods instead of DCT and a new normalization method in our work (within-block normalization as referred in this paper) are two factors that makes difference from previous works in literature. The best performance is achieved using a block size of 16times16, performing dimension reduction using approximate pairwise accuracy criterion (aPAC) and applying within-block mean and variance normalization.

Reducing dimensionality is a key preprocessing step in many data analysis applications to address the negative effects of the curse of dimensionality and collinearity on model performance and computational complexity, to denoise the data or to reduce storage requirements. Moreover, in many applications it is desirable to reduce the input dimensions by choosing a subset of variables that best represents the entire set without any a priori information available. Unsupervised variable selection techniques provide a solution to this second problem. An autoencoder, if properly regularized, can solve both unsupervised dimensionality reduction and variable selection, but the training of large neural networks can be prohibitive in time sensitive applications. We present an approach called Recovery of Linear Components (RLC), which serves as a middle ground between linear and non-linear dimensionality reduction techniques, reducing autoencoder training times while enhancing performance over purely linear techniques. With the aid of synthetic and real world case studies, we show that the RLC, when compared with an autoencoder of similar complexity, shows higher accuracy, similar robustness to overfitting, and faster training times. Additionally, at the cost of a relatively small increase in computational complexity, RLC is shown to outperform the current state-of-the-art for a semiconductor manufacturing wafer measurement site optimization application.

Background Prediction of inflammatory bowel disease (IBD) development is the subject of intense study. We have previously shown that significant changes occur in routine laboratory tests in the 5 years preceding IBD diagnosis. Using routinely collected laboratory results, we developed machine learning models to predict development of IBD. Methods We extracted data from the electronic medical records of Maccabi Health Services and included patients with IBD ≥ 16 years of age with a minimum of 5 years follow up and an age and sex matched healthy cohort. The outcome measured was entry into the IBD registry. The study population was split into training and validation cohorts and using laboratory values in the 5 years prior to IBD diagnosis as features, we assessed the performance of supervised learning models and performed interpretability analysis with unsupervised dimensionality reduction. Results 5643 patients with IBD and 17199 healthy people were included in this study. 3039 (53.8%) had Crohn’s disease (CD), 2322 (41.1%) ulcerative colitis (UC) and 282 (5%) indeterminate colitis. The mean age of the IBD and healthy cohorts at study inclusion was 39.3 ± 16.5 and 38.5 ± 15.6 years, respectively. Weighted nearest neighbour imputation was used to correct for data missingness. Unsupervised Uniform Manifold Approximation Projection dimensionality reduction correctly separated the cohort into healthy and IBD clusters (Figure 1A). Furthermore, this algorithm identified two distinct groups correlating to CD or UC diagnosis (Figure 1B). Supervised learning models, namely neural net (AUC 0.95), linear support vector machine (SVM) (AUC 0.93), radial basis function SVM (AUC 0.85) and logistic regression (AUC 0.93) models, accurately predicted IBD diagnosis (Figure 2). These models were repeated without C-reactive protein and the predictive value was not significantly changed. Moreover, we excluded data in the 2 years preceding IBD diagnosis to determine whether model performance was reduced and the predictive value remained the same. Interestingly, using only data from 3-5 years preceding IBD diagnosis, the model could better distinguish between CD and UC (Figure 3). Conclusion In this study, machine learning models could accurately predict a diagnosis of IBD using laboratory data from the 5 years preceding IBD diagnosis. Prospective studies are needed to test these models in a real world setting.

Recent years have witnessed a great success of social media websites. Tag-based image search is an important approach to access the image content of interest on these websites. However, the existing ranking methods for tag-based image search frequently return results that are irrelevant or lacking in diversity. Most of the existing methods estimate the relevance of images by using tags and visual characteristics either separately or sequentially. The proposed system uses an approach that utilize simultaneously both visual information and textual information in real time to estimate the relevance of user tagged image. The method used to determine the relevance estimation is the hypergraph learning approach. The hypergraph is a generalization of a graph in which an edge in the hypergraph can be connected to any number of vertices. In the proposed method each social image can be represented by the bag-of-visual words and bag-of-textual words, which can be obtained from the textual content and visual content of the particular image. A hypergraph can be constructed in which the vertices represent the social images for ranking and the each hyperedge represents the visual words or tags that are obtained from the image. In the hypergraph learning scheme, both the visual content and tag information are taken into consideration at same time. Different from the method used by the traditional hypergraph, in the proposed system a social image hypergraph is constructed where vertices represent the images and hyperedges represent the visual or textual terms. The set of pseudo-positive images are used to achieve the learning, where the weight of hyperedges are updated throughout the learning process. Thus only the most relevant images are given to the user.