Bayesian unidimensional scaling

Distributed feature selection (DFS) strategy for microarray gene expression data to improve the classification performance

Tracking recurrence of correlation structure in neuronal recordings

The present work strives to investigate the effect of using dimensionality reduction techniques (DRTs) on breast cancer (BC) classification problem. Primarily, we focused on the following five (DRTs): Auto-Encoders (AE), T-Distributed Stochastic Neighbor Embedding (T- SNE), Recursive Feature Elimination (RFE), Isometric Feature Mapping (Isomap), and Principle Component analysis (PCA). These methods are combined with two famous classifiers that are Support Vector Machine (SVM) and Multilayer perceptron (MLP). They are used for BC classification. Breast Cancer Wisconsin Diagnostic (WDBC) data set was used to validate the experiments of this work. The former was provided by the University of California, Irvine (UCI) machine learning repository. The results demonstrated that combining MLP with the chosen (DRTs) methods increased the classification accuracy for almost all built models by at least 0.7%. In addition, they revealed a decrease in the classification accuracy using SVM as a classifier for almost all built models.KeywordsMachine learningDeep learningDimensionality reductionNeural networksBreast cancer predictionMedical dataset

Hierarchically clustered heatmaps are a popular visualization technique to display high-dimensional biological datasets as figures in research publications. Most tools developed to visualize hierarchically clustered heatmaps generate static images. Clustergrammer is a web-based matrix visualization tool built, using the D3 JavaScript library, to enable experimental and computational-biologists to easily generate highly interactive heatmaps. Clustergrammer implements many interactive features including: zooming, panning, filtering, sorting, reordering, dimensionality reduction, gene set enrichment analysis, gene description text, and shareable persistent links. Clustergrammer can be used to visualize and analyze a diverse set of high-dimensional biological data including: gene expression, protein-protein interactions, and CyTOF single-cell proteomics data. Clustergrammer provides interactive visualizations for several web-tools including: Enrichr, GEN3VA, GEO2Enrichr, Harmonizome, and L1000CDS2. In addition, Clustergrammer can be embedded within IPython Jupyter Notebooks or as a standalone open-source JavaScript or Python library.

Machine learning methods and in particular random forests are promising approaches for prediction based on high dimensional omics data sets. They provide variable importance measures to rank predictors according to their predictive power. If building a prediction model is the main goal of a study, often a minimal set of variables with good prediction performance is selected. However, if the objective is the identification of involved variables to find active networks and pathways, approaches that aim to select all relevant variables should be preferred. We evaluated several variable selection procedures based on simulated data as well as publicly available experimental methylation and gene expression data. Our comparison included the Boruta algorithm, the Vita method, recurrent relative variable importance, a permutation approach and its parametric variant (Altmann) as well as recursive feature elimination (RFE). In our simulation studies, Boruta was the most powerful approach, followed closely by the Vita method. Both approaches demonstrated similar stability in variable selection, while Vita was the most robust approach under a pure null model without any predictor variables related to the outcome. In the analysis of the different experimental data sets, Vita demonstrated slightly better stability in variable selection and was less computationally intensive than Boruta.In conclusion, we recommend the Boruta and Vita approaches for the analysis of high-dimensional data sets. Vita is considerably faster than Boruta and thus more suitable for large data sets, but only Boruta can also be applied in low-dimensional settings.

Feature selection is an important step in data analysis to address the curse of dimensionality. Such dimensionality reduction techniques are particularly important when if a classification is required and the model scales in polynomial time with the size of the feature (e.g., some applications include genomics, life sciences, cyber-security, etc.). Feature selection is the process of finding the minimum subset of features that allows for the maximum predictive power. Many of the state-of-the-art information-theoretic feature selection approaches use a greedy forward search; however, there are concerns with the search in regards to the efficiency and optimality. A unified framework was recently presented for information-theoretic feature selection that tied together many of the works in over the past twenty years. The work showed that joint mutual information maximization (JMI) is generally the best options; however, the complexity of greedy search for JMI scales quadratically and it is infeasible on high dimensional datasets. In this contribution, we propose a fast approximation of JMI based on information theory. Our approach takes advantage of decomposing the calculations within JMI to speed up a typical greedy search. We benchmarked the proposed approach against JMI on several UCI datasets, and we demonstrate that the proposed approach returns feature sets that are highly consistent with JMI, while decreasing the run time required to perform feature selection.

Numerical modelling increasingly generates massive, high-dimensional spatio-temporal datasets. Exploring such datasets relies on effective visualization. This study presents a generic workflow to (i) project high-dimensional spatio-temporal data on a two-dimensional (2D) plane accurately (ii) compare dimensionality reduction techniques (DRTs) in terms of resolution and computational efficiency (iii) represent 2D projection spatially using a 2D perceptually uniform background color map. Machine learning (ML) based DRTs for data visualization i.e., principal component analysis (PCA), generative topographic mapping (GTM), t-distributed stochastic neighbor embedding (t-SNE) and uniform manifold approximation and projection (UMAP) are compared in terms of accuracy, resolution and computational efficiency to handle massive datasets. The accuracy of visualization is evaluated using a quality metric based on a co-ranking framework. The workflow is applied to an output of an Australian Water Resource Assessment (AWRA) model for Tasmania, Australia. The dataset consists of daily time series of nine components of the water balance at a 5 km grid cell resolution for the year 2017. The case study shows that PCA allows rapid visualization of global data structures, while t-SNE and UMAP allows more accurate representation of local trends. Furthermore, UMAP is computationally more efficient than t-SNE and least affected by the outliers compared to GTM.

BackgroundCanonical correlation analysis (CCA) is a multivariate statistical method which describes the associations between two sets of variables. The objective is to find linear combinations of the variables in each data set having maximal correlation. In genomics, CCA has become increasingly important to estimate the associations between gene expression data and DNA copy number change data. The identification of such associations might help to increase our understanding of the development of diseases such as cancer. However, these data sets are typically high-dimensional, containing a lot of variables relative to the number of objects. Moreover, the data sets might contain atypical observations since it is likely that objects react differently to treatments. We discuss a method for Robust Sparse CCA, thereby providing a solution to both issues. Sparse estimation produces canonical vectors with some of their elements estimated as exactly zero. As such, their interpretability is improved. Robust methods can cope with atypical observations in the data.ResultsWe illustrate the good performance of the Robust Sparse CCA method by several simulation studies and three biometric examples. Robust Sparse CCA considerably outperforms its main alternatives in (1) correctly detecting the main associations between the data sets, in (2) accurately estimating these associations, and in (3) detecting outliers.ConclusionsRobust Sparse CCA delivers interpretable canonical vectors, while at the same time coping with outlying observations. The proposed method is able to describe the associations between high-dimensional data sets, which are nowadays commonplace in genomics.Furthermore, the Robust Sparse CCA method allows to characterize outliers.Electronic supplementary materialThe online version of this article (doi:10.1186/s12918-016-0317-9) contains supplementary material, which is available to authorized users.

Interpretable machine learning models for gene expression datasets are important for understanding the decision-making process of a classifier and gaining insights on the underlying molecular processes of genetic conditions. Interpretable models can potentially support early diagnosis before full disease manifestation. This is particularly important yet, challenging for mental health. We hypothesise this is due to extreme heterogeneity issues which may be overcome and explained by personalised modelling techniques. Thus far, most machine learning methods applied to gene expression datasets, including deep neural networks, lack personalised interpretability. This paper proposes a new methodology named personalised constrained neuro fuzzy inference (PCNFI) for learning personalised rules from high dimensional datasets which are structurally and semantically interpretable. Case studies on two mental health related datasets (schizophrenia and bipolar disorders) have shown that the relatively short and simple personalised fuzzy rules provided enhanced interpretability as well as better classification performance compared to other commonly used machine learning methods. Performance test on a cancer dataset also showed that PCNFI matches previous benchmarks. Insights from our approach also indicated the importance of two genes (ATRX and TSPAN2) as possible biomarkers for early differentiation of ultra-high risk, bipolar and healthy individuals. These genes are linked to cognitive ability and impulsive behaviour. Our findings suggest a significant starting point for further research into the biological role of cognitive and impulsivity-related differences. With potential applications across bio-medical research, the proposed PCNFI method is promising for diagnosis, prognosis, and the design of personalised treatment plans for better outcomes in the future.

In the oil and gas industry, an extensive volume of logging data is routinely generated and acquired during operations. This data is accumulated across various phases of exploration, production, and subsequent operations. The challenge lies in extracting meaningful insights from this high-dimensional dataset. Constraints in data visualization (such as the three spatial axes, x, y, and z) and manipulation further complicate the process of drawing valuable conclusions. As a method to tackle this challenge, a technique has been devised to streamline data visualization by reducing high-dimensional data to a lower dimensionality, while retaining vital features crucial for big data analysis. Principal component analysis (PCA) emerges as a prominent approach employed for data dimensionality reduction. PCA simplifies multivariate datasets encompassing well-logging tools (e.g., gamma ray, density, neutron porosity, and resistivity) and drilling parameters such as rate of penetration, weight on bit, and vibration data, condensing them into a reduced number of factors known as principal components (PCs). The results of the dimensionality reduction technique show that various logging data and drilling parameters were condensed into a set of principal components (PC1, PC2…PCx), where x corresponds to the number of utilized variables. The analysis indicates that the first two components (PC1, PC2) capture most of data patterns. The graphical representation of these components (PCA biplot) reveals distinct clusters with clear patterns, facilitating the identification of separate electro-facies. Moreover, PC1 exhibits a strong correlation with lithology variations, enabling its utilization in well-to-well correlation. Additionally, regression analysis demonstrates a significant predictive relationship between PCA components and well logging variables, allowing the use of the R-squared regression technique to forecast a result curve based on PCA input. Higher principal components are found to be more associated with formation fluid, thus complementing the standard petrophysical analysis. The industry's current shift is from data collection to practical applications. By employing diverse dimensionality reduction techniques, the workflow enables the analysis of big data in a more comprehensive manner, unveiling concealed trends and insights. This approach not only facilitates machine-learning and artificial-intelligence applications but also enables a deeper understanding of the data through descriptive forms, thereby supporting industry advancements.

Non-negative Matrix Factorization (NMF) and Singular Value Decomposition (SVD) are widely recognized as pivotal dimensionality reduction techniques in the literature, particularly for deep learning applications involving large and high-dimensional datasets like SEM images. This study systematically evaluates the impact of SVD and NMF on the performance, efficiency, and energy consumption of four deep learning architectures: GoogleNet, AlexNet, ResNet, and SqueezeNet. By applying these techniques to reduce dataset dimensions, we observed that SVD excelled in computational efficiency, achieving up to 35% faster processing times compared to raw datasets. NMF, on the other hand, provided superior feature interpretability, which proved beneficial for tasks requiring meaningful pattern extraction. Energy consumption analysis revealed that SVD led to a 28% reduction in computational energy cost on average, making it a practical choice for resource-constrained environments. Among the evaluated models, ResNet consistently delivered the highest classification accuracy after dimensionality reduction, showing an improvement of 4-6% over models trained on non-reduced data. These findings underscore the critical role of dimensionality reduction in enhancing the scalability, energy efficiency, and classification accuracy of deep learning models, offering valuable insights for optimizing high-dimensional data applications in both academic and industrial contexts.

High-dimensional systems are increasingly common in fields such as remote sensing, communications, and biomedical signal processing, where data typically comprises multivariate signals with interdependent components. These systems often suffer from noise contamination, which can significantly reduce the fidelity of signal interpretation. Traditional noise-reduction techniques, such as Wiener filtering or wavelet-based methods, frequently fail to perform optimally when applied to high-dimensional data due to challenges associated with high variance, noise correlation, and large computational demands [1-5]. In this article, we review state-of-the-art signal-processing algorithms for noise reduction specifically designed for high-dimensional systems. We focus on dimensionality reduction techniques like Principal Component Analysis (PCA), Independent Component Analysis (ICA), and kernel-based approaches, which have shown promise in improving signal fidelity while reducing noise [6, 7]. Furthermore, we introduce a novel hybrid algorithm that combines dimensionality reduction with adaptive filtering, hypothesizing that this integration will yield better results compared to conventional methods that rely solely on adaptive filtering. The objective is to demonstrate that by reducing dimensionality before applying adaptive filtering, the SNR (signal-to-noise ratio) can be significantly improved. Simulation results on high-dimensional datasets in remote sensing and biomedical signal applications confirm that the proposed hybrid approach outperforms baseline techniques, showing higher robustness and efficiency in both noise suppression and computational load [8-11]. This research highlights key contributions in the field, focusing on the advantages of incorporating dimensionality reduction as a preprocessing stage in high-dimensional signal processing [12-15]. Future work may involve fine-tuning these algorithms for real-time applications in various domains.

High-dimensional datasets can lead to overfitting and computationally expensive model building on machine learning. This study uses a dimensionality reduction technique, namely feature selection techniques, to overcome these problems. Five feature selection methods were used, i.e., Chi-Square (CS), Information Gain (IG), Genetic Algorithm (GA), Particle Swarm Optimization (PSO), and Least Absolute Shrinkage and Selection Operator (LASSO), and three classifier methods viz. Naïve Bayes, Extreme Gradient Boosting (XGB), and RF Classifier. The dataset used is the Heart Attack Analysis & Prediction Dataset. In this study, three scenarios of the best feature selection were carried out, namely: 1. selection of the best feature using a specific feature selection, 2. the intersection of selection of the best feature from the same category, 3. the intersection of selection of the best feature from the five proposed feature selection methods. The performance model is measured using accuracy, precision, recall, f1-score, AUC, and training time. This study reveals that feature selection is very effective in improving the performance of prediction models. Based on the experiment results, the best feature selection is CS and IG in the Filter Category with the XGB model. The best feature selected improved the performance of accuracy, precision, recall, f1-score, and AUC, i.e., 1.7%, 1%, 2.3%, 1.6%, and 0.2%, respectively. Meanwhile, training time requirements decreased by 23.5%. Feature selection with specific techniques performs better than feature selection by selecting the best features from the same category feature selection technique or various other feature selection methods.

Visual analysis of dimensionality reduction quality for parameterized projections

Purpose: Lung cancer remains one of the leading causes of death worldwide, especially among active smokers, yet early detection is still difficult because traditional imaging methods have limited sensitivity for identifying early-stage abnormalities. This study was conducted to address the need for a more accurate computational approach capable of detecting lung cancer at a molecular level using gene expression data. The goal is to build a model that can reliably distinguish cancerous from non-cancerous samples based on genomic features. Methods: This study uses the GSE4115 gene-expression dataset consisting of 187 bronchial epithelial samples and 22,215 gene features. The Moth-Flame Optimization (MFO) algorithm was implemented to select the most informative subset of genes from this high-dimensional dataset. A Support Vector Machine (SVM) classifier was then trained using multiple kernels, with hyperparameter tuning performed to identify the optimal configuration for each kernel. Results: Experimental results show that the Polynomial kernel achieved the highest performance using 286 MFO-selected features, reaching an accuracy of 0.84 and an F1-score of 0.85. These results confirm that combining MFO with SVM improves classification performance compared to using raw gene data without feature selection. Novelty: This study provides the first application of MFO-based feature selection for lung cancer prediction in smokers using the GSE4115 dataset. The findings demonstrate the value of nature-inspired optimization for handling high-dimensional genomic data and offer a promising direction for developing early computational detection methods.