A Survey Of Dimensionality Reduction And Classification Methods

Financial distress identification remains an essential topic in the scientific literature due to its importance for society and the economy. The advancements in information technology and the escalating volume of stored data have led to the emergence of financial distress that transcends the realm of financial statements and its’ indicators (ratios). The feature space could be expanded by incorporating new perspectives on feature data categories such as macroeconomics, sectors, social, board, management, judicial incident, etc. However, the increased dimensionality results in sparse data and overfitted models. This study proposes a new approach for efficient financial distress classification assessment by combining dimensionality reduction and machine learning techniques. The proposed framework aims to identify a subset of features leading to the minimization of the loss function describing the financial distress in an enterprise. During the study, 15 dimensionality reduction techniques with different numbers of features and 17 machine-learning models were compared. Overall, 1,432 experiments were performed using Lithuanian enterprise data covering the period from 2015 to 2022. Results revealed that the artificial neural network (ANN) model with 30 ranked features identified using the Random Forest mean decreasing Gini (RF_MDG) feature selection technique provided the highest AUC score. Moreover, this study has introduced a novel approach for feature extraction, which could improve financial distress classification models.

Classification of human hand movements based on EMG signals using nonlinear dimensionality reduction and data fusion techniques

Real world data is high-dimensional like images, speech signals containing multiple dimensions to represent data. Higher dimensional data are more complex for detecting and exploiting the relationships among terms. Dimensionality reduction is a technique used for reducing complexity for analysing high dimensional data. It reduces the dimensions from the original input data. Dimensionality reduction methods can be of two types as feature extractions and feature selection techniques. Feature Extraction is a distinct form of Dimensionality Reduction to extract some important feature from input dataset. Two different approaches available for dimensionality reduction are supervised approach and unsupervised approach. One exclusive purpose of this survey is to provide an adequate comprehension of the different dimensionality reduction techniques that exist currently and also to introduce the applicability of any one of the prescribed methods that depends upon the given set of parameters and varying conditions.

Feature selection (FS), which reduces data dimensionality and enriches the performance of any suggested framework, is one of the most crucial parts of machine learning (ML). In real-world applications, computational, large dimensionality and noisy or ambiguous nature, storage complexity, high performance, and other difficulties beset FS work. As a pre-processing phase, dimension reduction is used to improve the accuracy of learning features and reduce training time by removing irrelevant data, noise, and duplicated features. The feature selection (FS) and feature extraction (FE) methods were used to conduct dimensionality reduction (DR). Due to the increasing rate of data creation, FS is a practical method for addressing several serious dimensionality problems, like decreasing duplication, eradicating unneeded data, and enhancing comprehensibility in the outcomes. The efficiency of data processing and storage can be enriched by FE that handles the challenge of selecting the most distinct, instructive, and constrained collection of features. This research focuses on contemporary feature selection and feature extraction algorithms that aid in reducing data dimensionality.KeywordsFeature selectionMachine learningFeature extractionDimensionality reductionData processing

Dimensionality reduction as a preprocessing step to machine learning is effective in removing irrelevant and redundant data, increasing learning accuracy, and improving result comprehensibility. However, the recent increase of dimensionality of data poses a severe challenge to many existing feature selection and feature extraction methods with respect to efficiency and effectiveness. In the field of machine learning and pattern recognition, dimensionality reduction is important area, where many approaches have been proposed. In this paper, some widely used feature selection and feature extraction techniques have analyzed with the purpose of how effectively these techniques can be used to achieve high performance of learning algorithms that ultimately improves predictive accuracy of classifier. An endeavor to analyze dimensionality reduction techniques briefly with the purpose to investigate strengths and weaknesses of some widely used dimensionality reduction methods is presented.

High-dimensional features often cause computational complexity and dimensionality curse. Feature selection and feature extraction are the two mainstream methods for dimensionality reduction. Feature selection but not feature extraction can preserve the critical information and maintain the physical meaning simultaneously. Herein, we proposed a dimensionality reduction method based on rough set theory (DRM-RST) for feature selection. We defined the hyperspectral image as a decision system, extracted the features as decision attributes, and selected the effective features based on information entropy. We used the Washington D.C. Mall dataset and New York dataset to evaluate the performance of DRM-RST on dimensionality reduction. Compared with full band classification, 184 or 185 redundant bands were removed in DRM-RST, respectively. DRM-RST achieved similar accuracy (overall accuracy >94%) by SVM classifier and reduced computing time by about 85%. We further compared the dimensionality reduction efficiency of DRM-RST against other popular methods, including ReliefF, Sequential Backward Elimination (SBE) and Information Gain (IG). The Producer’s accuracy (PA) and User’s accuracy (UA) of DRM-RST was greater than that of ReliefF and IG. DRM-RST showed greater stability of accuracy than SBE in dimensionality reduction when using for different datasets. Collectively, this study provides a new method for dimensionality reduction that can reduce computational complexity and alleviate dimensionality curse.

Distance similarity matrix using ensemble of dimensional data reduction techniques: Vibration and aerocoustic case studies

A Review of Dimensionality Reduction Techniques for Efficient Computation

Software defect prediction (SDP) is designed to assist software testing, which can reasonably allocate test resources to reduce costs and improve development efficiency. In order to improve the prediction performance, researchers have designed many defect-related features for SDP. However, feature redundancy (FR) and irrelevance caused by the increasing dimensions of data will greatly degrade the performance of defect prediction. In order to solve the problems, researchers have proposed various data dimensionality reduction methods. These methods can be simply divided into two categories of methods: feature selection and feature extraction. However, the two categories of methods have their own advantages and limitation. In this paper, we propose a Hybrid Feature Dimensionality Reduction Approach (HFDRA) for SDP, which combines the two different kinds of methods, to improve the performance of SDP. HFDRA approach can be divided into two stages: feature selection and feature extraction. First, HFDRA divides the original features into several feature subsets through a clustering algorithm in the feature selection stage. Then, in the feature extraction stage, kernel principal component analysis (KPCA) is used to reduce the dimensionality of each feature subset. Finally, the reduced-dimensional data is used to build the prediction model. In the empirical study, we use 22 projects from AEEEM, SOFTLAB, MORP, and ReLink as experiment object. In this paper, we first compare our approach with seven baseline methods and three state-of-the-art methods. Then, we analyze the relationship between FR and prediction performance. Experiment results show that our approach outperforms the state-of-the-art data dimensionality reduction methods for defect prediction.

Credit risk analysis forms an important criterion for the endurance of the banking sector. Understanding credit liability of customer enhances the security stability of the banker. This research is aimed at comparing the predictive accuracy of data mining techniques with feature selection and extraction techniques in the default of credit card clients and suggesting the most efficient technique for analyzing credit card default. Customer's default payments in Taiwan were used as the dataset. Machine Learning classification algorithms like K-nearest neighbor(KNN), Naive Bayesian(NB), Decision Tree(DT) and Support Vector Machines(SVM) were applied to the dataset. Feature filter methods like information gain (IG), gain ratio (GR) and chi-square (CS) and Wrapper based methods like Best First search(BFS) and Hill Climbing search (HCS) were used to identify minimum features required. Feature extraction methods such as Principal Component Analysis(PCA) and Linear Discriminant Anal-ysis(LDA) were also applied. The minimum features of feature selection algorithms along with PCA and LDA were then applied to Base classifiers and their accuracies were tabulated and compared. SVM with feature selection and extraction techniques had the best predictive accuracy.

The curse of dimensionality, i.e., the fact that feature spaces of increasing dimensionality with finite sample sizes tend to be empty, has given incentive to a plethora of research activities in various disciplines and diverse application fields, e.g., statistics or neural networks. Three major application fields are multivariate data classification, data analysis and data visualization. In this contribution, methods for dimensionality reduction from three decades of interdisciplinary research are browsed and their applicability in the above application domains is briefly discussed. Complementing techniques for ensuing interactive data visualization, data navigation and visual exploratory data analysis are presented, which exploit the remarkable human perceptive and associative capabilities for interactive visual exploratory data analysis and systematic recognition system design. The main focus of this paper is on the comparison of feature selection and feature extraction techniques and the potential benefit of their combination. Further, the interesting implications of dimensionality reduction for VLSI design and related area and power consumption are pointed out.

Aim) Pathological brain detection (PBD) systems aim to assist and even replace neuroradiologists to make decisions for patients. This review offers a comprehensive and quantitative comparison for PBD systems by artificial intelligence in magnetic resonance imaging (MRI) scanning. (Method) We first investigated four categories of brain diseases, including neoplastic disease, neurodegenerative disease, cerebrovascular disease, and inflammation. Next, we introduced important MRI techniques, such as the shimming, water and fat suppression, and three advanced imaging modalities (functional MRI, diffusion tensor imaging, and magnetic resonance spectroscopic imaging). Then, we discussed four image preprocessing techniques (image denoising, slice selection, brain extraction, spatial normalization, and intensity normalization), seven feature representation techniques (shape, moment, wavelet, statistics, entropy, gray level co-occurrence matrix, and Fourier transform), and two dimension reduction techniques (feature selection and feature extraction). Afterwards, we studied classification related methods: six learning models (decision tree, extreme learning machine, k-nearest neighbors, naive Bayes classifier, support vector machine, feed-forward neural network), five kernel functions (linear, homogeneous and inhomogeneous polynomial, radial basis function, and sigmoid), and three types of optimization methods (evolutionary algorithm, stochastic optimization, and swarm intelligence). (Results) We introduced three benchmark datasets and used K-fold stratified cross validation to avoid overfitting. We presented a detailed quantitative comparison among 44 state-of-theart PBD algorithms and discussed their advantages and limitations. (Discussions) Artificial intelligence is now making stride in the PBD field and enjoys a fair amount of success. In the future, semi-supervised learning and transfer learning techniques may be potential breakthroughs to develop PBD systems.

A modified genetic algorithm and weighted principal component analysis based feature selection and extraction strategy in agriculture

Parkinson's disease (PD) presents a growing global health challenge, with early detection being crucial for effective management and treatment. This study seeks to develop an innovative machine learning (ML) framework for the early detection of PD by integrating advanced techniques for data preprocessing, dimensionality reduction, feature selection, and ensemble classification, aiming to significantly improve detection accuracy and timeliness. The research employs a robust ML pipeline, beginning with data preprocessing using mean imputation, standardization, min-max scaling, and SMOTE (Synthetic Minority Over-sampling Technique) to handle imbalanced data. Dimensionality reduction is achieved through Principal Component Analysis (PCA), while feature selection is performed using SelectKBest coupled with the ANOVA F-test to identify the most relevant features. Four ensemble methods—Random Forest, Gradient Boosting, XGBoost, and Support Vector Machine (SVM)—are evaluated for classification. Among the classifiers tested, the Gradient Boosting model stands out with an impressive accuracy of 0.9487, demonstrating its superior performance in PD detection. Integrating multiple preprocessing, dimensionality reduction, and feature selection techniques proves essential in optimizing model performance, highlighting the importance of a multifaceted approach in handling complex datasets. This research introduces a comprehensive ML framework that combines multiple advanced techniques in a streamlined process, significantly improving the early detection of Parkinson's disease. Ensemble methods, combined with strategic feature selection and data balancing techniques, offer a novel approach that could be applied to other neurodegenerative disorders, expanding its potential impact beyond PD detection.

Breast cancer remains a critical global health concern, requiring advanced and reliable diagnostic methods for early detection and effective intervention. This work introduces an integrated ensemble framework that combines multiple dimensionality reduction (DR) techniques, including Principal Component Analysis (PCA), Non-negative Matrix Factorization (NMF), and Singular Value Decomposition (SVD), with robust machine learning (ML) classifiers for improved breast cancer detection. The publicly available Wisconsin Breast Cancer Dataset (WBCD) was utilized, with rigorous data preprocessing performed to address missing values, anomalies, and class imbalance through stratified sampling and median imputation. To mitigate overfitting and underfitting, dimensionality reduction was coupled with cross-validation and ensemble strategies. The predictive performance of Logistic Regression (LR), Support Vector Machine (SVM), Random Forest (RF), Decision Tree (DT), and Multi-Layer Perceptron (MLP) was systematically evaluated. Experimental results show that SVM consistently achieves a maximum accuracy of 97.9 % across all applied DR techniques, while MLP and LR also reach 97.9 % accuracy with PCA and NMF, though MLP exhibits performance variability depending on the selected DR method. The findings provide practical guidance for healthcare practitioners and researchers, supporting the adoption of explainable and scalable AI-driven diagnostic tools. Limitations include the reliance on a single dataset and the need for further validation on larger and more diverse clinical cohorts. Future work will focus on enhancing model interpretability, external validation, and real-world deployment in resource-constrained settings.