Feature selection using social network techniques

Feature selection is a dimensionality reduction method known as a main step in data mining and machine learning. The aim of feature selection is to remove redundant and unrelated features. In recent years several feature selection methods based on graph theory and social networking techniques have been proposed .In this study, a feature selection approach based on multi objective PSO algorithm and social network techniques is presented. In the proposed method, Fisher score, node centrality and edge centrality are used to construct the fitness function in order to present a multi objective particle swarm optimization (PSO) approach. The proposed method run over a variety of datasets and the results are compared with the well-known filter-based feature selection methods. The results show that the proposed method is effective and the performance of the proposed method better than other methods in some cases.

Heart disease is still one of the top causes of death globally, demanding early and effective prediction tools. Traditional predictive models frequently have limitations because they rely on single‐source statistics, which may not represent the entire scenario of a patient's complex health profile. Feature selection is an essential preprocessing step in machine learning that improves model efficiency by eliminating irrelevant or redundant features. This study evaluates the impact of feature selection on six classification modelsNaïve Bayes, Logistic Regression, Decision Tree, K‐Nearest Neighbors (KNN), Support Vector Machine (SVM), and Random Forestacross four datasets. Performance was assessed using accuracy, precision, recall, F1‐score, and area under the curve (AUC) before and after feature selection. The results indicate that feature selection had varying effects across different models and datasets. In Dataset‐01, Random Forest maintained its strong performance (98.70% accuracy) even after feature selection, while SVM and KNN saw noticeable drops, with accuracy decreasing from 88.96% to 79.87% and 87.01% to 81.17% respectively. For Dataset‐02, SVM showed improvement, increasing from 80.00% to 85.00%, while Naïve Bayes and KNN also improved, reaching 82.50% and 82.50% accuracy, respectively. In Dataset‐03, Logistic Regression and KNN experienced declines, with accuracy dropping from 88.76% to 83.71% and 91.57% to 87.08%, respectively, while Random Forest fell from 97.75% to 91.57%. Dataset‐04 showed mixed results, with Logistic Regression improving from 86.67% to 91.11%, but KNN and Random Forest dropping from 97.78% to 84.44% and 91.11% to 82.22%, respectively. These findings suggest that while feature selection enhances performance for some models, it can negatively impact others, particularly those reliant on a larger feature set. Future research should explore adaptive feature selection techniques to optimize model performance across diverse datasets.

Feature selection is the preliminary step in machine learning and data mining. It identifies the most important and relevant features within a dataset by eliminating the redundant or irrelevant features. The substantial benefits may include an improved performance in terms of high prediction accuracy, reduced computational complexity, and simply interpretable underlying models. In this paper, we present a novel framework to investigate and understand the importance of Monte Carlo tree search (MCTS) in feature selection for very high-dimensional datasets. We construct a binary feature selection tree where each node represents one of the two feature states: a feature is selected or not. The search starts with an empty root node reflecting that no feature is selected. Then, the search tree is expanded by adding nodes in an incremental fashion through MCTS-based simulations. Following tree and default policy, every iteration generates an initial feature subset, where a filter is used to select the top k features forming the candidate feature subset. The classification accuracy is used as the goodness or reward of the candidate feature subset and propagated backward up to the root node following the active path. Finally, the candidate subset with highest reward is selected as the best feature subset. Experiments are performed on 30 real-world datasets, including 14 very high-dimensional microarray datasets, and results are also compared with state-of-the-art methods in the literature, which proves the efficacy, validity, and significance of the proposed method.

Feature selection plays a central role in data analysis and is also a crucial step in machine learning, data mining and pattern recognition. Feature selection algorithm focuses mainly on the design of a criterion function and the selection of a search strategy. In this paper, a novel feature selection approach (NFSA) based on quantum genetic algorithm (QGA) and a good evaluation criterion is proposed to select the optimal feature subset from a large number of features extracted from radar emitter signals (RESs). The criterion function is given firstly. Then, detailed algorithm of QGA is described and its performances are analyzed. Finally, the best feature subset is selected from the original feature set (OFS) composed of 16 features of RESs. Experimental results show that the proposed approach reduces greatly the dimensions of OFS and heightens accurate recognition rate of RESs, which indicates that NFSA is feasible and effective.

High-feature dimensions is one of the obvious challenges facing large-scale hierarchical classification problem. One promising way to overcome such problem is reducing the features size by removing irrelevant, noise and redundant features. Filter-based feature selection methods are commonly used to construct smaller subset of most relevant features in either adaptive or global approach at each non leaf node within the hierarchy. Both the Feature selection approaches do not only reduce time and space complexity but also improve classification accuracy of the learning models. In this study, the effectiveness of the main approaches for constructing relevant features subset, that is adaptive and global approaches are analyzed using the two filter-based methods suitable for text classification. Experiment evaluation conducted on 20NG, IPC, and DMOZ-small datasets shows that implementing feature selection adaptively has more capability of selecting most relevant features, thus, achieving the best classification accuracy. Moreover, further evaluation shown integrating feature selection with LSHC records improve performance when compared to without feature selection.

With the continuous development of Internet technology, data gradually present a complicated and high-dimensional trend. These high-dimensional data have a large number of redundant features and irrelevant features, which bring great challenges to the existing machine learning algorithms. Feature selection is one of the important research topics in the fields of machine learning, pattern recognition and data mining, and it is also an important means in the data preprocessing stage. Feature selection is to look for the optimal feature subset from the original feature set, which would improve the classification accuracy and reduce the machine learning time. The traditional feature selection algorithm tends to ignore the kind of feature which has a weak distinguishing capacity as a monomer, whereas the feature group’s distinguishing capacity is strong. Therefore, a new dynamic interaction feature selection (DIFS) algorithm is proposed in this paper. Initially, under the theoretical framework of interactive information, it redefines the relevance, irrelevance and redundancy of the features. Secondly, it offers the computational formulas for calculating interactive information. Finally, under the eleven data sets of UCI and three different classifiers, namely, KNN, SVM and C4.5, the DIFS algorithm increases the classification accuracy of the FullSet by 3.2848% and averagely decreases the number of features selected by 15.137. Hence, the DIFS algorithm can not only identify the relevance feature effectively, but also identify the irrelevant and redundant features. Moreover, it can effectively improve the classification accuracy of the data sets and reduce the feature dimensions of the data sets.

Feature extraction is viewed as an important preprocessing step for pattern recognition, machine learning and data mining. Neighborhood rough set (NRS) based feature extracting algorithm is able to delete most of the redundant and irrelevant features, which avoid the step of data discretization and hence decreased the information lost in preprocess. In this paper, we firstly introduce the basic definitions and operations of NRS, and propose a palmprint recognition method based on NRS. The neighborhood model is used to reduce the attributes and extract the recognition features. Experimental results on PolyU palmprint database demonstrate that the proposed method is effective and feasible for palmprint recognition.

Neighborhood effective information ratio for hybrid feature subset evaluation and selection

The efficiency and the effectiveness of a machine learning (ML) model are greatly influenced by feature selection (FS), a crucial preprocessing step in machine learning that seeks out the ideal set of characteristics with the maximum accuracy possible. Due to their dominance over traditional optimization techniques, researchers are concentrating on a variety of metaheuristic (or evolutionary) algorithms and trying to suggest cutting-edge hybrid techniques to handle FS issues. The use of hybrid metaheuristic approaches for FS has thus been the subject of numerous research works. The purpose of this paper is to critically assess the existing hybrid FS approaches and to give a thorough literature review on the hybridization of different metaheuristic/evolutionary strategies that have been employed for supporting FS. This article reviews pertinent documents on hybrid frameworks that were published in the period from 2009 to 2022 and offers a thorough analysis of the used techniques, classifiers, datasets, applications, assessment metrics, and schemes of hybridization. Additionally, new open research issues and challenges are identified to pinpoint the areas that have to be further explored for additional study.

Feature Selection, one of the most important preprocessing steps in Machine Learning, is the process where we automatically or manually select those features which contribute most to our prediction variable or output in which we are interested. This subset of features has some very important benefits: it reduces the computational complexity of learning algorithms, saves time, improves accuracy, and the selected features can be insightful for the people involved in the problem domain. Among the different ways of performing feature selection such as filter, wrapper and hybrid, filter-based separability methods can be used as a feature ranking tool in binary classification problems, most popular being the Bhattacharya distance and the Jeffries–Matusita (JM) distance. However, these measures are parametric and their computation requires knowledge of the distribution from which the samples are drawn. In real life, we often come across instances where it is difficult to have an idea about the distribution of observations. In this paper, we have presented a new non-parametric approach for performing feature selection called the ‘Non-Parametric Distance Measure’. The experiment with the new measure is performed over nine datasets and the results are compared with other ranking-based methods for feature selection using those datasets. This experiment proves that the new box-plot-based method can provide greater accuracy and efficiency than the conventional ranking-based measures for feature selection such as the Chi-Square, Symmetric Uncertainty and Information Gain.

Feature selection is a fundamental pre‐processing step in machine learning that aims to reduce the dimensionality of a dataset by selecting the most effective features from the original features. This process is regarded as a combinatorial optimization problem, and the grey wolf optimizer (GWO), a novel meta‐heuristic algorithm, has gained popularity in feature selection due to its fast convergence speed and easy implementation. In this paper, an improved binary GWO algorithm incorporating a novel Population Adaptation strategy called PA‐BGWO is proposed. The PA‐BGWO takes into account the characteristics of the feature selection problem and designs three strategies. The proposed strategy includes an adaptive individual update procedure to enhance the exploitation ability and accelerate convergence speed, a head wolf fine‐tuned mechanism to exert the impact on each independent feature of the objective function, and a filter‐based method ReliefF for calculating feature weights with dynamically adjusted mutation probabilities based on the ranking features to effectively escape from local optima. Experimental comparisons with several state‐of‐the‐art feature selection methods on 15 classification problems demonstrate that the proposed approach can select a small feature subset with higher classification accuracy in most cases.

Background: Feature selection (FS), a crucial preprocessing step in machine learning, greatly reduces the dimension of data and improves model performance. This paper focuses on selecting features for medical data classification. Methods: In this work, a new form of ensemble FS method called weighted rank difference ensemble (WRD-Ensemble) has been put forth. It combines three FS methods to produce a stable and diverse subset of features. The three base FS approaches are Pearson’s correlation coefficient (PCC), reliefF, and gain ratio (GR). These three FS approaches produce three distinct lists of features, and then they order each feature by importance or weight. The final subset of features in this study is chosen using the average weight of each feature and the rank difference of a feature across three ranked lists. Using the average weight and rank difference of each feature, unstable and less significant features are eliminated from the feature space. The WRD-Ensemble method is applied to three medical datasets: chronic kidney disease (CKD), lung cancer, and heart disease. These data samples are classified using logistic regression (LR). Results: The experimental results show that compared to the base FS methods and other ensemble FS methods, the proposed WRD-Ensemble method leads to obtaining the highest accuracy value of 98.97% for CKD, 93.24% for lung cancer, and 83.84% for heart disease. Conclusion: The results indicate that the proposed WRD-Ensemble method can potentially improve the accuracy of disease diagnosis models, contributing to advances in clinical decision-making.

MFeature: Towards high performance evolutionary tools for feature selection

IBJA: An improved binary DJaya algorithm for feature selection

Granular maximum decision entropy-based monotonic uncertainty measure for attribute reduction