Visualizing Pseudo-Boolean Functions: Feature Selection and Regularization for Machine Learning

To develop a hypothesis-free model that best predicts response to MTX drug in RA patients utilizing biologically meaningful genetic feature selection of potentially functional single nucleotide polymorphisms (pfSNPs) through robust machine learning (ML) feature selection methods. MTX-treated RA patients with known response were divided in a 4:1 ratio into training and test sets. From the patients' exomes, potential features for classifier prediction were identified from pfSNPs and non-genetic factors through ML using recursive feature elimination with cross-validation incorporating the random forest classifier. Feature selection was repeated on random subsets of the training cohort, and consensus features were assembled into the final feature set. This feature set was evaluated for predictive potential using six ML classifiers, first by cross-validation within the training set, and finally by analysing its performance with the unseen test set. The final feature set contains 56 pfSNPs and five non-genetic factors. The majority of these pfSNPs are located in pathways related to RA pathogenesis or MTX action and are predicted to modulate gene expression. When used for training in six ML classifiers, performance was good in both the training set (area under the curve: 0.855-0.916; sensitivity: 0.715-0.892; and specificity: 0.733-0.862) and the unseen test set (area under the curve: 0.751-0.826; sensitivity: 0.581-0.839; and specificity: 0.641-0.923). Sensitive and specific predictors of MTX response in RA patients were identified in this study through a novel strategy combining biologically meaningful and machine learning feature selection and training. These predictors may facilitate better treatment decision-making in RA management.

Feature selection in machine learning: A new perspective

We propose a unifying FCA-based framework for some questions in data analysis and data mining, combining ideas from Rough Set Theory, JSM-reasoning, and feature selection in machine learning. Unlike the standard rough set model the indiscernibility relation in our paper is based on a quasi-order, not necessarily an equivalence relation. Feature selection, though algorithmically difficult in general, appears to be easier in many cases of scaled many-valued contexts, because the difficulties can at least partially be projected to the scale contexts. We propose a heuristic algorithm for this.

Human Action Recognition (HAR) is a vital part of the healthcare sector. Medical practitioners are experiencing difficulty in recognizing human physiological conditions due to the enormous quantity of sensory stimulation. Machine learning techniques help to predict human physiological conditions, providing medical practitioners to work more efficiently. Feature selection is an essential part of discovering new knowledge in the majority of real-world problems when there are a lot of features. Feature selection is extremely useful because it speeds up decisions and enhances classification performance. The importance of feature selection in machine learning is dimension reduction in a massive multivariate data collection. This paper presents an effective feature selection method known as the Enhanced Recursive Feature Elimination (EFRE) for selecting key features from the data set for HAR prediction. The experimental results reveal that the ERFE technique selects the most suitable features for HAR prediction. The performance of the proposed ERFE approach is tested using different performance evaluation metrics. The performance analysis shows that the ERFE method outperforms existing feature selection methods with 88% accuracy.

Privacy preserving in machine learning is a crucial issue in industry informatics since data used for training in industries usually contain sensitive information. Existing differentially private machine learning algorithms have not considered the impact of data correlation, which may lead to more privacy leakage than expected in industrial applications. For example, data collected for traffic monitoring may contain some correlated records due to temporal correlation or user correlation. To fill this gap, in this article, we propose a correlation reduction scheme with differentially private feature selection considering the issue of privacy loss when data have correlation in machine learning tasks. The proposed scheme involves five steps with the goal of managing the extent of data correlation, preserving the privacy, and supporting accuracy in the prediction results. In this way, the impact of data correlation is relieved with the proposed feature selection scheme, and moreover the privacy issue of data correlation in learning is guaranteed. The proposed method can be widely used in machine learning algorithms, which provide services in industrial areas. Experiments show that the proposed scheme can produce better prediction results with machine learning tasks and fewer mean square errors for data queries compared to existing schemes.

Nondestructive Testing (NDT) assessment of materials’ health condition is useful for classifying healthy from unhealthy structures or detecting flaws in metallic or dielectric structures. Performing structural health testing for coated/uncoated metallic or dielectric materials with the same testing equipment requires a testing method that can work on metallics and dielectrics such as microwave testing. Reducing complexity and expenses associated with current diagnostic practices of microwave NDT of structural health requires an effective and intelligent approach based on feature selection and classification techniques of machine learning. Current microwave NDT methods in general based on measuring variation in the S-matrix over the entire operating frequency ranges of the sensors. For instance, assessing the health of metallic structures using a microwave sensor depends on the reflection or/and transmission coefficient measurements as a function of the sweeping frequencies of the operating band. The aim of this work is reducing sweeping frequencies using machine learning feature selection techniques. By treating sweeping frequencies as features, the number of top important features can be identified, then only the most influential features (frequencies) are considered when building the microwave NDT equipment. The proposed method of reducing sweeping frequencies was validated experimentally using a waveguide sensor and a metallic plate with different cracks. Among the investigated feature selection techniques are information gain, gain ratio, relief, chi-squared. The effectiveness of the selected features were validated through performance evaluations of various classification models; namely, Nearest Neighbor, Neural Networks, Random Forest, and Support Vector Machine. Results showed good crack classification accuracy rates after employing feature selection algorithms.

The chapter gives a broad overview to feature selection with an importance on different selection techniques and algorithms. It utilizes datasets with diverse volume of features and instances to evaluate the ability of various feature selection algorithms. The datasets were selected based on well-defined conditions that act as standards for diverse settings. The performance outcomes of the algorithms on the datasets are tabulated and the insights gained are discussed in this chapter. Appendices 1 and 2 comprise the exact scores that each algorithm achieved for each dataset. The chapter improves the understanding of feature selection and extends the usefulness for selecting the appropriate algorithms for specific use cases.

In machine learning, Case Based Reasoning is a prominent technique for harvesting knowledge from past experiences. The past experiences are represented in the form of a repository of cases having a set of features. But each feature may not have the equal relevancy in describing a case. Measuring the relevancy of each feature is always a prime issue. A subset of relevant features describes a case with adequate accuracy. An appropriate subset of relevant features should be selected for improving the performance of the system and to reduce dimensionality. In case based domain, feature selection is a process of selecting an appropriate subset of relevant features. There are various real domains which are inherently case based and features are expressed in terms of linguistic variables. To assign a numerical weight to each linguistic feature, a lot of feature subset selection algorithms have been proposed. But the weighting values are usually determined using subjective judgement or a trial and error basis.

Generally, there are two types of approaches available for the detection of networks attacks, namely signature based and anomaly based. In this work, we will analyze the performance of anomaly-based detection model in SDN with the help of some common machine learning algorithms and feature selection mechanisms. We construct a mechanism of machine learning model for an intrusion detection system and train the model with the NSL-KDD Data set using feature selection technique. In order to enhance the performance of the classifier, some feature selection methods have been applied as a prepossessing of the data set. We have used five feature selection methods, namely Info Gain, Gain Ratio, CFS Subset Evaluator, Symmetric Uncertainty, and Chi-square test. A full data set of 41 features and a reduced data set after applying feature selection method has been experimented. A data set with feature selection ensures the highest accuracy with Random Forest classifier using Gain Ratio feature selection Evaluator.

Sentiment analysis of scientific citations has received much attention in recent years because of the increased availability of scientific publications. Scholarly databases are valuable sources for publications and citation information where researchers can publish their ideas and results. Sentiment analysis of scientific citations aims to analyze the authors’ sentiments within scientific citations. During the last decade, some review papers have been published in the field of sentiment analysis. Despite the growth in the size of scholarly databases and researchers’ interests, no one as far as we know has carried out an in-depth survey in a specific area of sentiment analysis in scientific citations. This paper presents a comprehensive survey of sentiment analysis of scientific citations. In this review, the process of scientific citation sentiment analysis is introduced and recently proposed methods with the main challenges are presented, analyzed and discussed. Further, we present related fields such as citation function classification and citation recommendation that have recently gained enormous attention. Our contributions include identifying the most important challenges as well as the analysis and classification of recent methods used in scientific citation sentiment analysis. Moreover, it presents the normal process, and this includes citation context extraction, public data sources, and feature selection. We found that most of the papers use classical machine learning methods. However, due to limitations of performance and manual feature selection in machine learning, we believe that in the future hybrid and deep learning methods can possibly handle the problems of scientific citation sentiment analysis more efficiently and reliably.

Alzheimer’s disease is a progressive neurodegenerative disorder for which early detection remains a significant challenge due to the complexity of clinical features and the high dimensionality of medical data. This study aims to improve the accuracy and reliability of Alzheimer’s disease detection by evaluating the performance of multiple machine learning algorithms integrated with intelligent feature selection strategies. Five classification models, Decision Tree, Naïve Bayes, Random Forest, Logistic Regression, and Deep Learning, were investigated under two experimental scenarios: without feature selection and with feature selection using Recursive Feature Elimination, Binary Particle Swarm Optimization, and Variance Threshold. Model performance was evaluated using K-fold cross-validation based on accuracy, precision, recall, and F1-score metrics. The results demonstrate that feature selection consistently enhances classification performance, particularly for conventional machine learning models such as Random Forest and Logistic Regression. Although the Deep Learning model achieves competitive accuracy, its reduced precision and F1-score indicate limitations when applied to reduced feature spaces. These findings highlight the importance of incorporating appropriate feature selection techniques to address data complexity and improve the effectiveness of early Alzheimer’s disease detection.

Type 1 Diabetes Mellitus (T1D) has been increasingly associated with alterations in the gut microbiome. However, the impact of taxonomic resolution, feature selection strategies, and machine learning methods on microbiome-based prediction remains incompletely understood. We analyzed publicly available 16S rRNA gene sequencing datasets from two geographic cohorts to evaluate microbiome-based prediction of T1D. Microbial features were constructed at multiple taxonomic levels and as full hierarchical taxonomic paths preserving phylogenetic structure. Machine learning models were trained using stratified cross-validation and cross-cohort validation frameworks. Feature selection was performed using Binary Particle Swarm Optimization (BPSO) to identify compact and predictive microbial signatures. Model performance was evaluated using AUC, Accuracy, F1 score, and Matthews Correlation Coefficient. Differential abundance analysis using the LinDA framework was used to support biological interpretation of selected taxa. Tree-based models, particularly Random Forest and XGBoost, achieved the strongest predictive performance across taxonomic representations. Taxonomic resolution influenced model behavior, with family-level features providing strong performance with compact feature sets, while higher-resolution representations did not consistently improve performance despite increased complexity. BPSO identified consistently selected taxa across validation frameworks, suggesting stable predictive signatures. Several of these taxa have been linked to inflammatory or metabolically altered gut environments. Cross-cohort validation showed reduced performance compared with within-study models, highlighting challenges in generalization. Machine learning combined with BPSO-based feature selection provides an effective framework for identifying predictive microbial signatures associated with T1D. Our findings highlight the importance of taxonomic resolution, feature stability, and cross-cohort validation in microbiome-based predictive modeling. Integrating evolutionary feature selection with machine learning and biological validation may improve the robustness and interpretability of candidate microbial signatures.

Although many studies have investigated the usage of machine learning (ML) algorithms for cross-project software defect prediction, limited works addressed cross-project defect number prediction (CPDNP). This study investigated the robustness of feature selection in five ML algorithms, such as Decision Trees, Random Forests, Gradient Boosted Trees, Generalized Linear Model, and Deep Neural Networks, for CPDNP. The results showed that Gradient Boosted Tree generated models with the lowest errors in most projects. However, models generated by Random Forests and Deep Neural Networks with feature selection were considered the most robust ones. Meanwhile, models generated by Decision Trees were the least robust. The feature selection was sufficient to generate robust ML models for CPDNP.

Synergic Integration of the miRNome, Machine Learning and Bioinformatics for the Identification of Potential Disease-Modifying Agents in Obstructive Sleep Apnea.

Dimensionality reduction as a preprocessing step to machine learning is effective in removing irrelevant and redundant data, increasing learning accuracy, and improving result comprehensibility. However, the recent increase of dimensionality of data poses a severe challenge to many existing feature selection and feature extraction methods with respect to efficiency and effectiveness. In the field of machine learning and pattern recognition, dimensionality reduction is important area, where many approaches have been proposed. In this paper, some widely used feature selection and feature extraction techniques have analyzed with the purpose of how effectively these techniques can be used to achieve high performance of learning algorithms that ultimately improves predictive accuracy of classifier. An endeavor to analyze dimensionality reduction techniques briefly with the purpose to investigate strengths and weaknesses of some widely used dimensionality reduction methods is presented.