Analysis of an Educational dataset using Classification Algorithm in Conjunction with Wrapper Feature Selection Methods

Finding defects early in a software system is a crucial task, as it creates adequate time for fixing such defects using available resources. Strategies such as symmetric testing have proven useful; however, its inability in differentiating incorrect implementations from correct ones is a drawback. Software defect prediction (SDP) is another feasible method that can be used for detecting defects early. Additionally, high dimensionality, a data quality problem, has a detrimental effect on the predictive capability of SDP models. Feature selection (FS) has been used as a feasible solution for solving the high dimensionality issue in SDP. According to current literature, the two basic forms of FS approaches are filter-based feature selection (FFS) and wrapper-based feature selection (WFS). Between the two, WFS approaches have been deemed to be superior. However, WFS methods have a high computational cost due to the unknown number of executions available for feature subset search, evaluation, and selection. This characteristic of WFS often leads to overfitting of classifier models due to its easy trapping in local maxima. The trapping of the WFS subset evaluator in local maxima can be overcome by using an effective search method in the evaluator process. Hence, this study proposes an enhanced WFS method that dynamically and iteratively selects features. The proposed enhanced WFS (EWFS) method is based on incrementally selecting features while considering previously selected features in its search space. The novelty of EWFS is based on the enhancement of the subset evaluation process of WFS methods by deploying a dynamic re-ranking strategy that iteratively selects germane features with a low subset evaluation cycle while not compromising the prediction performance of the ensuing model. For evaluation, EWFS was deployed with Decision Tree (DT) and Naïve Bayes classifiers on software defect datasets with varying granularities. The experimental findings revealed that EWFS outperformed existing metaheuristics and sequential search-based WFS approaches established in this work. Additionally, EWFS selected fewer features with less computational time as compared with existing metaheuristics and sequential search-based WFS methods.

<p>Soil properties could be assessed with reflectance spectroscopy (soil spectroscopy, SS) in vis-NIR region (400-2500 nm) through absorption features found in soil spectra. A high spectral resolution (up to 1 nm) drives to high dimensional and multicollinear data. This issue is usually addressed prior to modelling with feature extraction methods such as Principal Component Analysis, or embedded methods such as Partial Least Squares Regression (PLSR). Feature Selection (FS) wrapper methods are promising dimensionality reduction approaches barely used in SS. The objective of this study was two-fold: i) evaluate the performance of FS wrapper methods built from Random Forest (RF) algorithm to predict soil organic matter (SOM), clay and carbonates using laboratory spectroscopy, ii) test the performance of FS methods for dimensionality reduction in SS. The reflectance of 100 soil samples from Sierra de las Nieves National Park (Spain), was measured under laboratory conditions using an ASD FieldSpec Pro JR. A spectral preprocessing method, Continumm Removal (CR), was applied to raw spectra. The RF wrapper considered two different feature searching approaches: Sequential Forward Selection (SFS) and Sequential Flotant Forward Selection (SFFS). The performance of RF with FS (RF-FS) was compared to that of Partial Least Squares Regression (PLSR) and RF (without FS). Models were evaluated with R-squared, root mean squared error (RMSE) and ratio of prediction to deviation (RPD).</p><p>RF-FS models outperformed PLSR and RF models for the three SAP. RF-FS best models had a RPD of 2.19 for SOM, 1.64 for carbonates and 1.52 for clay, whereas PLSR models had RPD values of 1.59, 1.22 and 1.3, and RF 1.38, 1.23 and 1.23 for SOM, carbonates, and clay, respectively. Therefore, FS was useful in obtaining models with improved accuracy by reducing redundant features and avoiding multicollinearity (Hughes effect). The application of FS wrapper methods reduced the number of features in the RF-FS models to less than 1% of the starting features. Features were selected across all spectra from SOM and clay, and around 900, 1900 and 2350 nm for carbonates. This research, thus, shows an alternative to different feature extraction approaches for modelling soil properties based on FS methods and machine learning.</p>

Breast cancer is the most life-threatening disease among women worldwide. Nowadays, clinical and genomic data in breast cancer datasets are analyzed to identify the incidence and impact of the disease. This paper proposes an analysis on the clinical breast cancer datasets, METABRIC and SEER for the prediction of survivability of breast cancer patients through ensemble classifiers. It attempts to identify informative independent features for those classifiers using filter methods, Mutual Information and Chi-Square and wrapper methods, Sequential Forward Selection, Sequential Backward Selection and Recursive Feature Elimination in the feature selection sequence. An analysis on the performance of ensemble techniques namely, bagging, boosting and stacking shows that the ensemble techniques produce higher accuracy when compared to the accuracies produced by the base classifiers. Also, the stacking ensemble model produces higher accuracies of 99.2% and 99.8% than other ensemble techniques on METABRIC and SEER datasets respectively. The SBS wrapper method aids higher accuracy to all classifiers than SFS and RFE.

Development and validation of an interpretable machine learning model for cerebral small vessel disease risk assessment.

Objective. The identification of functional regions, in particular the subthalamic nucleus, through microelectrode recording (MER) is the key step in deep brain stimulation (DBS). To eliminate variability in a neurosurgeon’s judgment, this study presents an online identification method for identifying functional regions along the electrode trajectory. Approach. Functional regions can be identified through offline clustering and online identification based on the unsupervised random forest (RF) algorithm. We took 106 features from MER and the estimated anatomical distance to target as the dataset to train the RF model. To improve the prediction performance and reduce the computation time, a wrapper feature selection (FS) method was added into the algorithm. The method contains feature ranking based on out-of-bag error or silhouette index and feature subset search based on the roulette selection algorithm. Main results. To evaluate the optimization effect of the FS method on the unsupervised RF algorithm, we compared the results of the algorithm with or without FS on the DBS dataset. In addition, the optimization effect of FS on the computation time is evaluated. The results show that for offline clustering, the accuracy obtained with the selected features is higher than that obtained with all features, and the running time decreased from 259.7 s to 60.8 s in the iteration of the FS. The accuracy in online identification improved from 76.19% to 92.08% through FS. In addition, the functional region online identification time is 41.5 ms, which can meet the requirements of DBS surgery. Significance. In conclusion, using the FS method can improve the accuracy and reduce the computation time of the online identification of functional regions. In addition, the online identification method can provide valuable assistance for both neurosurgeons and stereotactic surgery robots in guiding implantation of the electrode in real time.

This paper presents a hybrid optimization strategy for wideband antenna design that leverages the strengths of both Random Forest (RF) and Differential Evolution (DE) algorithms. The strategy employs DE for iteratively updating antenna parameters and RF for feature selection in the process of antenna performance optimization. Initially, DE is applied to update antenna parameters for a predetermined number of iterations, generating a dataset of antenna performance metrics. This dataset is then used to train an RF model, which identifies the importance of each design variable. Feature selection, guided by the RF-derived importance, is applied to reduce the dimensionality of the search space. DE subsequently continues the optimization process within this reduced parameter space. Validation of this hybrid approach is performed through the design of a wideband slot antenna and compared against standalone DE, Genetic Algorithm (GA), Particle Swarm Optimization (PSO), and Simulated Annealing (SA). Results demonstrate that the proposed strategy significantly accelerates convergence, achieving the target reflection coefficient and gain with substantially fewer iterations than the other methods (reductions of 75.56%, 45%, 42.11%, and 50% compared to DE, GA, SA, and PSO, respectively). Furthermore, the hybrid strategy consistently finds superior solutions exhibiting lower loss values compared to the benchmark algorithms. The method offers a computationally efficient, interpretable, and effective approach to antenna optimization.

\n Contains fulltext :\n 210202.pdf (Author’s version postprint ) (Open Access)\n \n Contains fulltext :\n 210202.pdf (Publisher’s version ) (Open Access)\n

<p>Classification problems often have a large number of features, but not all of them are useful for classification. Irrelevant and redundant features may even reduce the classification accuracy. Feature selection is a process of selecting a subset of relevant features, which can decrease the dimensionality, shorten the running time, and/or improve the classification accuracy. There are two types of feature selection approaches, i.e. wrapper and filter approaches. Their main difference is that wrappers use a classification algorithm to evaluate the goodness of the features during the feature selection process while filters are independent of any classification algorithm. Feature selection is a difficult task because of feature interactions and the large search space. Existing feature selection methods suffer from different problems, such as stagnation in local optima and high computational cost. Evolutionary computation (EC) techniques are well-known global search algorithms. Particle swarm optimisation (PSO) is an EC technique that is computationally less expensive and can converge faster than other methods. PSO has been successfully applied to many areas, but its potential for feature selection has not been fully investigated. The overall goal of this thesis is to investigate and improve the capability of PSO for feature selection to select a smaller number of features and achieve similar or better classification performance than using all features. This thesis investigates the use of PSO for both wrapper and filter, and for both single objective and multi-objective feature selection, and also investigates the differences between wrappers and filters. This thesis proposes a new PSO based wrapper, single objective feature selection approach by developing new initialisation and updating mechanisms. The results show that by considering the number of features in the initialisation and updating procedures, the new algorithm can improve the classification performance, reduce the number of features and decrease computational time. This thesis develops the first PSO based wrapper multi-objective feature selection approach, which aims to maximise the classification accuracy and simultaneously minimise the number of features. The results show that the proposed multi-objective algorithm can obtain more and better feature subsets than single objective algorithms, and outperform other well-known EC based multi-objective feature selection algorithms. This thesis develops a filter, single objective feature selection approach based on PSO and information theory. Two measures are proposed to evaluate the relevance of the selected features based on each pair of features and a group of features, respectively. The results show that PSO and information based algorithms can successfully address feature selection tasks. The group based method achieves higher classification accuracies, but the pair based method is faster and selects smaller feature subsets. This thesis proposes the first PSO based multi-objective filter feature selection approach using information based measures. This work is also the first work using other two well-known multi-objective EC algorithms in filter feature selection, which are also used to compare the performance of the PSO based approach. The results show that the PSO based multiobjective filter approach can successfully address feature selection problems, outperform single objective filter algorithms and achieve better classification performance than other multi-objective algorithms. This thesis investigates the difference between wrapper and filter approaches in terms of the classification performance and computational time, and also examines the generality of wrappers. The results show that wrappers generally achieve better or similar classification performance than filters, but do not always need longer computational time than filters. The results also show that wrappers built with simple classification algorithms can be general to other classification algorithms.</p>

ObjectivePostoperative complications remain a major concern in laryngeal cancer surgery, often requiring invasive interventions or intensive care. This study aimed to develop and validate an interpretable machine learning (ML) model to preoperatively predict Clavien-Dindo Grade ≥ III complications and support risk-informed perioperative decision-making.MethodsWe conducted a retrospective study using a temporally split cohort of laryngeal cancer patients. Postoperative complications were graded using the Clavien-Dindo (CD) classification. Eight ML algorithms were trained and evaluated using receiver operating characteristic (ROC) curves, calibration plots, and decision curve analysis (DCA). Model interpretability was assessed using SHapley Additive exPlanations (SHAP). A web-based calculator was deployed for clinical use.ResultThe random forest (RF) model achieved the best performance, with an area under the curve (AUC) of 0.935 in the training set and 0.842 in the test set. The model demonstrated robust sensitivity and specificity for both surgical and medical complications. Calibration curves indicated strong agreement between predicted and actual outcomes. SHAP analysis identified eight key predictors—such as vocal cord mobility, tumor subsite, and nutritional status—that contributed most to risk estimation. A user-friendly web calculator was developed and is accessible at: https://qilushiny.shinyapps.io/qilupredicate/.ConclusionWe developed a clinically interpretable ML model that accurately predicts major postoperative complications in patients undergoing laryngeal cancer surgery. This tool provides individualized risk assessments that can guide surgical planning, optimize perioperative strategies, and enhance shared decision-making. Prospective multicenter validation is needed to confirm its utility in routine practice.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12893-025-03213-z.

Breast cancer ranks among the most prevalent cancers in women globally, with its treatment efficacy heavily reliant on the early identification and diagnosis of the disease. The importance of early detection and diagnosis cannot be overstated in enhancing the survival prospects of those afflicted with breast cancer. With the increasing application of machine learning technology in the medical field, algorithm-based diagnostic tools provide new possibilities for early prediction of breast cancer. In this study, we introduced a novel feature selection approach, which leverages Shapley additive explanation (SHAP) values as the basis for Recursive Feature Elimination (RFE), utilizing a Random Forest (RF) algorithm within the RFE framework. To address the data imbalance challenge, we incorporated Borderline-SMOTE1. The efficacy of the proposed method was assessed using five machine learning models, K-Nearest Neighbor (KNN), Random Forest (RF), Logistic Regression (LR), Support Vector Machine (SVM), and Light Gradient Boosting Machine (LightGBM), applied to the Wisconsin Breast Cancer Diagnosis (WBCD) datasets. Optimizing hyperparameters of five models using the Particle Swarm Optimization (PSO) algorithm. In the datasets, 26 features were filtered using our recommended algorithm, the LightGBM-PSO model demonstrated an outstanding performance. The model demonstrated an impressive accuracy of 99.0% in differentiating between benign and malignant cases, boasting a specificity and precision of 100%, a recall rate of 97.40%, an F-measure of 98.68%, an AUC of 0.9870, and a 10-fold cross-validation accuracy of 0.9808. Subsequently, we developed a corresponding online tool (https://breast-cancer-prediction-tool-cgbjlhkns7yig6bmzvztmc.streamlit.app/) based on this model for predicting the risk of breast cancer. Feature selection using recommended algorithm and optimization of the LightGBM model through PSO can significantly enhance the accuracy of breast cancer prediction. This could potentially improve the prognosis for patients diagnosed with breast cancer.

Optimisation and interpretation of machine and deep learning models for improved water quality management in Lake Loktak

Classification problem in biological Omics data has gained in popularity in recent years. In consideration of the high dimension of the Omics dataset, the importance of feature selection technology becomes apparent. Feature selection can not only improve the classification accuracy but also avoid overfitting which is a common issue in machine learning. There are three main categories of feature selection methods: filter methods, wrapper methods, and hybrid methods. Yet it is difficult for researchers to choose among them. In this study, we conducted a comprehensive comparison for filter methods, wrapper methods, and hybrid methods. Specifically, we selected information gain as the filter method, genetic algorithm, and binary particle swarms optimization as the wrapper methods, IG-GA and IG-BPSO as the hybrid methods for comparison. The experimental results show that the IG-BPSO, a hybrid method, has the highest classification accuracy among these feature selection methods on three Omics datasets.

Union With Recursive Feature Elimination: A Feature Selection Framework to Improve the Classification Performance of Multicategory Causes of Death in Colorectal Cancer

The accurate identification of collision positions on containers is critical in logistics and trade for enhancing cargo safety and determining accident liability. Traditional visual inspection methods are labor-intensive, time-consuming, and costly. This study leverages data from an Inertial Measurement Unit sensor and evaluates combinations of machine learning models and feature selection methods to identify the optimal approach for collision position detection. Five machine learning models (decision tree, k-nearest neighbors, support vector machine, random forest, and extreme gradient boosting) and five feature selection methods (Pearson’s correlation coefficient, mutual information, sequential forward selection, sequential backward selection, and extra trees) were assessed using three performance metrics: accuracy, execution time, and CPU utilization. Statistical analysis with the Friedman test confirmed significant differences in model and feature selection performance. The combination of k-nearest neighbors and extra trees achieved the highest accuracy of 97.1%, demonstrating that inexpensive IMU acceleration data can provide a cost-effective, efficient, and reliable solution for collision detection. This has strong practical implications for improving accident accountability and reducing inspection costs in the logistics industry.

The high-dimensional feature vectors of hyper spectral data often impose a high computational cost as well as the risk of “over fitting ” when classification is performed. Therefore it is necessary to reduce the dimensionality through ways like feature selection. Currently, there are two kinds of feature selection methods: filter methods and wrapper methods. The form kind requires no feedback from classifiers and estimates the classification performance indirectly. The latter kind evaluates the “goodness ” of selected feature subset directly based on the classification accuracy. Many experimental results have proved that the wrapper methods can yield better performance, although they have the disadvantage of high computational cost. In this paper, we present a Genetic Algorithm (GA) based wrapper method for classification of hyper spectral data using Support Vector Machine (SVM), a state-of-art classifier that has found success in a variety of areas. The genetic algorithm (GA), which seek to solve optimization problems using the methods of evolution, specifically survival of the fittest, was used to optimize both the feature subset, i.e. band subset, of hyper spectral data and SVM kernel parameters simultaneously. A special strategy was adopted to reduce computation cost caused by the high-dimensional feature vectors of hyper spectral data when the feature subset part of chromosome was designed. The GA-SVM method was realized using the ENVI/IDL language, and was then tested by applying to a HYPERION hyper spectral image. Comparison of the optimized results and the un-optimized results showed that the GA-SVM method could significantly reduce the computation cost while improving the classification accuracy. The number of bands used for classification was reduced from 198 to