Software Defect Prediction Using Wrapper Feature Selection Based on Dynamic Re-Ranking Strategy

The application of software fault prediction (SFP) to predict faulty components at the early stage has been investigated in various studies. Reducing feature redundancy is key to enhancing the predictive accuracy of SFP models. Feature selection methods are utilized to select and retain the features that contribute the most information while eliminating irrelevant or redundant features from software fault datasets. However, feature selection (FS) in the field of SFP remains a broad and continuously evolving field, encompassing a diverse range of techniques and methodologies. In this work, we study and perform empirical evaluation of ten wrapper FS methods, namely artificial butterfly optimization (ABO), atom search optimization (ASO), equilibrium optimizer (EO), Henry gas solubility optimization (HGSO), poor and rich optimization (PRO), generalized normal distribution optimization (GNDO), slime mold algorithm, Harris hawk’s optimization, pathfinder algorithm (PFA) and manta ray foraging optimization for resolving the data redundancy issue in SFP datasets. Experimental results on nine fault datasets from the PROMISE and AEEEM repositories show that the EO achieves the best performance, with PRO and HGSO ranking next. The comparative analysis revealed that ten wrapper-based FS methods demonstrated a substantial improvement in handling data redundancy issues for SFP.

Many feature selection methods based on supervised learning have investigated side information that considers labelled instances. In this paper, we present a new feature selection method for semi-supervised learning, helped by side information, and the selected features are useful for discovering natural clusters. To select salient features for clustering, most existing wrapper-based feature selection methods study features from the partitioned clusters that result from a prespecified clustering algorithm. The selected features often suffer from both noisy features participating in the partitioning process and bias in the clustering algorithm selection. To overcome these problems, our method does not need clustered information to train features. In particular, our method uses side information to select features directly from the nearest and the farthest neighbours regardless of any clustering algorithm helping to partition data into clusters; specifically, the side information is considered to form pairwise instance constraints to guide the feature selection process and assign a weight for each instance for training. Experimental results indicate that our method, tested on image datasets, outperforms many well known wrapper-based feature selection methods in selecting features for clustering.

Due to the prevalence of high dimensionality(having a large number of independent attributes), feature selection techniques (which reduce the feature subset to amore manageable size) have become quite popular. These reduced feature subsets can help improve the performance of classification models and can also inform researchers about which features are most relevant for the problem at hand. For this latter problem, it is often most important that the features chosen are consistent even in the face of changes(perturbations) to the dataset. While previous studies have considered the problem of finding so-called "stable" feature selection techniques, none has examined stability across all three major categories of feature selection technique: filter-based feature rankers (which use statistical measures to assign scores to each feature), filter-based subset evaluators (which also employ statistical approaches, but consider whole feature subsets at a time), and wrapper-based subset evaluation (which also considers whole subsets, but which builds classification models to evaluate these subsets). In the present study, we use two datasets from the domain of Twitter profile mining to compare the stability of five filter-based rankers, two filter-based subset evaluators, and five wrapper-based subset evaluators. We find that the rankers are most stable, followed by the filter-based subset evaluators, with the wrappers being the least stable. We also show that the relative performance among the techniques within each group is consistent across dataset and perturbation level. However, the relative stability of the two datasets does vary between the groups, showing that the effects are more complex than simply "one group is always more stable than another group".

Feature selection plays a critical role in improving the efficiency, accuracy, and interpretability of machine learning models, particularly when dealing with high-dimensional datasets. Among various approaches, wrapper-based feature selection methods are known for their ability to capture feature interactions by directly optimizing model performance. This study presents a comprehensive comparative analysis of six wrapper feature selection techniques—Recursive Feature Elimination (RFE), Sequential Forward Selection (SFS), Sequential Backward Selection (SBS), Genetic Algorithm (GA), Particle Swarm Optimization (PSO), and Differential Evolution (DE)—in conjunction with five widely used classification algorithms: Decision Tree, K-Nearest Neighbour, Random Forest, Logistic Regression, and Support Vector Machine. Experiments are conducted on an educational dataset comprising 395 student records with 30 attributes obtained from the UCI repository, using different feature subset sizes (all features, top 20, top 15, and top 10). Model performance is evaluated using accuracy, precision, recall, F1-score, and AUC. The results demonstrate that wrapper methods significantly enhance classification performance while reducing dimensionality, with GA and RFE consistently emerging as the most effective techniques across multiple classifiers. DE also shows strong performance, particularly with Logistic Regression and Random Forest, whereas PSO generally underperforms in terms of AUC. Furthermore, reducing the feature set does not adversely affect predictive accuracy and, in several cases, leads to improved generalization. The findings confirm the effectiveness of wrapper methods for educational data mining and provide practical insights for selecting optimal feature–classifier combinations.

Software defect detection is a critical research topic in the field of software engineering, aiming to identify potential defects during the development process to improve software quality and reduce maintenance costs. This study proposes a novel feature selection and defect prediction classification algorithm based on a multi-strategy enhanced Parrot Optimization (PO) algorithm. Firstly, to address the limitations of the original Parrot Optimization algorithm, such as strong dependence on the initial population, premature convergence, and insufficient global search capability, this article develops a multi-strategy enhanced Parrot Optimization algorithm (MEPO). Experiments conducted on eight benchmark test functions validate the superior performance of MEPO in terms of convergence speed and solution accuracy. Secondly, to mitigate the adverse impact of irrelevant features on model performance in traditional software defect prediction methods, this study introduces a binary multi-strategy enhanced Parrot Optimization algorithm (BMEPO) for optimizing feature selection. Comparative experiments demonstrate that BMEPO exhibits stronger competitiveness in feature selection quality and classification performance compared to advanced feature selection algorithms. Finally, to further enhance the classification performance of defect prediction, a heterogeneous data stacking ensemble learning algorithm (HEDSE) based on feature selection is proposed. Experimental evaluations on 16 open-source software defect datasets indicate that the proposed HEDSE outperforms existing methods, providing a novel and effective solution for software defect prediction. The proposed approaches in this study hold significant practical value, particularly in improving software quality, optimizing testing resource allocation, and reducing maintenance costs, offering broad potential for application in real-world software engineering scenarios.

<p>Classification aims to identify a class label of an instance according to the information from its characteristics or features. Unfortunately, many classification problems have a large feature set containing irrelevant and redundant features, which reduce the classification performance. In order to address the above problem, feature selection is proposed to select a small subset of relevant features. There are three main types of feature selection methods, i.e. wrapper, embedded and filter approaches. Wrappers use a classification algorithm to evaluate candidate feature subsets. In embedded approaches, the selection process is embedded in the training process of a classification algorithm. Different from the other two approaches, filters do not involve any classification algorithm during the selection process. Feature selection is an important process but it is not an easy task due to its large search space and complex feature interactions. Because of the potential global search ability, Evolutionary Computation (EC), especially Particle Swarm Optimization (PSO), has been widely and successfully applied to feature selection. However, there is potential to improve the effectiveness and efficiency of EC-based feature selection. The overall goal of this thesis is to investigate and improve the capability of EC for feature selection to select small feature subsets while maintaining or even improving the classification performance compared to using all features. Different aspects of feature selection are considered in this thesis such as the number of objectives (single-objective/multi-objective), the fitness function (filter/wrapper), and the searching mechanism. This thesis introduces a new fitness function based on mutual information which is calculated by an estimation approach instead of the traditional counting approach. Results show that the estimation approach works well on both continuous and discrete data. More importantly, mutual information calculated by the estimation approach can capture feature interactions better than the traditional counting approach. This thesis develops a novel binary PSO algorithm, which is the first work to redefine some core concepts of PSO such as velocity and momentum to suit the characteristics of binary search spaces. Experimental results show that the proposed binary PSO algorithm evolve better solutions than other binary EC algorithms when the search spaces are large and complex. Specifically, on feature selection, the proposed binary PSO algorithm can select smaller feature subsets with similar or better classification accuracies, especially when there are a large number of features. This thesis proposes surrogate models for wrapper-based feature selection. The surrogate models use surrogate training sets which are subsets of informative instances selected from the training set. Experimental results show that the proposed surrogate models assist PSO to reduce the computational cost while maintaining or even improving the classification performance compared to using only the original training set. The thesis develops the first wrapper-based multi-objective feature selection algorithm using MOEA/D. A new decomposition strategy using multiple reference points for MOEA/D is designed, which can deal with different characteristics of multi-objective feature selection such as highly discontinuous Pareto fronts and complex relationships between objectives. The experimental results show that the proposed algorithm can evolve more diverse non-dominated sets than other multi-objective algorithms. This thesis introduces the first PSO-based feature selection algorithm for transfer learning. In the proposed algorithm, the fitness function uses classification performance to reduce the differences between domains while maintaining the discriminative ability on the target domain. The experimental results show that the proposed algorithm can select feature subsets which achieve better classification performance than four state-of-the-art feature-based transfer learning algorithms.</p>

The Microarray technology permits simultaneous monitoring of several gene expressions for every sample. But unfortunately, this classification of such samples into the distinct classes has often been found to difficult as the actual number of genes (the features) will to a great extent exceed the actual number of the samples. There is a high dimensionality in gene and its expression data which is a huge challenge in many of the problems of classification. Cloud computing is that popular concept of computing which performs a processing of a data of huge volumes by making use of highly accessible resources that are geographically distributed and accessed by the users based on the policy of pay as per use. In spite of the several steps in that of the B Cell that are now elucidate the last few stages of that of the plasma cell (PC) based differentiation that have not been understood yet. The PCs had generated at the time of primary and humoral immune responses that have started their differentiation within their light zones for the germinal centers of such light zones of that of the lymph nodes or the ones in the red pulp of the spleen having a life span of about few days. The selection of the features will aim at the maintenance of their original features of such data and at this time it will seek at identifying their main features and will weed out all those that are irrelevant for building of a learning model that is impactful. Identifying one global maximum will be Non-deterministic Polynomial (NP)-hard and if the criterion is decomposable or possesses the properties that can make some approximate type of optimization easy. The artificial bee colony (ABC) is that one that is used widely and applied for solving all real world problems. The stochastic diffusion search (SDS) will be an efficient multi-agent global search based technique of optimization applied to that of several problems and here the hybrid ABC with that of the SDS feature selection will be proposed and these images will be grouped as either normal or abnormal PC of the bone marrow that is based on this gene expression data. The support vector machine is that algorithms of supervised learning that is capable of being able to solve the complex problems in classification. This proposed method of gene selection will yield a comparable performance for the classification on being compared to that of the currently existing classifiers providing another new insight in case of feature selection.

Multiobjective feature selection for key quality characteristic identification in production processes using a nondominated-sorting-based whale optimization algorithm

Previously, many researchers have performed analysis of various techniques for the software fault prediction (SFP). Oddly, the majority of such studies have shown the limited prediction capability and their performance for given software fault datasets was not persistent. In contrast to this, recently, ensemble techniques based SFP models have shown promising and improved results across different software fault datasets. However, many new as well as improved ensemble techniques have been introduced, which are not explored for SFP. Motivated by this, the paper performs an investigation on ensemble techniques for SFP. We empirically assess the performance of seven ensemble techniques namely, Dagging, Decorate, Grading, MultiBoostAB, RealAdaBoost, Rotation Forest, and Ensemble Selection. We believe that most of these ensemble techniques are not used before for SFP. We conduct a series of experiments on the benchmark fault datasets and use three distinct classification algorithms, namely, naive Bayes, logistic regression, and J48 (decision tree) as base learners to the ensemble techniques. Experimental analysis revealed that rotation forest with J48 as the base learner achieved the highest precision, recall, and G-mean 1 values of 0.995, 0.994, and 0.994, respectively and Decorate achieved the highest AUC value of 0.986. Further, results of statistical tests showed used ensemble techniques demonstrated a statistically significant difference in their performance among the used ones for SFP. Additionally, the cost-benefit analysis showed that SFP models based on used ensemble techniques might be helpful in saving software testing cost and effort for twenty out of twenty-eight used fault datasets.

The enhancement of Intrusion Detection Systems (IDS) is required to ensure protection of network resources and services. This is a hot research topic, especially in the presence of advanced intrusions and attacks. This paper provides a comparison between Distributed Cumulative Histogram (DCH) as a Feature Selection (FS) technique, Information Gain Ratio (IGR) FS and wrapper-based FS in terms of accuracy and Root Mean Square Error (RMSE). The utilization of DCH of the traffic instances in normal and attack cases allows us to compare the traffic charts. We can observe the difference between effective features and less effective ones. We verify the feasibility of using DCH as an FS technique in the field of anomaly detection with just six selected features giving more accurate results with most classifiers compared to the IGR and wrapper-based FS. We applied our experiments on the modern UNSW dataset with the WEKA simulation platform that contains a group of classification, feature reduction and selection techniques.

Pattern recognition involves identifying a new instance as a member of a class based on the features extracted from problem space. Not all features contribute to increase in accuracy of classification. Wrapper based methodology selects an optimal subset of reduced features with no much loss in classification accuracy. Inspired by the bio evolutionary behaviour of flowers in optimal reproduction, a wrapper based optimal feature selection methodology is devised. In this paper, a Binary Flower Pollination Algorithm (BFPA) is proposed to address the combinatorial problem of feature selection. Support Vector machine (SVM) is used as a subset evaluator to calculate the fitness function to be optimized. Proposed algorithm is evaluated using 5 datasets from UCI repository. Switching probability is varied to achieve a fair tradeoff between exploration and exploitation. Feature subset chosen by the proposed method serves as input to 6 traditional classifiers to analyse the success of the algorithm. Experimental results are also compared with existing literature and observed that the proposed methodology identifies feature subset with reduced features and improved accuracy.

Defects that occur in software products are a universal occurrence. Software defect prediction is usually carried out to determine the performance, accuracy, precision and performance of the prediction model or method used in research, using various kinds of datasets. Software defect prediction is one of the Software Engineering studies that is of great concern to researchers. This research was conducted to determine the performance of tree-based classification algorithms including Decision Trees, Random Forests and Deep Forests without using feature selection and using firefly feature selection. And also know the tree-based classification algorithm with firefly feature selection which can provide better software defect prediction performance. The dataset used in this study is the ReLink dataset which consists of Apache, Safe and Zxing. Then the data is divided into testing data and training data with 10-fold cross validation. Then feature selection is performed using the Firefly Algorithm. Each ReLink dataset will be processed by each tree-based classification algorithm, namely Decision Tree, Random Forest and Deep Forest according to the results of the firefly feature selection. Performance evaluation uses the AUC value (Area under the ROC Curve). Research was conducted using google collab and the average AUC value generated by Firefly-Decision Tree is 0.66, the average AUC value generated by Firefly-Random Forest is 0.77, and the average AUC value generated by Firefly-Deep Forest is 0, 76. The results of this study indicate that the approach using the Firefly algorithm with Random Forest classification can work better in predicting software damage compared to other tree-based algorithms. In previous studies, tree-based classification with hyperparameter tuning on software defect prediction datasets obtained quite good results. In another study, the classification performance of SVM, Naïve Bayes and K-nearest neighbor with firefly feature selection resulted in improved performance. Therefore, this research was conducted to determine the performance of a tree-based algorithm using the firefly selection feature.

This paper analyzes the underlying complexity and non-linearity of electroencephalogram (EEG) signals by computing a novel multi-scale entropy measure for the classification of seizure, seizure-free and normal EEG signals. The quality factor (Q) based multi-scale entropy measure is proposed to compute the entropy of the EEG signal in different frequency-bands of interest. The Q -based entropy (QEn) is computed by decomposing the signal with the tunable-Q wavelet transform (TQWT) into the number of sub-bands and estimating K-nearest neighbor (K-NN) entropies from various sub-bands cumulatively. The optimal selection of Q and the redundancy parameter (R) of TQWT showed better robustness for entropy computation in the presence of high- and low-frequency components. The extracted features are fed to the support vector machine (SVM) classifier with the wrapper-based feature selection method. The proposed method has achieved accuracy of 100% in classifying normal (eyes-open and eyes-closed) and seizure EEG signals, 99.5% in classifying seizure-free EEG signals (from the hippocampal formation of the opposite hemisphere of the brain) from seizure EEG signals and 98% in classifying seizure-free EEG signals (from the epileptogenic zone) from seizure EEG signals, respectively, using the SVM classifier. We have also achieved classification accuracies of 99% and 98.6% in classifying seizure versus non-seizure EEG signals and the individual three classes, namely normal, seizure-free and seizure EEG signals, respectively. The performance measure of the proposed multi-scale entropy has been found to be comparable with the existing state of the art epileptic EEG signals classification methods studied using the same database.

DroidRL: Feature selection for android malware detection with reinforcement learning

In the present state of affairs, the motive behind every educational organization is to uplift the academic achievement of students. Educational data mining (EDM) is an upward field of research, and it is very helpful for academic institutions to predict the academic performance of the students. Educational datasets are the basis of various predictive models. Quality of these models can be improved by using feature selection (FS). To get the required benefits from the available data, there must be some tools for analysis and prediction. In lieu of the above, machine learning/data mining are most suitable. In educational data mining, for better accuracy of prediction models’ and quality of various educational datasets, feature selection (FS) plays a vital role. Feature selection (FS) algorithms abolish inappropriate information from the repositories of educational background so that performance of classifier in terms of accuracy could be increased and the same could be used for better decision. In lieu of the above, a best feature selection algorithm must be selected. In this paper, two filter selection approaches namely correlation feature selection (CFS) and wrapper-based feature selection have been used to demonstrate the importance of selection of a feature subset for a classification problem. The present paper aims to find the detailed investigation of filter feature selection algorithms along with the classification algorithms on a given dataset. We found result with numerous numbers of features from various Feature selection algorithms and classifiers which will help the researcher to discover the most excellent mixture of filter feature selection algorithms and its associated classifiers. The result indicates that SMO and J48 have the highest accuracy measures with the correlation feature selection algorithms, while Naïve Bayes has the highest accuracy measures with the wrapper subset feature selection algorithms for predicting high, medium and low grade for the students.