• Home
  • Search
  • Performance evaluation of SFP models using ML/DL and feature selection via cost evaluation framework.
  • https://doi.org/10.1038/s41598-026-58865-5Copy DOI Icon

Performance evaluation of SFP models using ML/DL and feature selection via cost evaluation framework.

Show More
  • Abstract
  • Literature Map
  • Similar Papers
Abstract

The increasing complexity of software systems (SW) makes fault prevention challenging. Although significant amount of research has explored Software Fault Prediction (SFP). However the extensive comparative analysis of Deep Learning (DL), state-of-the-art, and Machine Learning (ML) techniques, with the use of a Cost Evaluation Framework (CEF) remains largely unexplored. Furthermore, the combined impact of different feature selection (FS) techniques and DL algorithms has not been comprehensively investigated. This research aims to perform a large-scale comparative analysis DL and ML-based SFP models, incorporating SMOTE and ten feature selection (FS) strategies to address class and feature redundancy. Additionally the study employs a Cost Evaluation Framework (CEF), in addition to traditional metrics such as Accuracy, and AUC to assess the effectiveness of DL algorithms in SFP. Furthermore, the study investigates whether DL-based SFP models can reduce testing costs in real-world projects compared to state-of-the-art and ML approaches. This study employs 32 methods to train SFP models, comprising 8 state-of-the-art techniques, 16 ML methods, and 6 DL approaches, evaluated against 54 open-source projects to assess performance. To address feature redundancy and develop an efficient SFP model, the study analysed 11 FS techniques, including one that utilised the original dataset with complete feature measurements. The research additionally examined the application of the Synthetic Minority Oversampling Technique (SMOTE) to rectify class imbalance. The study introduced a cost evaluation methodology to assess the cost-effectiveness of the developed fault prediction model relative to actual testing efforts and to ascertain whether the implementation of the fault prediction model yields profit. Accuracy, Area under the Receiver Operating Characteristic (ROC) Curve (AUC) and CEF performance metrics are used to analyse the performance of developed SFP models. The experimental outcome indicates AdaBoost (ABC), Random Forest (RFC), and Extremely Randomized Trees (ETC), excluding SMOTE, exhibit the highest median AUC values of 0.73, 0.72, and 0.72 compared to other methods. ETC, RFC, and BAGD (Bagging Decision Tree) exhibited superior prediction, attaining median AUC values of 0.94, 0.93, and 0.93, respectively, following the adoption of SMOTE, across1188 datasets. Without SMOTE, the DL2 algorithm attained optimal performance, evidenced by the lowest mean-rank value of 11.58. The DL5 model utilising SMOTE exhibited superior prediction, achieving a mean rank of 9.37 and an AUC of 0.82, indicating a significantly enhanced performance. The completed comparative evaluation of various feature selection methodologies indicated that the SMOTE-based Consistency Feature Subset Selection (FS3) achieved a marginally superior median AUC value of 0.70 and the lowest mean rank of 5.84, aligning more closely with the AUC achieved by all metrics, thus exhibiting improved performance compared to other techniques. The experimental analysis, evaluated using AUC, Accuracy, Statical description and Friedman test indicates that while SMOTE based ML models achieved slightly higher performance than SMOTE-based DL models, the differences were marginal, indicating that DL performed competitively and attained comparable predictive capability. FS techniques eliminated 70-75% of irrelevant features, enabling DL-based SFP models to maintain or improve performance using reduced datasets. The cost-analysis framework results demonstrate that according to the engineer's low, medium and high testing efficiency, the fault prediction model is better suited for projects with faulty classes below the average threshold value of 49.48%, 38.86%, and 27.17% respectively.

Similar Papers
  • Research Article
  • Citations1

ATE-FS: An Average Treatment Effect-based Feature Selection Technique for Software Fault Prediction

  • Feb 11, 2025
  • ACM Transactions on Intelligent Systems and Technology
  • Akshat Mangal +1
  • Research Article
  • Citations17

ELM and KELM based software defect prediction using feature selection techniques

  • Jul 04, 2019
  • Journal of Information and Optimization Sciences
  • Ishani Arora +1
  • Book Chapter
  • Citations4

Is Open-Source Software Valuable for Software Defect Prediction of Proprietary Software and Vice Versa?

  • Nov 25, 2017
  • Misha Kakkar +3
  • Conference Article
  • Citations5

Research on software defect prediction technology based on deep learning

  • Jan 01, 2021
  • Pengcheng Jiang
  • Components

Supp1-3131950.pdf

  • Dec 02, 2021
  • Gopi Krishnan Rajbahadur
  • Research Article
  • Citations23

Revisiting the Impact of Dependency Network Metrics on Software Defect Prediction

  • Jan 01, 2021
  • IEEE Transactions on Software Engineering
  • Lina Gong +3
  • PDF
  • Research Article
  • Citations133

Performance Analysis of Feature Selection Methods in Software Defect Prediction: A Search Method Approach

  • Jul 09, 2019
  • Applied Sciences
  • Abdullateef Oluwagbemiga Balogun +3
  • PDF
  • Research Article

Optimizing software defect prediction performance via call graph embedding and feature selection

  • Nov 11, 2025
  • Cluster Computing
  • Sweta Mehta +3
  • Research Article
  • Citations5

An Optimized Software Defect Prediction Model Based on PSO - ANFIS

  • Dec 01, 2021
  • Recent Advances in Computer Science and Communications
  • Misha Kakkar +3
  • Conference Article
  • Citations3

Evaluating Missing Values for Software Defect Prediction

  • Feb 01, 2019
  • Misha Kakkar +3
  • Research Article
  • Citations1

Effective software defect prediction with deep neural networks

  • Mar 01, 2026
  • Results in Engineering
  • Somya R Goyal
  • Research Article
  • Citations50

Empirical evaluation of the performance of data sampling and feature selection techniques for software fault prediction

  • Mar 17, 2023
  • Expert Systems with Applications
  • Sonika Chandrakant Rathi +5
  • Conference Article
  • Citations18

Wrapper-based Feature Selection for Imbalanced Data using Binary Queuing Search Algorithm

  • Oct 01, 2019
  • Thaer Thaher +3
  • Research Article

Towards Robust Intrusion Detection: Exploring Feature Selection, Balancing Strategies, and Deep Learning for Minority Class Optimization

  • Jan 01, 2026
  • International Journal of Advanced Computer Science and Applications
  • Khalid Labhalla +1
  • Conference Article
  • Citations4

Performance Analysis of Feature Selection Techniques in Software Defect Prediction using Machine Learning

  • Nov 19, 2022
  • Kunal Anand +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.