Early software defect prediction: A systematic map and review

Software defect prediction models help testers find program modules that have a high probability of having defects. A method-calling network can express the dependencies between methods in a program. Existing approaches do not sufficiently utilize method-calling network to characterize the structural features between methods. To address this problem, in this study, it is proposed for the first time that the characteristics of methods in a program are obtained by analyzing the method-calling network, and a new approach is proposed for defect prediction at the method-level. Specifically in this study, the method-calling network of the program was first constructed and the network metrics of the method-calling network were obtained. Next, the new network embedding technique (node2vec) was used to automatically encode the method-calling network structure into a low-dimensional vector to obtain the network embedding metrics. Finally, they were combined with code metrics to construct defect prediction models. We evaluated our approaches on 13 open-source software systems. The experimental results show that the proposed method improved the values of area under the receiver operating characteristic curve by 2.5% to 6.7% and Matthews correlation coefficient by 13% to 178.4% compared to the baselines. Therefore the method-calling network contains rich structural features between methods, and the structural features extend the features used for method-level defect prediction and further improve the performance of defect prediction models.

Software dependency network metrics extracted from the dependency graph of the software modules by the application of Social Network Analysis (SNA metrics) have been shown to improve the performance of the Software Defect prediction (SDP) models. However, the relative effectiveness of these SNA metrics over code metrics in improving the performance of the SDP models has been widely debated with no clear consensus. Furthermore, some of the common SDP scenarios like predicting the number of defects in a module (Defect-count) in Cross-version and Cross-project SDP contexts remain unexplored. Such lack of clear directive on the effectiveness of SNA metrics when compared to the widely used code metrics prevents us from potentially building better performing SDP models. Therefore, through a case study of 9 open source software projects across 30 versions, we study the relative effectiveness of SNA metrics when compared to code metrics across 3 commonly used SDP contexts (Within-project, Cross-version and Cross-project) and scenarios (Defect-count, Defect-classification (classifying if a module is defective) and Effort-aware (ranking the defective modules w.r.t to the involved effort)). We find the SNA metrics by themselves or along with code metrics improve the performance of SDP models over just using code metrics on 5 out of the 9 studied SDP scenarios (three SDP scenarios across three SDP contexts). However, we note that in some cases the improvements afforded by considering SNA metrics over or alongside code metrics might only be marginal, whereas in other cases the improvements could be potentially large. Based on these findings we suggest that the future work should: consider SNA metrics alongside code metrics in their SDP models; as well as consider Ego metrics and Global metrics, the two different types of the SNA metrics separately when training SDP models as they behave differently.

Software testing is a time-consuming and expensive process. Software fault prediction models are used to identify fault-prone classes automatically before system testing. These models can reduce the testing duration, project risks, resource and infrastructure costs. In this study, we propose a novel fault prediction model to improve the testing process. Chidamber-Kemerer Object-Oriented metrics and method-level metrics such as Halstead and McCabe are used as independent metrics in our Artificial Immune Recognition System based model. According to this study, class-level metrics based model which applies AIRS algorithm can be used successfully for fault prediction and its performance is higher than J48 based approach. A fault prediction tool which uses this model can be easily integrated into the testing process.

Software Defect Prediction (SDP) plays an active area in many research domain of Software Quality of Assurance (SQA). Many existing research studies are based on software traditional metric sets and defect prediction models are built in machine language to detect the bug for limited source code line. Inspired by the above existing system. In this paper, defect prediction is focused on predicting defects in source code. The objective of this thesis is to improve the software quality for accurate defect prediction is source code for file level. So, that it helps the developer to find the bug and fix the issue, to make better use of a resource which reduces the test effort, minimize the cost and improve the quality of software. A new approach is introduced to improve the prediction performance of Bidirectional RNNLM in Deep Neural Network. To build the defect prediction model a defect learner framework is proposed and first it need to build a Neural Language Model. Using this Language Model it helps to learn to deep semantic features in source code and it train & test the model. Based on language model it combined with software traditional metric sets to measure the code and find the defect. The probability of language model and metric set Cross-Entropy with Abstract Syntax Tree (CE-AST) metric is used to evaluate the defect proneness and set as a metric label. For classification the metric label K-NN classifier is used. BPTT algorithm for learning RNN will provide additional improvement, it improves the predictions performance to find the dynamic error.

In recent years, data mining algorithm have been successfully applied in software defects and metrics prediction throughout the software development life cycle (SDLC). Features dependencies and class imbalance are the two main issues in the traditional Bayesian models for defect prediction. Traditional supervised models build high prediction rate with limited single product defect and metric data. Various models have been introduced in the literature to establish a common framework for defect and metric prediction in academia, standards and industry, but none of them are accepted as a common tool in real-time software products. Also, predicting the defects cum metrics in multiple products is one of the common problems in real time associated products. In this proposed model, a new ensemble defect prediction classification model was implemented on multiple associated products to predict metric relationship, along with defects patterns. Experimental results show that proposed model has high true positive rate compare to traditional Bayesian network models.

Software industries have enormous demand for fault prediction of the faulty module and fault removal techniques. Many researchers have developed different fault prediction models to predict the fault at an early stage of the software development life cycle (SDLC). But the state-of-the-art model still suffers from the performance and generalize validation of the models. However, some researchers refer to data mining techniques, machine learning, and artificial intelligence play crucial roles in developing fault prediction models. A recent study stated that metric selection techniques also help to enhance the performance of models. Hence, to resolve the issue of improving the fault prediction model’s performance and validation, we have used data mining, instance selection, metric selection, and ensemble methods to beat the state-of-the-art results. For the validation, we have collected the 22 software projects from the four different software repositories. We have implemented three machine learning algorithms and three ensemble methods with two metric selection methods on 22 datasets. The statistical evaluation of the implemented model performed using Wilcoxon signed-rank test and the Friedman test followed by the Nemenyi test to find the significant model. As a result, the Random forest algorithm produces the best result with an average median of 95.43% (accuracy) and 0.96 (f-measure) on 22 software projects. Based on the Nemenyi test, Random forest (RF) is performing better with 4.54 (accuracy mean score) and 4.41 (f-measure mean score) shown in the critical diagram. Experimental study shows that data mining techniques with PCA provide better accuracy and f-measure.

Patterns are a means for capturing experience in solving general problems. They give general solutions to common and recurring problems. They exist at various phases of the software development life-cycle (SDLC), namely the analysis phase, the design phase, the coding phase and the testing phase. Patterns in each phase are identified independently. It is possible for the patterns of one phase to be related to patterns in other phases. This paper proposes pattern mapping to inter-relate the patterns of various phases in SDLC to address this issue. It is advantageous to capture the related patterns of different phases as a single unit for software development. Reusable integrated components (ICs) of patterns are proposed to capture the related patterns. A software IC is a group of inter-related patterns, that gives solutions for a recurring problem at various phases of SDLC. A representation mechanism for ICs is presented using the URA (Unified Representation of an Artifact) model, which captures the software development process. Also, this paper proposes a software development process that is based on reusable ICs of patterns. The representation of a software project developed using ICs is also discussed.

Software fault prediction (SFP) involves the identification ofpotentially fault-prone modules before the testing phase in the softwaredevelopment lifecycle. By predicting faults early in the development process, the SFP process enables software developers to focus their efforts oncomponents that may contain faults, thereby enhancing the overall quality and reliability of the software. Machine learning and deep learningtechniques have been widely applied to train SFP models. However, theseapproaches face several challenges, including irrelevant or redundant features, imbalanced datasets, overfitting, and complex model structures.The NASA dataset from the PROMISE repository is the most commonlyused dataset for fault prediction. Recently, the BugHunter dataset withits substantially larger number of instances was explored to train the SFPmodels. In this study, we present the comparative study of three deeplearning models, including Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM) andfour machine learning models as K-Nearest Neighbors (KNN), MultilayerPerceptron (MLP), Adaptive Boosting (AdaBoost), Extreme GradientBoosting (XGB) to investigate the performance of SFP models on theBugHunter dataset. We employ the Lasso method for feature selectionand apply the Synthetic Minority Oversampling Technique (SMOTE) toaddress the issue of imbalanced data, aiming to enhance the accuracyof the results. The experimental findings reveal that CNN and RNNoutperformed other machine learning models, achieving the best overallperformance.

SummaryContext: Software performance is crucial for ensuring the quality of software products. As one of the non‐functional requirements, the few efforts devoted to software performance have often been neglected until a later phase in the software development life cycle (SDLC). The lack of clarity of what software performance research literature is available prevents researchers from understanding what software performance research fields are available. It also creates difficulty for practitioners to adopt state‐of‐the‐art software performance techniques. Software performance research is not as organized as other established research topics such as software testing. Thus, it is essential to conduct a systematic mapping study as a first step to provide an overview of the latest research literature available in software performance. Objective: The objective of this systematic mapping study is to survey and map software performance research literature into suitable categories and to synthesize the literature data for future access and reference. Method: This systematic mapping study conducts a manual examination by querying research literature in noble journals and proceedings in software engineering in the past decade. We examine each paper manually and identify primary studies for further analysis and synthesis according to the pre‐defined inclusion criteria. Lastly, we map the primary studies based on their corresponding classification category. Results: This systematic mapping study provides a state‐of‐the‐art literature mapping in software performance research. We have carefully examined 222 primary studies out of 2000+ research literature. We have identified six software performance research categories and 15 subcategories. We generate the primary study mapping and report five research findings. Conclusions: Unlike established research fields, it is unclear what types of software performance research categories are available to the community. This work takes the systematic mapping study approach to survey and map the latest software performance research literature. The study results provide an overview of the paper distribution and a reference for researchers to navigate research literature on software performance.

The bearings of double toothed roller crushers in open pit mines are prone to failure during prolonged operation, resulting in reduced productivity and shortened equipment life. Therefore, a bearing fault prediction model based on asymmetric loss function gradient boosting algorithm and digital twin has been proposed for effective fault prediction of double toothed roller crusher bearings. First, the fault signal is extracted by Wigner-Ville time-frequency distribution and empirical mode decomposition. Then, the asymmetric loss function gradient boosting algorithm is used to model and predict the signal characteristics. Finally, a digital twin model is constructed to visualize bearing fault information by combining sensor data and historical fault data. The experimental results showed that the fault prediction model achieved a precision rate of 98%, a recall rate of 99%, and an F1 value of 98.5% in predicting bearing rolling element failures. Among them, the running time of fault prediction was 1 minute and 49s. The experimental results demonstrate that the proposed fault prediction model can learn complex feature representations from large amounts of data, with good fault prediction performance and efficiency, and has certain practical application value. The research results contribute to improving the accuracy and intelligence level of fault prediction methods, thereby ensuring the stable operation of open-pit mine double toothed roller crushers.

Despite the fact that a number of approaches have been proposed for effective and accurate prediction of software defects, yet most of these have not found widespread applicability. Our objective in this communication is to provide a framework which is expected to be more effective and acceptable for predicting the defects in multiple phases across software development lifecycle. The proposed framework is based on the use of neural networks for predicting defects in software development life cycle. Further, in order to facilitate the easy use of the framework by project managers, a software graphical user interface has been developed that allows input data (including effort and defect) to be fed easily for predicting defects. The proposed framework provides a probabilistic defect prediction approach where instead of a definite number, a defect range (minimum, maximum, and mean) is predicted. The claim of efficacy and superiority of proposed framework is established through results of a comparative study, involving the proposed frame-work and some well-known models for software defect prediction.

Software testing and defect prediction are widely recognized quality assurance activities that can contribute to the effectiveness and efficiency of Just-In-Time Software Quality Assurance (JIT-SQA) by automating processes and reducing software maintenance costs. However, most studies treat them separately, with limited exploration of how defect prediction can guide testing and how testing can support prediction. Existing methods are not tailored for JIT-SQA and often ignore change-specific characteristics such as concept drift and verification latency, leading to unstable defect prediction. Moreover, test annotations on recent code changes are rarely used to update models, missing an opportunity to improve accuracy. Beyond this, effectively leveraging test-annotated samples remains an open research question. On the one hand, these samples frequently suffer from one-sided label noise, as instances that are not identified as defective during testing may still contain defects. On the other hand, defect-inducing changes are relatively infrequent, leading to a significant class imbalance issue. Therefore, exploring how to utilize test annotations to mitigate class imbalance and enhance prediction robustness presents a critical challenge. To fill this research gap, we propose an automatic JIT-SQA framework that incorporates defect prediction and testing processes collaboratively. Specifically, we direct testing activities through defect prediction models and design a novel update mechanism that integrates waiting time with testing feedback to update the predictor in real time. Moreover, we further design a reinforcement learning-based dynamic selection mechanism to mitigate the instability in defect prediction performance over time. Our framework facilitates mutual guidance between automatic testing and defect prediction, leveraging shared knowledge within the JIT-SQA system. This integration holds significant potential for producing high-quality software within modern development timelines. Notably, this study is the first to investigate the integrated application of these two quality assurance activities within a unified framework for JIT-SQA throughout the software development process. We conducted experiments using both statically typed (Java) and dynamically typed (Python) languages by integrating various testing techniques. The results indicate that, after implementing the novel update and dynamic selection mechanisms, our framework significantly outperforms simple integrated frameworks of defect prediction and testing. Specifically, the Recall improves by 21.6%, 10.5%, and 6.3% compared with the testing-integrated versions of ODaSC, ECo-HumLa, and HumLa, respectively, while the F1 score improves by 13.2%, 7.0%, and 5.1%. At the same time, in terms of guiding testing activities, the total number of tests is reduced by 2.9%, 19.5%, and 24.8% compared with the same baselines. These findings demonstrate that our framework not only mitigates the adverse effects of concept drift, verification latency, and class imbalance in code evolution scenarios but also effectively guides testing to reduce resource consumption.

The demand for automated online software systems is increasing day by day, which triggered the need for high-quality and maintainable softwares at lower cost. Software defect prediction is one of the crucial tasks of the quality assurance process which improves the quality at lower cost by reducing the overall testing and maintenance efforts. Early detection of defects in the software development life cycle (SDLC) leads to the early corrections and ultimately timely delivery of maintainable software, which satisfies the customer and makes him confident towards the development team. In the last decade, many machine learning-based approaches for software defect prediction have been proposed to achieve the higher accuracy. Artificial Neural Network (ANN) is considered as one of the widely used machine learning techniques, which is included in most of the proposed defect prediction frameworks and models. This research provides a critical analysis of the latest literature, published from year 2015 to 2018 on the use of Artificial Neural Networks for software defect prediction. In this study, a systematic research process is followed to extract the literature from three widely used digital libraries including IEEE, Elsevier, and Springer, and then after following a thorough process, 8 most relevant research publications are selected for critical review. This study will serve the researchers by exploring the current trends in software defect prediction with the focus on ANNs and will also provide a baseline for future innovations, comparisons, and reviews.

Software defect prediction is an important issue in the process of software development and maintenance, which is related to the overall success or failure of software. This is because early software failure prediction can improve software quality, reliability and efficiency, and reduce software cost. However, developing robust defect prediction models is a challenging task and many techniques have been proposed in the literature. In this paper, a software defect prediction model based on Novel Hybrid Genetics Software Defect Prediction (NHGSDP) is proposed. The supervised NHGSDP algorithm has been used to predict future software failures based on historical data. The evaluation process shows that the NHGSDP algorithm can be used effectively with high accuracy. The collected results show that the NHGSDP method has better performance.

Software testing plays important role in software development life cycle (SDLC) while dealing with critical business applications. From SDLC 70% of time and cost is used for testing only, it is time consuming and costly phase. For this one of the cost effective technique can be used which predicts a module as fault-prone and non-fault prone prior to testing. Predicting a safe module during testing and after development increases the cost of projects. As need to allocate better-test resources for module. This process may take most of the time in preparation and may lose some modules untested which cause accidental failure. Defective software modules cause software failures, increase development and maintenance costs, and decrease customer satisfaction. In this research, we present a novel fault prediction technique that reduces the probability of false (pf) and increases the precision for detection of faulty modules. Software Reliability is becoming an essential attribute of any software system. To improve software quality and testing efficiency by constructing predictive models from code attributes to enable a timely identification of fault-prone modules.Software defect prediction model helps in early detection of faults and contribute to their efficient removal and producing a reliable software system.