Software Defect Prediction Using Process Metrics Systematic Literature Review: Dataset and Granularity Level

Mobile applications have been around for a long time and proved to be a new excited market where everyone want to engage themselves. They have become more important than Web pages nowadays. Companies are giving more preference to mobile apps as compared to Web sites because of their user friendliness , better visibility and ease of social networking. This paper compares static code metrics and process metrics for predicting defects in an open source mobile applications. Correlation coefficient, mean absolute error and root mean squared error with process metrics as predictors are significantly better than with code metrics as predictors. Also the combined model based on process and code metrics is better than the model based on code metrics. It is shown that process metrics based defect prediction models are better for mobile applications in all 7 machine learning techniques used for modelling.

The correct prediction of faulty modules or classes has a number of advantages such as improving the quality of software and assigning capable development resources to fix such faults. There have been different kinds of fault/defect prediction models proposed in literature, but a great majority of them makes use of static code metrics as independent variables for making predictions. Recently, process metrics have gained a considerable attention as alternative metrics to use for making trust-worthy predictions. The objective of this paper is to investigate different combinations of static code and process metrics for evaluating fault prediction performance. We have used publicly available data sets, along with a frequently used classifier, Naive Bayes, to run our experiments. We have, both statistically and visually, analyzed our experimental results. The statistical analysis showed evidence against any significant difference in fault prediction performances for a variety of different combinations of metrics. This reinforced earlier research results that process metrics are as good as predictors of fault proneness as static code metrics. Furthermore, the visual inspection of box plots revealed that the best set of metrics for fault prediction is a mix of both static code and process metrics. We also presented evidence in support of some process metrics being more discriminating than others and thus making them as good predictors to use.

Software metrics is one of the most active research areas of software engineering. In this paper, a process model for software process metrics is proposed, which defines the roles, activities and related supporting tools and methods for the implementation of software process metrics, and gives an emphasis on the description of the goals, tasks and methods for the activity of process data collection, validation and analysis. Architecture of a process metrics infrastructure supporting software process Metrics is presented. Furthermore, an application of this model in a large-scale software enterprise is given to demonstrate its effects on software process assessment and improvement. The research result is greatly helpful and can be widely used for effectively regulating and improving software process metrics, and promoting the mature level of the organization's process capability.

Recent studies of software defect prediction typically produce datasets, methods and frameworks which allow software engineers to focus on development activities in terms of defect-prone code, thereby improving software quality and making better use of resources. Many software defect prediction datasets, methods and frameworks are published disparate and complex, thus a comprehensive picture of the current state of defect prediction research that exists is missing. This literature review aims to identify and analyze the research trends, datasets, methods and frameworks used in software defect prediction research betweeen 2000 and 2013. Based on the defined inclusion and exclusion criteria, 71 software defect prediction studies published between January 2000 and December 2013 were remained and selected to be investigated further. This literature review has been undertaken as a systematic literature review. Systematic literature review is defined as a process of identifying, assessing, and interpreting all available research evidence with the purpose to provide answers for specific research questions. Analysis of the selected primary studies revealed that current software defect prediction research focuses on five topics and trends: estimation, association, classification, clustering and dataset analysis. The total distribution of defect prediction methods is as follows. 77.46% of the research studies are related to classification methods, 14.08% of the studies focused on estimation methods, and 1.41% of the studies concerned on clustering and association methods. In addition, 64.79% of the research studies used public datasets and 35.21% of the research studies used private datasets. Nineteen different methods have been applied to predict software defects. From the nineteen methods, seven most applied methods in software defect prediction are identified. Researchers proposed some techniques for improving the accuracy of machine learning classifier for software defect prediction by ensembling some machine learning methods, by using boosting algorithm, by adding feature selection and by using parameter optimization for some classifiers. The results of this research also identified three frameworks that are highly cited and therefore influential in the software defect prediction field. They are Menzies et al. Framework, Lessmann et al. Framework, and Song et al. Framework.

Software metrics are measures of software. Their primary use is to help us plan and predict software development. We can better control software quality and development effort if we can quantify the software. This is especially true in the early stages of development. Research has been done in the area of predicting software maintenance effort from software metrics. In general, software metrics can be classified into two categories: software product metrics and software process metrics. Software product metrics are measures of software products such as source code and design documents. Software process metrics are measures of software development process. For example, the size of the software is a measure of the software product itself, thus a product metrics. The effort required to design a software system may be influenced by how the software is designed, thus a process metrics. In this article, only software product metrics are addressed.

Ensuring software quality has been an issue since the very beginning of software industry. Now with the advancement in technology, tools and development cycles, new paradigms have emerged to tackle the challenges of software quality management. Software defect prediction (SDP) is one of those to help recognize the code blocks, classes, methods or files which contain the most bugs. It helps the programmer to prioritize those classes or methods for testing and bug fixing. It gives maximum coverage and hence reduces the cost of testing, bug fixing and saves time. This paper provides a way to discover challenges encountered during the SDP. We provide a comprehensive study of different defect prediction techniques, tools, datasets, algorithms, modelling tools and application areas of cross-project defect prediction, granularity levels used and SDP challenges.

Software dependency network metrics extracted from the dependency graph of the software modules by the application of Social Network Analysis (SNA metrics) have been shown to improve the performance of the Software Defect prediction (SDP) models. However, the relative effectiveness of these SNA metrics over code metrics in improving the performance of the SDP models has been widely debated with no clear consensus. Furthermore, some of the common SDP scenarios like predicting the number of defects in a module (Defect-count) in Cross-version and Cross-project SDP contexts remain unexplored. Such lack of clear directive on the effectiveness of SNA metrics when compared to the widely used code metrics prevents us from potentially building better performing SDP models. Therefore, through a case study of 9 open source software projects across 30 versions, we study the relative effectiveness of SNA metrics when compared to code metrics across 3 commonly used SDP contexts (Within-project, Cross-version and Cross-project) and scenarios (Defect-count, Defect-classification (classifying if a module is defective) and Effort-aware (ranking the defective modules w.r.t to the involved effort)). We find the SNA metrics by themselves or along with code metrics improve the performance of SDP models over just using code metrics on 5 out of the 9 studied SDP scenarios (three SDP scenarios across three SDP contexts). However, we note that in some cases the improvements afforded by considering SNA metrics over or alongside code metrics might only be marginal, whereas in other cases the improvements could be potentially large. Based on these findings we suggest that the future work should: consider SNA metrics alongside code metrics in their SDP models; as well as consider Ego metrics and Global metrics, the two different types of the SNA metrics separately when training SDP models as they behave differently.

Software defects and quality assurance are crucial aspects of software development that should be considered during the software development cycle. To ensure high-quality software, it is essential to have a robust quality assurance process in place. System reliability and quality are very key components that must be considered during software development, and this can only be achieved when software undergoes a thorough test process for errors, anomalies, defects, omissions, and bugs. Early software defect prediction and detection play an essential role in ensuring the reliability and quality of software systems, ensuring that software companies discover errors or defects early enough and allocate more resources to defect-prone modules. This study proposes the development of an enhanced classifier model for software defect prediction and detection. The aim is to harness the collective intelligence of selected base classifiers like Support Vector Machine, Logistic regression, Decision Trees, Random Forest, AdaBoost, Gradient Boosting, K-Nearest Neighbor, GaussianNB, and Multi-Layer Perception to improve accuracy, robustness, and generalization in identifying potential defects using a soft voting ensemble technique. The ensemble model leveraged the confidence probability of the soft voting technique and the generalization advantage of cross-validation leading to a more robust and dynamic model. The performance of the model with existing classifiers was evaluated using accuracy, F1 score, Precision, and area under the ROC curve (ROC- AUC) as the evaluation metrics. The results of the experiment revealed that the Proposed Classifier produced an overall Accuracy rate of 93%, and ROC AUC of 98%. The results demonstrate the effectiveness of our enhanced ensemble classifier in software defect detection and prediction. By harnessing the strengths of diverse base classifiers, our approach provides a robust and adaptive solution to the challenges of early detection and mitigating defects in software systems. This research contributes to the advancement of reliable software development practices and lays the foundation for future enhancements in ensemble-based defect detection methodologies.

BackgroundDevelopers spend more time fixing bugs refactoring the code to increase the maintainability than developing new features. Researchers investigated the code quality impact on fault-proneness, focusing on code smells and code metrics.ObjectiveWe aim at advancing fault-inducing commit prediction using different variables, such as SonarQube rules, product, process metrics, and adopting different techniques.MethodWe designed and conducted an empirical study among 29 Java projects analyzed with SonarQube and SZZ algorithm to identify fault-inducing and fault-fixing commits, computing different product and process metrics. Moreover, we investigated fault-proneness using different Machine and Deep Learning models.ResultsWe analyzed 58,125 commits containing 33,865 faults and infected by more than 174 SonarQube rules violated 1.8M times, on which 48 software product and process metrics were calculated. Results clearly identified a set of features that provided a highly accurate fault prediction (more than 95% AUC). Regarding the performance of the classifiers, Deep Learning provided a higher accuracy compared with Machine Learning models.ConclusionFuture works might investigate whether other static analysis tools, such as FindBugs or Checkstyle, can provide similar or different results. Moreover, researchers might consider the adoption of time series analysis and anomaly detection techniques.

Numerous studies have applied machine learning to the software defect prediction problem, i.e. predicting which modules will experience a failure during operation based on software metrics. However, skewness in defect-prediction datasets can mean that the resulting classifiers often predict the faulty (minority) class less accurately. This problem is well known in machine learning, and is often referred to as “learning from imbalanced datasets.” One common approach for mitigating skewness is to use stratification to homogenize class distributions; however, it is unclear what stratification techniques are most effective, both generally and specifically in software defect prediction. In this article, we investigate two major stratification alternatives (under-, and over-sampling) for software defect prediction using Analysis of Variance. Our analysis covers several modern software defect prediction datasets using a factorial design. We find that the main effect of under-sampling is significant at α = 0.05, as is the interaction between under- and over-sampling. However, the main effect of over-sampling is not significant.

Software Defect Prediction (SDP) plays an active area in many research domain of Software Quality of Assurance (SQA). Many existing research studies are based on software traditional metric sets and defect prediction models are built in machine language to detect the bug for limited source code line. Inspired by the above existing system. In this paper, defect prediction is focused on predicting defects in source code. The objective of this thesis is to improve the software quality for accurate defect prediction is source code for file level. So, that it helps the developer to find the bug and fix the issue, to make better use of a resource which reduces the test effort, minimize the cost and improve the quality of software. A new approach is introduced to improve the prediction performance of Bidirectional RNNLM in Deep Neural Network. To build the defect prediction model a defect learner framework is proposed and first it need to build a Neural Language Model. Using this Language Model it helps to learn to deep semantic features in source code and it train & test the model. Based on language model it combined with software traditional metric sets to measure the code and find the defect. The probability of language model and metric set Cross-Entropy with Abstract Syntax Tree (CE-AST) metric is used to evaluate the defect proneness and set as a metric label. For classification the metric label K-NN classifier is used. BPTT algorithm for learning RNN will provide additional improvement, it improves the predictions performance to find the dynamic error.

The world of software systems is amplified with the changing environment magnifying the demand for quality software. Software fault prediction is a requisite activity ensuring the development of economic, efficient and quality software. It is the procedure for the development of models which help to identify faults in modules during early phases of software development lifecycle. Software fault prediction is one of the most prevalent research disciplines. The existing study in this domain includes numerous modelling techniques and software metrics for the early predictions of software faults. This paper aims to explore some of the prominent studies for software fault prediction in the existing literature. In this paper, software fault prediction papers since 1990 to 2017 are investigated. The paper includes the analysis of the studies having empirical validation and a good source of publication. The paper reflects the methods, metrics, and datasets available in the literature for software fault prediction. In addition, the modelling techniques based on traditional and computational intelligence-based methods are also reviewed. This paper is an endeavour to assemble the existing techniques and metrics of software fault prediction with a motive to assist researchers for easy evaluation of suitable metrics for their own research scenarios.

Software defect prediction is an important software quality assurance technique. It utilizes historical project data and previously discovered defects to predict potential defects. However, most of existing methods assume that large amounts of labeled historical data are available for prediction, while in the early stage of the life cycle, projects may lack the data needed for building such predictors. In addition, most of existing techniques use static code metrics as predictors, while they omit change information that may introduce risks into software development. In this paper, we take these two issues into consideration, and propose a semi-supervised based defect prediction approach - extRF. extRF extends the classical supervised Random Forest algorithm by self-training paradigm. It also employs change burst information for improving accuracy of software defect prediction. We also conduct an experiment to evaluate extRF against three other supervised machine learners (i.e. Logistic Regression, Naïve Bayes, Random Forest) and compare the effectiveness of code metrics, change burst metrics, and a combination of them. Experimental results show that extRF trained with a small size of labeled dataset achieves comparable performance to some supervised learning approaches trained with a larger size of labeled dataset. When only 2% of Eclipse 2.0 data are used for training, extRF can achieve F-measure about 0:562, approximate to that of LR (a supervised learning approach) at labeled sampling rate of 50%. Besides, change burst metrics outperform code metrics in that F-measure rises to a peak value of 0:75 for Eclipse 3.0 and JDT.Core.

In order to improve software reliability, software defect prediction is applied to the process of software maintenance to identify potential bugs. Traditional methods of software defect prediction mainly focus on designing static code metrics, which are input into machine learning classifiers to predict defect probabilities of the code. However, the characteristics of these artificial metrics do not contain the syntactic structures and semantic information of programs. Such information is more significant than manual metrics and can provide a more accurate predictive model. In this paper, we propose a framework called defect prediction via attention-based recurrent neural network (DP-ARNN). More specifically, DP-ARNN first parses abstract syntax trees (ASTs) of programs and extracts them as vectors. Then it encodes vectors which are used as inputs of DP-ARNN by dictionary mapping and word embedding. After that, it can automatically learn syntactic and semantic features. Furthermore, it employs the attention mechanism to further generate significant features for accurate defect prediction. To validate our method, we choose seven open-source Java projects in Apache, using F1-measure and area under the curve (AUC) as evaluation criteria. The experimental results show that, in average, DP-ARNN improves the F1-measure by 14% and AUC by 7% compared with the state-of-the-art methods, respectively.

Development of next generation of smarter machines and services requires building an easy-to-use technology stack. These technologies include hardware, embedded software, data, and applications. Smarter machines support carrying out the precise job, better decision making, and create new ways of doing things to reduce cost and increase speed, accuracy, and automation. As technology meets iron through smart machines, there is increased embedded software within products. To provide distinctive customer experience for a solution system, along with reliability of hardware, development of reliable software also plays a critical role. This paper proposes models in which software reliability is a function of the number of residual faults and is measured with the help of software metrics based on development data. The intent is to establish a statistical relationship between product metrics (that deal with the measurement of the software product) or process metrics (the process by which it is developed) with measures of quality. Using both a pattern recognition algorithm approach for classifying fault proneness and applying fuzzy logic to software metrics for defect prediction is found to be beneficial for improving software reliability prediction in early development phases. This paper proposes non-parametric models like Artificial Neural Network from deep learning to predict expected number of failures utilizing past failure data for software reliability estimation and release readiness during launch.