Code Quality Risk Assessment (CQRA) for Mission Software

Early prediction of quality of service using interface-level metrics, code-level metrics, and antipatterns

Context: A large number of source code metrics are reported in the literature. It is necessary to systematically collect, describe and classify source code metrics to support research and practice. Objective: We aim to utilize existing secondary studies to develop a catalog of source code metrics together with their descriptions. The catalog will also provide information about which units of code (e.g., operators, operands, lines of code, variables, parameters, code blocks, or functions) are used to measure the internal quality attributes and the scope on which they are collected. Method: We conducted a tertiary study to identify secondary studies reporting source code metrics. We have classified the source code metrics according to the measured internal quality attributes, the units of code used in the measures, and the scope at which the source code metrics are collected. Results: From 711 secondary studies, we identified 52 relevant secondary studies. We reported 423 source code metrics together with their descriptions and the internal quality attributes they measure. Source code metrics predominantly incorporate function as a unit of code to measure internal quality attributes. In contrast, several source code metrics use more than one unit of code when measuring internal quality attributes. Nearly 51% of the source code metrics are collected at the class scope, while almost 12% and 15% of source code metrics are collected at module and application levels, respectively. Conclusions: Researchers and practitioners can use the extensive catalog to assess which source code metrics meet their individual needs based on the description and classification scheme presented.

Prediction of faults in a proposed software is helpful in deciding the amount of effort to be given for software development. We observed that, a good number of authors hypothesized that the performance of fault prediction model depends on the source code metrics which are used as input of the model. Feature selection technique is a process of selecting suitable set of source code metrics which may improve the performance of fault prediction model. In this work, genetic algorithm (GA) has been applied as feature selection technique to select the suitable set of source code metrics. This selected set of source code metrics are used as requisite input data to develop a classifier using five different classification techniques such as logistic regression, extreme learning machine, support vector machine (SVM) with three different kernel functions (linear, polynomial, and radial basis kernel functions) in order to predict the faulty and non-faulty classes. In this study, we propose a cost evaluation framework to perform cost based analysis for evaluating the effectiveness of fault prediction model. We perform experiments on thirty number of Java Open Source projects. From the obtained results, it is observed that the model developed using selected set of source code metrics obtained better result as compared to all metrics. From costs analysis framework, it is observed that the developed fault prediction model is best suitable for software with % of faulty classes less than the threshold value depending on fault identification efficiency (low-46.44%, median-45.37%, and high-36.63%).

Fault localization aims to support the debugging activities of human developers by highlighting the program elements that are suspected to be responsible for the observed failure. Spectrum Based Fault Localization (SBFL), an existing localization technique that only relies on the coverage and pass/fail results of executed test cases, has been widely studied but also criticized for the lack of precision and limited effort reduction. To overcome restrictions of techniques based purely on coverage, we extend SBFL with code and change metrics that have been studied in the context of defect prediction, such as size, age and code churn. Using suspiciousness values from existing SBFL formulas and these source code metrics as features, we apply two learn-to-rank techniques, Genetic Programming (GP) and linear rank Support Vector Machines (SVMs). We evaluate our approach with a ten-fold cross validation of method level fault localization, using 210 real world faults from the Defects4J repository. GP with additional source code metrics ranks the faulty method at the top for 106 faults, and within the top five for 173 faults. This is a significant improvement over the state-of-the-art SBFL formulas, the best of which can rank 49 and 127 faults at the top and within the top five, respectively.

A significant number of Free Software projects has been widely used and considered successful. However, there is an even larger number of them that cannot overcome the initial step towards building an active community of users and developers. In this study, we investigated whether there are relationships between source code metrics and attractiveness, i.e., the ability of a project to attract users and developers. To verify these relationships, we analyzed 6,773 Free Software projects from the SourceForge.net repository. The results indicated that attractiveness is indeed correlated to some source code metrics. This suggests that measurable attributes of the project source code somehow affect the decision to contribute and adopt a Free Software project. The findings described in this study show that it is relevant for project leaders to monitor source code quality, most specifically a few objective metrics, since these can have a positive influence in their chances of forming a community of contributors and users around the software, enabling further enhancement in its quality.

This paper presents an approach for the application of clustering algorithms to uncover computer programming learning profiles by using evidence extracted from source code metrics. A system for automatic assessment of programming activities featuring capture of source code metrics was developed, in order to build a dataset containing metrics extracted from programs developed by beginners in a computing course. The dataset was submitted to three clustering algorithms. The results were promising when clustering students according to these indicators using the K-means algorithm.

Maintenance is one of the most effort consuming activities in the software development lifecycle. Efficient maintenance within short release cycles depends highly on the underlying source code structure, in the sense that complex modules are more difficult to maintain. In this paper we attempt to unveil and discuss relationships between maintenance productivity, the structural quality of the source code and process metrics like the type of a release and the number of downloads. To achieve this goal, we developed a Bayesian Belief Network (BBN) involving several maintainability predictors and three managerial indices for maintenance (i.e., duration, production, and productivity) on 20 open source software projects. The results suggest that maintenance duration depends on inheritance, coupling, and process metrics. On the other hand maintenance production and productivity depend mostly on code quality metrics.

Web services are being embraced by IT industry in the recent past to enable rapid development of distributed systems with optimal cost. Web services in SOA are self-adaptable to context, which makes SOA widely recognized in IT system as the technology, which has the potential of improving the receptiveness of both business and IT organizations. Web services help in building Service Based Systems (SBS) like Paytm, Amazon, Paypal, e-bay etc. which evolves frequently to fit the new user requirements which impacts the evolvability and quality of software design. Similar to software systems built using other paradigms, Service based systems also suffer from bad or poor design choices as in anti-pattern, code smells etc. Anti-patterns are explicit structures in the design that indicates violation of fundamental design principles and negatively impact the design quality. Anti-patterns have obstructive influence on the maintainability and perception of software systems. Thus there is a rising need for the early prediction of anti-patterns and refactoring them to improve the software quality in terms of execution cost, maintenance cost and memory consumption. In this work, a frame work is proposed for significant feature selection from source code metrics which includes Wilcoxon signed rank test, Univariate logistic regression analysis and Cross-correlation analysis. Then the different sets of features from various steps along with the original source code metrics are considered and are used for anti-pattern detection using 13 machine learning algorithms. Experimental results show the approximation capability of different classifiers and data balancing techniques with the features selected from the various steps of feature validation framework in addition to the original features for developing anti-pattern prediction model. The results also shows that the prediction model built with by the ensemble techniques using the features obtained from proposed feature selection framework outperforms other techniques.KeywordsSoftware engineeringAnti-patternWeb serviceImbalanced dataService oriented architectureMachine learningPredictionSource code metric

In the field of software development, the application of code quality check tools has become a key factor in improving product quality and development efficiency. While many existing tools are effective at detecting common problems in code, there are still some limitations. Firstly, these tools rely on predefined rules that may not fully encompass real-world coding challenges. Secondly, a lack of consideration of dependencies leads to failure to report violations occurring across files or modules. Third, the metrics used by these tools primarily focus on object-oriented programming, limiting their ability to assess software quality from the perspective of nationalized standards. To address these issues, this work proposes a dependency-enhanced method namely ERD-CQC for code quality detection and measurement. ERD-CQC provides 88 detection rules and 45 metrics, supplementing checking rules in categories such as Circuit Breaking, Serializable, and Security. ERD-CQC constructs an infused graph by integrating abstract syntax trees (ASTs), entities, and dependencies for violation detection. Based on the detection results, ERD-CQC provides a code quality measurement system with 4 nationalized standard dimensions for the purpose of measuring code quality from multiple perspectives. To validate the effectiveness of ERD-CQC, we manually examined 647 compliant and 528 non-compliant code snippets. ERD-CQC achieves the recall and F1 score exceeding 98%. We also collected open-source projects and closed-source projects in the real world, containing a total of 4,319 non-compliant code snippets. On this real-world benchmark, the average F1 score of ERD-CQC is 11.44% higher than the advanced tool SonarQube. Finally, we visualized the quality measurement results based on metrics and found that open-source and closed-source projects have certain patterns in metric performance. Our work will benefit developers in checking, evaluating, and monitoring their software quality comprehensively.

Most code-based quality measurement approaches are based, at least partially, on values of multiple source code metrics. A class will often be classified as being of poor quality if the values of its metrics are above given thresholds, which are different from one metric to another. The metrics thresholds are calculated using various techniques. In this paper, we investigated two specific techniques: ROC curves and Alves rankings. These techniques are supposed to give metrics thresholds which are practical for code quality measurements or even for fault-proneness prediction. However, Alves Rankings technique has not been validated as being a good choice for fault-proneness prediction, and ROC curves only partially on few datasets. Fault-proneness prediction is an important field of software engineering, as it can be used by developers and testers as a test effort indication to prioritize tests. This will allow a better allocation of resources, reducing therefore testing time and costs, and an improvement of the effectiveness of testing by testing more intensively the components that are likely more fault-prone. In this paper, we wanted to compare empirically the selected threshold calculation methods used as part of fault-proneness prediction techniques. We also used a machine learning technique (Bayes Network) as a baseline for comparison. Thresholds have been calculated for different object-oriented metrics using four different datasets obtained from the PROMISE Repository and another one based on the Eclipse project.

In spite of the significant body of research on traditional source code metrics, there has been a general failure to produce conclusive evidence as to their effectiveness for measuring software quality. We describe and recommend a potentially much more powerful and sensitive quality assessment alternative, software quality guidelines. Software quality guidelines are presented as “designer metrics”, that is, user-defined rules or constraints relating to measurable features of a program's structure, semantics, and syntax that affect its quality. To provide a methodology for designing, applying, and validating software quality guidelines, we recommend and briefly summarize IEEE standard 1061. This standard gives a process for constructing and implementing a software quality metrics framework that can be tailor-made to meet quality requirements for a particular project and/or organization. Our paper then demonstrates how software quality guidelines fit within the IEEE framework and gives an example illustrating how user-defined guidelines can be applied to evaluate or assess the quality of an Ada source unit. This guideline-based assessment of quality is then compared with an analysis based on traditional McCabe and Halstead metrics. Finally, we introduce a tool being developed by SAIC, called AdaReVu, as an effective mechanism for implementing and applying user-defined quality guidelines for Ada source code.

Due to the constant evolution of technology, each day brings new programming languages, development paradigms, and ways of evaluating processes. This is no different with source code metrics, where there is always new metric classes. To use a software metric to support decisions, it is necessary to understand how to perform the metric collection, calculation, interpretation, and analysis. The tasks of collecting and calculating source code metrics are most often automated, but how should we monitor them during the software development cycle? Our research aims to assist the software engineer to monitor metrics of vulnerability threats present in the source code through a reference prediction model, considering that real world software have non-functional security requirements, which implies the need to know how to monitor these requirements during the software development cycle. As a first result, this paper presents an empirical study on the evolution of the Linux project. Based on static analysis data, we propose low complexity models to study flaws in the Linux source code. About 391 versions of the project were analyzed by mining the official Linux repository using an approach that can be reproduced to perform similar studies. Our results show that it is possible to predict the number of warnings triggered by a static analyzer for a given software project revision as long as the software is continuously monitored.

In the conventional techniques, it requires prior knowledge of faults or a special structure, which may not be realistic in practice while detecting the software faults. To deal with this problem, in this work, the proposed approach aims to predict the faults of the software utilizing the source code metrics. In addition, the purpose of this paper is to measure the capability of the software fault predictability in terms of accuracy, f-measure, precision, recall, Area Under ROC (Receiver Operating Characteristic) Curve (AUC). The study investigates the effect of the feature selection techniques for software fault prediction. As an experimental analysis, our proposed approach is validated from four publicly available datasets. The result predicted from Random Forest technique outperforms the other machine learning techniques in most of the cases. The effect of the feature selection techniques has increased the performance in few cases, however, in the maximum cases it is negligible or even the worse.

Software use is ubiquitous. The quality and the evolution of quality over long periods of time is therefore of notable importance. Software engineering research investigates software quality in multiple areas. One of these areas are predictive models, in which measurements of past changes to the source code or file contents are used to assess the quality of changes, files or even product releases. However, these predictive models have yet to transition from research to practice on a larger scale. In contrast, Automated Static Analysis Tools (ASATs) are used in practice and are also part of several software quality models. ASATs are able to warn developers about parts of the source code that violate best practices or match common defect patterns. One downside of ASATs are false positives, i.e., warnings about parts of the code which are not problematic. Developers have to manually assess the warnings and annotate the code or the ASAT configuration to mitigate this. Within this thesis, we investigate the evolution of software quality with a focus on a general purpose ASAT for Java. Our main objective is to determine if the use of an ASAT can improve software quality, as measured by defects, significantly enough to mitigate additional effort by the developers to use the ASAT. We combine multiple software engineering research techniques and data validation studies to improve the signal-to-noise ratio to increase the validity and stability of our results. We focus on a general purpose ASAT for the Java programming language due to the maturity of the language and the large number of projects available for this language. Both the language and the general purpose ASAT have been available for a long time, which allows us to include longer periods of time for our analyses. We study how the ASAT is applied, how the generated warnings evolve over long time periods, and how it affects the quality of the source code in terms of defects. In addition, we include the perspective of the developers regarding software quality improvement by measuring changes when developers intend to improve the quality of the source code. Our studies yield surprising insights. While our results show that ASATs have a positive impact on software quality, the magnitude of the impact is much smaller than expected. Moreover, we can show that corrective changes are the main driver of complexity in software projects. They introduce more complexity than feature additions or any other type of maintenance. In addition, we find that software quality estimation models benefit more from size and complexity metrics than static analysis warnings of an ASAT. Our study of developer intents to increase software quality mirrors this result.

The software industry's Development Lifecycle is changing dramatically, with traditional, heavy-weight processes giving way to simple, light-weight Agile software development (ASD) methods. ASD develops outstanding, high-quality software in short, rapid iterations that is responsive to changing business conditions. Adaptive planning, gradual design and deployment, and a time-bound strategy are all encouraged. Agile promotes a quick and flexible response to change. Agile systems have the ability to improve customer satisfaction, lower bug rates, shorten cycle times, and enable faster adaption to quickly changing company needs. Agile helps to deliver early but might compromise on the quality of software delivered. This research paper presents the performance analysis of Agile Scrum methodology towards Time to Market and Code Quality based on various project parameters like time to deliver the product, number of bugs and code smells and amount of duplicate code.