Software quality metrics in the automatic evaluation of Python introductory programming

Aims: Software quality assurance is a formal process for evaluating and documenting the quality of the work products during each stage of the software development lifecycle.The practice of applying software metrics to operational factors and to maintain factors is a complex task.Successful software quality assurance is highly dependent on software metrics.It needs linkage the software quality model and software metrics through quality factors in order to offer measure method for software quality assurance.The contributions of this paper build an appropriate method of Software quality metrics application in quality life cycle with software quality assurance.Design: The purpose approach defines some software metrics in the factors and discussed several software quality assurance model and some quality factors measure method.Methodology: This paper solves customer value evaluation problem are: Build a framework of combination of software quality criteria.Describes software metrics.Build Software quality metrics application in quality life cycle with software quality assurance.Results: From the appropriate method of Software quality metrics application in quality life cycle with software quality assurance, each activity in the software life cycle, there is one or more QA quality measure metrics focus on ensuring the quality of the process and the resulting product.Future research is need to extend and improve the methodology to extend metrics that have been validated on one project, using our criteria, valid measures of quality on future software project.

Although many improvements for software development are proposed by software engineers, the embedded system community faces a hard task in applying these improvements to software development, due to the strong dependence between software and hardware in embedded systems, which raises a trade-off between software quality, measured by traditional metrics, and optimization for a specific platform. Traditional software quality concerns reuse, abstraction, coupling, and coherence. Such concepts are usually in conflict with performance, memory footprint, and other physical metrics used for design evaluation in embedded systems. The purpose of this work is to evaluate the relationship between quality metrics for software products and physical metrics for embedded systems, in order to guide an embedded designer in selecting the best alternative design during design space exploration already at the model level. After the experiments, we could analyze the correlation between these metrics and highlight the behavior for quality and physical metrics, which help us to better understand the trade-off between reuse and optimization.

Software quality indicates how effective and efficient a software is. Various standards need to be used to evaluate software quality. One of the most important internationally accepted standards for measuring software quality is the ISO/IEC 25010 software quality standard. With this standard, the quality of a software product is evaluated with eight different metrics. These are functional suitability, performance, compatibility, usability, reliability, security, maintainability and portability metrics. In this study, we tried to determine the relationship between user feedback and the metrics of the ISO/IEC 25010 software quality standard. Machine learning (ML) and natural language processing (NLP) techniques were used to classify user comments. After the data preprocessing phase, vectors of user comments were extracted with the Tf-Idf method for NLP. As a machine learning method, classification was made using five different models: Extra Trees Classifier (ETC), Gaussian Process Classifier (GPC), MLP Classifier (MLPC), Bernoulli Naive Bayes Classifier (BNBC) and Support Vector Classifier (SVC). Our goal is to show how quality metrics can be classified into multiple classes using user notifications. The data set used has an unbalanced structure, containing 1681 user comments classified by software experts. Synthetic Minority Oversampling Technique (SMOTE) was used to address the imbalance in the dataset. The results were compared by applying the same classification models to unbalanced and balanced data sets. According to the results obtained, the best classification model is the Extra Trees Classifier model, which provides the highest accuracy rate of 87% according to the SMOTE applied data set. The results show that ML and NLP methods can be used effectively in the classification process of software quality metrics.

Leveraging and acting on the results of static analysis is a well-known best practice. Static analysis also is an integral part of software quality assurance which is highlighted by the inclusion of static analysis results in software quality models like ColumbusQM and Quamoco. Although there are studies that explore if tools for static analysis are used and how they are configured, few publications explore the longitudinal effects of acting on static analysis results on software and on the evolution of software. Especially effects on quality criteria, e.g., software quality metrics, defects, or readability are missing. With our research, we will bridge this gap and measure the effects of static analysis on software quality evolution. We will measure the effect the removal of code that generates static analysis warnings has on software quality metrics. Furthermore, we will measure long term effects on external quality attributes, e.g., reported issues and defects. Finally, we want to predict false positives of static analysis warnings by training predictive models on our collected data.

Source code refactorings and transformations are extensively used by embedded system developers to improve the quality of applications, often supported by various open source and proprietary tools. They either aim at improving the design time quality such as the maintainability and reusability of software artifacts, or the runtime quality such as performance and energy efficiency. However, an inherent trade-off between design- and run-time qualities is often present posing challenges to embedded software development. This work is a first step towards the investigation of the impact of transformations for improving the performance and the energy efficiency on software quality metrics and the impact of refactorings for increasing the design time quality on the execution time, the memory and the energy consumption. Based on a set of embedded applications from widely used benchmark suites and typical transformations and refactorings, we identify interrelations and trade-offs between the aforementioned metrics.

Virtual Reality (VR) Software is becoming more mainstream in recent years. It has provided an opportunity for VR practitioners to explore new domains and deliver cutting edge products. The success of the VR products depends primarily on the product contextual relevance and qualities exhibited. However, it is unclear how VR practitioners curb software quality challenges and improve the essence of the VR product over every release. In this paper, we present a Systematic Mapping Study of the software quality metrics adopted by VR practitioners for assessing the quality of their VR products. The study showed that practitioners used unique metrics to measure the quality of their VR products in addition to adopting some of existing enterprise software metrics. Further, we consolidate these metrics into different themes that future practitioners may use for developing VR products.

SUMMARYThis research paper presents a framework for ranking of software engineering metrics based on expert opinion elicitation and fuzzy‐based matrix methodology. The proposed methodology is able to accommodate the imprecise and inexact data involved in the problem of ranking of software engineering metrics, vagueness and ambiguity occurring during expert (human) decision making and to depart from the complexity of formulation of the objective and the constraint function. The matrices lend themselves to mechanical manipulations and are useful for analyzing and deriving systems functions expeditiously to meet the objectives. The current research is based on software engineering metrics identified in an earlier study conducted by Lawrence Livermore National Laboratory. A set of ranking criteria were identified. Software engineering metrics are then ranked in ascending order using experts' opinion in accordance with the value of Permanent function on their criteria matrix. The proposed methodology has also been compared with other known methodologies. Copyright © 2011 John Wiley & Sons, Ltd.

In spite of the significant body of research on traditional source code metrics, there has been a general failure to produce conclusive evidence as to their effectiveness for measuring software quality. We describe and recommend a potentially much more powerful and sensitive quality assessment alternative, software quality guidelines. Software quality guidelines are presented as “designer metrics”, that is, user-defined rules or constraints relating to measurable features of a program's structure, semantics, and syntax that affect its quality. To provide a methodology for designing, applying, and validating software quality guidelines, we recommend and briefly summarize IEEE standard 1061. This standard gives a process for constructing and implementing a software quality metrics framework that can be tailor-made to meet quality requirements for a particular project and/or organization. Our paper then demonstrates how software quality guidelines fit within the IEEE framework and gives an example illustrating how user-defined guidelines can be applied to evaluate or assess the quality of an Ada source unit. This guideline-based assessment of quality is then compared with an analysis based on traditional McCabe and Halstead metrics. Finally, we introduce a tool being developed by SAIC, called AdaReVu, as an effective mechanism for implementing and applying user-defined quality guidelines for Ada source code.

: Software metrics (or measurements) which predict software quality have been refined and enhanced. Metrics were classified as anomaly-detecting metrics which identify deficiencies in documentation or source code, predictive metrics which measure the logic of the design and implementation, and acceptance metrics which are applied to the end product to assess compliance with requirements. A Software Quality Measurement Manual was produced which contained procedures and guidelines for assisting software system developers in setting quality goals, applying metrics and making quality assessments.

Software is a product in serious need of quality control technology. Major effort notwithstanding, software engineering has produced few metrics for aspects of software quality that have the potential of being universally applicable. The present paper suggests that, although universal metrics are elusive, metrics that are applicable and useful in a fully defined setting are readily available. A theory is presented that a well‐defined software work group can articulate their operational concept of quality and derive useful metrics for that concept and their environment.

Developers are extracted from 17 open-source projects from GitHub. Projects are chosen that use the java programming language, the Spring framework and Maven/Gradle build tools. Along with these developers, 24 software engineering metrics are extracted for each of them. These metrics are either calculated by analyzing the source code or relative to project management metadata. Each of these developers then are manually searched for in professional social media such as LinkedIn or Twitter to be labeled with their experience level in their project. Outliers are statistically detected and manually re-assigned when needed. The resulting dataset contains 703 anonymized developers qualified by their 24 project-related software engineering metrics and labeled for their experience. It is suitable for empirical software engineering studies that need to connect developers’ level of experience to tangible software engineering metrics.

During the last two decades research on software engineering is concentrated on quality. The best approach to quality evaluation goes through determining well-defined metrics on software properties. One such property is module complexity, which is a view of the software that is related to how easily it can be modified. There has been work on constructing a metrics domain which measures the module complexity. Generally, PCA (Principal Component Analysis) is used for defining principal metrics in the domain. Since there are usually no labels for the software data, an unsupervised dimensionality reduction technique, such as PCA needs to be used for determining the most important metrics. In this study, we use the clustering similarity obtained when a certain subset of metrics and when the whole set of metrics are used, to determine the most important metrics. We measure the relative difference/similarity between clusterings using three different indices, namely Rand, Jaccard and Fowlkes-Mallow. We use both backward feature selection and PCA for dimensionality reduction. On the publicly available NASA data, we find out that instead of the whole set of 42 metrics, using only 15 dimensions, we get almost the same clustering performance. Therefore, instead of the whole set of software metrics, a smaller number of them could be used to evaluate the software quality.

: The Rome Air Development Center has developed the Software Quality Framework as a means to specify software quality goals and measure software quality. Much of the work to date has focused on metrics applicable to software code. This report describes an effort undertaken to measure the quality of software products earlier in the software development life cycle, during the design phase, and to automate the capture of metric data from design media. Metrics of software quality, primarily those related to the criterion simplicity (or conversely, complexity), were reviewed. This review includes those metrics previously developed in the Software Quality Framework. Two metrics, Halstead's Software Science and McCabe's Cyclomatic Complexity were chosen for their amenability to measurement during design and their potential for automation. Two design media were used: Design Aids for Real-Time Systems, an experimental automated design tool developed at the Charles Stark Draper Laboratory; and Ada as a program design language.

One of the first steps towards the effective Technical Debt (TD) management is the quantification and continuous monitoring of the TD principal. In the current state-ofresearch and practice the most common ways to assess TD principal are the use of: (a) structural proxies—i.e., most commonly through quality metrics; and (b) monetized proxies—i.e., most commonly through the use of the SQALE (Software Quality Assessment based on Lifecycle Expectations) method. Although both approaches have merit, they seem to rely on different viewpoints of TD and their levels of agreement have not been evaluated so far. Therefore, in this paper, we empirically explore this relation by analyzing data obtained from 20 open source software projects and build a regression model that establishes a relationship between them. The results of the study suggest that a model of seven structural metrics, quantifying different aspects of quality (i.e., coupling, cohesion, complexity, size, and inheritance) can accurately estimate TD principal as appraised by SonarQube. The results of this case study are useful to both academia and industry. In particular, academia can gain knowledge on: (a) the reliability and agreement of TD principal assessment methods and (b) the structural characteristics of software that contribute to the accumulation of TD, whereas practitioners are provided with an alternative evaluation model with reduced number of parameters that can accurately assess TD, through traditional software quality metrics and tools.

Automatic evaluation of source code can be an important mechanism for learning programming languages. Several tools and systems perform automatic evaluation of source code through the use of input/output tests, but very few make use of software engineering metrics. Considering the limitations of current work, we present in this paper a 4-stage approach for the automatic evaluation of source code: i) the source code is compiled and an error-check is performed; ii) the compiled code is tested against a set of JUnit (see IV-A) tests provided by the teaching staff; iii) a set of software engineering metrics are used to compare the student's solution with the teacher's solution; iv) and finally, based on the results of the previous stages, automatic feedback is provided to the student in order for them to identify the areas in which they need further study. At the end of the paper, we present the results of the system's evaluation that show that our approach can facilitate the learning and knowledge acquisition processes of the student while learning a programming language.