Smart prediction for refactorings in the software test code

Software quality depends not only on production code but also on the quality of test code. Among the factors that compromise this quality, test smells stand out, poor implementation patterns in tests that negatively affect the readability, maintainability, and reliability of the test suite. Although tools such as tsDetect and JNose support test smell detection, they operate post-hoc, requiring the code to be complete and analyzed outside the development environment. Other tools, like RAIDE, offer detection within the IDE via plugins; however, the analysis is not instantaneous and still occurs after the code is written. These reactive approaches result in delayed feedback and significantly increase the refactoring effort. In this context, we propose AriesLinter, a static analysis tool that can be integrated into major development environments and is capable of detecting test smells in real time, as the test code is being written. Built on top of the Checkstyle infrastructure, AriesLinter automatically detects 10 of the most recurrent test smells reported in the literature and commonly found in Java projects. The tool is cross-platform and compatible with multiple IDEs. By anticipating the detection of poor patterns and embedding it into the natural workflow of developers, AriesLinter acts proactively, reducing the likelihood of test smells being introduced into the codebase and fostering a culture of continuous attention to test quality. Demo video: https://youtu.be/Cj6mQGGHlvk

Bug reports are submitted by the software stakeholders to foster the location and elimination of bugs. However, in large-scale software systems, it may be impossible to track and solve every bug, and thus developers should pay more attention to High-Impact Bugs (HIBs). Previous studies analyzed textual descriptions to automatically identify HIBs, but they ignored the quality of code, which may also indicate the cause of HIBs. To address this issue, we integrate the features reflecting the quality of production (i.e. CK metrics) and test code (i.e. test smells) into our textual similarity based model to identify HIBs. Our model outperforms the compared baseline by up to 39% in terms of AUC-ROC and 64% in terms of F-Measure. Then, we explain the behavior of our model by using SHAP to calculate the importance of each feature, and we apply case studies to empirically demonstrate the relationship between the most important features and HIB. The results show that several test smells (e.g. Assertion Roulette, Conditional Test Logic, Duplicate Assert, Sleepy Test) and product metrics (e.g. NOC, LCC, PF, and ProF) have important contributions to HIB identification.

Tests are essential for ensuring code quality in software development. However, poor implementation practices can compromise the maintainability and evolution of test code, leading to additional costs and effort. These practices, known as test smells, have been the focus of numerous studies that expanded the catalog of known test smells. Despite this progress, little attention has been given to examining test smells from the perspective of programming languages, particularly their relationship with specific languages or test frameworks. As a result, researchers and practitioners have proposed various approaches and tools to detect and refactor the same test smells across different languages. In this work, we address this gap by analyzing test smells in the literature from a programming language perspective. We conducted a systematic mapping study, gathering data from papers published up to August 2025, examining 117 papers. We identified the most prevalent languages and frameworks associated with test smells, the most common test smells in each language, refactorings for addressing test smells, datasets of test smells, the criticality of specific test smells, and a collection of tools to detect or refactor test smells for various languages and frameworks. Our results show that while many test smells have been addressed, research remains limited to a few languages and frameworks. We also found datasets and detection/refactoring tools for some languages and analyzed the criticality of frequently mentioned test smells. This study highlights the need to expand research to more languages and develop more universal solutions.

Unit testing represents a key activity in software development and maintenance. Test suites with high internal quality facilitate maintenance activities, such as code comprehension and regression testing. Several guidelines have been proposed to help developers write good test suites. Unfortunately, such rules are not always followed resulting in the presence of bad test code smells (or simply test smells). Test smells have been defined as poorly designed tests and their presence may negatively affect the maintainability of test suites and production code. Despite the many studies that address code smells in general, until now there has been no empirical evidence regarding test smells (i) distribution in software systems nor (ii) their impact on the maintainability of software systems. This paper fills this gap by presenting two empirical studies. The first study is an exploratory analysis of 18 software systems (two industrial and 16 open source) aimed at analyzing the distribution of test smells in source code. The second study, a controlled experiment involving twenty master students, is aimed at analyzing whether the presence of test smells affects the comprehension of source code during software maintenance. The results show that (i) test smells are widely spread throughout the software systems studied and (ii) most of the test smells have a strong negative impact on the comprehensibility of test suites and production code.

Test smells are symptoms of sub-optimal design choices adopted when developing test cases. Previous studies have proved their harmfulness for test code maintainability and effectiveness. Therefore, researchers have been proposing automated, heuristic-based techniques to detect them. However, the performance of these detectors is still limited and dependent on tunable thresholds. We design and experiment with a novel test smell detection approach based on machine learning to detect four test smells. First, we develop the largest dataset of manually-validated test smells to enable experimentation. Afterward, we train six machine learners and assess their capabilities in within- and cross-project scenarios. Finally, we compare the ML-based approach with state-of-the-art heuristic-based techniques. The key findings of the study report a negative result. The performance of the machine learning-based detector is significantly better than heuristic-based techniques, but none of the learners able to overcome an average F-Measure of 51%. We further elaborate and discuss the reasons behind this negative result through a qualitative investigation into the current issues and challenges that prevent the appropriate detection of test smells, which allowed us to catalog the next steps that the research community may pursue to improve test smell detection techniques.

A multi-label classification approach for detecting test smells over java projects

Software metrics are created and used by the distinctive programming associations intended for assessing, guaranteeing program excellence, activity, and software recovery. Software metrics have turned into a basic part of programming growth and are utilized in each period of the product development life cycle. Software metrics essentially measure programming items like plan source code and help us in taking technical and administrative choices. The desire of this examination is to play out the relative investigation of static and dynamic metrics. In any case, software quality characteristics, for example, performance, execution time and dependability rely upon the dynamic exercises of the product artifact. Due to every one of these variables, we favor dynamic metrics instead of customary static metrics. With the assistance of customary static metrics, we are not capable to analyze different actualities of programming. There are various types of this OO static and dynamic equipments. In this paper we have played out a similar investigation of different OO static and dynamic metrics tools and find out the hybrid too is counted as best one extraction of both, static and dynamic characteristics from mobile Android applications. The source code and a Docker compartment is utilized by open source tool in only three phases pre-static, static and dynamic examination.

Test smells are sub-optimal design choices in the implementation of test code. As reported by recent studies, their presence might not only negatively affect the comprehension of test suites but can also lead to test cases being less effective in finding bugs in production code. Although significant steps toward understanding test smells, there is still a notable absence of studies assessing their association with software quality.<br/>In this paper, we investigate the relationship between the presence of test smells and the change- and defect-proneness of test code, as well as the defect-proneness of the tested production code. To this aim, we collect data on 221 releases of ten software systems and we analyze more than a million test cases to investigate the association of six test smells and their co-occurrence with software quality. Key results of our study include:(i) tests with smells are more change- and defect-prone, (ii) ‘Indirect Testing’, ‘Eager Test’, and ‘Assertion Roulette’ are the most significant smells for change-proneness and, (iii) production code is more defect-prone when tested by smelly tests.

Summary form only given. Software metrics have become an integral part of software development and are used during every phase of the software development life cycle. Research in the area of software metrics tends to focus predominantly on static metrics that are obtained by static analysis of the software artifact. But software quality attributes such as performance and reliability depend on the dynamic behavior of the software artifact. Estimating software quality attributes based on dynamic metrics for the software system are more accurate and realistic. The research presented in this paper attempts to narrow the gap between static metrics and dynamic metrics, and lay the foundation for a more systematic approach to estimate the dynamic behavior of a software system early in the software development cycle. Focusing on coupling metrics, we present an empirical study to analyze the relationship between static and dynamic coupling metrics and propose the concept of pseudo dynamic metrics to estimate the dynamic behavior early in the software development lifecycle.

Software coupling is considered one of the most crucial issues in the software industry. The measurement of coupling gives a quantitative indication of internal features that are correlated to external quality attributes such as reliability, maintainability, extendibility, functionality and usability of the software. Various static and dynamic metrics have been proposed by several academicians and industry experts to evaluate software coupling. Although the values of static metrics can be easily extracted from the source code without its execution, researchers have emphasized the significance of dynamic metrics over static metrics in terms of program comprehensibility, change proneness, maintainability, etc. This chapter presents some of the dynamic coupling measures and their comparative analysis: (a) parametric-based analysis of six parameters, i.e. quality focus, measured entity, theoretical validation, dynamic analysis approach, environment, and statistical analysis technique; (b) functional-based analysis. Only those metrics which possess the required amount of information have been considered in order to make a meaningful comparison possible. Based on the comparison, the observed results reveal that empirical validation conducted for the evaluation of dynamic coupling metrics is not adequate to make them applicable to industries.

Test smells are poorly designed tests and negatively affect the quality of test suites and production code. We present the largest catalog of test smells, along with a summary of guidelines, techniques, and tools used to deal with test smells.

Software metrics help us to make meaningful estimates for software products and guide us in taking managerial and technical decisions like budget planning, cost estimation, quality assurance testing, software debugging, software performance optimization, and optimal personnel task assignments. Many design metrics have proposed in literature to measure various constructs of Object Oriented (OO) paradigm such as class, coupling, cohesion, inheritance, information hiding and polymorphism and use them further in determining the various aspects of software quality. However, the use of conventional static metrics have found to be inadequate for modern OO software due to the presence of run time polymorphism, templates class, template methods, dynamic binding and some code left unexecuted due to specific input conditions. This gap gave a cue to focus on the use of dynamic metrics instead of traditional static metrics to capture the software characteristics and further deploy them for maintainability predictions. As the dynamic metrics are more precise in capturing the execution behavior of the software system, in the current empirical investigation with the use of open source code, we validate and verify the superiority of dynamic metrics over static metrics. Four machine learning models are used for making the prediction model while training is performed simultaneously using static as well as dynamic metric suite. The results are analyzed using prevalent prediction accuracy measures which indicate that predictive capability of dynamic metrics is more concise than static metrics irrespective of any machine learning prediction model. Results of this would be helpful to practitioners as they can use the dynamic metrics in maintainability prediction in order to achieve precise planning of resource allocation.

Automated testing (hereafter referred to as just `testing') has become an essential process for improving the quality of software systems. In fact, testing can help to point out defects and to ensure that production code is robust under many usage conditions. However, writing and maintaining high-quality test code is challenging and frequently considered of secondary importance. Managers, as well as developers, do not treat test code as equally important as production code, and this behaviour could lead to poor test code quality, and in the future to defect-prone production code. The goal of my research is to bring awareness to developers on the effect of poor testing, as well as helping them in writing better test code. To this aim, I am working on 2 different perspectives: (1) studying best practices on software testing, identifying problems and challenges of current approaches, and (2) building new tools that better support the writing of test code, that tackle the issues we discovered with previous studies. Pre-print: https://doi.org/10.5281/zenodo.1411241

Several important aspects of software product quality can be evaluated using dynamic metrics that effectively capture and reflect the software's true runtime behavior. While the extent of research in this field is still relatively limited, particularly when compared to research on static metrics, the field is growing, given the inherent advantages of dynamic metrics. The aim of this work is to systematically investigate the body of research on dynamic software metrics to identify issues associated with their selection, design and implementation. Mapping studies are being increasingly used in software engineering to characterize an emerging body of research and to identify gaps in the field under investigation. In this study we identified and evaluated 60 works based on a set of defined selection criteria. These studies were further classified and analyzed to identify their relativity to future dynamic metrics research. The classification was based on three different facets: research focus, research type and contribution type. We found a strong body of research related to dynamic coupling and cohesion metrics, with most works also addressing the abstract notion of software complexity. Specific opportunities for future work relate to a much broader range of quality dimensions.

This paper reports on the feasibility of using eye tracking as a tool for students’ performance assessment in a medical serious game. We are particularly interested in analyzing the relationship between learners’ visual behaviour and their performance while solving medical cases. The objective of this study is twofold. First, we analyze how the students visually explore the learning environment across different areas of interest. Second, we examine whether static and dynamic eye tracking metrics can have an impact on students’ reasoning performance. Results revealed statistically significant associations between eye movement metrics and students’ outcomes. Particularly dynamic metrics better reflected students’ analytical reasoning abilities. Our findings have implications for the educational technology community seeking to gain a deeper understanding of the students’ learning experience.