Investigation on Java Mutation Testing Tools

Mutation testing is the fault-based software testing approach that is widely applicable for assessing the effectiveness of a test suite. The test suite effectiveness is measured through artificial seeding of faults into the programs under test. Mutation testing tools (MTTs) are significant enablers of the conversion of mutation testing from the research perspective into the real life and mostly applicable testing process. Without using the automatic MTT, mutation testing can’t be really connected in this present reality and is unrealistic to be acknowledged by the industry. Authors analyze six open-source JAVA-based MTT (Jester, JavaMut, MuJava, Jumble, Judy, and Javalanche). The results are based on the performance of various JAVA programs and two real-life applications. Benchmark comparison among the MTT is presented in terms of mutants, mutation operator, mutation score, and quality output. On the basis of comparative analysis, the performance of each tool is explained with the protocol for finding the appropriate tool among the six MTTs. The results show that the MuJava performs best compared to the others.KeywordsMutation testingFault-based testingAdequacy criterionMutation testing tools

Mutation testing has been used to evaluate the quality of the test set and provides the confidence in testing activity. Generation as well as running of mutants needs practice. A huge amount of mutants are generated and therefore, it is a very costly activity in terms of time and effort. Automatic mutant generation and execution is essential to support testing. Automation achieves more attention, saves time and effort as developers and testers use automated tools. In the recent years, researchers have found that practitioners have diminutive knowledge about tools and their effectiveness. This paper compares five well-known publicly accessible mutation testing tools: ‘MuClipse’, ‘Judy’, ‘Jumble’, ‘Jester’ and PIT. This comparison uses a collection of Java classes taken from various easily accessible sources. Tests were designed and generated with the help of test generation techniques. The performance of each mutation tool was analyzed and was compared based on their mutation operators, mutation score and execution time.

A test suite plays a key role in software testing. Mutation testing is a powerful approach to measure the fault-detection ability of a test suite. The mutation testing process requires a large number of mutants to be generated and executed. Hence, mutation testing is also computationally expensive. To solve this problem, predictive mutation testing builds a classification model to predict the test result of each mutant. However, the existing predictive mutation testing methods only can be used to estimate the overall mutation scores of object-oriented programs. To overcome the shortcomings of the existing methods, we propose a new method to directly predict the mutation score for each statement in process-oriented programs. Compared with the existing predictive mutation testing methods, our method uses more dynamic program execution features, which more adequately reflect dynamic dependency relationships among the statements and more accurately reflects information propagation during the execution of test cases. By comparing the prediction effects of logistic regression, artificial neural network, random forest, support vector machine, and symbolic regression, we finally decide to use a single hidden layer feedforward neural network as the predictive model to predict the statement mutation scores. In our two experiments, the mean absolute errors between the statement mutation scores predicted by the neural network and the real statement mutation scores both approximately reach 0.12.

SummaryFor mutation testing, the huge cost of running test suites on a large number of mutants has been a serious obstacle. To resolve this problem, we propose a learning‐based mutant reduction techniqueMuTrain.MuTrainuses cost‐considerate linear regression (i.e., CLARS) to learn amutation model, which predicts the mutation score of a test suite based on the mutation testing results of a previous version of a target program. Then,MuTrainapplies the mutation model for subsequent versions to predict mutation scores with significantly fewer mutants. For effective mutant reduction and accurate mutation score prediction,MuTrainuses fine‐grained mutation operators refined from the existing coarse‐grained mutation operators. The experiment results show thatMuTrainreduces the number of mutants effectively (i.e., selecting only 1.6% of mutants). Moreover,MuTrainpredicts mutation score far more accurately than the existing mutant reduction techniques and random mutant selection. We also found thatMuTrainachieves much greater mutant reduction when it uses the fine‐grained mutation operators than the traditional coarse‐grained mutation operators (i.e., 1.6% vs. 14.6%).

Mutation testing is a technique in which faults (mutants) are injected into a program or application to assess its test suite effectiveness. It works by inserting mutants and running the application’s test suite to identify if the mutants are detected (killed) or not (survived) by the tests. Although computationally expensive, it has proven to be an effective method to assess application test suites. Several mutation testing frameworks and tools have been built for the various programing languages, however, very few tools have been built for the JavaScript language, more specifically, there is a lack of mutation testing tools for the Node.js runtime and npm based applications. The npm Registry is a public collection of modules of open-source code for Node.js, front-end web applications, mobile applications, robots, routers, and countless other needs of the JavaScript community. The over 700,000 packages hosted in npm are downloaded more than 5 billion times per week. More and more software is published in npm every day, representing a huge opportunity to share code and solutions, but also to share bugs and faulty software. In this paper, we briefly describe prior work for mutation operators in JavaScript and Node.js, and propose Mutode, an open source tool which leverages the npm package ecosystem to perform mutation testing for JavaScript and Node.js applications. We empirically evaluated Mutode effectiveness by running it on 12 of the top 20 npm modules that have automated test suites.

Traditionally, mutation testing is used as test set generation and/or test evaluation criteria once it is considered a good fault model. This paper uses mutation testing for evaluating an automated static analyzer. Since static analyzers, in general, report a substantial number of false positive warnings, the intention of this study is to define a prioritization approach of static warnings based on their correspondence with mutations. On the other hand, knowing that Mutation Test has a high application cost, another possibility is to try to identify mutations of some specific mutation operators, which an automatic static analyzer is not adequate to detect. Therefore, this information can be used to prioritize the order of incrementally applying mutation operators considering, firstly, those with no correspondence with static warnings. In both cases, contributing to the establishment of incremental strategies on using automatic static analysis or mutation testing or even a combination of them. We used mutation operators as a fault model to evaluate the direct correspondence between mutations and static warnings. The main advantage of using mutation operators is that they generate a large number of programs containing faults of different types, which can be used to decide the ones most probable to be detected by static analyzers. We provide evidences on the correspondence between mutations and some types of static warnings. The results obtained for a set of 19 open-source programs indicate that: 1) static warnings may be prioritized based on their correspondence level with mutations; 2) specific set of mutation operators and their mutations may be prioritized based on their correspondence level with warnings. It is possible to provide an incremental testing strategy aiming at reducing the cost of both static analysis and mutation testing using the correspondence information between these activities/artifacts.

One of the main problems to perform the Software Testing is to find a set of tests (subset from input domain of the problem) which is effective to detect the remaining bugs in the software. The Search-Based Software Testing (SBST) approach uses metaheuristics to find low cost set of tests with a high effectiveness to detect bugs. From several existing test criteria, Mutation Testing is considered quite promising to reveal bugs, despite its high computational cost, due to the great quantity of mutant programs generated. Therefore, this paper addresses the problem of selecting mutant programs and test cases in Mutation Testing context. To this end, it is proposed a Coevolutionary Genetic Algorithm (CGA) and the concept of Genetic Effectiveness, describing a new representation and implementing new genetic operators. The CGA is applied in five benchmarks and the results are compared to other five methods, showing a better performance of the proposed algorithm in subsets automatic selection with better mutation score and greater reduction of computational cost, specifically the amount of testing, when compared with exhaustive test.

Background & Problem Statement - Software testing is a critical phase in the software development lifecycle (SDLC), ensuring that applications function correctly, meet user requirements, and maintain high- quality standards. Traditional software testing approaches, including manual testing and rule-based automation, often face challenges in scalability, efficiency, and adaptability to dynamic software environments. Traditional testing methods are overwhelmed by complex software systems which slows down defect detection and extends both testing costs and release schedules. Machine Learning (ML) has emerged as a transformative solution, introducing predictive and adaptive capabilities that optimize test case selection, automate defect detection, and enhance overall software quality assurance (QA). This study explores the integration of ML in software testing, addressing the challenges of traditional QA methodologies and demonstrating how AI-driven frameworks improve testing efficiency. Methodology - To investigate the impact of ML in software testing, this research adopts a systematic approach by analyzing ML-driven test automation techniques, including predictive testing, adaptive test execution, and automated test case generation. Research reviews how Google Microsoft Facebook IBM and Deep Code put ML-based quality assurance frameworks into operation. The study leverages supervised learning, reinforcement learning, deep learning, and NLP-based techniques to demonstrate how ML models predict software defects, dynamically adapt test cases, and optimize testing resources. The research tests how ML-based testing models operate within CI/CD pipelines to improve ongoing testing and deployment flow. Analysis & Results - The analysis of ML-driven software testing reveals that predictive analytics improves early defect detection rates. It helps developers spend 37% less time debugging their work. Adaptive testing models, including self-healing test scripts, minimize maintenance costs by 50% and enhance test reliability in agile environments. The integration of NLP-based test case generation increases test coverage. NLP technology enables automatic connection between requirements and test cases at 89% success rate. Additionally, reinforcement learning techniques improve test case selection, reducing redundant test executions by 43%. Our research shows different ML methods work well to lessen incorrect error alerts. ML integration for QA surely increasing defect prediction accuracy and optimizing test execution time. Findings & Contributions - This research contributes to the field of AI-driven software testing by providing a comprehensive framework for ML-based QA methodologies. Our study shows that machine learning helps find more software problems better adapts test cases and lowers testing expenses to solve present software development needs. The study also identifies critical challenges, including data availability, model interpretability, and computational overhead, suggesting future research directions in Explainable AI (XAI), hybrid AI-ML testing models, and AI-driven security testing. As the industry moves toward AI-first software testing, this research paves the way for fully autonomous QA frameworks, enabling intelligent, scalable, and cost- effective software validation techniques. Keywords - Machine Learning, Software Testing, Quality Assurance, Predictive Testing, Adaptive Testing, Test Automation, Defect Prediction, Self- Healing Test Scripts, AI-Driven QA, Reinforcement

Software testing is a critical and time-consuming element of the software development life cycle (SDLC). Software testing determines the quality of a software before it is used by end users. The main goal is to find software errors or defects so that these errors or defects can be corrected in this testing phase. There are two general techniques for detecting software errors, namely: Black Box and White Box Testing. Black Box testing technique is used for validation, namely whether you have built the right software and White Box testing is used for verification, namely whether the software made is correct. This study aims to test the value index software using three methods of Black Box testing techniques, namely Equivalence partitioning (EP), Boundary Value Analysis (BVA) and Error Guessing. The testing process is carried out to find out how much error occurs in the value index software. The research method is qualitative descriptive and analytical. The results of this study indicate that the value index application has no shortcomings (error of 0%) both with BVA, EP and Error Guessing techniques.

Software testing plays important role in software development life cycle (SDLC) while dealing with critical business applications. From SDLC 70% of time and cost is used for testing only, it is time consuming and costly phase. For this one of the cost effective technique can be used which predicts a module as fault-prone and non-fault prone prior to testing. Predicting a safe module during testing and after development increases the cost of projects. As need to allocate better-test resources for module. This process may take most of the time in preparation and may lose some modules untested which cause accidental failure. Defective software modules cause software failures, increase development and maintenance costs, and decrease customer satisfaction. In this research, we present a novel fault prediction technique that reduces the probability of false (pf) and increases the precision for detection of faulty modules. Software Reliability is becoming an essential attribute of any software system. To improve software quality and testing efficiency by constructing predictive models from code attributes to enable a timely identification of fault-prone modules.Software defect prediction model helps in early detection of faults and contribute to their efficient removal and producing a reliable software system.

Mutation testing is a well known technique for estimating and improving the quality of test suites. Given a test suite T for a system S, mutation testing systematically creates mutants of S and executes T to measure how many mutants T detects. If T does not detect some (non-equivalent) mutants, T can be improved by adding test cases that detect those mutants. Mutants are created by applying mutation operators. Mutation operators are important because they define the characteristics of the system that are tested as well as the characteristics that are improved in the test suite. While mutation operators are well defined for a number of programming paradigms such as sequential or multi-threaded, to the best of our knowledge, mutation operators have not been defined for the actor programming model. In this paper, we define and classify mutation operators that can be used for mutation testing of actor programs.

Software quality assurance and testing is a vital phase in the software development lifecycle, oftentimes requiring a significant portion of the overall development budget. Providing authentic, real-world software quality assurance and testing experiences within the context of a Computer Science or Software Engineering curriculum is challenging. Based on prior experience leveraging Free and Open Source Software (FOSS) as an instructional medium for teaching Software Engineering concepts, this paper describes five increasingly involved learning interventions that utilize FOSS in a software testing course to provide students real-world hands-on software testing knowledge and experience. The proposed approach will be piloted and assessed in software testing courses at the undergraduate and graduate level.

Mutant generation for embedded systems using kernel-based software and hardware fault simulation

A probabilistic framework for mutation testing in deep neural networks

Cybersecurity risks are increasing in frequency and complexity, but many organizations struggle to plan and implement adequate protections at all stages of the software development life cycle (SDLC). Security is frequently added at the end of development (afterthought), and making effective use of safeguard space is difficult for IT leaders. The purpose of this study is to produce an all‐encompassing framework to adopt and ensure security throughout each phase of the SDLC, from planning through maintenance. The aim is to minimize vulnerabilities and improve the resilience of software by making “security by design” a structure that not only adopts security elegantly as a living document but also is built to be part of the development process. This study adopted a mixed‐methods approach. The initial stage of inquiry involved a systematic literature review (SLR) to identify common cybersecurity issues associated with each SDLC phase. The SLR was followed by an empirical survey of 71 software professionals from a variety of organizations. The survey was designed to gather perceived threats, current practices, and challenges associated with software development for survey participants' organizations. The data collected were analyzed and reviewed statistically, through chi‐square tests and ANOVA, to profile the variance relative to the size of the organization, geographic region, and experience level of the practitioner. The results noted several high‐risk challenges across the SDLC: underfunded security controls, imprecise requirements, insecure architecture, software bugs (i.e., injection vulnerabilities), inadequate testing, misconfigured production environments, and unreliable maintenance. The proposed framework provides cybersecurity mitigation techniques for each stage of the SDLC, such as leveraging security‐oriented design patterns, secure coding policies (i.e., input validation and authentication protocols), robust testing (i.e., penetration testing and code review), and continuous monitoring after deployment. The implementation of these measures leads to a significant risk reduction in the overall organizational security posture. The framework is a formalized end‐to‐end approach to secure software development by embedding security throughout the cycle. Embedding security as a part of the process versus an afterthought at every stage of the cycle creates a risk reduction impact. This integrated approach also provides organizations with the opportunity to foresee and mitigate events earlier in the cycle, along with general compliance mandates (i.e., GDPR, HIPAA, and PCI‐DSS), to provide more resilient, trustworthy software systems.