How to Overcome the Equivalent Mutant Problem and Achieve Tailored Selective Mutation Using Co-evolution

Manual test case generation is an exhaustive and time-consuming process. However, automated test data generation may reduce the efforts and assist in creating an adequate test suite embracing predefined goals. The quality of a test suite depends on its fault-finding behavior. Mutants have been widely accepted for simulating the artificial faults that behave similarly to realistic ones for test data generation. In prior studies, the use of search-based techniques has been extensively reported to enhance the quality of test suites. Symmetry, however, can have a detrimental impact on the dynamics of a search-based algorithm, whose performance strongly depends on breaking the “symmetry” of search space by the evolving population. This study implements an elitist Genetic Algorithm (GA) with an improved fitness function to expose maximum faults while also minimizing the cost of testing by generating less complex and asymmetric test cases. It uses the selective mutation strategy to create low-cost artificial faults that result in a lesser number of redundant and equivalent mutants. For evolution, reproduction operator selection is repeatedly guided by the traces of test execution and mutant detection that decides whether to diversify or intensify the previous population of test cases. An iterative elimination of redundant test cases further minimizes the size of the test suite. This study uses 14 Java programs of significant sizes to validate the efficacy of the proposed approach in comparison to Initial Random tests and a widely used evolutionary framework in academia, namely Evosuite. Empirically, our approach is found to be more stable with significant improvement in the test case efficiency of the optimized test suite.

Fault in a software program can be detected by mutation testing. However, mutation testing is an expensive process in a software testing domain. In this paper, we have introduced a method based on genetic algorithm and mutation analysis for unit testing process. Software industry produces high quality software in which software testing has an important role. First, we make a program/software and intent some mutant in this program/software, find most critical path and optimise test cases using genetic algorithm for the unit testing. Initially generated test cases are refined using genetic algorithm. We use a mutant function for measuring the adequacy of the test case set. The given mutant function is used to calculate a mutant score. We have achieved 100% path coverage and boundary coverage using mutation testing. The objective is to produce a set of good test cases for killing one or more undesired mutants and produces different mutant from original software/program. Unlike simple algorithms, genetic algorithms provide suitability for reducing the data generation at a comparable cost. An optimised test case has been generated by proposed approach for cost reduction and revealing or killing undesired test cases.

Fault in a software program can be detected by mutation testing. However, mutation testing is an expensive process in a software testing domain. In this paper, we have introduced a method based on genetic algorithm and mutation analysis for unit testing process. Software industry produces high quality software in which software testing has an important role. First, we make a program/software and intent some mutant in this program/software, find most critical path and optimise test cases using genetic algorithm for the unit testing. Initially generated test cases are refined using genetic algorithm. We use a mutant function for measuring the adequacy of the test case set. The given mutant function is used to calculate a mutant score. We have achieved 100% path coverage and boundary coverage using mutation testing. The objective is to produce a set of good test cases for killing one or more undesired mutants and produces different mutant from original software/program. Unlike simple algorithms, genetic algorithms provide suitability for reducing the data generation at a comparable cost. An optimised test case has been generated by proposed approach for cost reduction and revealing or killing undesired test cases.

Mutation testing can be used to measure the quality of a given test suite. But two flaws prevent wide acceptance. First, there are equivalent mutants - mutants that are semantically equivalent to the unmodified version and therefore unkillable. Manually identifying those mutants is time-consuming and error-prone. Second, initially there are often too few test cases. Mutation testing detects missing cases. But it is too time-consuming to manually write all the required test cases. This paper shows how to use symbolic execution to tackle both problems, i.e., to detect equivalent mutants and exclude them from further analysis, and to automatically generate test cases that kill the remaining mutants. Our evaluation uses a set of 252 publicly available mutants for which it is known that they are hard to classify. Despite the fact that detecting equivalent mutants is an undecidable problem in general, our fully automatic tool MutantDistiller correctly classifies all of them (13 equivalent, 239 non-equivalent). MutantDistiller also generates test cases that kill the nonequivalent mutants.

One approach widely adopted for software testing rely on mutation to use as their comparison criteria. However, in mutation testing, the number of useless mutants generated during the mutation process can reach up to 90% of the total number of mutants created, making the process costly and time-consuming. The concept of subsumption has been empirically used in mutation testing to establish fault hierarchies in which tests that detect certain failures guarantee the detection of others, drastically reducing the generation of these useless mutants during mutation analysis. The tools used for mutation testing do not yet present automated solutions for checking for dominance and, consequently, selection of mutants to be considered in the tests. Hence, in this paper we present a model of automatic identification of subsumption in mutation testing based on evolutionary algorithms, demonstrating evidence of the efficiency of the use of artificial computational approaches for automatic generation of graphs of dynamic dominances in complex problems. We consider in our approach the dynamic subsumption method and a visualization model from the generation of graphs.We show that our approach is feasible and useful.

Artificial neural networks (ANN) and evolutionary algorithms are two relatively young research areas that were subject to a steadily growing interest during the past years. This paper examines the use of different evolutionary algorithms, imperialist competitive algorithm (ICA), genetic algorithm (GA), ICA-GA and recursive ICA-GA (R-ICA-GA) to train a classification problem on a multi layer perceptron (MLP) neural network. All of named evolutionary training algorithms are compared together in this paper. The first goal of the paper is to apply new evolutionary optimization algorithms ICA-GA and R-ICA-GA for training the ANN and the second goal of the paper is to compare different evolutionary algorithms. It is shown that the ICA-GA has the best performance, in number of epochs, compared to the other algorithms. For this purpose, learning algorithms are applied on six known datasets (WINE, PIMA, WDBC, IRIS, SONAR and GLASS) which are used for classification problems.

Genetic algorithms are widely used to solve and optimize combinatorial problems and are more often applied for library design in combinatorial chemistry. Because of their flexibility, however, their implementation can be challenging. In this study, the influence of the representation of solid catalysts on the performance of genetic algorithms was systematically investigated on the basis of a new, constrained, multiobjective, combinatorial test problem with properties common to problems in combinatorial materials science. Constraints were satisfied by penalty functions, repair algorithms, or special representations. The tests were performed using three state-of-the-art evolutionary multiobjective algorithms by performing 100 optimization runs for each algorithm and test case. Experimental data obtained during the optimization of a noble metal-free solid catalyst system active in the selective catalytic reduction of nitric oxide with propene was used to build up a predictive model to validate the results of the theoretical test problem. A significant influence of the representation on the optimization performance was observed. Binary encodings were found to be the preferred encoding in most of the cases, and depending on the experimental test unit, repair algorithms or penalty functions performed best.

For problems involving uncertainties in design variables and parameters, a bi-objective evolutionary algorithm (EA) based approach to design optimization using evidence theory is proposed and implemented in this paper. In addition to a functional objective, a plausibility measure of failure of constraint satisfaction is minimized. Despite some interests in classical optimization literature, this is the first attempt to use evidence theory with an EA. Due to EA's flexibility in modifying its operators, nonrequirement of any gradient, its ability to handle multiple conflicting objectives, and ease of parallelization, evidence-based design optimization using an EA is promising. Results on a test problem and two engineering design problems show that the modified evolutionary multi-objective optimization algorithm is capable of finding a widely distributed trade-off frontier showing different optimal solutions corresponding to different levels of plausibility failure limits. Furthermore, a single-objective evidence-based EA is found to produce better optimal solutions than a previously reported classical optimization algorithm. Furthermore, the use of a graphical processing unit (GPU) based parallel computing platform demonstrates EA's performance enhancement around 160–700 times in implementing plausibility computations. Handling uncertainties of different types are getting increasingly popular in applied optimization studies and this EA based study is promising to be applied in real-world design optimization problems.

Mutation testing has a widespread reputation of being a rather powerful testing technique. However, its practical application requires the detection of equivalent mutants. Detecting equivalent mutants is cumbersome since it requires manual analysis, resulting in unbearable testing cost. To overcome this difficulty, researchers have proposed the use of mutant classification, an approach that helps isolating equivalent mutants. From this perspective, the present paper establishes and assesses possible mutant classification strategies. The conducted study suggests that while mutant classification can be useful in isolating equivalent mutants, it fails to kill some mutants. Indeed, the experimental results show that the proposed strategies achieve to kill approximately 95% of the introduced killable mutants.

Mutation testing is a fault‐based technique to test the quality of test suites by inducing artificial syntactic faults or mutants in a source program. However, some mutants have the same semantics as original program and cannot be detected by any test suite input known as equivalent mutants. Equivalent mutant problem (EMP) is undecidable as it requires manual human effort to identify a mutant as equivalent or killable. The constraint‐based testing (CBT) theory suggests the use of mathematical constraints which can help reveal some equivalent mutants using mutant features. In this paper, we consider three metrics of CBT theory, ie, reachability, necessity, and sufficiency to extract feature constraints from mutant programs. Constraints are extracted using program dependency graphs. Other features such as degree of significance, semantic distance, and information entropy of mutants are also extracted to build a binary classification model. Machine learning algorithms such as Random Forest, GBT, and SVM are applied under two application scenarios (split‐project and cross‐project) on ten Java programs to predict equivalent mutants. The analysis of the study demonstrates that that the proposed techniques not only improves the efficiency of the equivalent mutant detection but also reduces the effort required to perform it with small accuracy loss.

The ratio of equivalent mutants: A key to analyzing mutation equivalence

In order to solve the current problems( expensive testing cost and low efficiency of test case generation) in mutation test for Simulink models,a mutation testing process and an optimized set of mutation operators were proposed for Simulink models based on program mutation according to the current mutation operators for the Simulink models. Experiments show that this set of mutation operators can effectively reduce the generation number of mutation models without prejudice to the mutation score of testing case set,thus it will effectively save the testing cost. Then a search-based test case generation method for Simulink models mutation testing was described. The test case generation problem was transformed into the objective function minimization problem,and the test cases which can kill the mutation models were ultimately obtained through the optimization of objective function by algorithm of simulated annealing. Finally,the application of a typical case for the method verified the correctness and effectiveness.

This paper presents a theory of testing that integrates into Hoare and He’s Unifying Theory of Programming (UTP). We give test cases a denotational semantics by viewing them as specification predicates. This reformulation of test cases allows for relating test cases via refinement to specifications and programs. Having such a refinement order that integrates test cases, we develop a testing theory for fault-based testing. Fault-based testing uses test data designed to demonstrate the absence of a set of pre-specified faults. A well-known fault-based technique is mutation testing. In mutation testing, first, faults are injected into a program by altering (mutating) its source code. Then, test cases that can detect these errors are designed. The assumption is that other faults will be caught, too. In this paper, we apply the mutation technique to both, specifications and programs. Using our theory of testing, two new test case generation laws for detecting injected (anticipated) faults are presented: one is based on the semantic level of UTP design predicates, the other on the algebraic properties of a small programming language.

Evolutionary algorithms (EAs) have been widely used to search for optimal strategies for the planning and management of water resources systems, particularly reservoir operation. This study provides a comprehensive diagnostic assessment of state of the art of the non-animal-inspired EA applications to reservoir optimization. This type of EAs does not mimic biologic traits and group strategies of animal (wild) species. A search of pertinent papers was applied to the journal citation reports (JCRs). A bibliometric survey identified 14 pertinent non-animal-inspired EAs, such as the genetic algorithm (GA), simulated annealing (SA), and differential evolution (DE) algorithms, most of which have a number of modified versions. The characteristics of non-animal-inspired EAs and their modified versions were discussed to identify the difference between EAs and how each EA was improved. Additionally, the type of application of non-animal-inspired EAs to different case studies was investigated, and comparisons were made between the performance of the applied EAs in the studied literature. The survey revealed that the GA is the most frequently applied algorithm, followed by the DE algorithm. Non-animal-inspired EAs are superior to the classical methods of reservoir optimization (e.g., the non-linear programming and dynamic programming) due to faster convergence, diverse solution space, and efficient objective function evaluation. Several non-animal-inspired EAs of recent vintage have been shown to outperform the classic GA, which was the first evolutionary algorithm applied to reservoir operation.

Android is an operating system source which offers flexibility and support for most mobile applications, and easy access to social networks. It is important to understand the complexity of design, development, implementation, and testing of Android apps. A number of challenges may be faced in testing android applications, including the lack of testing processes and methods, testing experts being unavailable, poor in-house testing environment, and time restrictions. Mutation testing is a fault-based testing technique, applied by generating mutants and running the application with these mutants to analyze the killed and equivalent mutants. We defined a set of mutation operators according to the features of android applications: apps with content sharing, apps with multimedia, apps with graphics, and apps with user location and maps. We identified 42 mutation operators. In addition, we implemented a new tool, “µ-Android,” which automatically generates mutants and retrieves results to prove the efficiency of the test cases and enable the new operators.