Quality Evaluation of Object-Oriented and Standard Mutation Operators Applied to C# Programs

Meta-heuristics Genetic Algorithms (GA) and Ant Colony Optimisation (ACO) have both shown success when applied to permutation problems such as the Traveling Salesman Problem (TSP). Therefore, combining the approaches could prove beneficial. This paper presents an ACO inspired mutation operator for use in a GA based on Partial-ACO. Applied to a range of TSP instances the ACO mutation operator demonstrates considerable improvements over standard GA mutation operators finding solutions within a few percent of optimal for TSP instances of nearly 4000 cities. In fact, experiments demonstrate performance close to that of local search with standard mutation operators. Moreover, when integrating local search the ACO inspired mutation operator continues to provide improved results over standard mutation and local search.KeywordsAnt colony optimisationGenetic algorithmMutation

Traditionally, mutation testing is used as test set generation and/or test evaluation criteria once it is considered a good fault model. This paper uses mutation testing for evaluating an automated static analyzer. Since static analyzers, in general, report a substantial number of false positive warnings, the intention of this study is to define a prioritization approach of static warnings based on their correspondence with mutations. On the other hand, knowing that Mutation Test has a high application cost, another possibility is to try to identify mutations of some specific mutation operators, which an automatic static analyzer is not adequate to detect. Therefore, this information can be used to prioritize the order of incrementally applying mutation operators considering, firstly, those with no correspondence with static warnings. In both cases, contributing to the establishment of incremental strategies on using automatic static analysis or mutation testing or even a combination of them. We used mutation operators as a fault model to evaluate the direct correspondence between mutations and static warnings. The main advantage of using mutation operators is that they generate a large number of programs containing faults of different types, which can be used to decide the ones most probable to be detected by static analyzers. We provide evidences on the correspondence between mutations and some types of static warnings. The results obtained for a set of 19 open-source programs indicate that: 1) static warnings may be prioritized based on their correspondence level with mutations; 2) specific set of mutation operators and their mutations may be prioritized based on their correspondence level with warnings. It is possible to provide an incremental testing strategy aiming at reducing the cost of both static analysis and mutation testing using the correspondence information between these activities/artifacts.

A two-stage differential biogeography-based optimization algorithm and its performance analysis

Android applications have experienced an outstanding growth in recent years with a projected figure of 378 billion in the number of application downloads by 2020. As a result, verification and validation of Android apps and their graphical user interface through testing is crucial for quality assurance. One way to assess quality assurance is through the use of mutation testing. Mutation testing modifies the code of the System Under Test in small ways in order to produce faulty versions of the program called mutants and compare them to the original version. However, the cost of mutation testing can be expensive and time consuming. We propose a graphical user-friendly Android mutation tool called Edroid whose primary purpose is to mutate Android's main components such as activities, services, content providers and broadcast receivers using the source code of XML files. We introduce fourteen Android mutation operators, where ten of them are new, that are derived from operators used in different fields such as mutation testing for Graphical User Interface and XML schema. We evaluate Edroid on five real world Android applications using testing methodologies such as random testing and black box testing. Results from these experiments demonstrate that Edroid facilitates the generation of mutants. In addition and equally important, the set of proposed mutation operators are effective in revealing errors that improve the quality of a given test suite.

SummaryFor mutation testing, the huge cost of running test suites on a large number of mutants has been a serious obstacle. To resolve this problem, we propose a learning‐based mutant reduction techniqueMuTrain.MuTrainuses cost‐considerate linear regression (i.e., CLARS) to learn amutation model, which predicts the mutation score of a test suite based on the mutation testing results of a previous version of a target program. Then,MuTrainapplies the mutation model for subsequent versions to predict mutation scores with significantly fewer mutants. For effective mutant reduction and accurate mutation score prediction,MuTrainuses fine‐grained mutation operators refined from the existing coarse‐grained mutation operators. The experiment results show thatMuTrainreduces the number of mutants effectively (i.e., selecting only 1.6% of mutants). Moreover,MuTrainpredicts mutation score far more accurately than the existing mutant reduction techniques and random mutant selection. We also found thatMuTrainachieves much greater mutant reduction when it uses the fine‐grained mutation operators than the traditional coarse‐grained mutation operators (i.e., 1.6% vs. 14.6%).

In order to solve the current problems( expensive testing cost and low efficiency of test case generation) in mutation test for Simulink models,a mutation testing process and an optimized set of mutation operators were proposed for Simulink models based on program mutation according to the current mutation operators for the Simulink models. Experiments show that this set of mutation operators can effectively reduce the generation number of mutation models without prejudice to the mutation score of testing case set,thus it will effectively save the testing cost. Then a search-based test case generation method for Simulink models mutation testing was described. The test case generation problem was transformed into the objective function minimization problem,and the test cases which can kill the mutation models were ultimately obtained through the optimization of objective function by algorithm of simulated annealing. Finally,the application of a typical case for the method verified the correctness and effectiveness.

Mutation testing is a well known technique for estimating and improving the quality of test suites. Given a test suite T for a system S, mutation testing systematically creates mutants of S and executes T to measure how many mutants T detects. If T does not detect some (non-equivalent) mutants, T can be improved by adding test cases that detect those mutants. Mutants are created by applying mutation operators. Mutation operators are important because they define the characteristics of the system that are tested as well as the characteristics that are improved in the test suite. While mutation operators are well defined for a number of programming paradigms such as sequential or multi-threaded, to the best of our knowledge, mutation operators have not been defined for the actor programming model. In this paper, we define and classify mutation operators that can be used for mutation testing of actor programs.

A probabilistic framework for mutation testing in deep neural networks

Testing Deep Learning (DL) systems is a complex task as they do not behave like traditional systems would, notably because of their stochastic nature. Nonetheless, being able to adapt existing testing techniques such as Mutation Testing (MT) to DL settings would greatly improve their potential verifiability. While some efforts have been made to extend MT to the Supervised Learning paradigm, little work has gone into extending it to Reinforcement Learning (RL) which is also an important component of the DL ecosystem but behaves very differently from SL. This paper builds on the existing approach of MT in order to propose a framework, RLMutation, for MT applied to RL. Notably, we use existing taxonomies of faults to build a set of mutation operators relevant to RL and use a simple heuristic to generate test cases for RL. This allows us to compare different mutation killing definitions based on existing approaches, as well as to analyze the behavior of the obtained mutation operators and their potential combinations called Higher Order Mutation(s) (HOM). We show that the design choice of the mutation killing definition can affect whether or not a mutation is killed as well as the generated test cases. Moreover, we found that even with a relatively small number of test cases and operators we manage to generate HOM with interesting properties which can enhance testing capability in RL systems.

Software Testing is one of the most significant phases within the software development life cycle since software bugs can be costly and traumatic. However, the traditional software testing process is not enough on its own as some undiscovered faults might still exist due to the test cases’ inability to detect all underlying faults. Amidst the various proposed techniques of test suites’ efficiency detection comes mutation testing, one of the most effective approaches as declared by many researchers. Nevertheless, there is not enough research on how well the mutation testing tools adhere to the theory of mutation or how well their mutation operators are performing the tasks they were developed for. This research paper presents an investigative study on two different mutation testing tools for Java programming language, namely PIT and µJava. The study aims to point out the weaknesses and strengths of each tool involved through performing mutation testing on four different open-source Java programs to identify the best mutation tool among them. The study aims to further identify and compare the mutation operators of each tool by calculating the mutation score. That is, the operators’ performance is evaluated with the mutation score, with the presumption that the more prominent the number of killed mutants is, the higher the mutation score, thus the more effective the mutation operator and the affiliated tool.Â

Mutant generation for embedded systems using kernel-based software and hardware fault simulation

Mutation testing has been neglected by researchers because of the high cost associated with the technique. To manage this issue, researchers have developed cost reduction strategies that aim to reduce the overall cost of mutation, while maintaining the effectiveness and the efficiency of testing. The purpose of this research paper is to present a new cost reduction strategy that cuts the cost of mutation testing through reducing the number of mutation operators used. The experimental part of the paper focuses on the implementation of this strategy on five different java applications. The results of the experiment areused to evaluate the efficiency and quantify the savings of our approach compared to two other existing mutation testing strategies.

Construction quality evaluation of asphalt pavement based on BIM and GIS

Mutation testing is the fault-based software testing approach that is widely applicable for assessing the effectiveness of a test suite. The test suite effectiveness is measured through artificial seeding of faults into the programs under test. Mutation testing tools (MTTs) are significant enablers of the conversion of mutation testing from the research perspective into the real life and mostly applicable testing process. Without using the automatic MTT, mutation testing can’t be really connected in this present reality and is unrealistic to be acknowledged by the industry. Authors analyze six open-source JAVA-based MTT (Jester, JavaMut, MuJava, Jumble, Judy, and Javalanche). The results are based on the performance of various JAVA programs and two real-life applications. Benchmark comparison among the MTT is presented in terms of mutants, mutation operator, mutation score, and quality output. On the basis of comparative analysis, the performance of each tool is explained with the protocol for finding the appropriate tool among the six MTTs. The results show that the MuJava performs best compared to the others.KeywordsMutation testingFault-based testingAdequacy criterionMutation testing tools

User experience has a fundamental role in determining the effectiveness of image compression methods. This work presents the subjective evaluation of a new compression plug-in for current compression formats developed by Cogisen. The quality of image compression methods is often evaluated by objective metrics based on subjective quality datasets, rather than by using subjective quality evaluation tests. Cogisen's compression method follows an adaptive compression process that evaluates the saliency of any image and calculates the level of compression beyond which viewers shall be aware of image quality degradation. The Single Stimulus Continuous Quality Scale method was used to conduct the subjective quality evaluation of image compression. Pictures compressed by the Facebook Mobile lossy JPEG compression and by the Cogisen plug-in integrated in the Facebook Mobile compression settings were used. The results of the user quality evaluation of pictures show about a 45i¾?% compression improvement, with no loss in perceived image quality, for pictures compressed by the Cogisen plug-in compared to jpeg pictures as compressed by Facebook Mobile.