A Kidney Algorithm for Pairwise Test Suite Generation

Software Testing is an important aspect of the real time software development process. Software testing always assures the quality of software product. As associated with software testing, there are few very important issues where there is a need to pay attention on it in the process of software development test. These issues are generation of effective test case and test suite as well as optimization of test case and suite while doing testing of software product. The important issue is that testing time of the test case and test suite. It is very much important that after development of software product effective testing should be performed. So to overcome these issues of optimization, we have proposed new approach for test suite optimization using genetic algorithm (GA). Genetic algorithm is evolutionary in nature so it is often used for optimization of problem by researcher. In this paper, our aim is to study various selections methods like tournament selection, rank selection and roulette wheel selection and then we apply this genetic algorithm (GA) on various programs which will generate optimized test suite with parameters like fitness value of test case, test suite and take minimum amount of time for execution after certain preset generation. In this paper our main objectives as per the experimental investigation, we show that tournament selection works very fine as compared to other methods with respect fitness selection of test case and test suites, testing time of test case and test suites as well as number of requirements.

There exists a direct need to automate the process of test suite generation to get the most optimal results as testing accounts for more than 40% of total cost. One method to solve this problem is the use of meta-heuristic algorithms which iteratively improve the test data to reach the most optimized test suites. This study focuses on the performance evaluation of six meta-heuristic algorithms namely: hill-climbing algorithm (HCA), particle swarm optimization (PSO), firefly algorithm (FA), cuckoo search algorithm (CS), bat algorithm (BA) and artificial bee colony algorithm (ABC) by using their standard implementation to optimize the path coverage and branch coverage produced by the test data. The goal of the study was to find the best-suited algorithm to narrow down the future research in the field of test automation for path coverage-based optimization approaches. Each algorithm was first implemented to automatically generate test suites based on the program under test. This was followed by the performance evaluation of each algorithm for five programs written in Java. The algorithms were compared using process metrics: average time, best time, worst time and product metrics: path coverage & objective function values of the generated test suites. Results indicated ABC as the best-suited algorithm as it gave the most optimal test suites in reasonable time. BA was found to be the fastest but produced less optimal results. FA was found to be the slowest algorithm, while CS, PSO and HCA performed in between. These results show the relative performance of the six algorithms for this scenario and may be used by the future researchers to narrow down and improve the best performing algorithms for path coverage-based optimization approaches.

There occur frequent requirement changes in software systems even after the software has been developed. Regression testing is continuously performed to identify the undesired affects of these requirement changes on already tested system. Test suites grow enormously with these changes due to addition of new test cases for enhanced functionality. Optimization of test suite to perform regression testing within the budgetary and time restrictions is ultimate choice for a tester because "Retest all" test suite is un-economical and is not suitable choice. Test suite optimization can be either static or on the fly. With on the fly optimization, optimal suite keeps on changing with the requirement changes. On the fly optimization of test suite is preferable option for regression testing. Presently, static test suite optimization approaches exist. We have proposed an application specific, on the fly optimization approach for test suite optimization problem. We have implemented our approach on an academic testing problem. We use fuzzy logic to optimize the test suite with multiple optimization objectives. Our approach has been successful to generate on the fly optimized test suites for changing requirements. In future, we will implement this approach on considerably large sized testing problems.

The metaheuristic algorithm is a very important area of research that continuously improves in solving optimization problems. Nature-inspired is one of the metaheuristic algorithm classifications that has grown in popularity among researchers over the last few decades. Nature-inspired metaheuristic algorithms contribute significantly to tackling many standing complex problems (such as the combinatorial t-way testing problem) and achieving optimal results. One challenge in this area is the combinatorial explosion problem, which is always intended to find the most optimal final test suite that will cover all combinations of a given interaction strength. As such, test case generation has been selected as the most active research area in combinatorial t-way testing as Non-deterministic Polynomial-Time Hardness (NP-hard). However, not all metaheuristics are effectively adopted in combinatorial t-way testing, some proved to be effective and thus have been popular tools selected for optimization, whilst others were not. This research paper outlines a hundred and ten (110) outstanding nature-inspired metaheuristic algorithms for the last decades (2001 and 2021), such as the Coronavirus Optimization Algorithm, Ebola Optimization Algorithm, Harmony Search, Tiki-Taka Algorithm, and so on. The purpose of this review is to revisit and carry out an up-to-date review of these distinguished algorithms with their respective current states of use. This is to inspire future research in the field of combinatorial t-way testing for better optimization. Thus, we found that all metaheuristics have a simple structure that can be adopted in different areas to become more efficient in optimization. Finally, we suggested some future paths of investigation for researchers who are interested in the combinatorial t-way testing field to employ more of these algorithms by tuning their parameters settings to achieve an optimal solution.

Redundant test cases in newly generated test suites often remain undetected until execution and waste scarce project resources. In model-based testing, the testing process starts early on in the developmental phases and enables early fault detection. The redundancy in the test suites generated from models can be detected earlier as well and removed prior to its execution. The article presents a novel model-based test suite optimization technique involving UML activity diagrams by formulating the test suite optimization problem as an Equality Knapsack Problem. The aim here is the development of a test suite optimization framework that could optimize the model-based test suites by removing the redundant test cases. An evolution-based algorithm is incorporated into the framework and is compared with the performances of two other algorithms. An empirical study is conducted with four synthetic and industrial scale Activity Diagram models and results are presented.

Websites entail some exclusively novel challenges in this era of software quality focus. This sequence of fresh issues associated exclusively with website applications can be considered as an opportunity as well as a greater challenge to have a complete automated testing of websites. Despite the continuous research in the web application testing domain, further work is needed to quantitatively analyze the testing results especially with respect to obtaining the statistics such as the efficiency and coverage of the test cases, value of test suite with respect to coverage and efficiency and distribution of the fault reasons. Using artificial intelligence techniques, a complete framework has been proposed to address the above mentioned challenges. This research effort presents the preliminary results of this effort. The proposed framework would provide the facilities to cover analyze web based on the web modeling, test case generation, test suite optimization and related tool development. This research effort is a part of framework project and tries to determine the effect of introduction of ‘value’ factor in evolutionary testing.

Supply chain efficiency is a major challenge in today’s business environment, where efficient resource allocation and coordination of activities are essential for competitive advantage. Traditional efficiency strategies often struggle for resources for the complex and dynamic network. In response, bio-inspired metaheuristic algorithms have emerged as powerful tools to solve these optimization problems. Referring to the random search nature of metaheuristic algorithms and emphasizing that no metaheuristic algorithm is the best optimizer for all optimization applications, the No Free Lunch (NFL) theorem encourages researchers to design newer algorithms to be able to provide more effective solutions to optimization problems. Motivated by the NFL theorem, the innovation and novelty of this paper is in designing a new meta-heuristic algorithm called Bobcat Optimization Algorithm (BOA) that imitates the natural behavior of bobcats in the wild. The basic inspiration of BOA is derived from the hunting strategy of bobcats during the attack towards the prey and the chase process between them. The theory of BOA is stated and then mathematically modeled in two phases (i) exploration based on the simulation of the bobcat’s position change while moving towards the prey and (ii) exploitation based on simulating the bobcat’s position change during the chase process to catch the prey. The performance of BOA is evaluated in optimization to handle the CEC 2017 test suite for problem dimensions equal to 10, 30, 50, and 100, as well as to address CEC 2020. The optimization results show that BOA has a high ability in exploration, exploitation, and balance them during the search process in order to achieve a suitable solution for optimization problems. The results obtained from BOA are compared with the performance of twelve well-known metaheuristic algorithms. The findings show that BOA has been successful in handling the CEC 2017 test suite in 89.65, 79.31, 93.10, and 89.65% of the functions for the problem dimension equal to 10, 30, 50, and 100, respectively. Also, the findings show that in order to handle the CEC 2020 test suite, BOA has been successful in 100% of the functions of this test suite. The statistical analysis confirms that BOA has a significant statistical superiority in the competition with the compared algorithms. Also, in order to analyze the efficiency of BOA in dealing with real world applications, twenty-two constrained optimization problems from CEC 2011 test suite and four engineering design problems have been selected. The findings show that BOA has been successful in 90.90% of CEC2011 test suite optimization problems and in 100% of engineering design problems. In addition, the efficiency of BOA to handle SCM applications has been challenged to solve ten case studies in the field of sustainable lot size optimization. The findings show that BOA has successfully provided superior performance in 100% of the case studies compared to competitor algorithms.

The Antlion Optimization (ALO) algorithm is a meta-heuristic optimization algorithm based on the hunting of ants by antlions. The basic inadequacy of this algorithm is that it has long run time because of the random walk model used for the ant's movement. We improved some mechanisms in ALO algorithm, such as ants' random walking, reproduction, sliding ants towards antlion, elitism, and selection procedure. This proposed algorithm is called Improved Antlion Optimization (IALO) algorithm. To show the performance of the proposed IALO algorithm, we used different measurement metrics, such as mean best, standard deviation, optimality, accuracy, CPU time, and number of function evaluations (NFE). The proposed IALO algorithm was tested for different benchmark test functions taken from the literature. There are no studies regarding time analysis of ALO algorithm found in the literature. This study firstly aims to demonstrate the success of the proposed IALO algorithm especially in runtime analysis. Secondly, the IALO algorithm was also applied to the Quadratic Assignment Problem (QAP) known as a difficult combinatorial optimization problem. In QAP tests, the performance of the IALO algorithm was compared with the performances of the classic ALO algorithm and 14 well-known and recent meta-heuristic algorithms. The results of the benchmark test functions show that IALO algorithm is able to provide very competitive results in terms of mean best/standard deviation, optimality, accuracy, CPU time and NFE metrics. The CPU time results prove that IALO algorithm is faster than the classic ALO algorithm. As a result of the QAP tests, the proposed IALO algorithm has the best performance according to the mean cost, worst cost and standard deviation. The source codes of QAP with the proposed IALO algorithm are publicly available at https://github.com/uguryuzgec/QAP-with-IALO.

Test suite optimization is mandatory activity to cope with the limited resources and economical constraints. Aside from conventional optimization approaches, computational intelligence based approaches have also focused this research area and evolutionary algorithms have been successful to remarkably reduce the test suite size. These approaches classify the decision space by generating an optimal front called Pareto front. Despite the tremendous research work on interpretation of Pareto front, its visualization and usefulness for higher objectives is a challenging task. Pareto fronts contain multiple suitable solutions that can be converged using heuristics to a reduced set of possible solutions. We have proposed fuzzy based optimization approach in combination with all path coverage criterion to safely reduce a test suite to a single solution. We have initially implemented our proposed work on a testing problem having three objectives; however, it is scalable to any finite number of optimization objectives. We validated our approach by comparing it with evolutionary algorithms. We found that our approach significantly reduces the test suite to a precise test suite. We have concluded that our approach is capable to be automated and provide ‘on the fly’ optimal solution.

A modified reptile search algorithm for global optimization and image segmentation: Case study brain MRI images

Regression testing is an essential quality test technique during the maintenance phase of the software. It is executed to ensure the validity of the software after any modification. As software evolves, the test suite expands and may become too large to be executed entirely within a limited testing budget and/or time. So, to reduce the cost of regression testing, it is mandatory to reduce the size of the test suite by discarding the redundant test cases and selecting the most representative ones that do not compromise the effectiveness of the test suite in terms of some predefined criteria such as its fault-detection capability. This problem is known as test suite reduction (TSR); and it is known to be as nondeterministic polynomial-time complete (NP-complete) problem. This paper formulated the TSR problem as a multi-objective optimization problem; and adapted the heuristic binary bat algorithm (BBA) to resolve it. The BBA algorithm was adapted in order to enhance its exploration capabilities during the search for Pareto-optimal solutions. The effectiveness of the proposed multi-objective adapted binary bat algorithm (MO-ABBA) was evaluated using 8 test suites of different sizes, in addition to twelve benchmark functions. Experimental results showed that, for the same fault discovery rate, the MO-ABBA is capable of reducing the test suite size more than each of the multi-objective original binary bat (MO-BBA) and the multi-objective binary particle swarm optimization (MO-BPSO) algorithms. Moreover, MO-ABBA converges to the best solutions faster than each of the MO-BBA and the MO-BPSO.

Test suite reduction techniques decrease the cost of software testing by removing the redundant test cases from the test suite while still producing a reduced set of tests that yields the same level of code coverage as the original suite. Most of the existing approaches to reduction aim to decrease the size of the test suite. Yet, the difference in the execution cost of the tests is often significant and it may be costly to use a test suite consisting of a few long-running test cases. Thus, this paper proposes an algorithm, based on the concept of test irreplaceability, which creates a reduced test suite with a decreased execution cost. Leveraging widely used benchmark programs, the empirical study shows that, in comparison to existing techniques, the presented algorithm is the most effective at reducing the cost of running a test suite.

Testing and debugging have been the most significant steps of software development since it is tricky for engineers to create error-free software. Software testing takes place after coding with the goal of finding flaws. If errors are found, debugging would be done to identify the source of the errors so that they may be fixed. Detecting as well as locating defects are thus two essential stages in the creation of software. We have created a unique approach with the following two working phases to generate a minimized test suite that is capable of both detecting and localizing faults. In the initial test suite minimization process, the cases were generated and minimized based on the objectives such as D-score and coverage by the utilization of the proposed Blue Monkey Customized Black Widow (BMCBW) algorithm. After this test suite minimization, the fault validation is done which includes the process of fault detection and localization. For this fault validation, we have utilized an improved Long Short-Term Memory (LSTM). At 90% of the learning rate the accuracy of the presented work is 0.97%, 2.20%, 2.52%, 0.97% and 2.81% is better than the other extant models like AOA, COOT, BES, BMO and BWO methods. The results obtained proved that our Blue Monkey Customized Black Widow Optimization-based fault detection and localization approach can provide superior outcomes.

Ribonucleic Acid (RNA) has important structural and functional roles in the cell and plays roles in many stages of protein synthesis. The structure of RNA largely determines its function. Current physical methods for structure determination are time-consuming and expensive, thus the methods for the computational prediction of structure are necessary. Various algorithms that have been used for RNA structure prediction based in minimum free energy include dynamic programming (DP) and meta heuristic algorithms. One of the most recent meta heuristic algorithms is Musician's behavior-inspired harmony search (HS) algorithm that has been successful in numerous complex optimization problems. This paper builds on the previous work of the harmony search algorithm (HSRNAFold) which was used to find the RNA secondary structure with minimum free energy. In this paper, the accuracy of prediction is compared to the dynamic programming technique RNAFold. The results show that HSRNAFold is able to predict more accurate structures than RNAFold for all test sequences.

Constructing test cases from counterexamples generated by a model checker is an important method to perform test automation. In fact, one counterexample may cover multiple test goals, which leads to unnecessary calls to the model checker, and redundant test cases in test suite such that affect the testi ng performance. A method to test suite reduction based on satisfiability is proposed. The kripke model is translated in conjunction with test goals (trap properties) into CNFs. And then test goal t o generate counterexample is selected according to the hardness of the corresponding CNF, after that, model checking the selected test goal to generate counterexample. The generated counterexample is translated in conjunction with those uncovered test goals i nto CNFs. If the corresponding CNF is unsatisfiable then the test goal is pick ed out from the set of test goals. Meanwhile, the new generated test case is winnowed by test suite to reduce the redundancy before it is added into the test suite. Experimental results show that t he method proposed in this paper is effective for reducing the model checker calls and the length of the test suite. At the same time, the coverage and error detection capability of the test suite are not decl ined.