Test Data Generation Algorithm of Combinatorial Testing Based on Differential Evolution

Software testing and retesting occurs continuously during the software development lifecycle to detect errors as early as possible. As the software evolves the size of test suites also grows. When the no of test cases generated are more, obviously size of the test suite will also be more. So the testing time is to be minimized by reducing the execution time of the algorithm used for test data generation and also by introducing minimization procedure for test suite reduction. Due to limited resources and timing constraints for testing, test suite minimization techniques are needed to eliminate redundant test cases as possible. By considering multiple objectives rather than the coverage alone, the test cases are being generated which satisfies the testing requirements. Most of the existing techniques are code-based. In this article we present an approach by modifying an existing heuristic for test suite minimization. Genetic algorithm has been used for random test data generation and the output of GA is given to the minimization procedure for reducing the total no of generated test cases, collectively named as Hybrid Algorithm (HA). The results are satisfactory and show significant improvements in reducing test suite size with minimum execution time. Experiments have been done for simple to medium complexity java programs taken from SIR and execution time is reduced to 5,685ms for a test set. The results are compared with existing method Mutant Gene Algorithm and size of test suite is minimized upto 13.6% using Hybrid Algorithm.

Software testing is a fundamental step in the software development lifecycle. Its purpose is to evaluate the quality of software applications. Regression testing is an important testing methodology in software testing. The purpose of regression testing is to validate the software after each change of its code. This involves adding new test cases to the test suite and running the test suite as the software changes, making the test suite larger. The cost and time of the project are affected by the test suite size. The challenge is to run regression testing with a smaller number of test cases and larger amount of software coverage. Minimization of the test suite with maximization of the software coverage is an NP-complete problem. Search-based software engineering is an important topic in software engineering, which addresses software engineering optimization problems to find the optimal/approximate solution of the given problem. This study investigated an approach to reducing the regression testing effort and saving time. It also solved the regression testing optimization problem by achieving the maximum test suite coverage with the minimum test suite size. Several experiments were conducted to obtain the optimal solutions for the regression testing problem. We propose an optimization methodology that combines a genetic algorithm and a greedy algorithm to optimize regression testing by respectively maximizing the software test coverage and minimizing the test suite size. The proposed methodology can conveniently deliver fault-free, fully covered, and powerful programs for mission-critical functions. It can be applied to test a real-time system that has high requirements for reliability, security, and safety.

The Subspace Clustering Algorithm Based on Differential Evolution (DESC) performs well among the subspace clustering algorithms of global search. But the fixed number of iterations used may cause the algorithm to terminate when it does not converge or lead to overfitting. The improved algorithm proposed in this paper replaces the mutation operators of the DESC and uses the smoothness of the change in the optimal value of the objective function as the basis of judging convergence –– so that the algorithm can adaptively determine the termination timing and solve the above problems to a large extent. The experimental results show clearly that the proposed algorithm can terminate in the appropriate number of iterations in most cases and has certain advantages over the original algorithm in clustering accuracy and running time.

This study examines the implementation of Test Case Prioritization (TCP) using the Greedy Algorithm (GA) to enhance regression testing efficiency within a financial technology company's software development cycle. With testing durations increasing significantly, this study aims to address inefficiencies by applying the Greedy Algorithm to optimize test suite size and fault detection. The research methodology involves applying the Greedy Algorithm during the Regression Testing phase, comparing the prioritized suite with the original suite using metrics such as Average Percentage Fault Detection (APFD) and Test Suite Size Reduction (TSSR). Results show that the Greedy Algorithm achieved substantial improvements in both test suite size and fault detection effectiveness across different projects. For Project A, the test suite was reduced from 51-22 test cases, achieving a TSSR of 56.8%, with an APFD increase of 206.21%, rising from 0.0853-0.2613. Project B demonstrated even greater optimization, reducing the test suite from 36-8 test cases, resulting in a TSSR of 77.8% and an APFD improvement of 83.92%, rising from 0.3194-0.5875. These outcomes underscore the algorithm’s effectiveness in eliminating redundant test cases, accelerating testing, and enhancing fault detection thereby supporting the company's goal of faster release cycles without compromising quality.

Code coverage criteria define test objectives and provide actionable stopping conditions for creating test cases. They are also widely accepted as indicators of a test suite’s fault detection effectiveness. Several factors influence the relationship between code coverage and test suite effectiveness. However, previous empirical studies investigating this relationship tend to fail to control some of these factors, resulting in contradictory results. Through a systematic review of 417 previous studies investigating the effectiveness of test suites from various venues, such as journals, conferences, workshops, and book chapters, we have identified several factors that can impact experimental studies’ results and even threaten their validity. Some of these factors are well known, such as the test suite size (the number of test cases in the test suite), while some others are relatively unknown, such as the variety of execution traces (how different are test cases within a test suite in terms of structural coverage level). The list of factors we describe should be of interest to researchers and practitioners alike.

Software testing is a critical part of software development. As new test cases are generated over time due to software modifications, test suite sizes may grow significantly. Because of time and resource constraints for testing, test suite minimization techniques are needed to remove those test cases from a suite that, due to code modifications over time, have become redundant with respect to the coverage of testing requirements for which they were generated. Prior work has shown that test suite minimization with respect to a given testing criterion can significantly diminish the fault detection effectiveness (FDE) of suites. We present a new approach for test suite reduction that attempts to use additional coverage information of test cases to selectively keep some additional test cases in the reduced suites that are redundant with respect to the testing criteria used for suite minimization, with the goal of improving the FDE retention of the reduced suites. We implemented our approach by modifying an existing heuristic for test suite minimization. Our experiments show that our approach can significantly improve the FDE of reduced test suites without severely affecting the extent of suite size reduction

Boosted kernel search: Framework, analysis and case studies on the economic emission dispatch problem

Various Artificial Intelligent (AI) algorithms can be applied in solving optimization problems. Among the latest Evaluation Algorithm (EA) have been developed is Differential Evolution (DE). DE is developed based on an improved Genetic Algorithm and come with different strategies for faster optimization. However, the population trapped in local optimality and premature convergence to cause in DE algorithm have cause poor performance during optimization process. To overcome the drawbacks, mixed population update and bounce back strategy were introduced to modify and improve current DE algorithm. A Himmelblau function and real case from engineering problem were used to show the performance improvements of refined DE in optimization process.

Differential Evolution (DE) is a very simple but powerful Evolutionary Algorithm (EA) for real-world applications. There are two aspects affecting the overall optimization performance significantly, one is parameter control scheme and the other is trial vector generation strategy. As all the control parameters are employed in generating trial vectors, from this perspective of view, the trial vector generation strategy dominates the overall optimization performance. Therefore, trial vector generation strategies are often designed first, and then the corresponding parameter control schemes are well-tuned thereafter. Both of them constitute the main body of a new DE variant. Here in the paper, a novel DE variant with a new designed trial vector generation strategy as well as novel parameter control scheme is proposed to tackle freight trains scheduling problem. By incorporating both a time-stamp mechanism of the external archive and a novel parameter control scheme, the new algorithm can secure better optimization performance not only on man-made benchmarks but also on real-world applications. Both the man-made benchmarks from Congress on Evolutionary Computation (CEC) test suite and the real-world freight train scheduling problem are employed in the validation of the new DE variant. The experiment results show that it is competitive with other state-of-the-art DE techniques for these optimization problems.

Combinatorial testing (CT) is an important black-box testing method. In CT, the behavior of the system under test (SUT) is affected by several parameters/components. Then CT generates a combinatorial test suite. After the user executes a test suite and starts debugging, some test cases fail and some pass. From the perspective of a black box, the failures are caused by interaction of several parameters. It will be helpful if we can identify a small set of interacting parameters that caused the failures. This paper proposes a new automatic approach to identifying faulty interactions. It uses (pseudo-Boolean) constraint solving and optimization techniques to analyze the execution results of the combinatorial test suite. Experimental results show that the method is quite efficient and it can find faulty combinatorial interactions quickly. They also shed some light on the relation between the size of test suite and the ability of fault localization.

Matching and packing problems have formed an important class of NP-hard problems. There have been a number of recently developed techniques for parameterized algorithms for these problems, including greedy localization, color-coding plus dynamic programming, and randomized divide-and-conquer. In this paper, we provide further theoretical study on the structures of these problems, and develop improved algorithmic methods that combine existing and new techniques to obtain improved algorithms for matching and packing problems. For the 3-set packing problem, we present a deterministic algorithm of time O *(4.613k), which significantly improves the previous best deterministic algorithm of time O *(12.83k). For the 3-d matching problem, we develop a new randomized algorithm of running time O *(2.323k) and a new deterministic algorithm of running time O *(2.773k). Our randomized algorithm improves the previous best randomized algorithm of running time O *(2.523k), and our deterministic algorithm significantly improves the previous best deterministic algorithm of running time O *(12.83k). Our results also imply improved algorithms for various triangle packing problems in graphs.KeywordsTime ComplexityDynamic ProgrammingMatch ProblemPacking ProblemDynamic Programming AlgorithmThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Lifetime requirements and coverage demands are emphasized in wireless sensor networks. An area coverage algorithm based on differential evolution is developed in this study to obtain a given coverage ratio [Formula: see text]. The proposed algorithm maximizes the lifetime of wireless sensor networks to monitor the area of interest. To this end, we translate continuous area coverage into classical discrete point coverage, so that the optimization process can be realized by wireless sensor networks. Based on maintaining the ε-coverage performance, area coverage algorithm based on differential evolution takes the minimal energy as optimization objective. In area coverage algorithm based on differential evolution, binary differential evolution is redeveloped to search for an improved node subset and thus meet the coverage demand. Taking into account that the results of binary differential evolution are depended on the initial value, the resulting individual is not an absolutely perfect node subset. A compensation strategy is provided to avoid unbalanced energy consumption for the obtained node subset by introducing the positive and negative utility ratios. Under the helps of those ratios and compensation strategy, the resulting node subset can be added additional nodes to remedy insufficient coverage, and redundancy active nodes can be pushed into sleep state. Furthermore, balance and residual energy are considered in area coverage algorithm based on differential evolution, which can expand the scope of population exploration and accelerate convergence. Experimental results show that area coverage algorithm based on differential evolution possesses high energy and computation efficiencies and provides 90% network coverage.

Testers often represent systems under test in input parameter models. These contain parameters with associated values. Combinations of parameter values, with one value for each parameter, are potential test cases. In most models, some values of two or more parameters cannot be combined. Testers must then detect and avoid or remove these conflicts. This paper proposes two new methods for automatically handling such conflicts and compares these with two existing methods, based on the sizes of the final conflict-free test suites. A test suite reduction method, usable with three of the four investigated methods is also included in the study, resulting in seven studied conflict handling methods. In the experiment, the number and types of conflicts, as well as the size of the input parameter model and the coverage criterion used, are varied. All in all, 3854 test suites with a total of 929,158 test cases were generated. Two methods stand out as tractable and complementary. The best method (called the avoid methods) with respect to test suite size is to avoid selection of test cases with conflicts. However, this method cannot always be used. The second best method (called the replace method), removing conflicts from the final test suite, is completely general.

Differential evolution (DE), a global optimization algorithm, is the latest addition to the repository of evolutionary algorithm (EA). In spite of its potential in solving wide range of optimization problems, researchers are continuously working on its further improvement. The three control parameters of DE are population size (NP), mutation scale factor (F), and crossover rate (Cr). Analyzing the impact of these control parameters in DE’s performance and their control mechanisms is the popular research directions in DE’s literature. The objective of this paper is to observe and analyze the performance of DE algorithm for different values of one of its parameters, mutation scale factor (F), and to present the empirical validations on the superiority of a parameter adaptive mechanism of F, on an image segmentation problem. A detailed survey of DE literature pertaining to its parameter study and parameter setting is presented, and the research gaps are identified. Then, extending the study reported by Dhanalakshmy et al. (Dhanalakshmy D.M., Jeyakumar G., Shunmuga Velayutham C: Analytical Study and Empirical Validations on the Impact of Scale Factor Parameter of Differential Evolution Algorithm. In proceedings of International Conference on Pattern Recognition and Machine Intelligence -Lecture Notes in Computer Science vol. 11941, pp 328–336, (2019).), this paper presents empirical evidences to demonstrate the impact of F on DE’s convergence nature. The observations and inferences are presented with categorical analysis based on different values of F, using a well-formulated experimental setup with 14 benchmarking functions and 8 variants of DE. The performance measurements were kept the same as used in (Dhanalakshmy D.M., Jeyakumar G., Shunmuga Velayutham C: Analytical Study and Empirical Validations on the Impact of Scale Factor Parameter of Differential Evolution Algorithm. In proceedings of International Conference on Pattern Recognition and Machine Intelligence -Lecture Notes in Computer Science vol. 11941, pp. 328–336, 2019). It is followed by solving the image segmentation problem with classical DE algorithm and the DE algorithm with adaptive F (DE_aF) (Dhanya M Dhanalakshmy, M.S. Akhila, C.R. Vidhya, G. Jeyakumar: Improving the Search Efficiency of Differential Evolution Algorithm by Population Diversity Analysis and Adaptation of Mutation Step Sizes. International Journal on Advanced Intelligence Paradigms, 15(2):119–145, 2020) and comparing their performances.KeywordsEvolutionary algorithmDifferential evolutionParameter settingScale factorNature of convergence

Graphical user interfaces (GUIs) are used as front ends to most of today's software applications. The event-driven nature of GUIs presents new challenges for testing. One important challenge is test suite reduction. Conventional reduction techniques/tools based on static analysis are not easily applicable due to the increased use of multilanguage GUI implementations, callbacks for event handlers, virtual function calls, reflection, and multithreading. Moreover, many existing techniques ignore code in libraries and fail to consider the context in which event handlers execute. Consequently, they yield GUI test suites with seriously impaired fault-detection abilities. This paper presents a reduction technique based on the call-stack coverage criterion. Call stacks may be collected for any executing program with very little overhead. Empirical studies in this paper compare reduction based on call-stack coverage to reduction based on line, method, and event coverage, including variations that control for the size and optional consideration of library methods. These studies show that call-stack-based reduction provides unique trade-offs between the reduction in test suite size and the loss of fault detection effectiveness, which may be valuable in practice. Additionally, an analysis of the relationship between coverage requirements and fault-revealing test cases is presented.