Search-based software test data generation: a survey

Automated test data generation is a challenging problem for researchers in the area of software testing. Up until now, most of the work on test data generation is at unit level. Until level test data generation involves the execution of test path at unit level where interaction with other components is minimum. Test data generation for unit testing involves a single path and there is no usage of formal and actual parameters. The problem of automated test data generation becomes very challenging when we move to other levels of testing including integration testing and system level testing. At integration level, the variables are passed as arguments to other components and variables change their names; also multiple paths are executed from different components to ensure proper functionality. Recently evolutionary approaches have been proven a powerful tool for test data generation. In this paper, we have proposed a novel approach for test data generation for coupling based integration testing using particle swarm optimization. Up until now, there is no research for test data generation for coupling based integration testing using particle swarm optimization. Our approach takes the coupling path as input, containing different sub paths, and generates the test data using particle swarm optimization. We have also proposed architecture of tool for automation of our approach. In future, we will implement our proposed approach and will perform different experiments to prove its significance.

Software testing is the primary phase, which is performed during software development and it is carried by a sequence of instructions of test inputs followed by expected output.The Harmony Search (HS) algorithm is based on the improvisation process of music.In comparison to other algorithms, the HSA has gain popularity and superiority in the field of evolutionary computation.When musicians compose the harmony through different possible combinations of the music, at that time the pitches are stored in the harmony memory and the optimization can be done by adjusting the input pitches and generate the perfect harmony.The test case generation process is used to identify test cases with resources and also identifies critical domain requirements.In this paper, the role of Harmony search meta-heuristic search technique is analyzed in generating random test data and optimized those test data.Test data are generated and optimized by applying in a case study i.e. a withdrawal task in Bank ATM through Harmony search.It is observed that this algorithm generates suitable test cases as well as test data and gives brief details about the Harmony search method.It is used for test data generation and optimization.

Automatic test data generation is required to generate test cases dynamically for a specific software program. Manual generation of test data is too tedious and a time consuming task. This paper proposes a technique using Intelligent Water Drop (IWD) for automatic generation of test data. Correctly generated test data helps in reducing the effort while testing the software. This paper discusses different algorithms based on IWD to generate test data and path coverage over Control Flow Graph. Test data is generated keeping in mind all of the programming constraints like “if,” “while,” “do while,” etc., available in the program.

Structural test data generation using a memetic ant colony optimization based on evolution strategies

Test data generation is one of the most technically challenging steps of testing software, but most commercial systems currently incorporate very little automation for this step. This paper presents results from a project that is trying to find ways to incorporate test data generation into practical test processes. The results include a new procedure for automatically generating test data that incorporates ideas from symbolic evaluation, constraint-based testing, and dynamic test data generation. It takes an initial set of values for each input, and dynamically ‘pushes’ the values through the control-flow graph of the program, modifying the sets of values as branches in the program are taken. The result is usually a set of values for each input parameter that has the property that any choice from the sets will cause the path to be traversed. This procedure uses new analysis techniques, offers improvements over previous research results in constraint-based testing, and combines several steps into one coherent process. The dynamic nature of this procedure yields several benefits. Moving through the control flow graph dynamically allows path constraints to be resolved immediately, which is more efficient both in space and time, and more often successful than constraint-based testing. This new procedure also incorporates an intelligent search technique based on bisection. The dynamic nature of this procedure also allows certain improvements to be made in the handling of arrays, loops, and expressions; language features that are traditionally difficult to handle in test data generation systems. The paper presents the test data generation procedure, examples to explain the working of the procedure, and results from a proof-of-concept implementation. Copyright © 1999 John Wiley & Sons, Ltd.

Automatic test data generation from a software specification prepares test cases for software developers before their code development phase. Having test cases before coding, helps the developers to control their code to conform to the specification. In state-based specifications, paths from the initial state to the final state may be varied, this is called parallel paths. In order to generate test data which cover all behaviors in the specification, test data for each parallel path should be generated. This paper proposes an enhanced genetic algorithm(GA)-based approach to resolve the parallel paths from the UML state machine diagram. The proposed approach is improved from our previous study which uses a GA based test data generation method for only one path. The approach identifies parallel paths to cover all transitions. GA evolves a number of suitable test data sets - one test data set for each parallel path. The best test data for each parallel path are picked and used for calculating the overall coverage test data. The experimental results show improved coverage results with the enhanced approach for a number of case studies with parallel paths.

Conventional software testing is time consuming and expensive task. The techniques for automated test case generation of software is very important as it can reduce the time and cost of this process. Search based test data generation received lot of attention in last two decades for solving this problem. The well known metaheuristic search technique genetic algorithm is used to obtain a specific coverage in software testing. This technique automatically generates test data in order to obtain branch coverage in software testing. The authors first use a technique to identify the unique paths from a control flow graph and then apply genetic algorithm for test case generation. This paper defines the basic research issues of genetic algorithm like chromosome, fitness function, crossing over, mutation in this problem domain and uses a case study to illustrate it.

Evolutionary Testing has been shown a promising technology for the automatic test data generation. It reformulates test data generation as a metaheuristic search. A well- designed fitness function is essential to the efficiency of evolutionary search. Many efforts have been directed at the design and implementation of fitness function in recent years. However, previous work has just focused on the control dependency of the target on the branches in the same function. When function calls exist in the desired execution trace to the target, the evaluation of the test data on the coverage of these function calls, which should be provided to the evolutionary search, is not captured by the existing fitness function. In this case, the existing fitness function can not fairly evaluate the test data. And the evolutionary search will be hampered or even fail in severe cases. In this paper, a new term is first proposed to incorporate into the existing fitness function. It is applied to evaluate the test data's coverage of function calls along the desired path to the target. The new fitness function can evaluate the test data more fairly, resulting in a better guidance to the evolutionary search. This can be seen by the experiments carried out on two cases.

Software testing refers to a process that improves the quality of software systems through bug detection, but software testing is one of the time and cost-consuming stages in software development. Hence, software test automation is regarded as a solution, which can facilitate heavy and laborious tasks of testing. Problem: Automatic generation of data with maximum coverage of program branches is regarded as an NP-complete optimization problem. Several heuristic algorithms have been proposed for this problem. Failure to maximise branch coverage, the poor success rate in optimal test data generation, and low stable results are the major demerits of the previous methods. Goal: Enhancing the branch coverage rate of the generated test data, enhancing the success rate in generating the test data with maximum coverage, and enhancing the stability and speed criteria are the main goals of this study. Method: In this study, a combination of grey wolf optimization algorithm and genetic algorithm have been used to automatically generate optimal test data. The proposed hybrid method (Fuzuli <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> ) tries to generate test data with maximum branch coverage at the software source code level. Results: The results obtained from the proposed algorithm were compared with those of the following algorithms: Shuffled Frog Leaping Algorithm (SFLA), Artificial Bee Colony (ABC), Particle Swarm Optimization (PSO), and Genetic Algorithm (GA).The results obtained from running a wide range of tests on standard benchmark programs showed that the proposed algorithm outperforms other algorithms with an average coverage of %99.98, a success rate of %99.97, and an average output of 2.86.

A hybrid particle swarm genetic algorithm is purposed to apply in software testing using case automated generations. On the basis of classical genetic algorithm, the algorithm divided the population into “families”, influencing the convergence efficiency by crossover in family, keeping the diversity of the population by crossover between families; meanwhile, enhancing the speed of convergence by the PSO crossover (commixed the thought of PSO in genetic algorithm) According to the characteristics of software testing problems, we designed the corresponding fitness function and the encoding method. The results of data experiment were given to illustrate the effectiveness of the algorithm.

In search based test data generation, the problem of test data generation is reduced to that of function minimization or maximization.Traditionally, for branch testing, the problem of test data generation has been formulated as a minimization problem.In this paper we define an alternate maximization formulation and experimentally compare it with the minimization formulation.We use genetic algorithm and binary particle swarm optimization as the search technique and in addition to the usual operators we also employ a branch ordering strategy, memory and elitism.Results indicate that there is no significant difference in the performance or the coverage obtained through the two approaches and either could be used in test data generation when coupled with the branch ordering strategy, memory and elitism.

In search-based structural testing, metaheuristic search techniques have been frequently used to automate the test data generation. In Genetic Algorithms (GAs) for example, test data are rewarded on the basis of an objective function that represents generally the number of statements or branches covered. However, owing to the wide diversity of possible test data values, it is hard to find the set of test data that can satisfy a specific coverage criterion. In this paper, we introduce the use of Novelty Search (NS) algorithm to the test data generation problem based on statement-covered criteria. We believe that such approach to test data generation is attractive because it allows the exploration of the huge space of test data within the input domain. In this approach, we seek to explore the search space without regard to any objectives. In fact, instead of having a fitness-based selection, we select test cases based on a novelty score showing how different they are compared to all other solutions evaluated so far.

Due to the frequent non-existence of an automated oracle, test cases are often evaluated manually in practice. However, this fact is rarely taken into account by automatic test data generators, which seek to maximise a program's structural coverage only. The test data produced tends to be of a poor fit with the program's operational profile. As a result, each test case takes longer for a human to check, because the scenarios that arbitrary-looking data represent require time and effort to understand. This short paper proposes methods to extracting knowledge from programmers, source code and documentation and its incorporation into the automatic test data generation process so as to inject the realism required to produce test cases that are quick and easy for a human to comprehend and check. The aim is to reduce the so-called qualitative human oracle costs associated with automatic test data generation. The potential benefits of such an approach are demonstrated with a simple case study.

This paper proposes a genetic algorithm-based method to generate test cases. This method provides information for test case generation using state machine diagrams. Its feature is realizing automation through fewer generated test cases. In terms of automatic generation of test data based on path coverage, the goal is to build a function that can excellently assess the generated test data and guide the genetic algorithms to find the targeting parameter values.

Automatic Programming Assessment (APA) is one of the vital methods that has been applied around Computer Science education in realizing automated marking and grading on students’ programming exercises or assignments. APA is fundamentally relying upon a test data generation process to perform a dynamic testing. Recently in Software Testing (ST) research, it has been proven that the adoption of any Meta-Heuristic Search Techniques (MHSTs) is able to improve the efficiency of generating adequate and optimal test data. Unfortunately, current studies on APA have not yet usefully incorporated the techniques to include a better quality program testing coverage by considering the optimal size of generated test data. Thus, our study propose a method of generating and locating an adequate test data with optimal in size by adapting a MHST to satisfy the dynamic-functional testing in APA (or is called DyFunFPA-TDG method). In this paper, we merely focus on revealing the design of the method with a sample of the generated test cases to be used in APA. This method able to assist educators of elementary programming courses to provide the means of deriving and generating adequate test data with optimal in size regardless of having the expertise in specific knowledge of test cases design.KeywordsAutomatic programming assessmentTest data generationDynamic testingFlower pollination algorithm