A New Script Language Applicable To Symbolic Execution Systems

Test data generation is one of the most technically challenging steps of testing software, but most commercial systems currently incorporate very little automation for this step. This paper presents results from a project that is trying to find ways to incorporate test data generation into practical test processes. The results include a new procedure for automatically generating test data that incorporates ideas from symbolic evaluation, constraint-based testing, and dynamic test data generation. It takes an initial set of values for each input, and dynamically ‘pushes’ the values through the control-flow graph of the program, modifying the sets of values as branches in the program are taken. The result is usually a set of values for each input parameter that has the property that any choice from the sets will cause the path to be traversed. This procedure uses new analysis techniques, offers improvements over previous research results in constraint-based testing, and combines several steps into one coherent process. The dynamic nature of this procedure yields several benefits. Moving through the control flow graph dynamically allows path constraints to be resolved immediately, which is more efficient both in space and time, and more often successful than constraint-based testing. This new procedure also incorporates an intelligent search technique based on bisection. The dynamic nature of this procedure also allows certain improvements to be made in the handling of arrays, loops, and expressions; language features that are traditionally difficult to handle in test data generation systems. The paper presents the test data generation procedure, examples to explain the working of the procedure, and results from a proof-of-concept implementation. Copyright © 1999 John Wiley & Sons, Ltd.

In search based test data generation, the problem of test data generation is reduced to that of function minimization or maximization.Traditionally, for branch testing, the problem of test data generation has been formulated as a minimization problem.In this paper we define an alternate maximization formulation and experimentally compare it with the minimization formulation.We use genetic algorithm and binary particle swarm optimization as the search technique and in addition to the usual operators we also employ a branch ordering strategy, memory and elitism.Results indicate that there is no significant difference in the performance or the coverage obtained through the two approaches and either could be used in test data generation when coupled with the branch ordering strategy, memory and elitism.

A software system for the generation of test data for Ada programs

In the context of test data generation, symbolic execution gets more attention as computing power increases continuously. Experiments show that test generation tools based on symbolic execution can get high coverage and find bugs on real applications. However, symbolic execution still has limitations in handling some complex program structures such as pointers, arrays and library functions. To address the problem, this study proposes a technique called lazy symbolic execution, which combines symbolic execution with a lazy evaluation strategy. The authors approach is motivated by the observation that some program structures can be reasoned about symbolically and the others have to be evaluated concretely. Traditional symbolic execution can cope with the former well, whereas lazy symbolic evaluation is used to handle the latter. However, lazy symbolic evaluation introduces intermediate variables into path constraints. To eliminate those variables, concrete values for some input variables are first obtained by constraint solving or searching processes. Then, the given path is executed again using inputs consisting of concrete and symbolic values. The procedure is repeated until all intermediate variables are wiped out. The authors have implemented a prototype tool and performed some experiments. The empirical results show the effectiveness of their approach.

Software industry produces high quality software, in that software testing plays a vital role. In software testing automatic generation can reduce the process time and cost. Automatic generation of object-oriented unit tests in software testing is a challenging task especially in white box testing. In software engineering Software testing becomes challenging issue with the development of software tools. Using Genetic Algorithm will provide best results in the software testing toolkit. This paper suggests an upgrading of automated search-based test generation of test cases in order to obtain high branch coverage in white box testing. The proposed technique outperforms providing greater efficiency in testing technique which combines Genetic Algorithm and Symbolic Execution named GASE. GASE automatically generates test data in order to obtain high branch coverage in white box testing aiming effective testing process which can be extended for software testing in cloud computing where it becomes necessity.

This paper aims at incorporating immune operators in genetic algorithms as an advanced method for solving the problem of test data generation. The new proposed hybrid algorithm is called immune genetic algorithm (IGA). A full description of this algorithm is presented before investigating its application in the context of software test data generation using some benchmark programs. Moreover, the algorithm is compared with other evolutionary algorithms.

From the perspective of industry, formal methods over-promise and under-deliver. Theoretical computer scientists love the notion of proving programs correct, but have slowly come round to the realization that promises in grant proposals aren’t the same as delivering in the real world. Essentially we started seeing a slow erosion of the importance of the notion of soundness; completeness was dropped long before. The ideal of showing that programs behave according to their specification, became the reality of finding situations where they don’t. This maps perfectly onto an expensive activity well known to industry, namely software testing. This presentation looks at the happy marriage of techniques from formal methods and software testing. Software testing is expensive since it is time-consuming to derive tests to adequately cover the software’s behavior. Techniques from formal methods allow one to generate tests automatically (hence reducing costs) and systematically (hence increasing the likelihood of discovering errors). We look at the use of software model checking to find errors in complex code, and specifically, consider the evolution of one of the world’s most popular model checkers, JavaPathFinder (JPF). One of the core techniques in JPF is symbolic execution that, although introduced in the early seventies, has recently made a big comeback in the testing world. We discuss the reasons why it took this long for such a powerful technique to become popular (again) and how it is used within JPF. In addition we discuss some of the new advances in symbolic execution and how it is used for bug finding and test generation. Finally, we consider some of the new challenges facing the automated testing field and how formal techniques can be applied to address them.

Software testing refers to a process that improves the quality of software systems through bug detection, but software testing is one of the time and cost-consuming stages in software development. Hence, software test automation is regarded as a solution, which can facilitate heavy and laborious tasks of testing. Problem: Automatic generation of data with maximum coverage of program branches is regarded as an NP-complete optimization problem. Several heuristic algorithms have been proposed for this problem. Failure to maximise branch coverage, the poor success rate in optimal test data generation, and low stable results are the major demerits of the previous methods. Goal: Enhancing the branch coverage rate of the generated test data, enhancing the success rate in generating the test data with maximum coverage, and enhancing the stability and speed criteria are the main goals of this study. Method: In this study, a combination of grey wolf optimization algorithm and genetic algorithm have been used to automatically generate optimal test data. The proposed hybrid method (Fuzuli <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> ) tries to generate test data with maximum branch coverage at the software source code level. Results: The results obtained from the proposed algorithm were compared with those of the following algorithms: Shuffled Frog Leaping Algorithm (SFLA), Artificial Bee Colony (ABC), Particle Swarm Optimization (PSO), and Genetic Algorithm (GA).The results obtained from running a wide range of tests on standard benchmark programs showed that the proposed algorithm outperforms other algorithms with an average coverage of %99.98, a success rate of %99.97, and an average output of 2.86.

Automated test data generation is a challenging problem for researchers in the area of software testing. Up until now, most of the work on test data generation is at unit level. Until level test data generation involves the execution of test path at unit level where interaction with other components is minimum. Test data generation for unit testing involves a single path and there is no usage of formal and actual parameters. The problem of automated test data generation becomes very challenging when we move to other levels of testing including integration testing and system level testing. At integration level, the variables are passed as arguments to other components and variables change their names; also multiple paths are executed from different components to ensure proper functionality. Recently evolutionary approaches have been proven a powerful tool for test data generation. In this paper, we have proposed a novel approach for test data generation for coupling based integration testing using particle swarm optimization. Up until now, there is no research for test data generation for coupling based integration testing using particle swarm optimization. Our approach takes the coupling path as input, containing different sub paths, and generates the test data using particle swarm optimization. We have also proposed architecture of tool for automation of our approach. In future, we will implement our proposed approach and will perform different experiments to prove its significance.

Chapter 10 - Constraint-based heuristic algorithms for software test generation

Application of Negative Selection Algorithm (NSA) for test data generation of path testing

Structural test data generation using a memetic ant colony optimization based on evolution strategies

We present results for the "Impact Project Focus Area" on the topic of symbolic execution as used in software testing. Symbolic execution is a program analysis technique introduced in the 70s that has received renewed interest in recent years, due to algorithmic advances and increased availability of computational power and constraint solving technology. We review classical symbolic execution and some modern extensions such as generalized symbolic execution and dynamic test generation. We also give a preliminary assessment of the use in academia, research labs, and industry. © 2011 ACM.

In search-based structural testing, metaheuristic search techniques have been frequently used to automate the test data generation. In Genetic Algorithms (GAs) for example, test data are rewarded on the basis of an objective function that represents generally the number of statements or branches covered. However, owing to the wide diversity of possible test data values, it is hard to find the set of test data that can satisfy a specific coverage criterion. In this paper, we introduce the use of Novelty Search (NS) algorithm to the test data generation problem based on statement-covered criteria. We believe that such approach to test data generation is attractive because it allows the exploration of the huge space of test data within the input domain. In this approach, we seek to explore the search space without regard to any objectives. In fact, instead of having a fitness-based selection, we select test cases based on a novelty score showing how different they are compared to all other solutions evaluated so far.

Complex SQL queries are widely used today, but it is rather difficult to check if a complex query has been written correctly. Formal verification based on comparing a specification with an implementation is not applicable, since SQL queries are essentially a specification without any implementation. Queries are usually checked by running them on sample datasets and checking that the correct result is returned; there is no guarantee that all possible errors are detected. In this paper, we address the problem of test data generation for checking correctness of SQL queries, based on the query mutation approach for modeling errors. Our presentation focuses in particular on a class of join/outer-join mutations, comparison operator mutations, and aggregation operation mutations, which are a common cause of error. To minimize human effort in testing, our techniques generate a test suite containing small and intuitive test datasets. The number of datasets generated, is linear in the size of the query, although the number of mutations in the class we consider is exponential. Under certain assumptions on constraints and query constructs, the test suite we generate is complete for a subclass of mutations that we define, i.e., it kills all non-equivalent mutations in this subclass.