Generating Test Data for Path Coverage Based Testing Using Genetic Algorithms

Test data generation is one of the most technically challenging steps of testing software, but most commercial systems currently incorporate very little automation for this step. This paper presents results from a project that is trying to find ways to incorporate test data generation into practical test processes. The results include a new procedure for automatically generating test data that incorporates ideas from symbolic evaluation, constraint-based testing, and dynamic test data generation. It takes an initial set of values for each input, and dynamically ‘pushes’ the values through the control-flow graph of the program, modifying the sets of values as branches in the program are taken. The result is usually a set of values for each input parameter that has the property that any choice from the sets will cause the path to be traversed. This procedure uses new analysis techniques, offers improvements over previous research results in constraint-based testing, and combines several steps into one coherent process. The dynamic nature of this procedure yields several benefits. Moving through the control flow graph dynamically allows path constraints to be resolved immediately, which is more efficient both in space and time, and more often successful than constraint-based testing. This new procedure also incorporates an intelligent search technique based on bisection. The dynamic nature of this procedure also allows certain improvements to be made in the handling of arrays, loops, and expressions; language features that are traditionally difficult to handle in test data generation systems. The paper presents the test data generation procedure, examples to explain the working of the procedure, and results from a proof-of-concept implementation. Copyright © 1999 John Wiley & Sons, Ltd.

Software test-data generation is the process of identifying a set of data, which satisfies a given testing criterion. For solving this difficult problem there were a lot of research works, which have been done in the past. The most commonly encountered are random test-data generation, symbolic test-data generation, dynamic test-data generation, and recently, test-data generation based on genetic algorithms. This paper gives a survey of the majority of software test-data generation techniques based on genetic algorithms. It compares and classifies the surveyed techniques according to the genetic algorithms features and parameters. Also, this paper shows and classifies the limitations of these techniques.

The first step of decompilation and other reverse analysis for binary codes is constructing control flow graph of program. Due to the loss of structure in compiled code, extracting a control flow graph from an executable is not simply a matter. The usual way which scans the branch instructions in the disassembly file and creates the CFG can not suitable for multiply architecture and file format. This paper analyzes the structure of assembly program and designs a data information description language based on XML-reverse meta language, which is to describe structured disassembling information and then brings forward the control flow graph reconstruction algorithm. This method abstracts disassembly representations and supports multiple architectures. The testing result shows that this method can reconstruct control flow graph of binaries effectively.

In search based test data generation, the problem of test data generation is reduced to that of function minimization or maximization.Traditionally, for branch testing, the problem of test data generation has been formulated as a minimization problem.In this paper we define an alternate maximization formulation and experimentally compare it with the minimization formulation.We use genetic algorithm and binary particle swarm optimization as the search technique and in addition to the usual operators we also employ a branch ordering strategy, memory and elitism.Results indicate that there is no significant difference in the performance or the coverage obtained through the two approaches and either could be used in test data generation when coupled with the branch ordering strategy, memory and elitism.

Software testing refers to a process that improves the quality of software systems through bug detection, but software testing is one of the time and cost-consuming stages in software development. Hence, software test automation is regarded as a solution, which can facilitate heavy and laborious tasks of testing. Problem: Automatic generation of data with maximum coverage of program branches is regarded as an NP-complete optimization problem. Several heuristic algorithms have been proposed for this problem. Failure to maximise branch coverage, the poor success rate in optimal test data generation, and low stable results are the major demerits of the previous methods. Goal: Enhancing the branch coverage rate of the generated test data, enhancing the success rate in generating the test data with maximum coverage, and enhancing the stability and speed criteria are the main goals of this study. Method: In this study, a combination of grey wolf optimization algorithm and genetic algorithm have been used to automatically generate optimal test data. The proposed hybrid method (Fuzuli <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> ) tries to generate test data with maximum branch coverage at the software source code level. Results: The results obtained from the proposed algorithm were compared with those of the following algorithms: Shuffled Frog Leaping Algorithm (SFLA), Artificial Bee Colony (ABC), Particle Swarm Optimization (PSO), and Genetic Algorithm (GA).The results obtained from running a wide range of tests on standard benchmark programs showed that the proposed algorithm outperforms other algorithms with an average coverage of %99.98, a success rate of %99.97, and an average output of 2.86.

Automatic test data generation is required to generate test cases dynamically for a specific software program. Manual generation of test data is too tedious and a time consuming task. This paper proposes a technique using Intelligent Water Drop (IWD) for automatic generation of test data. Correctly generated test data helps in reducing the effort while testing the software. This paper discusses different algorithms based on IWD to generate test data and path coverage over Control Flow Graph. Test data is generated keeping in mind all of the programming constraints like “if,” “while,” “do while,” etc., available in the program.

The optimal regression testing problem is that of determining the minimum number of test cases needed for revalidating modified software in the maintenance phase. The present two natural optimization algorithms, namely simulated annealing and genetic algorithms, for solving this problem. The algorithms are based on an integer programming problem formulation and the program's control-flow graph. The main advantage of these algorithms is that they do not suffer from exponential explosion for realistic program sizes. The experimental results show that they find optimal or near-optimal number of retests in a reasonable time.

The main task of software testing is the generation of test data with high quality in restricted time. But for software with uncertainties, such as randomness, existing methods of generating test data often lose their effectiveness. For software with randomness, a test datum might cover the test target in one run, but obtains different results in the next run. Therefore, special testing methods must be exploited for this kind of software. In order to test software with randomness, we first propose a novel test adequacy criterion, and then build a mathematical model for generating test data according to the new criterion. Finally, we present a method of solving the optimization model using a set-based genetic algorithm. We apply the proposed method to test 12 programs, and compare with traditional genetic algorithm and the random method. From the experimental results we can see that, the proposed adequacy criterion is available for the software with randomness and the proposed algorithm can effectively generate test data.

Side-channels in software are an increasingly significant threat to the confidentiality of private user information, and the static detection of such vulnerabilities is a key challenge in secure software development. In this paper, we introduce a new technique for scalable detection of side- channels in software. Given a program and a cost model for a side-channel (such as time or memory usage), we decompose the control flow graph of the program into nested branch and loop components, and compositionally assign a symbolic cost expression to each component. Symbolic cost expressions provide an over-approximation of all possible observable cost values that components can generate. Queries to a satisfiability solver on the difference between possible cost values of a component allow us to detect the presence of imbalanced paths (with respect to observable cost) through the control flow graph. When combined with taint analysis that identifies conditional statements that depend on secret information, our technique answers the following question: Does there exist a pair of paths in the program's control flow graph, differing only on branch conditions influenced by the secret, that differ in observable side-channel value by more than some given threshold? Additional optimization queries allow us to identify the minimal number of loop iterations necessary for the above to hold or the maximal cost difference between paths in the graph. We perform symbolic execution based feasibility analyses to eliminate control flow paths that are infeasible. We implemented our techniques in a prototype, and we demonstrate its favourable performance against state-of-the-art tools as well as its effectiveness and scalability on a set of sizable, realistic Java server-client and peer-to-peer applications.

This paper describes a PhD proposal aiming at dealing with obfuscation in polymorphic virus. The major characteristic of polymorphic virus is the capability of infinitely self-modifying when infecting victim programs. It makes the traditional signature-based virus detection technique ineffective since this approach needs to collect all of signature instances. A recent emerging approach to counter this problem is abstracting the program from binary level, then extracting an abstracted model for further analysis. The most common model to be extracted is perhaps the control flow graph (CFG) of the binary program. However, this control-based abstraction approach is currently suffering from some advanced obfuscation techniques which change not only the signatures but also modify significantly the control flow of the programs. Thus, the control analysis will become quickly too complicated. Hence, we propose a novel approach of abstracting the binary code based on memory states. This approach allows us to detect useless instructions which are part of obfuscation code. Moreover, for the next step, our approach can be extended as new efficient technique for virus detection based on common abstracted pattern.

Control-Flow Integrity (CFI) enforces a control-flow graph (CFG) to limit attackers' ability to manipulate runtime control flow. CFI variations, enforcing different CFGs, achieve different degrees of attack surface reduction. To compare the security strength of different CFI policies, measuring the remaining attack surface is critical but challenging. Therefore, we propose MazeRunner, a framework that quantitatively estimates the attack surface of a CFI-hardened program. Methodology-wise, it takes a program's CFG, an attack model, and a security-violation policy as input to discover risky program points by an attack-aware data dependency tracking algorithm. Risky program points and the CFG are used to compute a metric for the remaining attack surface. We evaluate MazeRunner with 3 CFG types, 3 attack models, and 4 security-violation policies against 13 realistic benchmarks, and demonstrate that the new metric achieves higher precision than traditional metrics while maintaining completeness.

Conventional software testing is time consuming and expensive task. The techniques for automated test case generation of software is very important as it can reduce the time and cost of this process. Search based test data generation received lot of attention in last two decades for solving this problem. The well known metaheuristic search technique genetic algorithm is used to obtain a specific coverage in software testing. This technique automatically generates test data in order to obtain branch coverage in software testing. The authors first use a technique to identify the unique paths from a control flow graph and then apply genetic algorithm for test case generation. This paper defines the basic research issues of genetic algorithm like chromosome, fitness function, crossing over, mutation in this problem domain and uses a case study to illustrate it.

Structural Statistical Software Testing exploits the control flow graph of the program being tested to construct test cases. While test cases can easily be extracted from {em feasible paths} in the control flow graph, that is, paths which are actually exerted for some values of the program input, the feasible path region is a tiny fraction of the graph paths (less than $10^{-5}]$ for medium size programs). The S4T algorithm presented in this paper aims to address this limitation; as an Active Relational Learning Algorithm, it uses the few feasible paths initially available to sample new feasible paths. The difficulty comes from the non-Markovian nature of the feasible path concept, due to the long-range dependencies between the nodes in the control flow graph. Experimental validation on real-world and artificial problems demonstrates significant improvements compared to the state of the art.

Regression testing aims at showing that code has not been adversely affected by modification activities during maintenance. Regression test selection techniques reuse tests from an existing test suite to test a modified program. By reusing such a test suite to retest modified programs, maintainers or testers can reduce the required testing effort. This paper presents a regression test selection technique for AspectJ programs. The technique is based on various types of control flow graphs that can be used to select from the original test suite test cases that execute changed code for the new version of the AspectJ program. The code-base technique operates on the control flow graphs of AspectJ programs. The technique can be applied to modified individual aspects or classes as well as the whole program that uses modified aspects or classes.

Programmable Logic Controller (PLC) programs are vulnerable to tampering attacks with addition of malware, which can substantially cause severe physical destructions. In order to solve the problems, We propose a static analysis method of Binary PLC programs for the detection of PLC malwares. Firstly, disassemble the binary program into STL program. Secondly, constructed the control flow graph of STL program according to the characteristics of STL language. Thirdly, generate the program execution paths according to traverse the control flow graph and analysis loop paths. Finally the PLC program output and input mapping relations are obtained based on the program execution paths. The experiment shows that the proposed method has achieved the analysis of PLC programs and get each path predicate and the mapping relation between output and input.