Make test-zesti: A symbolic execution solution for improving regression testing

Software Testing is an important aspect of the real time software development process. Software testing always assures the quality of software product. As associated with software testing, there are few very important issues where there is a need to pay attention on it in the process of software development test. These issues are generation of effective test case and test suite as well as optimization of test case and suite while doing testing of software product. The important issue is that testing time of the test case and test suite. It is very much important that after development of software product effective testing should be performed. So to overcome these issues of optimization, we have proposed new approach for test suite optimization using genetic algorithm (GA). Genetic algorithm is evolutionary in nature so it is often used for optimization of problem by researcher. In this paper, our aim is to study various selections methods like tournament selection, rank selection and roulette wheel selection and then we apply this genetic algorithm (GA) on various programs which will generate optimized test suite with parameters like fitness value of test case, test suite and take minimum amount of time for execution after certain preset generation. In this paper our main objectives as per the experimental investigation, we show that tournament selection works very fine as compared to other methods with respect fitness selection of test case and test suites, testing time of test case and test suites as well as number of requirements.

By respecting program control-flow, flow-sensitive pointer analysis promises more precise results than its flow-insensitive counterpart. However, existing heap abstractions for C and C++ flow-sensitive pointer analyses model the heap by creating a single abstract heap object for each memory allocation. Two runtime heap objects which originate from the same allocation site are imprecisely modelled using one abstract object, which makes them share the same imprecise points-to sets and thus reduces the benefit of analysing heap objects flow-sensitively. On the other hand, equipping flow-sensitive analysis with context-sensitivity, whereby an abstract heap object would be created (cloned) per calling context, can yield a more precise heap model, but at the cost of uncontrollable analysis overhead when analysing larger programs. This paper presents TypeClone, a new type-based heap model for flow-sensitive analysis. Our key insight is to differentiate concrete heap objects lazily using type information at use sites within the program control-flow (e.g., when accessed via pointer dereferencing) for programs which conform to the strict aliasing rules set out by the C and C++ standards. The novelty of TypeClone lies in its lazy heap cloning: an untyped abstract heap object created at an allocation site is killed and replaced with a new object (i.e. a clone), uniquely identified by the type information at its use site, for flow-sensitive points-to propagation. Thus, heap cloning can be performed within a flow-sensitive analysis without the need for context-sensitivity. Moreover, TypeClone supports new kinds of strong updates for flow-sensitive analysis where heap objects are filtered out from imprecise points-to relations at object use sites according to the strict aliasing rules. Our method is neither strictly superior nor inferior to context-sensitive heap cloning, but rather, represents a new dimension that achieves a sweet spot between precision and efficiency. We evaluate our analysis by comparing TypeClone with state-of-the-art sparse flow-sensitive points-to analysis using the 12 largest programs in GNU Coreutils. Our experimental results also confirm that TypeClone is more precise than flow-sensitive pointer analysis and is able to, on average, answer over 15% more alias queries with a no-alias result.

Software testing is a fundamental step in the software development lifecycle. Its purpose is to evaluate the quality of software applications. Regression testing is an important testing methodology in software testing. The purpose of regression testing is to validate the software after each change of its code. This involves adding new test cases to the test suite and running the test suite as the software changes, making the test suite larger. The cost and time of the project are affected by the test suite size. The challenge is to run regression testing with a smaller number of test cases and larger amount of software coverage. Minimization of the test suite with maximization of the software coverage is an NP-complete problem. Search-based software engineering is an important topic in software engineering, which addresses software engineering optimization problems to find the optimal/approximate solution of the given problem. This study investigated an approach to reducing the regression testing effort and saving time. It also solved the regression testing optimization problem by achieving the maximum test suite coverage with the minimum test suite size. Several experiments were conducted to obtain the optimal solutions for the regression testing problem. We propose an optimization methodology that combines a genetic algorithm and a greedy algorithm to optimize regression testing by respectively maximizing the software test coverage and minimizing the test suite size. The proposed methodology can conveniently deliver fault-free, fully covered, and powerful programs for mission-critical functions. It can be applied to test a real-time system that has high requirements for reliability, security, and safety.

Testing is an important activity in engineering of industrial embedded software. In certain application domains (e.g., railway industry) engineering software is certified according to safety standards that require extensive software testing procedures to be applied for the development of reliable systems. Mutation analysis is a technique for creating faulty versions of a software for the purpose of examining the fault detection ability of a test suite. Mutation analysis has been used for evaluating existing test suites, but also for generating test suites that detect injected faults (i.e., mutation testing). To support developers in software testing, we propose a technique for producing test cases using an automated test generation approach that operates using mutation testing for software written in IEC 61131-3 language, a programming standard for safety-critical embedded software, commonly used for Programmable Logic Controllers (PLCs). This approach uses the Uppaal model checker and is based on a combined model that contains all the mutants and the original program. We applied this approach in a tool for testing industrial PLC programs and evaluated it in terms of cost and fault detection. For realistic validation we collected industrial experimental evidence on how mutation testing compares with manual testing as well as automated decision-coverage adequate test generation. In the evaluation, we used manually seeded faults provided by four industrial engineers. The results show that even if mutation-based test generation achieves better fault detection than automated decision coverage-based test generation, these mutation-adequate test suites are not better at detecting faults than manual test suites. However, the mutation-based test suites are significantly less costly to create, in terms of testing time, than manually created test suites. Our results suggest that the fault detection scores could be improved by considering some new and improved mutation operators (e.g., Feedback Loop Insertion Operator (FIO)) for PLC programs as well as higher-order mutations.

The availability of effective test suites is critical for the development and maintenance of reliable software systems. To increase test effectiveness, software developers tend to employ larger and larger test suites. The recent availability of software tools for automatic test generation makes building large test suites affordable, therefore contributing to accelerating this trend. However, large test suites, though more effective, are resources and time consuming and therefore cannot be executed frequently. Reducing them without decreasing code coverage is a needed compromise between efficiency and effectiveness of the test, hence enabling a more regular check of the software under development. We propose a novel approach, namely REDUNET, to reduce a test suite while keeping the same code coverage. We integrate this approach in a complete framework for the automatic generation of efficient and effective test suites, which includes test suite generation, code coverage analysis, and test suite reduction. Our approach formulates the test suite reduction as a set cover problem and applies integer linear programming and a network-based optimisation, which takes advantage of the properties of the control flow graph. We find the optimal set of test cases that keeps the same code coverage in fractions of seconds on real software projects and test suites generated automatically by Randoop. The results on ten real software systems show that the proposed approach finds the optimal minimisation and achieves up to 90% reduction and more than 50% reduction on all systems under analysis. On the largest project our reduction algorithm performs more than three times faster than both integer linear programming alone and the state-of-the-art heuristic Harrold Gupta Soffa.

We present results for the "Impact Project Focus Area" on the topic of symbolic execution as used in software testing. Symbolic execution is a program analysis technique introduced in the 70s that has received renewed interest in recent years, due to algorithmic advances and increased availability of computational power and constraint solving technology. We review classical symbolic execution and some modern extensions such as generalized symbolic execution and dynamic test generation. We also give a preliminary assessment of the use in academia, research labs, and industry. © 2011 ACM.

Software testing is the dominant method for quality assurance and control in software engineering [1] , [2] . Test suites serve as quality gates to safeguard against programming faults. But not every test suite is written equally. We usually gauge its quality using metrics such as code coverage. These assess how much of the code base has been covered. However, they do not tell if the tests actually test and verify the intentions. Mutation testing does this by deliberately injecting faults into the system under test and verifying how many of them the test suite can detect. For every injected fault that is not detected by the test suite, an additional test should be written. In the academic community, mutation testing is acknowledged as the most promising technique for automated assessment of the strength of a test suite [3] , [4] .

Software testing is a critical part of software development. As new test cases are generated over time due to software modifications, test suite sizes may grow significantly. Because of time and resource constraints for testing, test suite minimization techniques are needed to remove those test cases from a suite that, due to code modifications over time, have become redundant with respect to the coverage of testing requirements for which they were generated. Prior work has shown that test suite minimization with respect to a given testing criterion can significantly diminish the fault detection effectiveness (FDE) of suites. We present a new approach for test suite reduction that attempts to use additional coverage information of test cases to selectively keep some additional test cases in the reduced suites that are redundant with respect to the testing criteria used for suite minimization, with the goal of improving the FDE retention of the reduced suites. We implemented our approach by modifying an existing heuristic for test suite minimization. Our experiments show that our approach can significantly improve the FDE of reduced test suites without severely affecting the extent of suite size reduction

In the software industry, software testing becomes more important in the entire software development life cycle. Software testing is one of the fundamental components of software quality assurances. Software Testing Life Cycle (STLC)is a process involved in testing the complete software, which includes Regression Testing, Unit Testing, Smoke Testing, Integration Testing, Interface Testing, System Testing & etc. In the STLC of Regression testing, test case selection is one of the most important concerns for effective testing as well as cost of the testing process. During the Regression testing, executing all the test cases from existing test suite is not possible because that takes more time to test the modified software. This paper proposes new Hybrid approach that consists of modified Greedy approach for handling the test case selection and Genetic Algorithm uses effective parameter like Initial Population, Fitness Value, Test Case Combination, Test Case Crossover and Test Case Mutation for optimizing the tied test suite. By doing this, effective test cases are selected and minimized the tied test suite to reduce the cost of the testing process. Finally the result of proposed approach compared with conventional greedy approach and proved that our approach is more effective than other existing approach.

Symbolic execution is widely used in software test cases automatic generating, but itpsilas hard to use it in embedded software testing, for embedded software is related to hardware too close. And at present, embedded software testing tools can only generate flow chart and test cases' coverage information, can not generate test cases that satisfy designated coverage automatically. This paper brings forward an idea that realizes embedded software test cases automatic generating using symbolic execution. First translate the embedded program into an intermediate language program, which preserve all the necessary information from the origin embedded program and according hardware, then symbolic execute the intermediate language program to generate test cases automatically.

From the perspective of industry, formal methods over-promise and under-deliver. Theoretical computer scientists love the notion of proving programs correct, but have slowly come round to the realization that promises in grant proposals aren’t the same as delivering in the real world. Essentially we started seeing a slow erosion of the importance of the notion of soundness; completeness was dropped long before. The ideal of showing that programs behave according to their specification, became the reality of finding situations where they don’t. This maps perfectly onto an expensive activity well known to industry, namely software testing. This presentation looks at the happy marriage of techniques from formal methods and software testing. Software testing is expensive since it is time-consuming to derive tests to adequately cover the software’s behavior. Techniques from formal methods allow one to generate tests automatically (hence reducing costs) and systematically (hence increasing the likelihood of discovering errors). We look at the use of software model checking to find errors in complex code, and specifically, consider the evolution of one of the world’s most popular model checkers, JavaPathFinder (JPF). One of the core techniques in JPF is symbolic execution that, although introduced in the early seventies, has recently made a big comeback in the testing world. We discuss the reasons why it took this long for such a powerful technique to become popular (again) and how it is used within JPF. In addition we discuss some of the new advances in symbolic execution and how it is used for bug finding and test generation. Finally, we consider some of the new challenges facing the automated testing field and how formal techniques can be applied to address them.

A simulation study on some heuristics for test suite reduction

Software testing is one of the important stages of software development. In software development, developers always depend on testing to reveal bugs. In the maintenance stage test suite size grow because of integration of new technique. Addition of new technique force to create new test case which increase the size of test suite. In regression testing new test case may be added to the test suite during the whole testing process. These additions of test cases create possibility of presence of redundant test cases. Due to limitation of time and resource, reduction techniques should be used to identify and remove them. Research shows that a subset of the test case in a suit may still satisfy all the test objectives which is called as representative set. Redundant test case increase the execution cost of the test suite, in spite of NP-completeness of the problem there are few good reduction techniques have been available. In this paper a new approach for test case reduction is proposed. This algorithm use genetic algorithm technique iteratively with varying chromosome length to reduce test case in a test suit by finding representative set of test cases that are fulfill the testing criteria.

Software testing is important and time-consuming. A test suite, i.e., a set of test cases, plays a key role in validating the expected program behavior. In modern test-driven development, a test suite pushes the development progress. Software evolves over time; its test suite is executed to detect whether a new code change adds bugs to the existing code. Executing all test cases after each code change is unnecessary and may be impossible due to the limited development cycle. On the one hand, multiple test cases may focus on an identical piece of code; then several test cases cannot detect extra bugs. On the other hand, even executing a test suite once in a large project takes around one hour [1]; frequent code changes require much time for conducting testing. For instance, in Hadoop, a framework of distributed computing, 2,847 version commits are accepted within one year from September 2014 with a peak of 135 commits in one week [2].

Often the hardest part of coming to grips with any new development tool is knowing where to begin. Typically, this problem is worse when the tool offers as many choices as Spring. Fortunately, getting started with Spring isn’t actually that hard if you know where to look first. In this chapter, we present you with all the basic knowledge you need to get off to a flying start. Specifically, we will look at the following: Obtaining Spring: The first logical step is to obtain or build the Spring JAR files. If you want to get up and running quickly with the standard Spring distribution, simply download the latest Spring distribution from the Spring web site at www.springframework.org. However, if you want to be on the cutting edge of Spring developments, check out the latest version of the source code from Spring’s GitHub repository. If you use Maven for application development, you can simply add the dependencies for Spring into the project’s pom.xml (project object model) file, and Maven will download the JAR files for you. Please refer to the section “Spring Modules on the Maven Repository” for details. Spring packaging options: Spring packaging is modular; it allows you to pick and choose which components you want to use in your application and to include only those components when you are distributing your application. Spring dependencies: The full distribution of Spring includes a voluminous set of dependencies, but in many cases, you need only a subset of these dependencies. In this section, we look at which Spring features require which dependencies; this information helps you reduce the size of your application to the absolute minimum. Spring samples: Spring comes with a large selection of sample applications that make ideal reference points for building your own applications. In this section, we will take a look inside the sample applications to give you a feel for the amount of sample code that is available. If you couple this with the sample application you build during the course of this book, you should have more than enough of a codebase from which to start building your own applications. Test suite and documentation: One of the things members of the Spring community are most proud of is their comprehensive test suite and documentation set. Testing is a big part of what the team does. By using Clover (www.atlassian.com/software/clover), the team actively monitors the percentage of test coverage and is constantly striving to push this percentage higher. The documentation set provided with the standard distribution is also excellent. Putting a spring into “Hello World!”: All bad punning aside, we think the best way to get started with any new programming tool is to dive right in and write some code. We are going to look at some simple examples, including a full DI-based implementation of everyone’s favorite, “Hello World!” Don’t be alarmed if you don’t understand all the code examples right away; full discussions follow later in the book. KeywordsConfiguration FileSample ApplicationSpring ModulePublic ClassImplementation ClassThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.