Regression Test Reduction for Object-Oriented Software: A Control Call Graph Based Technique and Associated Tool

In the software industry, software testing becomes more important in the entire software development life cycle. Software testing is one of the fundamental components of software quality assurances. Software Testing Life Cycle (STLC)is a process involved in testing the complete software, which includes Regression Testing, Unit Testing, Smoke Testing, Integration Testing, Interface Testing, System Testing & etc. In the STLC of Regression testing, test case selection is one of the most important concerns for effective testing as well as cost of the testing process. During the Regression testing, executing all the test cases from existing test suite is not possible because that takes more time to test the modified software. This paper proposes new Hybrid approach that consists of modified Greedy approach for handling the test case selection and Genetic Algorithm uses effective parameter like Initial Population, Fitness Value, Test Case Combination, Test Case Crossover and Test Case Mutation for optimizing the tied test suite. By doing this, effective test cases are selected and minimized the tied test suite to reduce the cost of the testing process. Finally the result of proposed approach compared with conventional greedy approach and proved that our approach is more effective than other existing approach.

Test case prioritization is used in regression testing to schedule the execution order of test cases so as to expose faults earlier in testing. Over the past few years, many test case prioritization techniques have been proposed in the literature. Most of these techniques require data on dynamic execution in the form of code coverage information for test cases. However, the collection of dynamic code coverage information on test cases has several associated drawbacks including cost increases and reduction in prioritization precision. In this paper, we propose an approach to prioritizing test cases in the absence of coverage information that operates on Java programs tested under the JUnit framework-an increasingly popular class of systems. Our approach, JUnit test case Prioritization Techniques operating in the Absence of coverage information (JUPTA), analyzes the static call graphs of JUnit test cases and the program under test to estimate the ability of each test case to achieve code coverage, and then schedules the order of these test cases based on those estimates. To evaluate the effectiveness of JUPTA, we conducted an empirical study on 19 versions of four Java programs ranging from 2K-80K lines of code, and compared several variants of JUPTA with three control techniques, and several other existing dynamic coverage-based test case prioritization techniques, assessing the abilities of the techniques to increase the rate of fault detection of test suites. Our results show that the test suites constructed by JUPTA are more effective than those in random and untreated test orders in terms of fault-detection effectiveness. Although the test suites constructed by dynamic coverage-based techniques retain fault-detection effectiveness advantages, the fault-detection effectiveness of the test suites constructed by JUPTA is close to that of the test suites constructed by those techniques, and the fault-detection effectiveness of the test suites constructed by some of JUPTA's variants is better than that of the test suites constructed by several of those techniques.

It is the demand of our ever-advancing IT industry that software be updated in order to continue its use. Such a modification should not introduce any unwanted new faults in the system. For this, the existing test suite needs to be rerun, often called as regression testing. The main challenge during the regression testing process is not to exceed the desired time and budge deadlines. As a consequence various techniques such as test case selection, minimization and prioritization are used. This paper proposes and analyzes the effect of time constraint on an ant colony optimization based technique for Regression test selection and prioritization. It has been found that with an increase in the applied time constraint, there are more chances to get an optimum selected and prioritized test suite. Also it was found that the complexity of our algorithm depends on the size of the test suite and the applied time constraint and is independent of the number of faults being mutated or any other input variable.

One of the most expensive activities that occurs as software is developed and maintained is the testing (or retesting) of the software after it has been modified. Studies suggest that a significant portion of development and maintenance costs go to this retesting, which is known as regression testing. Reports estimate that regression testing consumes as much as 80% of the overall testing budget and can consume up to 50% of the cost of software maintenance. Rapidly changing software and computing environments present many challenges for effective and efficient regression testing in practice. Regression testing can be performed after changes are made to the software, such as after nightly or regular builds, before a new version of the software is released, every time the software is saved and compiled, such as in an agile development environment, or before patches, such as security patches, are released. Regardless of the environment or when it is performed, the goals of regression testing are the same: to improve confidence that the changes behave as intended and that they have not adversely affected unchanged parts of the software. Because regression testing is important, but expensive, much research has been performed, both in industry and in academia, to develop techniques to make regression testing more effective and efficient. This research has also produced many tools and systems that have been used for empirical studies that investigate the effectiveness, scalability, and practicality of the techniques. Researchers have developed techniques for addressing a number of issues related to regression testing, and, in this talk, I will discuss them in four areas. First, techniques attempt to reduce the regression testing time by creating effective regression test suites that test the changed part of the software, by identifying test cases in the regression test suite that do not need to be rerun on the changed software, and by identifying and removing obsolete test cases. Second, techniques can reuse test suites created for one version of the software by identifying those test cases that need to be rerun for testing subsequent versions of the software and by computing an effective ordering for running the test cases. Third, techniques can recycle test cases by monitoring executions to gather test inputs that can be used for retest-ing and by creating unit test cases from system test cases. Finally, techniques could recover test cases by identifying, manipulating, and transforming obsolete test cases, by generating new test cases from old ones, and by repairing test cases when the software changes. In this talk, I will overview the research in testing of evolving software, and discuss achievements to date in managing regression testing by reducing, reusing, recycling, and recovering test cases. I will also present the state of the research and the state of the practice in regression testing. Finally, I will discuss the current trends in both academia and industry, the challenges for solving the difficult problems that exist, the promise for testing of evolving software in the future, and the important open challenges for regression testing in the next decade.

Background: In today’s era, modifications in a software is a common requirement by customers. When changes are made to existing software, re-testing of all the test cases is required to ensure that the newly introduced changes do not have any unwanted effect on the behavior of the software. However, retesting of all the test cases would not only be time consuming but also expensive. Therefore, there is a need for a technique that reduces the number of tests to be performed. Regression testing is one of the ways to reduce the number of test cases. Selection technique is one such method which seeks to identify the test cases that are relevant to some set of recent changes. Objective: It is evident that most of the studies have used different selection techniques and have focused only on one parameter for achieving reduced test suite size without compromising the performance of regression testing. However, to the best of our knowledge, no study has taken two or more parameters of coverage, and/or execution time in a single testing. This paper presents a hybrid technique that combines both regression test selection using slicing technique and minimization of test cases using modified firefly algorithm with combination of parameters coverage and execution time in a single testing. Methods: A hybrid technique has been described that combines both selection and minimization. Selection of test cases is based upon slicing technique while minimization is done using firefly algorithm. Hybrid technique selects and minimizes the test suite using information on statement coverage and execution time. Results: The proposed technique gives 43.33% much superior result as compared to the other hybrid approach in terms of significantly reduced number of test cases. It shows that the resultant test cases were effective enough to cover 100% of the statements, for all the programs. The proposed technique was also tested on four different programs namely Quadratic, Triangle, Next day, Commission respectively for test suite selection and minimization which gave comparatively superior result in terms of reduction (%) in number of test cases required for testing. Conclusion: The combination of parameters used in slicing based approach, reduces the number of test cases making software testing an economical, feasible and time saving option without any fault in the source code. This proposed technique can be used by software practitioners/experts to reduce time, efforts and resources for selection and minimization of test cases.

In this chapter we present Verification-Aided Regression Testing, a novel extension of regression testing that is significantly less sensitive to the completeness of the validation test suite due to the use of model checking. We extend the use of test case executions from conventional direct fault discovery to generation of behavioral properties specific to the new version by (i) automatically producing properties that are proved to hold for the base version of a program, (ii) automatically identifying and checking on the upgraded program only the properties that, according to the developers’ intention, must be preserved by the upgrade, and (iii) reporting the faults and the corresponding counterexamples that are not revealed by the regression tests. Our empirical study on both open-source and industrial software systems show that Verification-Aided Regression Testing produces properties that can be extremely beneficial in increasing the effectiveness of regression testing by timely and automatically detecting faults unnoticed by existing test suites.

Software testing is a fundamental step in the software development lifecycle. Its purpose is to evaluate the quality of software applications. Regression testing is an important testing methodology in software testing. The purpose of regression testing is to validate the software after each change of its code. This involves adding new test cases to the test suite and running the test suite as the software changes, making the test suite larger. The cost and time of the project are affected by the test suite size. The challenge is to run regression testing with a smaller number of test cases and larger amount of software coverage. Minimization of the test suite with maximization of the software coverage is an NP-complete problem. Search-based software engineering is an important topic in software engineering, which addresses software engineering optimization problems to find the optimal/approximate solution of the given problem. This study investigated an approach to reducing the regression testing effort and saving time. It also solved the regression testing optimization problem by achieving the maximum test suite coverage with the minimum test suite size. Several experiments were conducted to obtain the optimal solutions for the regression testing problem. We propose an optimization methodology that combines a genetic algorithm and a greedy algorithm to optimize regression testing by respectively maximizing the software test coverage and minimizing the test suite size. The proposed methodology can conveniently deliver fault-free, fully covered, and powerful programs for mission-critical functions. It can be applied to test a real-time system that has high requirements for reliability, security, and safety.

A safe regression testing approach for safety critical systems

An empirical comparison of four Java-based regression test selection techniques

Software testing is one of the important stages of software development. In software development, developers always depend on testing to reveal bugs. In the maintenance stage test suite size grow because of integration of new technique. Addition of new technique force to create new test case which increase the size of test suite. In regression testing new test case may be added to the test suite during the whole testing process. These additions of test cases create possibility of presence of redundant test cases. Due to limitation of time and resource, reduction techniques should be used to identify and remove them. Research shows that a subset of the test case in a suit may still satisfy all the test objectives which is called as representative set. Redundant test case increase the execution cost of the test suite, in spite of NP-completeness of the problem there are few good reduction techniques have been available. In this paper a new approach for test case reduction is proposed. This algorithm use genetic algorithm technique iteratively with varying chromosome length to reduce test case in a test suit by finding representative set of test cases that are fulfill the testing criteria.

Test coverage optimization for large code problems

Regression testing is an expensive testing process used to validate software following modifications. The cost-effectiveness of regression testing techniques varies with characteristics of test suites. One such characteristic, test suite granularity, involves the way in which test inputs are grouped into test cases within a test suite. Various cost-benefits tradeoffs have been attributed to choices of test suite granularity, but almost no research has formally examined these tradeoffs. To address this lack, we conducted several controlled experiments, examining the effects of test suite granularity on the costs and benefits of several regression testing methodologies across six releases of two non-trivial software systems. Our results expose essential tradeoffs to consider when designing test suites for use in regression testing evolving systems.

Regression testing is very important process in software maintenance. Unfortunately, it is costly and time consuming to allow for the re-execution of all test cases during regression testing. The challenge in regression testing is the selection of best test cases from the existing test suite.This paper presents an evolutionary regression test case prioritization for object-oriented software based on extended system dependence graph model of the affected program using genetic algorithm. The approach is based on optimization of selected test case from dependency analysis of the source codes. The goal is to identify changes in a method's body due to data dependence, control dependence and dependent due to object relation such as inheritance and polymorphism, select the test cases based on affected statements and ordered them based on their fitness by using GA.The number of affected statements determined how fit a test case is good for regression testing. A case study is reported to provide evidence of the feasibility of the approach and its benefits in increasing the rate of fault detection and reduction in regression testing effort compared with retest-all. It was shown that our approach needs 30% of the test cases to cover all the faults, while 80% is needed to cover all the faults using retest-all, which is time consuming and costly.

Regression testing is an expensive maintenance process directed at validating modified software. Regression test selection techniques attempt to reduce the cost of regression testing by selecting tests from a program's existing test suite. Many regression test selection techniques have been proposed. Although there have been some analytical and empirical evaluations of individual techniques, to our knowledge only one comparative study, focusing on one aspect of two of these techniques, has been performed. We conducted an experiment to examine the relative costs and benefits of several regression test selection techniques. The experiment examined five techniques for reusing tests, focusing on their relative abilities to reduce regression testing effort and uncover faults in modified programs. Our results highlight several differences between the techniques, and expose essential tradeoffs that should be considered when choosing a technique for practical application.

In development of software applications Graphical User Interface (GUI) plays an important role. It contains many test cases where some are useful and others are unusable test cases. Testing is used to recover bugs from the software. GUI needs to perform regular testing because it often requires frequent changes which depend on requirements of the users. This type of application requires regression testing which can be tested frequently when the modification occurs. The old version has a test suite and the modified version has a new suite. We focus on the part where the modified test suite is affecting the old test suite by making them unfeasible. The empirical study of these test cases is that to make the unfeasible test cases to feasible test case like reusing of the test paths by varying input values. We probably find where the input values are making the old test suite to reuse in the modified test suite. This approach can make more efficient by reducing the cost of regression testing and consumption of less time for the new modified application.