Test coverage optimization for large code problems

Software testing is one of the important stages of software development. In software development, developers always depend on testing to reveal bugs. In the maintenance stage test suite size grow because of integration of new technique. Addition of new technique force to create new test case which increase the size of test suite. In regression testing new test case may be added to the test suite during the whole testing process. These additions of test cases create possibility of presence of redundant test cases. Due to limitation of time and resource, reduction techniques should be used to identify and remove them. Research shows that a subset of the test case in a suit may still satisfy all the test objectives which is called as representative set. Redundant test case increase the execution cost of the test suite, in spite of NP-completeness of the problem there are few good reduction techniques have been available. In this paper a new approach for test case reduction is proposed. This algorithm use genetic algorithm technique iteratively with varying chromosome length to reduce test case in a test suit by finding representative set of test cases that are fulfill the testing criteria.

Regression testing is an essential quality test technique during the maintenance phase of the software. It is executed to ensure the validity of the software after any modification. As software evolves, the test suite expands and may become too large to be executed entirely within a limited testing budget and/or time. So, to reduce the cost of regression testing, it is mandatory to reduce the size of the test suite by discarding the redundant test cases and selecting the most representative ones that do not compromise the effectiveness of the test suite in terms of some predefined criteria such as its fault-detection capability. This problem is known as test suite reduction (TSR); and it is known to be as nondeterministic polynomial-time complete (NP-complete) problem. This paper formulated the TSR problem as a multi-objective optimization problem; and adapted the heuristic binary bat algorithm (BBA) to resolve it. The BBA algorithm was adapted in order to enhance its exploration capabilities during the search for Pareto-optimal solutions. The effectiveness of the proposed multi-objective adapted binary bat algorithm (MO-ABBA) was evaluated using 8 test suites of different sizes, in addition to twelve benchmark functions. Experimental results showed that, for the same fault discovery rate, the MO-ABBA is capable of reducing the test suite size more than each of the multi-objective original binary bat (MO-BBA) and the multi-objective binary particle swarm optimization (MO-BPSO) algorithms. Moreover, MO-ABBA converges to the best solutions faster than each of the MO-BBA and the MO-BPSO.

Exhaustive product evolution and testing is required to ensure the quality of product. Regression testing is crucial to ensure software excellence. Regression test cases are applied to assure that new or adapted features do not relapse the existing features. As innovative features are included, new test cases are generated to assess the new functionality, and then included in the existing pool of test cases, thus escalating the cost and the time required in performing regression test and this unswervingly impacts the release, laid plan and the quality of the product. Hence there is a need to select minimal test cases that will test all the functionalities of the engineered product and it must rigorously test the functionalities that have high risk exposure. Test Suite-Test Case Refinement Technique will reduce regression test case pool size, reduce regression testing time, cost & effort and also ensure the quality of the engineered product. This technique is a regression test case optimization technique that is a hybrid of Test Case Minimization based on specifications and Test Case Prioritization based on risk exposure. This approach will facilitate achievement of quality product with decreased regression testing time and cost yet uncover same amount of errors as the original test cases.

One of the most expensive activities that occurs as software is developed and maintained is the testing (or retesting) of the software after it has been modified. Studies suggest that a significant portion of development and maintenance costs go to this retesting, which is known as regression testing. Reports estimate that regression testing consumes as much as 80% of the overall testing budget and can consume up to 50% of the cost of software maintenance. Rapidly changing software and computing environments present many challenges for effective and efficient regression testing in practice. Regression testing can be performed after changes are made to the software, such as after nightly or regular builds, before a new version of the software is released, every time the software is saved and compiled, such as in an agile development environment, or before patches, such as security patches, are released. Regardless of the environment or when it is performed, the goals of regression testing are the same: to improve confidence that the changes behave as intended and that they have not adversely affected unchanged parts of the software. Because regression testing is important, but expensive, much research has been performed, both in industry and in academia, to develop techniques to make regression testing more effective and efficient. This research has also produced many tools and systems that have been used for empirical studies that investigate the effectiveness, scalability, and practicality of the techniques. Researchers have developed techniques for addressing a number of issues related to regression testing, and, in this talk, I will discuss them in four areas. First, techniques attempt to reduce the regression testing time by creating effective regression test suites that test the changed part of the software, by identifying test cases in the regression test suite that do not need to be rerun on the changed software, and by identifying and removing obsolete test cases. Second, techniques can reuse test suites created for one version of the software by identifying those test cases that need to be rerun for testing subsequent versions of the software and by computing an effective ordering for running the test cases. Third, techniques can recycle test cases by monitoring executions to gather test inputs that can be used for retest-ing and by creating unit test cases from system test cases. Finally, techniques could recover test cases by identifying, manipulating, and transforming obsolete test cases, by generating new test cases from old ones, and by repairing test cases when the software changes. In this talk, I will overview the research in testing of evolving software, and discuss achievements to date in managing regression testing by reducing, reusing, recycling, and recovering test cases. I will also present the state of the research and the state of the practice in regression testing. Finally, I will discuss the current trends in both academia and industry, the challenges for solving the difficult problems that exist, the promise for testing of evolving software in the future, and the important open challenges for regression testing in the next decade.

Test case prioritization involves reordering the test cases in an order that helps in attaining certain performance goals. The rate of fault detection is one of the prime goals that we tend to achieve while doing prioritization. Test cases should run in an order to increase the possibility of fault detection and it should be achieved early during the test life cycle. To reduce the cost and time of regression testing, test case prioritization should be done with the intention of periodically modifying the test suite. The humongous set of test cases makes it redundant and cumbersome for the testers who ensure quality for an end application. The fault detection capability of a prioritized test suite is improved up to 15% using Modified PSO which forms the base algorithms for prioritization. The algorithm illustrated detects serious errors at earlier phases of testing process and effectiveness between prioritized and unprioritized test cases.

In development of software applications Graphical User Interface (GUI) plays an important role. It contains many test cases where some are useful and others are unusable test cases. Testing is used to recover bugs from the software. GUI needs to perform regular testing because it often requires frequent changes which depend on requirements of the users. This type of application requires regression testing which can be tested frequently when the modification occurs. The old version has a test suite and the modified version has a new suite. We focus on the part where the modified test suite is affecting the old test suite by making them unfeasible. The empirical study of these test cases is that to make the unfeasible test cases to feasible test case like reusing of the test paths by varying input values. We probably find where the input values are making the old test suite to reuse in the modified test suite. This approach can make more efficient by reducing the cost of regression testing and consumption of less time for the new modified application.

Multi-objective regression test selection in practice: An empirical study in the defense software industry

Regression testing is important for maintaining software quality. However, the cost of regression testing is relatively high. Test case prioritization is one way to reduce this cost. Test case prioritization techniques sort test cases for regression testing based on their importance. In this paper, we design and implement a test case prioritization method based on the location of a change. The method consists of three steps: (1) clustering test cases, (2) prioritizing the clusters with respect to the relevance of the clusters to a code change, and (3) test case prioritization within each cluster based on metrics. We propose a metric for measuring test case importance based on Requirement Complexity, Code Complexity, and Code Coverage. To evaluate our method, we apply it on a launch interceptor problem program, and measure the inclusiveness and precision for clusters of test cases with respect to code change in specific test cases. Our results show that our proposed change-based prioritization method increases the likelihood of executing more relevant test cases earlier.

Software testing is a fundamental step in the software development lifecycle. Its purpose is to evaluate the quality of software applications. Regression testing is an important testing methodology in software testing. The purpose of regression testing is to validate the software after each change of its code. This involves adding new test cases to the test suite and running the test suite as the software changes, making the test suite larger. The cost and time of the project are affected by the test suite size. The challenge is to run regression testing with a smaller number of test cases and larger amount of software coverage. Minimization of the test suite with maximization of the software coverage is an NP-complete problem. Search-based software engineering is an important topic in software engineering, which addresses software engineering optimization problems to find the optimal/approximate solution of the given problem. This study investigated an approach to reducing the regression testing effort and saving time. It also solved the regression testing optimization problem by achieving the maximum test suite coverage with the minimum test suite size. Several experiments were conducted to obtain the optimal solutions for the regression testing problem. We propose an optimization methodology that combines a genetic algorithm and a greedy algorithm to optimize regression testing by respectively maximizing the software test coverage and minimizing the test suite size. The proposed methodology can conveniently deliver fault-free, fully covered, and powerful programs for mission-critical functions. It can be applied to test a real-time system that has high requirements for reliability, security, and safety.

There occur frequent requirement changes in software systems even after the software has been developed. Regression testing is continuously performed to identify the undesired affects of these requirement changes on already tested system. Test suites grow enormously with these changes due to addition of new test cases for enhanced functionality. Optimization of test suite to perform regression testing within the budgetary and time restrictions is ultimate choice for a tester because "Retest all" test suite is un-economical and is not suitable choice. Test suite optimization can be either static or on the fly. With on the fly optimization, optimal suite keeps on changing with the requirement changes. On the fly optimization of test suite is preferable option for regression testing. Presently, static test suite optimization approaches exist. We have proposed an application specific, on the fly optimization approach for test suite optimization problem. We have implemented our approach on an academic testing problem. We use fuzzy logic to optimize the test suite with multiple optimization objectives. Our approach has been successful to generate on the fly optimized test suites for changing requirements. In future, we will implement this approach on considerably large sized testing problems.

Test suite reduction techniques decrease the cost of software testing by removing the redundant test cases from the test suite while still producing a reduced set of tests that yields the same level of code coverage as the original suite. Most of the existing approaches to reduction aim to decrease the size of the test suite. Yet, the difference in the execution cost of the tests is often significant and it may be costly to use a test suite consisting of a few long-running test cases. Thus, this paper proposes an algorithm, based on the concept of test irreplaceability, which creates a reduced test suite with a decreased execution cost. Leveraging widely used benchmark programs, the empirical study shows that, in comparison to existing techniques, the presented algorithm is the most effective at reducing the cost of running a test suite.

Regression testing and retesting of a modified Web Service can be very costly as it tends to generates large number of test cases. Regression testing cost can be reduced significantly by identifying and testing the modified portion of the Web Service only. This avoids the costly construction of new test cases and the unproductive rerunning of existing test cases when it can be guaranteed that the unmodified code of web service will produce the same results as it produced previously. In this paper, by designing Web service graph we propose an effective method of identifying the modified areas (in this paper area means code of web service to be tested) and use this information to reduce regression testing efforts. In this process, we define and use a set of metrics to categorize the testing of Web service. We demonstrate the applicability of the proposed approach using a case study that consisted of two steps. In the first step, we compared two versions of WSDL's to identify the inserted, modified and deleted areas of the Web services. Next, we use this information to generate the reduced test cases.

A safe regression testing approach for safety critical systems

Regression testing is one of the essential activities during the maintenance phase of software projects. It is executed to ensure the validity of an altered software. However, as the software evolves, regression testing becomes prohibitively expensive. In order to reduce the cost of regression testing, it is mandatory to reduce the size of the test suite by selecting the most representative test cases that do not compromise the effectiveness of the regression testing in terms of fault-detection capability. This problem is known as test suite reduction (TSR) problem, and it is known to be an NP-complete. The paper proposes a multi-objective adapted binary bat algorithm (ABBA) to solve the TSR problem. The original binary bat (OBBA) algorithm was adapted to enhance its exploration capabilities during the search for a Pareto-optimal surface. The effectiveness of the ABBA was evaluated using six Java programs with different sizes. Experimental results showed that for the same fault discovery rate, the ABBA is capable of reducing the test suite size more than the OBBA and the BPSO.

In the software industry, software testing becomes more important in the entire software development life cycle. Software testing is one of the fundamental components of software quality assurances. Software Testing Life Cycle (STLC)is a process involved in testing the complete software, which includes Regression Testing, Unit Testing, Smoke Testing, Integration Testing, Interface Testing, System Testing & etc. In the STLC of Regression testing, test case selection is one of the most important concerns for effective testing as well as cost of the testing process. During the Regression testing, executing all the test cases from existing test suite is not possible because that takes more time to test the modified software. This paper proposes new Hybrid approach that consists of modified Greedy approach for handling the test case selection and Genetic Algorithm uses effective parameter like Initial Population, Fitness Value, Test Case Combination, Test Case Crossover and Test Case Mutation for optimizing the tied test suite. By doing this, effective test cases are selected and minimized the tied test suite to reduce the cost of the testing process. Finally the result of proposed approach compared with conventional greedy approach and proved that our approach is more effective than other existing approach.