Similarity-based regression test case prioritization

Test-case prioritization is to schedule the execution order of test cases so as to maximize some objective (e.g., revealing faults early). The existing test-case prioritization approaches separate the process of test-case prioritization and the process of test-case execution by presenting the execution order of all test cases before programmers start running test cases. As the execution information of the modified program is not available for the existing test-case prioritization approaches, these approaches mainly rely on only the execution information of the previous program before modification. To address this problem, we present an adaptive test-case prioritization approach, which determines the execution order of test cases simultaneously during the execution of test cases. In particular, the adaptive approach selects test cases based on their fault-detection capability, which is calculated based on the output of selected test cases. As soon as a test case is selected and runs, the fault-detection capability of each unselected test case is modified according to the output of the latest selected test case. To evaluate the effectiveness of the proposed adaptive approach, we conducted an experimental study on eight C programs and four Java programs. The experimental results show that the adaptive approach is usually significantly better than the total test-case prioritization approach and competitive to the additional test-case prioritization approach. Moreover, the adaptive approach is better than the additional approach on some subjects (e.g, replace and schedule).

Test case prioritization is used in regression testing to schedule the execution order of test cases so as to expose faults earlier in testing. Over the past few years, many test case prioritization techniques have been proposed in the literature. Most of these techniques require data on dynamic execution in the form of code coverage information for test cases. However, the collection of dynamic code coverage information on test cases has several associated drawbacks including cost increases and reduction in prioritization precision. In this paper, we propose an approach to prioritizing test cases in the absence of coverage information that operates on Java programs tested under the JUnit framework-an increasingly popular class of systems. Our approach, JUnit test case Prioritization Techniques operating in the Absence of coverage information (JUPTA), analyzes the static call graphs of JUnit test cases and the program under test to estimate the ability of each test case to achieve code coverage, and then schedules the order of these test cases based on those estimates. To evaluate the effectiveness of JUPTA, we conducted an empirical study on 19 versions of four Java programs ranging from 2K-80K lines of code, and compared several variants of JUPTA with three control techniques, and several other existing dynamic coverage-based test case prioritization techniques, assessing the abilities of the techniques to increase the rate of fault detection of test suites. Our results show that the test suites constructed by JUPTA are more effective than those in random and untreated test orders in terms of fault-detection effectiveness. Although the test suites constructed by dynamic coverage-based techniques retain fault-detection effectiveness advantages, the fault-detection effectiveness of the test suites constructed by JUPTA is close to that of the test suites constructed by those techniques, and the fault-detection effectiveness of the test suites constructed by some of JUPTA's variants is better than that of the test suites constructed by several of those techniques.

Test case prioritization technology improves the efficiency of software testing by optimizing the execution order of test cases, which is an important research topic of software regression testing. In order to solve the problem of requirement-based test case prioritization, this paper proposed a solution based on ant colony optimization algorithm and gave its two different implementation methods: distance-based and index-based implementation. Firstly, a general indicator based on requirements was designed to evaluate the test cases. Secondly, the concept of test case attractivity was proposed, and the definition of the distance between test cases was given based on it. Finally, the main design strategies such as the pheromone update strategy, the optimal solution update strategy, and the local optimal mutation strategy were given. The experimental results show that the method has good global optimization ability, and its overall effect is better than particle swarm optimization algorithm, genetic algorithm and random testing.

Test case prioritization techniques attempt to reorder test cases in a manner that increases the rate at which faults are detected during regression testing. Coverage-based test case prioritization techniques typically use one of two overall strategies: a total strategy or an additional strategy . These strategies prioritize test cases based on the total number of code (or code-related) elements covered per test case and the number of additional (not yet covered) code (or code-related) elements covered per test case, respectively. In this article, we present a unified test case prioritization approach that encompasses both the total and additional strategies. Our unified test case prioritization approach includes two models ( basic and extended ) by which a spectrum of test case prioritization techniques ranging from a purely total to a purely additional technique can be defined by specifying the value of a parameter referred to as the f p value. To evaluate our approach, we performed an empirical study on 28 Java objects and 40 C objects, considering the impact of three internal factors (model type, choice of f p value, and coverage type) and three external factors (coverage granularity, test case granularity, and programming/testing paradigm), all of which can be manipulated by our approach. Our results demonstrate that a wide range of techniques derived from our basic and extended models with uniform f p values can outperform purely total techniques and are competitive with purely additional techniques. Considering the influence of each internal and external factor studied, the results demonstrate that various values of each factor have nontrivial influence on test case prioritization techniques.

A regression test case prioritization (TCP) technique reorders test cases for regression testing to achieve early fault detection. Most TCP techniques have been developed for regression testing of source code in an application. Most web applications rely on a database server for serving client requests. Any changes in the database result in erroneous client interactions and may bring down the entire web application. However, most prioritization techniques are unsuitable for prioritizing test suites for early detection of changes in databases. There are very few proposals in the literature for prioritization of test cases that can detect faults in the database early. We propose a new automated TCP technique for web applications that automatically identifies the database changes, prioritizes test cases related to database changes and executes them in priority order to detect faults early.

Regression testing is the process of testing a modified system using the old test suite. As the test suite size is large, system retesting consumes large amount of time and computing resources. This issue of retesting of software systems can be handled using a good test case prioritization technique. A prioritization technique schedules the test cases for execution so that the test cases with higher priority executed before lower priority. The objective of test case prioritization is to detect fault as early as possible so that the debuggers can begin their work earlier. In this paper we propose a new prioritization technique to prioritize the test cases to perform regression testing for Component Based Software System (CBSS). The components and the state changes for a component based software systems are being represented by UML state chart diagrams which are then converted into Component Interaction Graph (CIG) to describe the interrelation among components. Our prioritization algorithm takes this CIG as input along with the old test cases and generates a prioritized test suit taking into account total number of state changes and total number of database access, both direct and indirect, encountered due to each test case. Our algorithm is found to be very effective in maximizing the objective function and minimizing the cost of system retesting when applied to few JAVA projects.

Regression testing is important for maintaining software quality. However, the cost of regression testing is relatively high. Test case prioritization is one way to reduce this cost. Test case prioritization techniques sort test cases for regression testing based on their importance. In this paper, we design and implement a test case prioritization method based on the location of a change. The method consists of three steps: (1) clustering test cases, (2) prioritizing the clusters with respect to the relevance of the clusters to a code change, and (3) test case prioritization within each cluster based on metrics. We propose a metric for measuring test case importance based on Requirement Complexity, Code Complexity, and Code Coverage. To evaluate our method, we apply it on a launch interceptor problem program, and measure the inclusiveness and precision for clusters of test cases with respect to code change in specific test cases. Our results show that our proposed change-based prioritization method increases the likelihood of executing more relevant test cases earlier.

Test case prioritization techniques basically schedule the execution of test cases in a definite order such that to attain an objective function with greater efficiency.This scheduling of test cases ithe results of regression testing. Test case prioritization techniques order the test cases such that the most important ones are executed first encountering the faults first and thus makes the testing effective.In this paper an approach is presented which calculates the product of statement coverage and function calls . The results illustrate the effectiveness of formula computed with the help of APFD metric.

Prioritization of test case technique schedules the test suite in an order which will increase the efficiency and effectiveness of the system. The test case executed in an order which enhances the probability of finding the fault and also it finds the severe fault at the early stage. The prioritization of test case is proved to be useful for the regression testing. While much research is going on code coverage based prioritization technique and requirement based test case prioritization technique is not used by many. In this paper, test case prioritization technique based on fault of regression technique is proposed from the requirement specification, to improve the software quality and to increase the fault detection rate. In this paper, test case prioritization is done based the six main factors (i) requirement based on customer priority (ii) based on changes made in requirement specification (iii) complexity in Implementation of code (iv) re-usability (v) flow of application (vi) impact of the fault.

Considering that some intelligent software in mobile devices is related to location of sensors and devices, regression testing for it faces a major challenge. Test case prioritization (TCP), as a kind of regression test optimization technique, is beneficial to improve test efficiency. However, traditional TCP techniques may have limitations on testing intelligent software embedded in mobile devices because they do not take into account characteristics of mobile devices. This paper uses a smart mall as a scenario to design a novel location-based TCP technique for software embedded in mobile devices using the law of gravitation. First, test gravitation is proposed by applying the idea of universal gravitation. Second, a specific calculation model of test gravitation is designed for a smart mall scenario. Third, how to create a faulted test case set is designed by the pseudocode. Fourth, a location-based TCP using the law of gravitation algorithm is proposed, which utilizes test case information, fault information, and location information to prioritize test cases. Finally, an empirical evaluation is presented by using one industrial project. The observation, underlying the experimental results, is that our proposed TCP approach performs better than traditional TCP techniques. In addition, besides location information, the level of devices is also an important factor which affects the prioritization efficiency.

HSP: A hybrid selection and prioritisation of regression test cases based on information retrieval and code coverage applied on an industrial case study

To date, various test prioritization techniques have been developed, but the majority of these techniques consider a single objective that could limit the applicability of prioritization techniques by ignoring practical constraints imposed on regression testing. Multi-objective prioritization techniques try to reorder test cases so that they can optimize multiple goals that testers want to achieve. In this paper, we introduced a novel graph-based framework that maps the prioritization task to a graph traversal algorithm. To evaluate our approach, we performed an empirical study using 20 versions of four open source applications. Our results indicate that the use of the graph-based technique can improve the effectiveness and efficiency of test case prioritization technique.

Regression testing is a costly process used to validate the changed software code. Test case prioritisation (TCP) techniques enhance the cost-adequacy of regression testing by ordering the test cases such that more imperative test case runs earlier in the testing process. This paper proposes a heuristic time aware multi-criteria test case prioritisation (HTMCTCP). It considers various targets simultaneously and chooses the best test cases within the stipulated time budget. This technique swaps between the neighbourhood search and the global search in the huge search space of the problem to avoid getting stuck in local optima and also it guarantees fast convergence to the best solution. When the effectiveness of the technique is compared with the existing prioritisation approach like genetic algorithm (GA) and simulated annealing algorithm (SA), it is inferred that the proposed HTMCTCP technique indicates noteworthy change over those systems.

A web service may evolve autonomously, making peer web services in the same service composition uncertain as to whether the evolved behaviors may still be compatible to its originally collaborative agreement. Although peer services may wish to conduct regression testing to verify the original collaboration, the source code of the former service can be inaccessible to them. Traditional code-based regression testing strategies are inapplicable. The rich interface specifications of a web service, however, provide peer services with a means to formulate black-box testing strategies. In this paper, we formulate new test case prioritization strategies using tags embedded in XML messages to reorder regression test cases, and reveal how the test cases use the interface specifications of services. We evaluate experimentally their effectiveness on revealing regression faults in modified WS-BPEL programs. The results show that the new techniques can have a high probability of outperforming random ordering.

The goal of test case prioritization is to schedule the test cases in an order so that the faults in the software will be detected early during the regression testing. So far, a lot of test case prioritization techniques have been proposed and received considerable attention. However, there is currently no universally best approach because the existing techniques prioritize test cases under different considerations. Thus, we plan to propose a hybrid technique that takes into account more than one consideration and hope that this hybrid technique can further improve the effectiveness of test case prioritization. Although we did not construct the hybrid technique yet, we already have some preliminary experimental results that are collected from an investigation into real subject programs and their associated test suites. Therefore, this paper is dedicated to presenting our preliminary observations in this ongoing research project. Additionally, this paper also presents two assumptions based on the observations. These two assumptions should be useful for proposing the hybrid test case prioritization techniques in the future. Keywords-software maintenance; regression testing; test case prioritization; fault localization