Test case prioritization based on requirement correlations

Regression testing is an important and costly strategy in software testing. To decrease its cost, many methods have been proposed including the test case prioritization methods. The aim of the prioritization methods is to define an ideal order of test cases that allows for higher coverage and early fault detection with minimal number of executed test cases. However, the problem with most of the existing test case prioritization methods is the random sorting of test cases when two or more test cases record equal priority values. In this paper, the effectiveness of an enhanced weighted method using a unique priority value, UniVal, is proposed. Unival prioritizes test cases based on code coverage criteria with information from history of previous executions. In addition, a controlled experiment was executed, and the results were statistically analyzed to assess the effectiveness of the proposed method. The results indicates better performance with regard to prioritize test cases and achieve higher APFD values.  Â

Regression testing is important for maintaining software quality. However, the cost of regression testing is relatively high. Test case prioritization is one way to reduce this cost. Test case prioritization techniques sort test cases for regression testing based on their importance. In this paper, we design and implement a test case prioritization method based on the location of a change. The method consists of three steps: (1) clustering test cases, (2) prioritizing the clusters with respect to the relevance of the clusters to a code change, and (3) test case prioritization within each cluster based on metrics. We propose a metric for measuring test case importance based on Requirement Complexity, Code Complexity, and Code Coverage. To evaluate our method, we apply it on a launch interceptor problem program, and measure the inclusiveness and precision for clusters of test cases with respect to code change in specific test cases. Our results show that our proposed change-based prioritization method increases the likelihood of executing more relevant test cases earlier.

Mobile software application development process must be matured enough to handle the challenges (especially market related) associated with the development of high quality mobile software development. Ever increasing number of both mobile users and mobile applications had presented software engineers with the challenge of satisfying billions of users with high quality software applications to be delivered within deadline and budgets. Always there had been a lot of pressure to develop complex software categorized by thousands of requirements, under resource constrained environment. Requirement prioritization is one of the activities undertaken by software engineer to deliver partial software product to its customers such that most important requirements are implemented in the earliest releases. During next releases some changed and pending requirements are implemented, an activity that generates ripple effects. Such ripple effects need to be tested by executing modified source code against test cases of previous releases (regression testing). Regression testing is a very effortful activity that requires a software tester to select test cases that have high fault detection capability, execute the modified code against selected test cases and performing debugging. This regression testing activity can be lowered to the maximum extend by considering dependencies between requirements during the time of requirement prioritization. Thus requirement prioritization will be carried out not only against aspects like cost, time, risks, business values etc but against dependencies also. The aim is to implement almost all dependent highest priority requirements in current release so that implementation of new requirements is unlikely to have ripple effects. Changes in requirements might not be related to variable usage and definition and might not involve a change in functionality. In such cases there is no need to select already executed test cases of previous versions. Module dependencies can lead to test case selections of previous versions if changes of requirement lead to ripple effects. This paper aims to implement highest priority requirements such that regression testing is performed to minimum thereby improving development process of mobile applications. The proposed technique had been successfully evaluated on Android based notification software application that meets the specification of Aakash tablets.

Test case prioritization techniques attempt to reorder test cases in a manner that increases the rate at which faults are detected during regression testing. Coverage-based test case prioritization techniques typically use one of two overall strategies: a total strategy or an additional strategy . These strategies prioritize test cases based on the total number of code (or code-related) elements covered per test case and the number of additional (not yet covered) code (or code-related) elements covered per test case, respectively. In this article, we present a unified test case prioritization approach that encompasses both the total and additional strategies. Our unified test case prioritization approach includes two models ( basic and extended ) by which a spectrum of test case prioritization techniques ranging from a purely total to a purely additional technique can be defined by specifying the value of a parameter referred to as the f p value. To evaluate our approach, we performed an empirical study on 28 Java objects and 40 C objects, considering the impact of three internal factors (model type, choice of f p value, and coverage type) and three external factors (coverage granularity, test case granularity, and programming/testing paradigm), all of which can be manipulated by our approach. Our results demonstrate that a wide range of techniques derived from our basic and extended models with uniform f p values can outperform purely total techniques and are competitive with purely additional techniques. Considering the influence of each internal and external factor studied, the results demonstrate that various values of each factor have nontrivial influence on test case prioritization techniques.

A history-based cost-cognizant test case prioritization technique in regression testing

Considering that some intelligent software in mobile devices is related to location of sensors and devices, regression testing for it faces a major challenge. Test case prioritization (TCP), as a kind of regression test optimization technique, is beneficial to improve test efficiency. However, traditional TCP techniques may have limitations on testing intelligent software embedded in mobile devices because they do not take into account characteristics of mobile devices. This paper uses a smart mall as a scenario to design a novel location-based TCP technique for software embedded in mobile devices using the law of gravitation. First, test gravitation is proposed by applying the idea of universal gravitation. Second, a specific calculation model of test gravitation is designed for a smart mall scenario. Third, how to create a faulted test case set is designed by the pseudocode. Fourth, a location-based TCP using the law of gravitation algorithm is proposed, which utilizes test case information, fault information, and location information to prioritize test cases. Finally, an empirical evaluation is presented by using one industrial project. The observation, underlying the experimental results, is that our proposed TCP approach performs better than traditional TCP techniques. In addition, besides location information, the level of devices is also an important factor which affects the prioritization efficiency.

With the continuous evolution of software systems, test suites often grow very large. Rerunning all test cases may be impractical in regression testing under limited resources. Coverage-based test case prioritization techniques have been pro- posed to improve the effectiveness of regression testing. The origi- nal test suite often contains some test cases which are designed for exercising production features or exceptional behaviors, rather than for code coverage. Therefore, coverage-based prioritization techniques do not always generate satisfactory results. In this context, we propose a global similarity-based regression test case prioritization approach. The approach reschedules the execution order of test cases based on the distances between pair-wise test cases. We designed and conducted empirical studies on four C programs to validate the effectiveness of our proposed approach. Moreover, we also empirically compared the effects of six similarity measures on the global similarity-based test case prioritization approach. Experimental results illustrate that the global similarity-based regression test case prioritization approach using Euclidean distance is the most effective. This study aims at providing practical guidelines for picking the appropriate similarity measures.

HSP: A hybrid selection and prioritisation of regression test cases based on information retrieval and code coverage applied on an industrial case study

Prioritization of test case technique schedules the test suite in an order which will increase the efficiency and effectiveness of the system. The test case executed in an order which enhances the probability of finding the fault and also it finds the severe fault at the early stage. The prioritization of test case is proved to be useful for the regression testing. While much research is going on code coverage based prioritization technique and requirement based test case prioritization technique is not used by many. In this paper, test case prioritization technique based on fault of regression technique is proposed from the requirement specification, to improve the software quality and to increase the fault detection rate. In this paper, test case prioritization is done based the six main factors (i) requirement based on customer priority (ii) based on changes made in requirement specification (iii) complexity in Implementation of code (iv) re-usability (v) flow of application (vi) impact of the fault.

A regression test case prioritization (TCP) technique reorders test cases for regression testing to achieve early fault detection. Most TCP techniques have been developed for regression testing of source code in an application. Most web applications rely on a database server for serving client requests. Any changes in the database result in erroneous client interactions and may bring down the entire web application. However, most prioritization techniques are unsuitable for prioritizing test suites for early detection of changes in databases. There are very few proposals in the literature for prioritization of test cases that can detect faults in the database early. We propose a new automated TCP technique for web applications that automatically identifies the database changes, prioritizes test cases related to database changes and executes them in priority order to detect faults early.

The goal of test case prioritization is to schedule the test cases in an order so that the faults in the software will be detected early during the regression testing. So far, a lot of test case prioritization techniques have been proposed and received considerable attention. However, there is currently no universally best approach because the existing techniques prioritize test cases under different considerations. Thus, we plan to propose a hybrid technique that takes into account more than one consideration and hope that this hybrid technique can further improve the effectiveness of test case prioritization. Although we did not construct the hybrid technique yet, we already have some preliminary experimental results that are collected from an investigation into real subject programs and their associated test suites. Therefore, this paper is dedicated to presenting our preliminary observations in this ongoing research project. Additionally, this paper also presents two assumptions based on the observations. These two assumptions should be useful for proposing the hybrid test case prioritization techniques in the future. Keywords-software maintenance; regression testing; test case prioritization; fault localization

Software product line (SPL) testing is a challenging task, due to the huge number of variants sharing common functionalities to be taken into account for efficient testing. By adopting the concept of regression testing, incremental SPL testing strategies cope with this challenge by exploiting the reuse potential of test artifacts between subsequent variants under test. In previous work, we proposed delta-oriented test case prioritization for incremental SPL integration testing, where differences between architecture test model variants allow for reasoning about the order of reusable test cases to be executed. However, the prioritization left two issues open, namely (1) changes to component behavior are ignored, which may also influence component interactions and, (2) the weighting and ordering of similar test cases result in an unintended clustering of test cases. In this paper, we extend the test case prioritization technique by (1) incorporating changes to component behavior allowing for a more fine-grained analysis and (2) defining a dissimilarity measure to avoid clustered test case orders. We prototyped our test case prioritization technique and evaluated its applicability and effectiveness by means of a case study from the automotive domain showing positive results.

Test case prioritization is used in regression testing to schedule the execution order of test cases so as to expose faults earlier in testing. Over the past few years, many test case prioritization techniques have been proposed in the literature. Most of these techniques require data on dynamic execution in the form of code coverage information for test cases. However, the collection of dynamic code coverage information on test cases has several associated drawbacks including cost increases and reduction in prioritization precision. In this paper, we propose an approach to prioritizing test cases in the absence of coverage information that operates on Java programs tested under the JUnit framework-an increasingly popular class of systems. Our approach, JUnit test case Prioritization Techniques operating in the Absence of coverage information (JUPTA), analyzes the static call graphs of JUnit test cases and the program under test to estimate the ability of each test case to achieve code coverage, and then schedules the order of these test cases based on those estimates. To evaluate the effectiveness of JUPTA, we conducted an empirical study on 19 versions of four Java programs ranging from 2K-80K lines of code, and compared several variants of JUPTA with three control techniques, and several other existing dynamic coverage-based test case prioritization techniques, assessing the abilities of the techniques to increase the rate of fault detection of test suites. Our results show that the test suites constructed by JUPTA are more effective than those in random and untreated test orders in terms of fault-detection effectiveness. Although the test suites constructed by dynamic coverage-based techniques retain fault-detection effectiveness advantages, the fault-detection effectiveness of the test suites constructed by JUPTA is close to that of the test suites constructed by those techniques, and the fault-detection effectiveness of the test suites constructed by some of JUPTA's variants is better than that of the test suites constructed by several of those techniques.

Empirical studies often evaluate Test Case Prioritization (TCP) techniques by measuring their ability to uncover faults as early as possible. In this context, techniques reorder test cases using different sources of information. These prioritizations are evaluated considering faults detected by failure occurrences during testing execution. While this accounts for validity, results can be biased. For instance, conclusions regarding a TCP technique's behavior may be affected by the number of failures, faults and, in turn, the failure-fault relationship. We propose the use of fault abstractions, which relate faults to the related test cases that fail, to exercise techniques on preliminary empirical studies of system level TCP techniques. Rather than modeling specific classes of faults, our strategy is based on identifying cliques of test cases that are related according to a distance function as a fault abstraction, assuming that the test cases can fail due to a number of faults in common. Our strategy calculate these fault abstractions based on the set of maximal cliques obtained from test case distance graphs along with a set of essential test cases. Therefore, one can sample fault abstractions respecting different failure-fault relationships and apply on empirical studies. In our studies, even though our strategy is not predictive, a significant amount of fault abstractions calculated by the proposed technique corresponded to actual reported faults. Therefore Fault abstractions may represent the relationship between faults and failures to assess TCP techniques when none or a small amount of artifacts is available to perform a preliminary empirical evaluation.

Flower Pollination Algorithm (FPA) is a significant addition made to Nature Inspired Metaheuristic Optimization Algorithms (NIMOA). It is inspired by the pollination process of flowering plants. In this research, FPA is used for Test Case Prioritization (TCP) in Regression Testing (RT). The algorithm uses code coverage of test cases as the input. The algorithm has no prior information of faults covered by the test cases. This study deals with prioritizing (ordering) the test cases in such a way that only those test cases are executed that covers maximum faults in minimum time of execution. For validation of the results Average Percentage of Fault Detected (APFD) metrics is used. APFD values for different ordering of test cases is calculated for three applications written in Java. The empirical results of APFD metrics for FPA order (TS1) and FPA order (TSp) are better as compared to Random Order of Original Test Suite (TSo) and Reverse Random Order of (TSo). Therefore, this paper states that FPA for TCP gives efficient results in RT.