A Taxonomy of Information Attributes for Test Case Prioritisation: Applicability, Machine Learning

Regression testing is important for maintaining software quality. However, the cost of regression testing is relatively high. Test case prioritization is one way to reduce this cost. Test case prioritization techniques sort test cases for regression testing based on their importance. In this paper, we design and implement a test case prioritization method based on the location of a change. The method consists of three steps: (1) clustering test cases, (2) prioritizing the clusters with respect to the relevance of the clusters to a code change, and (3) test case prioritization within each cluster based on metrics. We propose a metric for measuring test case importance based on Requirement Complexity, Code Complexity, and Code Coverage. To evaluate our method, we apply it on a launch interceptor problem program, and measure the inclusiveness and precision for clusters of test cases with respect to code change in specific test cases. Our results show that our proposed change-based prioritization method increases the likelihood of executing more relevant test cases earlier.

HSP: A hybrid selection and prioritisation of regression test cases based on information retrieval and code coverage applied on an industrial case study

Test case prioritization involves reordering the test cases in an order that helps in attaining certain performance goals. The rate of fault detection is one of the prime goals that we tend to achieve while doing prioritization. Test cases should run in an order to increase the possibility of fault detection and it should be achieved early during the test life cycle. To reduce the cost and time of regression testing, test case prioritization should be done with the intention of periodically modifying the test suite. The humongous set of test cases makes it redundant and cumbersome for the testers who ensure quality for an end application. The fault detection capability of a prioritized test suite is improved up to 15% using Modified PSO which forms the base algorithms for prioritization. The algorithm illustrated detects serious errors at earlier phases of testing process and effectiveness between prioritized and unprioritized test cases.

Requirements based test prioritization using risk factors: An industrial study

Software needs modifications and requires revisions regularly. Owing to these revisions, retesting software becomes essential to ensure that the enhancements made, have not affected its bug-free functioning. The time and cost incurred in this process, need to be reduced by the method of test case selection and prioritization. It is observed that many nature-inspired techniques are applied in this area. African Buffalo Optimization is one such approach, applied to regression test selection and prioritization. In this paper, the proposed work explains and proves the applicability of the African Buffalo Optimization approach to test case selection and prioritization. The proposed algorithm converges in polynomial time (O(n2)). In this paper, the empirical evaluation of applying African Buffalo Optimization for test case prioritization is done on sample data set with multiple iterations. An astounding 62.5% drop in size and a 48.57% drop in the runtime of the original test suite were recorded. The obtained results are compared with Ant Colony Optimization. The comparative analysis indicates that African Buffalo Optimization and Ant Colony Optimization exhibit similar fault detection capabilities (80%), and a reduction in the overall execution time and size of the resultant test suite. The results and analysis, hence, advocate and encourages the use of African Buffalo Optimization in the area of test case selection and prioritization.

The validation of modified software depends on the success of Regression testing. For this, test cases are selected in such a way that can detect a maximum number of faults at the earliest stage of software development. The selection process in which the most beneficial test case are executed first is known as test case prioritization which improves the performance of execution of test cases in a specific or appropriate order. Many optimizing techniques like greedy algorithm, genetic algorithm, and metaheuristic search techniques have been used by many researchers for test case prioritization and optimization. This research paper presents a test case prioritization and optimization method using genetic algorithm by taking different factors of test cases like statement coverage data, requirements factors, risk exposure, and execution time.

Regression testing is the process of testing a modified system using the old test suite. As the test suite size is large, system retesting consumes large amount of time and computing resources. This issue of retesting of software systems can be handled using a good test case prioritization technique. A prioritization technique schedules the test cases for execution so that the test cases with higher priority executed before lower priority. The objective of test case prioritization is to detect fault as early as possible so that the debuggers can begin their work earlier. In this paper we propose a new prioritization technique to prioritize the test cases to perform regression testing for Component Based Software System (CBSS). The components and the state changes for a component based software systems are being represented by UML state chart diagrams which are then converted into Component Interaction Graph (CIG) to describe the interrelation among components. Our prioritization algorithm takes this CIG as input along with the old test cases and generates a prioritized test suit taking into account total number of state changes and total number of database access, both direct and indirect, encountered due to each test case. Our algorithm is found to be very effective in maximizing the objective function and minimizing the cost of system retesting when applied to few JAVA projects.

Test case prioritization techniques attempt to reorder test cases in a manner that increases the rate at which faults are detected during regression testing. Coverage-based test case prioritization techniques typically use one of two overall strategies: a total strategy or an additional strategy . These strategies prioritize test cases based on the total number of code (or code-related) elements covered per test case and the number of additional (not yet covered) code (or code-related) elements covered per test case, respectively. In this article, we present a unified test case prioritization approach that encompasses both the total and additional strategies. Our unified test case prioritization approach includes two models ( basic and extended ) by which a spectrum of test case prioritization techniques ranging from a purely total to a purely additional technique can be defined by specifying the value of a parameter referred to as the f p value. To evaluate our approach, we performed an empirical study on 28 Java objects and 40 C objects, considering the impact of three internal factors (model type, choice of f p value, and coverage type) and three external factors (coverage granularity, test case granularity, and programming/testing paradigm), all of which can be manipulated by our approach. Our results demonstrate that a wide range of techniques derived from our basic and extended models with uniform f p values can outperform purely total techniques and are competitive with purely additional techniques. Considering the influence of each internal and external factor studied, the results demonstrate that various values of each factor have nontrivial influence on test case prioritization techniques.

Test case prioritization is used in regression testing to schedule the execution order of test cases so as to expose faults earlier in testing. Over the past few years, many test case prioritization techniques have been proposed in the literature. Most of these techniques require data on dynamic execution in the form of code coverage information for test cases. However, the collection of dynamic code coverage information on test cases has several associated drawbacks including cost increases and reduction in prioritization precision. In this paper, we propose an approach to prioritizing test cases in the absence of coverage information that operates on Java programs tested under the JUnit framework-an increasingly popular class of systems. Our approach, JUnit test case Prioritization Techniques operating in the Absence of coverage information (JUPTA), analyzes the static call graphs of JUnit test cases and the program under test to estimate the ability of each test case to achieve code coverage, and then schedules the order of these test cases based on those estimates. To evaluate the effectiveness of JUPTA, we conducted an empirical study on 19 versions of four Java programs ranging from 2K-80K lines of code, and compared several variants of JUPTA with three control techniques, and several other existing dynamic coverage-based test case prioritization techniques, assessing the abilities of the techniques to increase the rate of fault detection of test suites. Our results show that the test suites constructed by JUPTA are more effective than those in random and untreated test orders in terms of fault-detection effectiveness. Although the test suites constructed by dynamic coverage-based techniques retain fault-detection effectiveness advantages, the fault-detection effectiveness of the test suites constructed by JUPTA is close to that of the test suites constructed by those techniques, and the fault-detection effectiveness of the test suites constructed by some of JUPTA's variants is better than that of the test suites constructed by several of those techniques.

Given that software systems are constantly changing at a fast rate during the software development process, Continuous integration testing that a cost-effective software development practice is characterized by constantly evolving test cases, rapid return, and limited performance time. Regression test is performed after any change in the software, so we must seek to optimize the regression test methods. One of the methods that have attracted a lot of attention today is the prioritization of the test using reinforcement learning. The purpose of this study was to survey and compare reinforcement learning reward functions, which are used to test case prioritization. We divided these reward functions into two categories: reward functions that depend on current information and reward functions which depend on historical information. The current information based reward functions rely only on the information of a previous implementation and the historical information-based reward functions rely on the whole information of the past. Each of these functions has the strengths and weaknesses that we have discussed in this study. Among the reward functions of these two categories, the HFCW reward function has better performance both in terms of fault detection and usage time. Keywords— Continuous integration, Test case prioritization, Machine learning, Reinforcement learning, Reward function

The process of regression testing is generally done under critical time and resource constraints. In the existing approaches, Test Case Prioritization (TCP), Test Case Selection (TCS), and Test Suite Reduction (TSR) have been treated as different optimization problems. This is based on simple heuristics that do not use valuable information that can be extracted from the program under test. Moreover, they do not allow for making internally consistent budget-conscious decisions. This paper introduces a unified approach that combines various variants of Graph Neural Networks (GNNs) and Reinforcement Learning (RL) for the solutions of TCP, TCS, and TSR. In our proposal, the representation of regression artifacts such as test cases, code entities, and faults is used as nodes in the typed graph, while the edges are used to show the relations between these artifacts. The relation-aware GNN is used for generating test case embeddings, reflecting the distance between the test cases and the changed areas as well as the areas known to have issues in the past. The actor-critic RL agent uses these test case embeddings and the budget to decide whether to run, skip, or discard test cases. The performance of our proposal outstrips coverage-based heuristics, history-based ranking, a GA-based search, an enhanced QPSO, and two learning-based ablations, as validated by our experiments on four Java projects from Defects4J. In terms of various programs and budgets, using GNN–RL has led to an improvement in the Average Percentage of Faults Detected (APFD) and Cost-cognizant Average Percentage of Faults Detected (APFDc) by approximately 6–9 percentage points on lower budgets. In addition, it is possible to obtain a 42% suite reduction and a 48% cost reduction while still retaining 98% of all fault-revealing tests in the final suite. The results show that it is possible to obtain promising solutions for adaptive regression testing using budget-aware reinforcement learning and graph-based representation learning.

This study examines the implementation of Test Case Prioritization (TCP) using the Greedy Algorithm (GA) to enhance regression testing efficiency within a financial technology company's software development cycle. With testing durations increasing significantly, this study aims to address inefficiencies by applying the Greedy Algorithm to optimize test suite size and fault detection. The research methodology involves applying the Greedy Algorithm during the Regression Testing phase, comparing the prioritized suite with the original suite using metrics such as Average Percentage Fault Detection (APFD) and Test Suite Size Reduction (TSSR). Results show that the Greedy Algorithm achieved substantial improvements in both test suite size and fault detection effectiveness across different projects. For Project A, the test suite was reduced from 51-22 test cases, achieving a TSSR of 56.8%, with an APFD increase of 206.21%, rising from 0.0853-0.2613. Project B demonstrated even greater optimization, reducing the test suite from 36-8 test cases, resulting in a TSSR of 77.8% and an APFD improvement of 83.92%, rising from 0.3194-0.5875. These outcomes underscore the algorithm’s effectiveness in eliminating redundant test cases, accelerating testing, and enhancing fault detection thereby supporting the company's goal of faster release cycles without compromising quality.

Regression testing is an important and costly strategy in software testing. To decrease its cost, many methods have been proposed including the test case prioritization methods. The aim of the prioritization methods is to define an ideal order of test cases that allows for higher coverage and early fault detection with minimal number of executed test cases. However, the problem with most of the existing test case prioritization methods is the random sorting of test cases when two or more test cases record equal priority values. In this paper, the effectiveness of an enhanced weighted method using a unique priority value, UniVal, is proposed. Unival prioritizes test cases based on code coverage criteria with information from history of previous executions. In addition, a controlled experiment was executed, and the results were statistically analyzed to assess the effectiveness of the proposed method. The results indicates better performance with regard to prioritize test cases and achieve higher APFD values.  Â

On customer's demand, new requirements are implemented in the software.The modified software may not work properly as earlier because of the new requirements added.So the modified software must be tested.Regression testing (RT) is defined as retesting of the modified software.It is performed using the already developed test suite and a newly developed test suite.The big software has a larger test suite size.For a single requirement change, to run the whole test cases is not beneficial for the development organisation.To make RT more effective, prioritisation of test suite is done.Here we present the genetic algorithm (GA) for the test case prioritisation (TCP).Different approaches have been discussed and implemented using the average percentage of fault detected (APFD) metric.The discussed approaches are applied over a single problem and the result is shown in the tabular form.APFD metric is applied to all the discussed approaches and suggested which one is better.This paper uses GA to arrange the test cases in a prioritised way on the basis of the fault detected.

Considering that some intelligent software in mobile devices is related to location of sensors and devices, regression testing for it faces a major challenge. Test case prioritization (TCP), as a kind of regression test optimization technique, is beneficial to improve test efficiency. However, traditional TCP techniques may have limitations on testing intelligent software embedded in mobile devices because they do not take into account characteristics of mobile devices. This paper uses a smart mall as a scenario to design a novel location-based TCP technique for software embedded in mobile devices using the law of gravitation. First, test gravitation is proposed by applying the idea of universal gravitation. Second, a specific calculation model of test gravitation is designed for a smart mall scenario. Third, how to create a faulted test case set is designed by the pseudocode. Fourth, a location-based TCP using the law of gravitation algorithm is proposed, which utilizes test case information, fault information, and location information to prioritize test cases. Finally, an empirical evaluation is presented by using one industrial project. The observation, underlying the experimental results, is that our proposed TCP approach performs better than traditional TCP techniques. In addition, besides location information, the level of devices is also an important factor which affects the prioritization efficiency.