A Meta-heuristic Test Case Prioritization Method Based on Hybrid Model

Flower Pollination Algorithm (FPA) is a significant addition made to Nature Inspired Metaheuristic Optimization Algorithms (NIMOA). It is inspired by the pollination process of flowering plants. In this research, FPA is used for Test Case Prioritization (TCP) in Regression Testing (RT). The algorithm uses code coverage of test cases as the input. The algorithm has no prior information of faults covered by the test cases. This study deals with prioritizing (ordering) the test cases in such a way that only those test cases are executed that covers maximum faults in minimum time of execution. For validation of the results Average Percentage of Fault Detected (APFD) metrics is used. APFD values for different ordering of test cases is calculated for three applications written in Java. The empirical results of APFD metrics for FPA order (TS1) and FPA order (TSp) are better as compared to Random Order of Original Test Suite (TSo) and Reverse Random Order of (TSo). Therefore, this paper states that FPA for TCP gives efficient results in RT.

Background: In Regression testing when any changes made to already tested program it should not affect to other part of program. When some part of code is modified then it is necessary to validate the modified code. In the life cycle of software development, regression testing is an important and expensive activity. Objective: Our objective is to save regression testing time by reducing the cost and time of retesting of the modified code as well as affected components of the program. Objective is to prioritize the test case for object oriented program based on early fault detection. Methods: A new model is proposed using fuzzy inference system for regression testing. This paper presents a fuzzy inference system based on the use of inputs like fault detection rate, execution time and requirement coverage. The proposed system allows tester to prioritize test case based on linguistic rules. Results: In this paper performance is measured by using Average Percentage Fault Detection metric. We analysed for both prioritized and non-prioritized test suite using average percentage fault detection metric. Conclusion: We developed fuzzy expert model which helps to takes better decision than other expert system for regression testing. We conclude that, prioritization of test case will decrease the regression testing time.

This study examines the implementation of Test Case Prioritization (TCP) using the Greedy Algorithm (GA) to enhance regression testing efficiency within a financial technology company's software development cycle. With testing durations increasing significantly, this study aims to address inefficiencies by applying the Greedy Algorithm to optimize test suite size and fault detection. The research methodology involves applying the Greedy Algorithm during the Regression Testing phase, comparing the prioritized suite with the original suite using metrics such as Average Percentage Fault Detection (APFD) and Test Suite Size Reduction (TSSR). Results show that the Greedy Algorithm achieved substantial improvements in both test suite size and fault detection effectiveness across different projects. For Project A, the test suite was reduced from 51-22 test cases, achieving a TSSR of 56.8%, with an APFD increase of 206.21%, rising from 0.0853-0.2613. Project B demonstrated even greater optimization, reducing the test suite from 36-8 test cases, resulting in a TSSR of 77.8% and an APFD improvement of 83.92%, rising from 0.3194-0.5875. These outcomes underscore the algorithm’s effectiveness in eliminating redundant test cases, accelerating testing, and enhancing fault detection thereby supporting the company's goal of faster release cycles without compromising quality.

The process of regression testing is generally done under critical time and resource constraints. In the existing approaches, Test Case Prioritization (TCP), Test Case Selection (TCS), and Test Suite Reduction (TSR) have been treated as different optimization problems. This is based on simple heuristics that do not use valuable information that can be extracted from the program under test. Moreover, they do not allow for making internally consistent budget-conscious decisions. This paper introduces a unified approach that combines various variants of Graph Neural Networks (GNNs) and Reinforcement Learning (RL) for the solutions of TCP, TCS, and TSR. In our proposal, the representation of regression artifacts such as test cases, code entities, and faults is used as nodes in the typed graph, while the edges are used to show the relations between these artifacts. The relation-aware GNN is used for generating test case embeddings, reflecting the distance between the test cases and the changed areas as well as the areas known to have issues in the past. The actor-critic RL agent uses these test case embeddings and the budget to decide whether to run, skip, or discard test cases. The performance of our proposal outstrips coverage-based heuristics, history-based ranking, a GA-based search, an enhanced QPSO, and two learning-based ablations, as validated by our experiments on four Java projects from Defects4J. In terms of various programs and budgets, using GNN–RL has led to an improvement in the Average Percentage of Faults Detected (APFD) and Cost-cognizant Average Percentage of Faults Detected (APFDc) by approximately 6–9 percentage points on lower budgets. In addition, it is possible to obtain a 42% suite reduction and a 48% cost reduction while still retaining 98% of all fault-revealing tests in the final suite. The results show that it is possible to obtain promising solutions for adaptive regression testing using budget-aware reinforcement learning and graph-based representation learning.

Background: Regression testing is very important stage of software maintenance and evolution phase. The software keeps on updating and to preserve the software quality, it needs to be retested every time it is updated. Due to limited resources, complete testing of the software becomes tedious task. The probable solution to this problem is to execute those test cases first that are more important prior to less important test cases. Objective: Optimization methods need to be acquired for efficient test case prioritization in minimum time, while maintaining the quality of the software. Various nature-inspired algorithms like genetic algorithm, particle swarm optimization and ant colony optimization, etc. have been applied for prioritizing test cases. In this paper, we have applied a relatively new nature-inspired optimization method, namely, Bat algorithm that utilizes the echolocation, loudness and pulse emission rate of bats in prioritizing the test cases. Methods: The proposed algorithm is experimented on sample case study of timetable management system and evaluated with the help of popular evaluation metric Average Percentage of Fault Detection. Results: Results have been compared with the baseline approaches i.e. untreated, random and reverse prioritization and well-established optimization method i.e. genetic algorithm and we have found a considerable increase in the value of evaluation metric. Conclusion: This preliminary study shows that the bat algorithm have great potential to solve test case prioritization problems.

On customer's demand, new requirements are implemented in the software.The modified software may not work properly as earlier because of the new requirements added.So the modified software must be tested.Regression testing (RT) is defined as retesting of the modified software.It is performed using the already developed test suite and a newly developed test suite.The big software has a larger test suite size.For a single requirement change, to run the whole test cases is not beneficial for the development organisation.To make RT more effective, prioritisation of test suite is done.Here we present the genetic algorithm (GA) for the test case prioritisation (TCP).Different approaches have been discussed and implemented using the average percentage of fault detected (APFD) metric.The discussed approaches are applied over a single problem and the result is shown in the tabular form.APFD metric is applied to all the discussed approaches and suggested which one is better.This paper uses GA to arrange the test cases in a prioritised way on the basis of the fault detected.

Regression testing (RT) is an expensive activity. It is applied on a modified program to enhance confidence and reliability by ensuring that the changes are accurately true and have not affected the unmodified portions of the SUT. Due to limited resources, it is not practical to re-run each test cases (TC). To improve the regression testing's effectiveness, the TCs should be arranged according to some objective function or criteria. Test case prioritization (TCP) arranges TCs in an order for execution that enhances their effectiveness by satisfying some testing goals. The highest priority assigned to TCs must execute before the TCs with low priority by virtue of some performance goal. Numerous goals are possible to achieve of which one such goal is rate of fault detection (RFT) in which the faults are surfaced as quickly as possible within the testing process. In this paper, a novel technique is suggested to prioritize the TCs that increase its effectiveness in detecting faults. The effectiveness of the proposed method is compared and matched with other prioritization approaches with the help of Average Percentage of Fault Detection (APFD) metric from which charts have been prepared.

The fully automatic Graphical User Interface tool for any application using novel model based test suite generation techniques for a GUI. They are unable to control response time and time intervals are based on relationship between GUI events handlers and test cases with their responsibilities. We present a novel prioritization algorithm that enhances event handlers for the automated GUI tool. The proposed tool generates GUI events, it Captures and Playback event responses to automatic verification point of the results for the test cases which are written to a log file and corresponding report will be generated. This novel algorithm was able to detect new test suite and ordering of test cases to reduce a GUI fault integration defects. The number of faults detected for a single event are found after generating test cases for the application. The Average Percentage of Fault Detection (APFD) and charts has been used to show the effectiveness of proposed algorithm to find fault detection rate.

This paper discusses regression testing in software maintenance, focusing on test case prioritization to verify modifications to software functionality efficiently. The primary goal is to rank test cases, prioritizing those covering more code or faults with minimal execution time. The challenge lies in prioritizing numerous test cases generated during development and maintenance. Various algorithms, including greedy approaches and meta-heuristic techniques, address this challenge. The paper introduces a ranking-based non-dominated sorting genetic algorithm (NSGA-2) for test case prioritization, emphasizing cases sensitive to faults caused by modifications or new functionality. Historical data is prioritized, with key objectives including the sensitive index, execution cost, and average percentage of fault detection (APFD). The proposed model is tested on handcrafted and benchmark Java-based applications, comparing its performance to state-of-the-art algorithms in test case prioritization.

Background: In today’s era, modifications in a software is a common requirement by customers. When changes are made to existing software, re-testing of all the test cases is required to ensure that the newly introduced changes do not have any unwanted effect on the behavior of the software. However, retesting of all the test cases would not only be time consuming but also expensive. Therefore, there is a need for a technique that reduces the number of tests to be performed. Regression testing is one of the ways to reduce the number of test cases. Selection technique is one such method which seeks to identify the test cases that are relevant to some set of recent changes. Objective: It is evident that most of the studies have used different selection techniques and have focused only on one parameter for achieving reduced test suite size without compromising the performance of regression testing. However, to the best of our knowledge, no study has taken two or more parameters of coverage, and/or execution time in a single testing. This paper presents a hybrid technique that combines both regression test selection using slicing technique and minimization of test cases using modified firefly algorithm with combination of parameters coverage and execution time in a single testing. Methods: A hybrid technique has been described that combines both selection and minimization. Selection of test cases is based upon slicing technique while minimization is done using firefly algorithm. Hybrid technique selects and minimizes the test suite using information on statement coverage and execution time. Results: The proposed technique gives 43.33% much superior result as compared to the other hybrid approach in terms of significantly reduced number of test cases. It shows that the resultant test cases were effective enough to cover 100% of the statements, for all the programs. The proposed technique was also tested on four different programs namely Quadratic, Triangle, Next day, Commission respectively for test suite selection and minimization which gave comparatively superior result in terms of reduction (%) in number of test cases required for testing. Conclusion: The combination of parameters used in slicing based approach, reduces the number of test cases making software testing an economical, feasible and time saving option without any fault in the source code. This proposed technique can be used by software practitioners/experts to reduce time, efforts and resources for selection and minimization of test cases.

Graph Neural Networks (GNNs) are widely applied in 3D computer vision tasks. However, validating their reliability is limited by the high cost of labeling large‐scale test data. Existing test case prioritization techniques for deep learning often rely on single‐dimensional heuristics or mutation‐based analysis (e.g., GraphPrior, PCPrior). However, these methods often limit fault diversity or incur prohibitive computational overhead. To address these issues, this paper proposes LeF‐MTP, a multi‐dimensional test prioritization framework. Unlike these state‐of‐the‐art methods that depend on computationally intensive external mutations, LeF‐MTP directly exploits the model's internal cognitive states. The method characterizes test cases through two dimensions: the predictive uncertainty and feature‐space confusability. These cognitive features are fused via a Random Forest classifier to predict failure probabilities. The resulting priority list is then optimized by a diversity‐aware re‐ranking process based on feature clustering. We evaluated the framework on four 3D point cloud datasets (including ScanObjectNN and S3DIS) across four GNN architectures. Experimental results demonstrate that LeF‐MTP achieves higher fault detection rates than heuristic and mutation‐based baselines, with Average Percentage of Fault Detection (APFD) improvements ranging from 8.79% to 74.69%. Moreover, an efficiency analysis demonstrates that the proposed approach reduces feature generation time compared to mutation‐based methods, validating its practicality for GNN testing.

One of the most expensive activities that occurs as software is developed and maintained is the testing (or retesting) of the software after it has been modified. Studies suggest that a significant portion of development and maintenance costs go to this retesting, which is known as regression testing. Reports estimate that regression testing consumes as much as 80% of the overall testing budget and can consume up to 50% of the cost of software maintenance. Rapidly changing software and computing environments present many challenges for effective and efficient regression testing in practice. Regression testing can be performed after changes are made to the software, such as after nightly or regular builds, before a new version of the software is released, every time the software is saved and compiled, such as in an agile development environment, or before patches, such as security patches, are released. Regardless of the environment or when it is performed, the goals of regression testing are the same: to improve confidence that the changes behave as intended and that they have not adversely affected unchanged parts of the software. Because regression testing is important, but expensive, much research has been performed, both in industry and in academia, to develop techniques to make regression testing more effective and efficient. This research has also produced many tools and systems that have been used for empirical studies that investigate the effectiveness, scalability, and practicality of the techniques. Researchers have developed techniques for addressing a number of issues related to regression testing, and, in this talk, I will discuss them in four areas. First, techniques attempt to reduce the regression testing time by creating effective regression test suites that test the changed part of the software, by identifying test cases in the regression test suite that do not need to be rerun on the changed software, and by identifying and removing obsolete test cases. Second, techniques can reuse test suites created for one version of the software by identifying those test cases that need to be rerun for testing subsequent versions of the software and by computing an effective ordering for running the test cases. Third, techniques can recycle test cases by monitoring executions to gather test inputs that can be used for retest-ing and by creating unit test cases from system test cases. Finally, techniques could recover test cases by identifying, manipulating, and transforming obsolete test cases, by generating new test cases from old ones, and by repairing test cases when the software changes. In this talk, I will overview the research in testing of evolving software, and discuss achievements to date in managing regression testing by reducing, reusing, recycling, and recovering test cases. I will also present the state of the research and the state of the practice in regression testing. Finally, I will discuss the current trends in both academia and industry, the challenges for solving the difficult problems that exist, the promise for testing of evolving software in the future, and the important open challenges for regression testing in the next decade.

Regression testing of the software during its maintenance phase, requires test case prioritization and selection due to the dearth of the allotted time. The resources and the time in this phase are very limited, thus testers tend to use regression testing methods such as test case prioritization and selection. The current study evaluates the effectiveness of testing with two major goals: (1) Least running time and (2) Maximum fault coverage possible. Ant Colony Optimization (ACO) is a well-known soft computing technique that draws its inspiration from nature and has been widely researched, implemented, analyzed, and validated for regression test prioritization and selection. Many versions of ACO approaches have been prolifically applied to find solutions to many non-polynomial time-solvable problems. Hence, an attempt has been made to enhance the performance of the existing ACO_TCSP algorithm without affecting its time complexity. There have been efforts to enhance the exploration space of various paths in each iteration and with elite exploitation, reducing the total number of iterations required to converge to an optimal path. Counterbalancing enhanced exploration with intelligent exploitation implies that the run time is not adversely affected, the same has also been empirically validated. The enhanced algorithm has been compared with the existing ACO algorithm and with the traditional approaches. The approach has also been validated on four benchmark programs to empirically evaluate the proposed Enhanced ACO_TCSP algorithm. The experiment revealed the increased cost-effectiveness and correctness of the algorithm. The same has also been validated using the statistical test (independent t-test). The results obtained by evaluating the proposed approach against other reference techniques using Average Percentage of Faults Detected (APFD) metrics indicate a near-optimal solution. The multiple objectives of the highest fault coverage and least running time were fruitfully attained using the Enhanced ACO_TCSP approach without compromising the complexity of the algorithm.

Regression testing is an important and costly strategy in software testing. To decrease its cost, many methods have been proposed including the test case prioritization methods. The aim of the prioritization methods is to define an ideal order of test cases that allows for higher coverage and early fault detection with minimal number of executed test cases. However, the problem with most of the existing test case prioritization methods is the random sorting of test cases when two or more test cases record equal priority values. In this paper, the effectiveness of an enhanced weighted method using a unique priority value, UniVal, is proposed. Unival prioritizes test cases based on code coverage criteria with information from history of previous executions. In addition, a controlled experiment was executed, and the results were statistically analyzed to assess the effectiveness of the proposed method. The results indicates better performance with regard to prioritize test cases and achieve higher APFD values.  Â

Software testing is one of the important stages of software development. In software development, developers always depend on testing to reveal bugs. In the maintenance stage test suite size grow because of integration of new technique. Addition of new technique force to create new test case which increase the size of test suite. In regression testing new test case may be added to the test suite during the whole testing process. These additions of test cases create possibility of presence of redundant test cases. Due to limitation of time and resource, reduction techniques should be used to identify and remove them. Research shows that a subset of the test case in a suit may still satisfy all the test objectives which is called as representative set. Redundant test case increase the execution cost of the test suite, in spite of NP-completeness of the problem there are few good reduction techniques have been available. In this paper a new approach for test case reduction is proposed. This algorithm use genetic algorithm technique iteratively with varying chromosome length to reduce test case in a test suit by finding representative set of test cases that are fulfill the testing criteria.