Regression Test Case Selection &PrioritizationUsing Dependence Graph and Genetic Algorithm

Background: In today’s era, modifications in a software is a common requirement by customers. When changes are made to existing software, re-testing of all the test cases is required to ensure that the newly introduced changes do not have any unwanted effect on the behavior of the software. However, retesting of all the test cases would not only be time consuming but also expensive. Therefore, there is a need for a technique that reduces the number of tests to be performed. Regression testing is one of the ways to reduce the number of test cases. Selection technique is one such method which seeks to identify the test cases that are relevant to some set of recent changes. Objective: It is evident that most of the studies have used different selection techniques and have focused only on one parameter for achieving reduced test suite size without compromising the performance of regression testing. However, to the best of our knowledge, no study has taken two or more parameters of coverage, and/or execution time in a single testing. This paper presents a hybrid technique that combines both regression test selection using slicing technique and minimization of test cases using modified firefly algorithm with combination of parameters coverage and execution time in a single testing. Methods: A hybrid technique has been described that combines both selection and minimization. Selection of test cases is based upon slicing technique while minimization is done using firefly algorithm. Hybrid technique selects and minimizes the test suite using information on statement coverage and execution time. Results: The proposed technique gives 43.33% much superior result as compared to the other hybrid approach in terms of significantly reduced number of test cases. It shows that the resultant test cases were effective enough to cover 100% of the statements, for all the programs. The proposed technique was also tested on four different programs namely Quadratic, Triangle, Next day, Commission respectively for test suite selection and minimization which gave comparatively superior result in terms of reduction (%) in number of test cases required for testing. Conclusion: The combination of parameters used in slicing based approach, reduces the number of test cases making software testing an economical, feasible and time saving option without any fault in the source code. This proposed technique can be used by software practitioners/experts to reduce time, efforts and resources for selection and minimization of test cases.

The process of regression testing is generally done under critical time and resource constraints. In the existing approaches, Test Case Prioritization (TCP), Test Case Selection (TCS), and Test Suite Reduction (TSR) have been treated as different optimization problems. This is based on simple heuristics that do not use valuable information that can be extracted from the program under test. Moreover, they do not allow for making internally consistent budget-conscious decisions. This paper introduces a unified approach that combines various variants of Graph Neural Networks (GNNs) and Reinforcement Learning (RL) for the solutions of TCP, TCS, and TSR. In our proposal, the representation of regression artifacts such as test cases, code entities, and faults is used as nodes in the typed graph, while the edges are used to show the relations between these artifacts. The relation-aware GNN is used for generating test case embeddings, reflecting the distance between the test cases and the changed areas as well as the areas known to have issues in the past. The actor-critic RL agent uses these test case embeddings and the budget to decide whether to run, skip, or discard test cases. The performance of our proposal outstrips coverage-based heuristics, history-based ranking, a GA-based search, an enhanced QPSO, and two learning-based ablations, as validated by our experiments on four Java projects from Defects4J. In terms of various programs and budgets, using GNN–RL has led to an improvement in the Average Percentage of Faults Detected (APFD) and Cost-cognizant Average Percentage of Faults Detected (APFDc) by approximately 6–9 percentage points on lower budgets. In addition, it is possible to obtain a 42% suite reduction and a 48% cost reduction while still retaining 98% of all fault-revealing tests in the final suite. The results show that it is possible to obtain promising solutions for adaptive regression testing using budget-aware reinforcement learning and graph-based representation learning.

In the software industry, software testing becomes more important in the entire software development life cycle. Software testing is one of the fundamental components of software quality assurances. Software Testing Life Cycle (STLC)is a process involved in testing the complete software, which includes Regression Testing, Unit Testing, Smoke Testing, Integration Testing, Interface Testing, System Testing & etc. In the STLC of Regression testing, test case selection is one of the most important concerns for effective testing as well as cost of the testing process. During the Regression testing, executing all the test cases from existing test suite is not possible because that takes more time to test the modified software. This paper proposes new Hybrid approach that consists of modified Greedy approach for handling the test case selection and Genetic Algorithm uses effective parameter like Initial Population, Fitness Value, Test Case Combination, Test Case Crossover and Test Case Mutation for optimizing the tied test suite. By doing this, effective test cases are selected and minimized the tied test suite to reduce the cost of the testing process. Finally the result of proposed approach compared with conventional greedy approach and proved that our approach is more effective than other existing approach.

In software maintenance, after modifying the software a system needs regression testing. Execution of regression testing confirms that any modified code has no adverse effect as well as does not introduce new faults in the existing functionality of the software. When working with object-oriented programming code-based testing is generally expensive. In this study, we proposed a technique for regression testing using unified modeling language (UML) diagrams and code-based analysis for object-oriented software. In this research work, the design and code based technique with an evolutionary approach are presented to select the best possible test cases from the test suite. We used the dependency graph for intermediate representation for the objectoriented program to identify the change. The selection of test cases is done at the design level using the UML model. The models are compared to identify the change between these two models. The proposed approached maximizes the value of APFD.

A new heuristic algorithm is proposed by this paper, on multi-objective optimization using metaheuristics and TSP (travelling salesman problems). Basic thinking behind this algorithm is minimizing the TSP path or tour by dividing the entire tour into blocks that are overlapped to each other and then improve each individual block separately. Although it is unproven that a good solution have small improvement chances if a node moved far way to a position compared to its original solution. By intensively searching each block, further improvement is possible in TSP path or tour that never be supported in various search methods and genetic algorithm. Proposed algorithm and computational experiment performance was tested, and these tests are carried out with support of already present instances of problem. According to the results represented by paper, the computation verifies that proposed algorithm can solve TSPs efficiently. Proposed algorithm is then used for selecting optimal test cases, thousands of those test cases which are selected after confirming that they identify bugs and they itself selected from a repository of test cases; these thousand test cases are those test cases which are selected from several thousand test cases because they detect bugs. Few test cases from repository act as milestones (nodes) and having certain weight associated with each, proposed algorithm based on TSP implemented over selected result and select the optimal result or path or solution. These selected optimal test cases or selected path are further used to perform the regression testing, by applying those test cases selected by proposed algorithm in order to remove most of the faults or bugs effectively, i.e. take less time and identify almost all the bugs with few test cases. Hence this proposed algorithm assures most effective solution for regression testing test case selection.

Testing a group of software artifacts that implement the same specification can be time consuming, especially when the test case repository is large. In the meantime, some test cases may cover the same aspects in the software, thus it is not necessary to apply all the test cases. This paper proposes a model-based adaptive test (MAT) case selection and ranking technique to eliminate duplicate test cases, i.e., test cases with the similar coverage, and rank the test cases according to their potency and coverage. This technique can be applied in various domains where multiple versions of applications are available to test, such as web service testing, n-version applications, regression testing, and standard-based testing. The MAT is based a statistical model based on earlier testing results, and the model can determine the next sets of test cases to minimize the testing effort. The MAT is then applied to testing of multi-versioned web services and the results shows that the MAT can reduce testing effort while still maintain the effectiveness of testing.

One of the most expensive activities that occurs as software is developed and maintained is the testing (or retesting) of the software after it has been modified. Studies suggest that a significant portion of development and maintenance costs go to this retesting, which is known as regression testing. Reports estimate that regression testing consumes as much as 80% of the overall testing budget and can consume up to 50% of the cost of software maintenance. Rapidly changing software and computing environments present many challenges for effective and efficient regression testing in practice. Regression testing can be performed after changes are made to the software, such as after nightly or regular builds, before a new version of the software is released, every time the software is saved and compiled, such as in an agile development environment, or before patches, such as security patches, are released. Regardless of the environment or when it is performed, the goals of regression testing are the same: to improve confidence that the changes behave as intended and that they have not adversely affected unchanged parts of the software. Because regression testing is important, but expensive, much research has been performed, both in industry and in academia, to develop techniques to make regression testing more effective and efficient. This research has also produced many tools and systems that have been used for empirical studies that investigate the effectiveness, scalability, and practicality of the techniques. Researchers have developed techniques for addressing a number of issues related to regression testing, and, in this talk, I will discuss them in four areas. First, techniques attempt to reduce the regression testing time by creating effective regression test suites that test the changed part of the software, by identifying test cases in the regression test suite that do not need to be rerun on the changed software, and by identifying and removing obsolete test cases. Second, techniques can reuse test suites created for one version of the software by identifying those test cases that need to be rerun for testing subsequent versions of the software and by computing an effective ordering for running the test cases. Third, techniques can recycle test cases by monitoring executions to gather test inputs that can be used for retest-ing and by creating unit test cases from system test cases. Finally, techniques could recover test cases by identifying, manipulating, and transforming obsolete test cases, by generating new test cases from old ones, and by repairing test cases when the software changes. In this talk, I will overview the research in testing of evolving software, and discuss achievements to date in managing regression testing by reducing, reusing, recycling, and recovering test cases. I will also present the state of the research and the state of the practice in regression testing. Finally, I will discuss the current trends in both academia and industry, the challenges for solving the difficult problems that exist, the promise for testing of evolving software in the future, and the important open challenges for regression testing in the next decade.

Regression testing as an essential activity in software development that has changed requirements. In practice, regression testing requires a lot of time so that an optimal strategy is needed. One approach that can be used to speed up execution time is the Regression Test Selection (RTS) approach. Currently, practitioners and academics have started to think about developing tools to optimize the process of implementing regression testing. Among them, STARTS and Ekstazi are the most popular regression testing tools among academics in running test case selection algorithms. This article discusses the comparison of the capabilities of the STARTS and Ekstazi features by using feature parameter evaluation. Both tools were tested with the same input data in the form of System Under Test (SUT) and test cases. The parameters used in the tool comparisons are platform technology, test case selection, functionality, usability and performance efficiency, the advantages, and disadvantages of the tool. he results of the trial show the differences and similarities between the features of STARTS and Ekstazi, so that it can be used by practitioners to take advantage of tools in the implementation of regression testing that suit their needs. In addition, experimental results show that the use of Ekstazi is more precise in sorting out important test cases and is more efficient, when compared to STARTS and regression testing with retest all.

Automated code-based test selection for software product line regression testing

Mobile software application development process must be matured enough to handle the challenges (especially market related) associated with the development of high quality mobile software development. Ever increasing number of both mobile users and mobile applications had presented software engineers with the challenge of satisfying billions of users with high quality software applications to be delivered within deadline and budgets. Always there had been a lot of pressure to develop complex software categorized by thousands of requirements, under resource constrained environment. Requirement prioritization is one of the activities undertaken by software engineer to deliver partial software product to its customers such that most important requirements are implemented in the earliest releases. During next releases some changed and pending requirements are implemented, an activity that generates ripple effects. Such ripple effects need to be tested by executing modified source code against test cases of previous releases (regression testing). Regression testing is a very effortful activity that requires a software tester to select test cases that have high fault detection capability, execute the modified code against selected test cases and performing debugging. This regression testing activity can be lowered to the maximum extend by considering dependencies between requirements during the time of requirement prioritization. Thus requirement prioritization will be carried out not only against aspects like cost, time, risks, business values etc but against dependencies also. The aim is to implement almost all dependent highest priority requirements in current release so that implementation of new requirements is unlikely to have ripple effects. Changes in requirements might not be related to variable usage and definition and might not involve a change in functionality. In such cases there is no need to select already executed test cases of previous versions. Module dependencies can lead to test case selections of previous versions if changes of requirement lead to ripple effects. This paper aims to implement highest priority requirements such that regression testing is performed to minimum thereby improving development process of mobile applications. The proposed technique had been successfully evaluated on Android based notification software application that meets the specification of Aakash tablets.

We consider regression testing of safety-critical systems consisting of black-box components. This scenario is common for automotive electronic systems where testing time is expensive and should be reduced without uncontrolled reduction of reliability. This requires a methodology to select test cases from a larger test suite for which a defined quality level of the resulting regression test cycle can be provided. With this in mind, we propose a method to select test cases based on a stochastic model. We are modeling test case failure probabilities as dependent random variables, therefore already observed test results have an influence on the estimation of further test case failure probabilities. Based on an information theoretical approach, we validate the mutual differential information degree between test case failure probabilities and compute a function which returns the risk for not selecting a test case, i.e., the probability of not selecting a failing test case. Depending on the mutual differential information significant reductions of testing time can be achieved while testing reliability is preserved at a quantifiable high level. We will validate theoretically our results and will show in an industrial case study the benefits of our method.

Regression tests ensure that existing functionality is not impaired by changes to an existing software system. However, executing complete test suites often takes much time. Therefore, a subset of tests has to be found that is sufficient to validate whether the system under test is still valid after it has been changed. This test case selection is especially important if regression tests are executed manually, since manual execution is time intensive and costly. To select manual test cases, many regression testing techniques exist that aim on achieving coverage of changed or even new code. Many of these techniques base on coverage data from prior test runs or logical properties such as annotated pre and post conditions in the source code. However, coverage information becomes outdated if a system is changed extensively. Also annotated logical properties are often not available in industrial software systems. We present an approach for test selection that is based on static analyses of the test suite and the system's source code. We combine trace link recovery using latent semantic indexing with the metric change coverage, which assesses the coverage of source code changes. The proposed approach works automatically without the need to execute tests beforehand or annotate source code. Furthermore, we present a first evaluation of the approach.

Regression testing, which is performed after changes are made to a software system, can be used before release of new versions of the system. However, practitioners often have little time to perform this regression testing because of the quick-release cycles of such modified systems. Thus, they may use a random-testing approach or perform little regression testing. This lack of adequate regression testing can cause bugs in untested parts of the program to be exposed only during production or field usage. To improve the efficiency of the regression testing, and thus enable its use before release, techniques that select and run only those test cases that are related to the changes or prioritize the test cases based on criticality or perceived effectiveness have been presented. These technique typically use some representation of the software such as a system model or the source code to perform the test selection and prioritization. However, in practice, access to a system modelor the source code may not be possible. To provide regression test selection in practice, we have developed, and present in this paper, a novel approach to regression test selection that, instead of using a system model or the source code, uses the system requirements and their associated test cases, which are typically available to developers/testers. The approach uses the set of system requirements, usually in natural language or some informal notation, that represent what is to be tested about the system. The technique uses these requirements, along with the set of test cases and their criticality that are associated with the requirements, to select test cases for use in regression testing. In this paper, we also present a case study that shows the potential effectiveness of our technique

Software testing (testing) is a crucial stage in software development. The success of the testing process will ensure the quality of the software. In the regression testing process, one issue is that not all test cases (retest all) in the test suite need to be executed. Retest all will consume massive resources, as well as a long time. Regression testing techniques seek to find ways to reduce test execution time. One of the regression testing techniques is test case selection, also known as regression test selection (RTS). This paper describes a study on babelRTS, an RTS algorithm, to see its effectiveness. Effectiveness is measured by comparing the execution time of the execution retest all and babelRTS. Experiments were carried out on five software under tests (SUT) that had some faults. Test cases are prepared by designing for each SUT. The results showed a reduction in time so that the effectiveness reached a maximum of 32%, and average of 23% .

Testing is considered as indispensable part in the development process life cycle of any software. Testing is performed to improve the performance, quality and reliability of the software. Designing of relevant test cases plays a major role in deciding the quality of testing. Hence test case designing activity is crucial for the success of testing. Combinatorial strategies are one of those most extensively used methods for generating test cases. These strategies detect failures which are caused by interactions of parameters in the Software under Test(SUT) and by using some sampling mechanisms a covering array test suite is generated. With increase in the number of parameter coverage, the t-way test size sets also increase exponentially which would result in explosion problem. This paper presents a technique to generate minimal test suites in Combinatorial testing(CT). Optimization technique used is Greedy computation strategy for mutation and selection of test cases, used in selecting the locally worst test case for elimination. That is the test case that does not cover maximum number of uncovered interactions from a pool of candidate test cases. Additionally, Branch and bound technique is used for cross over. It is used to branch for the possibilities of number of test cases as cross over inputs. If a particular test case is not possible the bounding happens, else branched for further selection of test cases as input. Experimental results show that Recast Ipog-D generates better/comparable results as compared to the existing algorithm. Additionally, Recast IPOG-D has also contributed to enhance many known Variable Strength Covering array(VSCA) that exist in literature.