A similarity-aware approach to testing based fault localization

Mutation-oriented test data augmentation for GUI software fault localization

Software Testing is an important aspect of the real time software development process. Software testing always assures the quality of software product. As associated with software testing, there are few very important issues where there is a need to pay attention on it in the process of software development test. These issues are generation of effective test case and test suite as well as optimization of test case and suite while doing testing of software product. The important issue is that testing time of the test case and test suite. It is very much important that after development of software product effective testing should be performed. So to overcome these issues of optimization, we have proposed new approach for test suite optimization using genetic algorithm (GA). Genetic algorithm is evolutionary in nature so it is often used for optimization of problem by researcher. In this paper, our aim is to study various selections methods like tournament selection, rank selection and roulette wheel selection and then we apply this genetic algorithm (GA) on various programs which will generate optimized test suite with parameters like fitness value of test case, test suite and take minimum amount of time for execution after certain preset generation. In this paper our main objectives as per the experimental investigation, we show that tournament selection works very fine as compared to other methods with respect fitness selection of test case and test suites, testing time of test case and test suites as well as number of requirements.

In order to improve the effectiveness of fault localization, various test suite reduction techniques have been proposed. However, excessive or improper reduction of test cases may lose some testing information, thus causing a negative impact on fault localization. In this paper, we propose a new similarity-based test suite reduction approach to improving spectrum-based fault localization. Firstly, this approach extracts the high suspicious statements in the faulty program and removes the coincidental passed test cases for test suite selection. Then, it selects similar passed test cases for each different failed test case from the new passed test set based on the similar proportion of their execution traces, and determines the final composition of each similar test set based on the contribution of failed test cases to fault localization. By using the execution information of each similar test set with a spectrum-based fault localization approach, the ranks of statements can be obtained. Finally, synthesizing all ranks of statements in each rank list, we obtain the final rank list of statements. Several experiments show that our approach can help reduce the debugging effort in terms of the percentage of statements needed to be inspected when locating faults in both single-fault and multi-fault programs. Moreover, the results demonstrate that the value of similar proportion have nontrivial influence on the effectiveness of fault localization.

With the emergence of social coding platforms, collaboration has become a key and dynamic aspect to the success of software projects. In such platforms, developers have to collaborate and deal with issues of collaboration in open‐source software development. Although collaboration is challenging, collaborative development produces better software systems than any developer could produce alone. Several approaches have investigated collaboration challenges, for instance, by proposing or evaluating models and tools to support collaborative work. Despite the undeniable importance of the existing efforts in this direction, there are few works on collaboration from perspectives of developers. In this work, we aim to investigate the perceptions of open‐source software developers on collaborations, such as motivations, techniques, and tools to support global, productive, and collaborative development. Following an ad hoc literature review, an exploratory interview study with 12 open‐source software developers fromGitHub, our novel approach for this problem also relies on an extensive survey with 121 developers to confirm or refute the interview results. We found different collaborative contributions, such as managing change requests. Besides, we observed that most collaborators prefer to collaborate with the core team instead of their peers. We also found that most collaboration happens in software development (60%) and maintenance (47%) tasks. Furthermore, despite personal preferences to work independently, developers still consider collaborating with others in specific task categories, for instance, software development. Finally, developers also expressed the importance of the social coding platforms, such asGitHub, to support maintainers, and contributors in making decisions and developing tasks of the projects. Therefore, these findings may help project leaders optimize the collaborations among developers and reduce entry barriers. Moreover, these findings may support the project collaborators in understanding the collaboration process and engaging others in the project.

Software testing is one of the important stages of software development. In software development, developers always depend on testing to reveal bugs. In the maintenance stage test suite size grow because of integration of new technique. Addition of new technique force to create new test case which increase the size of test suite. In regression testing new test case may be added to the test suite during the whole testing process. These additions of test cases create possibility of presence of redundant test cases. Due to limitation of time and resource, reduction techniques should be used to identify and remove them. Research shows that a subset of the test case in a suit may still satisfy all the test objectives which is called as representative set. Redundant test case increase the execution cost of the test suite, in spite of NP-completeness of the problem there are few good reduction techniques have been available. In this paper a new approach for test case reduction is proposed. This algorithm use genetic algorithm technique iteratively with varying chromosome length to reduce test case in a test suit by finding representative set of test cases that are fulfill the testing criteria.

Software testing is a critical part of software development. As new test cases are generated over time due to software modifications, test suite sizes may grow significantly. Because of time and resource constraints for testing, test suite minimization techniques are needed to remove those test cases from a suite that, due to code modifications over time, have become redundant with respect to the coverage of testing requirements for which they were generated. Prior work has shown that test suite minimization with respect to a given testing criterion can significantly diminish the fault detection effectiveness (FDE) of suites. We present a new approach for test suite reduction that attempts to use additional coverage information of test cases to selectively keep some additional test cases in the reduced suites that are redundant with respect to the testing criteria used for suite minimization, with the goal of improving the FDE retention of the reduced suites. We implemented our approach by modifying an existing heuristic for test suite minimization. Our experiments show that our approach can significantly improve the FDE of reduced test suites without severely affecting the extent of suite size reduction

The availability of effective test suites is critical for the development and maintenance of reliable software systems. To increase test effectiveness, software developers tend to employ larger and larger test suites. The recent availability of software tools for automatic test generation makes building large test suites affordable, therefore contributing to accelerating this trend. However, large test suites, though more effective, are resources and time consuming and therefore cannot be executed frequently. Reducing them without decreasing code coverage is a needed compromise between efficiency and effectiveness of the test, hence enabling a more regular check of the software under development. We propose a novel approach, namely REDUNET, to reduce a test suite while keeping the same code coverage. We integrate this approach in a complete framework for the automatic generation of efficient and effective test suites, which includes test suite generation, code coverage analysis, and test suite reduction. Our approach formulates the test suite reduction as a set cover problem and applies integer linear programming and a network-based optimisation, which takes advantage of the properties of the control flow graph. We find the optimal set of test cases that keeps the same code coverage in fractions of seconds on real software projects and test suites generated automatically by Randoop. The results on ten real software systems show that the proposed approach finds the optimal minimisation and achieves up to 90% reduction and more than 50% reduction on all systems under analysis. On the largest project our reduction algorithm performs more than three times faster than both integer linear programming alone and the state-of-the-art heuristic Harrold Gupta Soffa.

Database Management System (DBMS) developers have implemented extensive test suites to test their DBMSs. For example, the SQLite test suites contain over 92 million lines of code. Despite these extensive efforts, test suites are not systematically reused across DBMSs, leading to wasted effort. Integration is challenging, as test suites use various test case formats and rely on unstandardized test runner features. We present a unified test suite, SQuaLity, in which we integrated test cases from three widely-used DBMSs, SQLite, PostgreSQL, and DuckDB. In addition, we present an empirical study to determine the potential of reusing these systems' test suites. Our results indicate that reusing test suites is challenging: First, test formats and test runner commands vary widely; for example, SQLite has 4 test runner commands, while MySQL has 112 commands with additional features, to, for example, execute file operations or interact with a shell. Second, while some test suites contain mostly standard-compliant statements (e.g., 99% in SQLite), other test suites mostly test non-standardized functionality (e.g., 31% of statements in the PostgreSQL test suite are nonstandardized). Third, test reuse is complicated by various explicit and implicit dependencies, such as the need to set variables and configurations, certain test cases requiring extensions not present by default, and query results depending on specific clients. Despite the above findings, we have identified 3 crashes, 3 hangs, and multiple compatibility issues across four different DBMSs by executing test suites across DBMSs, indicating the benefits of reuse. Overall, this work represents the first step towards test-case reuse in the context of DBMSs, and we hope that it will inspire follow-up work on this important topic.

Guest Editorial: Special Section from the 11th International Conference on Quality Software (QSIC 2011)

Improving the effectiveness of test suite reduction for user-session-based testing of web applications

Bug or Fault localization is a process of identifying the specific location(s) or region(s) of source code (at various granularity levels such as the directory path, file, method or statement) that is faulty and needs to be modified to repair the defect. Fault localization is a routine task in software maintenance (corrective maintenance). Due to the increasing size and complexity of current software applications, automated solutions for fault localization can significantly reduce human effort and software maintenance cost.We present a technique (which falls into the class of static techniques for bug localization) for fault localization based on a character n-gram based Information Retrieval (IR) model. We frame the problem of bug localization as a relevant document(s) search task for a given query and investigate the application of character-level n-gram based textual features derived from bug reports and source-code file attributes. We implement the proposed IR model and evaluate its performance on dataset downloaded from two popular open-source projects (JBoss and Apache).We conduct a series of experiments to validate our hypothesis and present evidences to demonstrate that the proposed approach is effective. The accuracy of the proposed approach is measured in terms of the standard and commonly used SCORE and MAP (Mean Average Precision) metrics for the task of fault localization. Experimental results reveal that the median value for the SCORE metric for JBoss and Apache dataset is 99.03% and 93.70% respectively. We observe that for 16.16% of the bug reports in the JBoss dataset and for 10.67% of the bug reports in the Apache dataset, the average precision value (computed at all recall levels) is between 0.9 and 1.0.

Spectral fault localization is an automatic fault-localization technique to expedite debugging, which uses risk evaluation formula to rank the risk of fault existence in each program entity after collecting testing information. To assess the potential usefulness of a test suite and improves the accuracy for spectral fault localization, methods of assessing and optimizing test suite are proposed in this paper, which. Firstly, Average Ranking Cost of the test suite quality and two kinds of constrains are defined; and test suite quality assessment method based on these definitions is given. Secondly, a new test suite optimization method based on greedy algorithm is proposed. Finally, two widely used program databases (SIR and Defects4j) and 8 SFL techniques are applied to verify the effectiveness of our method; and the fault localization cost before and after optimizing test suites of test objects are analyzed using effect size. The largest effect size reaches 0.5398 and Each SFL technology has different degrees of improvement in the rankings of faulty statements in different programs by optimizing test suite.

Software testing and retesting occurs continuously during the software development lifecycle to detect errors as early as possible. As the software evolves the size of test suites also grows. When the no of test cases generated are more, obviously size of the test suite will also be more. So the testing time is to be minimized by reducing the execution time of the algorithm used for test data generation and also by introducing minimization procedure for test suite reduction. Due to limited resources and timing constraints for testing, test suite minimization techniques are needed to eliminate redundant test cases as possible. By considering multiple objectives rather than the coverage alone, the test cases are being generated which satisfies the testing requirements. Most of the existing techniques are code-based. In this article we present an approach by modifying an existing heuristic for test suite minimization. Genetic algorithm has been used for random test data generation and the output of GA is given to the minimization procedure for reducing the total no of generated test cases, collectively named as Hybrid Algorithm (HA). The results are satisfactory and show significant improvements in reducing test suite size with minimum execution time. Experiments have been done for simple to medium complexity java programs taken from SIR and execution time is reduced to 5,685ms for a test set. The results are compared with existing method Mutant Gene Algorithm and size of test suite is minimized upto 13.6% using Hybrid Algorithm.

An improving approach to analyzing change impact of C programs

Program comprehension is a required activity for many software development and maintenance tasks. A common means of understanding software is though UML diagrams. UML diagrams model the design and implementation of an application, describing both its structure and behaviour. However with rapid software development life cycles, UML diagrams often become inconsistent with the implementation of an application. This limits their effectiveness in program comprehension activities. This paper presents a program analysis framework to reverse engineer sequence diagrams from application source code. Part of this framework is the ability to filter out irrelevant operations from a sequence diagram in order to simplify the representation of an application. This is achieved by identifying important data, and following its use in the application. The purpose of this framework is to aid in program comprehension activities by providing up to date representations of an application; while also enabling developers to identify the logical operation of a program without interference from irrelevant or supporting operations.