Semantic context based coincidental correct test cases detection for fault localization

Mutation-oriented test data augmentation for GUI software fault localization

In order to improve the effectiveness of fault localization, various test suite reduction techniques have been proposed. However, excessive or improper reduction of test cases may lose some testing information, thus causing a negative impact on fault localization. In this paper, we propose a new similarity-based test suite reduction approach to improving spectrum-based fault localization. Firstly, this approach extracts the high suspicious statements in the faulty program and removes the coincidental passed test cases for test suite selection. Then, it selects similar passed test cases for each different failed test case from the new passed test set based on the similar proportion of their execution traces, and determines the final composition of each similar test set based on the contribution of failed test cases to fault localization. By using the execution information of each similar test set with a spectrum-based fault localization approach, the ranks of statements can be obtained. Finally, synthesizing all ranks of statements in each rank list, we obtain the final rank list of statements. Several experiments show that our approach can help reduce the debugging effort in terms of the percentage of statements needed to be inspected when locating faults in both single-fault and multi-fault programs. Moreover, the results demonstrate that the value of similar proportion have nontrivial influence on the effectiveness of fault localization.

Software security is crucial in every aspect of information technology. As today's software has become larger and more complex than ever before, software fault localization has become more time consuming and labor intensive accordingly. Consequently, a broad spectrum of software fault localization techniques are proposed and implemented. However, most of these techniques assume the existence of a test suite to enable their execution traces and results to be used in locating program bugs. However, this is not always true in practice. After the release of software, programmers begin debugging immediately after one failure is detected. Only the test case that results in the failure is available for programmers. In such a situation, fault localization techniques, which require multiple failed and successful test cases, cannot be applied. Hence, in this paper, we propose an innovative fault localization framework, CRAXfault, based on single-path concolic execution, which exploits the execution path of one failed test case to automatically generate numerous test cases that are then used in a fault localization technique (e.g., Tarantula, Ochiai, Crosstab, and DStar). CRAXfault is evaluated across 15 real-life programs. The results demonstrate a remarkable effectiveness of CRAXfault in the number of statements that must be examined before the first faulty statement for a given bug is identified and acceptable efficiency in the time cost for generating test cases.

Fault Localization Prioritization (FLP) aims at reordering existing test cases so that the location of detected faulty components can be identified earlier, using certain fault localization techniques. Although some researchers have proposed adaptive prioritization strategies with white-box code coverage information, such information may not always be available. In this paper, we address the FLP problem using black-box information derived from partitioning the input domain. Based on the well-known technique of Spectra-Based Fault Localization (SBFL), three test case prioritization strategies are designed following some basic SBFL heuristics. The implementation of these proposed strategies relies only on the partition information, and does not require any test case execution history. Experiments show that our strategies, when compared with pure random selection, result in a faster localization of faulty statements, reducing the number of test case executions required. Here, we analyze the characteristics and merits of the three proposed strategies.

Coincidental correctness (CC) is a situation during the execution of a test case, the buggy entity is executed, but the program behaves correctly as expected. Many automated fault localization (FL) techniques use runtime information to discover the underlying connection between the executed buggy entity and the failing test result. The existence of CC will weaken such connection, mislead the FL algorithms to build inaccurate models, and consequently, decrease the localization accuracy. To alleviate the adverse effect of CC on FL, CC detection techniques have been proposed to identify the possible CC tests via heuristic or machine learning algorithms. However, their performance on precision is not satisfactory since they overestimate the possible CC tests and are insufficient in learning the deep semantic features. In this work, we propose a novel <u>Tri</u>plet network-based <u>Co</u>incidental <u>Co</u>rrectness detection technique (<i>i.e.,</i> <b>TriCoCo</b>) to overcome the limitations of the prior works. <b>TriCoCo</b> narrows the possible CC tests by designing three features to identify genuine passing tests. Instead of using all tests as inputs by existing techniques, <b>TriCoCo</b> takes the identified genuine passing tests and failing ones to train a triplet model that can evaluate their relative distance. Finally, <b>TriCoCo</b> infers the probability of being a CC test of the test in the rest of the passing tests by using the trained triplet model. We conduct large-scale experiments to evaluate <b>TriCoCo</b> based on the widely-used Defects4J benchmark. The results demonstrate that <b>TriCoCo</b> can improve not only the precision of CC detection but also the effectiveness of FL techniques, <i>e.g.,</i> the precision of <b>TriCoCo</b> is 80.33<inline-formula><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> on average, and <b>TriCoCo</b> boosts the efficacy of DStar by 18<inline-formula><tex-math notation="LaTeX">$\%$</tex-math></inline-formula>–74<inline-formula><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> in terms of MFR metric when compared to seven state-of-the-art CC detection baselines.

Distance protection has been a challenging task due to the complexity of the power system. An effective protection scheme must perform fault detection, classification and location that help in quick restoration of the power supply. Although significant hybrid methods are available for fault detection, classification, and location, separately; there is always a need for an improved integrated approach performing fault detection, classification, and location. This chapter presents an integrated approach using autoregressive modeling and extreme learning machine (ELM). The performance of the scheme is investigated on 11 kV, 6 km medium voltage underground cable. The test system is simulated using PSCAD/EMTDC and the integrated protection scheme is implemented in MATLAB. The scheme uses half-cycle duration current samples. The data preprocessing is done using autoregressive (AR) modeling. The scheme detects the fault using the power of the critical pole. ELM binary classifier uses the AR coefficients to classify the type of fault. ELM in regression mode locates the fault distance. A total of 1260 data set is used to train the ELM fault classifier and ELM fault locator. The integrated protection scheme is tested with 10,800 test cases covering wide variation in operating conditions. The classification accuracy of the ELM fault classifier is 99.85% and the fault location error is less than 0.12% for the ELM fault locator, which is better than the other hybrid methods. Hence this integrated approach can be considered as quite suitable for protection of medium voltage underground cables.

Software testing is the key to ensuring software quality. Test case generation and software fault localization are two important research objects of testing. Spectrum-based fault localization (SBFL) method is a mainstream dynamic fault localization method, which inputs test cases into the program for execution and collects statement coverage information and execution results. By analysing this information, a fault report can be generated. The performance of SBFL is affected by test case quality. Therefore, this paper proposes a test case generation method based on improved genetic algorithm to assist SBFL. This method uses a small number of initial test cases to execute the program, then uses the XGBoost model to pre-process the spectrum information and generates a list of potential fault statements to guide the iterative process of the genetic algorithm. The experimental results show that the test cases generated using the improved genetic algorithm have certain advantages over the test cases generated by general genetic algorithm and the baseline test cases.

Software fault localization is commonly recognized as arduous and time consuming. Spectrum‐based fault localization (SBFL) has been widely used due to its lightness. However, the effectiveness of SBFL is limited since it only considers simple statistics on the coverage information, ignoring the tie problem that the spectrum matrixes of some statements are the same. Most existing deep learning‐based fault localization (DLFL) techniques convert the coverage information into a vector, which utilizes the spectrum in a simplified manner and still has limitations in practice. To solve the above problem, we propose an approach via the weighted execution graph and graph attention network (WEGAT). We use a graph structure to represent the coverage information between test cases and program elements. Then, we generate a weighted execution graph by applying the predicate execution sequence. Furthermore, we combine the weighted execution graph with the AST as an integrated graph, which is the input of the GAT for fault localization. We evaluate WEGAT in within‐project and cross‐project prediction scenarios on the Defects4J benchmark. Experimental results show that our approach outperforms traditional SBFL (Ochiai, DStar and Tarantula) and DLFL (TraPT, CNN‐FL, Grace, and AGFL) methods, effectively improving the accuracy of fault localization.

Spectrum-based fault localization aims to find the program statements that contain faults, according to the dynamic information of executing program. We propose a spectra-based fault localization method with test case reduction, which removes redundant test cases in fault localization. Experiments show that new method requires fewer test cases to obtain the better fault localization results.

Among the fault localization methods of the coverage-based regression test cases, the ranking of suspiciousness degree is seriously impacted due to the existence of similar test cases, the existence of isolated treatment of each program entity and the coincidental correctness test cases as well as the poor quality of regression test cases, which reduces the efficiency of fault localization. This paper proposes a method that combines the control dependence and data dependence, test cases covering table and test result with vector angular cosine weight to solve the aforesaid problems. Experimental results show that the efficiency of the proposed fault localization of regression test cases is better than that of Tarantula, Jaccard, Ochiai, PPDG and CP methods.

This article proposes a noniterative parameter-free fault location (PFFL) method for practically transposed lines. First, the proposed method accurately models the nonhomogeneity and asymmetry of practically transposed transmission lines by incorporating three untransposed line segments according to the practical transposition principle. To reduce the modeling complexity and to enable noniterative closed-form solutions for fault location, each untransposed line segment is modeled using lumped impedances in matrix form, and the shunt admittances are introduced at both line terminals to enhance the modeling accuracy. Finally, the analytical solution of fault location is derived, avoiding the divergence risks from iterative algorithms. The line or system parameters are not required throughout the entire fault location procedure, and it is compatible with both synchronized and unsynchronized phasor measurements at line terminals. Numerical experiments confirm the necessity of the proposed method considering nonhomogeneity and asymmetry of practically transposed lines: the proposed method exhibits higher fault location accuracy compared to the existing parameter-free method that assumes the ideal transposition. Moreover, the robustness of the proposed method is validated by further testing cases.

Designing test cases for remote asynchronous testing is error-prone. This is due to the difficulty to foresee all the disorders on the observations collected by the tester as well as the possible collisions between stimuli and observations. Designing correct synchronous test cases is easier, but transforming,them into correct asynchronous ones is a difficult task. Moreover, it is difficult to compare remote testing and local testing as in general sets of conformant implementations are not comparable.

Debugging is a time-consuming task in software development and maintenance. To accelerate this task, several approaches have been proposed to automate fault localization. In particular, testing based fault localization (TBFL), which utilizes the testing information to localize the faults, seem to be very promising. However, the similarity between test cases in the test suite has been ignored in the research on TBFL. In this paper, we investigate this similarity issue and propose a novel approach named similarity-aware fault localization (SAFL), which can calculate the suspicion probability of each statement with little impact by the similarity issue. To address and deal with the similarity between test cases, SAFL applies the theory of fuzzy sets to remove the uneven distribution of the test cases. We also performed an experimental study for two real-world programs at different size levels to evaluate SAFL together with another two approaches to TBFL. Experimental results show that SAFL is more effective than the other two approaches when the test suites contain injected redundancy, and SAFL can achieve a competitive result with normal test suites. SAFL can also be more effective than applying test suite reduction to current approaches to TBFL.

ObjectivesThe concurrent implementation of targeted (tPITC) and blanket provider-initiated testing and counselling (bPITC) is recommended by the World Health Organization (WHO) for HIV case-finding in generalized HIV epidemics. This study assessed the effectiveness of this intervention compared to symptom-based diagnostic HIV testing (DHT) in terms of HIV testing uptake, case detection and antiretroviral therapy (ART) enrollment among children and adolescents in Cameroon, where estimated HIV prevalence is relatively low at 3.7%.MethodsIn three hospitals where DHT was the standard practice before, tPITC and bPITC were implemented by inviting HIV-positive parents in care at the ART clinics to have their biological children (6 weeks-19 years) tested for HIV (tPITC). Concurrently, at the outpatient departments, similarly-age children/adolescents were systematically offered HIV testing via accompanying parents/guardians. The mean monthly number of children tested for HIV, identified HIV-positive and ART-enrolled were used to compare the outcomes of different HIV testing strategies before and after the intervention.ResultsIn comparing DHT to bPITC, there was a significant increase in the mean monthly number of children/adolescents tested for HIV (223.0 vs 348.3, p = 0.0073), but with no significant increase in the mean monthly number of children/adolescents: testing HIV-positive (10.5 vs 9.7, p = 0.7574) and ART- enrolled (7.3 vs 6.3, p = 0.5819). In comparing DHT to tPITC, there was no significant difference in the mean monthly number of children/adolescents: tested for HIV (223 vs 193.8, p = 0.4648); tested HIV-positive (10.5 vs 10.6, p = 0.9544), and ART-enrolled (7.3 vs 5.8, p = 0.4672). When comparing DHT versus bPITC+tPITC, there was a significant increase in the mean monthly number of children/adolescents: tested for HIV (223.0 to 542.2, p<0.0001), testing HIV-positive (10.5 vs 20.3, p = 0.0256), and ART-enrolled (7.3 vs 12.2, p = 0.0388).ConclusionsThese findings suggest that concurrent implementation of bPITC+tPITC was more effective compared to DHT in terms of HIV testing uptake, case detection and ART enrolment. However, considering that DHT and bPITC had comparable outcomes with regards to case detection and ART enrolment, bPITC+tPITC may not be efficient. Thus, this finding does not support concurrent bPITC+tPITC implementation as recommended by WHO. Rather, continued DHT+tPITC could effectively and efficiently accelerate HIV case detection and ART coverage among children and adolescents in Cameroon and similar low-prevalence context.

Test flakiness (non-deterministic behavior of test cases) is an increasingly serious concern in industrial practice. However, there are relatively little research results available that systematically address the analysis and mitigation of this phenomena. The dominant approach to handle flaky tests is still detecting and removing them from automated test executions. However, some reports showed that the amount of flaky test is in many cases so high that we should rather start working on approaches that operate in the presence of flaky tests. In this work, we investigate how test flakiness affects the effectiveness of Spectrum Based Fault Localization (SBFL), a popular class of software Fault Localization (FL), which heavily relies on test case execution outcomes. We performed a simulation based experiment to find out what is the relationship between the level of test flakiness and fault localization effectiveness. Our results could help the users of automated FL methods to understand the implications of flaky tests in this area and to design novel FL algorithms that take into account test flakiness.