Probabilistic reasoning in diagnosing causes of program failures

Mutation-oriented test data augmentation for GUI software fault localization

The availability of effective test suites is critical for the development and maintenance of reliable software systems. To increase test effectiveness, software developers tend to employ larger and larger test suites. The recent availability of software tools for automatic test generation makes building large test suites affordable, therefore contributing to accelerating this trend. However, large test suites, though more effective, are resources and time consuming and therefore cannot be executed frequently. Reducing them without decreasing code coverage is a needed compromise between efficiency and effectiveness of the test, hence enabling a more regular check of the software under development. We propose a novel approach, namely REDUNET, to reduce a test suite while keeping the same code coverage. We integrate this approach in a complete framework for the automatic generation of efficient and effective test suites, which includes test suite generation, code coverage analysis, and test suite reduction. Our approach formulates the test suite reduction as a set cover problem and applies integer linear programming and a network-based optimisation, which takes advantage of the properties of the control flow graph. We find the optimal set of test cases that keeps the same code coverage in fractions of seconds on real software projects and test suites generated automatically by Randoop. The results on ten real software systems show that the proposed approach finds the optimal minimisation and achieves up to 90% reduction and more than 50% reduction on all systems under analysis. On the largest project our reduction algorithm performs more than three times faster than both integer linear programming alone and the state-of-the-art heuristic Harrold Gupta Soffa.

Measurement and detection of redundancy in test suites attempt to achieve test minimization which in turn can help reduce test maintenance costs, and to also ensure the integrity of test cases. Test suite reduction based on coverage information has been discussed in many previous works. However, the applications of such techniques on real test suites and realistic measurements of redundancy have not yet been experimented thoroughly. To address such a need, we formulate in this paper two experimental metrics for coverage-based measurement of test redundancy in the context of JUnit test suites. We then evaluate the approach by measuring the redundancy of four real Java projects. The automated measures are compared with manual redundancy decisions (performed through an inspection by a tester). The results and lessons learned are interesting and somewhat surprising in that besides they show usefulness of coverage information, they present a set of shortcomings (in terms of precision) for the simplistic coverage-based redundancy measurement approach as discussed in the literature. The root-cause analysis of our observations identify several key lessons learned which should help the testing researchers and practitioners in devising better techniques for more precise measurement of test redundancy.

In the software multi-fault scenario, the failed test cases of different fault sources are classified by clustering, and then the program statements in each cluster are ranked with suspicion by the fault location technology based on program spectrum. Software multi-fault location technology decouples failed test cases by clustering their code coverage information. At present, there are many clustering strategies for software failed test cases. However, due to the characteristics of high dimension, complex structure and variable shape of the execution information of failed test cases, it is difficult for a single clustering algorithm to accurately identify the cluster structure, so that it cannot be decoupled accurately. In this paper, a variety of clustering algorithms and different parameters are selected to obtain multiple cluster members, according to the clustering members to get the similarity matrix, and then hierarchical clustering is used to obtain the final clustering results, to achieve the integration between different clustering algorithms, so that the algorithm can identify the complex cluster structure, and then improve the efficiency of fault location.

Autonomous vehicles are expected to predict the near future trajectories of the pedestrian who might wait or continue crossing the road at the un-signalized intersection. This study presents a new framework for pedestrian trajectory predictions, which integrates Dynamic Bayesian Networks (DBN) and sequence to sequence (Seq2Seq) learning through an adaptive weighting strategy. DBN anticipates the pedestrian’s future trajectories using a probabilistic graphical model that combines environmental clues and kinematic information. Seq2Seq predicts the future trajectories with a well-trained learning model based on the observed trajectories up to the current instant. The adaptive weighting strategy can allocate weights online for DBN and Seq2Seq using the stopping probability and prediction errors. A real-world pedestrian dataset is employed for evaluations and results show that the proposed model outperforms DBN and Seq2Seq. The mean errors and final destination errors of the proposed model when predicting one second ahead are 0.04m, 0.10m in crossing scenarios, and 0.06m, 0.17m in stopping scenarios. This study expects to provide active pedestrian protections for autonomous vehicles on urban roads.

Most of the approaches for dealing with uncertainty in the Semantic Web rely on the principle that this uncertainty is already asserted. In this paper, we propose a new approach to learn and reason about uncertainty in the Semantic Web. Using instance data, we learn the uncertainty of an OWL ontology, and use that information to perform probabilistic reasoning on it. For this purpose, we use Markov logic, a new representation formalism that combines logic with probabilistic graphical models.

Program slices are useful in debugging, testing, maintenance, and understanding of programs. The conventional notion of a program slice, the static slice , is the set of all statements that might affect the value of a given variable occurrence. In this paper, we investigate the concept of the dynamic slice consisting of all statements that actually affect the value of a variable occurrence for a given program input. The sensitivity of dynamic slicing to particular program inputs makes it more useful in program debugging and testing than static slicing. Several approaches for computing dynamic slices are examined. The notion of a Dynamic Dependence Graph and its use in computing dynamic slices is discussed. The Dynamic Dependence Graph may be unbounded in length; therefore, we introduce the economical concept of a Reduced Dynamic Dependence Graph, which is proportional in size to the number of dynamic slices arising during the program execution.

In order to improve the effectiveness of fault localization, various test suite reduction techniques have been proposed. However, excessive or improper reduction of test cases may lose some testing information, thus causing a negative impact on fault localization. In this paper, we propose a new similarity-based test suite reduction approach to improving spectrum-based fault localization. Firstly, this approach extracts the high suspicious statements in the faulty program and removes the coincidental passed test cases for test suite selection. Then, it selects similar passed test cases for each different failed test case from the new passed test set based on the similar proportion of their execution traces, and determines the final composition of each similar test set based on the contribution of failed test cases to fault localization. By using the execution information of each similar test set with a spectrum-based fault localization approach, the ranks of statements can be obtained. Finally, synthesizing all ranks of statements in each rank list, we obtain the final rank list of statements. Several experiments show that our approach can help reduce the debugging effort in terms of the percentage of statements needed to be inspected when locating faults in both single-fault and multi-fault programs. Moreover, the results demonstrate that the value of similar proportion have nontrivial influence on the effectiveness of fault localization.

Automated grading allows for the scalable assessment of large programming courses, often using test cases to determine the correctness of students' programs. However, test suites can vary in multiple ways, such as quality, size, and coverage. In this paper, we investigate how much test suites with varying properties can impact generated grades, and how these properties cause this impact. We conduct a study on artificial faulty programs that simulate students' programming mistakes and test suites generated from manually written tests. We find that these test suites generate greatly varying grades, with the standard deviation of grades for each fault typically representing ~84% of the grades not apportioned to the fault. We show that different properties of test suites can influence the grades that they produce, with coverage typically making the greatest effect, and mutation score and the potentially redundant repeated coverage of lines also having a significant impact. We offer suggestions based on our findings to assist tutors with building grading test suites that assess students' code in a fair and consistent manner. These suggestions include ensuring that test suites have 100% coverage, avoiding unnecessarily recovering lines, and checking test suites using real or artificial faults.

Software developers use testing to gain and maintain confidence in the correctness of a software system. Automated reduction and prioritization techniques attempt to decrease the time required to detect faults during test suite execution. This paper uses the Harrold Gupta Soffa, delayed greedy, traditional greedy, and 2-optimal greedy algorithms for both test suite reduction and prioritization. Even though reducing and reordering a test suite is primarily done to ensure that testing is cost-effective, these algorithms are normally configured to make greedy choices with coverage information alone. This paper extends these algorithms to greedily reduce and prioritize the tests by using both test cost (e.g., execution time) and the ratio of code coverage to test cost. An empirical study with eight real world case study applications shows that the ratio greedy choice metric aids a test suite reduction method in identifying a smaller and faster test suite. The results also suggest that incorporating test cost during prioritization allows for an average increase of 17% and a maximum improvement of 141% for a time sensitive evaluation metric called coverage effectiveness.

Probabilistic graphical models, such as partially observable Markov decision processes (POMDPs), have been used in stochastic spoken dialog systems to handle the inherent uncertainty in speech recognition and language understanding. Such dialog systems suffer from the fact that only a relatively small number of domain variables are allowed in the model, so as to ensure the generation of good-quality dialog policies. At the same time, the non-language perception modalities on robots, such as vision-based facial expression recognition and Lidar-based distance detection, can hardly be integrated into this process. In this paper, we use a probabilistic commonsense reasoner to “guide” our POMDP-based dialog manager, and present a principled, multimodal dialog management (MDM) framework that allows the robot's dialog belief state to be seamlessly updated by both observations of human spoken language, and exogenous events such as the change of human facial expressions. The MDM approach has been implemented and evaluated both in simulation and on a real mobile robot using guidance tasks.

Test case prioritization is used in regression testing to schedule the execution order of test cases so as to expose faults earlier in testing. Over the past few years, many test case prioritization techniques have been proposed in the literature. Most of these techniques require data on dynamic execution in the form of code coverage information for test cases. However, the collection of dynamic code coverage information on test cases has several associated drawbacks including cost increases and reduction in prioritization precision. In this paper, we propose an approach to prioritizing test cases in the absence of coverage information that operates on Java programs tested under the JUnit framework-an increasingly popular class of systems. Our approach, JUnit test case Prioritization Techniques operating in the Absence of coverage information (JUPTA), analyzes the static call graphs of JUnit test cases and the program under test to estimate the ability of each test case to achieve code coverage, and then schedules the order of these test cases based on those estimates. To evaluate the effectiveness of JUPTA, we conducted an empirical study on 19 versions of four Java programs ranging from 2K-80K lines of code, and compared several variants of JUPTA with three control techniques, and several other existing dynamic coverage-based test case prioritization techniques, assessing the abilities of the techniques to increase the rate of fault detection of test suites. Our results show that the test suites constructed by JUPTA are more effective than those in random and untreated test orders in terms of fault-detection effectiveness. Although the test suites constructed by dynamic coverage-based techniques retain fault-detection effectiveness advantages, the fault-detection effectiveness of the test suites constructed by JUPTA is close to that of the test suites constructed by those techniques, and the fault-detection effectiveness of the test suites constructed by some of JUPTA's variants is better than that of the test suites constructed by several of those techniques.

DyHDGE: Dynamic heterogeneous transaction graph embedding for safety-centric fraud detection in financial scenarios

A probabilistic expert system for predicting the risk of Legionella in evaporative installations

This paper presents an experimental study on the emulation of software faults by fault injection. In a first experiment, a set of real software faults has been compared with faults injected by a SWIFI tool (Xception) to evaluate the accuracy of the injected faults. Results revealed the limitations of Xception (and other SWIFI tools) in the emulation of different classes of software faults (about 44% of the software faults cannot be emulated). The use of field data about real faults was discussed and software metrics were suggested as an alternative to guide the injection process when field data is nor available. In a second experiment, a set of rules for the injection of errors meant to emulate classes of software faults was evaluated. The fault triggers used seem to be the cause for the observed strong impact of the faults in the target system and in the program results. The results also show the influence in the fault emulation of aspects such as code size, complexity of data structures, and recursive versus sequential execution.