Test suite optimization under multi-objective constraints for software fault detection and localization: Hybrid optimization based model

Software testing is a critical part of software development. As new test cases are generated over time due to software modifications, test suite sizes may grow significantly. Because of time and resource constraints for testing, test suite minimization techniques are needed to remove those test cases from a suite that, due to code modifications over time, have become redundant with respect to the coverage of testing requirements for which they were generated. Prior work has shown that test suite minimization with respect to a given testing criterion can significantly diminish the fault detection effectiveness (FDE) of suites. We present a new approach for test suite reduction that attempts to use additional coverage information of test cases to selectively keep some additional test cases in the reduced suites that are redundant with respect to the testing criteria used for suite minimization, with the goal of improving the FDE retention of the reduced suites. We implemented our approach by modifying an existing heuristic for test suite minimization. Our experiments show that our approach can significantly improve the FDE of reduced test suites without severely affecting the extent of suite size reduction

Software Testing is an important aspect of the real time software development process. Software testing always assures the quality of software product. As associated with software testing, there are few very important issues where there is a need to pay attention on it in the process of software development test. These issues are generation of effective test case and test suite as well as optimization of test case and suite while doing testing of software product. The important issue is that testing time of the test case and test suite. It is very much important that after development of software product effective testing should be performed. So to overcome these issues of optimization, we have proposed new approach for test suite optimization using genetic algorithm (GA). Genetic algorithm is evolutionary in nature so it is often used for optimization of problem by researcher. In this paper, our aim is to study various selections methods like tournament selection, rank selection and roulette wheel selection and then we apply this genetic algorithm (GA) on various programs which will generate optimized test suite with parameters like fitness value of test case, test suite and take minimum amount of time for execution after certain preset generation. In this paper our main objectives as per the experimental investigation, we show that tournament selection works very fine as compared to other methods with respect fitness selection of test case and test suites, testing time of test case and test suites as well as number of requirements.

There occur frequent requirement changes in software systems even after the software has been developed. Regression testing is continuously performed to identify the undesired affects of these requirement changes on already tested system. Test suites grow enormously with these changes due to addition of new test cases for enhanced functionality. Optimization of test suite to perform regression testing within the budgetary and time restrictions is ultimate choice for a tester because "Retest all" test suite is un-economical and is not suitable choice. Test suite optimization can be either static or on the fly. With on the fly optimization, optimal suite keeps on changing with the requirement changes. On the fly optimization of test suite is preferable option for regression testing. Presently, static test suite optimization approaches exist. We have proposed an application specific, on the fly optimization approach for test suite optimization problem. We have implemented our approach on an academic testing problem. We use fuzzy logic to optimize the test suite with multiple optimization objectives. Our approach has been successful to generate on the fly optimized test suites for changing requirements. In future, we will implement this approach on considerably large sized testing problems.

Testing is an important activity in engineering of industrial embedded software. In certain application domains (e.g., railway industry) engineering software is certified according to safety standards that require extensive software testing procedures to be applied for the development of reliable systems. Mutation analysis is a technique for creating faulty versions of a software for the purpose of examining the fault detection ability of a test suite. Mutation analysis has been used for evaluating existing test suites, but also for generating test suites that detect injected faults (i.e., mutation testing). To support developers in software testing, we propose a technique for producing test cases using an automated test generation approach that operates using mutation testing for software written in IEC 61131-3 language, a programming standard for safety-critical embedded software, commonly used for Programmable Logic Controllers (PLCs). This approach uses the Uppaal model checker and is based on a combined model that contains all the mutants and the original program. We applied this approach in a tool for testing industrial PLC programs and evaluated it in terms of cost and fault detection. For realistic validation we collected industrial experimental evidence on how mutation testing compares with manual testing as well as automated decision-coverage adequate test generation. In the evaluation, we used manually seeded faults provided by four industrial engineers. The results show that even if mutation-based test generation achieves better fault detection than automated decision coverage-based test generation, these mutation-adequate test suites are not better at detecting faults than manual test suites. However, the mutation-based test suites are significantly less costly to create, in terms of testing time, than manually created test suites. Our results suggest that the fault detection scores could be improved by considering some new and improved mutation operators (e.g., Feedback Loop Insertion Operator (FIO)) for PLC programs as well as higher-order mutations.

Fault detection, classification, and location estimation are paramount in transmission lines to ensure uninterrupted power supply, enhance reliability, and safety. Faster and more accurate fault localization is critical for prompt repair and efficient power restoration, thus minimizing economic losses. This paper presents a novel Fault Detection Isolation and Localization (FDIL) scheme, a machine learning (ML)-based framework, to increase the accuracy of fault location (FL) prediction. The proposed framework utilizes the phasor voltage, current, and frequency data obtained from optimally placed Phasor Measurement Units (PMUs) for fault detection, isolation, and localization. This paper also proposes a hybrid deep learning-based approach using Bidirectional Long Short-Term Memory (Bi-LSTM) networks and an attention mechanism for FL prediction. Extensive simulations using the IEEE 9-bus system, with different types and locations of faults, are performed on the OPAL-RT simulator-based testbed to generate and train the ML models. The performance of the proposed scheme is evaluated by calculating the Mean Absolute Error (MAE), Mean Squared Error (MSE), and R-squared (<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</i><sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup>) metric. The results demonstrate that the proposed approach achieves high accuracy in FL prediction, with an MAE of 0.0228 km, an MSE of 0.001, and an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</i><sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> of 0.9894. Compared to the best-performing technique reported in the literature (CNN-LSTM, MAE = 0.211km), the proposed approach achieves an 89.2% reduction in localization error. The proposed scheme and model outperform the contemporary model, as reported in various literature, making it an effective solution for real-time fault monitoring in transmission systems.

Redundant test cases in newly generated test suites often remain undetected until execution and waste scarce project resources. In model-based testing, the testing process starts early on in the developmental phases and enables early fault detection. The redundancy in the test suites generated from models can be detected earlier as well and removed prior to its execution. The article presents a novel model-based test suite optimization technique involving UML activity diagrams by formulating the test suite optimization problem as an Equality Knapsack Problem. The aim here is the development of a test suite optimization framework that could optimize the model-based test suites by removing the redundant test cases. An evolution-based algorithm is incorporated into the framework and is compared with the performances of two other algorithms. An empirical study is conducted with four synthetic and industrial scale Activity Diagram models and results are presented.

Mutation-oriented test data augmentation for GUI software fault localization

Test minimization techniques aim at identifying and eliminating redundant test cases from test suites in order to reduce the total number of test cases to execute, thereby improving the efficiency of testing. In the context of software product line, we can save effort and cost in the selection and minimization of test cases for testing a specific product by modeling the product line. However, minimizing the test suite for a product requires addressing two potential issues: 1) the minimized test suite may not cover all test requirements compared with the original suite; 2) the minimized test suite may have less fault revealing capability than the original suite. In this paper, we apply weight-based Genetic Algorithms (GAs) to minimize the test suite for testing a product, while preserving fault detection capability and testing coverage of the original test suite. The challenge behind is to define an appropriate fitness function, which is able to preserve the coverage of complex testing criteria (e.g., Combinatorial Interaction Testing criterion). Based on the defined fitness function, we have empirically evaluated three different weight-based GAs on an industrial case study provided by Cisco Systems, Inc. Norway. We also presented our results of applying the three weight-based GAs on five existing case studies from the literature. Based on these case studies, we conclude that among the three weight-based GAs, Random-Weighted GA (RWGA) achieved significantly better performance than the other ones.

There exists a direct need to automate the process of test suite generation to get the most optimal results as testing accounts for more than 40% of total cost. One method to solve this problem is the use of meta-heuristic algorithms which iteratively improve the test data to reach the most optimized test suites. This study focuses on the performance evaluation of six meta-heuristic algorithms namely: hill-climbing algorithm (HCA), particle swarm optimization (PSO), firefly algorithm (FA), cuckoo search algorithm (CS), bat algorithm (BA) and artificial bee colony algorithm (ABC) by using their standard implementation to optimize the path coverage and branch coverage produced by the test data. The goal of the study was to find the best-suited algorithm to narrow down the future research in the field of test automation for path coverage-based optimization approaches. Each algorithm was first implemented to automatically generate test suites based on the program under test. This was followed by the performance evaluation of each algorithm for five programs written in Java. The algorithms were compared using process metrics: average time, best time, worst time and product metrics: path coverage & objective function values of the generated test suites. Results indicated ABC as the best-suited algorithm as it gave the most optimal test suites in reasonable time. BA was found to be the fastest but produced less optimal results. FA was found to be the slowest algorithm, while CS, PSO and HCA performed in between. These results show the relative performance of the six algorithms for this scenario and may be used by the future researchers to narrow down and improve the best performing algorithms for path coverage-based optimization approaches.

Boolean expressions occur frequently in descriptions of computer systems, but they tend to be complex and error-prone in complex systems. The modified condition decision coverage (MCDC) criterion in system testing is an important testing technique for Boolean expression, as its usage mandated by safety standards such as DO-178 [1] (avionics) and ISO26262 [2] (automotive). In this paper, we develop an algorithm to generate optimal MCDC test suites for Boolean expressions. Our algorithm is based on SAT solving and generates minimal MCDC test suites. Experiments on a real-world avionics system confirm that the technique can construct minimal MCDC test suites within reasonable times, and improves significantly upon prior techniques.

• Introduces a novel method using Hilbert-Huang Transform (HHT) with LSTM networks for fault detection in DC microgrids. • Targets marine and satellite DC microgrids, addressing challenges with no zero-crossing points in traditional methods. • Demonstrates superior performance, outperforming SVM and Decision Tree techniques in fault detection time and accuracy. • Validates the method using MATLAB/Simulink, effectively detecting faults with resistance up to 1.3 ohms. • Ensures compliance with IEEE 8570 standards, enhancing reliability and applicability in real-world microgrid systems. Fault detection in DC microgrids is a major challenge because traditional protection methods like wavelet and Fourier transforms rely on zero-crossing points, which the DC signal lacks. This paper introduces a new approach combining the Hilbert-Huang Transform (HHT) and a long short-term memory (LSTM) neural network to improve fault detection and classification in DC microgrids. The proposed method first uses Empirical Mode Decomposition (EMD) to break down the fault current signal into multiple intrinsic mode function (IMFs). These IMFs capture important fault-related information at different frequency levels. Next, the Hilbert Transform is applied to these IMFs to obtain amplitude and frequency features over time. These features are used to create a set of numerical descriptors that summarize the fault characteristics. These extracted features are then fed into a two-layer LSTM neural network that can learn patterns in time-series data. The LSTM model is trained to classify the input into three categories: no fault, Pole-Ground (P-G) Fault, or Pole-Pole (P-P) Fault. Simulation tests were carried out using a MATLAB Simulink model of a 20kWatt DC microgrid under different fault conditions, noise levels, and load transients. Results show that the proposed HHT-LSTM model detects faults faster and more accurately than conventional techniques like Support Vector Machines (SVM) and Decision Trees (DT). It also meets the fault detection time requirements set by IEEE8570 and aligns with IEC standards. The model remains robust even under noisy conditions, making it reliable for real-time fault protection in marine, satellite, and an effective solution for enhancing fault detection in modern DC microgrids.

A fault detection and location algorithm on three-terminal transmission line is proposed. Vast literature on fault location shows that to simplify the analysis, line charging currents are often ignored. To improve the accuracy of fault location, synchronized current and voltage measurements from three ends of the three-terminal line are utilized to detect and locate distance to fault while considering the line charging currents. The method first detects the zone at which the fault has occurred and then locates it from the respective line ends. Since three-terminal lines provide solutions for connecting existing system to the grid, facilitating bulk transfer of power, thus the fault detection and location on the same is important to maintain reliability and security of the system. Performance of the proposed algorithm is tested for various fault types, fault locations, fault resistances and fault inception angles. The 200-kV system is simulated in ATP/EMTP; however, the algorithm is developed in MATLAB. The extensive simulation studies demonstrate accuracy of the proposed scheme.

Power systems are becoming increasingly complex due to the integration of several power electronic devices. Protection of such systems and augmentation of reliability and stability depend on limiting the fault currents. Analysis of faults is essential for the selection of appropriate relays and other safety systems. In this research work fault analysis, fault detection, and data prediction of long transmission lines are discussed. Different parameters are analyzed at each point of distance using MATLAB/Simulink for long transmission lines. Through these results, it is concluded that with the increase in distance, the parameters change, so the detection response of the Mho relay also varies. Moreover, in this research work, the Mho relay response at different points is tested through Simulink software for long transmission lines. As it is concluded that manually it is very difficult to test the response of Mho relay at each point, the artificial intelligence technique of long short-term memory (LSTM) is used for the prediction of the data for the next long transmission line. Lastly, in this work, a long short-term memory network is used to predict the parameters. The LSTM is then used to find the predicted data of subsequent transmission line from previous data. The artificial intelligence technique successfully utilized for the detection of long transmission line fault.

Spectral fault localization is an automatic fault-localization technique to expedite debugging, which uses risk evaluation formula to rank the risk of fault existence in each program entity after collecting testing information. To assess the potential usefulness of a test suite and improves the accuracy for spectral fault localization, methods of assessing and optimizing test suite are proposed in this paper, which. Firstly, Average Ranking Cost of the test suite quality and two kinds of constrains are defined; and test suite quality assessment method based on these definitions is given. Secondly, a new test suite optimization method based on greedy algorithm is proposed. Finally, two widely used program databases (SIR and Defects4j) and 8 SFL techniques are applied to verify the effectiveness of our method; and the fault localization cost before and after optimizing test suites of test objects are analyzed using effect size. The largest effect size reaches 0.5398 and Each SFL technology has different degrees of improvement in the rankings of faulty statements in different programs by optimizing test suite.

ABSTRACTFaults in power transmission systems pose significant challenges due to the complexity and length of transmission lines. Effective fault detection, classification and location are essential for preventing further damage to the power grid. While travelling wave‐based algorithms are commonly used for fault location, they often focus on identifying the fault's location without classifying the fault type. Accurate classification is crucial for enabling efficient and timely responses from protection systems. This paper introduces an integrated model for fault detection, classification and location using voltage signals from a single terminal of a series‐compensated transmission line with a static synchronous series compensator (SSSC). The Gabor Transform (GT) is utilised for feature extraction, enabling both fault detection and classification. Travelling wave theory is then applied to identify the faulty segment and estimate the fault location. Additionally, a novel technique adaptively calculates the threshold value during the protection algorithm's execution. The proposed method is validated through a comprehensive analysis of various fault scenarios and sensitivity analysis. Numerical simulations in MATLAB/Simulink show that the model achieves 100% accuracy for fault detection, classification, and faulty segment identification, with 99.7925% accuracy for fault location estimation, demonstrating its effectiveness in fault management.