Fault Localization from the Semantic Code Search Perspective

Code search is an essential task in software development. Developers often search the internet and other code databases for necessary source code snippets to ease the development efforts. Code search techniques also help learn programming as novice programmers or students can quickly retrieve (hopefully good) examples already used in actual software projects. Given the recurrence of the code search activity in software development, there is an increasing interest in the research community. To improve the code search experience, the research community suggests many code search tools and techniques. These tools and techniques leverage several different ideas and claim a better code search performance. However, it is still challenging to illustrate a comprehensive view of the field considering that existing studies generally explore narrow and limited subsets of used components. This study aims to devise a grounded approach to understanding the procedure for code search and build an operational taxonomy capturing the critical facets of code search techniques. Additionally, we investigate evaluation methods, benchmarks, and datasets used in the field of code search.

In software engineering, empirical studies on automated fault localization (FL) methods mainly focus on general software, and substantial progress has been made. However, the applicability and efficacy of these methods in specialized, domain-specific software like industrial software remains under-explored. Such specialized software is usually characterized by complex inputs and iterative computing paradigms, which could significantly influence the effectiveness of existing FL methods. To address this gap, this study takes a typical categorical of industrial software (i.e., computer-aided engineering (CAE) projects) as a case study, to investigate the feasibility and effectiveness of state-of-the-art FL methods within CAE projects. Through the reproduction of 76 real-world bugs from three widely used CAE projects (i.e., FDS, deal.II, and MFEM), we find that even the most precise FL methods require developers to examine on average 467.18 statements before finding bugs and can take 208.13 hours to execute. The complex inputs and long-term computation characteristics of CAE projects further increase the difficulty of FL. Moreover, FL on CAE also faces challenges, such as insufficient differentiation of coverage information and missing CAE-specific FL features. Based on our findings, we improve FL on CAE projects by proposing a set of CAE main module-based features, which improve the best-performed FL method in this study (i.e., DeepFL) by 35.93% and 45%, in terms of MAR and MFR , respectively.

Fault localization (FL) and automated program repair (APR) are two main tasks of automatic software debugging. Compared with traditional methods, deep learning-based approaches have been demonstrated to achieve better performance in FL and APR tasks. However, the existing deep learning-based FL methods ignore the deep semantic features or only consider simple code representations. And for APR tasks, existing template-based APR methods are weak in selecting the correct fix templates for more effective program repair, which are also not able to synthesize patches via the embedded end-to-end code modification knowledge obtained by training models on large-scale bug-fix code pairs. Moreover, in most of FL and APR methods, the model designs and training phases are performed separately, leading to ineffective sharing of updated parameters and extracted knowledge during the training process. This limitation hinders the further improvement in the performance of FL and APR tasks. To solve the above problems, we propose a novel approach called MTL-TRANSFER, which leverages a multi-task learning strategy to extract deep semantic features and transferred knowledge from different perspectives. First, we construct a large-scale open-source bug datasets and implement 11 multi-task learning models for bug detection and patch generation sub-tasks on 11 commonly used bug types, as well as one multi-classifier to learn the relevant semantics for the subsequent fix template selection task. Second, an MLP-based ranking model is leveraged to fuse spectrum-based, mutation-based and semantic-based features to generate a sorted list of suspicious statements. Third, we combine the patches generated by the neural patch generation sub-task from the multi-task learning strategy with the optimized fix template selecting order gained from the multi-classifier mentioned above. Finally, the more accurate FL results, the optimized fix template selecting order, and the expanded patch candidates are combined together to further enhance the overall performance of APR tasks. Our extensive experiments on widely-used benchmark Defects4J show that MTL-TRANSFER outperforms all baselines in FL and APR tasks, proving the effectiveness of our approach. Compared with our previously proposed FL method TRANSFER-FL (which is also the state-of-the-art statement-level FL method), MTL-TRANSFER increases the faults hit by 8/11/12 on Top-1/3/5 metrics (92/159/183 in total). And on APR tasks, the number of successfully repaired bugs of MTL-TRANSFER under the perfect localization setting reaches 75, which is 8 more than our previous APR method TRANSFER-PR. Furthermore, another experiment to simulate the actual repair scenarios shows that MTL-TRANSFER can successfully repair 15 and 9 more bugs (56 in total) compared with TBar and TRANSFER, which demonstrates the effectiveness of the combination of our optimized FL and APR components.

Code search greatly improves developers’ coding efficiency by retrieving reusable code segments with natural language queries. Despite the continuous efforts in improving both the effectiveness and efficiency of code search, two issues remained unsolved. First, programming languages have inherent strong structural linkages, and feature mining of code as text form would omit the structural information contained inside it. Second, there is a potential semantic relationship between code and query, it is challenging to align code and text across sequences so that vectors are spatially consistent during similarity matching.To tackle both issues, in this paper, a code search model named CSSAM (Code Semantics and Structures Attention Matching) is proposed. By introducing semantic and structural matching mechanisms, CSSAM effectively extracts and fuses multidimensional code features. Specifically, the cross and co-attention layer was developed to facilitate high-latitude spatial alignment of code and query at the token level. By leveraging the residual interaction, a matching module is designed to preserve more code semantics and descriptive features, which enhances the relevance between the code and its corresponding query text. Besides, to improve the model’s comprehension of the code’s inherent structure, a code representation structure named CSRG (Code Semantic Representation Graph) is proposed for jointly representing abstract syntax tree nodes and the data flow of the codes. According to the experimental results on two publicly available datasets containing 475k and 330k code segments, CSSAM significantly outperforms the baselines in terms of achieving the highest SR@1/5/10, MRR, and NDCG@50 on both datasets, respectively. Moreover, the ablation study is conducted to quantitatively measure the impact of each key component of CSSAM on the efficiency and effectiveness of code search, which offers insights into the improvement of advanced code search solutions.

DC fault current is contributed by various distributed energy resources in DC distribution systems. Most DC distribution systems have low line impedance. The fault current increases fast because of the low impedance. Some converters in DC distribution systems include Fault Current Limiting (FCL) function. The controlled fault currents at different locations are very close. Thus, it is important to design a reliable and fast fault detection and location method for DC distribution systems. This paper proposes a novel local measurement-based fault location algorithm for DC distribution systems. The proposed fault location algorithm can estimate the equivalent inductance from a protective device to a fault in less than one millisecond. The performance of the developed protection algorithm was validated in both numerical simulation and hardware testing.

Optimal placement of voltage measurements for wide area fault location observability considering the uncertainty of network parameters

The use of modern methods for determining the fault location (FL) on overhead power lines (OHРLs), which have high accuracy and high speed, allows you to quickly find and repair damage. Various physical principles and algorithms for calculating the distance to the site of damage are used in FL OHРLs devices from different manufacturers. Some of algorithms for FL on OHРLs use the emergency mode parameters (EMP), other algorithms use measurement results based on wave methods. Many random factors that determine the magnitude of the error in calculating the distance to the FL affect the operation of FL devices by EMP. Methods based on deterministic procedures used in well-known FL devices for OHРLs do not take into account the influence of random factors, which significantly increases the time to search for the fault. The authors have developed a new method of FL on OHРLs using multi-hypothesis sequential analysis and the Armitage algorithm. The task of recognizing a faulted section of an OHРL is formulated as a statistical problem. To do this, the inspection area of the OHРL is divided into many sections, followed by the implementation of the procedure for FL. The developed method makes it possible to adapt the distortions of currents and voltages on the emergency mode oscillograms to the conditions for estimating their parameters. The results of the calculations proved that the implementation of the developed method has practically no effect on the speed of the FL algorithm for the OHРL by EMP. This ensures the uniqueness of determining the faulted section of the OHРL under the influence of random factors, which leads to a significant reduction in the inspection area of the OHРL. The application of the developed method in FL devices for OHРLs will ensure the required reliability of power supply to consumers and reduce losses from power outages by minimizing the time to search for a fault.

Source code search tools are designed to help developers locate code relevant to their task. The effectiveness of a search technique often depends on properties of user queries, the code being searched, and the specific task at hand. Thus, new code search techniques should ideally be evaluated in realistic situations that closely reflect the complexity, purpose of use, and context encountered during actual search sessions. This paper explores what can be learned from using an online paired interleaving approach, originally used for evaluating internet search engines, to comparatively observe and assess the effectiveness of code search tools in the field. We present a case study in which we implemented online paired interleaving for code search, deployed the tool in an IDE for developers at multiple companies, and analyzed results from over 300 user queries during their daily software maintenance tasks. We leveraged the results to direct further improvement of a search technique, redeployed the tool and analyzed results from over 600 queries to validate that an improvement in search was achieved in the field. We also report on the characteristics of user queries collected during the study, which are significantly different than queries currently used in evaluations.

Over the years of software development, a vast amount of source code has been accumulated. Many code search tools were proposed to help programmers reuse previously-written code by performing free-text queries over a large-scale codebase. Our experience shows that the accuracy of these code search tools are often unsatisfactory. One major reason is that existing tools lack of query understanding ability. In this paper, we propose CodeHow, a code search technique that can recognize potential APIs a user query refers to. Having understood the potentially relevant APIs, CodeHow expands the query with the APIs and performs code retrieval by applying the Extended Boolean model, which considers the impact of both text similarity and potential APIs on code search. We deploy the backend of CodeHow as a Microsoft Azure service and implement the front-end as a Visual Studio extension. We evaluate CodeHow on a large-scale codebase consisting of 26K C# projects downloaded from GitHub. The experimental results show that when the top 1 results are inspected, CodeHow achieves a precision score of 0.794 (i.e., 79.4% of the first returned results are relevant code snippets). The results also show that CodeHow outperforms conventional code search tools. Furthermore, we perform a controlled experiment and a survey of Microsoft developers. The results confirm the usefulness and effectiveness of CodeHow in programming practices.

Based on the traveling wave fault location principle, the traveling wave fault location method of Teed circuit is studied, and the idea of comprehensively using the three-terminal measurement data of Teed-circuit for fault location is proposed, and a new criterion for fault branch identification and a new method for fault location are proposed accordingly. The branch identification criterion considers the influence of the actual fault location error factors on the branch misjudgment to ensure the validity of the branch identification; and the proposed fault location method makes full use of the three-terminal measurement data of Teed-circuit. Meanwhile, the influence caused by the uncertainty of wave velocity on the fault location result is eliminated in the fault location expression, and the fault location error caused by the line sags is eliminated to a certain extent. The ATP/EMTP simulation results show that the method proposed in this paper can accurately identify fault branches and locate fault points on Teed-circuit.

Chapter 12 - Utility practices on fault location

Fault location information is critical for operating and maintaining transmission networks. Some of the challenges in calculating accurate fault location include fault resistance, zero-sequence mutual coupling, load, system nonhomogeneity, and transmission lines composed of multiple sections with considerably different characteristics. This paper presents a fully automated real-time fault location system that provides accurate fault location information for multiple transmission lines and makes the results available to system operators and maintenance personnel. The system performs the following tasks: • Retrieves all triggered event reports from relays and digital fault recorders (DFRs) automatically. With these reports, the system identifies the faulted transmission line(s) within the transmission network. • Determines the fault type and calculates the fault location and fault resistance using multiterminal fault location methods for different types of power lines, including overhead lines, underground cables, and composite lines that include both overhead line and underground cable sections. • Uses fault location methods that are accurate under fault resistance, parallel-line mutual coupling, system nonhomogeneity, and load. • Supports event reports with a fixed sampling rate or a variable sampling rate that is a multiple of the power system operating frequency. • Supports event reports generated by relays and DFRs connected to an IRIG-B time source. This paper compares fault location results calculated from a field event report that was obtained from an actual power system with actual fault location data.

Automated program repair (APR) tools apply fault localization (FL) techniques to identify the locations of likely faults to be repaired. The effectiveness, performance, and repair correctness of APR depends in part on the FL method used. If FL does not identify the location of a fault, the application of an APR tool will not be effective--it will fail to repair the fault. If FL assigns the actual faulty statement a low priority for repair, APR performance will be reduced by increasing the time required to find a potential repair. In addition, the correctness of a generated repair will be decreased since APR will modify fault-free statements that are assigned a higher priority for repair than an actual faulty statement. We conducted a controlled experiment to evaluate the impact of ten FL techniques on APR effectiveness, performance, and repair correctness using a brute force APR tool applied to faulty versions of the Siemens Suite and two other large programs: space and sed. All FL techniques were effective in identifying all faults; however, Wong3 and Ample1 were the least effective FL techniques since they assigned the lowest priority for repair in more than 26 % of the trials. We obtained the worst APR performance significantly when Ample1 was used since it generated a large number of variants in 29.11 % of the trials, and took the longest time to produce potential repairs. Jaccard FL improved repair correctness by generating more validated repairs---potential repairs that pass a set of regression tests, and generating potential repairs that failed fewer regression tests. Also Jaccard's performance is noteworthy in that it never generated a large number of variants during the repair process compared to the alternatives.

Fault location in electrical energy distribution networks is an important task, as faults in distribution grids are among the main causes of electricity supply disruption. Fault location in the distribution systems, however, is a challenging task because of the topology of the distribution networks, as well as the main and side branches. Therefore, it is necessary to address these challenges through an intelligent approach to fault location. In this paper, fault location in electric energy distribution networks is addressed considering the changes in fault distance and fault resistance in the presence of different fault types. A new method for fault location is developed for conditions where the minimum information is available and only information at the beginning of the feeder is used. This facilitates wide adoption of the technique as it does not require significant investments in instrumentation and measurement. The proposed intelligent method is based on the impedance and transient state estimation. This technique employs a specific impedance analysis for determining possible fault locations considering the unbalanced performance of distribution systems, distances, and different fault resistances. To determine the real faulty section, real fault frequency component analysis and the simulated faults at possible fault locations are used. At this stage of the process, it is possible to eliminate multiple estimations with the help of comparison and identification of the similarities. Therefore, a real faulty section is determined. It is observed that some conditions of electric energy distribution networks affect the accuracy and performance of the proposed method significantly; thus, a detailed investigation is conducted to neutralize these conditions. Simulation results and calculations based on MATLAB along with a practical test of the proposed method in power network simulator confirm a satisfactory performance.

Quick and accurate location of faults is the basic task of fault location as well as an important research subject in power system relay protection. The traveling wave principles of double terminal fault locating method is described, and the main causes of the generation of errors in actual application are investigated. A novel double terminal fault locating method is presented based on the existing fault locating method. The arriving intervals of the travelling wave from the fault location, reflected wave from fault location and reflected wave from opposite bus are recorded. The new fault location method is applied in Zhuzhou power grid, and the results show that it has high location accuracy.