Duplication Detection for Software Bug Reports based on Topic Model

Does Deep Learning improve the performance of duplicate bug report detection? An empirical study

We propose an approach called Bug report assignment with topic modeling and heterogeneous network analysis (BAHA) to automatically assign bug reports to developers. Existing studies adopt social network analysis to characterize the collaboration of developers. The networks used in these studies are all homogenous. In real practice of bug resolution, different developers collaborate on different bug reports that makes the homogenous network unable to capture this information. We use heterogeneous network to describe the relations between reporters, bug reports and developers to characterize developers' collaboration. Experiments on Eclipse JDT project show that BAHA outperforms the state of art methods on automatic bug report assignment.

Bug reporting is essentially an uncoordinated process. The same bugs could be repeatedly reported because users or testers are unaware of previously reported bugs. As a result, extra time could be spent on bug triaging and fixing. In order to reduce redundant effort, it is important to provide bug reporters with the ability to search for previously reported bugs. The search functions provided by the existing bug tracking systems are using relatively simple ranking functions, which often produce unsatisfactory results. In this paper, we adopt Ranking SVM, a Learning to Rank technique to construct a ranking model for effective bug report search. We also propose to use the knowledge of Wikipedia to discover the semantic relations among words and documents. Given a user query, the constructed ranking model can search for relevant bug reports in a bug tracking system. Unlike related works on duplicate bug report detection, our approach retrieves existing bug reports based on short user queries, before the complete bug report is submitted. We perform evaluations on more than 16,340 Eclipse and Mozilla bug reports. The evaluation results show that the proposed approach can achieve better search results than the existing search functions provided by Bugzilla and Lucene. We believe our work can help users and testers locate potential relevant bug reports more precisely.

Bug deduplication is a hot topic in software engineering information retrieval research, but it is often not deployed. Typically to de-duplicate bug reports developers rely upon the search capabilities of the bug report software they employ, such as Bugzilla, Jira, or Github Issues. These search capabilities range from simple SQL string search to IR-based word indexing methods employed by search engines. Yet too often these searches do very little to stop the creation of duplicate bug reports. Some bug trackers have more than 10\% of their bug reports marked as duplicate. Perhaps these bug tracker search engines are not enough? In this paper we propose a method of attempting to prevent duplicate bug reports before they start: continuous querying. That is as the bug reporter types in their bug report their text is used to query the bug database to find duplicate or related bug reports. This continuous querying allows the reporter to be alerted to duplicate bug reports as they report the bug, rather than formulating queries to find the duplicate bug report. Thus this work ushers in a new way of evaluating bug report deduplication techniques, as well as a new kind of bug deduplication task. We show that simple IR measures show some promise for addressing this problem but also that further research is needed to refine this novel process that is integrate-able into modern bug report systems.

As text data continues to grow quickly, it is increasingly important to develop intelligent systems to help people manage and make use of vast amounts of text data (big text data''). As a new family of effective general approaches to text data retrieval and analysis, probabilistic topic models---notably Probabilistic Latent Semantic Analysis (PLSA), Latent Dirichlet Allocations (LDA), and their many extensions---have been studied actively in the past decade with widespread applications. These topic models are powerful tools for extracting and analyzing latent topics contained in text data; they also provide a general and robust latent semantic representation of text data, thus improving many applications in information retrieval and text mining. Since they are general and robust, they can be applied to text data in any natural language and about any topics. This tutorial systematically reviews the major research progress in probabilistic topic models and discuss their applications in text retrieval and text mining. The tutorial provides (1) an in-depth explanation of the basic concepts, underlying principles, and the two basic topic models (i.e., PLSA and LDA) that have widespread applications, (2) an introduction to EM algorithms and Bayesian inference algorithms for topic models, (3) a hands-on exercise to allow the tutorial attendants to learn how to use the topic models implemented in the MeTA Open Source Toolkit and experiment with provided data sets, (4) a broad overview of all the major representative topic models that extend PLSA or LDA, and (5) a discussion of major challenges and future research directions.

Duplicate bug report detection is one of the major problems in software triage systems like Bugzilla to deal with end user requests. User request contains some categorical and especially textual fields which need feature extraction for duplicate detection. Contextual and topical features are acquired using calculating cosine similarity between term frequency or inverse document frequency or BM25F technique from a pair of bug reports against some topics. This research proposes the individual Manhattan distance similarity approach instead of cosine distance similarity for every topic in contextual features to expand the feature dimension which can increase the accuracy of the duplicate bug report detection process. The four famous datasets of bug reports have used for evaluation of the proposed method including Android, Eclipse, Mozilla, and Open Office which the experimental results indicate performance improvement for four contextual features including general, cryptography, network, and Java topics.

For a large and complex software system, the project team could receive a large number of bug reports. Some bug reports could be duplicates as they essentially report the same problem. It is often tedious and costly to manually check if a newly reported bug is a duplicate of an already reported bug. In this paper, we propose BugSim, a method that can automatically retrieve duplicate bug reports given a new bug report. BugSim is based on learning to rank concepts. We identify textual and statistical features of bug reports and propose a similarity function for bug reports based on the features. We then construct a training set by assembling pairs of duplicate and non-duplicate bug reports. We train the weights of features by applying the stochastic gradient descent algorithm over the training set. For a new bug report, we retrieve candidate duplicate reports using the trained model. We evaluate BugSim using more than 45,100 real bug reports of twelve Eclipse projects. The evaluation results show that the proposed method is effective. On average, the recall rate for the top 10 retrieved reports is 76.11%. Furthermore, BugSim outperforms the previous state-of-art methods that are implemented using SVM and BM25Fext.

Users can use bug tracking systems (BTS) such as Bugzilla, JIRA, and GitHub to report defects they detect while using or testing the software applications. Bugzilla has specified the bug's information such as bug ID, description, product, classification, component, platform, operating system, bug status, resolution, priority, and severity. The severity assigned to the bug report is conducted manually. So, it takes some considerable time to assign the severity manually, and also it is a tedious task. In this research, we used a deep learning-based technique to develop the prediction model for severity. Our proposed approach is based on the convolutional neural network (CNN) for severity prediction of the bug reports. First, we pre-process the textual content in bug reports using natural language processing (NLP) approaches. Then we extract the features from the textual context (short description) using the Bag-of-Words feature extraction method. Finally, we train a CNN-based classifier to severity prediction based on its input. Then our result is compared with the Support Vector Machine (SVM) and Temporal Convolutional Network (TCN) to find a better model for severity prediction. The final results show that the proposed approach based on the CNN classifier performs better than the other approaches, and it shows an 81% accuracy while others have low accuracy, like 61% and 48% for SVM and TCN, respectively. The proposed model's performance was evaluated using the Bugzilla dataset, which included over 25,000 bug reports. The accuracy, precision, recall, and F1-Score are used to measure the performance.

It is a challenging task to accurately assign bug reports to an expert developer in both Open source project (OSP) as well as closed source projects (CSP). Objective of any triaging techniques is to reduce tossing length thereby minimizing bug fixing time. Developing such an efficient triaging technique is even more challenging for CSP due to limited accessibility of bug reports. As a result, most of the existing triaging techniques have been trained and tested using bugs from OSP leaving CSP untapped to a larger extent. In this paper, we have built a metric based score for identifying and ranking an expert developer independent of the nature of the project. We have proposed two metrics based on information theory i.e. developer specialization and bug importance score. Developer specialization score and Importance score are computed using information gain on bug component and bug importance fields. In order to validate our proposed approach, we choose bug reports from a closed source project developed by company XYZ, pvt. Ltd. India (the real name of the company is not disclosed due to confidentiality reason). Our proposed triaging approach has achieved good prediction results up to 88.9% on the chosen closed source project dataset.

The automated task of locating the potential buggy files in a software project given a bug report is called bug localization. Bug localization helps developers focus on crucial files. However, the existing automated bug localization approaches face a key challenge, called lexical mismatch. Specifically, the terms used in bug reports to describe a bug are different from the terms and code tokens used in source files. To address that, we present a novel approach that uses deep neural network (DNN) in combination with rVSM, an information retrieval (IR) technique. rVSM collects the feature on the textual similarity between bug reports and source files. DNN is used to learn to relate the terms in bug reports to potentially different code tokens and terms in source files. Our empirical evaluation on real-world bug reports in the open-source projects shows that DNN and IR complement well to each other to achieve higher bug localization accuracy than individual models. Importantly, our new model, DNNLOC, with a combination of the features built from DNN, rVSM, and project's bug-fixing history, achieves higher accuracy than the state-of-the-art IR and machine learning techniques. In half of the cases, it iscorrect with just a single suggested file. In 66% of the time, acorrect buggy file is in the list of three suggested files. With 5 suggested files, it is correct in almost 70% of the cases.

Along with the exponential growth of text data on the Web, particularly of the user-generated content, comes an increasing need for hierarchically organizing documents, retrieving documents accurately, and discovering evolutionary trends of various popular topics from the data. However, all of these are challenging due to the diversity, heterogeneity, noisiness and time-sensitivity of Web 2.0 data. Motivated by this, we tackle the challenges at a fundamental level, by proposing a novel topic modeling method with ontological guidance. It may be used to discover topic language models formalizing various terms relevant to given topics using the Web data. The topic model takes into account both the ontological relationships amongst the topics defined in a topic taxonomy and also word co-occurrence patterns in the data to automatically identify the portions in the data relevant to the topics. Then, it estimates language models for these topics from these relevant portions. At an application level, we use the topic model to propose novel approaches for three different tasks, namely hierarchical text classification without labeled data, information retrieval with pseudo-relevance feedback, and discovering topic evolutionary trends. Our classification experiment on the IPTC (International Press and Telecommunications Council) taxonomy, containing more than 1100 topics, shows that our approach achieves a performance of 67% in terms of the hierarchical version of the F-1 measure, without using any labeled data. Our retrieval experiments on five benchmark datasets show that compared to baseline retrieval (without pseudo-relevance feed-back), our approach improves on average 39% in terms of mean average precision. Finally, for the last task, using blog data, our approach discovers meaningful insights on how the crowd responds to various news topics such as the language used to discuss each topic, how this language drifts over time, and when the crowd’s focus on a topic increases, reaches a peak, and declines.

With a great number of software applications that have been developed, software maintenance has become an important and challenging task, particularly due to the increasing scale of software projects. Even if developers can create and update bug reports in bug repositories to support software maintenance, a large software project receives a large number of bug reports each day. For reducing the workload of developers, many researchers and software engineers have begun recommending appropriate developers to fix bugs. This process is called bug triage and is a hot research topic for software maintenance. In this paper, we propose a hybrid bug triage algorithm, combining a probability model and an experience model to rank all candidate developers for fixing a new bug. For this study, we adopted the smoothed Unigram Model (UM) instead of the traditional Vector Space Model (VSM) to search similar bug reports. In the probability model, we used a social network to analyze the probability of fixing a new bug for a candidate developer. We first proposed to add a new feature (the number of re-opened bugs) in order to get the fixing probability. In the experience model, we considered the number of fixed bugs and fixing cost for each candidate developer as the estimate factor. In addition, we introduced a new concept, activity factor, to better model developers' experience. We performed the experiments on two large-scale, open source projects. The results show that our method can effectively recommend the best developer for fixing bugs.

A significant amount of time is spent by software developers in investigating bug reports. It is useful to indicate when a bug report will be closed, since it would help software teams to prioritise their work. Several studies have been conducted to address this problem in the past decade. Most of these studies have used the frequency of occurrence of certain developer activities as input attributes in building their prediction models. However, these approaches tend to ignore the temporal nature of the occurrence of these activities. In this paper, a novel approach using Hidden Markov Models and temporal sequences of developer activities is proposed. The approach is empirically demonstrated in a case study using eight years of bug reports collected from the Firefox project. Our proposed model correctly identifies bug reports with expected bug fix times. We also compared our proposed approach with the state of the art technique in the literature in the context of our case study. Our approach results in approximately 33 percent higher F-measure than the contemporary technique based on the Firefox project data.

Software engineering researchers have been applying tools and techniques from information retrieval (IR) to problems such as bug localization to lower the manual effort required to perform maintenance tasks. The central challenge when using an IR-based tool is the formation of a high-quality query. When performing bug localization, one easily accessible source of query words is the bug report. A recent paper investigated the sufficiency of this source by using a genetic algorithm (GA) to build high quality queries. Unfortunately, the GA in essence cheats as it makes use of query performance when evolving a good query. This raises the question, is it feasible to attain similar results without cheating? One approach to providing cheat-free queries is to employ automatic summarization. The performance of the resulting summaries calls into question the sufficiency of the bug reports as a source of query words. To better understand the situation, Information Need Analysis (INA) is applied to quantify both how well the GA is performing and, perhaps more importantly, how well a bug report captures the vocabulary needed to perform IR-based bug localization. The results find that summarization shows potential to produce high-quality queries, but it requires more training data. Furthermore, while bug reports provide a useful source of query words, they are rather limited and thus query expansion techniques, perhaps in combination with summarization, will likely produce higher-quality queries.

One of the most important tasks related to managing bug reports is localizing the fault so that a fix can be applied. As such, prior work has aimed to automate this task of bug localization by formulating it as an information retrieval problem, where potentially buggy files are retrieved and ranked according to their textual similarity with a given bug report. However, there is often a notable semantic gap between the information contained in bug reports and identifiers or natural language contained within source code files. For user-facing software, there is currently a key source of information that could aid in bug localization, but has not been thoroughly investigated - information from the graphical user interface (GUI).