Characterizing duplicate bugs: Perceptions of practitioners and an empirical analysis

Bug deduplication is a hot topic in software engineering information retrieval research, but it is often not deployed. Typically to de-duplicate bug reports developers rely upon the search capabilities of the bug report software they employ, such as Bugzilla, Jira, or Github Issues. These search capabilities range from simple SQL string search to IR-based word indexing methods employed by search engines. Yet too often these searches do very little to stop the creation of duplicate bug reports. Some bug trackers have more than 10\% of their bug reports marked as duplicate. Perhaps these bug tracker search engines are not enough? In this paper we propose a method of attempting to prevent duplicate bug reports before they start: continuous querying. That is as the bug reporter types in their bug report their text is used to query the bug database to find duplicate or related bug reports. This continuous querying allows the reporter to be alerted to duplicate bug reports as they report the bug, rather than formulating queries to find the duplicate bug report. Thus this work ushers in a new way of evaluating bug report deduplication techniques, as well as a new kind of bug deduplication task. We show that simple IR measures show some promise for addressing this problem but also that further research is needed to refine this novel process that is integrate-able into modern bug report systems.

Bug reporting is an uncoordinated process that is often the cause of redundant workload in triaging and fixing bugs due to many duplicated bug reports. Furthermore, quite often, same bugs are repeatedly reported as users or testers are unaware of whether they have been reported from the search query results. In order to reduce both the users and developers' efforts, the quality of search in a bug tracking system is crucial. However, all existing search functions in a bug tracking system produce results with undesired relevance and ranking. Hence, it is essential to provide an effective search function to any bug tracking system.Learning to rank (LTR) is a supervised machine learning technique that is used to construct a ranking model from training data. We propose a novel approach by using LTR to search for potentially related bug reports in a bug tracking system. Our method uses a set of proposed features of bug reports and queries. A preliminary evaluation shows that our approach can enhance the quality of searching for similar bug reports, therefore, relieving the burden of developers in dealing with duplicate bug reports.

Free/Open Source Software (FOSS) communities often use open bug reporting to allow users to participate by reporting bugs. This practice can lead to more duplicate reports, as users can be less rigorous about researching existing bug reports. This paper examines how FOSS projects deal with duplicate bug reports. We examined 12 FOSS projects: 4 small, 4 medium and 4 large, where size was determined by number of code contributors. First, we found that contrary to what has been reported from studies of individual large projects like Mozilla and Eclipse, duplicate bug reports are a problem for FOSS projects, especially medium-sized, which struggle with a large number of submissions without the resources of large projects. Second, we found that the focus of a project does not affect the number of duplicate bug reports. Our findings indicate a need for additional scaffolding and training for bug reporters.

For a large and complex software system, the project team could receive a large number of bug reports. Some bug reports could be duplicates as they essentially report the same problem. It is often tedious and costly to manually check if a newly reported bug is a duplicate of an already reported bug. In this paper, we propose BugSim, a method that can automatically retrieve duplicate bug reports given a new bug report. BugSim is based on learning to rank concepts. We identify textual and statistical features of bug reports and propose a similarity function for bug reports based on the features. We then construct a training set by assembling pairs of duplicate and non-duplicate bug reports. We train the weights of features by applying the stochastic gradient descent algorithm over the training set. For a new bug report, we retrieve candidate duplicate reports using the trained model. We evaluate BugSim using more than 45,100 real bug reports of twelve Eclipse projects. The evaluation results show that the proposed method is effective. On average, the recall rate for the top 10 retrieved reports is 76.11%. Furthermore, BugSim outperforms the previous state-of-art methods that are implemented using SVM and BM25Fext.

Does Deep Learning improve the performance of duplicate bug report detection? An empirical study

In software maintenance, bug reports play an important role in the correctness of software packages. Unfortunately, the duplicatebug report problem arises because there are too many duplicate bug reports in various software projects. Handling with duplicate bug reports is thus time-consuming and has high cost of software maintenance. Therefore, this research introduces a detection scheme based on the extended class centroid information (ECCI) to enhance thedetection performance. This method is extended from the previous one, which used only centroid method without considering the effects of both inner and inter class. Besides, this method also improved the previous use of normalized cosine in identifying the similarity between two bug reports by denormalized cosine. The effectiveness of ECCI is proved through the empirical study with three open-source projects: SVN, Argo UML and Apache. The experimental results show thatECCI outperforms other detection schemes by about 10% in all cases.

For open source software such as Firefox, Eclipse, Subversion,. . . they usually have a system for bug management that sent by users. These bug reports help the system identify various software bugs which makes software maintenance better. However, a situation occurs that there are many error reports sent to the processing repository that these bug reports were previously reported by different users, this is called duplicate bug reports. In this paper, we introduce a multi-feature model combined with weighted improvements from CFC (ClassFeature-Centroid) to detect duplicate bug reportsautomatically. We have experimented on three projects of Mozilla, Eclipse and OpenOffice. The results show that our method can improve 8-12% better as compared to the compared methods.

For large open source projects (e.g., Eclipse, Mozilla), developers usually utilize bug reports to facilitate software maintenance tasks such as fixer assignment. However, there are a large portion of short reports in bug repositories. We find that 78.1% of bug reports only include less than 100 words in Eclipse and require bug fixers to spend more time on resolving them due to limited informative contents. To address this problem, in this paper, we propose a novel approach to enrich bug reports. Concretely, we design a sentence ranking algorithm based on a new textual similarity metric to select the proper contents for bug report enrichment. For the enriched bug reports, we conduct a user study to assess whether the additional sentences can provide further help to fixer assignment. Moreover, we assess whether the enriched versions can improve the performance of automated fixer recommendation. In particular, we perform three popular automated fixer recommendation approaches on the enriched bug reports of Eclipse, Mozilla, and GNU Compiler Collection (GCC). The experimental results show that enriched bug reports improve the average F-measure scores of the automated fixer recommendation approaches by up to 10% for DREX, 13.37% for DRETOM, and 8% for DevRec when top-10 bug fixers are recommended.

Bug reporting is essentially an uncoordinated process. The same bugs could be repeatedly reported because users or testers are unaware of previously reported bugs. As a result, extra time could be spent on bug triaging and fixing. In order to reduce redundant effort, it is important to provide bug reporters with the ability to search for previously reported bugs. The search functions provided by the existing bug tracking systems are using relatively simple ranking functions, which often produce unsatisfactory results. In this paper, we adopt Ranking SVM, a Learning to Rank technique to construct a ranking model for effective bug report search. We also propose to use the knowledge of Wikipedia to discover the semantic relations among words and documents. Given a user query, the constructed ranking model can search for relevant bug reports in a bug tracking system. Unlike related works on duplicate bug report detection, our approach retrieves existing bug reports based on short user queries, before the complete bug report is submitted. We perform evaluations on more than 16,340 Eclipse and Mozilla bug reports. The evaluation results show that the proposed approach can achieve better search results than the existing search functions provided by Bugzilla and Lucene. We believe our work can help users and testers locate potential relevant bug reports more precisely.

In this paper data mining is applied on bug database to discover the similar and duplicate bugs. Whenever a new bug will be entered in the bug database through bug tracking system, it will be matched against the existing bugs and duplicate and similar bugs will be mined from the bug database. Similar kind of bugs are resolved in almost in same manners. So if a bug is found somewhere similar to other existing bug which is already resolved then its resolution will take less time, since some of the bug analysis part is similar to existing one, hence it will save time. In the existing tradition developers must have to manually identify duplicate bug reports, but this identification process is time-consuming and exacerbates the already high cost of software maintenance. So if the similar and duplicate bugs can be found out using some approach it will be a cost and time saving activity. Based on this concept a weight similarity measurement model based object orinted approach is described here in this paper to discover similar and duplicate bugs in the bug database.

In this data package, you will find two folders: `artifacts` and `outputs` `artifacts` contains the videos we collected in our user study, the model files for the different models we evaluated, and the detailed results that we generated (See `Detailed Results` section for more information). The `videos` folder is broken down by user, where each user has a folder contain the apps they were given to create videos for. Each of the apps contain folders that denote the bug they generated a report for. Finally, instead these bug folders there is the actual video-based bug report as an mp4 file. The `user_assignment.csv` just contains the finalized assignments of users to corresponding bug reports. In the `models` folder, you will find the two models we evaluated (SIFT, SimCLR, and OCR+IR). In each folder you will find the corresponding trained codebook files that we generated for SIFT and SimCLR. These codebook files are pickle files that contain the binary representation of the trained codebooks. Additionally, in the SimCLR folder, you will find a checkpoint and pytorch model file that contains all the necessary information for reloading our trained SimCLR model. For the `OCR+IR` folder, you will find all of the code for the OCR+IR model as well as the intermediate output for this particular model, other models' outputs are stored in the `outputs` folder. The `outputs` folder contains all of the intermediate outputs of our code, except for OCR+IR. In the `results` folder, you will find all of the raw rankings and metrics for the SIFT and SimCLR model for all combinations of video-based bug reports per app. <strong>NOTE: SIFT is missing the 10k raw ranking and metrics, but will be provided in a future version. </strong>`evaluation_setting` contains a json file that contains all of the duplicate detection tasks we used for evaluating our models, i.e. `setting 2` (See paper for more details). `user_rankings_weighted_all` and `user_results_weighted_all` contain converted version of the raw rankings and metrics for the SIFT and SimCLR model to match `setting 2`. `extracted_text` contains the output of running the OCR model, i.e. the frames of the videos and the text from each frame. Lastly, `combined` contains the results of the combined tango approach. You can find a spreadsheet containing the results for all of the different configurations we tested at `tango_reproduction_package/artifacts/detailed_results.xlsx`. In this excel file, we have multiple sheets. `overall` shows the performance of the different model configurations averaged across all apps. `overall_comb` shows the combined performance of the visual and textual model configurations averaged across all apps. Additionally, `per-app` and `per-app-comb` has the performance of the single and combined model configurations per app, respectively. Lastly, we provide the overall performance in sheet `overall_user_study` and `overall_user_study_comb` of the single and combined model configurations on the settings (used only APOD app) given to the users for evaluating how much time and effort tango can save developers. All sheets show the performance in terms of mRR (`avg_rr`), the standard deviation of recipical rank, median (`med_rr`), and quartile 1 and 3 (`q#_rr`). The same is true for mAP (`avg_ap`). We also show the performance in terms of average rank including standard deviation, and quartiles. Lastly, we providing HIT@1-5, 7, and 10 (`h#`). Sheets that contain the `weight` column have information regarding how much weight is given to the visual and textual information. A value of `0.1` means that the textual information received a weight of `0.1` while the visual information was given a weight of `0.9`. For values containing two numbers, e.g. `0.1-0.0`, refers to the weighting scheme introduced in the paper for when there may be high overlap in vocabulary between duplicate and non-duplicates (See paper for more details). If an app does not have high overlap, then a weight of `0.1` is used for the textual information, else the textual information is not considered, i.e., weight of `0.0`. To use our data to reproduce our results please visit our github repository: https://github.com/ncoop57/tango

With the increasing number of bug reports coming into the open bug repository, it is impossible to triage bug reports manually by software managers. This paper proposes a novel approach called En-LDA (Entropy optimized Latent Dirichlet Allocation (LDA)) for automatic bug report assignment. Specifically, we propose entropy to optimize the number of topics of the LDA model and further use the entropy optimized LDA to capture the expertise and interest of developers in bug resolution. A developer’s interest in a topic is modeled by the number of the developer’s comments on bug reports of the topic divided by the number of all the developer’s comments. A developer’s expertise in a topic is modeled by the number of the developer’s comments on bug reports of the topic divided by the number of all developers’ comments on the topic. Given a new bug report, En-LDA recommends a ranked list of developers who are potentially adequate to resolve the new bug. Experiments on Eclipse JDT and Mozilla Firefox projects show that En-LDA can achieve high recall up to 84% and 58%, and precision up to 28% and 41%, respectively, which indicates promising aspects of the proposed approach.

A bug-tracking system such as Bugzilla contains bug reports (BRs) collected from various sources such as development teams, testing teams, and end users. When bug reporters submit bug reports to a bug-tracking system, the bug reporters need to label the bug reports as security bug reports (SBRs) or not, to indicate whether the involved bugs are security problems. These SBRs generally deserve higher priority in bug fixing than not-security bug reports (NSBRs). However, in the bug-reporting process, bug reporters often mislabel SBRs as NSBRs partly due to lack of security domain knowledge. This mislabeling could cause serious damage to software-system stakeholders due to the induced delay of identifying and fixing the involved security bugs. To address this important issue, we developed a new approach that applies text mining on natural-language descriptions of BRs to train a statistical model on already manually-labeled BRs to identify SBRs that are manually-mislabeled as NSBRs. Security engineers can use the model to automate the classification of BRs from large bug databases to reduce the time that they spend on searching for SBRs. We evaluated the model's predictions on a large Cisco software system with over ten million source lines of code. Among a sample of BRs that Cisco bug reporters manually labeled as NSBRs in bug reporting, our model successfully classified a high percentage (78%) of the SBRs as verified by Cisco security engineers, and predicted their classification as SBRs with a probability of at least 0.98.

Bug fixing is a critical activity in the software development process. In issue tracking systems such as JIRA, each bug report is assigned a priority level to indicate the urgency and importance level of the bug. The priority may change during the bug fixing process, indicating that the urgency and importance level of the bug will change with the bug fixing. However, manually evaluating priority changes for bugs is a tedious process that heavily relies on the subjective judgment of developers and project managers, leading to incorrect priority changes and thus hindering timely bug fixes. Given the lack of research on bug priority change prediction, we propose a novel two-phase bug report priority change prediction method based on bug fixing evolution features and class imbalance handling strategy. Specifically, we divided the bug lifecycle into two phases: bug reporting and bug fixing, and constructed bug priority change prediction models for each phase. To evaluate the performance of our method, we conducted experiments on a bug dataset constructed from 32 non-trivial Apache projects. The experimental results show that our proposed bug fixing evolution features and the adopted class imbalance handling strategy can effectively improve the performance of prediction models. The F1-score of the prediction model constructed for the bug reporting phase reached 0.798, while the F1-weighted and F1-macro of the prediction model constructed for the bug fixing phase were 0.712 and 0.613, respectively. Furthermore, we explored the cross-project applicability of our prediction models and their performance at different priority levels. The findings indicate large variations in model performance across different projects, although the overall scores remain decent. Meanwhile, the predictive performance across various priority levels remained relatively consistently high.

In Software Development Life Cycle, fixing defect bugs is one of the essential activities of the software maintenance phase. Bug severity indicates how major or minor the bug impacts on the execution of the system and how rapidly the developer should fix it. Triaging a vast amount of new bugs submitted to the software bug repositories is a cumbersome and time-consuming process. Manual triage might lead to a mistake in assigning the appropriate severity level for each bug. As a consequence, a delay for fixing severe software bugs will take place. However, the whole process of assigning the severity level for bug reports should be automated. In this paper, we aim to build prediction models that will be utilized to determine the class of the severity (severe or non-severe) of the reported bug. To validate our approach, we have constructed a dataset from historical bug reports stored in JIRA bug tracking system. These bug reports are related to different closed-source projects developed by INTIX Company located in Amman, Jordan. We compare eight popular machine learning algorithms, namely Naive Bayes, Naive Bayes Multinomial, Support Vector Machine, Decision Tree (J48), Random Forest, Logistic Model Trees, Decision Rules (JRip) and K-Nearest Neighbor in terms of accuracy, F-measure and Area Under the Curve (AUC). According to the experimental results, a Decision Tree algorithm called Logistic Model Trees achieved better performance compared to other machine learning algorithms in terms of Accuracy, AUC and F-measure with values of 86.31, 0.90 and 0.91, respectively.