Multiclass Labelling For Bug Report Severity Prediction Using Semantic Analysis And Machine Learning

Issue tracking systems such as Bugzilla are tools to facilitate collaboration between software maintenance professionals. Popular issue tracking systems consists of discussion forums to facilitate bug reporting and comment posting. We observe that several comments posted in issue tracking system contains link to external websites such as YouTube (video sharing website), Twitter (micro-blogging website), Stack overflow (a community-based question and answering website for programmers), Wikipedia and focused discussions forums. Stack overflow is a popular community-based question and answering website for programmers and is widely used by software engineers as it contains answers to millions of questions (an extensive knowledge resource) posted by programmers on diverse topics. We conduct a series of experiments on open-source Google Chromium and Android issue tracker data (publicly available real-world dataset) to understand the role and impact of Stack overflow in issue resolution. Our experimental results show evidences of several references to Stack overflow in threaded discussions and demonstrate correlation between a lower mean time to repair (in one dataset) with presence of Stack overflow links. We also observe that the average number of comments posted in response to bug reports are less when Stack overflow links are presented in contrast to bug reports not containing Stack overflow references. We conduct experiments based on textual similarly analysis (content-based linguistic features) and contextual data analysis (exploited metadata such as tags associated to a Stack overflow question) to recommend Stack overflow questions for an incoming bug report. We perform empirical analysis to measure the effectiveness of the proposed method on a dataset containing ground-truth and present our insights. We present the result of a survey (of Google Chromium Developers) that we conducted to understand practitioner's perspective and experience.

Bug fixing is a critical activity in the software development process. In issue tracking systems such as JIRA, each bug report is assigned a priority level to indicate the urgency and importance level of the bug. The priority may change during the bug fixing process, indicating that the urgency and importance level of the bug will change with the bug fixing. However, manually evaluating priority changes for bugs is a tedious process that heavily relies on the subjective judgment of developers and project managers, leading to incorrect priority changes and thus hindering timely bug fixes. Given the lack of research on bug priority change prediction, we propose a novel two-phase bug report priority change prediction method based on bug fixing evolution features and class imbalance handling strategy. Specifically, we divided the bug lifecycle into two phases: bug reporting and bug fixing, and constructed bug priority change prediction models for each phase. To evaluate the performance of our method, we conducted experiments on a bug dataset constructed from 32 non-trivial Apache projects. The experimental results show that our proposed bug fixing evolution features and the adopted class imbalance handling strategy can effectively improve the performance of prediction models. The F1-score of the prediction model constructed for the bug reporting phase reached 0.798, while the F1-weighted and F1-macro of the prediction model constructed for the bug fixing phase were 0.712 and 0.613, respectively. Furthermore, we explored the cross-project applicability of our prediction models and their performance at different priority levels. The findings indicate large variations in model performance across different projects, although the overall scores remain decent. Meanwhile, the predictive performance across various priority levels remained relatively consistently high.

A bug-tracking system such as Bugzilla contains bug reports (BRs) collected from various sources such as development teams, testing teams, and end users. When bug reporters submit bug reports to a bug-tracking system, the bug reporters need to label the bug reports as security bug reports (SBRs) or not, to indicate whether the involved bugs are security problems. These SBRs generally deserve higher priority in bug fixing than not-security bug reports (NSBRs). However, in the bug-reporting process, bug reporters often mislabel SBRs as NSBRs partly due to lack of security domain knowledge. This mislabeling could cause serious damage to software-system stakeholders due to the induced delay of identifying and fixing the involved security bugs. To address this important issue, we developed a new approach that applies text mining on natural-language descriptions of BRs to train a statistical model on already manually-labeled BRs to identify SBRs that are manually-mislabeled as NSBRs. Security engineers can use the model to automate the classification of BRs from large bug databases to reduce the time that they spend on searching for SBRs. We evaluated the model's predictions on a large Cisco software system with over ten million source lines of code. Among a sample of BRs that Cisco bug reporters manually labeled as NSBRs in bug reporting, our model successfully classified a high percentage (78%) of the SBRs as verified by Cisco security engineers, and predicted their classification as SBRs with a probability of at least 0.98.

For a large and complex software system, the project team could receive a large number of bug reports. Some bug reports could be duplicates as they essentially report the same problem. It is often tedious and costly to manually check if a newly reported bug is a duplicate of an already reported bug. In this paper, we propose BugSim, a method that can automatically retrieve duplicate bug reports given a new bug report. BugSim is based on learning to rank concepts. We identify textual and statistical features of bug reports and propose a similarity function for bug reports based on the features. We then construct a training set by assembling pairs of duplicate and non-duplicate bug reports. We train the weights of features by applying the stochastic gradient descent algorithm over the training set. For a new bug report, we retrieve candidate duplicate reports using the trained model. We evaluate BugSim using more than 45,100 real bug reports of twelve Eclipse projects. The evaluation results show that the proposed method is effective. On average, the recall rate for the top 10 retrieved reports is 76.11%. Furthermore, BugSim outperforms the previous state-of-art methods that are implemented using SVM and BM25Fext.

When a new bug report is received, developers usually need to reproduce the bug and perform code reviews to find the cause, a process that can be tedious and time consuming. A tool for ranking all the source files with respect to how likely they are to contain the cause of the bug would enable developers to narrow down their search and improve productivity. This project introduces an adaptive ranking approach that leverages project knowledge through functional decomposition of source code, API descriptions of library components, the bugfixing history, the code change history, and the file dependency graph. Given a bug report, the ranking score of each source file is computed as a weighted combination of an array of features, where the weights are trained automatically on previously solved bug reports using a learning-to-rank technique. I applied SVM (Support Virtual Machine) to classify the bug reports to identify, which category the bug belongs to. It helps to fix the critical defects early. The ranking system evaluated on six large scale open source Java projects, using the beforefix version of the project for every bug report. The experimental results show that the learning-to-rank approach outperforms three recent state-of-the-art methods. In particular, proposed method makes correct recommendations within the top 10 ranked source files for over 70 percent of the bug reports in the Eclipse Platform and Tomcat projects.

For large open source projects (e.g., Eclipse, Mozilla), developers usually utilize bug reports to facilitate software maintenance tasks such as fixer assignment. However, there are a large portion of short reports in bug repositories. We find that 78.1% of bug reports only include less than 100 words in Eclipse and require bug fixers to spend more time on resolving them due to limited informative contents. To address this problem, in this paper, we propose a novel approach to enrich bug reports. Concretely, we design a sentence ranking algorithm based on a new textual similarity metric to select the proper contents for bug report enrichment. For the enriched bug reports, we conduct a user study to assess whether the additional sentences can provide further help to fixer assignment. Moreover, we assess whether the enriched versions can improve the performance of automated fixer recommendation. In particular, we perform three popular automated fixer recommendation approaches on the enriched bug reports of Eclipse, Mozilla, and GNU Compiler Collection (GCC). The experimental results show that enriched bug reports improve the average F-measure scores of the automated fixer recommendation approaches by up to 10% for DREX, 13.37% for DRETOM, and 8% for DevRec when top-10 bug fixers are recommended.

This paper proposes a supervised machine learning approach for predicting the root cause of a given bug report. Knowing the root cause of a bug can help developers in the debugging process - either directly or indirectly by choosing proper tool support for the debugging task. We mined 54755 closed bug reports from the issue trackers of 103 GitHub projects and applied a set of heuristics to create a benchmark consisting of 10459 reports. A subset was manually classified into three groups (semantic, memory, and concurrency) based on the bugs' root causes. Since the types of root cause are not equally distributed, a combination of keyword search and random selection was applied. Our data set for the machine learning approach consists of 369 bug reports (122 concurrency, 121 memory, and 126 semantic bugs). The bug reports are used as input to a natural language processing algorithm. We evaluated the performance of several classifiers for predicting the root causes for the given bug reports. Linear Support Vector machines achieved the highest mean precision (0.74) and recall (0.72) scores. The created bug data set and classification are publicly available.

Free/Open Source Software (FOSS) communities often use open bug reporting to allow users to participate by reporting bugs. This practice can lead to more duplicate reports, as users can be less rigorous about researching existing bug reports. This paper examines how FOSS projects deal with duplicate bug reports. We examined 12 FOSS projects: 4 small, 4 medium and 4 large, where size was determined by number of code contributors. First, we found that contrary to what has been reported from studies of individual large projects like Mozilla and Eclipse, duplicate bug reports are a problem for FOSS projects, especially medium-sized, which struggle with a large number of submissions without the resources of large projects. Second, we found that the focus of a project does not affect the number of duplicate bug reports. Our findings indicate a need for additional scaffolding and training for bug reporters.

One of the key tasks related to ensuring mobile app quality is the reporting, management, and resolution of bug reports. As such, researchers have committed considerable resources toward automating various tasks of the bug management process for mobile apps, such as reproduction and triaging. However, the success of these automated approaches is largely dictated by the characteristics and properties of the bug reports they operate upon. As such, understanding mobile app bug reports is imperative to drive the continued advancement of report management techniques. While prior studies have examined high-level statistics of large sets of reports, we currently lack an in-depth investigation of how the information typically reported in mobile app issue trackers relates to the specific details generally required to reproduce the underlying failures. In this paper, we perform an in-depth analysis of 180 reproducible bug reports systematically mined from Android apps on GitHub and investigate how the information contained in the reports relates to the task of reproducing the described bugs. In our analysis, we focus on three pieces of information: the environment needed to reproduce the bug report, the steps to reproduce (S2Rs), and the observed behavior. Focusing on this information, we characterize failure types, identify the modality used to report the information, and characterize the quality of the information within the reports. We find that bugs are reported in a multi-modal fashion, the environment is not always provided, and S2Rs often contain missing or non-specific enough information. These findings carry with them important implications on automated bug reproduction techniques as well as automated bug report management approaches more generally.

With a great number of software applications that have been developed, software maintenance has become an important and challenging task, particularly due to the increasing scale of software projects. Even if developers can create and update bug reports in bug repositories to support software maintenance, a large software project receives a large number of bug reports each day. For reducing the workload of developers, many researchers and software engineers have begun recommending appropriate developers to fix bugs. This process is called bug triage and is a hot research topic for software maintenance. In this paper, we propose a hybrid bug triage algorithm, combining a probability model and an experience model to rank all candidate developers for fixing a new bug. For this study, we adopted the smoothed Unigram Model (UM) instead of the traditional Vector Space Model (VSM) to search similar bug reports. In the probability model, we used a social network to analyze the probability of fixing a new bug for a candidate developer. We first proposed to add a new feature (the number of re-opened bugs) in order to get the fixing probability. In the experience model, we considered the number of fixed bugs and fixing cost for each candidate developer as the estimate factor. In addition, we introduced a new concept, activity factor, to better model developers' experience. We performed the experiments on two large-scale, open source projects. The results show that our method can effectively recommend the best developer for fixing bugs.

For complex and popular software, project teams could receive a large number of bug reports. It is often tedious and costly to manually assign these bug reports to developers who have the expertise to fix the bugs. Many bug triage techniques have been proposed to automate this process. In this paper, we describe our study on applying conventional bug triage techniques to projects of different sizes. We find that the effectiveness of a bug triage technique largely depends on the size of a project team (measured in terms of the number of developers). The conventional bug triage methods become less effective when the number of developers increases. To further improve the effectiveness of bug triage for large projects, we propose a novel recommendation method called Bug Fixer, which recommends developers for a new bug report based on historical bug-fix information. Bug Fixer constructs a Developer-Component-Bug (DCB) network, which models the relationship between developers and source code components, as well as the relationship between the components and their associated bugs. A DCB network captures the knowledge of "who fixed what, where". For a new bug report, Bug Fixer uses a DCB network to recommend to triager a list of suitable developers who could fix this bug. We evaluate Bug Fixer on three large-scale open source projects and two smaller industrial projects. The experimental results show that the proposed method outperforms the existing methods for large projects and achieves comparable performance for small projects.

A large open source project consists of a wide range of bug reports. In open source project, bug reports are available and these reports can be modified by anyone. Bugs are software defects whose prediction is highly difficult. To detect the bugs, machine learning classifier has been proposed. It segregates the bug reports and developers and it learns the type of report suitable for each developer. Bug triage is the process of assigning the bug for the appropriate developer. The techniques include preprocessing, machine learning classifier, instance selection and feature selection. The aim of this paper is to attain a data set reduction in bug triage by including the representative values along with the statistical values of the bug data set. Our work considers the dataset from the open source project Eclipse. We focus on reducing the data scale and thereby improving the accuracy. This can be achieved by building a representative model for prediction of reduction orders by including the summary, metadata. Our proposed work attains an accuracy result of 96.5% that is better when compared with existing work.

[Context and motivation] Traces between issues in issue tracking systems connect bug reports to software features, they connect competing implementation ideas for a software feature or they identify duplicate issues. However, the trace quality is usually very low. To improve the trace quality between requirements, features, and bugs, information retrieval algorithms for automated trace retrieval can be employed. Prevailing research focusses on structured and well-formed documents, such as natural language requirement descriptions. In contrast, the information in issue tracking systems is often poorly structured and contains digressing discussions or noise, such as code snippets, stack traces, and links. Since noise has a negative impact on algorithms for automated trace retrieval, this paper asks: [Question/Problem] Do information retrieval algorithms for automated traceability perform effectively on issue tracking system data? [Results] This paper presents an extensive evaluation of the performance of five information retrieval algorithms. Furthermore, it investigates different preprocessing stages e.g. stemming or differentiating code snippets from natural language and evaluates how to take advantage of an issue's structure e.g. title, description, and comments to improve the results. The results show that algorithms perform poorly without considering the nature of issue tracking data, but can be improved by project-specific preprocessing and term weighting. [Contribution] Our results show how automated trace retrieval on issue tracking system data can be improved. Our manually created gold standard and an open-source implementation based on the OpenTrace platform can be used by other researchers to further pursue this topic.

As fixing bugs is a large part of software development and maintenance, estimating the time to fix bugs -bug fixing time- is extremely useful when planning software projects. Therefore, in this study, we propose a way to estimate bug fixing time using bug reports. First, we classify previous bug reports with meta fields by applying a k-NN method. Next, we compute the similarity of the new bug and previous bugs by using data from bug reports. Finally, we estimate how long it will take to fix the new bug using the time it took to repair similar bugs. In this study, we perform experiments with open source software. The results of these experiments show that our approach effectively estimates the bug fixing time.

In this paper, we present a new approach to localize a bug in the software source file hierarchy. The proposed approach uses log files of the revision control system and bug reports information in open bug repository of open source projects to train a Support Vector Machine (SVM) classifier. Our approach employs textual information in summary and description of bugs reported to the bug repository, in order to form machine learning features. The class labels are revision paths of fixed issues, as recorded in the log file of the revision control system. Given an unseen bug instance, the trained classifier can predict which part of the software source file hierarchy (revision path) is more likely to be related to this issue. Experimental results on more than 2000 bug reports of ‘UI’component of the Eclipse JDT project from the initiation date of the project until November 24, 2009 (about 8 years) using this approach, show weighted precision and recall values of about 98% on average.