Improving Just-In-Time Comment Updating via AST Edit Sequence

During software development and maintenance, code comments are often missing, inadequate, or they do not match the actual code content. In response to this problem, the research community has proposed a method for updating natural language comments based on code changes. However, there are two major limitations of this method that must be addressed: the long-term and non-temporal dependencies in the source code. To address these limitations, we propose a new model called code semantic learning–comment update (CSL2CU). The code semantic learning component of CSL2CU uses a self-attention mechanism and a positional encoding mechanism. It also uses a relative positional representation to model pairwise relationships between source code tags, thereby improving its ability to capture long-term dependencies and non-temporal dependencies of source code tagging ability. The comment-update component of CSL2CU is used to generate new comments based on old comments and code editing. The experimental results show that the CSL2CU model outperforms the three baselines used in exact match, BLEU, METEOR, and SARI.

“When you can measure what you are speaking about and express it in numbers, you know something about it; but when you cannot measure, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science.” LORD WILLIAM KELVIN (1824 – 1907). During the last decade, the quantification of software engineering process has got a pace due to availability of a huge amount of software repositories. These repositories include source code, bug, communication among developers/users, changes in code, etc. Researchers are trying to find out useful information from these repositories for improving the quality of software.The absence of bugs in the software is a major factor that decides the quality of software. In the available literature, researchers have proposed and implemented a plethora of bug prediction approaches varying in terms of accuracy, complexity and input data. The code change metric based bug prediction is proven to be very useful. In the literature, decay functions have been proposed that decay the complexity of code changes over a period of time in either exponential or linear fashion but they do not fit in open source software development paradigm because in open source software development paradigm, the development team is geographical dispersed and there is an irregular fluctuation in the code changes and bug detection/fixing process. The complexity of code changes reduces over a period of time that may be less than exponential or more than linear. This paper presents the method that quantifies the code change metric and also proposed decay functions that capture the variability in the decay curves represented the complexity of code changes. The proposed decay functions model the complexity of code changes which reduces over a period of time and follows different types of decay curves. We have collected the source code change data of Mozilla components and applied simple linear regression (SLR) and support vector regression (SVR) techniques to validate the proposed method and predict the bugs yet to come in future based on the current year complexity of code changes (entropy). The performance of proposed models has been compared using different performance criteria namely R2, Adjusted R2, Variation and Root Mean Squared Prediction Error (RMSPE).

Identifying bug immediately when it is introduced can help improve the validity and effectiveness of bug fixing. Predicting bugs in software code changes makes such identification possible. Buggy changes, changes that introduce bugs into source code, can be viewed as anomalies relative to clean changes for that they are rare and irregular. Thus, anomaly detection techniques can be applied to buggy change prediction. Isolation Forest, which detects anomalies based on the hypothesis that the anomalies have the shortest average path length on the constructed random forest, has exhibited its good performance on anomaly detection compared to other anomaly detection methods. In this paper, we adopt it in predicting bugs in software code changes. Empirical study with eight practical open source projects are conducted to validate the effective of Isolation Forest in bug prediction in software code changes. Results of the empirical study show that compared to traditional classification methods used in literature, Isolation Forest can achieve better clean precision, buggy recall, buggy F-measure, AUC and Gmean.

Currently, developers are often obligated to enhance code quality. High-quality code is often accompanied with comprehensive summaries, including code documentation and function explanations, which are invaluable for maintenance and further development. Regrettably, few software projects provide sufficient code comments owing to the high costs associated with human labeling. Contemporary researchers in software engineering concentrate on the methods for automated comment generating. Initial algorithms depended on handwritten templates or information retrieval methods. With the advancement of machine learning, researchers construct automated models for machine translation instead. Nonetheless, the produced code comments remain inadequate owing to the significant disparity between code structure and normal language. This study introduces a unique deep learning model, At-ComGen, which utilizes hybrid attention for the automated creation of source code comments. Utilizing two separate LSTM encoders, our approach integrates essential tokens from source code functions with the code structure, represented by a corresponding Abstract Syntax Tree. In contrast to earlier data-driven models, our methodology utilizes code syntax and semantics in the generation of comments. The hybrid attention method, used for comment creation for the first time to our knowledge, enhances the quality of code comments. The tests demonstrate that At-ComGen is efficacious and surpasses other prevalent methodologies. Machine comments from Seq2Seq and CODE-NN disregard code structure underlying DeepCom and At-ComGen. At-ComGen has 59.3%, 36.4%, 43.3%, and 13.1% higher comment BLEU values than baseline models for a 5-line function. Even though model performance reduces with comment length, At-ComGen's comments often outperform others. 5–10-word machine comments work best. For reference length 10, At-ComGen has 38.2%, 23.7%, 9.3%, and 4.4% greater BLEU values than the other baseline models.

Code comments are one of the effective means for assisting programmers to understand the source code. High-quality code comments play an important role in areas such as software maintenance and software reuse. Good code comments can help programmers understand the role of source code and facilitate comprehension of programs and software maintenance tasks quickly. But in reality, most programmers only pay attention to the code and ignore comments and documents, which greatly reduce the program's readability and maintainability. This paper has compiled the relevant research on code comments so far, mainly including four aspects: automatic generation of code comments, consistency of code comments, classification of code comments, and quality evaluation of code comments. By analyzing relevant methods in this research field, it provides more complete information for future research.

In the era of Big Code, when researchers seek to study an increasingly large number of repositories to support their findings, the data processing stage may require manipulating millions and more of records. In this work we focus on studies involving fine-grained AST level source code changes. We present how we extended the CodeDistillery source code mining framework with data manipulation capabilities, aimed to alleviate the processing of large datasets of fine grained source code changes. The capabilities we have introduced allow researchers to highly automate their repository mining process and streamline the data acquisition and processing phases. These capabilities have been successfully used to conduct a number of studies, in the course of which dozens of millions of fine-grained source code changes have been processed.

Just-in-time comment update is a promising way to reduce the burden of developers during software maintenance and evolution. Existing approaches can be divided into two categories: the heuristic-based approach and the deep-learning-based approach. The heuristic-based approach is restricted to a specific type of comment updates (i.e., code-indicative updates), but performs well on such type. The effectiveness of deep-learning-based approach is limited but it can handle diverse comment updates. Considering the complementary advantages of existing approaches, an intuitive idea is to combine them for better performance. To investigate this idea, we first conduct a pre-study experiment which shows that to construct an effective comment updater by combining heuristic-based and deep-learning-based approaches, we need to tackle two main challenges: 1) the heuristic-based approach may bring side effects to cases which cannot be updated by it; and 2) the current deep-learning-based approach is with limited effectiveness. Then, we propose a novel two-phase approach named Toper to cope with these two challenges and effectively perform comment updates. In the first phase, Toper integrates nine distinctive features identified through our large-scale empirical analysis into a predictive model, which can predict whether the contents of the comment updates can be found in the corresponding code changes, namely, the comment updates are code-indicative updates. If so, the updates are then generated by an off-the-shelf heuristic-based approach; otherwise, Toper leverages a deep learning model, which we specially designed for non-code-indicative updates, to infer the new comment based on the old comment and code change. Motivated by our manual observation on the limitation of existing approaches on non-code-indicative updates, our deep learning model adopts the Abstract Syntax Tree path technique, which can capture the program structure information for effectively embedding code changes. Our evaluation shows that our approach outperforms the state-of-the-art by around 20% with respect to the number of correct comments it generates. Via in-depth analysis, we illustrate the rationale of each design decision as well as point out potential directions.

A software system undergoes many code changes, which may affect code quality and maintainability. Refactoring techniques are a common strategy for restructuring source code and improving internal architecture. However, refactoring activities are limited to a catalog of well-known modifications, and we intend to investigate modifications to the source code, looking for patterns that affect its internal quality. In this work, we present a process for identifying code improvements and then learning from the modifications. This process extracts syntactic patterns of code changes and classifies them based on their impact on software maintainability. We executed the classification process on 132,819 code changes from seven GitHub repositories, obtaining 2405 significant code change patterns. Most code change patterns involve simple modifications to structural elements that have only a slight impact on maintainability. These modifications include renaming or moving code elements, simple refactoring, and adding a new implementation. We analyzed both simple modification patterns and many modifications to identify their impact on internal source code quality. The results show how significant a subset of identified patterns is to internal source code quality, as it improves or reduces maintainability variation.

Regression test prioritization (RTP) is an active research field, aiming at re‐ordering the tests in a test suite to maximize the rate at which faults are detected. A number of RTP strategies have been proposed, leveraging different factors to reorder tests. Some techniques include an analysis of changed source code, to assign higher priority to tests stressing modified parts of the codebase. Still, most of these change‐based solutions focus on simple text‐level comparisons among versions. We believe that measuring source code changes in a more refined way, capable of discriminating between mere textual changes (e.g., renaming of a local variable) and more structural changes (e.g., changes in the control flow), could lead to significant benefits in RTP, under the assumption that major structural changes are also more likely to introduce faults. To this end, we propose two novel RTP techniques that leverage tree kernels (TK), a class of similarity functions largely used in Natural Language Processing on tree‐structured data. In particular, we apply TKs to abstract syntax trees of source code, to more precisely quantify the extent of structural changes in the source code, and prioritize tests accordingly. We assessed the effectiveness of the proposals by conducting an empirical study on five real‐world Java projects, also used in a number of RTP‐related papers. We automatically generated, for each considered pair of software versions (i.e., old version, new version) in the evolution of the involved projects, 100 variations with artificially injected faults, leading to over 5k different software evolution scenarios overall. We compared the proposed prioritization approaches against well‐known prioritization techniques, evaluating both their effectiveness and their execution times. Our findings show that leveraging more refined code change analysis techniques to quantify the extent of changes in source code can lead to relevant improvements in prioritization effectiveness, while typically introducing negligible overheads due to their execution.

Commit messages, which summarize the source code changes in natural language, are essential for program comprehension and software evolution understanding. Unfortunately, due to the lack of direct motivation, commit messages are sometimes neglected by developers, making it necessary to automatically generate such messages. State-of-the-art adopts learning based approaches such as neural machine translation models for the commit message generation problem. However, they tend to ignore the code structure information and suffer from the out-of-vocabulary issue. In this paper, we propose CoDiSum to address the above two limitations. In particular, we first extract both code structure and code semantics from the source code changes, and then jointly model these two sources of information so as to better learn the representations of the code changes. Moreover, we augment the model with copying mechanism to further mitigate the out-of-vocabulary issue. Experimental evaluations on real data demonstrate that the proposed approach significantly outperforms the state-of-the-art in terms of accurately generating the commit messages.

Code comments can help in program comprehension and are considered as important artifacts to help developers in software maintenance. However, the comments are mostly missing or are outdated, specially in complex software projects. As a result, several automatic comment generation models are developed as a solution. The recent models explore the integration of external knowledge resources such as Unified Modeling Language class diagrams to improve the generated comments. In this paper, we propose API2Com, a model that leverages the Application Programming Interface Documentations (API Docs) as a knowledge resource for comment generation. The API Docs include the description of the methods in more details and therefore, can provide better context in the generated comments. The API Docs are used along with the code snippets and Abstract Syntax Trees in our model. We apply the model on a large Java dataset of over 130,000 methods and evaluate it using both Transformer and RNN-base architectures. Interestingly, when API Docs are used, the performance increase is negligible. We therefore run different experiments to reason about the results. For methods that only contain one API, adding API Docs improves the results by 4% BLEU score on average (BLEU score is an automatic evaluation metric used in machine translation). However, as the number of APIs that are used in a method increases, the performance of the model in generating comments decreases due to long documentations used in the input. Our results confirm that the API Docs can be useful in generating better comments, but, new techniques are required to identify the most informative ones in a method rather than using all documentations simultaneously.

Many software processes advocate that the test code should co-evolve with the production code. Prior work usually studies such co-evolution based on production-test co-evolution samples mined from software repositories. A production-test co-evolution sample refers to a pair of a test code change and a production code change where the test code change triggers or is triggered by the production code change. The quality of the mined samples is critical to the reliability of research conclusions. Existing studies mined production-test co-evolution samples based on the following assumption: if a test class and its associated production class change together in one commit, or a test class changes immediately after the changes of the associated production class within a short time interval, this change pair should be a production-test co-evolution sample . However, the validity of this assumption has never been investigated. To fill this gap, we present an empirical study, investigating the reasons for test code updates occurring after the associated production code changes, and revealing the pervasive existence of noise in the production-test co-evolution samples identified based on the aforementioned assumption by existing works. We define a taxonomy of such noise, including six categories (i.e., adaptive maintenance, perfective maintenance, corrective maintenance, indirectly related production code update, indirectly related test code update, and other reasons). Guided by the empirical findings, we propose CHOSEN (an identifi C ation met H od O f production-te S t co- E volutio N ) based on a two-stage strategy. CHOSEN takes a test code change and its associated production code change as input, aiming to determine whether the production-test change pair is a production-test co-evolution sample. Such identified samples are the basis of or are useful for various downstream tasks. We conduct a series of experiments to evaluate our method. Results show that (1) CHOSEN achieves an AUC of 0.931 and an F1-score of 0.928, significantly outperforming existing identification methods, and (2) CHOSEN can help researchers and practitioners draw more accurate conclusions on studies related to the co-evolution of production and test code. For the task of Just-In-Time (JIT) obsolete test code detection, which can help detect whether a piece of test code should be updated when developers modify the production code, the test set constructed by CHOSEN can help measure the detection method’s performance more accurately, only leading to 0.76% of average error compared with ground truth. In addition, the dataset constructed by CHOSEN can be used to train a better obsolete test code detection model, of which the average improvements on accuracy, precision, recall, and F1-score are 12.00%, 17.35%, 8.75%, and 13.50% respectively.

Research Question: The objective of this article is to present the model for simulating the financial viability of a Just-in-Time Maintenance Program in an agribusiness company. Motivation: Maintenance management in agro-industrial environments lacks effective systems to optimize the production process. The existing maintenance program have proven to be ineffective, from the point of view that unscheduled production downtime continues to occur. It is important to highlight that the types of maintenance used today, considering that their characteristics are not able to reduce or minimize the errors caused by human limited capacity, which has contributed to the increase of unscheduled production stoppages, causing equipment unavailability, low quality of final products and, mainly, failing to deliver products as agreed with customers. Idea: The maintenance program called Just-In-Time Maintenance Program - Maintenance Just in Time – consists of an integrated system for monitoring physical phenomena, with sensors installed in the equipment, connected to a central diagnosis able to predict failures in real time. With this program, it is possible to know in advance what may happen, taking the necessary measures regarding the acquisition of parts, repairs, and personnel allocation, among others, making these stops compatible or readjusting them with the production plan to minimize the undesirable stops. Data: The choice of the company participating in the research is made due to its continuous production process. The present study used a non-probabilistic sample for convenience with that provided by the observed company. The information collected in the agribusiness regarding the selling price, cost of raw material, cost of electricity, and total expenditure on maintenance, refer to the year 2020. Tools: A mathematical model was developed that estimates variations in the company's productivity and the consequent financial impacts. The results, operational and financial, of the model allow assessing the feasibility of implementing each of these programs. Findings: The results obtained indicate losses with the current maintenance program and show the potential for implementing the Just-In-Time Maintenance Program. This research concludes that it is possible to reduce unscheduled stoppages that cause production interruptions, increasing profit, with the implementation of the Just-In-Time Maintenance Program. Contribution: This article contributes to the literature as it presents an equipment maintenance model based on operational conditions, in contrast to other existing maintenance programs.

Source code comments are a valuable instrument to preserve design decisions and to communicate the intent of the code to programmers and maintainers. Nevertheless, commenting source code and keeping comments up-to-date is often neglected for reasons of time or programmers obliviousness. In this paper, we investigate the question whether developers comment their code and to what extent they add comments or adapt them when they evolve the code. We present an approach to associate comments with source code entities to track their co-evolution over multiple versions. A set of heuristics are used to decide whether a comment is associated with its preceding or its succeeding source code entity. We analyzed the co-evolution of code and comments in eight different open source and closed source software systems. We found with statistical significance that (1) the relative amount of comments and source code grows at about the same rate; (2) the type of a source code entity, such as a method declaration or an if-statement, has a significant influence on whether or not it gets commented; (3) in six out of the eight systems, code and comments co-evolve in 90% of the cases; and (4) surprisingly, API changes and comments do not co-evolve but they are re-documented in a later revision. As a result, our approach enables a quantitative assessment of the commenting process in a software system. We can, therefore, leverage the results to provide feedback during development to increase the awareness of when to add comments or when to adapt comments because of source code changes.

Augmenting commit classification by using fine-grained source code changes and a pre-trained deep neural language model