Deep Just-In-Time Consistent Comment Update via Source Code Changes

Code comments are valuable for program comprehension and software maintenance. However, comments can be inconsistent or out-of-date after code changes. To tackle this problem, Just-In-Time (JIT) comment updating aims to automatically update comments with code changes. Existing approaches for this task use edit sequences of source code to model code changes. Meanwhile, recent researches indicate that neural models based on abstract syntax trees (AST) can help represent source code. In this paper, we propose a new method to learn code changes by combining code edit sequences with AST edit sequences, so that the generated new comment can be more accurate. Our approach utilizes three encoders to encode code edit sequences, AST edit sequences and old comment token sequences, respectively. The outputs of the encoders are then decoded into a sequence of edit actions, which is parsed to generate a new comment. The proposed method is evaluated on a public dataset using seven metrics, and the experimental results show that our approach outperforms the baselines. Furthermore, when the new comment has a larger edit distance than the old one, our model shows better performance.

Code comments are one of the effective means for assisting programmers to understand the source code. High-quality code comments play an important role in areas such as software maintenance and software reuse. Good code comments can help programmers understand the role of source code and facilitate comprehension of programs and software maintenance tasks quickly. But in reality, most programmers only pay attention to the code and ignore comments and documents, which greatly reduce the program's readability and maintainability. This paper has compiled the relevant research on code comments so far, mainly including four aspects: automatic generation of code comments, consistency of code comments, classification of code comments, and quality evaluation of code comments. By analyzing relevant methods in this research field, it provides more complete information for future research.

Software Quality Assurance (SQA) is essential in software development and many defect prediction methods based on machine learning have been proposed to identify defective modules. However, most existing defect prediction models do not provide good defect prediction results, and the semantic features reflecting the detective patterns may not be well-captured via traditional feature extraction methods. More information such as code comments should be also be embedded to generate semantic features respecting the source code functionality. Therefore, how to embed code comments for defect prediction is a big challenge, and another problem is that many comments of source code are missing in real-world applications. In this paper, we propose a novel defect prediction model named CAP-CNN (Convolutional Neural Network for Comments Augmented Programs), which is a deep learning model that automatically embeds code comments in generating semantic features from the source code for software defect prediction. To overcome the missing comments problem, a novel training strategy is used in CAP-CNN that the network encodes and absorb comments information to generate semantic features automatically during training process, which does not need testing modules to contain comments. Experimental results on several widely-used software data sets indicate that the comment features are able to improve defect prediction performance.

Pengembangan perangkat lunak adalah tugas kompleks dan membutuhkan adaptasi untuk mengakomodasi kebutuhan pengguna. Untuk membuat konsep dan perubahan perangkat lunak, dalam pemeliharaan, sekarang telah dikembangkan lebih mudah dalam pengembangan perangkat lunak, pola model-view-controller, yang merupakan arsitektur yang dapat membantu memfasilitasi dalam pengembangan dan pemeliharaan perangkat luna. Hal ini, karena dalam arsitektur model tiga-lapis, yaitu: tampilan dan pengontrolan dalam pembangunan dilakukan secara independen, sehingga dapat memberikan dahan dalam pengembangan dan pemeliharaan. Selain itu, arsitektur ini juga dapat melihat hal-hal yang sederhana dan menarik bagi pengguna. Software sistem on-line test adalah perangkat lunak yang memerlukan interaksi dengan pengguna, dan pemeliharaan perangkat adaptif. Karena sistem ujian on-line memerlukan pengembangan perangkat lunak untuk mengakomodasi kebutuhan ini berkembang dengan cepat. Makalah ini untuk menganalisis Model-View-Controller dan mencoba pembangunan, untuk menerapkannya dalam pengembangan perangkat lunak sistem pengujian on-line.

In recent years, researchers have intensively investigated various topics in test prioritization, which aims to re-order tests to increase the rate of fault detection during regression testing. While the main research focus in test prioritization is on proposing novel prioritization techniques and evaluating on more and larger subject systems, little effort has been put on investigating the threats to validity in existing work on test prioritization. One main threat to validity is that existing work mainly evaluates prioritization techniques based on simple artificial changes on the source code and tests. For example, the changes in the source code usually include only seeded program faults, whereas the test suite is usually not augmented at all. On the contrary, in real-world software development, software systems usually undergo various changes on the source code and test suite augmentation. Therefore, it is not clear whether the conclusions drawn by existing work in test prioritization from the artificial changes are still valid for real-world software evolution. In this paper, we present the first empirical study to investigate this important threat to validity in test prioritization. We reimplemented 24 variant techniques of both the traditional and time-aware test prioritization, and investigated the impacts of software evolution on those techniques based on the version history of 8 real-world Java programs from GitHub. The results show that for both traditional and time-aware test prioritization, test suite augmentation significantly hampers their effectiveness, whereas source code changes alone do not influence their effectiveness much.

“When you can measure what you are speaking about and express it in numbers, you know something about it; but when you cannot measure, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science.” LORD WILLIAM KELVIN (1824 – 1907). During the last decade, the quantification of software engineering process has got a pace due to availability of a huge amount of software repositories. These repositories include source code, bug, communication among developers/users, changes in code, etc. Researchers are trying to find out useful information from these repositories for improving the quality of software.The absence of bugs in the software is a major factor that decides the quality of software. In the available literature, researchers have proposed and implemented a plethora of bug prediction approaches varying in terms of accuracy, complexity and input data. The code change metric based bug prediction is proven to be very useful. In the literature, decay functions have been proposed that decay the complexity of code changes over a period of time in either exponential or linear fashion but they do not fit in open source software development paradigm because in open source software development paradigm, the development team is geographical dispersed and there is an irregular fluctuation in the code changes and bug detection/fixing process. The complexity of code changes reduces over a period of time that may be less than exponential or more than linear. This paper presents the method that quantifies the code change metric and also proposed decay functions that capture the variability in the decay curves represented the complexity of code changes. The proposed decay functions model the complexity of code changes which reduces over a period of time and follows different types of decay curves. We have collected the source code change data of Mozilla components and applied simple linear regression (SLR) and support vector regression (SVR) techniques to validate the proposed method and predict the bugs yet to come in future based on the current year complexity of code changes (entropy). The performance of proposed models has been compared using different performance criteria namely R2, Adjusted R2, Variation and Root Mean Squared Prediction Error (RMSPE).

Currently, developers are often obligated to enhance code quality. High-quality code is often accompanied with comprehensive summaries, including code documentation and function explanations, which are invaluable for maintenance and further development. Regrettably, few software projects provide sufficient code comments owing to the high costs associated with human labeling. Contemporary researchers in software engineering concentrate on the methods for automated comment generating. Initial algorithms depended on handwritten templates or information retrieval methods. With the advancement of machine learning, researchers construct automated models for machine translation instead. Nonetheless, the produced code comments remain inadequate owing to the significant disparity between code structure and normal language. This study introduces a unique deep learning model, At-ComGen, which utilizes hybrid attention for the automated creation of source code comments. Utilizing two separate LSTM encoders, our approach integrates essential tokens from source code functions with the code structure, represented by a corresponding Abstract Syntax Tree. In contrast to earlier data-driven models, our methodology utilizes code syntax and semantics in the generation of comments. The hybrid attention method, used for comment creation for the first time to our knowledge, enhances the quality of code comments. The tests demonstrate that At-ComGen is efficacious and surpasses other prevalent methodologies. Machine comments from Seq2Seq and CODE-NN disregard code structure underlying DeepCom and At-ComGen. At-ComGen has 59.3%, 36.4%, 43.3%, and 13.1% higher comment BLEU values than baseline models for a 5-line function. Even though model performance reduces with comment length, At-ComGen's comments often outperform others. 5–10-word machine comments work best. For reference length 10, At-ComGen has 38.2%, 23.7%, 9.3%, and 4.4% greater BLEU values than the other baseline models.

In the era of Big Code, when researchers seek to study an increasingly large number of repositories to support their findings, the data processing stage may require manipulating millions and more of records. In this work we focus on studies involving fine-grained AST level source code changes. We present how we extended the CodeDistillery source code mining framework with data manipulation capabilities, aimed to alleviate the processing of large datasets of fine grained source code changes. The capabilities we have introduced allow researchers to highly automate their repository mining process and streamline the data acquisition and processing phases. These capabilities have been successfully used to conduct a number of studies, in the course of which dozens of millions of fine-grained source code changes have been processed.

Towards exact and inexact approximate matching of executable binaries

Unit testing represents a key activity in software development and maintenance. Test suites with high internal quality facilitate maintenance activities, such as code comprehension and regression testing. Several guidelines have been proposed to help developers write good test suites. Unfortunately, such rules are not always followed resulting in the presence of bad test code smells (or simply test smells). Test smells have been defined as poorly designed tests and their presence may negatively affect the maintainability of test suites and production code. Despite the many studies that address code smells in general, until now there has been no empirical evidence regarding test smells (i) distribution in software systems nor (ii) their impact on the maintainability of software systems. This paper fills this gap by presenting two empirical studies. The first study is an exploratory analysis of 18 software systems (two industrial and 16 open source) aimed at analyzing the distribution of test smells in source code. The second study, a controlled experiment involving twenty master students, is aimed at analyzing whether the presence of test smells affects the comprehension of source code during software maintenance. The results show that (i) test smells are widely spread throughout the software systems studied and (ii) most of the test smells have a strong negative impact on the comprehensibility of test suites and production code.

In software development, source code is repeatedly changed due to various reasons. Similar code changes are called change patterns. Identifying change patterns is useful to support software development in a variety of ways. For example, change patterns can be used to collect ingredients for code completion or automated program repair. Many research studies have proposed various techniques that detect change patterns. For example, Negara et al. proposed a technique that derives change patterns from the edit scripts. Negara's technique can detect fine-grained change patterns, but we consider that there is room to improve their technique. We found that Negara's technique occasionally generates change patterns from structurally-different changes, and we also uncovered that the reason why such change patterns are generated is that their technique performs text comparisons in matching changes. In this study, we propose a new change mining technique to detect change patterns only from structurally-identical changes by taking into account the structure of the abstract syntax trees. We implemented the proposed technique as a tool, TC2P, and we compared it with Negara's technique. As a result, we confirmed that TC2P was not only able to detect change patterns more adequately than the prior technique but also to detect change patterns that were not detected by the prior technique.

Love them or hate them, videos are a pervasive format for delivering online education at scale. They are especially popular for computer programming tutorials since videos convey expert narration alongside the dynamic effects of editing and running code. However, these screencast videos simply consist of raw pixels, so there is no way to interact with the code embedded inside of them. To expand the design space of learner interactions with programming videos, we developed Codemotion, a computer vision algorithm that automatically extracts source code and dynamic edits from existing videos. Codemotion segments a video into regions that likely contain code, performs OCR on those segments, recognizes source code, and merges together related code edits into contiguous intervals. We used Codemotion to build a novel video player and then elicited interaction design ideas from potential users by running an elicitation study with 10 students followed by four participatory design workshops with 12 additional students. Participants collectively generated ideas for 28 kinds of interactions such as inline code editing, code-based skimming, pop-up video search, and in-video coding exercises.

Source code comments are a valuable instrument to preserve design decisions and to communicate the intent of the code to programmers and maintainers. Nevertheless, commenting source code and keeping comments up-to-date is often neglected for reasons of time or programmers obliviousness. In this paper, we investigate the question whether developers comment their code and to what extent they add comments or adapt them when they evolve the code. We present an approach to associate comments with source code entities to track their co-evolution over multiple versions. A set of heuristics are used to decide whether a comment is associated with its preceding or its succeeding source code entity. We analyzed the co-evolution of code and comments in eight different open source and closed source software systems. We found with statistical significance that (1) the relative amount of comments and source code grows at about the same rate; (2) the type of a source code entity, such as a method declaration or an if-statement, has a significant influence on whether or not it gets commented; (3) in six out of the eight systems, code and comments co-evolve in 90% of the cases; and (4) surprisingly, API changes and comments do not co-evolve but they are re-documented in a later revision. As a result, our approach enables a quantitative assessment of the commenting process in a software system. We can, therefore, leverage the results to provide feedback during development to increase the awareness of when to add comments or when to adapt comments because of source code changes.

Code comments are essential for programming comprehension. Nevertheless, developers often neglect to update comments after modifying the source code. Wrong code comments may lead to bugs in the maintenance process, thus affecting the reliability of the software. So, timely comment mismatch detection is crucial for software development and maintenance. However, existing works have the following two limitations: 1) the lack of use of code structural and sequential information, and 2) the ignorance of existing associations between code and comments. In this paper, we propose a new model called ASKDetector (AST-Semantic and Key features fusion based mismatch Detector). For the first limitation, we encode code with an attention-based preorder traversal abstract syntax tree sequence to obtain both order and structural information. And CodeBERT is utilized to capture contextual semantic features further. For the second one, we encode extracted association information between the code snippets and comments to reduce the semantic gap. The correlations between the encoders are learned through a fusion layer and a multi-layer perceptron. The experimental results prove that our detector outperforms the state-of-the-art model in evaluation metrics, where our F1 and accuracy exceed an average of 3.4%.

Technical debt is a metaphor to reflect the tradeoff software engineers make between short-term benefits and long-term stability. Self-admitted technical debt (SATD), a variant of technical debt, has been proposed to identify debt that is intentionally introduced during software development, e.g., temporary fixes and workarounds. Previous studies have leveraged human-summarized patterns (which represent n-gram phrases that can be used to identify SATD) or text-mining techniques to detect SATD in source code comments. However, several characteristics of SATD features in code comments, such as vocabulary diversity, project uniqueness, length, and semantic variations, pose a big challenge to the accuracy of pattern or traditional text-mining-based SATD detection, especially for cross-project deployment. Furthermore, although traditional text-mining-based method outperforms pattern-based method in prediction accuracy, the text features it uses are less intuitive than human-summarized patterns, which makes the prediction results hard to explain. To improve the accuracy of SATD prediction, especially for cross-project prediction, we propose a Convolutional Neural Network-- (CNN) based approach for classifying code comments as SATD or non-SATD. To improve the explainability of our model’s prediction results, we exploit the computational structure of CNNs to identify key phrases and patterns in code comments that are most relevant to SATD. We have conducted an extensive set of experiments with 62,566 code comments from 10 open-source projects and a user study with 150 comments of another three projects. Our evaluation confirms the effectiveness of different aspects of our approach and its superior performance, generalizability, adaptability, and explainability over current state-of-the-art traditional text-mining-based methods for SATD classification.