Hybrid Attention Approach for Source Code Comment Generation

Code comments are one of the effective means for assisting programmers to understand the source code. High-quality code comments play an important role in areas such as software maintenance and software reuse. Good code comments can help programmers understand the role of source code and facilitate comprehension of programs and software maintenance tasks quickly. But in reality, most programmers only pay attention to the code and ignore comments and documents, which greatly reduce the program's readability and maintainability. This paper has compiled the relevant research on code comments so far, mainly including four aspects: automatic generation of code comments, consistency of code comments, classification of code comments, and quality evaluation of code comments. By analyzing relevant methods in this research field, it provides more complete information for future research.

Code comments are valuable for program comprehension and software maintenance. However, comments can be inconsistent or out-of-date after code changes. To tackle this problem, Just-In-Time (JIT) comment updating aims to automatically update comments with code changes. Existing approaches for this task use edit sequences of source code to model code changes. Meanwhile, recent researches indicate that neural models based on abstract syntax trees (AST) can help represent source code. In this paper, we propose a new method to learn code changes by combining code edit sequences with AST edit sequences, so that the generated new comment can be more accurate. Our approach utilizes three encoders to encode code edit sequences, AST edit sequences and old comment token sequences, respectively. The outputs of the encoders are then decoded into a sequence of edit actions, which is parsed to generate a new comment. The proposed method is evaluated on a public dataset using seven metrics, and the experimental results show that our approach outperforms the baselines. Furthermore, when the new comment has a larger edit distance than the old one, our model shows better performance.

Software Quality Assurance (SQA) is essential in software development and many defect prediction methods based on machine learning have been proposed to identify defective modules. However, most existing defect prediction models do not provide good defect prediction results, and the semantic features reflecting the detective patterns may not be well-captured via traditional feature extraction methods. More information such as code comments should be also be embedded to generate semantic features respecting the source code functionality. Therefore, how to embed code comments for defect prediction is a big challenge, and another problem is that many comments of source code are missing in real-world applications. In this paper, we propose a novel defect prediction model named CAP-CNN (Convolutional Neural Network for Comments Augmented Programs), which is a deep learning model that automatically embeds code comments in generating semantic features from the source code for software defect prediction. To overcome the missing comments problem, a novel training strategy is used in CAP-CNN that the network encodes and absorb comments information to generate semantic features automatically during training process, which does not need testing modules to contain comments. Experimental results on several widely-used software data sets indicate that the comment features are able to improve defect prediction performance.

Technical debt is a metaphor to reflect the tradeoff software engineers make between short-term benefits and long-term stability. Self-admitted technical debt (SATD), a variant of technical debt, has been proposed to identify debt that is intentionally introduced during software development, e.g., temporary fixes and workarounds. Previous studies have leveraged human-summarized patterns (which represent n-gram phrases that can be used to identify SATD) or text-mining techniques to detect SATD in source code comments. However, several characteristics of SATD features in code comments, such as vocabulary diversity, project uniqueness, length, and semantic variations, pose a big challenge to the accuracy of pattern or traditional text-mining-based SATD detection, especially for cross-project deployment. Furthermore, although traditional text-mining-based method outperforms pattern-based method in prediction accuracy, the text features it uses are less intuitive than human-summarized patterns, which makes the prediction results hard to explain. To improve the accuracy of SATD prediction, especially for cross-project prediction, we propose a Convolutional Neural Network-- (CNN) based approach for classifying code comments as SATD or non-SATD. To improve the explainability of our model’s prediction results, we exploit the computational structure of CNNs to identify key phrases and patterns in code comments that are most relevant to SATD. We have conducted an extensive set of experiments with 62,566 code comments from 10 open-source projects and a user study with 150 comments of another three projects. Our evaluation confirms the effectiveness of different aspects of our approach and its superior performance, generalizability, adaptability, and explainability over current state-of-the-art traditional text-mining-based methods for SATD classification.

Code comments are a key software component for program comprehension and software maintainability. High-quality code and comments are urgently needed by data-driven models widely used in tasks like code summarization. Many existing approaches for assessing the quality of comments are machine learning based classification algorithms or rely on heuristic rules. These approaches are difficult to capture the complicated features of text data and are often limited in accuracy, efficiency, and generalization ability. In this paper, we convert the quality assessment of code comments into a classification problem based on the multi-input neural network. We summarize the input, the code and comments, into vectors using the attention-based Bi-LSTM model and the weighted GloVe model, respectively, and concatenate the code vectors and the comment vectors as the input of the Multiple-Layer Perceptron classifier for the comment quality assessment. Experimental results show that our approach, in general, outperforms the previous technique, on both our labeled dataset and the public dataset, with the F1-score of 96.91% and 91.90%, respectively. Using the training set and the testing set from distinct sources, our approach can still achieve reasonable performance, which demonstrates its generalization ability.

Assessing the similarity between code components plays a pivotal role in a number of Software Engineering (SE) tasks, such as clone detection, impact analysis, refactoring, etc. Code similarity is generally measured by relying on manually defined or hand-crafted features, e.g., by analyzing the overlap among identifiers or comparing the Abstract Syntax Trees of two code components. These features represent a best guess at what SE researchers can utilize to exploit and reliably assess code similarity for a given task. Recent work has shown, when using a stream of identifiers to represent the code, that Deep Learning (DL) can effectively replace manual feature engineering for the task of clone detection. However, source code can be represented at different levels of abstraction: identifiers, Abstract Syntax Trees, Control Flow Graphs, and Bytecode. We conjecture that each code representation can provide a different, yet orthogonal view of the same code fragment, thus, enabling a more reliable detection of similarities in code. In this paper, we demonstrate how SE tasks can benefit from a DL-based approach, which can automatically learn code similarities from different representations.

Code comments can help in program comprehension and are considered as important artifacts to help developers in software maintenance. However, the comments are mostly missing or are outdated, specially in complex software projects. As a result, several automatic comment generation models are developed as a solution. The recent models explore the integration of external knowledge resources such as Unified Modeling Language class diagrams to improve the generated comments. In this paper, we propose API2Com, a model that leverages the Application Programming Interface Documentations (API Docs) as a knowledge resource for comment generation. The API Docs include the description of the methods in more details and therefore, can provide better context in the generated comments. The API Docs are used along with the code snippets and Abstract Syntax Trees in our model. We apply the model on a large Java dataset of over 130,000 methods and evaluate it using both Transformer and RNN-base architectures. Interestingly, when API Docs are used, the performance increase is negligible. We therefore run different experiments to reason about the results. For methods that only contain one API, adding API Docs improves the results by 4% BLEU score on average (BLEU score is an automatic evaluation metric used in machine translation). However, as the number of APIs that are used in a method increases, the performance of the model in generating comments decreases due to long documentations used in the input. Our results confirm that the API Docs can be useful in generating better comments, but, new techniques are required to identify the most informative ones in a method rather than using all documentations simultaneously.

Internet code search is quite popular research area. StackOverflow allows developers to ask and answer questions about code. Previous approach to search code on StackOverflow uses tf-idf method that based on number of occurrences of words to recommend source code. This method has the disadvantage that variable or method identifiers are considered as normal words, even though identifiers are often a combination of two or more words. For example, there is an identifier named “randomString”. In that case, if we search using a keyword “random” the system probably will not recommend “randomString” because both words are different. Concept location can tackle this problem. Concept location has been used widely to obtain the correlation between code with a specific concepts or features. Previous research of concept location only focused on source code's comments, and relation among the objects within the source code. This research proposes a mechanism for finding code on StackOverflow uses Latent Dirichlet Allocation (LDA) using concept location in the preprocessing stage. Questions, answers, and code snippets about Java programming are downloaded from StackOverflow to a local repository. Corpuses are generated by extracting questions, answers and code snippets. Inferencing concept location from source code is created using LDA algorithm. Developers query concepts and then system will recommend source code based on the relevant concepts. The result of the experiment shows that the system is able to recommend source code with 48% average of precision and 58% average of recall.

Code comments are essential for programming comprehension. Nevertheless, developers often neglect to update comments after modifying the source code. Wrong code comments may lead to bugs in the maintenance process, thus affecting the reliability of the software. So, timely comment mismatch detection is crucial for software development and maintenance. However, existing works have the following two limitations: 1) the lack of use of code structural and sequential information, and 2) the ignorance of existing associations between code and comments. In this paper, we propose a new model called ASKDetector (AST-Semantic and Key features fusion based mismatch Detector). For the first limitation, we encode code with an attention-based preorder traversal abstract syntax tree sequence to obtain both order and structural information. And CodeBERT is utilized to capture contextual semantic features further. For the second one, we encode extracted association information between the code snippets and comments to reduce the semantic gap. The correlations between the encoders are learned through a fusion layer and a multi-layer perceptron. The experimental results prove that our detector outperforms the state-of-the-art model in evaluation metrics, where our F1 and accuracy exceed an average of 3.4%.

Existing research suggests that a considerable fraction (5-10%) of the source code of large scale computer programs is duplicate code (clones). Detection and removal of such clones promises decreased software maintenance costs of possibly the same magnitude. Previous work was limited to detection of either near misses differing only in single lexems, or near misses only between complete functions. The paper presents simple and practical methods for detecting exact and near miss clones over arbitrary program fragments in program source code by using abstract syntax trees. Previous work also did not suggest practical means for removing detected clones. Since our methods operate in terms of the program structure, clones could be removed by mechanical methods producing in-lined procedures or standard preprocessor macros. A tool using these techniques is applied to a C production software system of some 400 K source lines, and the results confirm detected levels of duplication found by previous work. The tool produces macro bodies needed for clone removal, and macro invocations to replace the clones. The tool uses a variation of the well known compiler method for detecting common sub expressions. This method determines exact tree matches; a number of adjustments are needed to detect equivalent statement sequences, commutative operands, and nearly exact matches. We additionally suggest that clone detection could also be useful in producing more structured code, and in reverse engineering to discover domain concepts and their implementations.

Disfluencies in spoken language remain a challenge to both language processing applications and human perception. Disfluency identification and removal are therefore beneficial steps to improve performance of spoken language understanding tasks. In particular, it is important for close captioning video conferences. We investigated different approaches for disfluency identification and removal, ranging from rule-based, translation models and supervised classification using either Conditional Random Fields (CRF) or Deep Neural Networks (DNN). As supervised classifier in our task requires huge amount of human annotation and labeling, the rule-based approach and translation model allow us to use less human labeling than supervised classification. In the rule-based approach, we used regular expressions for matching simple disfluent words and obtained 81.01 in the Bilingual Evaluation Understudy (BLEU) score, a way to scale the reconstruction quality. Secondly, we used a Weighted Finite State Transducer (WFST) to translate phrases that have complex disfluencies into more fluent ones and obtained 72.02 score in BLEU measure. We applied for the first time DNN with the same feature set as a CRF baseline system and obtained better performance for one disfluency type, repeat, with precision at 81.7% and recall 82.0%. Furthermore, the harmonic mean F1 score for overall weighted disfluencies was improved by 3.7% when we used CRF system with additional word vector feature trained by a neural network. We constructed a DNN-CRF hybrid system by using voting algorithm. For the disfluency type, false start, our F1 result is 52.2%. This is significant improvement over a CRF system’s baseline 36.7%. It has been found that BLEU score is improved from 81.01 to 82.80 by elaborately combining all the three approaches.

During software development and maintenance, code comments are often missing, inadequate, or they do not match the actual code content. In response to this problem, the research community has proposed a method for updating natural language comments based on code changes. However, there are two major limitations of this method that must be addressed: the long-term and non-temporal dependencies in the source code. To address these limitations, we propose a new model called code semantic learning–comment update (CSL2CU). The code semantic learning component of CSL2CU uses a self-attention mechanism and a positional encoding mechanism. It also uses a relative positional representation to model pairwise relationships between source code tags, thereby improving its ability to capture long-term dependencies and non-temporal dependencies of source code tagging ability. The comment-update component of CSL2CU is used to generate new comments based on old comments and code editing. The experimental results show that the CSL2CU model outperforms the three baselines used in exact match, BLEU, METEOR, and SARI.

Context: Software development teams constantly opt for faster, lower quality solutions to solve current problems without planning for the future. This situation will have a negative long-term impact and is called technical debt. Similar to a financial debt, technical debts require interest payments and must be managed and detected so that the team can evaluate the best way to deal with them. One way to detect technical debts is through classification of source code comments. Developers often insert comments warning of the need to improve their own code in the future. This is known as Self-Admitted Technical Debt (SATD). Objective: Combine Word2vec for word embedding with a Long short-term memory (LSTM) neural network model to identify SATDs from comments in source code and compare with other studies and LSTM without word embedding. Method: We plan and execute an experimental process with model’s effectiveness data validation. Results: In general, the classification improves when all SATD types were grouped in a single label. In relation to other studies, the LSTM model with Word2vec achieved better recall and f-measure. The LSTM model without word embedding achieves greater recall, but perform worse in precision and f-measure. Conclusion: We found evidence that LSTM models combined with word embedding are promising for the development a more effective SATD classifier.

Modern software developers tend to engage in social coding platforms to reuse code snippets to expedite the development process, while the codes on such platforms are often suffering from comments being mismatched, missing or outdated. This puts the code search and comprehension in difficulty, and increases the burden of maintenance for software building upon these codes. As summarizing code is beneficial yet it is very expensive for manual operation, in this paper, we elaborate an automatic and effective code summarization paradigm to address this laborious challenge. We represent a given code snippet as an abstract syntax tree (AST), and generate a set of compositional root-to-leaf paths to make the AST accessible regarding code context and structure in a less complex yet expressive way. Accordingly, we design a tree-based transformer model, called TreeXFMR, on these paths to summarize source code in a hierarchical attention operation. This yields two advantages on code representation learning: (1) attention mechanisms at token-and path-level attend the semantics and interactions of source code from different aspects; (2) bi-level positional encodings introduced reveal the intra- and inter-path structure of AST and improve the unambiguity of the representations. During decoding, TreeXFMR attends such learned representations to produce each output of natural language word. We further pre-train the transformer to achieve faster and better training convergence results. Extensive experiments on the code collection from GitHub demonstrate the effectiveness of TreeXFMR, which significantly outperforms state-of-the-art baselines.

Plagiarism source code is a program that is made up of other programs that have same syntax structure. In this research, the approach used to detect plagiarism is tree-based by building abstract syntax tree base on grammar on two predefined plagiarism files source code. Damerau-Levenshtein Distance Algorithm will calculate the tree structure formed minimum distance value to get the percentage of similarity. Previously, the application calculated the value of threshold obtained from the average value of plagiarism plot paired pairs, and then were reduced to its standard deviation to be able to declare that both files are plagiarism or not. This research analyzes the best use of grammar between jexer rule or a combination of lexer and parser rule, the best use of preprocessing combination and the best use of distance value of Damerau-Levenshtein Distance Algorithm. Based on the tests performed, the use of grammar lexer and parser rule resulted the highest accuracy of 97.435 % by taking 118,115 seconds and threshold used is 88.2314 %.The combination of preprocessing resulted highest accuracy of 97.435% by using whole preprocessing existing or by using preprocessing comment only. For the best distance value is 4 with highest accuracy 97.435 %.Plagiarisme source code adalah jika sebagai sebuah program yang terbentuk dari program lainnya dan memiliki struktur syntax yang sama. Dalam penelitian ini, pendekatan yang digunakan untuk mendeteksi plagiarisme adalah tree-based dengan membangun abstract syntax tree atas dua berkas source code terduga plagiat berdasarkan grammar yang telah dirancang. Struktur tree yang terbentuk akan dihitung nilai jarak minimumnya dengan Damerau-Levenshtein Distance Algorithm untuk mendapatkan persentase kemiripan. Sebelumnya, aplikasi menghitung nilai threshold yang didapatkan dari nilai rata-rata kemiripan pasangan berkas plagiat yang dikurangi dengan simpangan bakunya untuk dapat menyatakan kedua berkas masukan plagiat atau tidak. Penelitian ini menganalisis penggunaan grammar terbaik antara lexer rule atau kombinasi lexer dan parser rule, penggunaan kombinasi preprocessing terbaik serta penggunaan nilai jarak terbaik pada Damerau-Levenshtein Distance Algorithm. Berdasarkan pengujian yang dilakukan, penggunaan grammar lexer dan parser rule menghasilkan akurasi tertinggi yaitu 97.435 % dengan memakan waktu 118,115 detik dengan nilai threshold 88.2314 %. Kombinasi preprocessing yang menghasilkan akurasi tertinggi 97.435 % menggunakan seluruh preprocessing yang ada atau dengan menggunakan prerpocessing comment saja. Untuk nilai jarak terbaik adalah nilai jarak sebesar 4 dengan akurasi tertinggi, yaitu 97.435 %.