On Precision of Code Clone Detection Tools

This paper presents a novel code clone search technique that is accurate, incremental, and scalable to hundreds of million lines of code. Our technique incorporates multiple code representations (i.e., a technique to transform code into various representations to capture different types of clones), query reduction (i.e., a technique to select clone search keywords based on their uniqueness), and a customised ranking function (i.e., a technique to allow a specific clone type to be ranked on top of the search results) to improve clone search performance. We implemented the technique in a clone search tool, called Siamese, and evaluated its search accuracy and scalability on three established clone data sets. Siamese offers the highest mean average precision of 95% and 99% on two clone benchmarks compared to seven state-of-the-art clone detection tools, and reported the largest number of Type-3 clones compared to three other code search engines. Siamese is scalable and can return cloned code snippets within 8 seconds for a code corpus of 365 million lines of code. Using an index of 130,719 GitHub projects, we demonstrate that Siamese’s incremental indexing capability dramatically decreases the index preparation time for large-scale data sets with multiple releases of software projects. The paper discusses the applications of Siamese to facilitate software development and research with two use cases including online code clone detection and clone search with automated license analysis.

A code clone is a code fragment identical or similar to another code fragment in source code. Some of code clones are considered as a factor of bug replications and make it more difficult to maintain software. Various code clone detection tools have been proposed so far. However, in most algorithms adopted by existing clone detection tools, if program statements are reordered, they are not detected as code clones. In this research, we examined how clone detection results change by rearranging the order of program statements. We performed preprocessing to rearranging the order of program statements using program dependency graph (PDG). We compared clone detection results with and without preprocessing. As a result, by rearranging the order of program statements, the number of detected code clones is almost the same in most projects. We classified newly detected or disappeared clones manually. From our experimental results, we show that there is no newly detected clone whose statements are reordered and that there are four disappeared clones whose statements are reordered. We think three out of the four clones occurred by copy-and-paste operations. Therefore, we conclude that rearranging the order of program statements is not effective to detect reordered code clones.

Investigating the code fragments of code clones detected by code clone detection tools is a time-consuming task, especially when a large number of reference source files are available. This paper proposes (i) a method for clustering a clone class, which is detected by code clone detection tools using syntactic similarity, based on topic similarity by considering its code fragments as sequences of words and (ii) a method for assigning short tags to clusters of the clustering result. We also report an experiment of applying the proposed method to packages of an open source operating system.

Context: Code recommendation plays an important role in improving programming efficiency and software quality. Existing approaches mainly focus on method- or API-level recommendations, which limits their effectiveness to local code contexts. From a multi-stage recommendation perspective, class-level code recommendation aims to efficiently narrow a large candidate code space while preserving essential structural information. Objective: This paper proposes RioCC, a class-level code recommendation framework that leverages deep forest-based code clone detection to progressively reduce the candidate space and improve recommendation efficiency in large-scale code spaces. Method: RioCC models the recommendation process as a coarse-to-fine candidate reduction procedure. In the coarse-grained stage, a quick search-based filtering module performs rapid candidate screening and initial similarity estimation, effectively pruning irrelevant candidates and narrowing the search space. In the fine-grained stage, a deep forest-based analysis with cascade learning and multi-grained scanning captures context- and structure-aware representations of class-level code fragments, enabling accurate similarity assessment and recommendation. This two-stage design explicitly separates coarse candidate filtering from detailed semantic matching to balance efficiency and accuracy. Results: Experiments on a large-scale dataset containing 192,000 clone pairs from BigCloneBench and a collected code pool show that RioCC consistently outperforms state-of-the-art methods, including CCLearner, Oreo, and RSharer, across four types of code clones, while significantly accelerating the recommendation process with comparable detection accuracy. Conclusions: By explicitly formulating class-level code recommendation as a staged retrieval and refinement problem, RioCC provides an efficient and scalable solution for large-scale code recommendation and demonstrates the practical value of integrating lightweight filtering with deep forest-based learning.

Code segments are similar or identical to each other by copy-paste the original code at different locations with or without modifications are called code clones. Code clones are either by intentional or unintentional. In software development reprocess, the original code is most common. If any bug exists in original code segment, the bug remains in all the cloned code. As a result, bug propagation in cloning process affects in software maintenance. Detection and corrections are required. Code clone detection appears as an ongoing research area to solve these issues. In this paper, we carried out literature survey of 43 studies to summarize techniques and tools utilized for code clone detection. Six code clone detection techniques are based on text, token, tree, metric, graph, and hybrid to detect type-1, type-2, type-3, and type-4 clones. Our study is concluded with several studies that exist to detect the types 1–4 clones. There is a requirement of novel approach to develop to detect all four types of clones together. In addition, need to develop more approaches to simplify to create program dependency graph for type-4 clone detection.

In evaluating code clone detection tools, a benchmark is created to measure their precision and recall. Benchmarks in previous research have either of the following issues: the first one is that they depend on the code clone definitions of benchmark creators; the second one is that they are not code clones occurring in actual development process. To get rid of both the two issues, we propose a methodology that creates code clone references based on code clones occurring in development process without any human judgements. More concretely, we use multiple revisions included in the source code repository of target software to identify merged methods in the past development process. We regard merged methods as real code clones. The authors' benchmark can evaluate detection accuracy of code clone detection tools more objectivity.

Adult schistosome cDNA libraries as a source of antigens for the study of experimental and human schistosomiasis

The construction of large-scale engineering project usually relies on the development of many new software systems, whereas it would be very expensive and time consuming if these new software systems are completely developed anew. In order to solve the problem, a model of legacy software reuse is proposed in this paper. The model is based on code clone detection. Firstly, code clone in source code of legacy software is detected by means of code clone detection tool. Secondly, abstract syntax trees of the functions which contain code clone are created. Thirdly, the degree of variation between the functions which contain the code clone belongs to the same clone set is calculated according to their abstract syntax trees, and then some functions whose similarities of abstract syntax trees are in the allowed range are combined. Finally, the combined functions and other frequently invoked functions are refactored into new functions or encapsulated into new classes, and all of these functions or classes can be reused as components in the development of new software systems. The test result shows that the reuse method based on this model can shrink the scope for searching the reusable component in legacy software systems, and thus improve the efficiency of legacy software reuse.

Code clone detection is an important problem for software maintenance and evolution. Many approaches consider either structure or identifiers, but none of the existing detection techniques model both sources of information. These techniques also depend on generic, handcrafted features to represent code fragments. We introduce learning-based detection techniques where everything for representing terms and fragments in source code is mined from the repository. Our code analysis supports a framework, which relies on deep learning, for automatically linking patterns mined at the lexical level with patterns mined at the syntactic level. We evaluated our novel learning-based approach for code clone detection with respect to feasibility from the point of view of software maintainers. We sampled and manually evaluated 398 file- and 480 method-level pairs across eight real-world Java systems; 93% of the file- and method-level samples were evaluated to be true positives. Among the true positives, we found pairs mapping to all four clone types. We compared our approach to a traditional structure-oriented technique and found that our learning-based approach detected clones that were either undetected or suboptimally reported by the prominent tool Deckard. Our results affirm that our learning-based approach is suitable for clone detection and a tenable technique for researchers.

It has been noted in recent years that the presence of code clones makes software maintenance more difficult. Unintended code inconsistencies may occur due to the presence of code clones. In order to avoid problems caused by code clones, it is necessary to identify where code clones exist in a software system. Consequently, various kinds of code clone detection techniques have been proposed before now. In particular, incremental code clone detection attracts much attention in the last few years, and line-based and token-based incremental detection methods have been proposed. In incremental detections, code clone detection results or their intermediate products persist by using databases, and it is used in next code clone detection. However, no incremental detection technique has been proposed for PDG-based detection, which requires much more time to detect code clones than line-or token-based detection. In this paper, we propose a PDG-based incremental code clone detection technique for improving practicality of PDG-based detection. A prototype tool has been developed, and it has been applied to open source software. We confirmed that detection time is extremely shortened and its detection result is almost the same as one of an existing PDG-based detection technique.

Exact or nearly similar code fragments in a software system's source code are referred to as code clones. It is often the case that updates (i.e., changes) to a code clone will need to be propagated to its related code clones to preserve their similarity and to maintain source code consistency. When there is a delay in propagating the changes (possibly because the developer is unaware of the related cloned code), the system might behave incorrectly. A delay in propagating a change is referred to as `late propagation,' and a number of studies have investigated this phenomenon. However, these studies did not investigate the intensity of late propagation nor how late propagation differs by clone type. In this research, we investigate late propagation separately for each of the three clone types (Type 1, Type 2, and Type 3). According to our experimental results on thousands of revisions of eight diverse subject systems written in two programming languages, late propagation occurs more frequently in Type 3 clones compared with the other two clone types. More importantly, there is a higher probability that Type 3 clones will experience buggy late propagations compared with the other two clone types. Also, we discovered that block clones are more involved in late propagation than method clones. Refactoring and tracking of Similarity Preserving Change Pattern (SPCP) clones (i.e., the clone fragments that evolve following a SPCP) can help us minimize the occurrences of late propagation in clones.

Code clones are prevalent in software systems due to many factors in software development. Detecting code clones and managing consistency between them along code evolution can be very useful for reducing clone-related bugs and maintenance costs. Despite some early attempts at detecting code clones and managing the consistency between them, the state-of-the-art tool can only handle simple code clones whose structures are identical or quite similar. However, existing empirical studies show that clones can have quite different structures with their evolution, which can easily go beyond the capability of the state-of-the-art tool. In this paper, we propose CCSync, a novel, rule-directed approach, which paves the structure differences between the code clones and synchronizes them even when code clones become quite different in their structures. The key steps of this approach are, given two code clones, to (1) extract a synchronization rule from the relationship between the clones, and (2) once one code fragment is updated, propagate the modifications to the other following the synchronization rule. We have implemented a tool for CCSync and evaluated its effectiveness on five Java projects. Our results shows that there are many code clones suitable for synchronization, and our tool achieves precisions of up to 92% and recalls of up to 84%. In particular, more than 76% of our generated revisions are identical with manual revisions.

Code clone is similar source code that occurs in a program. Code clone affects the maintainability of software. Therefore, various code clone detection techniquees such as text based comparison technique, token based comparison technique, tree based comparison technique, metric based comparison technique and graph based comparison technique have been proposed to detect code clone. Apart from these techniquees, models also have been used to detect code clone. Model consist of step by step process that is utilized to detect code clones. This work looks into analyzing existing code clone detection models through SWOT analysis. The strength and weakness of these code clone detection models is discussed. The output of this work shows that code clone detection is a more procedural way of detecting code clone through process and the opportunity of expanding the research area is vast.

Similar or identical code portions which are generated by copying and reusing code portions within the source code are named as code clones. While so many works have been conducted to detect these clones, they generally use string comparison techniques and very few of them take advantage of popular learning based approaches, such as deep learning. This paper proposes a new approach based on a popular and successful image classification technique named as convolutional neural network. It simply tokenizes each candidate clone pair in order to generate image files. Then, convolutional neural network is used to classify these image data with labels “clone” and “not clone”. In order to train and test the network, clone and not clone pairs are chosen from a public database including six million methods. As a result, the approach gives 99% accuracy, effectively detects clones and not clones with 2-5% false alarms rates at method granularity.

So far, many approaches for detecting code clones have been proposed based on the different degrees of normalizations (e.g. removal of white spaces, tokenization, and regularization of identifiers). Different degrees of normalizations lead to different granularities of source code to be detect as code clones. To investigate how the normalizations impact the code clone detection, this study proposes six approaches for detecting code clones with preprocessing input source files using different degrees of normalizations. More precisely, each normalization is applied to the input source files and then equivalence class partitioning is performed to the files in the preprocessing. After that, code clones are detected from a set of files that are representatives of each equivalence class using a token-based code clone detection tool named CCFinder. The proposed approaches can be categorized into two types, approaches with non-normalization and normalization. The former is the detection of only identical files without any normalization. Meanwhile, the latter category is the detection of identical files with different degrees of normalizations such as removal of all lines containing macros. From the case study, we observed that our proposed approaches detect code clones faster than the approach that uses only CCFinder. We also found the approach with non-normalization is the fastest among the proposed approaches in many cases.