Supervised Deep Features for Software Functional Clone Detection by Exploiting Lexical and Syntactical Information in Source Code

The ecosystem of open source software (OSS) has been growing considerably in size. In addition, code clones - code fragments that are copied and pasted within or between software systems - are also proliferating. Although code cloning may expedite the process of software development, it often critically affects the security of software because vulnerabilities and bugs can easily be propagated through code clones. These vulnerable code clones are increasing in conjunction with the growth of OSS, potentially contaminating many systems. Although researchers have attempted to detect code clones for decades, most of these attempts fail to scale to the size of the ever-growing OSS code base. The lack of scalability prevents software developers from readily managing code clones and associated vulnerabilities. Moreover, most existing clone detection techniques focus overly on merely detecting clones and this impairs their ability to accurately find "vulnerable" clones. In this paper, we propose VUDDY, an approach for the scalable detection of vulnerable code clones, which is capable of detecting security vulnerabilities in large software programs efficiently and accurately. Its extreme scalability is achieved by leveraging function-level granularity and a length-filtering technique that reduces the number of signature comparisons. This efficient design enables VUDDY to preprocess a billion lines of code in 14 hour and 17 minutes, after which it requires a few seconds to identify code clones. In addition, we designed a security-aware abstraction technique that renders VUDDY resilient to common modifications in cloned code, while preserving the vulnerable conditions even after the abstraction is applied. This extends the scope of VUDDY to identifying variants of known vulnerabilities, with high accuracy. In this study, we describe its principles and evaluate its efficacy and effectiveness by comparing it with existing mechanisms and presenting the vulnerabilities it detected. VUDDY outperformed four state-of-the-art code clone detection techniques in terms of both scalability and accuracy, and proved its effectiveness by detecting zero-day vulnerabilities in widely used software systems, such as Apache HTTPD and Ubuntu OS Distribution.

Code clone detection is an important problem for software maintenance and evolution. Many approaches consider either structure or identifiers, but none of the existing detection techniques model both sources of information. These techniques also depend on generic, handcrafted features to represent code fragments. We introduce learning-based detection techniques where everything for representing terms and fragments in source code is mined from the repository. Our code analysis supports a framework, which relies on deep learning, for automatically linking patterns mined at the lexical level with patterns mined at the syntactic level. We evaluated our novel learning-based approach for code clone detection with respect to feasibility from the point of view of software maintainers. We sampled and manually evaluated 398 file- and 480 method-level pairs across eight real-world Java systems; 93% of the file- and method-level samples were evaluated to be true positives. Among the true positives, we found pairs mapping to all four clone types. We compared our approach to a traditional structure-oriented technique and found that our learning-based approach detected clones that were either undetected or suboptimally reported by the prominent tool Deckard. Our results affirm that our learning-based approach is suitable for clone detection and a tenable technique for researchers.

Over the past few years, several software clone detection techniques have been developed. The software clones are the consequence of copied/pasted activity in software development which eventuates at different level of abstraction and may have different inception in a software system. This paper presents an efficient approach for the detection of type-1 software clones. The proposed detect type-1 software clones with high precision, recall, portability, and scalability. The type-1 clones generated by using mutation operator-based editing taxonomy.

A code clone is a code fragment identical or similar to another code fragment in source code. Some of code clones are considered as a factor of bug replications and make it more difficult to maintain software. Various code clone detection tools have been proposed so far. However, in most algorithms adopted by existing clone detection tools, if program statements are reordered, they are not detected as code clones. In this research, we examined how clone detection results change by rearranging the order of program statements. We performed preprocessing to rearranging the order of program statements using program dependency graph (PDG). We compared clone detection results with and without preprocessing. As a result, by rearranging the order of program statements, the number of detected code clones is almost the same in most projects. We classified newly detected or disappeared clones manually. From our experimental results, we show that there is no newly detected clone whose statements are reordered and that there are four disappeared clones whose statements are reordered. We think three out of the four clones occurred by copy-and-paste operations. Therefore, we conclude that rearranging the order of program statements is not effective to detect reordered code clones.

Duplicate code is an example of bad smells, which are usually been refactored after the detection to improve the quality of programs. Locate the duplicate code at the programming phase may reduce the cost of maintenance, but the challenge is it need to detect duplicate code between an incomplete code fragment with complete files, which the existing tools are hard to be applied to this scenario. In this paper, we propose an AST-sequence-based duplicate code detection approach for onsite programming code. The abstract syntax tree (AST) is extracted from source code and then is transformed into an encoded sequence. A local sequence alignment algorithm is used to find highly similar subsequences. After the post-processing, similar regions will be found between two code fragments according to the subsequences. We have developed a prototype tool as a plugin for Visual Studio Code. Experimental results indicate that our approach is effective in finding highly similar regions between cross-granularity code fragments, which can facilitate duplicate code detection for incomplete onsite programming code.

Maintaining software systems becomes more difficult as their size and complexity increase.One factor that makes software maintenance more difficult is the presence of code clones.A code clone is defined as a code fragment occurring more than once in identical or similar form into a software system.For example, the presence of code clones is a big factor of overlooking some places that should be modified simultaneously.One technique that helps the number of code clones is Refactoring.There are several research efforts that provide support to refactor code clones, but unfortunately some code clones cannot or should not be refactored (ex.stereotyped process, absence of abstraction functionality, performance enhancement).In order to support maintaining the consistency among code clones, we propose a simultaneous modification support method.Given a software system, firstly, a maintainer identifies a code fragment that must be modified.Then, only the code clones between the identified code fragment and the source files of the software system are detected.We developed a simultaneous modification support tool, Libra, and applied it to open source software systems.The results showed that Libra was a good searching tool as much as grep, which is a useful tool of UNIX.

Assessing the similarity between code components plays a pivotal role in a number of Software Engineering (SE) tasks, such as clone detection, impact analysis, refactoring, etc. Code similarity is generally measured by relying on manually defined or hand-crafted features, e.g., by analyzing the overlap among identifiers or comparing the Abstract Syntax Trees of two code components. These features represent a best guess at what SE researchers can utilize to exploit and reliably assess code similarity for a given task. Recent work has shown, when using a stream of identifiers to represent the code, that Deep Learning (DL) can effectively replace manual feature engineering for the task of clone detection. However, source code can be represented at different levels of abstraction: identifiers, Abstract Syntax Trees, Control Flow Graphs, and Bytecode. We conjecture that each code representation can provide a different, yet orthogonal view of the same code fragment, thus, enabling a more reliable detection of similarities in code. In this paper, we demonstrate how SE tasks can benefit from a DL-based approach, which can automatically learn code similarities from different representations.

Code clones have to be made explicit and be managed in software maintenance. Researchers have developed many clone detection tools to detect and analyze code clones in software systems. These tools report code clones as similar code fragments in source files. However, clone-related maintenance tasks (e.g., refactorings) often involve a group of code clones appearing in larger syntactic context (e.g., code clones in sibling classes or code clones calling similar methods). Given a list of low-level code-fragment clones, developers have to manually summarize from bottom up low-level code clones that are relevant to the syntactic context of a maintenance task. In this paper, we present a clone summarization technique to summarize code clones with respect to their common syntactic context. The clone summarization allows developers to locate and maintain code clones in a top-down manner by type hierarchy and usage dependencies. We have implemented our approach in the Clonepedia tool and conducted a user study on JHotDraw with 16 developers. Our results show that Clonepedia users can better locate and refactor code clones, compared with developers using the Clone Detective tool.

Code clones are prevalent in software systems due to many factors in software development. Detecting code clones and managing consistency between them along code evolution can be very useful for reducing clone-related bugs and maintenance costs. Despite some early attempts at detecting code clones and managing the consistency between them, the state-of-the-art tool can only handle simple code clones whose structures are identical or quite similar. However, existing empirical studies show that clones can have quite different structures with their evolution, which can easily go beyond the capability of the state-of-the-art tool. In this paper, we propose CCSync, a novel, rule-directed approach, which paves the structure differences between the code clones and synchronizes them even when code clones become quite different in their structures. The key steps of this approach are, given two code clones, to (1) extract a synchronization rule from the relationship between the clones, and (2) once one code fragment is updated, propagate the modifications to the other following the synchronization rule. We have implemented a tool for CCSync and evaluated its effectiveness on five Java projects. Our results shows that there are many code clones suitable for synchronization, and our tool achieves precisions of up to 92% and recalls of up to 84%. In particular, more than 76% of our generated revisions are identical with manual revisions.

As Code Clones are defined on the notion of similarity in code fragments, it is necessary to first know what a code is meant by in accordance with Code Clones. A Source Code Fragment, which is a sequence of source code lines, is the basic entity that is used to analyze similarity/relation between Code Clones. For analysis, removal, avoidance, and management of Code Clones we have to first detect clones in software systems. There are more than 40 clone detection tools that implement some clone detection techniques to detect clones, but it is not well-defined what could be the appropriate minimum threshold for Clone length and with which unit of estimation. This paper, on the basis of Code Clone literature, presents different Units of Measurement of Clone Size and a comprehensive review of minimum Clone Size based on a particular technique used in Clone Detection and also argues that a unique Unit of Measurement and Minimum Clone Size should be presented.

Copy-and-paste of code is a common practice in software development, especially with the availability of myriad open-source projects. Reusing existing code fragments may accelerate the development process and ensure better quality of software if high-quality code fragments are reused. Therefore, code cloning might not be avoided even though it can increase software maintenance cost as bugs can be propagated inadvertently by code cloning and go beyond the boundary of a system. Generally speaking, reusable code clones are highly popular, have fewer bugs, and can stand the test of time thus exist in the system for a long term. Current code reuse studies primarily leverage API usage and method clone structures to provide coding recommendations. However, the existing approaches focus on the functional utility of code snippets without considering the quality aspects, such as fault resiliency, when analyzing code reuse. Without clone quality in mind, the virtues of code cloning are severely diminished. To help developers determine if the code clones are reusable or not, we leverage Machine Learning classifiers and Large Language Models (LLMs) to automatically identify reusable code clones from three perspectives: clone prevalence (i.e., the number of clone siblings), clone fault-resiliency (i.e., the percentage of non-buggy commits versus buggy commits), and clone longevity (i.e., the clone genealogy length). Our approach achieves a median AUC of 0.73, with an F1-score of 0.89, based on experiments conducted on 60 open-source projects, consisting of 30 Java and 30 C projects, which collectively encompass 538,598 commits. The results show that CountFollowers (i.e., number of people following the contributors that wrote the clones), SimilarityClonePaths (i.e., the Jaccard similarity coefficient among clone paths), and CountContributors (i.e., number of distinct contributors that access the clone group) provide the most explanatory power that contributes to correctly classifying reusable code clones. Hence, practitioners can utilize our classifiers and insights from our findings to make more reliable use of code clones and prioritize the use of high-quality clones in their clone management activities.

SUMMARY Although code clones (i.e. code fragments that have similar or identical code fragments in the source code) are regarded as a factor that increases the complexity of software maintenance, tools for supporting clone refactoring (i.e. merging a set of code clones into a single method or function) are not commonly used. To promote the development of refactoring tools that can be more widely utilized, we present an investigation of clone refactoring carried out in the development of open source software systems. In the investigation, we identified the most frequently used refactoring patterns and discovered how merged code clone token sequences and

Clone code is the code fragment that is identical or similar in syntax or semantics, which has great impact on software development and maintenance. According to the large amount of clone code in software and its complex changes, researchers have proposed many methods to eliminate the harmful clone code, in which refactoring is an effective measure. In this paper, concepts of clone code and refactoring are introduced firstly. And then the main methods of current clone code refactoring are compared and analyzed, so the related tools of clone code refactoring can be elaborated and their advantages and disadvantages are summarized. At last, the shortcomings and limitations of the clone code refactoring are discussed.

Activities such as copying, pasting and modifying codes usually lead to a lot of code clones in the software systems. These code clones have a great impact on software development and maintenance. This paper presents a new clone detection approach. That is to uses a algorithm based on suffix array to efficiently search repeated token substrings and finally to detect code clones. We also develop a clone detection tool SaCD in order to implement the method. This tool has detected statement clones in 31 open source software systems of C, C++ and Java language. The experimental results show that the SaCD can efficiently detect code clones at the statement granularity in the software.

In recent times, the topic of human facial age estimation attracted much attention. This is due to its ability to improve biometrics systems. Recently, several applications that are based on the demographic attributes estimation have been developed. These include law enforcement, re-identification in videos, planed marketing, intelligent advertising, social media, and human-computer interaction. The main contributions of the paper are as follows. Firstly, it extends some handcrafted models that are based on the Pyramid Multi Level (PML) face representation. Secondly, it evaluates the performance of two different kinds of features that are handcrafted and deep features. It compares handcrafted and deep features in terms of accuracy and computational complexity. The proposed scheme of study includes the following three main steps: 1) face preprocessing; 2) feature extraction (two different kinds of features are studied: handcrafted and deep features); 3) feeding the obtained features to a linear regressor. In addition, we investigate the strengths and weaknesses of handcrafted and deep features when used in facial age estimation. Experiments are run on three public databases (FG-NET, PAL and FACES). These experiments show that both handcrafted and deep features are effective for facial age estimation.