Increasing clone maintenance support by unifying clone detection and refactoring activities

This report documents the program and the outcomes of Dagstuhl Seminar 12071 ``Software Clone Management Towards Industrial Application''. Software clones are identical or similar pieces of code or design. A lot of research has been devoted to software clones. Unlike previous research, this seminar put a particular emphasis on industrial application of software clone management methods and tools and aimed at gathering concrete usage scenarios of clone management in industry, which will help to identify new industrially relevant aspects in order to shape the future research. Talks were presented by industrial participants and working groups were formed to discuss issues in clone detection, presentation, and refactoring. In addition we developed a unified conceptual model to capture clone information required to support a common notion of clone data and for interoperability to foster exchange of data among researchers and tools in practice. The main focus of current research is clones in source code -- therefore, we also looked into ways of extending our research to other types of software artifacts. Last but not least, we discussed how clone management activities may be integrated into the process of software development.

So far, a lot of techniques have been developed on the detection of code clones (i.e., duplicated code) in large-scale source code. Currently, the code clone research community is gradually shifting its focus of attention from the detection to the management (e.g., clone refactoring). During clone management, developers need to understand how and why code clones are scattered in source code, and then decide how to handle those code clones. In this paper, we present a clone analysis tool with tag cloud visualization. This tool is aimed at helping to understand why code clone are concentrated in a part of a software system by generating tag clouds from a collection of identifier names in source code.

This paper presents a technique to detect clones in source code by comparing the operations performed in the statements comprising a function. The key concept used is that two functions are considered clones if the statements in the functions perform the same operation up to a certain extent. This could be ascertained by categorizing the available statement types based on the operations performed (for instance, addition, multiplication, function invocation, etc). Then, a category is assigned to each statement present in every function in the source code. Comparisons are then made between functions by comparing the categories of the statements to each other. If one function contains exactly the same categories of statement as another (same operations performed in both the functions), or contains a subset of statement categories (operations performed in one function are subset of another), then these functions are judged to be clones.

Code clone is one of the factors that makes software maintenance more difficult. Once a developer find a defect in a code fragment, he/she has to inspect the all of the code clones of the code fragment. In this study, we investigated the effectiveness of query-based use of a vector-based clone detection tool for supporting simultaneous fixing of buggy clones in source code and compared it with the query-based use of a token-based clone detection tool CCFinder.

Source code cloning does not happen within a single system only. It can also occur between one system and another. We use the term code sibling to refer to a code clone that evolves in a different system than the code from which it originates. Code siblings can only occur when the source code copyright owner allows it and when the conditions imposed by such license are not incompatible with the license of the destination system. In some situations copying of source code fragments are allowed—legally—in one direction, but not in the other. In this paper, we use clone detection, license mining and classification, and change history techniques to understand how code siblings—under different licenses—flow in one direction or the other between Linux and two BSD Unixes, FreeBSD and OpenBSD. Our results show that, in most cases, this migration appears to happen according to the terms of the license of the original code being copied, favoring always copying from less restrictive licenses towards more restrictive ones. We also discovered that sometimes code is inserted to the kernels from an outside source.

While finding clones in source code has drawn considerable attention, there has been only very little work in finding similar fragments in binary code and intermediate languages, such as Java bytecode. Some recent studies showed that it is possible to find distinct sets of clone pairs in bytecode representation of source code, which are not always detectable at source code-level. In this paper, we present a bytecode clone detection approach, called SeByte, which exploits the benefits of compilers (the bytecode representation) for detecting a specific type of semantic clones in Java bytecode. SeByte is a hybrid metric-based approach that takes advantage of both, Semantic Web technologies and Set theory. We use a two-step analysis process: (1) Pattern matching via Semantic Web querying and reasoning, and (2) Content matching, using Jaccard coefficient for set similarity measurement. Semantic Web-based pattern matching helps us to find method blocks which share similar patterns even in case of extreme dissimilarity (e.g., numerous repetitions or large gaps). Although it leads to high recall, it gives high false positive rate. We thus use the content matching (via Jaccard) to reduce false positive rate by focusing on content semantic resemblance. Our evaluation of four Java systems and five other tools shows that SeByte can detect a large number of semantic clones that are either not detected or supported by source code based clone detectors.

Excessive code duplication is a bane of modern software development. Several experimental studies show that on average 15 percent of a software system can contain source code clones - repeatedly reused fragments of similar code. While code duplication may increase the speed of initial software development, it undoubtedly leads to problems during software maintenance and support. That is why many developers agree that software clones should be detected and dealt with at every stage of software development life cycle. This paper is a brief survey of current state-of-the-art in clone detection. First, we highlight main sources of code cloning such as copy-and-paste programming, mental code patterns and performance optimizations. We discuss reasons behind the use of these techniques from the developer's point of view and possible alternatives to them. Second, we outline major negative effects that clones have on software development. The most serious drawback duplicated code have on software maintenance is increasing the cost of modifications - any modification that changes cloned code must be propagated to every clone instance in the program. Software clones may also create new software bugs when a programmer makes some mistakes during code copying and modification. Increase of source code size due to duplication leads to additional difficulty of code comprehension. Third, we review existing clone detection techniques. Classification based on used source code representation model is given in this work. We also describe and analyze some concrete examples of clone detection techniques highlighting main distinctive features and problems that are present in practical clone detection. Finally, we point out some open problems in the area of clone detection. Currently questions like "What is a code clone?", "Can we predict the impact clones have on software quality" and "How can we increase both clone detection precision and recall at the same time? " stay open to further research. We list the most important questions in modern clone detection and explain why they continue to remain unanswered despite all the progress in clone detection research.

Dynamic web pages composed of inter-woven (tangled) source code written in multiple programming languages (e.g., HTML, PHP, JavaScript, CSS) makes it difficult to analyze and manage clones in web applications. Despite more than a decade of research on software clones, there are not many studies towards the investigation of code clones in web applications. In this paper, we present an in-depth study on the patterns (i.e., forking and templating) of exact and near-miss code clones in two industrial dynamic web applications having distinct architecture. The findings of our study confirm the believed patterns for cloning and suggest that specialized techniques and tool support are necessary for effectively managing clones in the tangled source code of dynamic web applications.

In this research we have developed an algorithmic approach to convert source code clones to aspects. Code cloning is the process of duplicating code or creating replication of code fragments. In this work, we use an existing code-clone detection tool to identify code clones in a source code. Secondly, we design algorithms to convert the code clones into aspects and do aspect composition with the original source code. Thirdly, we implement a prototype based on the algorithms. Fourthly, we carry out a performance analysis on the aspects composed source code and our analysis shows that the aspect composed code performs as well as the original code and even better in terms of execution times.

In this paper we discuss a new application of Semantic Web and Artificial Intelligence in software analysis research. We show on a concrete example - clone detection for object-oriented source code that transitivity closure computation can provide added value to the clone detection community. Our novel approach models the domain of discourse knowledge as a mixture of source code patterns and inheritance trees represented as Directed Acyclic Graphs. Our approach promotes the use of Semantic Web and inference engines in source code analysis. More specifically we take advantage of the Semantic Web and its support for knowledge modeling and transitive closure computation to detect semantic source code clones not detected by traditional detection tools.

A code clone is a code fragment identical or similar to another code fragment in source code. Some of code clones are considered as a factor of bug replications and make it more difficult to maintain software. Various code clone detection tools have been proposed so far. However, in most algorithms adopted by existing clone detection tools, if program statements are reordered, they are not detected as code clones. In this research, we examined how clone detection results change by rearranging the order of program statements. We performed preprocessing to rearranging the order of program statements using program dependency graph (PDG). We compared clone detection results with and without preprocessing. As a result, by rearranging the order of program statements, the number of detected code clones is almost the same in most projects. We classified newly detected or disappeared clones manually. From our experimental results, we show that there is no newly detected clone whose statements are reordered and that there are four disappeared clones whose statements are reordered. We think three out of the four clones occurred by copy-and-paste operations. Therefore, we conclude that rearranging the order of program statements is not effective to detect reordered code clones.

Code clones have to be made explicit and be managed in software maintenance. Researchers have developed many clone detection tools to detect and analyze code clones in software systems. These tools report code clones as similar code fragments in source files. However, clone-related maintenance tasks (e.g., refactorings) often involve a group of code clones appearing in larger syntactic context (e.g., code clones in sibling classes or code clones calling similar methods). Given a list of low-level code-fragment clones, developers have to manually summarize from bottom up low-level code clones that are relevant to the syntactic context of a maintenance task. In this paper, we present a clone summarization technique to summarize code clones with respect to their common syntactic context. The clone summarization allows developers to locate and maintain code clones in a top-down manner by type hierarchy and usage dependencies. We have implemented our approach in the Clonepedia tool and conducted a user study on JHotDraw with 16 developers. Our results show that Clonepedia users can better locate and refactor code clones, compared with developers using the Clone Detective tool.

The ecosystem of open source software (OSS) has been growing considerably in size. In addition, code clones - code fragments that are copied and pasted within or between software systems - are also proliferating. Although code cloning may expedite the process of software development, it often critically affects the security of software because vulnerabilities and bugs can easily be propagated through code clones. These vulnerable code clones are increasing in conjunction with the growth of OSS, potentially contaminating many systems. Although researchers have attempted to detect code clones for decades, most of these attempts fail to scale to the size of the ever-growing OSS code base. The lack of scalability prevents software developers from readily managing code clones and associated vulnerabilities. Moreover, most existing clone detection techniques focus overly on merely detecting clones and this impairs their ability to accurately find "vulnerable" clones. In this paper, we propose VUDDY, an approach for the scalable detection of vulnerable code clones, which is capable of detecting security vulnerabilities in large software programs efficiently and accurately. Its extreme scalability is achieved by leveraging function-level granularity and a length-filtering technique that reduces the number of signature comparisons. This efficient design enables VUDDY to preprocess a billion lines of code in 14 hour and 17 minutes, after which it requires a few seconds to identify code clones. In addition, we designed a security-aware abstraction technique that renders VUDDY resilient to common modifications in cloned code, while preserving the vulnerable conditions even after the abstraction is applied. This extends the scope of VUDDY to identifying variants of known vulnerabilities, with high accuracy. In this study, we describe its principles and evaluate its efficacy and effectiveness by comparing it with existing mechanisms and presenting the vulnerabilities it detected. VUDDY outperformed four state-of-the-art code clone detection techniques in terms of both scalability and accuracy, and proved its effectiveness by detecting zero-day vulnerabilities in widely used software systems, such as Apache HTTPD and Ubuntu OS Distribution.

Software clone detection, aiming at identifying out code fragments with similar functionalities, has played an important role in software maintenance and evolution. Many clone detection approaches have been proposed. However, most of them represent source codes with hand-crafted features using lexical or syntactical information, or unsupervised deep features, which makes it difficult to detect the functional clone pairs, i.e., pieces of codes with similar functionality but differing in both syntactical and lexical level. In this paper, we address the software functional clone detection problem by learning supervised deep features. We formulate the clone detection as a supervised learning to hash problem and propose an end-to-end deep feature learning framework called CDLH for functional clone detection. Such framework learns hash codes by exploiting the lexical and syntactical information for fast computation of functional similarity between code fragments. Experiments on software clone detection benchmarks indicate that the CDLH approach is effective and outperforms the state-of-the-art approaches in software functional clone detection.

Code clone is similar source code that occurs in a program. Code clone affects the maintainability of software. Therefore, various code clone detection techniquees such as text based comparison technique, token based comparison technique, tree based comparison technique, metric based comparison technique and graph based comparison technique have been proposed to detect code clone. Apart from these techniquees, models also have been used to detect code clone. Model consist of step by step process that is utilized to detect code clones. This work looks into analyzing existing code clone detection models through SWOT analysis. The strength and weakness of these code clone detection models is discussed. The output of this work shows that code clone detection is a more procedural way of detecting code clone through process and the opportunity of expanding the research area is vast.