- Research Article
18
- 10.1016/j.cose.2024.104098
Survey of source code vulnerability analysis based on deep learning
- Sep 03, 2024
- Computers & Security
- Chen Liang + 4 more +4
Survey of source code vulnerability analysis based on deep learning
Program semantics learning is the core and fundamental for various code intelligent tasks e.g., vulnerability detection, clone detection. A considerable amount of existing works propose diverse approaches to learn the program semantics for different tasks and these works have achieved state-of-the-art performance. However, currently, a comprehensive and systematic study on evaluating different program representation techniques across diverse tasks is still missed. From this starting point, in this paper, we conduct an empirical study to evaluate different program representation techniques. Specifically, we categorize current mainstream code representation techniques into four categories i.e., Feature-based, Sequence-based, Tree-based, and Graph-based program representation technique and evaluate its performance on three diverse and popular code intelligent tasks i.e., Code Classification, Vulnerability Detection, and Clone Detection on the public released benchmark. We further design three research questions (RQs) and conduct a comprehensive analysis to investigate the performance. By the extensive experimental results, we conclude that (1) The graph-based representation is superior to the other selected techniques across these tasks. (2) Compared with the node type information used in tree-based and graph-based representations, the node textual information is more critical to learning the program semantics. (3) Different tasks require the task-specific semantics to achieve their highest performance, however combining various program semantics from different dimensions such as control dependency, data dependency can still produce promising results.
Survey of source code vulnerability analysis based on deep learning
Survey of source code vulnerability analysis based on deep learning
On the impact of multiple source code representations on software engineering tasks — An empirical study
On the impact of multiple source code representations on software engineering tasks — An empirical study
Proceedings of the 3rd Textgraphs Workshop on Graph-Based Algorithms for Natural Language Processing - TextGraphs '08
Recent years have shown an increased interest in bringing the field of graph theory into Natural Language Processing. In many NLP applications entities can be naturally represented as nodes in a graph and relations between them can be represented as edges. Recent research has shown that graph-based representations of linguistic units as diverse as words, sentences and documents give rise to novel and efficient solutions in a variety of NLP tasks, ranging from part of speech tagging, word sense disambiguation and parsing to information extraction, semantic role assignment, summarization and sentiment analysis. The contribution of the graph representation, in addition to its intuitiveness, resides in the possibility to relate linguistic entities beyond pairwise comparison. This volume contains papers accepted for presentation at the TextGraphs-3 2008 Workshop on Graph-Based Algorithms for Natural Language Processing. This event took place on August 24, 2008, in Manchester, UK, immediately following COLING 2008, the 22nd International Conference on Computational Linguistics. It was the third workshop on this topic, building on the success of the first and second TextGraphs workshop at HLT-NAACL 2006 and 2007. The workshop aimed at bringing together researchers working on problems related to the use of graph-based algorithms for Natural Language Processing and on the theory of graph-based methods. It addressed a broad spectrum of research areas to foster exchange of ideas and to help identify principles of using the graph notions that go beyond an ad-hoc usage. We issued calls for both regular and short, late-breaking papers. Six regular and three short papers were accepted for presentation, based on the careful reviews of our program committee. We are indebted to all program committee members for their thoughtful, high quality and elaborate reviews, especially considering our extremely tight time frame for reviewing. The papers appearing in this volume have surely benefited from their expert feedback. This year's workshop attracted papers employing graphs in a wide range of settings, so we are proud to present a very diverse program this year. N. Hathout acquires morphological structure from a lexicon employing the bipartite graph between headwords' formal semantic features. Mapping of text to a graph-based meaning representation is conducted by S. Muresan, using a recent grammar formalism. A. B. Masse et al. lay out a general theoretical framework for addressing the symbol grounding problem in digital dictionaries. A. Moschitti and F.M. Zanzotto use Kernel methods on tree pairs for recognizing textual entailment. Combining co-occurrence and phonological similarity, K. Ichioka and F. Fukumoto semantically cluster onomatopoetic words in Japanese. D. Rao et al. examine several random walk based approaches to measure word similarity. B. McGillivray et al. address cluster overlapping with correspondence analysis and apply their method to cluster English and Italian verbs and nouns. A domain-specific summarization method ranking nodes in a graph of concepts is introduced by L. Plaza Morales et al. The topology of associative concept dictionaries is modeled by H. Akama et al., who report interesting scale free properties of such networks.
Read moreTCCCD: Triplet-Based Cross-Language Code Clone Detection
Code cloning is a common practice in software development, where developers reuse existing code to accelerate programming speed and enhance work efficiency. Existing clone-detection methods mainly focus on code clones within a single programming language. To address the challenge of code clone instances in cross-platform development, we propose a novel method called TCCCD, which stands for Triplet-Based Cross-Language Code Clone Detection. Our approach is based on machine learning and can accurately detect code clone instances between different programming languages. We used the pre-trained model UniXcoder to map programs written in different languages into the same vector space and learn their code representations. Then, we fine-tuned TCCCD using triplet learning to improve its effectiveness in cross-language clone detection. To assess the effectiveness of our proposed approach, we conducted thorough comparative experiments using the dataset provided by the paper titled CLCDSA (Cross Language Code Clone Detection using Syntactical Features and API Documentation). The experimental results demonstrated a significant improvement of our approach over the state-of-the-art baselines, with precision, recall, and F1-measure scores of 0.96, 0.91, and 0.93, respectively. In summary, we propose a novel cross-language code-clone-detection method called TCCCD. TCCCD leverages the pre-trained model UniXcode for source code representation and fine-tunes the model using triplet learning. In the experimental results, TCCCD outperformed the state-of-the-art baselines in terms of the precision, recall, and F1-measure.
Read moreNeural Methods for Programming: A Comprehensive Survey and Future Directions
The advancement of neural-based models has driven significant progress in modern code intelligence, accelerating the development of intelligent programming tools such as code assistants and automated software engineering systems. This study presents a comprehensive and systematic survey of neural methods for programming tasks within the broader context of software development. Guided by six research questions, this study synthesizes insights from more than 250 scientific papers, the majority of which were published between 2015 and 2025, with earlier foundational works (dating back to the late 1990s) included for historical context. The analysis spans 18 major programming tasks, including code generation, code translation, code clone detection, code classification, and vulnerability detection. The survey methodologically examines the development and evolution of neural approaches, the datasets employed, and the performance evaluation metrics adopted in this field. It traces the progress in neural techniques from early code modeling approaches to advanced Code-specific Large Language Models (Code LLMs), emphasizing their advantages over traditional rule-based and statistical methods. A taxonomy of evaluation metrics and a categorized summary of datasets and benchmarks reveal both progress and persistent limitations in data coverage and evaluation practices. The review further distinguishes neural models designed for natural language processing and programming languages, highlighting the structural and functional characteristics that influence model performance. Finally, the study discusses emerging trends, unresolved challenges, and potential research directions, underscoring the transformative role of neural-based architectures, particularly Code LLMs, in enhancing programming and software design activities and shaping the future of AI-driven software development.
Read moreLearning and Fusing Multi-View Code Representations for Function Vulnerability Detection
The explosive growth of vulnerabilities poses a significant threat to the security of software systems. While various deep-learning-based vulnerability detection methods have emerged, they primarily rely on semantic features extracted from a single code representation structure, which limits their ability to detect vulnerabilities hidden deep within the code. To address this limitation, we propose S2FVD, short for Sequence and Structure Fusion-based Vulnerability Detector, which fuses vulnerability-indicative features learned from the multiple views of the code for more accurate vulnerability detection. Specifically, S2FVD employs either well-matched or carefully extended neural network models to extract vulnerability-indicative semantic features from the token sequence, attributed control flow graph (ACFG) and abstract syntax tree (AST) representations of a function, respectively. These features capture different perspectives of the code, which are then fused to enable S2FVD to accurately detect vulnerabilities that are well-hidden within a function. The experiments conducted on two large vulnerability datasets demonstrated the superior performance of S2FVD against state-of-the-art approaches, with its accuracy and F1 scores reaching 98.07% and 98.14% respectively in detecting the presence of vulnerabilities, and 97.93% and 97.94%, respectively, in pinpointing specific vulnerability types. Furthermore, with regard to the real-world dataset D2A, S2FVD achieved average performance gains of 6.86% and 14.84% in terms of accuracy and F1 metrics, respectively, over the state-of-the-art baselines. This ablation study also confirms the superiority of fusing the semantics implied in multiple distinct code views to further enhance vulnerability detection performance.
Read moreVulnerability Detection via Multiple-Graph-Based Code Representation
During software development and maintenance, vulnerability detection is an essential part of software quality assurance. Even though many program-analysis-based and machine-learning-based approaches have been proposed to automatically detect vulnerabilities, they rely on explicit rules or patterns defined by security experts and suffer from either high false positives or high false negatives. Recently, an increasing number of studies leverage deep learning techniques, especially Graph Neural Network (GNN), to detect vulnerabilities. These approaches leverage program analysis to represent the program semantics as graphs and perform graph analysis to detect vulnerabilities. However, they suffer from two main problems: (i) Existing GNN-based techniques do not effectively learn the structural and semantic features from source code for vulnerability detection. (ii) These approaches tend to ignore fine-grained information in source code. To tackle these problems, in this paper, we propose a novel vulnerability detection approach, named <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MGVD</small> (<bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</b> <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ultiple</small>-<bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</b> <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">raph-Based</small> <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">V</b> <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ulnerability</small> <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D</b> <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">etection)</small>, to detect vulnerable functions. To effectively learn the structural and semantic features from source code, <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MGVD</small> uses three different ways to represent each function into multiple forms, i.e., two statement graphs and a sequence of tokens. Then we encode such representations to a three-channel feature matrix. The feature matrix contains the structural feature and the semantic feature of the function. And we add a weight allocation layer to distribute the weights between structural and semantic features. To overcome the second problem, <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MGVD</small> constructs each graph representation of the input function using multiple different graphs instead of a single graph. Each graph focuses on one statement in the function and its nodes denote the related statements and their fine-grained code elements. Finally, <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MGVD</small> leverages CNN to identify whether this function is vulnerable based on such feature matrix. We conduct experiments on 3 vulnerability datasets with a total of 30,341 vulnerable functions and 127,931 non-vulnerable functions. The experimental results show that our method outperforms the state-of-the-art by 9.68% – 10.28% in terms of F1-score.
Read morePVDetector: Pretrained Vulnerability Detection on Vulnerability-enriched Code Semantic Graph
Automated vulnerability detection is a critical issue in software security. The advent of deep learning (DL) has led to numerous studies employing DL to detect vulnerabilities in software source code. However, existing approaches still perform poorly, particularly with real-world vulnerabilities, due to the difficulty in accurately capturing their properties. To this end, we introduce PVDetector, a DL-based approach that utilizes rich code semantics, incorporates vulnerability knowledge, and leverages pretrained code representations for precise vulnerability detection. At its core, PVDetector employs a new model called Vulnerability-enriched Code Semantic Graph (VCSG), which accurately characterizes functions by distinguishing the semantics of identical variables and more finely capturing control dependencies, data dependencies, and vulnerability relationships. Additionally, we introduce four pretraining tasks specifically designed to learn the semantics of control, data, vulnerability, and variables from the VCSG model. These pretraining tasks significantly enhance PVDetector's capability to detect vulnerabilities in downstream tasks. Experimental results indicate that PVDetector outperforms SOTAs by 5.0%-12.5% in precision, 0.2%-9.7% in recall, and 3.0%-15.1% in F1-score. Additionally, it supports six programming languages and demonstrates high efficiency (e.g., 10.6 \(\times\) faster than DeepDFA). When applied to seven software products, PVDetector discovered 55 vulnerabilities, including 10 silently patched flaws that had not been previously reported.
Read moreRule-directed code clone synchronization
Code clones are prevalent in software systems due to many factors in software development. Detecting code clones and managing consistency between them along code evolution can be very useful for reducing clone-related bugs and maintenance costs. Despite some early attempts at detecting code clones and managing the consistency between them, the state-of-the-art tool can only handle simple code clones whose structures are identical or quite similar. However, existing empirical studies show that clones can have quite different structures with their evolution, which can easily go beyond the capability of the state-of-the-art tool. In this paper, we propose CCSync, a novel, rule-directed approach, which paves the structure differences between the code clones and synchronizes them even when code clones become quite different in their structures. The key steps of this approach are, given two code clones, to (1) extract a synchronization rule from the relationship between the clones, and (2) once one code fragment is updated, propagate the modifications to the other following the synchronization rule. We have implemented a tool for CCSync and evaluated its effectiveness on five Java projects. Our results shows that there are many code clones suitable for synchronization, and our tool achieves precisions of up to 92% and recalls of up to 84%. In particular, more than 76% of our generated revisions are identical with manual revisions.
Read moreA Novel Source Code Representation Approach Based on Multi-Head Attention
Code classification and code clone detection are crucial for understanding and maintaining large software systems. Although deep learning surpasses traditional techniques in capturing the features of source code, existing models suffer from low processing power and high complexity. We propose a novel source code representation method based on the multi-head attention mechanism (SCRMHA). SCRMHA captures the vector representation of entire code segments, enabling it to focus on different positions of the input sequence, capture richer semantic information, and simultaneously process different aspects and relationships of the sequence. Moreover, it can calculate multiple attention heads in parallel, speeding up the computational process. We evaluate SCRMHA on both the standard dataset and an actual industrial dataset, and analyze the differences between these two datasets. Experiment results in code classification and clone detection tasks show that SCRMHA consumes less time and reduces complexity by about one-third compared with traditional source code feature representation methods. The results demonstrate that SCRMHA reduces the computational complexity and time consumption of the model while maintaining accuracy.
Read moreVulnerability detection through cross-modal feature enhancement and fusion
Vulnerability detection through cross-modal feature enhancement and fusion
A Smart Contract Vulnerability Detection Model Based on Syntactic and Semantic Fusion Learning
As a trusted decentralized application, smart contracts manage a large number of digital assets on the blockchain. Vulnerability detection of smart contracts is an important part of ensuring the security of digital assets. At present, many researchers extract features of smart contract source code for vulnerability detection based on deep learning methods. However, the current research mainly focuses on the single representation form of the source code, which cannot fully obtain the rich semantic and structural information contained in the source code, so it is not conducive to the detection of various and complex smart contract vulnerabilities. Aiming at this problem, this paper proposes a vulnerability detection model based on the fusion of syntax and semantic features. The syntactic and semantic representation of the source code is obtained from the abstract syntax tree and control flow graph of the smart contract through TextCNN and Graph Neural Network. The syntactic and semantic features are fused, and the fused features are used to detect vulnerabilities. Experiments show that the detection accuracy and recall rate of this model have been improved on the detection tasks of five types of vulnerabilities, with an average precision of 96% and a recall rate of 90%, which can effectively identify smart contract vulnerabilities.
Read moreAn Empirical Study on the Characteristics of Reusable Code Clones
Copy-and-paste of code is a common practice in software development, especially with the availability of myriad open-source projects. Reusing existing code fragments may accelerate the development process and ensure better quality of software if high-quality code fragments are reused. Therefore, code cloning might not be avoided even though it can increase software maintenance cost as bugs can be propagated inadvertently by code cloning and go beyond the boundary of a system. Generally speaking, reusable code clones are highly popular, have fewer bugs, and can stand the test of time thus exist in the system for a long term. Current code reuse studies primarily leverage API usage and method clone structures to provide coding recommendations. However, the existing approaches focus on the functional utility of code snippets without considering the quality aspects, such as fault resiliency, when analyzing code reuse. Without clone quality in mind, the virtues of code cloning are severely diminished. To help developers determine if the code clones are reusable or not, we leverage Machine Learning classifiers and Large Language Models (LLMs) to automatically identify reusable code clones from three perspectives: clone prevalence (i.e., the number of clone siblings), clone fault-resiliency (i.e., the percentage of non-buggy commits versus buggy commits), and clone longevity (i.e., the clone genealogy length). Our approach achieves a median AUC of 0.73, with an F1-score of 0.89, based on experiments conducted on 60 open-source projects, consisting of 30 Java and 30 C projects, which collectively encompass 538,598 commits. The results show that CountFollowers (i.e., number of people following the contributors that wrote the clones), SimilarityClonePaths (i.e., the Jaccard similarity coefficient among clone paths), and CountContributors (i.e., number of distinct contributors that access the clone group) provide the most explanatory power that contributes to correctly classifying reusable code clones. Hence, practitioners can utilize our classifiers and insights from our findings to make more reliable use of code clones and prioritize the use of high-quality clones in their clone management activities.
Read moreResearch on the Tools of Clone Code Refactoring
Clone code is the code fragment that is identical or similar in syntax or semantics, which has great impact on software development and maintenance. According to the large amount of clone code in software and its complex changes, researchers have proposed many methods to eliminate the harmful clone code, in which refactoring is an effective measure. In this paper, concepts of clone code and refactoring are introduced firstly. And then the main methods of current clone code refactoring are compared and analyzed, so the related tools of clone code refactoring can be elaborated and their advantages and disadvantages are summarized. At last, the shortcomings and limitations of the clone code refactoring are discussed.
Read moreDetermining Degree of Relevance of Reviews Using a Graph-Based Text Representation
Reviews are text-based feedback provided by reviewers to authors. The quality of a review can be determined by identifying how relevant it is to the work that the review was written for as well as its similarity to existing well-written and coherent reviews. Relevance between two pieces of text can be determined by identifying semantic and syntactic similarities between them. In this paper, we make use of string-based metrics that incorporate concepts of paraphrasing and plagiarism to determine matching between texts. We use a graph-based text representation technique. We use the k-nearest neighbor classification algorithm to build a supervised model and classify text as LOW, MEDIUM or HIGH based on values of the metrics. We evaluate our approach on three data sets from student assignments and show that our model achieves an average accuracy of 63%.
Read more