Research on Social Comment Text Classification Based on BERT and Graph Neural Network

Graph neural networks induced by concept lattices for classification

As a trusted decentralized application, smart contracts manage a large number of digital assets on the blockchain. Vulnerability detection of smart contracts is an important part of ensuring the security of digital assets. At present, many researchers extract features of smart contract source code for vulnerability detection based on deep learning methods. However, the current research mainly focuses on the single representation form of the source code, which cannot fully obtain the rich semantic and structural information contained in the source code, so it is not conducive to the detection of various and complex smart contract vulnerabilities. Aiming at this problem, this paper proposes a vulnerability detection model based on the fusion of syntax and semantic features. The syntactic and semantic representation of the source code is obtained from the abstract syntax tree and control flow graph of the smart contract through TextCNN and Graph Neural Network. The syntactic and semantic features are fused, and the fused features are used to detect vulnerabilities. Experiments show that the detection accuracy and recall rate of this model have been improved on the detection tasks of five types of vulnerabilities, with an average precision of 96% and a recall rate of 90%, which can effectively identify smart contract vulnerabilities.

Path-aware multi-hop graph towards improving graph learning

Traditional GCN based methods capture the propagation structure between posts, but do not fully model dynamic semantic information, such as the role of specific users on the propagation path and the context of post content that changes over time, leading to a decrease in the accuracy of rumor detection. Therefore, we propose an innovative path attention graph convolution network (PAGCN) framework, which effectively solves this limitation by integrating propagation structure and semantic representation learning. PAGCN first uses the graph neural network (GNN) to model the information transmission path, focusing on the differences between rumor and fact information in communication behavior, such as the differences between depth first and breadth first dissemination modes. Then, in order to enhance the ability of semantic understanding, we design a multi head attention mechanism based on convolutional neural network (CNN), which extracts deep contextual relationships from text content. Furthermore, by introducing the comparative learning technology, PAGCN can adaptively optimize the representation of structural and semantic features, dynamically focus on the most discriminative features, and significantly improve the sensitivity to subtle patterns in rumor propagation. The experimental verification on three benchmark datasets of twitter15, twitter16, and Weibo, shows that the proposed PAGCN performs best among the 17 comparison models, and the accuracy rates on twitter15 and Weibo datasets are 90.9% and 93.9%, respectively, which confirms the effectiveness of the framework in capturing propagation structure and semantic information at the same time.

SCN_GNN: A GNN-based fraud detection algorithm combining strong node and graph topology information

In recent years, smart contract vulnerability detection methods mostly view smart contract source code as natural language for processing, which cannot fully capture the semantic and structural features of the source code and has a high rate of false positives and missing positives. To improve the accuracy of vulnerability detection, this paper uses Graph Neural Network to obtain the semantic and structural information of the source code and Convolutional Neural Network to assist learning. We propose a graph neural network-based vulnerability detection model for smart contracts, which transforms smart contracts into control flow graphs, learns graph embedding using graph neural networks, and introduces Convolutional Neural Networks to learn the node order information of control flow graphs, and finally performs vulnerability detection using graph embedding and node order information. Experimenting on real datasets, our accuracy and F1 values are improved and the model can effectively detect smart contract vulnerabilities.

Graph Neural Networks (GNNs) are deep learning methods which provide the current state of the art performance in node classification tasks. GNNs often assume homophily – neighboring nodes having similar features and labels–, and therefore may not be at their full potential when dealing with non-homophilic graphs. In this work, we focus on addressing this limitation and enable Graph Attention Networks (GAT), a commonly used variant of GNNs, to explore the structural information within each graph locality. Inspired by the positional encoding in the Transformers, we propose a framework, termed Graph Attentional Networks with Positional Embeddings (GAT-POS), to enhance GATs with positional embeddings which capture structural and positional information of the nodes in the graph. In this framework, the positional embeddings are learned by a model predictive of the graph context, plugged into an enhanced GAT architecture, which is able to leverage both the positional and content information of each node. The model is trained jointly to optimize for the task of node classification as well as the task of predicting graph context. Experimental results show that GAT-POS reaches remarkable improvement compared to strong GNN baselines and recent structural embedding enhanced GNNs on non-homophilic graphs.

Graph Neural Networks (GNNs) play a very important role today. It does analyze not only the graph data itself, but also the data connectivity of the graph. The quality of a GNN is thus altered by the result of extracted graph structure information. The extraction could be enhanced by GNN model design or directly from the training dataset with a GNN-decoupled method. In this paper, we propose RankedDrop, a new sampling method to improve the extraction of graph structure information. This approach is based on droppingout technique, and it adopts a spatial-aware selection of edges to drop. It takes into account structure information of the graph to control the dropping-out, and its random selection of edges to be dropped is under the control of a probability generated with respect to graph’s topological importance. Our experiments point out that RankedDrop provides high-quality and robust training results compared to the leading solutions. Furthermore, RankedDrop could be a framework plugin and combined with GNN model improvements to maximize GNN quality. Furthermore, RankedDrop could be a plugin for AI frameworks like MindSpore and combined with GNN model improvements to maximize GNN quality.

Graph Neural Networks (GNNs) have become the state-of-the-art method for many applications on graph structured data. GNNs are a model for graph representation learning, which aims at learning to generate low dimensional node embeddings that encapsulate structural and feature-related information. GNNs are usually trained in an end-to-end fashion, leading to highly specialized node embeddings. While this approach achieves great results in the single-task setting, the generation of node embeddings that can be used to perform multiple tasks (with performance comparable to single-task models) is still an open problem. We propose the use of meta-learning to allow the training of a GNN model capable of producing multi-task node embeddings. In particular, we exploit the properties of optimization-based meta-learning to learn GNNs that can produce general node representations by learning parameters that can quickly (i.e. with a few steps of gradient descent) adapt to multiple tasks. Our experiments show that the embeddings produced by a model trained with our purposely designed meta-learning procedure can be used to perform multiple tasks with comparable or, surprisingly, even higher performance than both single-task and multi-task end-to-end models.

Minnan nursery rhymes (MNRs), an integral part of Minnan intangible cultural heritage (ICH), are shared by southern Fujian, Taiwan, and overseas Chinese communities. Preserving and analyzing MNRs, especially their emotional evolution over time, is crucial but challenging due to shifting cultural contexts. Traditional sentiment analysis methods often overlook intrinsic relationships among nursery rhymes. To address this, we construct an MNR network using textual feature vectors and cosine similarity thresholds, enabling the exploration of structural and emotional patterns. Inspired by graph neural networks, we propose a Joint-GraphSAGE model for sentiment classification, which effectively captures complex relationships and semantic nuances. Comparative experiments with classical machine learning and deep learning models demonstrate that the Joint-GraphSAGE model significantly outperforms baselines in sentiment classification tasks for both traditional and modern MNRs. This approach not only enhances sentiment analysis accuracy for MNRs, but also offers new perspectives for studying ICH cultural connections.

Existing multivariate time series forecasting (MTSF) methods have begun to explore architectures built entirely upon Graph Neural Networks (GNNs). Directly connecting variables across different timestamps with a fully connected adjacency matrix introduces unnecessary noise and ignores the inherent temporal ordering between nodes. Besides this, the limited depth of Spatial graph convolutional layers restricts the ability of the model to capture long-term dependencies. We propose <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{S}^{\text{2}}$</tex>-<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{GNN}$</tex>, a multi-scale adaptive graph neural network method, which fusion spatial and spectral graph convolutions. An adaptive mechanism is used to capture the unidirectional relationship between nodes at different timestamps, which enables the construction of a graph structure that preserves the temporal order. Multi-order neighbor information and original node features are fused at each layer of spatial graph convolution to improve the local feature extraction capability of the spatial graph convolution. <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{S}^{\text{2}}$</tex>-GNN introduces spectral graph convolution and applies singular value decomposition (SVD) in the frequency domain to decompose the adjacency matrix and implement spectral graph convolution, which enables the extraction of key structural information and provides a compact representation of global spatio-temporal dependencies. We conduct extensive experiments on 7 public datasets for time series forecasting. The experimental results show that the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{S}^{\text{2}}$</tex>-GNN outperforms typical baseline models in time series forecasting tasks.

BackgroundLung cancer is a global disease with high lethality, with early screening being considerably helpful for improving the 5-year survival rate. Multimodality features in early screening imaging are an important part of the prediction for lung adenocarcinoma, and establishing a model for adenocarcinoma diagnosis based on multimodal features is an obvious clinical need. Through our practice and investigation, we found that graph neural networks (GNNs) are excellent platforms for multimodal feature fusion, and the data can be completed using the edge-generation network. Therefore, we propose a new lung adenocarcinoma multiclassification model based on multimodal features and an edge-generation network.MethodsAccording to a ratio of 80% to 20%, respectively, the dataset of 338 cases was divided into the training set and the test set through 5-fold cross-validation, and the distribution of the 2 sets was the same. First, the regions of interest (ROIs) cropped from computed tomography (CT) images were separately fed into convolutional neural networks (CNNs) and radiomics processing platforms. The results of the 2 parts were then input into a graph embedding representation network to obtain the fused feature vectors. Subsequently, a graph database based on the clinical and semantic features was established, and the data were supplemented by an edge-generation network, with the fused feature vectors being used as the input of the nodes. This enabled us to clearly understand where the information transmission of the GNN takes place and improves the interpretability of the model. Finally, the nodes were classified using GNNs.ResultsOn our dataset, the proposed method presented in this paper achieved superior results compared to traditional methods and showed some comparability with state-of-the-art methods for lung nodule classification. The results of our method are as follows: accuracy (ACC) =66.26% (±4.46%), area under the curve (AUC) =75.86% (±1.79%), F1-score =64.00% (±3.65%), and Matthews correlation coefficient (MCC) =48.40% (±5.07%). The model with the edge-generating network consistently outperformed the model without it in all aspects.ConclusionsThe experiments demonstrate that with appropriate data=construction methods GNNs can outperform traditional image processing methods in the field of CT-based medical image classification. Additionally, our model has higher interpretability, as it employs subjective clinical and semantic features as the data construction approach. This will help doctors better leverage human–computer interactions.

Graphs help to define the relationships between entities in the data. These relationships, represented by edges, often provide additional context information which can be utilised to discover patterns in the data. Graph Neural Networks (GNNs) employ the inductive bias of the graph structure to learn and predict on various tasks. The primary operation of graph neural networks is the feature aggregation step performed over neighbours of the node based on the structure of the graph. In addition to its own features, for each hop, the node gets additional combined features from its neighbours. These aggregated features help define the similarity or dissimilarity of the nodes with respect to the labels and are useful for tasks like node classification. However, in real‐world data, features of neighbours at different hops may not correlate with the node's features. Thus, any indiscriminate feature aggregation by GNN might cause the addition of noisy features leading to degradation in model's performance. In this work, we show that selective aggregation of node features from various hops leads to better performance than default aggregation on the node classification task. Furthermore, we propose a Dual‐Net GNN architecture with a classifier model and a selector model. The classifier model trains over a subset of input node features to predict node labels while the selector model learns to provide optimal input subset to the classifier for the best performance. These two models are trained jointly to learn the best subset of features that give higher accuracy in node label predictions. With extensive experiments, we show that our proposed model outperforms both feature selection methods and state‐of‐the‐art GNN models with remarkable improvements up to 27.8%.

A central question in neuroscience is how self-organizing dynamic interactions in the brain emerge on their relatively static structural backbone. Due to the complexity of spatial and temporal dependencies between different brain areas, fully comprehending the interplay between structure and function is still challenging and an area of intense research. In this paper we present a graph neural network (GNN) framework, to describe functional interactions based on the structural anatomical layout. A GNN allows us to process graph-structured spatio-temporal signals, providing a possibility to combine structural information derived from diffusion tensor imaging (DTI) with temporal neural activity profiles, like that observed in functional magnetic resonance imaging (fMRI). Moreover, dynamic interactions between different brain regions discovered by this data-driven approach can provide a multi-modal measure of causal connectivity strength. We assess the proposed model’s accuracy by evaluating its capabilities to replicate empirically observed neural activation profiles, and compare the performance to those of a vector auto regression (VAR), like that typically used in Granger causality. We show that GNNs are able to capture long-term dependencies in data and also computationally scale up to the analysis of large-scale networks. Finally we confirm that features learned by a GNN can generalize across MRI scanner types and acquisition protocols, by demonstrating that the performance on small datasets can be improved by pre-training the GNN on data from an earlier study. We conclude that the proposed multi-modal GNN framework can provide a novel perspective on the structure-function relationship in the brain. Accordingly this approach appears to be promising for the characterization of the information flow in brain networks.

Motif-aware curriculum learning for node classification.