Local and global self-attention enhanced graph convolutional network for skeleton-based action recognition

Graph convolutional networks (GCNs) have revealed great promise in skeleton-based gait recognition by modeling human joints as a spatiotemporal graph. However, most current spatiotemporal GCN (ST-GCN) methods rely on a fixed graph topology and a static partitioning scheme. This fixed approach limits the model’s ability to capture complex relationships between distant body joints. Besides, Temporal Convolutional Networks (TCNs) are commonly used to model temporal dynamics. However, they have shortcomings in capturing temporal patterns at multiple scales. To overcome these constraints, this paper introduces a new architecture, i.e., SATGCN, that integrates a graph attention combination module (GACM) with a multi-scale temporal convolutional network (MSTCN). In GACM, a fusion approach is used, in which the attention module dynamically learns joint relationships with GCN at each frame. It enables the network to capture local spatial topology and long-range dependencies between physically distant body parts. Following the spatial module, the multi-scale TCN analyzes joint features over time at multiple scales, capturing both short- and long-range motion patterns across the gait sequence. The model is evaluated on the CASIA-B gait recognition dataset under normal, bag-carrying, and clothing conditions, achieving accuracies of 96.97%, 93.10%, and 90.65%, respectively. The performance is about 0.97%, 1.80%, and 0.65% better than that of previous works in the three conditions. The results reveal the effectiveness of combining an attention-guided GCN with multi-scale temporal convolution for skeleton-based gait recognition.

Detecting anomalous traffic behaviors with seasonal deep Kalman filter graph convolutional neural networks

Video object detection has made significant progress in recent years thanks to convolutional neural networks (CNNs) and vision transformers (ViTs). Typically, CNNs excel at capturing local features but struggle to model global representations. Conversely, ViTs are adept at capturing long-range global features but face challenges in representing local feature details. Off-the-shelf video object detection methods solely rely on CNNs or ViTs to conduct feature aggregation, which hampers their capability to simultaneously leverage global and local information, thereby resulting in limited detection performance. In this paper, we propose a Transformer-GraphFormer Blender Network (TGBFormer) for video object detection, with three key technical improvements to fully exploit the advantages of transformers and graph convolutional networks while compensating for their limitations. First, we develop a spatial-temporal transformer module to aggregate global contextual information, constituting global representations with long-range feature dependencies. Second, we introduce a spatial-temporal GraphFormer module that utilizes local spatial and temporal relationships to aggregate features, generating new local representations that are complementary to the transformer outputs. Third, we design a global-local feature blender module to adaptively couple transformer-based global representations and GraphFormer-based local representations. Extensive experiments demonstrate that our TGBFormer establishes new state-of-the-art results on the ImageNet VID dataset. Particularly, our TGBFormer achieves 86.5% mAP while running at around 41.0 FPS on a single Tesla A100 GPU.

A novel anomaly detection method for multivariate time series based on spatial-temporal graph learning

Graph convolutional networks (GCNs) can effectively learn graph data features and are widely used to predict the bearing remaining useful life (RUL). However, most existing GCN-based methods are based on single-node relationships to construct the graph structure while ignoring potential complex relationships between nodes. To enrich graph feature information and improve bearing RUL prediction accuracy, a novel bearing RUL prediction method is proposed in this article, which use spatial–temporal information fusion graph convolution network. The proposed method mines implicit temporal dependence and spatial correlation in graph samples by constructing PathGraph and RadiusGraph, which effectively improves the feature diversity of graph samples. In addition, to efficiently extract the spatial–temporal features of graph samples and alleviate the over-smoothing problem in GCNs, a feature extraction module combining graph attention network (GAT) and graph convolution network–temporal convolution network (GCN-TCN) is designed, where GAT dynamically assigns edge weights to capture spatial dependencies, while GCN-TCN enhances global temporal dependency capture. The experimental validation results show that the proposed method effectively improves bearing RUL prediction accuracy.

Graph convolutional neural networks have difficulties in model training takes a lot of time and lack of universality in meta-path design when processing knowledge graphs. To solve these issues, this paper proposes an efficient recommendation algorithm integrating knowledge graphs with graph convolutional networks(EKG-GCN). The algorithm is mainly based on the graph convolution algorithm. Firstly, the relationship between users and entities is standardized and scored, then a new neighborhood aggregation method is proposed. Some nodes in the knowledge graph are set as central nodes. A distance-influence function is set up according to the distance from other nodes to the central node, and the feature extraction of information in the knowledge graph is completed according to the distance-influence function. Finally, the knowledge graph features are used as an auxiliary to complete the recommendation task. In the experiment, three data sets were selected to compare with other advanced models, and it was found that the model is in an advantageous position in terms of training time and recommendation effect, and has a certain degree of universality.

LightGCAN: A lightweight graph convolutional attention network for user preference modeling and personalized recommendation

The main core purpose of artificial emotional intelligence is to recognize human emotions. Technologies such as facial, semantic, or brainwave recognition applications have been widely proposed. However, the abovementioned recognition techniques for emotional features require a large number of training samples to obtain high accuracy. Human behaviour pattern can be trained and recognized by the continuous movement of the Spatial Temporal Graph Convolution Network (ST-GCN). However, this technology does not distinguish between the speed of delicate emotions, and the speed of human behaviour and delicate changes of emotions cannot be effectively distinguished. This research paper proposes Spatial Temporal Variation Convolutional Network training for human emotion recognition, using skeleton detection technology to calculate the degree of skeleton point change between consecutive actions and using the nearest neighbour algorithm to classify speed levels and train the ST-GCN recognition model to obtain the emotional state. Application of the speed change recognition ability of the Spatial Temporal Variation Graph Convolution Network (STV-GCN) to artificial emotional intelligence calculation makes it possible to efficiently recognize the delicate actions of happy, sad, fear, and angry in human behaviour. The STV-GCN technology proposed in this paper is compared with ST-GCN and can effectively improve the recognition accuracy by more than 50%.

Traffic prediction has attracted a lot of attention in recent years. However, it is challenging due to the dynamic and heterogeneous spatial–temporal correlations. Most of the existing methods adopt the attention mechanism to capture the dynamic spatial features, but it is difficult for the attention mechanism to learn the real-time change of spatial dependencies, which restrict accurate spatial dependencies learning. Furthermore, most methods are out at elbows when solving heterogeneous spatial–temporal data. To overcome these problems, we propose a novel traffic prediction model called Dynamic spatial–temporal Heterogeneous Graph Convolution Network. Different from the existing methods, we have designed a dynamic localized graph and a corresponding adaptive localized graph convolution network, which are capable of simultaneously capturing dynamic and heterogeneous spatial correlations. We propose a gated adaptive temporal convolution network to capture the temporal heterogeneity of traffic data and enjoy global receptive fields. Finally, a global correlations fusion network is provided to incorporate the global spatial–temporal correlations. Compared with 11 baselines, our proposed model achieves state-of-the-art performance in the accuracy of prediction.

Accurate spatiotemporal prediction of landslide displacement is crucial for early disaster warning and risk mitigation. However, existing deep learning-based models typically focus on either temporal or spatial features independently, without adequately capturing the interactions and dependencies between them, leading to reduced prediction accuracy. In this study, we propose a novel InSAR-based landslide displacement prediction method that integrates spatial and temporal features through a graph neural network (GNN) framework. Taking the Heifangtai landslide area in the Chinese Loess Plateau as a case study, we first utilize SBAS-InSAR techniques on multi-orbit Sentinel-1A data to derive surface deformation time series, and apply a three-dimensional decomposition method to extract east-west, vertical, and north-south displacement components. Four typical landslide regions in the Heifangtai area are selected as experimental scenarios, and their corresponding spatial nodes are defined through a grid-based approach to construct an adjacency matrix, forming a spatial graph. We then develop a hybrid model, GCN-MHSA-TCN (GMTC), which combines Graph Convolutional Networks (GCN), Multi-Head Self-Attention (MHSA), and Temporal Convolutional Networks (TCN) to jointly learn spatiotemporal dependencies for accurate landslide displacement forecasting. Experimental results demonstrate that the proposed method effectively captures both local and long-range spatiotemporal correlations, achieving improved prediction accuracy and robustness compared to existing approaches. Moreover, across the four typical landslide regions, the GMTC model achieves an average root mean square error (RMSE) of 0.13 mm, representing the best performance among all evaluated prediction models. The predicted displacement time series exhibit high consistency with observed values and demonstrate greater temporal stability. This model offers a promising tool for supporting landslide monitoring, hazard assessment, and emergency evacuation planning based on InSAR time-series data.

Coronary heart disease (CHD) has become one of the most serious public health issues due to its high morbidity and mortality rates. Most of the existing coronary heart disease risk prediction models manually extract features based on shallow machine learning methods. It only focuses on the differences between local patient features and ignores the interaction modeling between global patients. Its accuracy is still insufficient for individualized patient management strategies. In this paper, we propose CHD prediction as a graph node classification task for the first time, where nodes can represent individuals in potentially diseased populations and graphs intuitively represent associations between populations. We used an adaptive multi-channel graph convolutional neural network (AM-GCN) model to extract graph embeddings from topology, node features, and their combinations through graph convolution. Then, the adaptive importance weights of the extracted embeddings are learned by using an attention mechanism. For different situations, we model the relationship of the CHD population with the population graph and the K-nearest neighbor graph method. Our experimental evaluation explored the impact of the independent components of the model on the CHD disease prediction performance and compared it to different baselines. The experimental results show that our new model exhibits the best experimental results on the CHD dataset, with a 1.3% improvement in accuracy, a 5.1% improvement in AUC, and a 4.6% improvement in F1-score compared to the nongraph model.

As a current research hotspot, graph convolution networks (GCNs) have provided new opportunities for tree species classification in multi-source remote sensing images. To solve the challenge of limited label information, a new tree species classification model was proposed by using the semi-supervised graph convolution fusion method for hyperspectral images (HSIs) and multispectral images (MSIs). In the model, the graph-based attribute features and pixel-based features are fused to deepen the correlation of multi-source images to improve accuracy. Firstly, the model employs the canonical correlation analysis (CCA) method to maximize the correlation of multi-source images, which explores the relationship between information from various sources further and offers more valuable insights. Secondly, convolution calculations were made to extract features and then map graph node fusion, which not only reduces redundancy features but also enhances compelling features. Finally, the relationship between representative descriptors is captured through the use of hyperedge convolution in the training process, and the dominant features on the graph are fully mined. The tree species are classified through two fusion feature operations, leading to improved classification performance compared to state-of-the-art methods. The fusion strategy can produce a complete classification map of the study areas.

This paper proposes a novel Quantum Spatial Graph Convolutional Neural Network (QSGCNN) model that can directly learn a classification function for graphs of arbitrary sizes. The main idea is to define a new quantum-inspired spatial graph convolution associated with pre-transformed fixed-sized aligned grid structures of graphs, in terms of quantum information propagation between grid vertices of each graph. We show that the proposed QSGCNN model can significantly reduce either the information loss or the notorious tottering problem arising in existing spatially-based Graph Convolutional Network (GCN) models. Experiments on benchmark graph datasets demonstrate the effectiveness of the proposed QSGCNN model.

Skeleton-based human action recognition has great potential for human behavior analysis owing to its simplicity and robustness in varying environments. This paper presents a spatial and temporal attention-enhanced graph convolution network (STAEGCN) for human action recognition. The spatial-temporal attention module in the network uses convolution embedding for positional information and adopts multi-head self-attention mechanism to extract spatial and temporal attention separately from the input series of the skeleton. The spatial and temporal attention are then concatenated into an entire attention map according to a specific ratio. The proposed spatial and temporal attention module was integrated with an adaptive graph convolution network to form the backbone of STAEGCN. Based on STAEGCN, a two-stream skeleton-based human action recognition model was trained and evaluated. The model performed better on both NTU RGB+D and Kinetics 400 than 2s-AGCN and its variants. It was proven that the strategy of decoupling spatial and temporal attention and combining them in a flexible way helps improve the performance of graph convolution networks in skeleton-based human action recognition.

Geometric deep learning provides a principled and universal way for the integration of imaging and non-imaging modes in the medical field. Graph Convolutional Networks (GCNs) have been widely explored in a variety of problems, such as disease prediction, segmentation, and matrix completion. Using large, multi-modal data sets, graphs can capture the interaction of individual elements represented as nodes in the graphs. In particular, in medical applications, nodes can represent individuals (patients or healthy controls) in a potentially large population and are accompanied by a set of features, while the edges of the graph contain the associations between subjects in an intuitive way. This representation allows the inclusion of rich imaging and non-imaging information as well as individual subject characteristics in the task of disease classification. This article gives an overview of graph convolutional neural networks, focusing on the application of graph convolutional neural networks in disease prediction, and discusses the challenges and prospects of graph convolutional neural networks in disease prediction.