Safer and Efficient Factory by Predicting Worker Trajectories Using Spatio-Temporal Graph Attention Networks

Spatio-temporal graph signal analysis has a significant impact on a wide range of applications, including hand/body pose action recognition. To achieve effective analysis, spatio-temporal graph convolutional networks (ST-GCN) leverage the powerful learning ability to achieve great empirical successes; however, those methods need a huge amount of high-quality training data and lack theoretical interpretation. To address this issue, the spatio-temporal graph scattering transform (ST-GST) was proposed to put forth a theoretically interpretable framework; however, the empirical performance of this approach is constrained by the fully mathematical design. To benefit from both sides, this work proposes a novel complementary mechanism to organically combine the spatio-temporal graph scattering transform and neural networks, resulting in the proposed spatio-temporal graph complementary scattering networks (ST-GCSN). The essence is to leverage the mathematically designed graph wavelets with pruning techniques to cover major information and use trainable networks to capture complementary information. The empirical experiments on hand pose action recognition show that the proposed ST-GCSN outperforms both ST-GCN and ST-GST.

In the realm of intelligent transportation systems, accurately predicting vehicle trajectories is paramount for enhancing road safety and optimizing traffic flow management. Addressing the impacts of complex traffic environments and efficiently modeling the diverse behaviors of vehicles are the key challenges at present. To achieve precise prediction of vehicle trajectories, it is essential to fully consider the dynamic changes in traffic conditions and the long-term dependencies of time-series data. In response to these challenges, we propose the Memory-Enhanced Spatio-Temporal Graph Network (MESTGN), an innovative model that integrates a Spatio-Temporal Graph Convolutional Network (STGCN) with an attention-enhanced Long Short-Term Memory (LSTM)-based sequence to sequence (Seq2Seq) encoder–decoder structure. MESTGN utilizes STGCN to capture the complex spatial dependencies between vehicles and reflects the interactions within the traffic network through road traffic data and network topology, which significantly influences trajectory prediction. Additionally, the model focuses on historical vehicle trajectory data points using an attention-weighted mechanism under a traditional LSTM prediction architecture, calculating the importance of critical trajectory points. Finally, our experiments conducted on the urban traffic dataset ApolloSpace validate the effectiveness of our proposed model. We demonstrate that MESTGN shows a significant performance improvement in vehicle trajectory prediction compared with existing mainstream models, thereby confirming its increased prediction accuracy.

This study proposes a vessel position prediction method using attention spatiotemporal graph convolutional networks, which addresses the issue of low prediction accuracy due to less consideration of inter-feature dependencies in current vessel trajectory prediction methods. First, the method cleans the vessel trajectory data and uses the Time-ratio trajectory compression algorithm to compress the trajectory data, avoiding data redundancy and providing feature points for vessel trajectories. Second, the Spectral Temporal Graph Neural Network (StemGNN) extracts the correlation matrix that describes the relationship between multiple variables as a priori matrix input to the prediction model. Then the vessel trajectory prediction model is constructed, and the attention mechanism is added to the spatial and temporal dimensions of the trajectory data based on the spatio-temporal graph convolutional network at the same time as the above operations are performed on different time scales. Finally, the features extracted from different time scales are fused through the full connectivity layer to predict the future trajectories. Experimental results show that this method achieves higher accuracy and more stable prediction results in trajectory prediction. The attention-based spatio-temporal graph convolutional networks effectively capture the spatio-temporal correlations of the main features in vessel trajectories, and the spatio-temporal attention mechanism and graph convolution have certain interpretability for the prediction results.

Four-dimensional (4D) trajectory prediction plays a critical role in modern air traffic management, enabling applications such as conflict detection, anomaly monitoring, and congestion mitigation. However, existing methods have limited information sources when modeling potential spatial correlations between aircraft in complex airspace scenarios, and their final trajectory inference ability is weak, resulting in lower prediction accuracy. Faced with these challenges, we propose Modern Temporal and Dual Attention based Spatiotemporal Graph Convolutional Network (MTDA-STGCN), which employs a self-attention mechanism to reconstruct the adjacency matrix to enhance the ability of capturing global node correlations. This adjacency matrix reconstructed with the self-attention mechanism is dynamically optimized throughout the training process of network, offering a more nuanced reflection of the inter-node relationships compared to traditional algorithms. Subsequently, our model uses graph attention to extract additional global features for modeling accuracy interactions between aircraft. Finally, the output is input into the Modern Temporal Prediction Network (MTPN) to obtain the predicted trajectory probability distribution. The experiments on real-world ADS-B datasets demonstrate that MTDA-STGCN outperforms existing 4D trajectory prediction algorithms on all datasets. The proposed dual-attention framework significantly enhances the capture of node spatial correlations, while the MTPN module effectively improves the accuracy of the predicted results.

Spatio-temporal heterogeneous graph using multivariate earth observation time series: Application for drought forecasting

In order to provide intelligent and efficient healthcare services in the Internet of Medical Things (IoMT), human action recognition (HAR) can play a crucial role. As a result of their stringent requirements, such as high computational complexity and memory efficiency, classical HAR techniques are not applicable to modern and intelligent healthcare services, e.g., IoMT. To address these issues, we present in this paper a novel HAR technique for healthcare services in IoMT. This model, referred to as the spatio-temporal graph convolutional network (STGCN), primarily aims at skeleton-based human–machine interfaces. By independently extracting spatial and temporal features, STGCN significantly reduces information loss. Spatio-temporal information is extracted independently of the exact spatial and temporal point, ensuring the extraction of useful features for HAR. Using only joint data and fewer parameters, we demonstrate that our proposed STGCN achieved 92.2% accuracy on the skeleton dataset. Unlike multi-channel methods, which use a combination of joint and bone data and have a large number of parameters, multi-channel methods use both joint and bone data. As a result, STGCN offers a good balance between accuracy, memory consumption, and processing time, making it suitable for detecting medical conditions.

Optimizing the energy consumption of an MEC (Multi-Access Edge Computing) system is a crucial challenge for operation cost reduction and environmental conservation. In this paper, we address an MECS (MEC Server) sleep control problem that aims to reduce the energy consumption of the system while providing users with a reasonable service delay by adjusting the number of active MECSs according to the load imposed on the system. To tackle the problem, we identify two crucial issues that influence the design of an effective sleep control technique and propose methods to address each of these issues. The first issue is accurately predicting the system load. Changes in system load are spatio-temporally correlated among MECSs. By leveraging such correlation information with STGCN (Spatio-Temporal Graph Convolutional Network), we enhance the prediction accuracy of task arrival rates for each MECS. The second issue is rapidly selecting MECSs to sleep when the load distribution over an MEC system is given. The problem of choosing sleep MECS is a combinatorial optimization problem with high time complexity. To address the issue, we employ a genetic algorithm and quickly determine the optimal sleep MECS with the predicted load information for each MECS. Through simulation studies, we verify that compared to the LSTM (Long Short-Term Memory)-based method, our method increases the energy efficiency of an MEC system while providing a compatible service delay.

This study introduces the MSTA-GNet (Multi-Scale Spatiotemporal Attention Graph Network), a novel deep learning model which integrates spatiotemporal self-attention mechanisms to model heterogeneous dependencies in traffic networks. The primary objective of the study is to improve existing traffic flow prediction models to address the inadequacies of traditional models in complex big data environments. Key innovations of the MSTA-GNet model include positional encoding and global and local self-attention mechanisms to capture long-term and short-term dependencies. Using the PeMS (Performance Measurement System) dataset, the study conducted performance comparison experiments among various deep learning models, including LSTM (Long Short-Term Memory), GCN (Graph Convolutional Network), DCRNN (Diffusion Convolutional Recurrent Neural Network), STGCN (Spatiotemporal Graph Convolutional Network), STMetaNet (Spatiotemporal Meta Network), and MSTA-GNet. The results showed that MSTA-GNet significantly outperformed other models with improvements of 13.4%, 11.8%, and 9.7% in Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE) metrics, respectively. Ablation studies further validated the significance of attention mechanisms, feature extraction, convolutional layers, and graph networks, confirming the effectiveness and practical application of MSTA-GNet in traffic flow prediction. This research provides important insights for AI-based congestion management, support for low-carbon traffic networks, and optimization of local traffic operations, demonstrating its significant practical value in intelligent transportation systems.

Accurate spatiotemporal forecasting of electric vehicle (EV) charging load is essential for smart grid management and efficient charging service operation. This paper introduced a novel spatiotemporal graph convolutional network with cross-attention (STGCN-Attention) for multi-factor charging load prediction. The model demonstrated a strong capability to capture cross-scale spatiotemporal correlations by adaptively integrating historical charging load, charging pile occupancy, dynamic electricity prices, and meteorological data. Evaluations in real-world charging scenarios showed that the proposed model achieved superior performance in hour forecasting, reducing Mean Absolute Error (MAE) by 9% and 16% compared to traditional STGCN and LSTM models, respectively. It also attained approximately 30% higher accuracy than 24 h prediction. Furthermore, the study identified an optimal 1-2-1 multi-scale temporal window strategy (hour–day–week) and revealed key driver factors. The combined input of load, occupancy, and electricity price yielded the best results (RMSE = 37.21, MAE = 27.34), while introducing temperature and precipitation raised errors by 2–8%, highlighting challenges in fine-grained weather integration. These findings provided actionable insights for real-time and intraday charging scheduling.

Anomaly detection is an important problem that has been researched in several domains. Based on the available data patterns, various supervised and unsupervised anomaly detection techniques have been introduced. In this paper, a novel anomaly detection technique for location aware geospatial big dataset is outlined. Specifically, we focus on anomaly detection in spatiotemporal complex networks. The outlined technique incorporates components of anomaly quantification and decision making on spatiotemporal graphs and embeds simultaneous learning and detection procedures. The magnitude of an anomaly at each time step is quantified to signify the pattern of anomalous behavior in the spatiotemporal network. We illustrate the efficacy of the proposed method by detecting and indicating the time and location of a single or multiple anomalies in an illustrative traffic network problem. Theoretical experiments on a suite of six randomly generated traffic network problems have been performed. The performance of the proposed algorithm with tuned parameters on this random set of problem instances clearly establishes the effectiveness and applicability of the introduced solution procedure.

In autonomous driving scenarios, pedestrian trajectory prediction is an important research direction. Based on the spatio-temporal graph convolutional neural network, we propose a new pedestrian trajectory prediction algorithm. The new algorithm constructs a series of new models around pedestrian intention estimation. The construction of the estimation algorithm considers the following aspects: the contextual information of pedestrians and the surrounding environment, the “pedestrian ego-vehicle” interaction combined with the vehicle speed estimation, the pedestrian’s own skeletal structure information and body language estimation, which includes head joints and the relative structural relationship of the torso joints, including whether it is out of the same plane, is rotated, and so on. Skeleton information feature extraction and construction adopts the method of graph convolutional neural network to structure pedestrians into joints in the form of graphs in non-Euclidean space, and further adopts spatial temporal graph convolutional network for feature extraction and learning. The new method is named a “head-torso”-based spatial temporal graph convolutional network (HT-STGCN). On the dataset PID, the novel method achieves substantial improvements over mainstream methods. Experimental results show that combining HT-STGCN with observed action can improve trajectory prediction.

The reason why China’s table tennis can continue to this day is because it is an important heritage sports event in the country. In order to ensure the high-level performance of the Chinese table tennis team in the game, the identification and detection of its sports characteristics is a very meaningful work. With the development of computer technology, the use of multimedia intelligence technology to conduct research on competitive tactics has become a general consensus in the sports world. This paper explores the recognition and detection technology of the strength of table tennis hitting movements from the perspective of biomechanics, aiming to provide scientific basis and technical support for improving the performance of athletes. Based on the principles of biomechanics and combined with multimedia intelligent algorithms, this study developed a spatiotemporal graph convolutional network (STGCN) and a motion detection method based on Kinect technology to identify and quantify the hitting strength of table tennis players. The experimental results of this paper show that in the recognition of different types of movements based on the STGCN method, the correct recognition rate of 100 groups of movement strength is 84%, and the correct recognition rate of 500 groups of movement strength is 91.6%; in the recognition of different types of movement strength based on Kinect, the correct recognition rate of 100 groups of movement strength is 97%, and the correct recognition rate of 500 groups of movement strength is 99.6%; it can be seen that no matter how many groups of hitting movements are made, the correct recognition rate of strength based on Kinect is higher than that of STGCN.

IntroductionAccurately recognizing and understanding human motion actions presents a key challenge in the development of intelligent sports robots. Traditional methods often encounter significant drawbacks, such as high computational resource requirements and suboptimal real-time performance. To address these limitations, this study proposes a novel approach called Sports-ACtrans Net.MethodsIn this approach, the Swin Transformer processes visual data to extract spatial features, while the Spatio-Temporal Graph Convolutional Network (ST-GCN) models human motion as graphs to handle skeleton data. By combining these outputs, a comprehensive representation of motion actions is created. Reinforcement learning is employed to optimize the action recognition process, framing it as a sequential decision-making problem. Deep Q-learning is utilized to learn the optimal policy, thereby enhancing the robot's ability to accurately recognize and engage in motion.Results and discussionExperiments demonstrate significant improvements over state-of-the-art methods. This research advances the fields of neural computation, computer vision, and neuroscience, aiding in the development of intelligent robotic systems capable of understanding and participating in sports activities.

Spatio-temporal Graph Convolutional Networks (ST-GCNs) have shown great performance in the context of skeleton-based action recognition. Nevertheless, ST-GCNs use raw skeleton data as vertex features. Such features have low dimensionality and might not be optimal for action discrimination. Moreover, a single layer of temporal convolution is used to model short-term temporal dependencies but can be insufficient for capturing both long-term. In this paper, we extend the Spatio-Temporal Graph Convolutional Network for skeleton-based action recognition by introducing two novel modules, namely, the Graph Vertex Feature Encoder (GVFE) and the Dilated Hierarchical Temporal Convolutional Network (DH-TCN). On the one hand, the GVFE module learns appropriate vertex features for action recognition by encoding raw skeleton data into a new feature space. On the other hand, the DH-TCN module is capable of capturing both short-term and long-term temporal dependencies using a hierarchical dilated convolutional network. Experiments have been conducted on the challenging NTU RGB-D 60, NTU RGB-D 120 and Kinetics datasets. The obtained results show that our method competes with state-of-the-art approaches while using a smaller number of layers and parameters; thus reducing the required training time and memory.

Spatio-temporal Graph Convolutional Networks (ST-GCNs) have shown great performance in the context of skeleton-based action recognition. Nevertheless, ST-GCNs use raw skeleton data as vertex features. Such features have low dimensionality and might not be optimal for action discrimination. Moreover, a single layer of temporal convolution is used to model short-term temporal dependencies but can be insufficient for capturing both long-term. In this paper, we extend the Spatio-Temporal Graph Convolutional Network for skeleton-based action recognition by introducing two novel modules, namely, the Graph Vertex Feature Encoder (GVFE) and the Dilated Hierarchical Temporal Convolutional Network (DH-TCN). On the one hand, the GVFE module learns appropriate vertex features for action recognition by encoding raw skeleton data into a new feature space. On the other hand, the DH-TCN module is capable of capturing both short-term and long-term temporal dependencies using a hierarchical dilated convolutional network. Experiments have been conducted on the challenging NTU RGB-D 60, NTU RGB-D 120 and Kinetics datasets. The obtained results show that our method competes with state-of-the-art approaches while using a smaller number of layers and parameters; thus reducing the required training time and memory.