GSTGM: Graph, spatial–temporal attention and generative based model for pedestrian multi-path prediction

Pedestrian trajectory prediction is a critical research area with numerous domains, e.g., blind navigation, autonomous driving systems, and service robots. There exist two challenges in this research field: spatio-temporal interaction modeling among pedestrians and the uncertainty of pedestrian trajectories. To tackle these challenges, we propose a spatio-temporal interaction aware and trajectory distribution aware graph convolution network. First, we propose a spatio-temporal interaction aware module that integrates a graph convolutional network and self-attention mechanism to model spatio-temporal interactions among pedestrians. Second, we design a trajectory distribution aware module to learn latent trajectory distribution information from the measured trajectories at observed and future times. This can provide knowledge-rich trajectory distribution information for the multimodality of the predicted trajectories. Finally, to address the problem of the propagation and accumulation of prediction errors, we design a trajectory decoder to generate the multimodal future trajectories. The proposed model is evaluated utilizing videos recorded by a camera sensor in crowded areas and can be applied to predict multiple pedestrians’ future trajectories from in-vehicle cameras. Experimental results demonstrate that the proposed approach can achieve superior results on the average displacement error (ADE) and final displacement error (FDE) metrics to state-of-the-art approaches and can predict socially acceptable future trajectories.

Pedestrian trajectory prediction is fundamental to a wide range of scientific research work and industrial applications. Most of the current advanced trajectory prediction methods incorporate context information such as pedestrian neighbourhood, labelled static obstacles, and the background scene into the trajectory prediction process. In contrast to these methods which require rich contexts, the method in our paper focuses on predicting a pedestrian's future trajectory using his/her observed part of the trajectory only. Our method, which we refer to as LVTA, is a Location-Velocity-Temporal Attention LSTM model where two temporal attention mechanisms are applied to the hidden state vectors from the location and velocity LSTM layers. In addition, a location-velocity attention layer embedded inside a tweak module is used to improve the predicted location and velocity coordinates before they are passed to the next time step. Extensive experiments conducted on three large benchmark datasets and comparison with eleven existing trajectory prediction methods demonstrate that LVTA achieves competitive prediction performance. Specifically, LVTA attains 9.19 pixels Average Displacement Error (ADE) and 17.28 pixels Final Displacement Error (FDE) for the Central Station dataset, and 0.46 metres ADE and 0.92 metres FDE for the ETH&UCY datasets. Furthermore, evaluation on using LVTA to generate trajectories of different prediction lengths and on new scenes without the need of retraining confirms that it has good generalizability.

Accurate and fast prediction of pedestrian trajectory is very important. However, the interaction between pedestrians is complex, pedestrians are affected not only by their own motion but also by the neighboring pedestrians. Some methods do not extract the pedestrian motion features completely. It is easy to ignore a few of important features of pedestrians. The Spatial-Temporal Attention Graph Convolutional Network (ST-AGNN) for pedestrian trajectory prediction is proposed in this paper. ST-AGNN model can extract the spatial interaction features of each time step and stack the spatial features to obtain the spatial-temporal features. It can predict the future trajectory by using Temporal Convolutional Network (TCN). We evaluate the average displacement error (ADE) and final displacement error (FDE) of the proposed method on ETH and UCY datasets. The experimental results show that the proposed method is effective in accuracy and efficiency.

Predicting and planning the future trajectories of various traffic participants is an important task with multiple applications, including autonomous vehicles, service robots, and intelligent transportation. However, the diversity of heterogeneous agents including pedestrians, bicycles, and vehicles in traffic scenarios presents substantial challenges to this task. Current models do not fully capture the implicit and explicit interaction relationships among these heterogeneous agents and often overlook the significance of extracting implicit correlations from agent features. To address these issues, we introduce a novel model for trajectory prediction: Heterogeneous Graph Structure Contrast Optimization (HGSCO). To accurately capture the interaction relationships among heterogeneous agents, HGSCO constructs semantic graph structures representing implicit relationships and meta-path graph structures representing explicit relationships. Then, HGSCO introduces a cross-view contrastive learning approach, which optimizes the heterogeneous graph structure by maximizing mutual information between the two types of graph structures. The model can provide precise interaction relationships among heterogeneous agents by effectively fusing these two graph representations with a gated fusion method. We utilized video data captured by camera sensors in complex environments with multiple agents to conduct experiments. Our proposed model achieved an 11.5% and 6.7% reduction in Average Displacement Error (ADE) across these datasets, respectively, and a reduction of 15.6% and 8.1% in Final Displacement Error (FDE). The results demonstrate that HGSCO significantly surpasses existing state-of-the-art methods regarding trajectory prediction accuracy.

Accurate pedestrian trajectory prediction is crucial for applications such as autonomous driving and crowd surveillance. This paper proposes the OV-SKTGCNN model, an enhancement to the Social-STGCNN model, aimed at addressing its low prediction accuracy and limitations in dealing with forces between pedestrians. By rigorously dividing monocular and binocular overlapping visual regions and utilizing different influence factors, the model pedestrian interactions more realistically. The Kolmogorov-Arnold Networks (KANs) combined with Temporal Convolutional Networks (TCNs) greatly improve the ability to extract temporal features. Experimental results on the ETH and UCY datasets demonstrate that the model reduces the Final Displacement Error (FDE) by an average of 23% and the Average Displacement Error (ADE) by 18% compared to Social-STGCNN. The proposed OV-SKTGCNN model demonstrates improved prediction accuracy and better captures the subtleties of pedestrian movements.

Pedestrian trajectory prediction enables faster progress in autonomous driving and robot navigation where complex social and environmental interactions involve. Previous models use grid-based pooling or global attention to measure social interactions and use Recurrent Neural Network (RNN) to generate sequences. However, these methods can not extract latent features from temporal and spatial information simultaneously. To address the limitation of previous work, we propose a Self-Centered Star Graph with Attention (SCSG Attention) framework. Firstly, pedestrians’ historical trajectories are encoded. Then multi-head attention mechanism plays a role as enhancement of social interaction awareness and simulation of physical attention from human beings. Lastly, the self-centered star graph decoder can aggregate temporal and spatial features and make predictions. Experiments are conducted on public benchmark datasets and measured with uniform standards. Our results show an improvement over the state-of-the-art algorithms by \(38\%\) on average displacement error (ADE) and \(19\%\) on final displacement error (FDE). Furthermore, it is demonstrated that the star graph has better performance in efficiency of training convergence and ends up with better results in limited time.

Pedestrian trajectory prediction from egocentric monocular video is hindered by camera motion, intermittent occlusions, and complex social interactions. We present NIM-STGCN, a unified framework whose core contribution is a differentiable view normalization(GVN) that couples an enhanced differentiable PnP layer (ED-PnP) with an $$\textrm{SE}(3)$$ SE ( 3 ) warp to align past observations into a single virtual static camera frame. Because GVN is trained end-to-end, forecasting losses back-propagate to pose estimation, yielding geometrically cleaner inputs. On the normalized histories, a lightweight Gated Convolutional Imputation Module (GCIM) recovers missing bounding-box measurements while preserving observed entries, and an efficient spatio-temporal GCN encodes agent dynamics and interactions (optionally augmented by a physics-guided kinematics–interaction prior, PKIM). A Gaussian-mixture predictor produces multi-modal futures and is optimized with a sequence-level negative log-likelihood together with a time-weighted position loss. Extensive experiments on the JAAD and PIE benchmarks show that NIM-STGCN reduces Average Displacement Error (ADE) and Final Displacement Error (FDE) by 12–18 % compared to state-of-the-art methods. Code is available at https://github.com/fantot/NIM-STGCN . Graphical abstract

This work remedies the drawbacks of existing methods in pedestrian trajectory prediction for video surveillance. A traditional two-stage approach, which first detects and tracks pedestrians before forecasting their subsequent trajectories, relies solely on historical trajectories to predict future movements, overlooking the rich contextual information within videos. Although some one-stage methods have been introduced to streamline this process, they still struggle with complex model architectures due to their reliance on pixel-wise flow estimation. To resolve these issues, we propose a simple yet effective Transformer-based one-stage model, called PredDETR, that directly anticipates the future trajectories of multiple pedestrians from videos. Following the end-to-end paradigm of DETR, our PredDETR model not only associates the bounding boxes of each pedestrian across subsequent frames but also predicts their future trajectories using an additional predictive decoder. Taking past raw video frames as input, the proposed PredDETR directly forecasts the future locations of each pedestrian in a non-autoregressive manner. The empirical results validate that, despite its simpler design, PredDETR is a compelling method compared to a previous one-stage approach.

Non-probability sampling network based on anomaly pedestrian trajectory discrimination for pedestrian trajectory prediction

Pedestrian trajectory prediction is an important research area with significant applications in autonomous driving and intelligent surveillance. However, existing studies on pedestrian trajectory prediction often suffer from a noticeable discrepancy between predicted and actual trajectories, due to incomplete extraction of pedestrian trajectory features and the randomness of the pedestrian walking process. The key objective of this article is to address this issue by proposing a method that can reasonably simulate the randomness of pedestrian walking and comprehensively extract pedestrian trajectory features. To achieve this, a novel social informer model built upon the informer model is proposed in this article. The social informer utilizes a transformer encoder-based interaction module to comprehensively extract pedestrian trajectory features, which are input into the informer model for further processing. Additionally, an adaptive variance mechanism is proposed to determine the optimal variance and accurately simulate the random nature of pedestrian walking. Finally, the proposed model is evaluated in a comparative experiment on ETH and UCY datasets, with results demonstrating that the proposed model outperforms other models, exhibiting improved accuracy and performance.

This paper presents a novel framework for accurate pedestrian trajectory prediction in intersection corners or near crosswalks. Given prior knowledge of curbside geometry (i.e. angle made by intersecting curbs at the corner point of interest and the coordinates of the corner itself), the presented framework can accurately predict pedestrian trajectories even in new, unseen intersections. This is achieved by learning motion primitives in a common frame, called the curbside coordinate frame. A key insight in developing this common frame is to ensure that trajectories from intersections with different geometries, representing the same behavior, are spatially similar in the common frame. Motion primitives learned in such a common frame, can then be easily generalized to predict in new intersections, with different geometries than the ones trained on. We test our algorithm on real pedestrian trajectory datasets collected at two intersections, with distinctly different curbside and crosswalk geometries. A comparison of our algorithm with [1] demonstrates improved prediction accuracies of pedestrian trajectory prediction in the case of same training and test intersections, and the improvement of accuracy in the most different training and test intersections scenarios. The result also shows additional context, such as information about pedestrian traffic lights, if available, can be easily incorporated in our prediction model for further improvement in prediction accuracy.

Vehicle trajectory prediction is an important research basis for the decision making and path planning of the intelligent and connected vehicle. In the connected vehicle environment, vehicles share information and drive cooperatively, and the intelligent and connected vehicles are able to obtain more accurate and rich perception information, which provides a data basis for accurate prediction of vehicle trajectories. However, attaining accurate and effective vehicle trajectory predictions poses technical challenges due to insufficient extraction of vehicular spatial–temporal interaction features. In this paper, we propose a vehicle trajectory prediction model based on graph convolutional neural network (GCN) in a connected vehicle environment. Specifically, using the driving scene information obtained by the intelligent and connected vehicle, the spatial graph and temporal graph are constructed based on the spatial interaction coefficient (SIC) and self-attention mechanism, respectively. Then, the graph data are entered into the interaction extraction module, and the spatial interaction features and temporal interaction features are extracted separately using the graph convolutional networks, which are fused to obtain the spatial–temporal interaction information. Finally, the interaction features are learned based on the convolutional neural networks to output the future trajectory information of all vehicles in the scene by one forward operation rather than a step-by-step process. The ablation experiment results show that the method proposed in this study to model the spatiotemporal interaction among vehicles based on SIC and self-attention mechanism reduces the prediction error by 5% and 12%, respectively. The results from the model comparison experiment show that the proposed method engenders an 8% improvement in prediction accuracy over the state-of-the-art solution, providing technical and theoretical support for trajectory prediction research of intelligent and connected vehicles.

In autonomous driving scenarios, pedestrian trajectory prediction is an important research direction. Based on the spatio-temporal graph convolutional neural network, we propose a new pedestrian trajectory prediction algorithm. The new algorithm constructs a series of new models around pedestrian intention estimation. The construction of the estimation algorithm considers the following aspects: the contextual information of pedestrians and the surrounding environment, the “pedestrian ego-vehicle” interaction combined with the vehicle speed estimation, the pedestrian’s own skeletal structure information and body language estimation, which includes head joints and the relative structural relationship of the torso joints, including whether it is out of the same plane, is rotated, and so on. Skeleton information feature extraction and construction adopts the method of graph convolutional neural network to structure pedestrians into joints in the form of graphs in non-Euclidean space, and further adopts spatial temporal graph convolutional network for feature extraction and learning. The new method is named a “head-torso”-based spatial temporal graph convolutional network (HT-STGCN). On the dataset PID, the novel method achieves substantial improvements over mainstream methods. Experimental results show that combining HT-STGCN with observed action can improve trajectory prediction.

Pedestrian trajectory prediction can provide decision support for self-driving cars and robots to avoid potential collision risks. Modelling of social relationships and multimodal prediction are core challenges in pedestrian trajectory prediction. However, there are two problems with existing methods: (1) these methods usually focus on modelling pairwise or group interactions, ignoring the importance of high-order interactions; and (2) existing methods struggle to balance the performance between deterministic and multimodal predictions. To address these issues, we propose a higher-order graph attention network for pedestrian trajectory prediction. Specifically, we first construct spatial and temporal graphs to model social interactions and temporal dependencies. To achieve high-order interaction modelling, we design a higher-order graph attention module to capture pedestrian social and temporal interactions of different orders by stacking multiple layers of graph attention. Subsequently, we design a deformable spatial-temporal sparse module to adaptively eliminate social and temporal redundant interactions. To balance deterministic and multimodal prediction performance, we design a two-stage prediction strategy. The first stage predicts pedestrian deterministic trajectories, and the second stage refines pedestrian deterministic trajectories to complete both deterministic and multimodal predictions. Quantitative analysis on the ETH, UCY, and SDD datasets demonstrates that our method outperforms existing methods in both deterministic and multimodal predictions. Qualitative analysis shows that our method can effectively describe complex social relationships and accomplish reliable trajectory prediction.

This article introduces a novel model for low-quality pedestrian trajectory prediction, the social nonstationary transformers (NSTransformers), that merges the strengths of NSTransformers and spatiotemporal graph transformer (STAR). The model can capture social interaction cues among pedestrians and integrate features across spatial and temporal dimensions to enhance the precision and resilience of trajectory predictions. We also propose an enhanced loss function that combines diversity loss with logarithmic root mean squared error (log-RMSE) to guarantee the reasonableness and diversity of the generated trajectories. This design adapts well to complex pedestrian interaction scenarios, thereby improving the reliability and accuracy of trajectory prediction. Furthermore, we integrate a generative adversarial network (GAN) to model the randomness inherent in pedestrian trajectories. Compared to the conventional standard Gaussian distribution, our GAN approach better simulates the intricate distribution found in pedestrian trajectories, enhancing the trajectory prediction's diversity and robustness. Experimental results reveal that our model outperforms several state-of-the-art methods. This research opens the avenue for future exploration in low-quality pedestrian trajectory prediction.