GA-STT: Human Trajectory Prediction With Group Aware Spatial-Temporal Transformer

In the field of trajectory prediction, when new scene data is continuously introduced, the system may forget the historical trajectory information that has been learned before, leading to the phenomenon of decreasing the prediction accuracy of previous scenes, which is called catastrophic forgetting. The Multi-Memo Net framework proposed in this paper is an instance-based approach that stores typical trajectories through a memory bank thereby establishing a direct link between the current and historical scenes. During training, self-encoders and decoders are used to efficiently memorize the feature expressions of past trajectories and future intentions; during prediction, the controller finds similar scenes that have occurred in history in the memory bank based on the past trajectories and contextual information for trajectory prediction, and ultimately obtains a multimodal trajectory prediction through K-means clustering, and while predicting, the model can keep updating the content of the memory bank to learn, thus iteratively optimizing the model prediction performance. Thus, the model prediction performance is iteratively optimized. Compared with baseline models such as Kalman filter and Memory Augmented Networks for Multiple Trajectory Prediction (MANTRA) model, the method in this paper has significant improvement in Average Displacement Error (ADE) and Final Displacement Error (FDE) metrics.

Pedestrian trajectory prediction is fundamental to a wide range of scientific research work and industrial applications. Most of the current advanced trajectory prediction methods incorporate context information such as pedestrian neighbourhood, labelled static obstacles, and the background scene into the trajectory prediction process. In contrast to these methods which require rich contexts, the method in our paper focuses on predicting a pedestrian's future trajectory using his/her observed part of the trajectory only. Our method, which we refer to as LVTA, is a Location-Velocity-Temporal Attention LSTM model where two temporal attention mechanisms are applied to the hidden state vectors from the location and velocity LSTM layers. In addition, a location-velocity attention layer embedded inside a tweak module is used to improve the predicted location and velocity coordinates before they are passed to the next time step. Extensive experiments conducted on three large benchmark datasets and comparison with eleven existing trajectory prediction methods demonstrate that LVTA achieves competitive prediction performance. Specifically, LVTA attains 9.19 pixels Average Displacement Error (ADE) and 17.28 pixels Final Displacement Error (FDE) for the Central Station dataset, and 0.46 metres ADE and 0.92 metres FDE for the ETH&UCY datasets. Furthermore, evaluation on using LVTA to generate trajectories of different prediction lengths and on new scenes without the need of retraining confirms that it has good generalizability.

Predicting and planning the future trajectories of various traffic participants is an important task with multiple applications, including autonomous vehicles, service robots, and intelligent transportation. However, the diversity of heterogeneous agents including pedestrians, bicycles, and vehicles in traffic scenarios presents substantial challenges to this task. Current models do not fully capture the implicit and explicit interaction relationships among these heterogeneous agents and often overlook the significance of extracting implicit correlations from agent features. To address these issues, we introduce a novel model for trajectory prediction: Heterogeneous Graph Structure Contrast Optimization (HGSCO). To accurately capture the interaction relationships among heterogeneous agents, HGSCO constructs semantic graph structures representing implicit relationships and meta-path graph structures representing explicit relationships. Then, HGSCO introduces a cross-view contrastive learning approach, which optimizes the heterogeneous graph structure by maximizing mutual information between the two types of graph structures. The model can provide precise interaction relationships among heterogeneous agents by effectively fusing these two graph representations with a gated fusion method. We utilized video data captured by camera sensors in complex environments with multiple agents to conduct experiments. Our proposed model achieved an 11.5% and 6.7% reduction in Average Displacement Error (ADE) across these datasets, respectively, and a reduction of 15.6% and 8.1% in Final Displacement Error (FDE). The results demonstrate that HGSCO significantly surpasses existing state-of-the-art methods regarding trajectory prediction accuracy.

Pedestrian trajectory prediction is a critical research area with numerous domains, e.g., blind navigation, autonomous driving systems, and service robots. There exist two challenges in this research field: spatio-temporal interaction modeling among pedestrians and the uncertainty of pedestrian trajectories. To tackle these challenges, we propose a spatio-temporal interaction aware and trajectory distribution aware graph convolution network. First, we propose a spatio-temporal interaction aware module that integrates a graph convolutional network and self-attention mechanism to model spatio-temporal interactions among pedestrians. Second, we design a trajectory distribution aware module to learn latent trajectory distribution information from the measured trajectories at observed and future times. This can provide knowledge-rich trajectory distribution information for the multimodality of the predicted trajectories. Finally, to address the problem of the propagation and accumulation of prediction errors, we design a trajectory decoder to generate the multimodal future trajectories. The proposed model is evaluated utilizing videos recorded by a camera sensor in crowded areas and can be applied to predict multiple pedestrians’ future trajectories from in-vehicle cameras. Experimental results demonstrate that the proposed approach can achieve superior results on the average displacement error (ADE) and final displacement error (FDE) metrics to state-of-the-art approaches and can predict socially acceptable future trajectories.

Vehicles' trajectory prediction is a topic with growing interest in recent years, as there are applications in several domains ranging from autonomous driving to traffic congestion prediction and urban planning. Predicting trajectories starting from Floating Car Data (FCD) is a complex task that comes with different challenges, namely Vehicle to Infrastructure (V2I) interaction, Vehicle to Vehicle (V2V) interaction, multimodality, and generalizability. These challenges, especially, have not been completely explored by state-of-the-art works. In particular, multimodality and generalizability have been neglected the most, and this work attempts to fill this gap by proposing and defining new datasets, metrics, and methods to help understand and predict vehicle trajectories. We propose and compare Deep Learning models based on Long Short-Term Memory and Generative Adversarial Network architectures; in particular, our GAN-3 model can be used to generate multiple predictions in multimodal scenarios. These approaches are evaluated with our newly proposed error metrics N-ADE and N-FDE, which normalize some biases in the standard Average Displacement Error (ADE) and Final Displacement Error (FDE) metrics. Experiments have been conducted using newly collected datasets in four large Italian cities (Rome, Milan, Naples, and Turin), considering different trajectory lengths to analyze error growth over a larger number of time-steps. The results prove that, although LSTM-based models are superior in unimodal scenarios, generative models perform best in those where the effects of multimodality are higher. Space-time and geographical analysis are performed, to prove the suitability of the proposed methodology for real cases and management services.

Meta-IRLSOT++: A meta-inverse reinforcement learning method for fast adaptation of trajectory prediction networks

Pedestrian trajectory prediction in urban environments has emerged as a critical research area with extensive applications across various domains. Accurate prediction of pedestrian trajectories is essential for the safe navigation of autonomous vehicles and robots in pedestrian-populated environments. Effective prediction models must capture both the spatial interactions among pedestrians and the temporal dependencies governing their movements. Existing research primarily focuses on forecasting a single trajectory per pedestrian, limiting its applicability in real-world scenarios characterised by diverse and unpredictable pedestrian behaviours. To address these challenges, this paper introduces the Graph Convolutional Network, Spatial–Temporal Attention, and Generative Model (GSTGM) for pedestrian trajectory prediction. GSTGM employs a spatiotemporal graph convolutional network to effectively capture complex interactions between pedestrians and their environment. Additionally, it integrates a spatial–temporal attention mechanism to prioritise relevant information during the prediction process. By incorporating a time-dependent prior within the latent space and utilising a computationally efficient generative model, GSTGM facilitates the generation of diverse and realistic future trajectories. The effectiveness of GSTGM is validated through experiments on real-world scenario datasets. Compared to the state-of-the-art models on benchmark datasets such as ETH/UCY, GSTGM demonstrates superior performance in accurately predicting multiple potential trajectories for individual pedestrians. This superiority is measured using metrics such as Final Displacement Error (FDE) and Average Displacement Error (ADE). Moreover, GSTGM achieves these results with significantly faster processing speeds.

Accurate and fast prediction of pedestrian trajectory is very important. However, the interaction between pedestrians is complex, pedestrians are affected not only by their own motion but also by the neighboring pedestrians. Some methods do not extract the pedestrian motion features completely. It is easy to ignore a few of important features of pedestrians. The Spatial-Temporal Attention Graph Convolutional Network (ST-AGNN) for pedestrian trajectory prediction is proposed in this paper. ST-AGNN model can extract the spatial interaction features of each time step and stack the spatial features to obtain the spatial-temporal features. It can predict the future trajectory by using Temporal Convolutional Network (TCN). We evaluate the average displacement error (ADE) and final displacement error (FDE) of the proposed method on ETH and UCY datasets. The experimental results show that the proposed method is effective in accuracy and efficiency.

Pedestrian trajectory prediction enables faster progress in autonomous driving and robot navigation where complex social and environmental interactions involve. Previous models use grid-based pooling or global attention to measure social interactions and use Recurrent Neural Network (RNN) to generate sequences. However, these methods can not extract latent features from temporal and spatial information simultaneously. To address the limitation of previous work, we propose a Self-Centered Star Graph with Attention (SCSG Attention) framework. Firstly, pedestrians’ historical trajectories are encoded. Then multi-head attention mechanism plays a role as enhancement of social interaction awareness and simulation of physical attention from human beings. Lastly, the self-centered star graph decoder can aggregate temporal and spatial features and make predictions. Experiments are conducted on public benchmark datasets and measured with uniform standards. Our results show an improvement over the state-of-the-art algorithms by \(38\%\) on average displacement error (ADE) and \(19\%\) on final displacement error (FDE). Furthermore, it is demonstrated that the star graph has better performance in efficiency of training convergence and ends up with better results in limited time.

Accurate pedestrian trajectory prediction is crucial for applications such as autonomous driving and crowd surveillance. This paper proposes the OV-SKTGCNN model, an enhancement to the Social-STGCNN model, aimed at addressing its low prediction accuracy and limitations in dealing with forces between pedestrians. By rigorously dividing monocular and binocular overlapping visual regions and utilizing different influence factors, the model pedestrian interactions more realistically. The Kolmogorov-Arnold Networks (KANs) combined with Temporal Convolutional Networks (TCNs) greatly improve the ability to extract temporal features. Experimental results on the ETH and UCY datasets demonstrate that the model reduces the Final Displacement Error (FDE) by an average of 23% and the Average Displacement Error (ADE) by 18% compared to Social-STGCNN. The proposed OV-SKTGCNN model demonstrates improved prediction accuracy and better captures the subtleties of pedestrian movements.

Pedestrian trajectory prediction from egocentric monocular video is hindered by camera motion, intermittent occlusions, and complex social interactions. We present NIM-STGCN, a unified framework whose core contribution is a differentiable view normalization(GVN) that couples an enhanced differentiable PnP layer (ED-PnP) with an $$\textrm{SE}(3)$$ SE ( 3 ) warp to align past observations into a single virtual static camera frame. Because GVN is trained end-to-end, forecasting losses back-propagate to pose estimation, yielding geometrically cleaner inputs. On the normalized histories, a lightweight Gated Convolutional Imputation Module (GCIM) recovers missing bounding-box measurements while preserving observed entries, and an efficient spatio-temporal GCN encodes agent dynamics and interactions (optionally augmented by a physics-guided kinematics–interaction prior, PKIM). A Gaussian-mixture predictor produces multi-modal futures and is optimized with a sequence-level negative log-likelihood together with a time-weighted position loss. Extensive experiments on the JAAD and PIE benchmarks show that NIM-STGCN reduces Average Displacement Error (ADE) and Final Displacement Error (FDE) by 12–18 % compared to state-of-the-art methods. Code is available at https://github.com/fantot/NIM-STGCN . Graphical abstract

Forecasting pedestrian trajectories in dynamic scenes remains a critical problem in various applications, such as autonomous driving and socially aware robots. Such forecasting is challenging due to human-human and human-object interactions and future uncertainties caused by human randomness. Generative model-based methods handle future uncertainties by sampling a latent variable. However, few studies explored the generation of the latent variable. In this work, we propose the trajectory predictor with pseudo Oracle (TPPO), which is a generative model-based trajectory predictor. The first pseudo oracle is pedestrians’ moving directions, and the second one is the latent variable estimated from ground truth trajectories. A social attention module is used to aggregate neighbors’ interactions based on the correlation between pedestrians’ moving directions and future trajectories. This correlation is inspired by the fact that pedestrians’ future trajectories are often influenced by pedestrians in front. A latent variable predictor is proposed to estimate latent variable distributions from observed and ground-truth trajectories. Moreover, the gap between these two distributions is minimized during training. Therefore, the latent variable predictor can estimate the latent variable from observed trajectories to approximate that estimated from ground-truth trajectories. We compare the performance of TPPO with related methods on several public datasets. Results demonstrate that TPPO outperforms state-of-the-art methods with low average and final displacement errors. The ablation study shows that the prediction performance will not dramatically decrease as sampling times decline during tests.

Intelligent and ubiquitous systems like smart surveillance systems, autonomous service robots, and crowd-oriented urban infrastructure, among others, require the proper human trajectory forecasting. Nevertheless, most of the current learning-based methods only primarily investigate the social behaviour of agents but disregard the physical restrictions of the environment, and the physically infeasible predictions of the obstacles, walls, or non-walkable areas. These limitations are addressed by this paper via an environment-constrained graph neural network (EC-GNN) framework, in human trajectory forecasting under dynamic crowd behaviour, offered in the paper. The suggested method will represent pedestrian interactions in the form of a dynamic graph where agents take the role of nodes and social influences of the agents is represented by adaptive edge relationships whereas scene-conscious constraints by using structured map and obstacle embedding. The projected socially compliant and physically realistic agent-environment trajectory forecasts of the framework are provided by combining the agent-agent interaction properties and agent-environment representations. The large-scale experiments on the benchmark datasets in crowd trajectory prove that the proposed EC-GNN shows a way to achieve great improvements in the Average Displacement Error (ADE) and Final Displacement Error (FDE) and the rates of collisions and obstacle violations in the face of different crowd densities and the complex layouts of the scene. Thesefindings confirm the fact that the ability to solve problems concerning the environment constraints into the graph-based learning is effective to have robust and scalable human trajectory forecasts in intelligent environments.

The Detection Transformer (DETR) has revolutionized the design of CNN-based object detection systems, showcasing impressive performance. However, its potential in the domain of multi-frame 3D object detection remains largely unexplored. In this paper, we present STEMD, a novel end-to-end framework that enhances the DETR-like paradigm for multi-frame 3D object detection by addressing three key aspects specifically tailored for this task. First, to model the inter-object spatial interaction and complex temporal dependencies, we introduce the spatial-temporal graph attention network, which represents queries as nodes in a graph and enables effective modeling of object interactions within a social context. To solve the problem of missing hard cases in the proposed output of the encoder in the current frame, we incorporate the output of the previous frame to initialize the query input of the decoder. Finally, it poses a challenge for the network to distinguish between the positive query and other highly similar queries that are not the best match. And similar queries are insufficiently suppressed and turn into redundant prediction boxes. To address this issue, our proposed IoU regularization term encourages similar queries to be distinct during the refinement. Through extensive experiments, we demonstrate the effectiveness of our approach in handling challenging scenarios, while incurring only a minor additional computational overhead.

Autonomy is becoming more alluring in today's world due to its applications in various fields such as defense, transportation and exploration of unknown terrain. Autonomy in transportation will be common in the near future and prediction of trajectory helps in decision making. Trajectory prediction is crucial and significant for efficient navigation. The behavior of vehicles is not typical like most other dynamic objects due to physical factors such as vehicle size, driver behavior, etc. and social factors such as avoidance of obstacles, an inter-vehicle distance of neighboring vehicles, etc. data-driven vehicle-trajectory prediction model using Generative Adversarial Networks (GANs) is proposed which uses the partial history of the vehicles and learns the complex behavior of a vehicle. The proposed scheme feeds the model the sequence of vehicle positions as the coordinates that are obtained from a traffic scene and result in the probabilistic vehicle position based on vehicle velocities, prioritization and the above social and physical factors. The model is evaluated on aerial views of traffic data such as VisDrone data set and the TRAF data set. The proposed model can learn vehicle behaviors such as overcoming, merging, etc. Independent of vehicle type, the proposed model predicts the future path of each vehicle in the scene given the history of the vehicle and the results are measured using the average and final displacement errors.