Goal-Guided and Interaction-Aware State Refinement Graph Attention Network for Multi-Agent Trajectory Prediction

Occupant behavioral patterns, once extracted, could reveal cues about activities and space usage that could effectively get used for building systems to achieve energy savings. The ability to accurately predict the trajectories of occupants inside a room branched into different zones has many notable and compelling applications. For example - efficient space utilization and floor plans, intelligent building operations, crowd management, comfortable indoor environment, security, and evacuation or managing personnel. This paper proposes future occupant trajectory prediction using state-of-the-art time series prediction methods, i.e., Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) models. These models are being implemented and compared to forecast occupant trajectories at a given time and location in a non-intrusive and reliable manner. The considered test-space for the collection of the dataset is a multi-utility area in an instrumented public building. The deployed 3D Stereo Vision Cameras capture the spatial location coordinates (x- and y- coordinates) from a bird's view angle without eliciting any other information that could reveal confidential data or uniquely identify a person. Our results showed that the GRU model forecasts were considerably more accurate than the LSTM model for the trajectory prediction. GRU prediction model achieved a Mean Squared Error (MSE) of 30.72 cm between actual and predicted location coordinates, and LSTM achieved an MSE of 47.13 cm, respectively, for multiple occupant trajectories within the monitored area. Another evaluation metric Mean Absolute Error (MAE) is used, and the GRU prediction model achieved an MAE of 3.14 cm, and the LSTM model achieved an MAE of 4.07 cm. The GRU model guarantees a high-fidelity occupant trajectory prediction for any given case with higher accuracy when compared to the baseline LSTM model.

Trajectory prediction is critical for ensuring the safety, reliability, and scalability of Unmanned Aerial Vehicle (UAV) in urban environments. Despite advances in deep learning, existing methods often struggle with dynamic UAV conditions, such as rapid directional changes and limited forecasting horizons, while lacking comprehensive real-time validation and generalization capabilities. This study addresses these challenges by proposing a gated recurrent unit (GRU)-based deep learning framework optimized through Look_Back and Forward_Length labeling to capture complex temporal patterns. The model demonstrated state-of-the-art performance, surpassing existing unmanned aerial vehicles (UAV) and aircraft trajectory prediction approaches, including FlightBERT++, in terms of both accuracy and robustness. It achieved reliable long-range predictions up to 4 s, and its real-time feasibility was validated due to its efficient resource utilization. The model’s generalization capability was confirmed through evaluations on two independent UAV datasets, where it consistently predicted unseen trajectories with high accuracy. These findings highlight the model’s ability to handle rapid maneuvers, extend prediction horizons, and generalize across platforms. This work establishes a robust trajectory prediction framework with practical applications in collision avoidance, mission planning, and anti-drone systems, paving the way for safer and more scalable UAV operations.

Understanding crowd motion dynamics and forecasting the future pedestrian trajectories are critical to various applications, e.g. autonomous driving and surveillance system. This task is challenging because when pedestrians plan the future paths in real crowd scenes, they will distinguish the priorities of following their predetermined destinations and responding to the motion behaviors of neighboring pedestrians. However, most of the existing methods ignore the problem of intention-interaction trade-off. In this paper, we tackle this problem by a hierarchical network, which achieves dynamically reasoning predetermined destinations and future trajectories. A novel graph structure called Intention-Interaction Graph (IIG) is designed to jointly model the self intentions and social interactions. To aggregate information in IIG, Interaction Gated Graph Attention Networks (IGGAN) consisting of a gate mechanism and an attention mechanism is proposed, thus achieving reasoning the influence degree of neighboring pedestrians and destinations. Experimental results on multiple widely used pedestrian trajectory prediction datasets, including two datasets in ETH and three datasets in UCY, demonstrate the effectiveness of the proposed model.

Forecasting pedestrian trajectories in dynamic scenes remains a critical problem in various applications, such as autonomous driving and socially aware robots. Such forecasting is challenging due to human-human and human-object interactions and future uncertainties caused by human randomness. Generative model-based methods handle future uncertainties by sampling a latent variable. However, few studies explored the generation of the latent variable. In this work, we propose the trajectory predictor with pseudo Oracle (TPPO), which is a generative model-based trajectory predictor. The first pseudo oracle is pedestrians’ moving directions, and the second one is the latent variable estimated from ground truth trajectories. A social attention module is used to aggregate neighbors’ interactions based on the correlation between pedestrians’ moving directions and future trajectories. This correlation is inspired by the fact that pedestrians’ future trajectories are often influenced by pedestrians in front. A latent variable predictor is proposed to estimate latent variable distributions from observed and ground-truth trajectories. Moreover, the gap between these two distributions is minimized during training. Therefore, the latent variable predictor can estimate the latent variable from observed trajectories to approximate that estimated from ground-truth trajectories. We compare the performance of TPPO with related methods on several public datasets. Results demonstrate that TPPO outperforms state-of-the-art methods with low average and final displacement errors. The ablation study shows that the prediction performance will not dramatically decrease as sampling times decline during tests.

Background: Today, self-driving cars are already on the roads. However, driving safety remains a huge challenge. Trajectory prediction of traffic targets is one of the important tasks of an autonomous driving environment perception system, and its output trajectory can provide necessary information for decision control and path planning. Although there are many patents and articles related to trajectory prediction, the accuracy of trajectory prediction still needs to be improved. Objective: This paper aimed to propose a novel scheme that considers multi-feature independent encoding trajectory prediction (MFIE). Methods: MFIE is an independently coded trajectory prediction algorithm that consists of a spacetime interaction module and trajectory prediction module, and considers speed characteristics and road characteristics. In the spatiotemporal interaction module, an undirected and weightless static traffic graph is used to represent the interaction between vehicles, and multiple graph convolution blocks are used to perform data mining on the historical information of target vehicles, capture temporal features, and process spatial interaction features. In the trajectory prediction module, three long short-term memory (LSTM) encoders are used to encode the trajectory feature, motion feature, and road constraint feature independently. The three hidden features are spliced into a tensor, and the LSTM decoder is used to predict the future trajectory. Results: On datasets, such as Apollo and NGSIM, the proposed method has shown lower prediction error than traditional model-driven and data-driven methods, and predicted more target vehicles at the same time. It can provide a basis for vehicle path planning on highways and urban roads, and it is of great significance to the safety of autonomous driving. Conclusion: This paper has proposed a multi-feature independent encoders’ trajectory prediction data-driven algorithm, and the effectiveness of the algorithm is verified with a public dataset. The trajectory prediction algorithm considering multi-feature independent encoders provides some reference value for decision planning.

Future mobile networks are envisioned to have critical limitations in terms of latency, energy usage, capacity and network resources since these networks are expected to become extremely dense and complex. The rapid enormous advances in recent technologies such as Internet of Things (IoT) highlights the urgent need for network performance enhancement as well. To this end, self-organizing networks are a promising solution to push the network performance to the next level. These scalable networks can dynamically adapt to possible changes in the network. Smart mobility management, in particular mobility prediction, is a subsection of self-organizing functions which are mainly based on the machine learning techniques. In this paper, we propose to estimate user’s future trajectory using machine learning approaches for a better network management. We propose a novel bidirectional trajectory prediction model called BTPM to model the user mobility behavior. The proposed method exploits the potential benefits of bidirectional gated recurrent unit (GRU) for having an accurate prediction. Moreover, we introduce a data preprocessing phase to obtain better results with significantly lower execution time. The proposed approach takes full advantage of data analysis in both directions (backward and forward) in order to provide a long-term prediction and model user’s mobility even with complex patterns. Experimental results show that the proposed bidirectional approach significantly improves the performance of the mobility predictor in terms of model accuracy, robustness and execution time. It achieves a model error of 0.014 and decreases the execution time up to 97%.

Trajectory prediction of a moving object has imperative significance in both research and practical applications, ranging from target tracking, security surveillance and autonomous vehicle driving. For improving the efficacy of such prediction, a novel approach of data stream learning coupled with Kalman-filter and evolving correlated horizons feature selection (KF-ECH-FS) is proposed. KF has traditionally been used as a control-feedback-loop mechanism that corrects the errors from past trials, to predict the next-step position in trajectory prediction. In our fusion model here, KF and its windowed version are being used as predictor variables in a multivariate time series forecasting process. The predictor variables which serve as additional features aid in improving the trajectory prediction when only the relevant features are being selected in the incremental learning manner by multi-variate data stream analytics. Our proposed ECH-FS solves the problem of model overfitting when many features after expansion by time-series windowing are evaluated and selected along the learning process. A simple and efficient feature selection heuristics, Auto-encoder is used, along with data stream learning by Gate Recurrent Unit. The results, through an experimentation over a sample case of camera surveillance of accident prevention, show that our proposed KF-ECH-FS is superior to either KF or windowing alone in 1-step horizon trajectory prediction.

For autonomous driving, accurate trajectory prediction is paramount, necessitating effective harnessing of spatiotemporal data. This study proposes an innovative Spatiotemporal Transformer-based model, enhancing trajectory prediction precision by leveraging a multi-head self-attention mechanism. This mechanism intricately captures both inter-vehicular interactions and temporal trajectory dependencies. The model, structured around an LSTM-based encoder-decoder framework, innovatively considers spatial interactions among observed and future trajectories during the decoding process. Evaluated on the NGSIM dataset, our model demonstrates superior predictive accuracy over existing methodologies, underscoring the efficacy of Spatiotemporal Transformers in complex dynamic environments.

Predicting the future trajectories of target agents is critical for the safe operation of autonomous vehicles in complex environments. Accurate trajectory prediction requires the extraction of both temporal and spatial information. To address the limitations of existing static attention methods, which are constrained by fixed historical ranges and spatial radii under limited computational resources, a novel trajectory prediction approach using dynamic spatio-temporal attention is proposed. DSTNet incorporates a multi-scale spatial sparse attention module and a multi-scale historical causal attention module, both aimed at dynamically encoding the spatio-temporal dependencies of the data. These modules are unified with multi-modal interactive attention in a three-attention spatio-temporal module. Additionally, DSTNet includes an anchor-based trajectory refinement module, which adaptively optimizes the coarse-grained trajectories generated by the backbone network. Compared with state-of-the-art methods, DSTNet demonstrates outstanding performance on the Argoverse 1 and Argoverse 2 motion prediction benchmarks, delivering accurate and stable future trajectories. Notably, the model has been successfully deployed on an autonomous vehicle.

Meta-IRLSOT++: A meta-inverse reinforcement learning method for fast adaptation of trajectory prediction networks

Aircraft trajectory prediction is a challenging problem in air traffic control, especially for conflict detection. Traditional trajectory predictors require a variety of inputs such as flight-plans, aircraft performance models, meteorological forecasts, etc. Many of these data are subjected to environmental uncertainties. Further, limited information about such inputs, especially the lack of aircraft tactical intent, makes trajectory prediction a challenging task. In this work, we propose a deep learning model that performs trajectory prediction by modeling and incorporating aircraft tactical intent. The proposed model adopts the encoder-decoder architecture and makes use of the convolutional layer as well as Gated Recurrent Units (GRUs). The proposed model does not require explicit information about aircraft performance and wind data. Results demonstrate that the provision of enriched aircraft intent, together with appropriate model design, could improve the prediction error up to 30% at a prediction horizon of 10 minutes (from 4.9 nautical miles to 3.4 nautical miles). The model also guarantees the mean error growth rate with increasing look-ahead time to be lower than 0.2 nautical miles per minute. In addition, the model offers a very low variance in the prediction, which satisfies the variance-standard specified by EUROCONTROL (EU Organization for Safety and Navigation of Air Traffic) for trajectory predictors. The proposed model also outperforms the state-of-the-art trajectory prediction model, where the Root Mean Square Error (RMSE) is reduced from 0.0203 to 0.0018 for latitude prediction, and from 0.0482 to 0.0021 for longitude prediction in a single prediction step of 15 seconds look-ahead. We showed that the pre-trained model on ADS-B data maintains its high performance, in terms of cross-track and along-track errors, when being validated in the Bluesky Air Traffic Simulator. The proposed model would significantly improve the performance of conflict detection systems where such trajectory prediction models are needed.

Pedestrian behavior anticipation is a key challenge in the design of assistive and autonomous driving systems suitable for urban environments. An intelligent system should be able to understand the intentions or underlying motives of pedestrians and to predict their forthcoming actions. To date, only a few public datasets were proposed for the purpose of studying pedestrian behavior prediction in the context of intelligent driving. To this end, we propose a novel large-scale dataset designed for pedestrian intention estimation (PIE). We conducted a large-scale human experiment to establish human reference data for pedestrian intention in traffic scenes. We propose models for estimating pedestrian crossing intention and predicting their future trajectory. Our intention estimation model achieves 79% accuracy and our trajectory prediction algorithm outperforms state-of-the-art by 26% on the proposed dataset. We further show that combining pedestrian intention with observed motion improves trajectory prediction. The dataset and models are available at http://data.nvision2.eecs.yorku.ca/PIE_dataset/.

Pedestrian trajectory prediction is a critical research area with numerous domains, e.g., blind navigation, autonomous driving systems, and service robots. There exist two challenges in this research field: spatio-temporal interaction modeling among pedestrians and the uncertainty of pedestrian trajectories. To tackle these challenges, we propose a spatio-temporal interaction aware and trajectory distribution aware graph convolution network. First, we propose a spatio-temporal interaction aware module that integrates a graph convolutional network and self-attention mechanism to model spatio-temporal interactions among pedestrians. Second, we design a trajectory distribution aware module to learn latent trajectory distribution information from the measured trajectories at observed and future times. This can provide knowledge-rich trajectory distribution information for the multimodality of the predicted trajectories. Finally, to address the problem of the propagation and accumulation of prediction errors, we design a trajectory decoder to generate the multimodal future trajectories. The proposed model is evaluated utilizing videos recorded by a camera sensor in crowded areas and can be applied to predict multiple pedestrians’ future trajectories from in-vehicle cameras. Experimental results demonstrate that the proposed approach can achieve superior results on the average displacement error (ADE) and final displacement error (FDE) metrics to state-of-the-art approaches and can predict socially acceptable future trajectories.

Accurate prediction of mesoscale eddy trajectories requires efficient models with large-size available data instances to capture the main eddy characteristics. However, there is a lack of salient attention mechanisms that can recognize the demand of extracting the aggregated features over multidimensional eddy data sources. Additionally, deep learning techniques are very important for eddy trajectory prediction to dynamically capture properties of mesoscale eddies in the South China Sea. In this article, we propose a spatio-temporal attention-based deep learning framework that can orchestrate heterogeneous data integration and propagation trajectory forecast together. It consists of a novel autoencoder equipped with channel and spatial attention mechanisms (CSA-encoder), and a gated recurrent unit (GRU) network with temporal attention layer (TA-GRU). CSA-encoder compresses stereoscopic eddy data with convolutional layers and generates the small-scale and high-quality dataset as the input of TA-GRU. The finer grained TA-GRU method is extended to accurately predict eddy trajectories with more valuable imagery information so that the temporal attention mechanism can automatically select relevant regions within the next 14 days. Our cross-validation results demonstrate that our framework averagely achieves a lower distance error (9 km) and 54% performance improvement over the baseline GRU technique in the next one day, and outperforms two state-of-the-art techniques of long short-term memory and recurrent neural network by 54.9% and 65.6%, respectively.

Real-time stochastic multi-ship trajectory modeling is crucial for maritime safety. However, it remains challenging due to the uncertainty of dynamic vessel intentions and their complex interactions. Most existing studies rely on deterministic social data from historical time steps for modeling, which often fail to capture the future states of interacting ships, leading to unrealistic trajectory overlaps. Recent research has demonstrated that diffusion models excel in trajectory prediction due to their high generation quality, training stability, and diversity. However, their slow sampling speed limits real-time perception in maritime environments, as generating high-quality trajectories typically requires hundreds of denoising steps. To address these challenges, we propose a Multi-Ship Future interaction trajectory prediction approach based on a Pre-initializer Diffusion model (MFPD). By training a parameterized pre-initializer to directly learn the joint distribution of multiple denoising steps in the reverse diffusion process, our method significantly reduces the time cost of denoising while retaining only a few steps for fine-tuning the distribution. Specifically, in addition to encoding historical trajectory information and social interactions as state embeddings, we also incorporate future trajectory and multimodal maritime environmental information as input condition embeddings to fully capture potential future interactions and environmental features. Experimental results demonstrate that the proposed model significantly improves performance on two real-world datasets while greatly accelerating the sampling speed, demonstrating the superiority in real-world maritime environments.