Freeway Trajectory Prediction via SpatiotemporalTransformers

In vehicle trajectory prediction, constructing the interactive relationships among vehicles within the traffic environment poses a significant challenge. Existing models predominantly focus on temporal dependencies within vehicle histories and spatial correlations among neighboring vehicles, overlooking the continuous influence of historical vehicle states on the current time step and the interplay of multiple sequences over time. To address these limitations, we propose a method for multimodal vehicle trajectory prediction that integrates Hybrid Time-step Scene Interaction (HTSI) into the spatiotemporal relationships. Firstly, we introduce the HTSI module, comprising Multi-step Temporal Information Aggregation (MTIA) and Single-step Temporal Information Aggregation (STIA) methods. MTIA utilizes multi-head attention mechanisms to capture temporal dependencies between consecutive frames, thereby generating new time series amalgamating the ongoing influence of historical time states on the current timestamp. Simultaneously, STIA employs multi-head attention mechanisms to capture the spatial dimension weights of multiple time series and, by aggregating spatial interaction features at each timestamp, generates new time series fused with spatial interaction influences. Subsequently, feature extraction is performed through LSTM layers. Moreover, we propose an improved DIPM pooling module, improving the model’s long-term prediction capability by selectively reusing historical hidden states. Ultimately, based on training results from the HighD and NGSIM datasets, our model demonstrates significant advantages in long-term prediction compared to other state-of-the-art trajectory prediction models. Specifically, within the 5 s prediction window, the model achieved a root mean square error (RMSE) of 2.79 m on the NGSIM dataset, representing a 33.62% improvement over the baseline model’s average accuracy. Additionally, on the HighD dataset, the model attained an RMSE of 2.16 m, reflecting a 33.43% enhancement. The crucial code can be obtained from the provided link: https://github.com/gyhhq/Prediction-trajectory .

Background: Today, self-driving cars are already on the roads. However, driving safety remains a huge challenge. Trajectory prediction of traffic targets is one of the important tasks of an autonomous driving environment perception system, and its output trajectory can provide necessary information for decision control and path planning. Although there are many patents and articles related to trajectory prediction, the accuracy of trajectory prediction still needs to be improved. Objective: This paper aimed to propose a novel scheme that considers multi-feature independent encoding trajectory prediction (MFIE). Methods: MFIE is an independently coded trajectory prediction algorithm that consists of a spacetime interaction module and trajectory prediction module, and considers speed characteristics and road characteristics. In the spatiotemporal interaction module, an undirected and weightless static traffic graph is used to represent the interaction between vehicles, and multiple graph convolution blocks are used to perform data mining on the historical information of target vehicles, capture temporal features, and process spatial interaction features. In the trajectory prediction module, three long short-term memory (LSTM) encoders are used to encode the trajectory feature, motion feature, and road constraint feature independently. The three hidden features are spliced into a tensor, and the LSTM decoder is used to predict the future trajectory. Results: On datasets, such as Apollo and NGSIM, the proposed method has shown lower prediction error than traditional model-driven and data-driven methods, and predicted more target vehicles at the same time. It can provide a basis for vehicle path planning on highways and urban roads, and it is of great significance to the safety of autonomous driving. Conclusion: This paper has proposed a multi-feature independent encoders’ trajectory prediction data-driven algorithm, and the effectiveness of the algorithm is verified with a public dataset. The trajectory prediction algorithm considering multi-feature independent encoders provides some reference value for decision planning.

Accurate trajectory prediction of surrounding road users is critical to autonomous driving systems. In mixed traffic flows, road users with different kinds of behaviors and styles bring complexity to the environment, which requires considering interactions among road users when anticipating their future trajectories. This paper presents a long-term interactive trajectory prediction method for surrounding vehicles using a hierarchical multi-sequence learning network. In contrast to non-interactive method which assumes that road users are independent of each other, this method can automatically learn high-level dependencies among multiple interacting vehicles through the proposed structural-LSTM (long short-term memory) network. Specifically, structural-LSTM first assigns one LSTM for each interacting vehicle. Then these LSTMs share their cell states and hidden states with their spatial-neighboring LSTMs by a radial connection, and recurrently analyze the output state of itself as well as the other LSTMs in a deeper layer. Finally based on all output states, the network predicts trajectories for surrounding vehicles. The proposed method is evaluated on the NGSIM dataset, and its results show that satisfyingly accurate prediction performance of long-term trajectories of surrounding vehicles is accessible, e.g., longitudinal and lateral RMS error can be reduced to less than 1.93m and 0.31m over 5s time horizon, respectively.

Vehicle trajectory information is a crucial part of improving the efficiency and the safety of the ITS. Data missing or irregular sampling in the real-world road traffic makes it hard to obtain accurate real-time vehicle trajectories. In this paper, we focus on trajectory imputation and prediction tasks with small data (magnitude set as 10 1 and 10 2 ). Limited by insufficient data, the simulation results of the existing data-driven algorithms are unsatisfactory. With car-following models integrated as prior physical information to constrain the training process, we design the car-following-informed neural network (CFINN). A multi-head self-attention layer is attached to the fully connected network layer to extract vehicle features. Different from the structure of most neural networks in regression analysis, an extra physics-based dataset is constructed in the CFINN. The loss function consists of two parts including the given trajectory's fitting error and the generated trajectory's residual error. We embed the gated recurrent unit-based encoder–decoder layer to the CFINN framework for trajectory predictions. The rationality and the superiority of our model are validated on the NGSIM dataset and the HighD dataset. Compared with baseline models in both single-vehicle and queue-typed trajectory imputation experiments, lower error can be achieved via the CFINN and coefficients of car-following models can be calibrated. According to driving regimes derived from CFINN-based trajectory prediction experiments, we discuss the impact of cut-in behaviours on the target vehicle and carry out the kinetic analysis. The novel neural network model driven by both data and physical knowledge provides technical support in vehicle status assessments and trajectory predictions.

Occupant behavioral patterns, once extracted, could reveal cues about activities and space usage that could effectively get used for building systems to achieve energy savings. The ability to accurately predict the trajectories of occupants inside a room branched into different zones has many notable and compelling applications. For example - efficient space utilization and floor plans, intelligent building operations, crowd management, comfortable indoor environment, security, and evacuation or managing personnel. This paper proposes future occupant trajectory prediction using state-of-the-art time series prediction methods, i.e., Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) models. These models are being implemented and compared to forecast occupant trajectories at a given time and location in a non-intrusive and reliable manner. The considered test-space for the collection of the dataset is a multi-utility area in an instrumented public building. The deployed 3D Stereo Vision Cameras capture the spatial location coordinates (x- and y- coordinates) from a bird's view angle without eliciting any other information that could reveal confidential data or uniquely identify a person. Our results showed that the GRU model forecasts were considerably more accurate than the LSTM model for the trajectory prediction. GRU prediction model achieved a Mean Squared Error (MSE) of 30.72 cm between actual and predicted location coordinates, and LSTM achieved an MSE of 47.13 cm, respectively, for multiple occupant trajectories within the monitored area. Another evaluation metric Mean Absolute Error (MAE) is used, and the GRU prediction model achieved an MAE of 3.14 cm, and the LSTM model achieved an MAE of 4.07 cm. The GRU model guarantees a high-fidelity occupant trajectory prediction for any given case with higher accuracy when compared to the baseline LSTM model.

Accurate trajectory prediction is essential for ensuring flight safety, autonomous navigation, and efficient control of unmanned aerial vehicles (UAVs). However, existing neural network approaches often focus on a single platform, limiting their generalizability across UAVs with different dynamics. This study evaluates three neural architectures for spatiotemporal trajectory prediction using high-frequency flight data. Two UAV configurations, such as a fixed-wing (Cessna) and a multirotor (Phantom), were simulated in XPlane 11 under identical environmental conditions. Flight data of latitude, longitude, and altitude sampled at 80 Hz were preprocessed using a sliding-window segmentation method to preserve temporal dependencies. Backpropagation Neural Network (BPNN), Recurrent Neural Network (RNN), and Long Short-Term Memory (LSTM) models were trained using standardised preprocessing and tuned hyperparameters. Results show that recurrent models outperform the feedforward BPNN in capturing temporal patterns. For the Cessna dataset, the RNN achieved balanced accuracy with a mean-squared error (MSE) of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$5.5 \times 10^{-8}$</tex>, while the LSTM obtained the lowest MSE (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$2.7 \times 10^{-8}$</tex>). In the more dynamic Phantom dataset, the BPNN remained competitive (MSE <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\approx 1.8 \times 10^{-7}$</tex>), whereas RNN and LSTM maintained superior altitude prediction. These results highlight the suitability of lightweight recurrent architectures for real-time UAV trajectory prediction.

Accurate trajectory prediction plays a crucial role in optimizing the performance of Internet of Vehicles (IoV) systems, reducing data transmission overhead, and enhancing communication network security. However, the expanding sensing range in IoV has led to increasingly complex spatial-temporal interactions, posing significant challenges for future trajectory prediction endeavors. Currently, predominant approaches involve constructing spatial-temporal interactions through various attention mechanisms. Nevertheless, these methods often yield numerous redundant interactions, potentially resulting in unstable predictions and diffuse interactions. Consequently, there is a pressing need to enhance the application of these methods in trajectory prediction within IoV contexts. Motivated by these challenges, our work introduces a sparse spatial-temporal Transformer (SSTT) to predict vehicle trajectories. SSTT consists of two main Transformer modules: the sparse spatial Transformer and the local-global temporal Transformer. We integrate a learnable sparse plugin into the former to minimize extraneous information in spatial interactions. This plugin enables SSTT to focus more effectively on critical interactive neighbor vehicles by optimizing attention weight distribution, thereby enhancing optimization convergence and prediction accuracy. For the latter, local time windows are employed to capture temporal local correlations and extend the attentional receptive field. Experimental results conducted on three real-world datasets demonstrate that SSTT achieves state-of-the-art performance, and even when only 15% of the training data is used, it can still outperform SOAT baselines. This study presents novel ideas and methodologies for advancing trajectory prediction techniques within the IoV paradigm. The code and our model will be available at GitHub.

Objectives With the continuous advancement of autonomous driving technology, accurate vehicle trajectory prediction plays a crucial role in preventing potential traffic collisions. To address the limitations of existing methods—specifically the insufficient exploitation of historical trajectory information and the inaccurate modeling of multi-vehicle interactions—this study proposes a method that combines multi-scale feature extraction with trajectory optimization, enabling precise prediction of target vehicle trajectories. Methods Targeting urban roads and highway scenarios with naturalistic driving data, surrounding interactive vehicles are first identified using the Pearson correlation coefficient, and historical feature inputs are selected via the Maximal Information Coefficient (MIC). The proposed CTS-Informer model integrates CNN and TCN decoders to capture local and long-range spatiotemporal dependencies from historical trajectories. It further leverages Informer’s sparse attention and self-attention distillation to model global motion trends. Finally, a trajectory correction and smoothing module refines the output for enhanced continuity and realism in the predicted target vehicle trajectory. Results The proposed model was validated on the NGSIM dataset, demonstrating superior performance over existing baseline methods. For a 5-s prediction horizon, the model achieved a 25–48% reduction in ADE and a 12–42% reduction in RMSE. Component ablation experiments further showed that removing the CNN, TCN, or trajectory correction and smoothing module led to a performance drop of 12.8–32.6%, highlighting the contribution of each module to overall accuracy. Conclusions The study demonstrates that the CTS-Informer model delivers strong long-term prediction performance, with each component contributing effectively to overall accuracy. The results validate the effectiveness of multi-scale spatiotemporal feature fusion for extended-horizon prediction. Furthermore, the model provides reliable trajectory boundaries, providing robust reference input for autonomous driving decision-making systems.

Trajectory prediction for challenging scenarios has always been a significant problem in the field due to the complexity of dynamic scenarios and interactions. Furthermore, there is often a dynamic gap between evaluating and validating methods on fixed datasets and real driving scenarios. This letter forms part of a series of reports emanating from the IEEE Transactions on Intelligent Vehicles's Decentralized and Hybrid Workshops (DHW) dedicated to the field of Scenarios Engineering. Our research proposes a scenario engineering-based calibration and validation framework for trajectory prediction of autonomous vehicles to more effectively validate the performance of the method in challenging scenarios. First, Scenarios Engineering (SE) uses OpenSCENARIO and real dataset to generate high-definition maps for challenging driving scenarios. Then, the vectorization approach is employed to extract contextual details from the scene and agent trajectory information from the HD map, and the graph neural network is used to model the high-order interaction to realize the interactive trajectory prediction. Compared with the traditional method, the trajectory prediction can be calibrated through SE so that the prediction process can use more traffic information and attribute characteristics, and improve the evaluation index of prediction. The DHW discusses a practical case to verify the potential of the trajectory prediction framework based on scenarios generation in improving the authenticity and accuracy of trajectory prediction.

Vehicle trajectories at urban road intersections are influenced by diverse spatiotemporal factors, with significant variations across different locations. To address these complexities, real-world trajectory data were collected from intersections in Xi’an, Shaanxi Province, China, and raw trajectories were extracted using the You Only Look Once v8 and BoT-SORT algorithms. This study presents the Spatiotemporal Generative Attention Network (STGANT), a novel generative adversarial network-based framework designed specifically for vehicle trajectory prediction at intersections. The STGANT integrates innovative mechanisms, such as agent attention, social pooling, and Long Short-Term Memory, capturing spatial and temporal dependencies among vehicles. By incorporating an improved attention module, STGANT achieves a balance between accuracy and computational efficiency, enabling region-specific training while reducing overall computational load. Comprehensive experiments demonstrate that the STGANT outperforms existing models for prediction accuracy, computational complexity, and parameter efficiency. Qualitative analysis further verifies that the STGANT effectively captures the intricate spatiotemporal dynamics of vehicle interactions, resulting in reliable and accurate trajectory predictions. This study contributes to the enhancement of safety and operational efficiency in intelligent transportation systems at urban intersections.

Accurate trajectory prediction is critical for ensuring the safety of autonomous vehicles (AVs) in complex traffic environments. Recently, fully perception-based trajectory prediction methods have gained significant attention due to their capability to holistically model interactions within traffic scenarios. However, existing methods struggle to address occlusion in real-world traffic conditions and are constrained by fixed perception radius, leading to either excessive computational overhead or reduced prediction accuracy. These limitations pose a critical challenge in balancing predictive performance and computational efficiency in multi-agent trajectory forecasting. To tackle these issues, we propose a novel occlusion-aware trajectory prediction framework, named OIPAR. Specifically, OIPAR first applies vectorized normalization to environmental data and integrates occlusion reasoning with trajectory prediction, enabling efficient handling of occlusions. Moreover, it introduces an adaptive perception radius iteration module that dynamically adjusts the perception range to enhance prediction rationality. Our experimental results demonstrate that OIPAR offers flexible configurability, allowing the adoption of adaptive strategies to accommodate diverse and complex traffic environments.

A number of air traffic management decision support tools (DST) are being developed to help air traffic managers and controllers improve capacity, efficiency, and safety in the National Airspace System. Although DST functionality may vary widely, trajectory prediction algorithms can be found at the core of most DST. A methodology is presented for the automated statistical analysis of trajectory prediction accuracy as a function of phase of flight (level-flight, climb, descent) and look-ahead time. The methodology is focused on improving trajectory prediction algorithm performance for DST applications such as conflict detection and arrival metering. The methodology has been implemented in software and tested with air traffic data. Aggregate trajectory prediction accuracy statistics are computed and displayed in histogram format based on 2,774 large commercial jet flights from five different days of Fort Worth Center air traffic data. The results show that trajectory prediction anomalies can be detected by examining error distributions for large numbers of trajectory predictions. The ability of the trajectory analysis methodology to detect the effects of subsequent changes to the trajectory prediction algorithm and to aircraft performance model parameters was also demonstrated.

Accurate prediction of future person location and movement trajectory from an egocentric wearable camera can benefit a wide range of applications, such as assisting visually impaired people in navigation, and the development of mobility assistance for people with disability. In this work, a new egocentric dataset was constructed using a wearable camera, with 8,250 short clips of a targeted person either walking 1) toward, 2) away, or 3) across the camera wearer in indoor environments, or 4) staying still in the scene, and 13,817 person bounding boxes were manually labelled. Apart from the bounding boxes, the dataset also contains the estimated pose of the targeted person as well as the IMU signal of the wearable camera at each time point. An LSTM-based encoder-decoder framework was designed to predict the future location and movement trajectory of the targeted person in this egocentric setting. Extensive experiments have been conducted on the new dataset, and have shown that the proposed method is able to reliably and better predict future person location and trajectory in egocentric videos captured by the wearable camera compared to three baselines.

Vehicle trajectory prediction and driving intention recognition are essential technologies for improving safety and optimizing traffic efficiency in autonomous driving systems. Although traditional LSTM models are effective in trajectory prediction, their complexity and computational requirements impede practical implementation efficiency. To overcome this challenge, this study proposes a hybrid model, MTF-GRU, which integrates multi-feature fusion. Initially, the datasets are preprocessed in this study through denoising, feature extraction, and timing extraction to capture vehicle information from single and fused multi-features. Subsequently, a GRU encoding-decoding model is developed. The encoder processes the feature data to generate context vectors, while the decoder employs a combination of recursive and teaching-driven input modes. Furthermore, a teaching rate control mechanism is integrated to dynamically convert context vectors into future trajectories. The proposed model is validated using the NGSIM datasets, demonstrating superior prediction performance with multi-feature inputs outperforming single features by reducing the average endpoint displacement error by 20.5%. Our model also achieves improved accuracy rates, particularly excelling in long-term predictions with an endpoint displacement error of only 2.31 meters at 5 seconds. Moreover, the overall accuracy rate for lane change intention recognition reaches 91.3%. The model's computational efficiency supports practical deployment in real-time autonomous systems, while future efforts will integrate multi-modal sensor data to enhance adaptability in complex urban scenarios and extreme conditions. Further validation will extend to diverse traffic environments and edge computing platforms to optimize real-world robustness.

This article introduces a novel model for low-quality pedestrian trajectory prediction, the social nonstationary transformers (NSTransformers), that merges the strengths of NSTransformers and spatiotemporal graph transformer (STAR). The model can capture social interaction cues among pedestrians and integrate features across spatial and temporal dimensions to enhance the precision and resilience of trajectory predictions. We also propose an enhanced loss function that combines diversity loss with logarithmic root mean squared error (log-RMSE) to guarantee the reasonableness and diversity of the generated trajectories. This design adapts well to complex pedestrian interaction scenarios, thereby improving the reliability and accuracy of trajectory prediction. Furthermore, we integrate a generative adversarial network (GAN) to model the randomness inherent in pedestrian trajectories. Compared to the conventional standard Gaussian distribution, our GAN approach better simulates the intricate distribution found in pedestrian trajectories, enhancing the trajectory prediction's diversity and robustness. Experimental results reveal that our model outperforms several state-of-the-art methods. This research opens the avenue for future exploration in low-quality pedestrian trajectory prediction.