Vehicle trajectory prediction and generation using LSTM models and GANs.

<div class="section abstract"><div class="htmlview paragraph">Safety is the cornerstone for Advanced Driver Assistance Systems (ADAS) and Autonomous Driving Systems (ADS). To assess the safety of a traffic situation, it is essential to predict motion states of traffic participants in the future with mathematic models. Accurate vehicle trajectory prediction is an important prerequisite for reasonable traffic situation risk assessment and appropriate decision making. Vehicle trajectory prediction methods can be generally divided into motion model based methods and maneuver model based methods. Vehicle trajectory prediction based on motion models can be accurate and reliable only in the short term. While vehicle trajectory prediction based on maneuver models present more satisfactory performance in the long term, these maneuver models rely on machine learning methods. Abundant data should be collected to train the maneuver recognition model, which increases complexity and lowers real-time performance. In this paper, a vehicle trajectory prediction method based on motion model and maneuver model fusion with Interactive Multiple Model (IMM) is proposed. Firstly, Constant Turn Rate and Acceleration (CTRA) motion model and Unscented Kalman Filter (UKF) are used to predict vehicle trajectory with uncertainty in the future. Then, vehicle trajectory prediction based on simplified maneuver recognition model is conducted, using temporal and spatial relationship between vehicle historical trajectory and lane lines. After that, vehicle trajectory prediction by integrating motion model and maneuver model with IMM is conducted. Finally, the proposed method is compared with CTRA motion model based vehicle trajectory prediction and lane keeping model (LKM) based vehicle trajectory prediction in two simulation test scenarios. The simulation results indicates that the IMM-based method achieves both excellent prediction accuracy and appropriate prediction uncertainty in the whole prediction horizon. This research can be used to support decision making for Advanced Driver Assistance Systems (ADAS) and Autonomous Driving Systems and leads to improvement of traffic safety.</div></div>

Vehicle trajectory information is a crucial part of improving the efficiency and the safety of the ITS. Data missing or irregular sampling in the real-world road traffic makes it hard to obtain accurate real-time vehicle trajectories. In this paper, we focus on trajectory imputation and prediction tasks with small data (magnitude set as 10 1 and 10 2 ). Limited by insufficient data, the simulation results of the existing data-driven algorithms are unsatisfactory. With car-following models integrated as prior physical information to constrain the training process, we design the car-following-informed neural network (CFINN). A multi-head self-attention layer is attached to the fully connected network layer to extract vehicle features. Different from the structure of most neural networks in regression analysis, an extra physics-based dataset is constructed in the CFINN. The loss function consists of two parts including the given trajectory's fitting error and the generated trajectory's residual error. We embed the gated recurrent unit-based encoder–decoder layer to the CFINN framework for trajectory predictions. The rationality and the superiority of our model are validated on the NGSIM dataset and the HighD dataset. Compared with baseline models in both single-vehicle and queue-typed trajectory imputation experiments, lower error can be achieved via the CFINN and coefficients of car-following models can be calibrated. According to driving regimes derived from CFINN-based trajectory prediction experiments, we discuss the impact of cut-in behaviours on the target vehicle and carry out the kinetic analysis. The novel neural network model driven by both data and physical knowledge provides technical support in vehicle status assessments and trajectory predictions.

In the field of trajectory prediction, when new scene data is continuously introduced, the system may forget the historical trajectory information that has been learned before, leading to the phenomenon of decreasing the prediction accuracy of previous scenes, which is called catastrophic forgetting. The Multi-Memo Net framework proposed in this paper is an instance-based approach that stores typical trajectories through a memory bank thereby establishing a direct link between the current and historical scenes. During training, self-encoders and decoders are used to efficiently memorize the feature expressions of past trajectories and future intentions; during prediction, the controller finds similar scenes that have occurred in history in the memory bank based on the past trajectories and contextual information for trajectory prediction, and ultimately obtains a multimodal trajectory prediction through K-means clustering, and while predicting, the model can keep updating the content of the memory bank to learn, thus iteratively optimizing the model prediction performance. Thus, the model prediction performance is iteratively optimized. Compared with baseline models such as Kalman filter and Memory Augmented Networks for Multiple Trajectory Prediction (MANTRA) model, the method in this paper has significant improvement in Average Displacement Error (ADE) and Final Displacement Error (FDE) metrics.

Vehicle trajectory prediction and driving intention recognition are essential technologies for improving safety and optimizing traffic efficiency in autonomous driving systems. Although traditional LSTM models are effective in trajectory prediction, their complexity and computational requirements impede practical implementation efficiency. To overcome this challenge, this study proposes a hybrid model, MTF-GRU, which integrates multi-feature fusion. Initially, the datasets are preprocessed in this study through denoising, feature extraction, and timing extraction to capture vehicle information from single and fused multi-features. Subsequently, a GRU encoding-decoding model is developed. The encoder processes the feature data to generate context vectors, while the decoder employs a combination of recursive and teaching-driven input modes. Furthermore, a teaching rate control mechanism is integrated to dynamically convert context vectors into future trajectories. The proposed model is validated using the NGSIM datasets, demonstrating superior prediction performance with multi-feature inputs outperforming single features by reducing the average endpoint displacement error by 20.5%. Our model also achieves improved accuracy rates, particularly excelling in long-term predictions with an endpoint displacement error of only 2.31 meters at 5 seconds. Moreover, the overall accuracy rate for lane change intention recognition reaches 91.3%. The model's computational efficiency supports practical deployment in real-time autonomous systems, while future efforts will integrate multi-modal sensor data to enhance adaptability in complex urban scenarios and extreme conditions. Further validation will extend to diverse traffic environments and edge computing platforms to optimize real-world robustness.

Vehicle trajectory prediction is an important research basis for the decision making and path planning of the intelligent and connected vehicle. In the connected vehicle environment, vehicles share information and drive cooperatively, and the intelligent and connected vehicles are able to obtain more accurate and rich perception information, which provides a data basis for accurate prediction of vehicle trajectories. However, attaining accurate and effective vehicle trajectory predictions poses technical challenges due to insufficient extraction of vehicular spatial–temporal interaction features. In this paper, we propose a vehicle trajectory prediction model based on graph convolutional neural network (GCN) in a connected vehicle environment. Specifically, using the driving scene information obtained by the intelligent and connected vehicle, the spatial graph and temporal graph are constructed based on the spatial interaction coefficient (SIC) and self-attention mechanism, respectively. Then, the graph data are entered into the interaction extraction module, and the spatial interaction features and temporal interaction features are extracted separately using the graph convolutional networks, which are fused to obtain the spatial–temporal interaction information. Finally, the interaction features are learned based on the convolutional neural networks to output the future trajectory information of all vehicles in the scene by one forward operation rather than a step-by-step process. The ablation experiment results show that the method proposed in this study to model the spatiotemporal interaction among vehicles based on SIC and self-attention mechanism reduces the prediction error by 5% and 12%, respectively. The results from the model comparison experiment show that the proposed method engenders an 8% improvement in prediction accuracy over the state-of-the-art solution, providing technical and theoretical support for trajectory prediction research of intelligent and connected vehicles.

In this paper, a system solution is designed to improve the service efficiency of intersection. For the purpose, a method of trajectory prediction for vehicle is put forward, which is based on the basis of realize multi-targets tracking. The characteristics of traffic video, structural of road profile and movement of vehicles have been analyzed. We designed the multi-targets tracking algorithm based on particle filter for traffic video sequence, in order to improve the accuracy and rapidity, the heuristic method for multi-targets data correlation is proposed. Based on the results of tracking, we analyze the rule of movement parameters of vehicle as the path has changed, according to the features of structured road the vehicle running status parameters is simplified, and a Hidden Markov Model (HMM) has trained, which is used for vehicle trajectory prediction. In the end, Analysis the basic reason of traffic conflict and the dynamic model of traffic conflict zone is designed. Experimental results demonstrate that, the method of tracking based on particle filter has good tracking performance in traffic video; the vehicle trajectory can be predicted rapidly in the required time, and traffic conflict zone is produced accurately.

Pedestrian trajectory prediction is a critical research area with numerous domains, e.g., blind navigation, autonomous driving systems, and service robots. There exist two challenges in this research field: spatio-temporal interaction modeling among pedestrians and the uncertainty of pedestrian trajectories. To tackle these challenges, we propose a spatio-temporal interaction aware and trajectory distribution aware graph convolution network. First, we propose a spatio-temporal interaction aware module that integrates a graph convolutional network and self-attention mechanism to model spatio-temporal interactions among pedestrians. Second, we design a trajectory distribution aware module to learn latent trajectory distribution information from the measured trajectories at observed and future times. This can provide knowledge-rich trajectory distribution information for the multimodality of the predicted trajectories. Finally, to address the problem of the propagation and accumulation of prediction errors, we design a trajectory decoder to generate the multimodal future trajectories. The proposed model is evaluated utilizing videos recorded by a camera sensor in crowded areas and can be applied to predict multiple pedestrians’ future trajectories from in-vehicle cameras. Experimental results demonstrate that the proposed approach can achieve superior results on the average displacement error (ADE) and final displacement error (FDE) metrics to state-of-the-art approaches and can predict socially acceptable future trajectories.

Motion prediction for the leading vehicle is a critical task for connected autonomous vehicles. It provides a method to model the leading-following vehicle behavior and analysis their interactions. In this study, a joint time-series modeling approach for leading vehicle trajectory prediction considering different driving styles is proposed. The proposed method enables a precise and personalized trajectory prediction for the leading vehicle based on limited inter-vehicle communication signals, such as the vehicle speed and acceleration of the front vehicles. Three different driving styles are first recognized based on an unsupervised clustering algorithm, namely, Gaussian Mixture Model (GMM). The GMM generates a specific driving style for each vehicle based on the speed, acceleration, jerk, time, and space headway features of the leading vehicle. The feature importance of driving style recognition is also evaluated based on the Maximal Information Coefficient (MIC) algorithm. Then, a personalized joint time series modeling (JTSM) method based on the Long Short-Term Memory (LSTM) Recurrent Neural Network model (RNN) is proposed to predict the front vehicle trajectories. The JTSM contains a common LSTM layer and different fully connected regression layers for different driving styles. The proposed method is tested with the Next Generation Simulation (NGSIM) data on the US101, and I-80 highway dataset. The JTSM is tested for making predictions one to five seconds ahead. Results indicate that the proposed personalized JTSM approach shows a significant advantage over the baseline algorithms.

Vehicle trajectories at urban road intersections are influenced by diverse spatiotemporal factors, with significant variations across different locations. To address these complexities, real-world trajectory data were collected from intersections in Xi’an, Shaanxi Province, China, and raw trajectories were extracted using the You Only Look Once v8 and BoT-SORT algorithms. This study presents the Spatiotemporal Generative Attention Network (STGANT), a novel generative adversarial network-based framework designed specifically for vehicle trajectory prediction at intersections. The STGANT integrates innovative mechanisms, such as agent attention, social pooling, and Long Short-Term Memory, capturing spatial and temporal dependencies among vehicles. By incorporating an improved attention module, STGANT achieves a balance between accuracy and computational efficiency, enabling region-specific training while reducing overall computational load. Comprehensive experiments demonstrate that the STGANT outperforms existing models for prediction accuracy, computational complexity, and parameter efficiency. Qualitative analysis further verifies that the STGANT effectively captures the intricate spatiotemporal dynamics of vehicle interactions, resulting in reliable and accurate trajectory predictions. This study contributes to the enhancement of safety and operational efficiency in intelligent transportation systems at urban intersections.

Machine learning for autonomous vehicle's trajectory prediction: A comprehensive survey, challenges, and future research directions

Pedestrian trajectory prediction is fundamental to a wide range of scientific research work and industrial applications. Most of the current advanced trajectory prediction methods incorporate context information such as pedestrian neighbourhood, labelled static obstacles, and the background scene into the trajectory prediction process. In contrast to these methods which require rich contexts, the method in our paper focuses on predicting a pedestrian's future trajectory using his/her observed part of the trajectory only. Our method, which we refer to as LVTA, is a Location-Velocity-Temporal Attention LSTM model where two temporal attention mechanisms are applied to the hidden state vectors from the location and velocity LSTM layers. In addition, a location-velocity attention layer embedded inside a tweak module is used to improve the predicted location and velocity coordinates before they are passed to the next time step. Extensive experiments conducted on three large benchmark datasets and comparison with eleven existing trajectory prediction methods demonstrate that LVTA achieves competitive prediction performance. Specifically, LVTA attains 9.19 pixels Average Displacement Error (ADE) and 17.28 pixels Final Displacement Error (FDE) for the Central Station dataset, and 0.46 metres ADE and 0.92 metres FDE for the ETH&UCY datasets. Furthermore, evaluation on using LVTA to generate trajectories of different prediction lengths and on new scenes without the need of retraining confirms that it has good generalizability.

Vehicle trajectory prediction is essential to autonomous driving tasks. Accurate trajectory prediction of other traffic agents can significantly improve the ability of scene understanding and eventually improve the driving safety level of autonomous vehicles. Popular trajectory prediction methods leverage traffic rules by considering maps or lane graphs as a part of the input. However, this implicit representation is insufficient and can not guarantee an accurate understanding of traffic rules through the model. In this paper, we introduce a novel framework called forecasting using continuous unlikelihood and signal temporal logic (FCUS), which incorporates traffic rules in vehicle motion forecasting by explicitly modeling them as continuous signal temporal logic (STL) features into a GAN-style neural network. By doing so, a strong rule prior can be introduced. Meanwhile, an auxiliary unlikelihood loss is adopted to make the model more rule-aware. Extensive experiments evaluated on the real-world driving dataset show that our framework enables state-of-the-art prediction models to improve 8.92% performance on MinADE <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">5</inf> and 19.73% on avoiding rule violation. Demo videos and code are available at https://chantsss.github.io/FCUS/.

PB-Trajectron: Physics bounded neural network for generalized trajectory prediction

Accurate trajectory prediction of vehicles is essential for reliable autonomous driving. To maintain consistent performance as a vehicle driving around different cities, it is crucial to adapt to changing traffic circumstances and achieve lifelong trajectory prediction model. To realize it, catastrophic forgetting is a main problem to be addressed. In this paper, a divergence measurement method based on conditional Kullback-Leibler divergence is proposed first to evaluate spatiotemporal dependency difference among varied driving circumstances. Then based on generative replay, a novel lifelong vehicle trajectory prediction framework is developed. The framework consists of a conditional generation model and a vehicle trajectory prediction model. The conditional generation model is a generative adversarial network conditioned on position configuration of vehicles. After learning and merging trajectory distribution of vehicles across different cities, the generation model replays trajectories with prior samplings as inputs, which alleviates catastrophic forgetting. The vehicle trajectory prediction model is trained by the replayed trajectories and achieves consistent prediction performance on visited cities. A lifelong experiment setup is established on four open datasets including five tasks. Spatiotemporal dependency divergence is calculated for different tasks. Even though these divergence, the proposed framework exhibits lifelong learning ability and achieves consistent performance on all tasks.

Background: Today, self-driving cars are already on the roads. However, driving safety remains a huge challenge. Trajectory prediction of traffic targets is one of the important tasks of an autonomous driving environment perception system, and its output trajectory can provide necessary information for decision control and path planning. Although there are many patents and articles related to trajectory prediction, the accuracy of trajectory prediction still needs to be improved. Objective: This paper aimed to propose a novel scheme that considers multi-feature independent encoding trajectory prediction (MFIE). Methods: MFIE is an independently coded trajectory prediction algorithm that consists of a spacetime interaction module and trajectory prediction module, and considers speed characteristics and road characteristics. In the spatiotemporal interaction module, an undirected and weightless static traffic graph is used to represent the interaction between vehicles, and multiple graph convolution blocks are used to perform data mining on the historical information of target vehicles, capture temporal features, and process spatial interaction features. In the trajectory prediction module, three long short-term memory (LSTM) encoders are used to encode the trajectory feature, motion feature, and road constraint feature independently. The three hidden features are spliced into a tensor, and the LSTM decoder is used to predict the future trajectory. Results: On datasets, such as Apollo and NGSIM, the proposed method has shown lower prediction error than traditional model-driven and data-driven methods, and predicted more target vehicles at the same time. It can provide a basis for vehicle path planning on highways and urban roads, and it is of great significance to the safety of autonomous driving. Conclusion: This paper has proposed a multi-feature independent encoders’ trajectory prediction data-driven algorithm, and the effectiveness of the algorithm is verified with a public dataset. The trajectory prediction algorithm considering multi-feature independent encoders provides some reference value for decision planning.