LGINet: Autonomous vehicle trajectory prediction using lane graphs and imitative learning policy

Aiming at the problems of large simplification and low accuracy of traditional trajectory prediction models, combined with the characteristics of time sequence, continuity, and interactivity of UUV trajectories, an unknown UUV trajectory prediction method based on a gated unit recursive (GRU) neural network model is proposed. The MinMaxScaler method is used to normalize the trajectory data; in order to improve the prediction accuracy, the adaptive parameter adjustment algorithm and adam algorithm are used to optimize the network structure of the established GRU-based trajectory prediction model, without human intervention for parameter adjustment. Finally, the trajectory prediction accuracy of GRU trajectory prediction model and BP trajectory prediction model are analyzed through simulation experiments. The experimental results show that the root mean square error is about 3.96m on the x-axis and 1.00m on the y-axis. The trajectory prediction model based on GRU has faster prediction speed and higher prediction accuracy for unknown UUV trajectory after bearings-only detection.

Trajectory prediction is one of the key technologies of the autonomous driving system. The accurate and efficient prediction can assist the autonomous driving system in making correct decisions and ensuring driving safety. This paper summarizes the research on trajectory prediction for autonomous vehicles based on deep learning. First, trajectory prediction models’ input representation and output types are analyzed. Then, deep learning-based trajectory prediction methods are reviewed with their advantages and limitations. Finally, future research directions are proposed to address the shortcomings of existing methods in terms of multi-agent interaction modeling and generalization capability.

Accurate trajectory prediction of vehicles is essential for reliable autonomous driving. To maintain consistent performance as a vehicle driving around different cities, it is crucial to adapt to changing traffic circumstances and achieve lifelong trajectory prediction model. To realize it, catastrophic forgetting is a main problem to be addressed. In this paper, a divergence measurement method based on conditional Kullback-Leibler divergence is proposed first to evaluate spatiotemporal dependency difference among varied driving circumstances. Then based on generative replay, a novel lifelong vehicle trajectory prediction framework is developed. The framework consists of a conditional generation model and a vehicle trajectory prediction model. The conditional generation model is a generative adversarial network conditioned on position configuration of vehicles. After learning and merging trajectory distribution of vehicles across different cities, the generation model replays trajectories with prior samplings as inputs, which alleviates catastrophic forgetting. The vehicle trajectory prediction model is trained by the replayed trajectories and achieves consistent prediction performance on visited cities. A lifelong experiment setup is established on four open datasets including five tasks. Spatiotemporal dependency divergence is calculated for different tasks. Even though these divergence, the proposed framework exhibits lifelong learning ability and achieves consistent performance on all tasks.

Data-driven trajectory prediction with weather uncertainties: A Bayesian deep learning approach

Accurately predicting trajectory of vehicles is a key capability for autonomous vehicles during real-world driving scenarios, which requires the autonomous vehicles to perceive the surrounding environment and analyze the track histories to judge the driving intentions of the focused surrounding vehicles in order to make decisions for the next trajectory route. In this paper, we propose a hybrid attentional trajectory prediction model incorporating both temporal attentional and spatial attentional mechanisms, which has a temporal attentional model that can be used to capture important temporal information affecting trajectory prediction and a spatial attentional mechanism that can better introduce spatial dependencies and potential driving intention information of surrounding and target vehicles. The models are evaluated for performance on publicly available highD and NGSIM datasets. By comparing the results with other state-of-the-art models for trajectory prediction, it is found that the model achieves an improvement in prediction performance, indicating that this model can more accurately learn driving information from the spatio-temporal perspective of the target driving vehicle’s attention. Further, by qualitatively analyzing the distribution of attention weights from the spatio-temporal perspective, we assessed the interpretability of the model on the maneuverability judgment of the vehicle.

Aircraft trajectory prediction is a challenging problem in air traffic control, especially for conflict detection. Traditional trajectory predictors require a variety of inputs such as flight-plans, aircraft performance models, meteorological forecasts, etc. Many of these data are subjected to environmental uncertainties. Further, limited information about such inputs, especially the lack of aircraft tactical intent, makes trajectory prediction a challenging task. In this work, we propose a deep learning model that performs trajectory prediction by modeling and incorporating aircraft tactical intent. The proposed model adopts the encoder-decoder architecture and makes use of the convolutional layer as well as Gated Recurrent Units (GRUs). The proposed model does not require explicit information about aircraft performance and wind data. Results demonstrate that the provision of enriched aircraft intent, together with appropriate model design, could improve the prediction error up to 30% at a prediction horizon of 10 minutes (from 4.9 nautical miles to 3.4 nautical miles). The model also guarantees the mean error growth rate with increasing look-ahead time to be lower than 0.2 nautical miles per minute. In addition, the model offers a very low variance in the prediction, which satisfies the variance-standard specified by EUROCONTROL (EU Organization for Safety and Navigation of Air Traffic) for trajectory predictors. The proposed model also outperforms the state-of-the-art trajectory prediction model, where the Root Mean Square Error (RMSE) is reduced from 0.0203 to 0.0018 for latitude prediction, and from 0.0482 to 0.0021 for longitude prediction in a single prediction step of 15 seconds look-ahead. We showed that the pre-trained model on ADS-B data maintains its high performance, in terms of cross-track and along-track errors, when being validated in the Bluesky Air Traffic Simulator. The proposed model would significantly improve the performance of conflict detection systems where such trajectory prediction models are needed.

High-performance trajectory prediction is at the core of strategic airspace capacity and safety enhancement. Current state-of-the-art trajectory prediction models are based on a three-dimensional point-mass model, using often predefined settings from existing databases rather than real-time information available onboard the aircraft to determine aircraft dynamics. As a result trajectory prediction performance is limited by the accuracy of these settings. This paper addresses this limitation and proposes a high-performance four-dimensional trajectory prediction model for civil aircraft. The model includes a new flight-control system and an enhanced flight-script. The latter incorporates new taxonomy and content enabling better definition of aircraft intent. The performance of the trajectory prediction model is assessed using data acquired during a real flight trial, and shown to be significantly better than the current models. It has the potential to meet the required navigation performance for departure, en-route and non-precision-approach phases of flight.

This paper develops a pedestrian potentially dangerous behaviour prediction method based on attention‐long‐short‐term memory (Attention‐LSTM) architecture to predict pedestrian trajectory and intention for the unexpected pedestrian crossing accident avoidance. To extract the road scene information for short periods of time, and improve the accuracy of subsequent intention inference and trajectory prediction, the panoramic segmentation is used to extrapolate pedestrian instances and segment areas of the environment. Next, an encoder–decoder framework based on Attention‐LSTM model is proposed to infer a pedestrian's intention to run or walk out into oncoming traffic straight and to predict the future trajectory. The proposed network involves two parts: temporal feature encoder and multi‐task decoder. The temporal feature encoder is mainly used to selectively emphasize the temporal features using attention mechanism, and then LSTM is employed for its encoding. In the multi‐task decoder, a multi‐head self‐attention mechanism and LSTM are used to forecast the pedestrians’ intention and future trajectory, respectively. Extensive experiments on pedestrian intention estimation (PIE) datasets demonstrate that the authors’ proposed approach surpasses prior studies in terms of prediction accuracy in trajectory and intention. This study can not only effectively avoid serious accidents caused by illegal road crossing but also achieve early warning for collision avoidance.

Vehicle trajectory information is a crucial part of improving the efficiency and the safety of the ITS. Data missing or irregular sampling in the real-world road traffic makes it hard to obtain accurate real-time vehicle trajectories. In this paper, we focus on trajectory imputation and prediction tasks with small data (magnitude set as 10 1 and 10 2 ). Limited by insufficient data, the simulation results of the existing data-driven algorithms are unsatisfactory. With car-following models integrated as prior physical information to constrain the training process, we design the car-following-informed neural network (CFINN). A multi-head self-attention layer is attached to the fully connected network layer to extract vehicle features. Different from the structure of most neural networks in regression analysis, an extra physics-based dataset is constructed in the CFINN. The loss function consists of two parts including the given trajectory's fitting error and the generated trajectory's residual error. We embed the gated recurrent unit-based encoder–decoder layer to the CFINN framework for trajectory predictions. The rationality and the superiority of our model are validated on the NGSIM dataset and the HighD dataset. Compared with baseline models in both single-vehicle and queue-typed trajectory imputation experiments, lower error can be achieved via the CFINN and coefficients of car-following models can be calibrated. According to driving regimes derived from CFINN-based trajectory prediction experiments, we discuss the impact of cut-in behaviours on the target vehicle and carry out the kinetic analysis. The novel neural network model driven by both data and physical knowledge provides technical support in vehicle status assessments and trajectory predictions.

Pedestrian trajectory prediction is a critical research area with numerous domains, e.g., blind navigation, autonomous driving systems, and service robots. There exist two challenges in this research field: spatio-temporal interaction modeling among pedestrians and the uncertainty of pedestrian trajectories. To tackle these challenges, we propose a spatio-temporal interaction aware and trajectory distribution aware graph convolution network. First, we propose a spatio-temporal interaction aware module that integrates a graph convolutional network and self-attention mechanism to model spatio-temporal interactions among pedestrians. Second, we design a trajectory distribution aware module to learn latent trajectory distribution information from the measured trajectories at observed and future times. This can provide knowledge-rich trajectory distribution information for the multimodality of the predicted trajectories. Finally, to address the problem of the propagation and accumulation of prediction errors, we design a trajectory decoder to generate the multimodal future trajectories. The proposed model is evaluated utilizing videos recorded by a camera sensor in crowded areas and can be applied to predict multiple pedestrians’ future trajectories from in-vehicle cameras. Experimental results demonstrate that the proposed approach can achieve superior results on the average displacement error (ADE) and final displacement error (FDE) metrics to state-of-the-art approaches and can predict socially acceptable future trajectories.

Understanding crowd motion dynamics and forecasting the future pedestrian trajectories are critical to various applications, e.g. autonomous driving and surveillance system. This task is challenging because when pedestrians plan the future paths in real crowd scenes, they will distinguish the priorities of following their predetermined destinations and responding to the motion behaviors of neighboring pedestrians. However, most of the existing methods ignore the problem of intention-interaction trade-off. In this paper, we tackle this problem by a hierarchical network, which achieves dynamically reasoning predetermined destinations and future trajectories. A novel graph structure called Intention-Interaction Graph (IIG) is designed to jointly model the self intentions and social interactions. To aggregate information in IIG, Interaction Gated Graph Attention Networks (IGGAN) consisting of a gate mechanism and an attention mechanism is proposed, thus achieving reasoning the influence degree of neighboring pedestrians and destinations. Experimental results on multiple widely used pedestrian trajectory prediction datasets, including two datasets in ETH and three datasets in UCY, demonstrate the effectiveness of the proposed model.

Predicting the future trajectories of target agents is critical for the safe operation of autonomous vehicles in complex environments. Accurate trajectory prediction requires the extraction of both temporal and spatial information. To address the limitations of existing static attention methods, which are constrained by fixed historical ranges and spatial radii under limited computational resources, a novel trajectory prediction approach using dynamic spatio-temporal attention is proposed. DSTNet incorporates a multi-scale spatial sparse attention module and a multi-scale historical causal attention module, both aimed at dynamically encoding the spatio-temporal dependencies of the data. These modules are unified with multi-modal interactive attention in a three-attention spatio-temporal module. Additionally, DSTNet includes an anchor-based trajectory refinement module, which adaptively optimizes the coarse-grained trajectories generated by the backbone network. Compared with state-of-the-art methods, DSTNet demonstrates outstanding performance on the Argoverse 1 and Argoverse 2 motion prediction benchmarks, delivering accurate and stable future trajectories. Notably, the model has been successfully deployed on an autonomous vehicle.

In vehicle trajectory prediction, constructing the interactive relationships among vehicles within the traffic environment poses a significant challenge. Existing models predominantly focus on temporal dependencies within vehicle histories and spatial correlations among neighboring vehicles, overlooking the continuous influence of historical vehicle states on the current time step and the interplay of multiple sequences over time. To address these limitations, we propose a method for multimodal vehicle trajectory prediction that integrates Hybrid Time-step Scene Interaction (HTSI) into the spatiotemporal relationships. Firstly, we introduce the HTSI module, comprising Multi-step Temporal Information Aggregation (MTIA) and Single-step Temporal Information Aggregation (STIA) methods. MTIA utilizes multi-head attention mechanisms to capture temporal dependencies between consecutive frames, thereby generating new time series amalgamating the ongoing influence of historical time states on the current timestamp. Simultaneously, STIA employs multi-head attention mechanisms to capture the spatial dimension weights of multiple time series and, by aggregating spatial interaction features at each timestamp, generates new time series fused with spatial interaction influences. Subsequently, feature extraction is performed through LSTM layers. Moreover, we propose an improved DIPM pooling module, improving the model’s long-term prediction capability by selectively reusing historical hidden states. Ultimately, based on training results from the HighD and NGSIM datasets, our model demonstrates significant advantages in long-term prediction compared to other state-of-the-art trajectory prediction models. Specifically, within the 5 s prediction window, the model achieved a root mean square error (RMSE) of 2.79 m on the NGSIM dataset, representing a 33.62% improvement over the baseline model’s average accuracy. Additionally, on the HighD dataset, the model attained an RMSE of 2.16 m, reflecting a 33.43% enhancement. The crucial code can be obtained from the provided link: https://github.com/gyhhq/Prediction-trajectory .

Pedestrian trajectory prediction in dynamic, multi-agent traffic junction scene is an important problem in the context of self-driving cars. Accurately predicting the trajectory of the agent, especially the pedestrian with high randomness, is of great significance to autonomous driving technology. In this paper, we propose Multi-PPTP, a novel trajectory prediction model that first utilizes a composite rasterized map to model the complex representations and interactions of road components, including dynamic obstacles (e.g., pedestrians, vehicles, and cyclists) and static road information (e.g., lanes, traffic lights, and sidewalks). Then it uses MapNet and AgentNet to extract spatio-temporal features by deep convolutional networks and LSTM to automatically derive relevant features, and next exploits Interaction-AttNet aggregate features to learn the high interactions among all components by affine transformation and multi-head attention mechanism. Additionally, we also propose a series of unique loss functions to predict multiple possible trajectories of each pedestrian while estimating their probabilities. Following extensive offline evaluation and comparison to the state-of-the-art models, our approach outperforms significantly the state-of-the-art models on our large scale in-house and public nuScenes dataset.

The long-term prediction of hypersonic glide vehicle (HGV) trajectory based on radar tracking data still faces challenges. Existing approaches are hard to provide satisfactory performances in long-term HGV trajectory prediction tasks due to insufficient long-range dependency extraction ability and cumulative errors. To address the above issues, a trajectory prediction model physics-informed Transformer (PIT) is proposed in this work. The top$\tau$-mean attention mechanism and the generative decoder are designed in this model to extract the trajectories' long-range dependency and reduce the cumulative errors. Furthermore, the physical knowledge is applied at both the input and output ends of the model, which further improves the model's prediction accuracy even in the case of missing a small amount of radar tracking data. A simulated HGV trajectory dataset is established based on the dynamic equations for the PIT trajectory prediction model. The experimental results show that the PIT performs better than other state-of-the-art deep neural network models. The additional experiments on a small-scale dataset and datasets with missing tracking data further verify the excellent performance and robustness of the PIT in these cases.