Pedestrian Trajectory Prediction in Crowded Environments Using Social Attention Graph Neural Networks

Pedestrian path prediction is a very challenging problem because scenes are often crowded or contain obstacles. Existing state-of-the-art long short-term memory (LSTM)-based prediction methods have been mainly focused on analyzing the influence of other people in the neighborhood of each pedestrian while neglecting the role of potential destinations in determining a walking path. In this article, we propose classifying pedestrian trajectories into a number of route classes (RCs) and using them to describe the pedestrian movement patterns. Based on the RCs obtained from trajectory clustering, our algorithm, which we name the prediction of pedestrian paths by LSTM (PoPPL), predicts the destination regions through a bidirectional LSTM classification network in the first stage and then generates trajectories corresponding to the predicted destination regions through one of the three proposed LSTM-based architectures in the second stage. Our algorithm also outputs probabilities of multiple predicted trajectories that head toward the destination regions. We have evaluated PoPPL against other state-of-the-art methods on two public data sets. The results show that our algorithm outperforms other methods and incorporating potential destination prediction improves the trajectory prediction accuracy.

Pedestrian trajectory prediction can provide decision support for self-driving cars and robots to avoid potential collision risks. Modelling of social relationships and multimodal prediction are core challenges in pedestrian trajectory prediction. However, there are two problems with existing methods: (1) these methods usually focus on modelling pairwise or group interactions, ignoring the importance of high-order interactions; and (2) existing methods struggle to balance the performance between deterministic and multimodal predictions. To address these issues, we propose a higher-order graph attention network for pedestrian trajectory prediction. Specifically, we first construct spatial and temporal graphs to model social interactions and temporal dependencies. To achieve high-order interaction modelling, we design a higher-order graph attention module to capture pedestrian social and temporal interactions of different orders by stacking multiple layers of graph attention. Subsequently, we design a deformable spatial-temporal sparse module to adaptively eliminate social and temporal redundant interactions. To balance deterministic and multimodal prediction performance, we design a two-stage prediction strategy. The first stage predicts pedestrian deterministic trajectories, and the second stage refines pedestrian deterministic trajectories to complete both deterministic and multimodal predictions. Quantitative analysis on the ETH, UCY, and SDD datasets demonstrates that our method outperforms existing methods in both deterministic and multimodal predictions. Qualitative analysis shows that our method can effectively describe complex social relationships and accomplish reliable trajectory prediction.

Non-probability sampling network based on anomaly pedestrian trajectory discrimination for pedestrian trajectory prediction

Pedestrian trajectory prediction is an important research area with significant applications in autonomous driving and intelligent surveillance. However, existing studies on pedestrian trajectory prediction often suffer from a noticeable discrepancy between predicted and actual trajectories, due to incomplete extraction of pedestrian trajectory features and the randomness of the pedestrian walking process. The key objective of this article is to address this issue by proposing a method that can reasonably simulate the randomness of pedestrian walking and comprehensively extract pedestrian trajectory features. To achieve this, a novel social informer model built upon the informer model is proposed in this article. The social informer utilizes a transformer encoder-based interaction module to comprehensively extract pedestrian trajectory features, which are input into the informer model for further processing. Additionally, an adaptive variance mechanism is proposed to determine the optimal variance and accurately simulate the random nature of pedestrian walking. Finally, the proposed model is evaluated in a comparative experiment on ETH and UCY datasets, with results demonstrating that the proposed model outperforms other models, exhibiting improved accuracy and performance.

Pedestrian trajectory prediction is fundamental to a wide range of scientific research work and industrial applications. Most of the current advanced trajectory prediction methods incorporate context information such as pedestrian neighbourhood, labelled static obstacles, and the background scene into the trajectory prediction process. In contrast to these methods which require rich contexts, the method in our paper focuses on predicting a pedestrian's future trajectory using his/her observed part of the trajectory only. Our method, which we refer to as LVTA, is a Location-Velocity-Temporal Attention LSTM model where two temporal attention mechanisms are applied to the hidden state vectors from the location and velocity LSTM layers. In addition, a location-velocity attention layer embedded inside a tweak module is used to improve the predicted location and velocity coordinates before they are passed to the next time step. Extensive experiments conducted on three large benchmark datasets and comparison with eleven existing trajectory prediction methods demonstrate that LVTA achieves competitive prediction performance. Specifically, LVTA attains 9.19 pixels Average Displacement Error (ADE) and 17.28 pixels Final Displacement Error (FDE) for the Central Station dataset, and 0.46 metres ADE and 0.92 metres FDE for the ETH&UCY datasets. Furthermore, evaluation on using LVTA to generate trajectories of different prediction lengths and on new scenes without the need of retraining confirms that it has good generalizability.

Pedestrian trajectory prediction via physical-guided position association learning

This paper presents a novel framework for accurate pedestrian trajectory prediction in intersection corners or near crosswalks. Given prior knowledge of curbside geometry (i.e. angle made by intersecting curbs at the corner point of interest and the coordinates of the corner itself), the presented framework can accurately predict pedestrian trajectories even in new, unseen intersections. This is achieved by learning motion primitives in a common frame, called the curbside coordinate frame. A key insight in developing this common frame is to ensure that trajectories from intersections with different geometries, representing the same behavior, are spatially similar in the common frame. Motion primitives learned in such a common frame, can then be easily generalized to predict in new intersections, with different geometries than the ones trained on. We test our algorithm on real pedestrian trajectory datasets collected at two intersections, with distinctly different curbside and crosswalk geometries. A comparison of our algorithm with [1] demonstrates improved prediction accuracies of pedestrian trajectory prediction in the case of same training and test intersections, and the improvement of accuracy in the most different training and test intersections scenarios. The result also shows additional context, such as information about pedestrian traffic lights, if available, can be easily incorporated in our prediction model for further improvement in prediction accuracy.

Pedestrian trajectory prediction is imperative in specific fields, such as crowd management and collision prevention in automated driving environments. In this study, a novel long-short-term memory (LSTM)-based deep neural network capable of simulating the different walking behaviours of individuals with and without disabilities was designed. This network consists of three modules: the Disability module, the Environmental module, and the Trajectory Prediction module. Data from a large-scale pedestrian walking behaviour experiment involving individuals with disabilities were used to train and test the network. These data correspond to several experiments. Each experiment attempts to capture the essence of individuals’ walking behaviour in different situations. By sequencing and normalising the input data and applying regularisation techniques, the network was successfully trained. The results were compared to state-of-the-art models, demonstrating that the network can predict pedestrians’ trajectories more accurately, especially when pedestrian heterogeneity is involved.

In recent years, with the increasing flow of people in public places, people's awareness of safety and personal space have continuously increasing. Therefore, a large number of surveillance cameras are installed in public places, generating a large amount of surveillance data. Trajectory prediction based on the surveillance data has become a hot research direction in the field of computer vision and crowd behavior analysis. It can be used in traffic control and management, road planning, prevention of safety accidents and many other critical domains. The purpose of this paper is to propose a more efficient trajectory prediction algorithm. This paper proposes to use the convLSTM (Convolutional Long Short-Term Memory) neural network to predict pedestrian trajectory, and obtains better results than its original model LSTM (Long Short-Term Memory) network in the acquisition of space-time relationships. Combined with the mixed density network (MDN), the network model uses a probability distribution instead of a definite location to predict the trajectory, which greatly improves the efficiency and accuracy of trajectory prediction.

Pedestrian trajectory prediction is one of the important tasks required for autonomous navigation and social robots in human environments. Previous studies focused on estimating social forces among individual pedestrians. However, they did not consider the social forces of groups on pedestrians, which results in over-collision avoidance problems. To address this problem, we present a Disentangled Multi-Relational Graph Convolutional Network (DMRGCN) for socially entangled pedestrian trajectory prediction. We first introduce a novel disentangled multi-scale aggregation to better represent social interactions, among pedestrians on a weighted graph. For the aggregation, we construct the multi-relational weighted graphs based on distances and relative displacements among pedestrians. In the prediction step, we propose a global temporal aggregation to alleviate accumulated errors for pedestrians changing their directions. Finally, we apply DropEdge into our DMRGCN to avoid the over-fitting issue on relatively small pedestrian trajectory datasets. Through the effective incorporation of the three parts within an end-to-end framework, DMRGCN achieves state-of-the-art performances on a variety of challenging trajectory prediction benchmarks.

Pedestrian trajectory prediction is a critical research area with numerous domains, e.g., blind navigation, autonomous driving systems, and service robots. There exist two challenges in this research field: spatio-temporal interaction modeling among pedestrians and the uncertainty of pedestrian trajectories. To tackle these challenges, we propose a spatio-temporal interaction aware and trajectory distribution aware graph convolution network. First, we propose a spatio-temporal interaction aware module that integrates a graph convolutional network and self-attention mechanism to model spatio-temporal interactions among pedestrians. Second, we design a trajectory distribution aware module to learn latent trajectory distribution information from the measured trajectories at observed and future times. This can provide knowledge-rich trajectory distribution information for the multimodality of the predicted trajectories. Finally, to address the problem of the propagation and accumulation of prediction errors, we design a trajectory decoder to generate the multimodal future trajectories. The proposed model is evaluated utilizing videos recorded by a camera sensor in crowded areas and can be applied to predict multiple pedestrians’ future trajectories from in-vehicle cameras. Experimental results demonstrate that the proposed approach can achieve superior results on the average displacement error (ADE) and final displacement error (FDE) metrics to state-of-the-art approaches and can predict socially acceptable future trajectories.

Speech Emotion Recognition (SER) plays a vital role in various real-world applications, from mental health diagnostics to human-computer interaction. This research examines the integration of Stacked Long Short-Term Memory (LSTM) networks and Hidden Markov Models (HMM) for SER, addressing the limitations of traditional models like Gaussian Mixture Models (GMM) and Support Vector Machines (SVM). While GMMs and SVMs offer decent performance, their inability to model temporal dynamics limits their accuracy in detecting complex emotions. The proposed hybrid model leverages LSTM’s strength in handling sequential data and HMM’s ability to model emotional transitions, making it highly suitable for real-world noisy environments. Preprocessing techniques such as MFCC and LPCC are applied to enhance feature extraction, and benchmark datasets like IEMOCAP and RAVDESS are used for evaluation. Finally, The Systematic review highlights the superior role of the hybrid model's performance on SER and sets the stage for a significant shift in future research in addressing bias and fairness in SER systems by combining LSTM and HMM as hybrid model. In recent studies, context-aware emotion recognition models are commonly evaluated using existing datasets such as IEMOCAP, MELD, DailyDialog, and SEMAINE, which provide conversational and multimodal emotional data. Features used often include acoustic features as MFCCs, spectrograms, prosodic features, linguistic features as BERT or transformer embeddings, and visual cues such as facial landmarks or video frames. Many models also incorporate contextual embeddings from transformers, graph attention networks, or sequential memory modules to capture speaker history and emotional shifts.

Most Graph Neural Networks (GNNs) follow the message-passing scheme. Residual connection is an effective strategy to tackle GNNs' over-smoothing issue and performance reduction issue on non-homophilic networks. Unfortunately, the coarse-grained residual connection still suffers from class-imbalanced over-smoothing issue, due to the fixed and linear combination of topology and attribute in node representation learning. To make the combination flexible to capture complicated relationship, this paper reveals that the residual connection needs to be node-dependent, layer-dependent, and related to both topology and attribute. To alleviate the difficulty in specifying complicated relationship, this paper presents a novel perspective on GNNs, i.e., the representations of one node in different layers can be seen as a sequence of states. From this perspective, existing residual connections are not flexible enough for sequence modeling. Therefore, a novel node-dependent residual connection, i.e., Long Short-Term Graph Memory Network (LSTGM) is proposed to employ Long Short-Term Memory (LSTM), to model the sequence of node representation. To make the graph topology fully employed, LSTGM innovatively enhances the updated memory and three gates with graph topology. A speedup version is also proposed for effective training. Experimental evaluations on real-world datasets demonstrate their effectiveness in preventing over-smoothing issue and handling networks with heterophily.

Pedestrian trajectory prediction in urban environments has emerged as a critical research area with extensive applications across various domains. Accurate prediction of pedestrian trajectories is essential for the safe navigation of autonomous vehicles and robots in pedestrian-populated environments. Effective prediction models must capture both the spatial interactions among pedestrians and the temporal dependencies governing their movements. Existing research primarily focuses on forecasting a single trajectory per pedestrian, limiting its applicability in real-world scenarios characterised by diverse and unpredictable pedestrian behaviours. To address these challenges, this paper introduces the Graph Convolutional Network, Spatial–Temporal Attention, and Generative Model (GSTGM) for pedestrian trajectory prediction. GSTGM employs a spatiotemporal graph convolutional network to effectively capture complex interactions between pedestrians and their environment. Additionally, it integrates a spatial–temporal attention mechanism to prioritise relevant information during the prediction process. By incorporating a time-dependent prior within the latent space and utilising a computationally efficient generative model, GSTGM facilitates the generation of diverse and realistic future trajectories. The effectiveness of GSTGM is validated through experiments on real-world scenario datasets. Compared to the state-of-the-art models on benchmark datasets such as ETH/UCY, GSTGM demonstrates superior performance in accurately predicting multiple potential trajectories for individual pedestrians. This superiority is measured using metrics such as Final Displacement Error (FDE) and Average Displacement Error (ADE). Moreover, GSTGM achieves these results with significantly faster processing speeds.

DATA-DRIVEN APPROACH TO HOLISTIC SITUATIONAL AWARENESS IN CONSTRUCTION SITE SAFETY MANAGEMENT