Recurrent Aligned Network for Generalized Pedestrian Trajectory Prediction

Domain Adaptation (DA) methods are usually carried out by means of simply reducing the marginal distribution differences between the source and target domains, and subsequently using the resultant trained classifier, namely source classifier, for use in the target domain. However, in many cases, the true predictive distributions of the source and target domains can be vastly different especially when their class distributions are skewed, causing the issues of sample selection bias in DA. Hence, DA methods which leverage the source labeled data may suffer from poor generalization in the target domain, resulting in negative transfer. In addition, we observed that many DA methods use either a source classifier or a linear combination of source classifiers with a fixed weighting for predicting the target unlabeled data. Essentially, the labels of the target unlabeled data are spanned by the prediction of these source classifiers. Motivated by these observations, in this paper, we propose to construct many source classifiers of diverse biases and learn the weight for each source classifier by directly minimizing the structural risk defined on the target unlabeled data so as to heal the possible sample selection bias. Since the weights are learned by maximizing the margin of separation between opposite classes on the target unlabeled data, the proposed method is established here as Maximal Margin Target Label Learning (MMTLL), which is in a form of Multiple Kernel Learning problem with many label kernels. Extensive experimental studies of MMTLL against several state-of-the-art methods on the Sentiment and Newsgroups datasets with various imbalanced class settings showed that MMTLL exhibited robust accuracies on all the settings considered and was resilient to negative transfer, in contrast to other counterpart methods which suffered significantly in prediction accuracy.

Transfer learning and domain adaptation (DA) methods have been utilized in bearing prognostic and health management, but most of the current DA methods do not take into account the feature scale change of degraded features when aligning the feature distribution, and these methods are more suitable for the classification problem, which is more robust to the feature scale change. However, they perform poorly in regression problems. In addition, most of the remaining useful life (RUL) prediction methods require preprocessing such as statistical feature extraction on the signal, which makes the prediction process complicated. To solve the above problems, a DA method based on the representation subspace distance (RSD) is proposed for predicting the bearing RUL under different operating conditions. First, the proposed convolutional neural network (CNN) self-attention (SA) long short term memory network model is utilized to extract the deep features from the original signal, which overcomes the limitations of the CNN in extracting time series. Then, the RSD in the Riemannian geometry of the Grassmann manifold is proposed as a domain transfer loss to learn domain invariant features. The modified method can align the feature distribution of the source domain and the target domain without changing the feature scale. At the same time, the bases mismatch penalization is introduced to avoid destroying the semantic information of the features in the process of domain alignment. Finally, the effectiveness of the proposed method is verified by experiments on four types of transfer tasks, and its superiority is also demonstrated by comparison with other advanced methods.

Pedestrian trajectory prediction can provide decision support for self-driving cars and robots to avoid potential collision risks. Modelling of social relationships and multimodal prediction are core challenges in pedestrian trajectory prediction. However, there are two problems with existing methods: (1) these methods usually focus on modelling pairwise or group interactions, ignoring the importance of high-order interactions; and (2) existing methods struggle to balance the performance between deterministic and multimodal predictions. To address these issues, we propose a higher-order graph attention network for pedestrian trajectory prediction. Specifically, we first construct spatial and temporal graphs to model social interactions and temporal dependencies. To achieve high-order interaction modelling, we design a higher-order graph attention module to capture pedestrian social and temporal interactions of different orders by stacking multiple layers of graph attention. Subsequently, we design a deformable spatial-temporal sparse module to adaptively eliminate social and temporal redundant interactions. To balance deterministic and multimodal prediction performance, we design a two-stage prediction strategy. The first stage predicts pedestrian deterministic trajectories, and the second stage refines pedestrian deterministic trajectories to complete both deterministic and multimodal predictions. Quantitative analysis on the ETH, UCY, and SDD datasets demonstrates that our method outperforms existing methods in both deterministic and multimodal predictions. Qualitative analysis shows that our method can effectively describe complex social relationships and accomplish reliable trajectory prediction.

There is a large amount of out-of-distribution data (OOD) in remote sensing, which hinders high-accuracy segmentation models under the assumption of independent identical distribution (i.i.d.) from stable and reliable performance in real-world remote sensing applications. And Domain Adaptation (DA) is presented to seamlessly extend classifiers to the label-scarce target domain in the presence of the label-sufficient source domain with different data distributions. However, given that the domain shift, i.e. the distribution difference between the two domains, is more serious in remote sensing images, the current DA methods for image segmentation in Computer Vision (CV) typically perform unsatisfactorily in remote sensing, even suffering from the negative domain alignment. To this end, this paper proposes the Self-Training Adversarial Domain Adaptation (STADA) method for remote sensing image segmentation, which not only performs adversarial learning to extract domain-invariant features, but also implements Self-Training using pseudo-labels in the target domain denoised by the conditional adversarial loss for classifier adaptation. The ISPRS and WHU datasets are employed to conduct extensive experiments to investigate the effectiveness of STADA and the specific effect of its each DA component. And the experimental results demonstrate that STADA outperforms other state-of-the-art DA methods in the remote sensing image segmentation task.

In real-world hyperspectral image (HSI) classification, the limited annotated examples usually lead to the insufficiently trained classifier, which further generates low classification accuracy. To overcome this challenge, Domain Adaptation (DA) methods have been developed to transfer learnable knowledge from the external HSIs with sufficient labeled examples ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., the source domain) to the interested HSI with scarce labeled examples ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., the target domain). Conventional DA approaches often pseudo-label the examples with high classification confidence and then incorporate them in the training process. However, due to the significant domain gap, relying solely on the confident examples may not be adequate to achieve satisfactory performance. Therefore, this paper proposes an Interactive Easy-to-Hard Domain Adaptation method (IEH-DA) to arrange the adaptation process so that the “easy” examples are adapted ahead of the “hard” ones. In an early stage, the easy examples with high pseudo-labeling confidence are selected for the adversarial learning based DA. In a later stage, the “hard” examples with high informativity are further selected, and they are interactively labeled by human expert to provide accurate supervision information for adaptation. As a result, the examples in target domain are used in an easy-to-hard way, which forms a curriculum sequence for orderly model training. Extensive experiments conducted on typical public datasets demonstrate that IEH-DA outperforms other state-of-the-art DA methods for HSI classification.

Model fitting is a fundamental component in computer vision for salient data selection, feature extraction and data parameterization. Conventional approaches such as the RANSAC family show limitations when dealing with data containing multiple models, high percentage of outliers or sample selection bias, commonly encountered in computer vision applications. In this paper, we present a novel model evaluation function based on Gaussian-weighted Jensen–Shannon divergence, and integrate into a particle swarm optimization (PSO) framework using ring topology. We avoid two problems from which most regression algorithms suffer, namely the requirements to specify inlier noise scale and the number of models. The novel evaluation method is generic and does not require any estimation of inlier noise. The continuous and meta-heuristic exploration facilitates estimation of each individual model while delivering the number of models automatically. Tests on datasets comprised of inlier noise and a large percentage of outliers (more than 90 % of the data) demonstrate that the proposed framework can efficiently estimate multiple models without prior information. Superior performance in terms of processing time and robustness to inlier noise is also demonstrated with respect to state of the art methods.

Motion segmentation in real-world scenes is a fundamental component in computer vision. There exists a variety of motion recognition algorithms, each with varying degrees of accuracy and computational complexity. The most widely used techniques, in the case of static cameras, are those based on frame difference. Those methods have a significant weakness when it comes to detect slow moving objects. Therefore, we introduce in this paper a novel approach that aims to improve motion segmentation by proposing an accumulative wavelet based frame differencing technique. Moreover, in the proposed approach we exploit a combination of several techniques to efficiently enhance the quality of motion segmentation results. The approach's performance on real-world video sequences shows that comparing frames using the 2D wavelet transform increases motion segmentation quality.

Pedestrian trajectory prediction enables faster progress in autonomous driving and robot navigation where complex social and environmental interactions involve. Previous models use grid-based pooling or global attention to measure social interactions and use Recurrent Neural Network (RNN) to generate sequences. However, these methods can not extract latent features from temporal and spatial information simultaneously. To address the limitation of previous work, we propose a Self-Centered Star Graph with Attention (SCSG Attention) framework. Firstly, pedestrians’ historical trajectories are encoded. Then multi-head attention mechanism plays a role as enhancement of social interaction awareness and simulation of physical attention from human beings. Lastly, the self-centered star graph decoder can aggregate temporal and spatial features and make predictions. Experiments are conducted on public benchmark datasets and measured with uniform standards. Our results show an improvement over the state-of-the-art algorithms by \(38\%\) on average displacement error (ADE) and \(19\%\) on final displacement error (FDE). Furthermore, it is demonstrated that the star graph has better performance in efficiency of training convergence and ends up with better results in limited time.

Deep multi-Wasserstein unsupervised domain adaptation

The discriminability and transferability of models are two important factors for the success of domain adaptation methods. Recently, some domain adaptation methods have improved models by adding a discriminant information extraction module. However, these methods need to carefully balance the discriminability and transferability of a model. To address this problem, we propose a new deep domain adaptation method, Gaussian Process-based Transfer Kernel Learning (GPTKL), which can perform domain knowledge transfer and improve the discrimination ability of the model simultaneously. GPTKL uses the kernel similarity between all samples in the source and target domains as a priori information to establish a cross-domain Gaussian process. By maximizing its likelihood function, GPTKL reduces the domain discrepancy between the source and target domains, thereby enhancing generalization across domains. At the same time, GPTKL introduces the deep kernel learning strategy into the cross-domain Gaussian process to learn a transfer kernel function based on deep features. Through transfer kernel learning, GPTKL learns a deep feature space with both discriminability and transferability. In addition, GPTKL uses cross-entropy and mutual information to learn a classification model shared by the source and target domains. Experiments on four benchmarks show that GPTKL achieves superior classification performance over state-of-the-art methods.

Conventional machine learning relies on two presumptions: (1) the training and testing datasets follow the same independent distribution, and (2) an adequate quantity of samples is essential for achieving optimal model performance during training. Nevertheless, meeting these two assumptions can be challenging in real-world scenarios. Domain adaptation (DA) is a subfield of transfer learning that focuses on reducing the distribution difference between the source domain (Ds) and target domain (Dt) and subsequently applying the knowledge gained from the Ds task to the Dt task. The majority of current DA methods aim to achieve domain invariance by aligning the marginal probability distributions of the Ds. and Dt. Recent studies have pointed out that aligning marginal probability distributions alone is not sufficient and that alignment of conditional probability distributions is equally important for knowledge migration. Nonetheless, unsupervised DA presents a more significant difficulty in aligning the conditional probability distributions because of the unavailability of labels for the Dt. In response to this issue, there have been several proposed methods by researchers, including pseudo-labeling, which offer novel solutions to tackle the problem. In this paper, we systematically analyze various pseudo-labeling algorithms and their applications in unsupervised DA. First , we summarize the pseudo-label generation methods based on the single and multiple classifiers and actions taken to deal with the problem of imbalanced samples. Second, we investigate the application of pseudo-labeling in category feature alignment and improving feature discrimination. Finally, we point out the challenges and trends of pseudo-labeling algorithms. As far as we know, this article is the initial review of pseudo-labeling techniques for unsupervised DA.

Pedestrian trajectory prediction in urban environments has emerged as a critical research area with extensive applications across various domains. Accurate prediction of pedestrian trajectories is essential for the safe navigation of autonomous vehicles and robots in pedestrian-populated environments. Effective prediction models must capture both the spatial interactions among pedestrians and the temporal dependencies governing their movements. Existing research primarily focuses on forecasting a single trajectory per pedestrian, limiting its applicability in real-world scenarios characterised by diverse and unpredictable pedestrian behaviours. To address these challenges, this paper introduces the Graph Convolutional Network, Spatial–Temporal Attention, and Generative Model (GSTGM) for pedestrian trajectory prediction. GSTGM employs a spatiotemporal graph convolutional network to effectively capture complex interactions between pedestrians and their environment. Additionally, it integrates a spatial–temporal attention mechanism to prioritise relevant information during the prediction process. By incorporating a time-dependent prior within the latent space and utilising a computationally efficient generative model, GSTGM facilitates the generation of diverse and realistic future trajectories. The effectiveness of GSTGM is validated through experiments on real-world scenario datasets. Compared to the state-of-the-art models on benchmark datasets such as ETH/UCY, GSTGM demonstrates superior performance in accurately predicting multiple potential trajectories for individual pedestrians. This superiority is measured using metrics such as Final Displacement Error (FDE) and Average Displacement Error (ADE). Moreover, GSTGM achieves these results with significantly faster processing speeds.

Predicting the future trajectories of pedestrians is a challenging problem that has a range of application, from crowd surveillance to autonomous driving. In literature, methods to approach pedestrian trajectory prediction have evolved, transitioning from physics-based models to data-driven models based on recurrent neural networks. In this work, we propose a new approach to pedestrian trajectory prediction, with the introduction of a novel 2D convolutional model. This new model outperforms recurrent models, and it achieves state-of-the-art results on the ETH and TrajNet datasets. We also present an effective system to represent pedestrian positions and powerful data augmentation techniques, such as the addition of Gaussian noise and the use of random rotations, which can be applied to any model. As an additional exploratory analysis, we present experimental results on the inclusion of occupancy methods to model social information, which empirically show that these methods are ineffective in capturing social interaction.

Objective. The scarcity of electroencephalogram (EEG) data, coupled with individual and scenario variations, leads to considerable challenges in real-world EEG-based driver fatigue detection. We propose a domain adaptation method that utilizes EEG data collected from a laboratory to supplement real-world EEG data and constructs a cross-scenario and cross-subject driver fatigue detection model for real-world scenarios. Approach. First, we collect EEG data from subjects participating in a driving experiment conducted in both laboratory and real-world scenarios. To address the issue of data scarcity, we build a real-world fatigued driving detection model by integrating the real-world data with the laboratory data. Then, we propose a method named cross-scenario and cross-subject domain adaptation (CS2DA), which aims to eliminate the domain shift problem caused by individual variances and scenario differences. Adversarial learning is adopted to extract the common features observed across different subjects within the same scenario. The multikernel maximum mean discrepancy (MK-MMD) method is applied to further minimize scenario differences. Additionally, we propose a conditional MK-MMD constraint to better utilize label information. Finally, we use seven rules to fuse the predicted labels. Main results. We evaluate the CS2DA method through extensive experiments conducted on the two EEG datasets created in this work: the SEED-VLA and the SEED-VRW datasets. Different domain adaptation methods are used to construct a real-world fatigued driving detection model using data from laboratory and real-world scenarios, as well as a combination of both. Our findings show that the proposed CS2DA method outperforms the existing traditional and adversarial learning-based domain adaptation approaches. We also find that combining data from both laboratory and real-world scenarios improves the performance of the model. Significance. This study contributes two EEG-based fatigue driving datasets and demonstrates that the proposed CS2DA method can effectively enhance the performance of a real-world fatigued driving detection model.

Great progress has been achieved in domain adaptation in decades. Existing works are always based on an ideal assumption that testing target domains are independent and identically distributed with training target domains. However, due to unpredictable corruptions (e.g., noise and blur) in real data, such as web images and real-world object detection, domain adaptation methods are increasingly required to be corruption robust on target domains. We investigate a new task, corruption-agnostic robust domain adaptation (CRDA), to be accurate on original data and robust against unavailable-for-training corruptions on target domains. This task is non-trivial due to the large domain discrepancy and unsupervised target domains. We observe that simple combinations of popular methods of domain adaptation and corruption robustness have suboptimal CRDA results. We propose a new approach based on two technical insights into CRDA, as follows: (1) an easy-to-plug module called domain discrepancy generator (DDG) that generates samples that enlarge domain discrepancy to mimic unpredictable corruptions; (2) a simple but effective teacher-student scheme with contrastive loss to enhance the constraints on target domains. Experiments verify that DDG maintains or even improves its performance on original data and achieves better corruption robustness than baselines. Our code is available at: https://github.com/YifanXu74/CRDA .