High-risk powered two-wheelers scenarios generation for autonomous vehicle testing using WGAN

Generative Adversarial Networks (GANs) have been used to model the underlying probability distribution of sample based datasets. GANs are notoriuos for training difficulties and their dependence on arbitrary hyperparameters. One recent improvement in GAN literature is to use the Wasserstein distance as loss function leading to Wasserstein Generative Adversarial Networks (WGANs). Using this as a basis, we show various ways in which the Wasserstein distance is estimated for the task of generative modelling. Additionally, the secrets in training such models are shown and summarized at the end of this work. Where applicable, we extend current works to different algorithms, different cost functions, and different regularization schemes to improve generative models.

The design and operation of modern energy systems are heavily influenced by time-dependent and uncertain parameters, e.g., renewable electricity generation, load-demand, and electricity prices. These are typically represented by a set of discrete realizations known as scenarios. A popular scenario generation approach uses deep generative models (DGM) that allow scenario generation without prior assumptions about the data distribution. However, the validation of generated scenarios is difficult, and a comprehensive discussion about appropriate validation methods is currently lacking. To start this discussion, we provide a critical assessment of the currently used validation methods in the energy scenario generation literature. In particular, we assess validation methods based on probability density, auto-correlation, and power spectral density. Furthermore, we propose using the multifractal detrended fluctuation analysis (MFDFA) as an additional validation method for non-trivial features like peaks, bursts, and plateaus. As representative examples, we train generative adversarial networks (GANs), Wasserstein GANs (WGANs), and variational autoencoders (VAEs) on two renewable power generation time series (photovoltaic and wind from Germany in 2013 to 2015) and an intra-day electricity price time series form the European Energy Exchange in 2017 to 2019. We apply the four validation methods to both the historical and the generated data and discuss the interpretation of validation results as well as common mistakes, pitfalls, and limitations of the validation methods. Our assessment shows that no single method sufficiently characterizes a scenario but ideally validation should include multiple methods and be interpreted carefully in the context of scenarios over short time periods.

In industrial applications of machinery fault diagnostics, deep learning has been widely adopted to process large amounts of monitoring data. Unfortunately, due to the domain discrepancy, diagnostic models trained with source domain data suffer from degraded diagnostic performance on the target domain. To address this problem, a Wasserstein bi-classifier adversarial learning network (WBALN) is proposed. Specifically, WBALN consists of a feature extractor, two classifiers, and a discrepancy metric based on Wasserstein distance. A two-stream optimization strategy is used in the training process, which involves jointly performing bi-classifier adversarial learning and Wasserstein generative adversarial network (WGAN)-based adversarial learning. In the bi-classifier training stream, a min-max game is conducted between a discrepancy detector composed of two classifiers and the feature extractor to reduce the disparity between these classifiers. In the WGAN-based training stream, a Wasserstein adversarial discrepancy (WAD) is applied in combination with the original classifier as a domain discriminator, which achieves fault diagnosis and distribution alignment through a unified objective. This WAD enables WBALN to achieve sufficient feature alignment using the predicted discriminative information. In addition, the utilization of the nuclear-norm is useful for ensuring the determinacy and diversity of predictions. Except for ordinary domain adaptation, WBALN is also extended for challenging problems about inter-class imbalanced domain adaptation. The performance of the proposed WBALN is verified through multiple experiments on two bearing datasets.

Feature extraction (FE) is a crucial research area in hyperspectral image (HSI) processing. Recently, due to the powerful ability of deep learning (DL) to extract spatial and spectral features, DL-based FE methods have shown great potentials for HSI processing. However, most of the DL-based FE methods are supervised, and the training of them suffers from the absence of labeled samples in HSIs severely. The training issue of supervised DL-based FE methods limits their application on HSI processing. To address this issue, in this paper, a novel modified generative adversarial network (GAN) is proposed to train a DL-based feature extractor without supervision. The designed GAN consists of two components, which are a generator and a discriminator. The generator can focus on the learning of real probability distributions of data sets and the discriminator can extract spatial-spectral features with superior invariance effectively. In order to learn upsampling and downsampling strategies adaptively during FE, the proposed generator and discriminator are designed based on a fully deconvolutional subnetwork and a fully convolutional subnetwork, respectively. Moreover, a novel min-max cost function is designed for training the proposed GAN in an end-to-end fashion without supervision, by utilizing the zero-sum game relationship between the generator and discriminator. Besides, the proposed modified GAN replaces the original Jensen-Shannon divergence with the Wasserstein distance, aiming to mitigate the unstability and difficulty of the training of GAN frameworks. Experimental results on three real data sets validate the effectiveness of the proposed method.

The terahertz (THz) metamaterial sensor design is typically complex and requires substantial expertise in physics. To simplify this process, we propose a novel reverse design model based on an improved conditional generative adversarial network that integrates self-attention generative adversarial network and Wasserstein generative adversarial network (WGAN) networks, and is referred to as the self-attention conditional Wasserstein GAN (SACW-GAN) model. By using the target response of the sensor as the input to the generator network, and incorporating labeling information, an attention mechanism, and the Wasserstein distance, we achieve effective reverse design of THz metamaterial sensors. The simulation results demonstrate the model's high performance, with spectral and image accuracies of 95% and 97%, respectively. This deep learning approach offers new perspectives and methodologies for the reverse design and application of THz metamaterial sensors, significantly advancing the field.

Autonomous vehicles operate without direct human intervention, which introduces safety risks that differ from those of conventional vehicles. Although many studies have examined safety issues related to autonomous driving, high-risk situations have often been defined using single indicators, making it difficult to capture the complex and evolving nature of accident risk. To address this limitation, this study proposes a structured framework for defining and analyzing high-risk situations throughout the traffic accident process. High-risk situations are described using three complementary indicators: accident likelihood, accident severity, and accident duration. These indicators explain how risk emerges, increases, and persists over time. Based on this concept, a framework for traffic accident visualization analysis is developed to support phase-specific risk assessment and visualization. The framework combines accident-phase information with factor-level risk contributions, allowing systematic identification of key factors and their interactions across different accident stages. Using combinations of the three indicators, high-risk situations are classified into twenty-seven distinct types, providing a clear typology for complex accident scenarios involving autonomous vehicles. The applicability of the proposed framework is demonstrated through two representative accident scenarios with different risk characteristics. The results show that the framework effectively captures interactions among multiple risk factors, explains how risk levels change from pre-crash to post-crash phases, and identifies contributing factors that are difficult to detect using conventional traffic accident investigation methods. Overall, the proposed framework offers a practical basis for autonomous vehicle accident analysis, safety evaluation, and policy-related decision-making.

Simulation is an indispensable tool in the development and testing of autonomous vehicles (AVs), offering an efficient and safe alternative to road testing. An outstanding challenge with simulation-based testing is the generation of safety-critical scenarios, which are essential to ensure that AVs can handle rare but potentially fatal situations. This paper addresses this challenge by introducing a novel framework CaDRE, to generate realistic, diverse, and controllable safetycritical scenarios. Our approach optimizes for both the quality and diversity of scenarios by employing a unique formulation and algorithm that integrates real-world scenarios, domain knowledge, and black-box optimization. We validate the effectiveness of our framework through extensive testing in three representative types of traffic scenarios. The results demonstrate superior performance in generating diverse and highquality scenarios with greater sample efficiency than existing reinforcement learning (RL) and sampling-based methods.

Nowadays, deep learning methods based on a virtual environment are widely applied to research and technology development for autonomous vehicle’s smart sensors and devices. Learning various driving environments in advance is important to handle unexpected situations that can exist in the real world and to continue driving without accident. For training smart sensors and devices of an autonomous vehicle well, a virtual simulator should create scenarios of various possible real-world situations. To create reality-based scenarios, data on the real environment must be collected from a real driving vehicle or a scenario analysis process conducted by experts. However, these two approaches increase the period and the cost of scenario generation as more scenarios are created. This paper proposes a scenario generation method based on deep learning to create scenarios automatically for training autonomous vehicle smart sensors and devices. To generate various scenarios, the proposed method extracts multiple events from a video which is taken on a real road by using deep learning and generates the multiple event in a virtual simulator. First, Faster-region based convolution neural network (Faster-RCNN) extracts bounding boxes of each object in a driving video. Second, the high-level event bounding boxes are calculated. Third, long-term recurrent convolution networks (LRCN) classify each type of extracted event. Finally, all multiple event classification results are combined into one scenario. The generated scenarios can be used in an autonomous driving simulator to teach multiple events that occur during real-world driving. To verify the performance of the proposed scenario generation method, experiments using real driving video data and a virtual simulator were conducted. The results for deep learning model show an accuracy of 95.6%; furthermore, multiple high-level events were extracted, and various scenarios were generated in a virtual simulator for smart sensors and devices of an autonomous vehicle.

Generative adversarial networks (GANs) have been impactful on many problems and applications but suffer from unstable training. The Wasserstein GAN (WGAN) leverages the Wasserstein distance to avoid the caveats in the minmax two-player training of GANs but has other defects such as mode collapse and lack of metric to detect the convergence. We introduce a novel inferential Wasserstein GAN (iWGAN) model, which is a principled framework to fuse autoencoders and WGANs. The iWGAN model jointly learns an encoder network and a generator network motivated by the iterative primal-dual optimization process. The encoder network maps the observed samples to the latent space and the generator network maps the samples from the latent space to the data space. We establish the generalization error bound of the iWGAN to theoretically justify its performance. We further provide a rigorous probabilistic interpretation of our model under the framework of maximum likelihood estimation. The iWGAN, with a clear stopping criteria, has many advantages over other autoencoder GANs. The empirical experiments show that the iWGAN greatly mitigates the symptom of mode collapse, speeds up the convergence, and is able to provide a measurement of quality check for each individual sample. We illustrate the ability of the iWGAN by obtaining competitive and stable performances for benchmark datasets.

Traffic simulation is an effective tool for urban planners, traffic engineers, and researchers to study traffic. In particular, microscopic traffic simulation, which simulates individual vehicles’ movements within a transportation network, has demonstrated its importance in analyzing and managing transportation systems. However, integrating data from various sources, generating traffic scenarios, and importing information into traffic simulators to conduct microscopic simulations have always been a challenge. This paper presents a solution to overcome this challenge: RealTwin, a comprehensive tool for automated scenario generation for microscopic traffic simulation. Following a streamlined scenario generation and calibration workflow, RealTwin effectively bridges gaps between traffic data from various sources and traffic simulators, making microscopic traffic simulation more accessible for researchers and engineers across various levels of expertise. Using RealTwin to generate a real-world traffic scenario in Simulation of Urban Mobility (SUMO), VISSIM, and AIMSUN, RealTwin’s ability is demonstrated in the construction of realistic and consistent traffic scenarios in different simulators. Furthermore, this paper introduces and illustrates RealTwin’s capability for technology (e.g., autonomous vehicle) scenario generation. This feature can contribute to more comprehensive microscopic simulations, facilitating the analysis of potential effects of various technological innovations on mobility, energy efficiency, and safety. Finally, RealTwin is used to calibrate a simulation in SUMO. The calibration module enhances RealTwin’s ability to generate consistent simulations across different platforms and more realistic simulations that reflect real-world traffic operations.

Simulation-based testing has emerged as an essential tool for verifying and validating autonomous vehicles (AVs). However, contemporary methodologies, such as deterministic and imitation learning-based driver models, struggle to capture the variability of human-like driving behavior. Given these challenges, we propose HAD-Gen, a general framework for realistic traffic scenario generation that simulates diverse human-like driving behaviors. The framework first clusters the vehicle trajectory data into different driving styles according to safety features. It then employs maximum entropy inverse reinforcement learning on each of the clusters to learn the reward function corresponding to each driving style. Using these reward functions, the method integrates offline reinforcement learning pre-training and multi-agent reinforcement learning algorithms to obtain general and robust driving policies. Multi-perspective simulation results in highway scenarios show that our proposed scenario generation framework can generate diverse, human-like driving behaviors with strong generalization capability. The proposed framework achieved a 90.96% goal-reaching rate, an off-road rate of 2.08%, and a collision rate of 6.91% in new unseen driving scenarios, outperforming prior approaches by over 20% in goal-reaching performance. The source code is released at https://github.com/RoboSafe-Lab/HAD-Gen.

In recent AI-driven disease diagnosis, the success ofmodels hasdepended mainly on extensive data sets and advanced algorithms. However,creating traditional data sets for rare or emerging diseases presentssignificant challenges. To address this issue, this study introducesa direct-self-attention Wasserstein generative adversarial network(DSAWGAN) designed to improve diagnostic capabilities in infectiousdiseases with limited data availability. DSAWGAN enhances convergencespeed, stability, and image quality by integrating attention modulesand leveraging the Wasserstein distance optimization. We comparedDSAWGAN-generated images with traditional data augmentation and otherimage generation techniques, evaluating their effectiveness usingclassification neural networks for diagnostic accuracy. This modelintegration was then applied to a mobile app, enabling rapid, portable,and cost-effective diagnostic testing across various concentrationranges. Using only half of the raw data (n = 1500),DSAWGAN achieves an accuracy increase from 98.00 to 99.33%. Even withjust 10% of the original data (n = 300), a neuralnetwork trained with the augmented data set maintains an accuracyof 92.67%, demonstrating the approach’s effectiveness in resource-limitedsettings.

Generative adversarial networks (GAN) are currently a hotly debated research topic in the field of machine vision; however, they possess various shortcomings that cannot be overlooked, such as unstable generated samples, collapsed modes, and slow convergence. The present paper combines the advantages of the super-resolution GAN model (SRGAN) and Wasserstein GAN with gradient penalty (WGAN-GP) and proposes a hybrid model, known as super-resolution Wasserstein generative adversarial network with gradient penalty (SRWGAN-GP). The generator of SRWGAN-GP utilizes the residuals block and Wasserstein distance to increase the resolution of the generated samples and solve the shortcomings of the unstable GAN training. Besides, the discriminator of SRWGAN-GP consists of a local discriminator and a global discriminator to enhance image recognition capacity. Finally, the discriminator is extracted for image recognition. The results of comparative experiments indicate that this method can effectively enhance the accuracy of image recognition.

Transitioning to a driverless city: Evaluating a hybrid system for autonomous and non-autonomous vehicles

Evaluating autonomous ground vehicles requires evaluating their mobility performance. Since autonomous vehicles are envisioned to make decisions in a variety of situations and environments too diverse to practically assess with only physical testing, their development, and evaluation will necessarily include the use of simulations. These simulations must represent reality sufficiently to represent the decisions that the vehicles would make in real-world. In this paper we present our Scoping Autonomous Vehicle Simulation (SAVeS) platform for benchmarking the performance of simulated environments for autonomous ground vehicle testing <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .