Image Recognition Based on Super-Resolution Wasserstein Generative Adversarial Nets with Gradient Penalty

Single image super-resolution (SISR) is of great importance as a low-level computer vision task. The fast development of Generative Adversarial Network (GAN) based deep learning architectures realises an efficient and effective SISR to boost the spatial resolution of natural images captured by digital cameras. However, the SISR for medical images is still a very challenging problem. This is due to (1) compared to natural images, in general, medical images have lower signal to noise ratios, (2) GAN based models pre-trained on natural images may synthesise unrealistic patterns in medical images which could affect the clinical interpretation and diagnosis, and (3) the vanilla GAN architecture may suffer from unstable training and collapse mode that can also affect the SISR results. In this paper, we propose a novel lesion focused SR (LFSR) method, which incorporates GAN to achieve perceptually realistic SISR results for brain tumour MRI images. More importantly, we test and make comparison using recently developed GAN variations, e.g., Wasserstein GAN (WGAN) and WGAN with Gradient Penalty (WGAN-GP), and propose a novel multi-scale GAN (MS-GAN), to achieve a more stabilised and efficient training and improved perceptual quality of the super-resolved results. Based on both quantitative evaluations and our designed mean opinion score, the proposed LFSR coupled with MS-GAN has performed better in terms of both perceptual quality and efficiency.

Recently, generative adversarial networks (GANs) are widely applied to increase the amounts of imbalanced input samples in fault diagnosis. However, the existing GAN-based methods have convergence difficulties and training instability, which affect the fault diagnosis efficiency. This paper develops a novel framework for imbalanced fault classification based on Wasserstein generative adversarial networks with gradient penalty (WGAN-GP), which interpolates randomly between the true and generated samples to ensure that the transition region between the true and false samples satisfies the Lipschitz constraint. The process of feature learning is visualized to show the feature extraction process of WGAN-GP. To verify the availability of the generated samples, a stacked autoencoder (SAE) is set to classify the enhanced dataset composed of the generated samples and original samples. Furthermore, the exhibition of the loss curve indicates that WGAN-GP has better convergence and faster training speed due to the introduction of the gradient penalty. Three bearing datasets are employed to verify the effectiveness of the developed framework, and the results show that the proposed framework has an excellent performance in mechanical fault diagnosis under the imbalanced training dataset.

ECG databases are usually highly imbalanced due to the abundance of Normal ECG and scarcity of abnormal cases. As such, deep learning classifiers trained on imbalanced datasets usually perform poorly, especially on minor classes. One solution is to generate realistic synthetic ECG signals using Generative Adversarial Networks (GAN) to augment imbalanced datasets. In this study, we combined conditional GAN with WGAN-GP and developed AC-WGAN-GP in 1D form for the first time to be applied on MIT-BIH Arrhythmia dataset. We investigated the impact of data augmentation on arrhythmia classification. Two models were employed for ECG generation: (i) unconditional GAN; Wasserstein GAN with gradient penalty (WGAN-GP) is trained on each class individually; (ii) conditional GAN; one Auxiliary Classifier WGAN-GP (AC-WGAN-GP) model is trained on all classes and then used to generate synthetic beats in all classes. Two scenarios are defined for each case: (a) unscreened; all the generated synthetic beats were used, and (b) screened; only high-quality beats are selected and used, based on their Dynamic Time Warping (DTW) distance to a designated template. The state-of-the-art ResNet classifier (EcgResNet34) is trained on each of the four augmented datasets and the performance metrics (precision/recall/F1-Score micro- and macro-averaged, confusion matrices, multiclass precision-recall curves) were compared with those of the original imbalanced case. We also used a simple metric Net Improvement. All the three metrics show consistently that unconditional GAN with raw generated data creates the best improvements.

Accurate recognition of characters imprinted on ship bodies is essential for ensuring operational efficiency, safety, and security in the maritime industry. However, the limited availability of datasets of specialized digits and characters poses a challenge. To overcome this challenge, we propose a generative adversarial network (GAN) model for augmenting the limited dataset of special digits and characters in ship markings. We evaluated the performance of various GAN models, and the Wasserstein GAN with Gradient Penalty (WGAN-GP) and Wasserstein GAN with divergence (WGANDIV) models demonstrated exceptional performance in generating high-quality synthetic images that closely resemble the original imprinted characters required for augmenting the limited datasets. And the evaluation metric, Fréchet inception distance, further validated the outstanding performance of the WGAN-GP and WGANDIV models, establishing them as optimal choices for dataset augmentation to enhance the accuracy and reliability of recognition systems.

The effect of loss function on conditional generative adversarial networks

Thanks to the availability of large-scale data, deep Convolutional Neural Networks (CNNs) have witnessed success in various applications of computer vision. However, the performance of CNNs on Synthetic Aperture Radar (SAR) image classification is unsatisfactory due to the lack of well-labeled SAR data, as well as the differences in imaging mechanisms between SAR images and optical images. Therefore, this paper addresses the problem of SAR image classification by employing the Generative Adversarial Network (GAN) to produce more labeled SAR data. We propose special GANs for generating SAR images to be used in the training process. First, we incorporate the quadratic operation into the GAN, extending the convolution to make the discriminator better represent the SAR data; second, the statistical characteristics of SAR images are integrated into the GAN to make its value function more reasonable; finally, two types of parallel connected GANs are designed, one of which we call PWGAN, combining the Deep Convolutional GAN (DCGAN) and Wasserstein GAN with Gradient Penalty (WGAN-GP) together in the structure, and the other, which we call CNN-PGAN, applying a pre-trained CNN as a discriminator to the parallel GAN. Both PWGAN and CNN-PGAN consist of a number of discriminators and generators according to the number of target categories. Experimental results on the TerraSAR-X single polarization dataset demonstrate the effectiveness of the proposed method.

Seismic inverse modeling method based on generative adversarial networks

This study presents a novel approach using Wasserstein Generative Adversarial Networks with Gradient Penalty (WGAN-GP) to generate synthetic electroencephalography (EEG) and electrocardiogram (ECG) waveforms. The synthetic EEG data represent concentration and relaxation mental states, while the synthetic ECG data correspond to normal and abnormal states. By addressing the challenges of limited biophysical data, including privacy concerns and restricted volunteer availability, our model generates realistic synthetic waveforms learned from real data. Combining real and synthetic datasets improved classification accuracy from 92% to 98.45%, highlighting the benefits of dataset augmentation for machine learning performance. The WGAN-GP model achieved 96.84% classification accuracy for synthetic EEG data representing relaxation states and optimal accuracy for concentration states when classified using a fusion of convolutional neural networks (CNNs). A 50% combination of synthetic and real EEG data yielded the highest accuracy of 98.48%. For EEG signals, the real dataset consisted of 60-s recordings across four channels (TP9, AF7, AF8, and TP10) from four individuals, providing approximately 15,000 data points per subject per state. For ECG signals, the dataset contained 1200 real samples, each comprising 140 data points, representing normal and abnormal states. WGAN-GP outperformed a basic generative adversarial network (GAN) in generating reliable synthetic data. For ECG data, a support vector machine (SVM) classifier achieved an accuracy of 98% with real data and 95.8% with synthetic data. Synthetic ECG data improved the random forest (RF) classifier’s accuracy from 97% with real data alone to 98.40% when combined with synthetic data. Statistical significance was assessed using the Wilcoxon signed-rank test, demonstrating the robustness of the WGAN-GP model. Techniques such as discrete wavelet transform, downsampling, and upsampling were employed to enhance data quality. This method shows significant potential in addressing biophysical data scarcity and advancing applications in assistive technologies, human-robot interaction, and mental health monitoring, among other medical applications.

Generative adversarial networks (GANs) with gaming abilities have been widely applied in image generation. However, gamistic generators and discriminators may reduce the robustness of the obtained GANs in image generation under varying scenes. Enhancing the relation of hierarchical information in a generation network and enlarging differences of different network architectures can facilitate more structural information to improve the generation effect for image generation. In this paper, we propose an enhanced GAN via improving a generator for image generation (EIGGAN). EIGGAN applies a spatial attention to a generator to extract salient information to enhance the truthfulness of the generated images. Taking into relation the context account, parallel residual operations are fused into a generation network to extract more structural information from the different layers. Finally, a mixed loss function in a GAN is exploited to make a tradeoff between speed and accuracy to generate more realistic images. Experimental results show that the proposed method is superior to popular methods, i.e., Wasserstein GAN with gradient penalty (WGAN-GP) in terms of many indexes, i.e., Frechet Inception Distance, Learned Perceptual Image Patch Similarity, Multi-Scale Structural Similarity Index Measure, Kernel Inception Distance, Number of Statistically-Different Bins, Inception Score and some visual images for image generation.

As deep learning is showing unprecedented success in medical image analysis tasks, the lack of sufficient medical data is emerging as a critical problem. While recent attempts to solve the limited data problem using Generative Adversarial Networks (GAN) have been successful in generating realistic images with diversity, most of them are based on image-to-image translation and thus require extensive datasets from different domains. Here, we propose a novel model that can successfully generate 3D brain MRI data from random vectors by learning the data distribution. Our 3D GAN model solves both image blurriness and mode collapse problems by leveraging alpha-GAN that combines the advantages of Variational Auto-Encoder (VAE) and GAN with an additional code discriminator network. We also use the Wasserstein GAN with Gradient Penalty (WGAN-GP) loss to lower the training instability. To demonstrate the effectiveness of our model, we generate new images of normal brain MRI and show that our model outperforms baseline models in both quantitative and qualitative measurements. We also train the model to synthesize brain disorder MRI data to demonstrate the wide applicability of our model. Our results suggest that the proposed model can successfully generate various types and modalities of 3D whole brain volumes from a small set of training data.

<p>Appearance defects frequently occur during cigarette manufacturing due to production equipment or raw materials. Appearance defects significantly impact the quality of tobacco products. Since manual inspection cannot keep pace with the demands of high-speed production lines, rapid and accurate automated classification and detection are essential. Supervised learning is predominantly employed in research on automated classification of product quality appearance defects. However, supervised learning necessitates substantial labeled data for training, which is time-consuming to annotate and prone to errors. This paper proposes a self-supervised learning-based classification model for cigarette appearance defects. This is a generative adversarial network (GAN) model based on masked autoencoders (MAE), called MAE-GAN. First, this model combines MAE as a generator with a simple discriminator to form a generative adversarial network according to the principle of mask reconstruction in MAE. The generator reconstructs the images to learn their features. Second, the model also integrates MAE's loss function into the GAN's loss function. This lets the model focus on pixel-level losses during training. As a result, model performance is improved. Third, a Wasserstein GAN with gradient penalty (WGAN-GP) is added to stabilize the training process. In addition, this paper preprocesses cigarette images through segmentation and recombination. Neural networks typically accept images with the same width and height. Due to the narrow shape of cigarette images, if the image is directly transformed into a square and fed into a neural network, the details of the image will be severely lost. This paper segments the cigarette image into three parts and recombines them into images with similar length and width, greatly improving classification accuracy.</p>

Automated classification of human gait, a critical indicator of neuromuscular health, is often hindered by the dependence of supervised machine learning on extensive labeled pathological datasets, which are scarce and difficult to obtain. This paper explores the feasibility of a paradigm towards unsupervised learning, proposing a framework based on a Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP). The WGAN-GP is trained exclusively on healthy gait patterns from a single young subject's shankmounted inertial measurement unit (IMU) to build a model of healthy movement. The framework utilizes a reconstruction-based anomaly detection strategy, where abnormalities are quantified by the magnitude of the error when attempting to reconstruct a new gait cycle from the learned healthy model. Evaluated using real-world data from healthy subjects (both young and old) and for individuals with Parkinson's disease, the model showed strong performance, achieving an area under the curve (AUC) of 0.96. Notably, the framework also demonstrated sensitivity to non-pathological, age-related variations in gait. This feasibility study demonstrates the ability of the proposed WGAN-GP-based unsupervised detection method as a data-efficient and generalizable alternative. Consequently, it paves the way for future validation on larger clinical datasets to characterize mobility impairments in various disorders and age groups.

Generative adversarial networks for overlapped and imbalanced problems in impact damage classification

It is possible to generate artificial EEG signals using generative adversarial networks (GANs), but the physiological plausibility of these signals is not always considered, even though plausibility is important for the trustworthiness of generated EEG. Here, for the first time, we investigate how two key factors, input noise type (white vs. 1/f) and event-related potential (ERP)-based penalties, affect the plausibility of EEG trials generation, using a Wasserstein GAN with gradient penalty (WGAN-GP). ERP penalties were introduced as loss terms to penalize excessive high-frequency oscillations, thereby suppressing them during training. We evaluated physiological plausibility through visual inspection of ERP waveforms and power spectra (PS), statistical comparisons of EEG features (bandpower, entropy, P3 amplitude and latency, Petrosian fractal dimension, and Hjorth complexity), dimensional similarity by principal component analysis, t-distributed stochastic neighbor embedding, kernel density estimation, and decomposition of periodic and aperiodic components. Results show that WGAN-GP models using 1/f noise input preserved spectral characteristics better than white noise models, which introduced high-frequency oscillations. ERP-based penalties reduced these oscillations in white noise models, especially the 0.5-5Hz bandpass-filtered ERP penalty, improving ERP waveforms and PS. However, ERP penalties with 1/f noise models sometimes disrupted the PS. In summary, using white noise and a 0.5-5Hz passband penalty best reproduced ERP waveforms and PS, while using 1/f noise and a 0.5-3Hz passband penalty achieved the most physiologically plausible feature distributions. There is a trade-off between learning time and frequency domain features. Input noise and ERP penalties must be aligned with the intended application.

In this era of unprecedented rapid development of artificial intelligence, researchers are moving forward to develop new neural networks. Nevertheless, few have considered optimizing the extant generative adversarial network (GAN) to achieve the most effective and accurate solutions to provide optimal results for the challenges. This study aims to investigate the critical role of parameter optimization in GAN neural networks, with a particular focus on the Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) architecture applied to the generation of medical images, especially images depicting brain tumors. Therefore, the project used Kaggles brain tumor data set as a canvas to conduct an in-depth study of the impact of these parameters on the training model and generated results. For the starter, to investigate how alterations in the learning rate impact the model, this article selects a series of values for meticulous analysis to determine the most effective configuration. Then, evaluate and find a better and more suitable choice between Adam and SGD optimizers through comparison, focusing on their impact on training dynamics. As the last one, this study examines how the Tanh activation function constrains pixel values and shapes image realism through comparative results. By dissecting and understanding the interaction of these parameters in detail, we lay the foundation for optimizing GAN neural networks, increasing their efficiency, and producing accurate solutions for accurate diagnostics and healthcare applications. The journey through the labyrinth of GAN parameter tuning ultimately provided valuable insights into seamlessly synthesizing medical images.