Blind Image Separation Method Based on Cascade Generative Adversarial Networks

Generative adversarial networks (GANs) have shown striking performances in computer vision applications to augment virtual training samples (VTS). However, the VTS generating by GANs in the context of hyperspectral image classification suffer from structural inconsistency due to the insufficient number of training samples in order to learn high-order features from the discriminator. This work addresses the scarcity of training samples by designing a GAN, in which the performance of discriminator is improved to produce more structurally coherent VTS. In the proposed method, by splitting the discriminator into two parts, GAN undertakes two tasks: the main task is to learn to distinguish between real and fake samples, and the auxiliary task is to learn to distinguish structurally corrupted and real samples. With this setup, GAN will produce real-like VTS with a higher variation than conventional GAN. Furthermore, in order to reduce the computational cost, subspace-based dimension reduction was performed to obtain the dominant features around the training samples to generate meaningful patterns from the original ones to be used in the learning phase. Based on the experimental results on real, and well-known hyperspectral benchmark images, the proposed method improves the performance compared with GANs-related, and conventional data augmentation strategies <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .

Semantic segmentation of moving objects in dynamic backgrounds faces core challenges such as background interference and blurred target features. This study proposes an innovative architecture that integrates Generative Adversarial Network (GAN) with Transformers. The GAN module enhances adaptability to dynamic backgrounds through adversarial training, while the self-attention mechanism in the Transformer captures long-range semantic dependencies. A gated fusion strategy is designed to achieve dynamic balancing of multimodal features. The method employs a conditional GAN to generate dynamic background samples with variations in illumination and motion blur. A Transformer-based encoder-decoder structure is used to model global contextual relationships. A temporal attention module is introduced to incorporate motion vector fields, improving temporal consistency. Additionally, a KL-divergence (KL) constrained semantic consistency loss optimizes the plausibility of generated samples. Experiments are conducted on both a multi-dimensional simulated dataset and the real-world KITTI dataset. Results show that the proposed model achieves an average Intersection over Union (IoU) of 85.6% in standard dynamic scenes, outperforming DeepLabv3 + by 9.2% points. In low-light and high-speed motion scenarios, the robustness index reaches 92.0%, 8.5 points higher than baseline models. Ablation studies demonstrate that removing the Transformer leads to a 6.7% drop in mIoU, while excluding the feature fusion module reduces robustness by 4.0%, confirming the necessity of both components. Temporal analysis reveals that the model maintains a stable performance of 84.5–86.5% over 20-frame sequences, with fluctuation reduced by 63% compared to baseline. The adversarial training improves the model’s adaptability to lighting changes by 5.3%. The multi-head self-attention (MSA) mechanism reduces long-range misclassification by 6.7%. The gated fusion strategy lowers false positive rates in background-disturbed regions by 12.8%. This framework optimizes segmentation through a generator-segmenter feedback loop, effectively balancing dynamic background noise suppression and semantic fidelity. The contributions are threefold: (1) The first semantic segmentation framework to deeply integrate GANs and Transformers. (2) A theoretical model for dynamic feature gating and semantic consistency constraints. (3) A standardized evaluation system covering 10 dynamic background types and five illumination gradients. This study provides key technical support for real-time environmental perception in autonomous driving and intelligent surveillance, advancing both the theoretical and practical frontiers of dynamic scene understanding.

Generative Adversarial Network is the topic of interest in today’s research in the field of image processing and computer vision. A basic GAN model was introduced by Ian Goodfellow et al. in 2014. After that advancement in the field of research in GAN models has been application specific. In computer vision and image to image translation GANs are playing very effective role either in the case of face detection and recognition or in image resolution enhancement and image augmentation. This paper represents a concise overview of various GAN models along with their features and applications. Pix2Pix and conditional GAN models work upon paired datasets while other models like cycle GAN, discover GAN, dual GAN, info GAN, deep convolutional GAN etc. work upon unpaired datasets. Various image datasets which are commonly used for training of generator and discriminator networks are also discussed in this paper. Since partial mode collapse is a common problem to occur during training process for all models, therefore various normalization techniques are also preferred during the training of generator and discriminator networks. As the advancements in GAN models are increasing at a very fast rate, soon these models will also be preferred in commercial applications.

The inverse design method based on a generative adversarial network (GAN) combined with a simulation neural network (sim-NN) and the self-attention mechanism is proposed in order to improve the efficiency of GAN for designing nanophotonic devices. The sim-NN can guide the model to produce more accurate device designs via the spectrum comparison, whereas the self-attention mechanism can help to extract detailed features of the spectrum by exploring their global interconnections. The nanopatterned power splitter with a 2 μm × 2 μm interference region is designed as an example to obtain the average high transmission (>94%) and low back-reflection (<0.5%) over the broad wavelength range of 1200~1650 nm. As compared to other models, this method can produce larger proportions of high figure-of-merit devices with various desired power-splitting ratios.

Evolutionary conditional GANs for supervised data augmentation: The case of assessing berry number per cluster in grapevine

Crop yield estimation and prediction constitutes a key issue in agricultural management, particularly under the context of demographic pressure and climate change. Currently, the main challenge in estimating crop yields based on remotely sensed data and data-driven methods is how to cope with small datasets and the limited amount of annotated samples. In order to cope with small datasets and the limited amount of annotated samples and improve the accuracy of winter wheat yield estimation in the Guanzhong Plain, PR China, this study proposed a method of combining generative adversarial networks (GANs) and convolutional neural network (CNN) for comprehensive growth monitoring of winter wheat, in which the remotely sensed leaf area index (LAI), vegetation temperature condition index (VTCI) and meteorological data at four growth stages of winter wheat during 2012–2017 were generated as the inputs of multi-layer convolutional neural networks (CNNs), and GAN was employed to artificially increase the number of training samples. Then, a linear regression model between the simulated comprehensive growth monitoring (I) and the measured yields was established to estimate yields of winter wheat in the Guanzhong Plain pixel by pixel. The final results showed when GAN was used to double the size of the training samples, and the simulation values obtained by CNN based on augmented samples using GAN provided a better training (R2 = 0.95, RMSE = 0.05), validation (R2 = 0.54, RMSE = 0.16) and testing (R2 = 0.50, RMSE = 0.14) performance than that just using the original samples. The achieved best pixel-scale yield estimation accuracy of winter wheat (R2 = 0.50, RMSE = 591.46 kg/ha) in the Guanzhong Plain. These results showed that small samples can be enlarged by GAN, thus, more important features for reflecting the growth conditions and yields of winter wheat from the remotely sensed indices and meteorological indices can be extracted, and indicated that CNN accompanied with GAN could contribute a lot to the comprehensive growth monitoring and yield estimation of winter wheat and data augmentation methods are extremely useful for the application of small samples in deep learning.

Terahertz communications are envisioned as a promising technology for the sixth generation and beyond wireless systems, which can support wireless links with Terabits-per-second (Tbps) data rates. As the foundation of designing terahertz communications, channel modeling and characterization are crucial to scrutinize the potential of this spectrum. However, current channel modeling in the terahertz band heavily relies on time-consuming and costly measurements. Here, we propose a transfer learning enabled transformer based generative adversarial network to mitigate this problem in terahertz channel modeling. Specifically, as a fundamental building block, a generative adversarial network is exploited to generate channel parameters. To improve the accuracy, a transformer structure with a self-attention mechanism is incorporated in generative adversarial network. Still incurring errors compared with ground-truth measurement, a transfer learning is designed to solve the mismatch between the formulated network and measurement. The proposed method can achieve high accuracy in channel modeling, while requiring only rather limited amount of measurement, which is a promising complement of current channel modeling techniques.

Hyperspectral remote sensing image classification based on residual generative Adversarial Neural Networks

In the field of mobile malware detection, the problem of sample imbalance often occurs in the dataset, making the classifier unable to learn features through sufficient data during the training process. This research used the generative adversarial networks (GAN). In this paper, features of malwares are transformed into image expressions, and data is generated from a small number of malicious families to balance and expand the original dataset. We also compare other data augmentation techniques to explore whether they are beneficial to identify a small number of malicious samples. Experiments show that both traditional techniques and GAN can improve the accuracy of classification, but GAN can more effectively improve the classification model to detect that the dataset originally has a small number of datasets and the recognition accuracy is lower. The experimental results show that in the different datasets of 4,000 data in Drebin and 20,000 data in AMD, the types with a relatively small number of samples are augmented by the GAN. Compared with before and after data augmentation, the difference in F1-score accuracy can reach 5%~20%.

Due to the limitation of the establishment of pathological voice database, the number of samples is often insufficient and imbalanced, which causes the defect of many deep learning methods to be used in pathological voice database. In this paper, generative adversarial network (GAN) is used to generate the features of voice data to improve the imbalanced distribution of samples. GAN uses a generator and a discriminator for adversarial training to generate data similar to the original data distribution. Back propagation generative adversarial network (BPGAN) and deep convolution generative adversarial network (DCGAN) are used to generate vector features and matrix features respectively. It is found that the recall of minority sample is significantly higher when using balanced training set added data generated by GAN than imbalanced training set. Scatter diagram of generated feature vector and grayscale diagram of feature matrix are drawn. By adding the feature vector generated by BPGAN and DCCGAN, the recall of the test set for the minority sample is increased by 4.75%, and 21% respectively. The results show that GAN can generate different feature sets of pathological voice effectively.

Among the wide variety of image generative models, two models stand out: Variational Auto Encoders (VAE) and Generative Adversarial Networks (GAN). GANs can produce realistic images, but they suffer from mode collapse and do not provide simple ways to get the latent representation of an image. On the other hand, VAEs do not have these problems, but they often generate images less realistic than GANs. In this article, we explain that this lack of realism is partially due to a common underestimation of the natural image manifold dimensionality. To solve this issue we introduce a new framework that combines VAE and GAN in a novel and complementary way to produce an auto-encoding model that keeps VAEs properties while generating images of GAN-quality. We evaluate our approach both qualitatively and quantitatively on five image datasets.

Recently, deep learning has developed rapidly, which has made significant progress in tasks such as target detection and classification. Compared with traditional methods, using deep learning techniques contribute to achieve higher detection accuracy, recognition rate, and other better performance with big data set. In the fields of radar and sonar especially like underwater acoustic signals, training samples are scarce due to the difficulty of the collection or security reason, which leads to poor performance of the classification models, as those need big training samples. In this paper, we present a novel framework based on Generative Adversarial Networks (GAN) to resolve the problem of insufficient samples for the underwater acoustic signals. Our method preprocesses the audio samples to the gray-scale spectrum images, so that, those can fit the GAN to captures the features and reduce the complexity at the same time. Then our method utilizes an independent classification network outside the GAN to evaluate the generated samples by GAN. The experimental results show that the samples generated by our approach outperform existing methods with higher quality, which can significantly improve the prediction accuracy of the classification model.

Imbalanced image datasets consist of image datasets where there is a significant disparity in the number of samples across different classes. With imbalanced image datasets, learning algorithms often tend to be biased toward the majority class samples. This leads to poor classification of minority class samples as their training is not properly conducted. It becomes more complicated when the number of samples in the minority class is very low. In this paper, a novel hierarchical algorithm is proposed for generating new data using Capsule Generative Adversarial Networks (CapsGAN) to address the class imbalance problem in imbalanced image datasets. Unlike common GAN models, the proposed method incorporates an auxiliary CapsNet to identify high‐value images in both minority and majority classes. This identification is based on the ability to detect complex relationships between low‐level and high‐level features present in capsule networks. Furthermore, the proposed CapsGAN model is conditioned to generate minority class samples based on feature vectors of last capsule layer to achieve a more balanced image dataset. For evaluating the performance of the proposed model, an image dataset called CICS was collected and introduced. Extensive experiments were also conducted using various online image datasets from different domains, with varying numbers of classes and data sizes. The experimental results demonstrated that the proposed model can generate high‐quality samples in cases where the image dataset or the number of minority class samples is relatively small. Furthermore, the proposed model has maintained an accuracy of over 80% in an imbalanced ratio of 1:60.

Unsupervised anomaly detection defines an abnormal event as an event that does not conform to expected behavior. In the field of unsupervised anomaly detection, it is a pioneering work that leverages the difference between a future frame predicted by a generative adversarial network and its ground truth to detect an abnormal event. Based on the work, we improve the ability of video prediction framework to detect abnormal events by enhancing the difference between prediction results for normal and abnormal events. We incorporate super-resolution and self-attention mechanism to design a generative adversarial network. We propose an auto-encoder as a generator, which incorporates dense residual networks and self-attention. Moreover, we propose a new discriminator, which introduces self-attention on the basis of a relativistic discriminator. To predict a future frame with higher quality for normal events, we impose a constraint on the motion in video prediction by fusing optical flow and gradient difference between frames. We also introduce a perception constraint in video prediction to enrich the texture details of a frame. The AUC of our method on CUHK Avenue and Shanghai Tech datasets reaches 89.2% and 75.7% respectively, which is better than most existing methods. In addition, we propose a processing flow that can realize real-time anomaly detection in videos. The average running time of our video prediction framework is 37 frames per second. Among all real-time methods for abnormal event detection in videos, our method is competitive with the state-of-the-art methods.

The actual production process is often faced with insufficient fault data. Traditional data augmentation algorithms are prone to problems such as overfitting and pattern collapse. To address the issues, a tractor transmission systems vibration fault data augmentation and diagnosis method based on deep convolutional generative adversarial network is proposed. First, the original vibration signal is converted to a time–frequency diagram, which serves as the input to the generative adversarial network. Subsequently, the two timescale update rule strategy and gradient penalty are applied to stabilize the training process of the generative adversarial network, and the self-attention mechanism is introduced to enhance the feature extraction capability of the generative adversarial network. Furthermore, an enhanced AlexNet based on the convolutional block attention module is developed to improve the diagnostic performance. The approach was verified using datasets from Case Western Reserve University (CWRU), Huazhong University of Science and Technology (HUST), and a laboratory gearbox dataset. Results show 99% diagnostic accuracy for CWRU, over 97.5% for the laboratory dataset, and 98% accuracy under a single condition for the HUST gear dataset, with accuracy exceeding 95.3% under other conditions. This indicates that the method improves diagnostic precision and model generalization under data scarcity.