Transfer learning enabled transformer-based generative adversarial networks for modeling and generating terahertz channels

In modern wireless communication systems, wireless channel modeling has always been a fundamental task in system design and performance optimization. Traditional channel modeling methods, such as ray-tracing and geometry- based stochastic channel models, require in-depth domain-specific knowledge and technical expertise in radio signal propagations across electromagnetic fields. To avoid these difficulties and complexities, a novel generative adversarial network (GAN) framework is proposed for the first time to address the problem of autonomous wireless channel modeling without complex theoretical analysis or data processing. Specifically, the GAN is trained by raw measurement data to reach the Nash equilibrium of a MinMax game between a channel data generator and a channel data discriminator. Once this process converges, the resulting channel data generator is extracted as the target channel model for a specific application scenario. To demonstrate, the distribution of a typical additive white Gaussian noise channel is successfully approximated by using the proposed GAN-based channel modeling framework, thus verifying its good performance and effectiveness.

Semantic segmentation of moving objects in dynamic backgrounds faces core challenges such as background interference and blurred target features. This study proposes an innovative architecture that integrates Generative Adversarial Network (GAN) with Transformers. The GAN module enhances adaptability to dynamic backgrounds through adversarial training, while the self-attention mechanism in the Transformer captures long-range semantic dependencies. A gated fusion strategy is designed to achieve dynamic balancing of multimodal features. The method employs a conditional GAN to generate dynamic background samples with variations in illumination and motion blur. A Transformer-based encoder-decoder structure is used to model global contextual relationships. A temporal attention module is introduced to incorporate motion vector fields, improving temporal consistency. Additionally, a KL-divergence (KL) constrained semantic consistency loss optimizes the plausibility of generated samples. Experiments are conducted on both a multi-dimensional simulated dataset and the real-world KITTI dataset. Results show that the proposed model achieves an average Intersection over Union (IoU) of 85.6% in standard dynamic scenes, outperforming DeepLabv3 + by 9.2% points. In low-light and high-speed motion scenarios, the robustness index reaches 92.0%, 8.5 points higher than baseline models. Ablation studies demonstrate that removing the Transformer leads to a 6.7% drop in mIoU, while excluding the feature fusion module reduces robustness by 4.0%, confirming the necessity of both components. Temporal analysis reveals that the model maintains a stable performance of 84.5–86.5% over 20-frame sequences, with fluctuation reduced by 63% compared to baseline. The adversarial training improves the model’s adaptability to lighting changes by 5.3%. The multi-head self-attention (MSA) mechanism reduces long-range misclassification by 6.7%. The gated fusion strategy lowers false positive rates in background-disturbed regions by 12.8%. This framework optimizes segmentation through a generator-segmenter feedback loop, effectively balancing dynamic background noise suppression and semantic fidelity. The contributions are threefold: (1) The first semantic segmentation framework to deeply integrate GANs and Transformers. (2) A theoretical model for dynamic feature gating and semantic consistency constraints. (3) A standardized evaluation system covering 10 dynamic background types and five illumination gradients. This study provides key technical support for real-time environmental perception in autonomous driving and intelligent surveillance, advancing both the theoretical and practical frontiers of dynamic scene understanding.

The inverse design method based on a generative adversarial network (GAN) combined with a simulation neural network (sim-NN) and the self-attention mechanism is proposed in order to improve the efficiency of GAN for designing nanophotonic devices. The sim-NN can guide the model to produce more accurate device designs via the spectrum comparison, whereas the self-attention mechanism can help to extract detailed features of the spectrum by exploring their global interconnections. The nanopatterned power splitter with a 2 μm × 2 μm interference region is designed as an example to obtain the average high transmission (>94%) and low back-reflection (<0.5%) over the broad wavelength range of 1200~1650 nm. As compared to other models, this method can produce larger proportions of high figure-of-merit devices with various desired power-splitting ratios.

Intelligent reflecting surface (IRS) is a promising new technology that is able to create a favorable wireless signal propagation environment by collaboratively reconfiguring the passive reflecting elements, yet with low hardware and energy cost. In IRS-aided wireless communication systems, channel modeling is a fundamental task for communication algorithm design and performance optimization, which however is also very challenging since in-depth domain knowledge and technical expertise in radio signal propagations are required, especially for modeling the high-dimensional cascaded base station (BS)-IRS and IRS-user channels (also referred to as the reflected channels). In this paper, we propose a model-driven generative adversarial network (GAN)-based channel modeling framework to autonomously learn the reflected channel distribution, without complex theoretical analysis or data processing. The designed GAN (also named as IRS-GAN) is trained to reach the Nash equilibrium of a minimax game between a generative model and a discriminative model, where the special structure of the reflected channels is incorporated to improve the modeling accuracy. Simulation results are presented to validate the effectiveness of the proposed IRS-GAN framework for IRS-related channel modeling.

Terahertz (THz) communication networks, combined with Intelligent Reflecting Surfaces (IRS), hold significant potential for advancing the capabilities of Sixth-Generation (6G) wireless systems. The integration of these technologies offers opportunities for ultra-fast data transmission and substantial improvements in network capacity, meeting the growing demand for higher bandwidth and faster connectivity. This study investigates the use of flying IRS-assisted Unmanned Aerial Vehicles (UAVs) within a THz communication framework, aiming to overcome challenges in environments with complex signal propagation, such as urban settings. To facilitate this integration, we introduce the Fly-IRS algorithm, designed to optimize key aspects of network performance, including user device grouping, IRS phase shifts, and UAV positioning. These factors are critical for maximizing data transmission rates, boosting network capacity, and minimizing service disruptions that affect user experience. Our approach leverages a combination of Multi-Agent Deep Reinforcement Learning (MADRL) and Particle Swarm Optimization (PSO) to effectively handle user grouping, IRS phase shift optimization, and UAV trajectory planning. By merging the strengths of both techniques, the Fly-IRS algorithm enhances system adaptability and efficiency. Simulation results demonstrate significant improvements in data rates, with the Fly-IRS algorithm delivering up to 10% better performance compared to the MADRL method alone. These findings highlight the transformative potential of integrating THz communication networks with IRS and UAV technologies, emphasizing their ability to shape future wireless communication systems. This innovation could lead to more reliable, high-capacity networks, capable of addressing the advanced requirements of 6G, offering improved service quality even in difficult environments.

To solve the challenge of single-channel blind image separation (BIS) caused by unknown prior knowledge during the separation process, we propose a BIS method based on cascaded generative adversarial networks (GANs). To ensure that the proposed method can perform well in different scenarios and to address the problem of an insufficient number of training samples, a synthetic network is added to the separation network. This method is composed of two GANs: a U-shaped GAN (UGAN), which is used to learn image synthesis, and a pixel-to-attention GAN (PAGAN), which is used to learn image separation. The two networks jointly complete the task of image separation. UGAN uses the unpaired mixed image and the unmixed image to learn the mixing style, thereby generating an image with the “true” mixing characteristics which addresses the problem of an insufficient number of training samples for the PAGAN. A self-attention mechanism is added to the PAGAN to quickly extract important features from the image data. The experimental results show that the proposed method achieves good results on both synthetic image datasets and real remote sensing image datasets. Moreover, it can be used for image separation in different scenarios which lack prior knowledge and training samples.

Deep neural networks (DNNs), e.g., convolutional neural network (CNN), are able to learn effective features from wafer maps for dimensional reduction and feature extraction. However, very large image data are needed to train DNNs to obtain high generalization performance. It is still a difficult task due to the lack of sufficient labeled images with various defects. This article proposes a semisupervised deep transfer learning algorithm called joint feature and label adversarial network (JFLAN). JFLAN uses CNNs to extract transferable features of wafer maps and then introduces a multilayer domain adaptation and pseudolabel learning block based on the generative adversarial network (GAN). This effectively reduces the distribution discrepancy and the among-class distance of the transferable features. Finally, JFLAN transfers knowledge from wafer image source data collected offline and then achieves the goal of significantly improved accuracy of wafer defect recognition and realizes online adaptive defect recognition. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The defect recognition on wafer maps plays a key role to recognize fault sources of the semiconductor manufacturing processes. Transfer learning is able to use the existing labeled data to assist in the classification of unlabeled data and is very effective to solve the problem of small samples and nonstationary generalization errors. In particular, the infusion of adversarial learning in transfer learning will provide a new idea for deep feature learning. This article provides a novel method based on transfer learning to implement wafer map defect recognition (WMDR) to quickly identify defect root causes for yield enhancement. This article provides a novel way for quality control of semiconductor manufacturing processes based on transfer and adversarial learning.

We address the problem of segmenting cell contours from microscopy images of human induced pluripotent Retinal Pigment Epithelial stem cells (iRPE) using Convolutional Neural Networks (CNN). Our goal is to compare the accuracy gains of CNN-based segmentation by using (1) un-annotated images via Generative Adversarial Networks (GAN), (2) annotated out-of-bio-domain images via transfer learning, and (3) a priori knowledge about microscope imaging mapped into geometric augmentations of a small collection of annotated images. First, the GAN learns an abstract representation of cell objects. Next, this unsupervised learned representation is transferred to the CNN segmentation models which are further fine-tuned on a small number of manually segmented iRPE cell images. Second, transfer learning is applied by pre-training a part of the CNN segmentation model with the COCO dataset containing semantic segmentation labels. The CNN model is then adapted to the iRPE cell domain using a small set of annotated iRPE cell images. Third, augmentations based on geometrical transformations are applied to a small collection of annotated images. All these approaches to training CNN-based segmentation model are compared to a baseline CNN model trained on a small collection of annotated images. For very small annotation counts, the results show accuracy improvements up to 20 % by the best approach in comparison to the accuracy achieved using a baseline U-Net model. For larger annotation counts these approaches asymptotically approach the same accuracy.

Automatic image annotation (AIA) has been adopted in different applications such as image retrieval and classification. Deep Learning is used in AIA to extract image features and then convert these features into text descriptions and labels. However, conventional AIA models that employ deep learning methods suffer from various shortcomings, such as poor annotation performance. This work proposes an AIA model based on convolutional neural networks (CNNs), generative adversarial networks (GANs), and transfer learning. GANs have attracted a lot of interest because of its ability to generate data without explicitly using probability density. Thus, it has proven its usefulness in image annotation and image augmentation. In this work, an Auxiliary classifier-GAN (ACGAN) has been used, where the discriminator predicts the class of an image rather than taking it as a given input; therefore, the stabilization of the training stage is ensured, and the generation of high-quality images is provided. Transfer learning is also used to enhance the performance of the classification. The proposed model outperforms the best state-of-the-art models in terms of MiAP, F-measure and error rate using ImageClef, ESPGame and IAPR-TC12 datasets.

Terahertz (THz) communications are celebrated as key enablers for converged localization and sensing in future sixth-generation (6G) wireless communication systems and beyond. Instead of being a byproduct of the communication system, localization in 6G is indispensable for location-aware communications. Towards this end, we aim to identify the prospects, challenges, and requirements of THz localization techniques. We first review the history and trends of localization methods and discuss their objectives, constraints, and applications in contemporary communication systems. We then detail the latest advances in THz communications and introduce THz-specific channel and system models. Afterward, we formulate THz-band localization as a 3D position/orientation estimation problem, detailing geometry-based localization techniques and describing potential THz localization and sensing extensions. We further formulate the offline design and online optimization of THz localization systems, provide numerical simulation results, and conclude by providing lessons learned and future research directions. Preliminary results illustrate that under the same transmission power and array footprint, THz-based localization outperforms millimeter wave-based localization. In other words, the same level of localization performance can be achieved at THz-band with less transmission power or a smaller footprint.

Unsupervised anomaly detection defines an abnormal event as an event that does not conform to expected behavior. In the field of unsupervised anomaly detection, it is a pioneering work that leverages the difference between a future frame predicted by a generative adversarial network and its ground truth to detect an abnormal event. Based on the work, we improve the ability of video prediction framework to detect abnormal events by enhancing the difference between prediction results for normal and abnormal events. We incorporate super-resolution and self-attention mechanism to design a generative adversarial network. We propose an auto-encoder as a generator, which incorporates dense residual networks and self-attention. Moreover, we propose a new discriminator, which introduces self-attention on the basis of a relativistic discriminator. To predict a future frame with higher quality for normal events, we impose a constraint on the motion in video prediction by fusing optical flow and gradient difference between frames. We also introduce a perception constraint in video prediction to enrich the texture details of a frame. The AUC of our method on CUHK Avenue and Shanghai Tech datasets reaches 89.2% and 75.7% respectively, which is better than most existing methods. In addition, we propose a processing flow that can realize real-time anomaly detection in videos. The average running time of our video prediction framework is 37 frames per second. Among all real-time methods for abnormal event detection in videos, our method is competitive with the state-of-the-art methods.

The actual production process is often faced with insufficient fault data. Traditional data augmentation algorithms are prone to problems such as overfitting and pattern collapse. To address the issues, a tractor transmission systems vibration fault data augmentation and diagnosis method based on deep convolutional generative adversarial network is proposed. First, the original vibration signal is converted to a time–frequency diagram, which serves as the input to the generative adversarial network. Subsequently, the two timescale update rule strategy and gradient penalty are applied to stabilize the training process of the generative adversarial network, and the self-attention mechanism is introduced to enhance the feature extraction capability of the generative adversarial network. Furthermore, an enhanced AlexNet based on the convolutional block attention module is developed to improve the diagnostic performance. The approach was verified using datasets from Case Western Reserve University (CWRU), Huazhong University of Science and Technology (HUST), and a laboratory gearbox dataset. Results show 99% diagnostic accuracy for CWRU, over 97.5% for the laboratory dataset, and 98% accuracy under a single condition for the HUST gear dataset, with accuracy exceeding 95.3% under other conditions. This indicates that the method improves diagnostic precision and model generalization under data scarcity.

The wireless channel model is a deterministric or stochastic numerical description of physical propagation enviroments based on channel measurement or analysis and simulation on the basis of propagation theories. It provides an effective and simple means to approximately express the channel characteristics of the wireless transmission. Channel models can be used for link and system-level performance evaluation, optimization, and comparative studies on wireless communications technologies, algorithms, products and system. Channel model is of great importance to all wireless communications (including 5G mobile communication) research. This chapter focuses on the progress in 5G channel measurement and channel modeling. Section 3.1 introduces the requirements for 5G wireless channel model. Section 3.2 introduces and compares five commonly used channel modeling methods. Section 3.3 introduces the knowledge on channel measurement, channel measurement system and the currently ongoing 5G wireless channel measurement activities. Section 3.4 describes several channel parameter extraction algorithms and statistical analysis of channel parameters. Section 3.5 introduces several existing channel models. Section 3.6 gives an example to show the generating process of stochastic channel models. And the last section is the summary of this chapter.

Extracting traffic information from images plays an increasingly significant role in Internet of vehicle. However, due to the high-speed movement and bumps of the vehicle, the image will be blurred during image acquisition. In addition, in rainy days, as a result of the rain attached to the lens, the target will be blocked by rain, and the image will be distorted. These problems have caused great obstacles for extracting key information from transportation images, which will affect the real-time judgment of vehicle control system on road conditions, and further cause decision-making errors of the system and even have a bearing on traffic accidents. In this paper, we propose a motion-blurred restoration and rain removal algorithm for IoV based on generative adversarial network and transfer learning. Dynamic scene deblurring and image de-raining are both among the challenging classical research directions in low-level vision tasks. For both tasks, firstly, instead of using ReLU in a conventional residual block, we designed a residual block containing three 256-channel convolutional layers, and we used the Leaky-ReLU activation function. Secondly, we used generative adversarial networks for the image deblurring task with our Resblocks, as well as the image de-raining task. Thirdly, experimental results on the synthetic blur dataset GOPRO and the real blur dataset RealBlur confirm the effectiveness of our model for image deblurring. Finally, as an image de-raining task based on transfer learning, we can fine-tune the pre-trained model with less training data and show good results on several datasets used for image rain removal.

The Viterbi algorithm is widely adopted in digital communication systems because of its capability of realizing maximum-likelihood signal sequence detection. However, implementation of the Viterbi algorithm requires instantaneous channel state information (CSI) to be available at the receiver. This is difficult to satisfy in some emerging communication systems such as molecular communications, underwater optical communications, etc, where the underlying channel models are highly complex or completely unknown. ViterbiNet, developed in the prior literature, is a promising framework to cope with this challenge, where deep learning (DL) techniques are combined with the Viterbi Algorithm to enable near-optimal signal detection without CSI. This paper offers a non-trivial variation of ViterbiNet based on generative adversarial networks (GAN). Specifically, a novel architecture using GAN is designed to directly learn the channel transition probability (CTP) from receiver observations, which is the only part of the Viterbi algorithm that is channel-dependent. With the learned CTP, the classical Viterbi algorithm can be implemented without modifications. To make the proposed architecture applicable to time-varying channels, we further develop two methods to fine-tune the learned CTP online. In the first method, pilots within each frame are exploited to update the CTP learning network; In the second method, a decision-directed approach is devised to generate training data in real-time, which is utilized to re-train the learning network. By combining these two approaches, the receiver is able to track the dynamic channel conditions without being trained from scratch. Numerical simulations demonstrate the superiority of the proposed design compared to existing methods.