Adversarial Data Hiding in Digital Images

Generative Adversarial Networks, as a powerful generative model, show great potential in generating adversarial samples and defending against adversarial attacks. In this paper, using Generative Adversarial Networks (GANs) as the basic framework, we design a network attack sample generation method based on Deep Convolutional Generative Adversarial Networks (DCGANs) and an adversarial sample defence method based on multi-scale GANs, and verify the practicality of the two methods through experiments, respectively. Compared with the three adversarial sample generation methods of AE-CDA, AE-DEEP and AE-ATTACK, the DCGAN-based adversarial sample generation method in this paper can interfere with the detection function of the anomaly detection model more effectively, and has better stability and versatility, and can maintain a relatively stable attack effect on a wide range of models and datasets. On the MNIST dataset, the classification accuracy of the adversarial sample defence method proposed in this paper is only slightly lower than that of the APE-GAN defence method on the JSMA adversarial samples, with a maximum classification accuracy of 98.69%. The maximum classification accuracy reaches 98.69%, and the time consumption is 1.5 s, which is only slightly larger than that of the APE-GAN defence method of 1.2 s. Thus, the time consumption of this paper’s multi-scale GAN-based adversarial sample defense method is smaller or equal to that of other comparative defense methods when systematic errors are ignored. The purpose of this paper is to provide a technical reference on how to eliminate adversarial perturbations using generative adversarial networks.

It is important to detect adversarial samples in the physical world that are far away from the training data distribution. Some adversarial samples can make a machine learning model generate a highly overconfident distribution in the testing stage. Thus, we proposed a mechanism for detecting adversarial samples based on semisupervised generative adversarial networks (GANs) with an encoder-decoder structure; this mechanism can be applied to any pretrained neural network without changing the network’s structure. The semisupervised GANs also give us insight into the behavior of adversarial samples and their flow through the layers of a deep neural network. In the supervised scenario, the latent feature (or the discriminator’s output score information) of the semi-supervised GAN and the target network's logit information are used as the input of logistic regression classifier to detect the adversarial samples. In the unsupervised scenario, first, we proposed a one-class classier based on the semisupervised Gaussian mixture conditional generative adversarial network (GM-CGAN) to fit the joint feature information of the normal data, and then, we used a discriminator network to detect normal data and adversarial samples. In both supervised scenarios and unsupervised scenarios, experimental results show that our method outperforms latest methods.

Although adversarial sampling is a trendy topic in computer vision, very few works consider the integral constraint: The result of the attack is a digital image whose pixel values are integers. This is not an issue at first sight since applying a rounding after forging an adversarial sample trivially does the job. Yet, this paper shows theoretically and experimentally that this operation has a big impact. The adversarial perturbations are fragile signals whose quantization destroys its ability to delude an image classifier. This paper presents a new quantization mechanism which preserves the adversariality of the perturbation. Its application outcomes to a new look at the lessons learnt in adversarial sampling.

Magnetic induction tomography (MIT) is a technique used for imaging electromagnetic properties of objects using eddy current effects. The non-linear characteristics had led to more difficulties with its solution especially in dealing with low conductivity imaging materials such as biological tissues. Two methods that could be applied for MIT image processing which is the Generative Adversarial Network (GAN) and the Algebraic Reconstruction Technique (ART). ART is widely used in the industry due to its ability to improve the quality of the reconstructed image at a high scanning speed. GAN is an intelligent method which would be able to carry out the training process. In the GAN method, the MIT principle is used to find the optimum global conductivity distribution and it is described as a training process and later, reconstructed by a generator. The output is an approximate reconstruction of the distribution’s internal conductivity image. Then, the results were compared with the previous traditional algorithm, namely the regularization algorithm of BPNN and Tikhonov Regularization method. It turned out that GAN had able to adjust the non-linear relationship between input and output. GAN was also able to solve non-linear problems that cannot be solved in the previous traditional algorithms, namely Back Propagation Neural Network (BPNN) and Tikhonov Regularization method. There are several other intelligent algorithms such as CNN (Convolution Neural Network) and K-NN (K-Nearest Neighbor), but such algorithms have not been able to produce the expected image quality. Thus, further study is still needed for the improvement of the image quality. The expected result in this study is the comparison of these two techniques, namely ART and GAN to get the best results on the image reconstruction using MIT. Thus, it is shown that GAN is a better candidate for this purpose.

Generative artificial intelligence, especially with regard to the generative adversarial network (GAN), is an important research area in radiology as evidenced by a number of literature reviews on the role of GAN in radiology published in the last few years. However, no review article about GAN in pediatric radiology has been published yet. The purpose of this paper is to systematically review applications of GAN in pediatric radiology, their performances, and methods for their performance evaluation. Electronic databases were used for a literature search on 6 April 2023. Thirty-seven papers met the selection criteria and were included. This review reveals that the GAN can be applied to magnetic resonance imaging, X-ray, computed tomography, ultrasound and positron emission tomography for image translation, segmentation, reconstruction, quality assessment, synthesis and data augmentation, and disease diagnosis. About 80% of the included studies compared their GAN model performances with those of other approaches and indicated that their GAN models outperformed the others by 0.1–158.6%. However, these study findings should be used with caution because of a number of methodological weaknesses. For future GAN studies, more robust methods will be essential for addressing these issues. Otherwise, this would affect the clinical adoption of the GAN-based applications in pediatric radiology and the potential advantages of GAN could not be realized widely.

PurposeMagnetic resonance imaging (MRI)-guided radiotherapy enables adaptive treatment plans based on daily anatomical changes and accurate organ visualization. However, the bias field artifact can compromise image quality, affecting diagnostic accuracy and quantitative analyses. This study aims to assess the impact of bias field correction on 0.35 T pelvis MRIs by evaluating clinical anatomy visualization and generative adversarial network (GAN) auto-segmentation performance.Materials and methods3D simulation MRIs from 60 prostate cancer patients treated on MR-Linac (0.35 T) were collected and preprocessed with the N4ITK algorithm for bias field correction. A 3D GAN architecture was trained, validated, and tested on 40, 10, and 10 patients, respectively, to auto-segment the organs at risk (OARs) rectum and bladder. The GAN was trained and evaluated either with the original or the bias-corrected MRIs. The Dice similarity coefficient (DSC) and 95th percentile Hausdorff distance (HD95th) were computed for the segmented volumes of each patient. The Wilcoxon signed-rank test assessed the statistical difference of the metrics within OARs, both with and without bias field correction. Five radiation oncologists blindly scored 22 randomly chosen patients in terms of overall image quality and visibility of boundaries (prostate, rectum, bladder, seminal vesicles) of the original and bias-corrected MRIs. Bennett’s S score and Fleiss’ kappa were used to assess the pairwise interrater agreement and the interrater agreement among all the observers, respectively.ResultsIn the test set, the GAN trained and evaluated on original and bias-corrected MRIs showed DSC/HD95th of 0.92/5.63 mm and 0.92/5.91 mm for the bladder and 0.84/10.61 mm and 0.83/9.71 mm for the rectum. No statistical differences in the distribution of the evaluation metrics were found neither for the bladder (DSC: p = 0.07; HD95th: p = 0.35) nor for the rectum (DSC: p = 0.32; HD95th: p = 0.63). From the clinical visual grading assessment, the bias-corrected MRI resulted mostly in either no change or an improvement of the image quality and visualization of the organs’ boundaries compared with the original MRI.ConclusionThe bias field correction did not improve the anatomy visualization from a clinical point of view and the OARs’ auto-segmentation outputs generated by the GAN.

The increasing complexity of power systems as well as the rapid expansion of renewable energy resources and deregulation of electricity industry lead to continual demand for operating power systems at higher efficiencies. The modern power systems are pushed to operate close to their secure limits. When subjected to a severe contingency, the stressed power systems may lose its dynamic security, resulting in catastrophic consequences such as cascading failure and even wide-spread blackout. Under this new reality, online and real-time power system stability assessments have great impacts on avoiding blackout events. The traditional stability methods are mainly based on time-domain simulation, which are not fast enough to satisfy the online and real-time application requirements. In recent years, with the development of advanced real-time system monitoring and communication techniques, the data-driven methods have been identified as powerful tools to help power systems counteract large disturbance and avoid instability. To solve above threats, this Ph.D. research aims to develop data-analytics methodologies from power grid measurement data to recognize the system stability, covering both pre-fault dynamic security assessment (DSA) and post-fault real-time short-term voltage stability (STVS) assessment areas under some extreme scenarios and practical issues. Specific objectives of this Ph.D. research are to develop an intelligent system for the future smart grid. Based on several kinds of promising advanced data-analytics technologies, including randomized learning, incremental broad learning (BL), transfer learning (TL), adversarial machine learning (ML), and generative adversarial network (GAN), etc, an intelligent system will be developed to interpret the power grid stability characteristics, and to support stability-constrained operation control actions. Hence, it can make power systems achieve seven goals, including extensibility, efficiency, robustness, security, personalization, reliability, and effectiveness. The outcomes of this Ph.D. research will enable a more secure, intelligent, and knowledge-transparent future of the next-generation smart grid. The main contributions through this Ph.D. research are as follows: Firstly, the temporal-adaptive data-driven methods are proposed based on optimized hybrid ensemble learning mechanism for online STVS assessment. Compared to the traditional data-driven stability methods, the proposed methods can balance the tradeoff between the stability accuracy and speed and select the most suitable Pareto set. Secondly, in the area of DSA, an online updating DSA method is developed by using incremental BL technique. The incremental BL method only needs to calculate the pseudoinverse of new state matrix without the computations of the whole output weights matrix. In this way, it does not need re-training the whole model to continuously maintain and improve the online DSA accuracy. Besides, the proposed method can achieve under three different increment scenarios, including increment of enhancement hidden nodes, features, and new training instances. Thirdly, in the area of missing phasor measurement unit (PMU) data, two different fully robust data-driven stability methods are developed, respectively for pre-fault DSA and post-fault STVS assessment. The former one is based on GAN without observability-constrained feature subsets to ensure its availability under any PMU missing scenario. The latter one is based on super-resolution perception and incremental BL, which can achieve both satisfactory stability accuracy and speed. Fourthly, in order to accurately quantify the robustness of the ML-based model under adversarial perturbations on safety-critical power system stability problem, an adversarial robustness verification method for data-driven stability models is proposed. Besides, a model-free and attack-independent robust index are defined to evaluate the ability of the ML-based stability models against adversarial examples, which can be used to select the candidate model and provide formal robustness guarantee for stability application in practice. Rigorous mathematic proof is provided for the robust index under both linear and non-linear binary scenarios. Moreover, the proposed robust index can be used to mitigate the adversarial examples via adversarial training. Finally, three different TL-based DSA methods are proposed to use one known fault to evaluate other related but different unknown faults. The first two methods aim to reduce the distribution differences between the trained data and unknown data with the complete unlearned faults. The third one considers the different faults with incomplete data scenario. Through applying the adversarial training process, the DSA model trained by one fault can work for a different but related fault by confusing domain discriminator. With the property of feature extractor network, the proposed integrated TL-based method can solve the missing data issue for the unknown faults. All the methodologies have been tested on benchmark power systems to verify their effectiveness, and comparative and comparative studies with existing methods in the literature are conducted where applicable.

Owing to their great success in unsupervised tasks, generative adversarial networks (GANs) are widely adopted for supervised (conditional) image-generation tasks such as in-painting. Unfortunately, designing an objective function using GANs is not trivial. For supervised tasks, the generator trained only with GAN loss does not yield the corresponding target for the given input because GAN is trained to match the data distribution, not to find the exact answer. Therefore, the loss function of a generator is often formulated by linearly combining supervised loss and GAN loss as similar to multi-task learning, expecting that each loss's weakness is complemented. Contrary to expectation, both losses cause a conflict in practice due to different optimum of each loss, yielding low objective and subjective image qualities. To address this problem, we empirically investigated the conflict caused by using a conventional GAN with pixel-wise losses; we then propose a novel (relativistic) accuracy-aware discriminator. Based on the proposed discriminator, we developed an accuracy-aware GAN (AAGAN) and proved its optimality under an ideal assumption. We then propose a relativistic accuracy-aware GAN (RAAGAN) by considering practical assumptions. Experimental results on supervised tasks demonstrated that the proposed schemes alleviated the competition between losses and outperformed conventional GANs in terms of both objective and subjective qualities.

PurposeTo develop generative adversarial networks (GANs) that synthesize realistic anterior segment optical coherence tomography (AS-OCT) images and evaluate deep learning (DL) models that are trained on real and synthetic datasets for detecting angle closure.MethodsThe GAN architecture was adopted and trained on the dataset with AS-OCT images collected from the Joint Shantou International Eye Center of Shantou University and the Chinese University of Hong Kong, synthesizing open- and closed-angle AS-OCT images. A visual Turing test with two glaucoma specialists was performed to assess the image quality of real and synthetic images. DL models, trained on either real or synthetic datasets, were developed. Using the clinicians’ grading of the AS-OCT images as the reference standard, we compared the diagnostic performance of open-angle vs. closed-angle detection of DL models and the AS-OCT parameter, defined as a trabecular-iris space area 750 µm anterior to the scleral spur (TISA750), in a small independent validation dataset.ResultsThe GAN training included 28,643 AS-OCT anterior chamber angle (ACA) images. The real and synthetic datasets for DL model training have an equal distribution of open- and closed-angle images (all with 10,000 images each). The independent validation dataset included 238 open-angle and 243 closed-angle AS-OCT ACA images. The image quality of real versus synthetic AS-OCT images was similar, as assessed by the two glaucoma specialists, except for the scleral spur visibility. For the independent validation dataset, both DL models achieved higher areas under the curve compared with TISA750. Two DL models had areas under the curve of 0.97 (95% confidence interval, 0.96–0.99) and 0.94 (95% confidence interval, 0.92–0.96).ConclusionsThe GAN synthetic AS-OCT images appeared to be of good quality, according to the glaucoma specialists. The DL models, trained on all-synthetic AS-OCT images, can achieve high diagnostic performance.Translational RelevanceThe GANs can generate realistic AS-OCT images, which can also be used to train DL models.

The vast application of artificial intelligence in numerous fields—image generation being one of them—has been made possible by the quick development of deep learning. Generative Adversarial Networks (GAN) can generate high-quality images through an adversarial training mechanism. The use and performance of GAN and its conditional variation, CGAN, in the field of handwritten digital image generation, are thoroughly examined in this research. The basic GAN and CGAN models, based on the PyTorch deep learning framework and the Modified National Institute of Standards and Technology (MNIST) dataset, are applied to generate handwritten digital images respectively. To assess and compare the variations between the two models concerning the fineness of image generation, the loss changes, and other relevant factors, the generation outcomes and the loss changes that occur during the training phase are documented. The experimental results demonstrate that, compared with the basic GAN, CGAN exhibits notable advantages in terms of image quality stability, the avoidance of model collapse, and the control of image categories. Furthermore, an investigation of other cutting-edge generating models indicates that there is still room for optimization in the CGAN network structure to improve its performance for increasingly intricate generative tasks.

Training Generative Adversarial Networks (GAN) to generate high-quality images typically requires large datasets. Network pruning during training has recently emerged as a significant advancement for data-efficient GAN. However, simple and straightforward pruning can lead to the risk of losing key information, resulting in suboptimal results due to GAN’s competitive dynamics between generator (G) and discriminator (D). Addressing this, we present RG-GAN, a novel approach that marks the first incorporation of dynamic weight regeneration and pruning in GAN training to improve the quality of the generated samples, even with limited data. Specifically, RG-GAN initiates layer-wise dynamic pruning by removing less important weights to the quality of the generated images. While pruning enhances efficiency, excessive sparsity within layers can pose a risk of model collapse. To mitigate this issue, RG-GAN applies a dynamic regeneration method to reintroduce specific weights when they become important, ensuring a balance between sparsity and image quality. Though effective, the sparse network achieved through this process might eliminate some weights important to the combined G and D performance, a crucial aspect for achieving stable and effective GAN training. RG-GAN addresses this loss of weights by integrating learned sparse network weights back into the dense network at the previous stage during a follow-up regeneration step. Our results consistently demonstrate RG-GAN’s robust performance across a variety of scenarios, including different GAN architectures, datasets, and degrees of data scarcity, reinforcing its value as a generic training methodology. Results also show that data augmentation exhibits improved performance in conjunction with RG-GAN. Furthermore, RG-GAN can achieve fewer parameters without compromising, and even enhancing, the quality of the generated samples. Code can be found at this link: https://github.com/IntellicentAI-Lab/RG-GAN

EEG-to-image translation is a promising direction in brain-computer interfaces (BCIs), enabling advanced forms of communication by generating visual representations directly from brain signals. However, EEG data are inherently noisy and exhibit substantial inter-subject variability, making them challenging to model. Existing approaches rely on Generative Adversarial Networks (GANs), which are powerful but notoriously unstable during training. In this study, we investigated whether incorporating a forward diffusion process into the GAN training loop could improve the training stability and performance. We propose DiffGAN, a diffusion-enhanced GAN that combines an LSTM-based feature extractor trained using contrastive learning with a GAN generator regularized through a forward diffusion process. To evaluate the performance of our proposed model, we conducted experiments on an EEG dataset collected during observation of object images from 10 distinct categories. The performance of DiffGAN was compared to a baseline Deep Convolutional GAN using the Inception Score (IS) and entropy-based metrics that evaluated the quality and diversity of generated images. Results showed that DiffGAN consistently outperformed the baseline model across all categories, improving the average IS from 4.01 to 4.63. The improvements were primarily driven by gains in diversity, with a slight decrease in image quality. These findings demonstrate that diffusion-based regularization improves the stability of EEG-to-image GANs, offering a promising direction for future BCI applications.

Surface defect recognition (SDC) is essential in intelligent manufacturing. Deep learning (DL) is a research hotspot in SDC. Limited defective samples are available in most real-world cases, which poses challenges for DL methods. Given such circumstances, generating defective samples by generative adversarial networks (GANs) is applied. However, insufficient samples and high-frequency texture details in defects make GANs very hard to train, yield mode collapse, and poor image quality, which can further impact SDC. To solve these problems, this article proposes a new GAN called contrastive GAN, which can be trained to generate diverse defects with only extremely limited samples. Specifically, a shared data augmentation (SDA) module is proposed for avoiding overfitting. Then, a feature attention matching (FAM) module is proposed to align features for improving the quality of generated images. Finally, a contrastive loss based on hypersphere is employed to constrain GANs to generate images that differ from the traditional transform. Experiments show that the proposed GAN generates defective images with higher quality and lower variance between real defects compared to other GANs. Synthetic images contribute to pretrained DL networks with accuracies of up to 95.00%–99.56% for Northeastern University (NEU) datasets of different sizes and 91.84% for printed circuit board (PCB) cases, which proves the effectiveness of the proposed method.

Frame interpolation, which generates an intermediate frame given adjacent ones, finds various applications such as frame rate up-conversion, video compression, and video streaming. Instead of using complex network models and additional data involved in the state-of-the-art frame interpolation methods, this paper proposes an approach based on an end-to-end generative adversarial network. A combined loss function is employed, which jointly considers the adversarial loss (difference between data models), reconstruction loss, and motion blur degradation. The objective image quality metric values reach a PSNR of 29.22 dB and SSIM of 0.835 on the UCF101 dataset, similar to those of the state-of-the-art approach. The good visual quality is notably achieved by approximately one-fifth computational time, which entails possible real-time frame rate up-conversion. The interpolated output can be further improved by a GAN based refinement network that better maintains motion and color by image-to-image translation.

This study aims to explore the feasibility of employing generative adversarial networks (GAN) to generate synthetic contrast-enhanced ultrasound (CEUS) from grayscale ultrasound images of patients with thyroid nodules while dispensing with the need for ultrasound contrast agents injection. Patients who underwent preoperative thyroid CEUS examinations between January 2020 and July 2022 were collected retrospectively. The cycle-GAN framework integrated paired and unpaired learning modules was employed to develop the non-invasive image generation process. The syntheticCEUS imageswas generated in three phases: pre-arterial, plateau, and venous. The evaluation included quantitative similarity metrics, classification performance, and qualitative assessment by radiologists. CEUS videos of 360 thyroid nodules from 314 patients (45 years ± 12 [SD]; 272 women) in the internal dataset and 202 thyroid nodules from 183 patients (46 years ± 13 [SD]; 148 women) in the external dataset were included. In the external testing dataset, quantitative analysis revealed a significant degree of similarity between real and synthetic CEUS images (structure similarity index, 0.89 ± 0.04; peak signal-to-noise ratio, 28.17 ± 2.42). Radiologists deemed 126 of 132 [95%] synthetic CEUS images diagnostically useful. The accuracy of radiologists in distinguishing between real and synthetic images was 55.6% (95% CI: 0.49, 0.63), with an AUC of 61.0% (95% CI: 0.65, 0.68). No statistically significant difference (p > 0.05) was observed when radiologists assessed peak intensity and enhancement patterns using real CEUS and synthetic CEUS. Both quantitative analysis and radiologist evaluations exhibited that synthetic CEUS images generated by generative adversarial networks were similar to real CEUS images. QuestionIt is feasible to generate synthetic thyroid contrast-enhanced ultrasound images using generative adversarial networks without ultrasound contrast agents injection. FindingsCompared to real contrast-enhanced ultrasound images, synthetic contrast-enhanced ultrasound images exhibited high similarity and image quality. Clinical relevanceThis non-invasive and intelligent transformation may reduce the requirement for ultrasound contrast agents in certain cases, particularly in scenarios where ultrasound contrast agents administration is contraindicated, such as in patients with allergies, poor tolerance, or limited access to resources.