A Novel GAN based on Progressive Growing Transformer with Capsule Embedding

Inspired by the state-of-the-art performance of MLP-Mixer on image classification task, multilayer perceptron (MLP) model attracts a number of researcher's attention again. Although various MLP architectures have been proposed, most of them focus on image classification domain. In this paper, we extend MLP to the task of image generation based on generative adversarial network (GAN). The pure MLP model is not friendly to small dataset because it is a data-hungry architecture. Thus, we leverage a hybrid model to solve the problem which uses MLP blocks as generator and CNN blocks as discriminator. Experimental results demonstrate that our model outperforms the pure CNN network on CIFAR- 10 dataset in view of two different evaluation metrics. Besides, we apply eight widely popular MLP structures to our generator to validate which one achieves the most excellent performance in image generation task.

paper, we focus on exploring the relationship between weave patterns and their mechanical properties in woven fiber composites through Machine Learning. Specifically, we explore the interactions between woven architectures and in-plane stiffness properties through Deep Convolutional Neural Network (DCNN) and Generative Adversarial Network (GAN). Our research is important for exploring how woven composite’s pattern is related to its mechanical properties and accelerating woven composite design as well as optimization. We focus on two tasks: (1) Stiffness prediction: Predicting in-plane stiffness properties for given weave patterns. Our DCNN extracts high-level features through several convolutional and fully connected layers to determine the final predictions. (2) Weave pattern prediction: Predicting weave patterns for target stiffness properties, which can be treated as the reverse task of the first one. Due to many-to-one mapping between weave patterns and the composite properties, we utilize a Decoder Neural Network as our baseline model and compare its performance with GAN and Genetic Algorithm. We represent the weave patterns as 2D checkerboard models and use finite element analysis (FEA) to determine in-plane stiffness properties, which serve as input data for our ML framework. We show that: (1) for stiffness prediction, DCNN can predict stiffness values for a given weave pattern with relatively high accuracy (above 93%); (2) for weave pattern prediction, the GAN model gives the best prediction accuracy (above 92%) while Decoder Neural Network has the best time efficiency. HAOTIAN FENG

Brain tumors are formed due to the abnormal growth of the cells that get multiplied and become in uncontrollable perspective. Tumors can damage brain cells by pressing down the skull, which consociate begins in, which negatively affects human health. In advanced stages, a brain tumor is a more dangerous infection that cannot be relieved. Brain tumor detection and early prevention are necessary in today's world. An extreme learning machine (ELM) is a widely adopted algorithm in machine learning. It is proposed to use classification models in brain tumor imaging. This classification is based on the techniques implemented: Convolutional Neural Networks(CNN) and Generative Adversarial Networks(GAN). CNN efficiently solves the convex optimization problem and is faster, requiring less human effort. The algorithmic architecture of GAN uses two neural networks, pocking one against the other. These networks are implemented in various fields to classify brain Tumor images. The present study mainly aims to introduce a new proposed classification system of preschool children brain imaging with Hybrid Convolutional Neural Networks and with the techniques of GAN. The proposed technique is compared with the existing hybrid-CNN and hybrid-GAN techniques. The outcomes are encouraging because the loss is deduced, and the accuracy facet increases. The proposed system achieved a training accuracy of 97.8% and a validation accuracy of 89%. The outcomes of the studies show that ELM in the platform of GAN for preschool children brain imaging classification has achieved higher predictive performance than the traditional classification mechanisms in increasingly complex scenarios. Time elapsed for training brain images samples finds inference value for training samples and time elapsed value increased by 28.9855%. Approximation ratio for cost based on probability, finding Approximation ratio for low probability range is increased by 88.1%. The combination of CNN, GAN and hybrid-CNN, hybrid-GAN, and hybrid CNN + GAN, compared with the proposed hybrid system, increased Detection latency for low range learning rate by 3.31%.

We consider the task of photo-realistic unconditional image generation (generate high quality, diverse samples that carry the same visual content as the image) on mobile platforms using Generative Adversarial Networks (GANs). In this paper, we propose a novel approach to trade-off image generation accuracy of a GAN for the energy consumed (compute) at run-time called Scale-Energy Trade-off GAN (SETGAN). GANs usually take a long time to train and consume a huge memory hence making it difficult to run on edge devices. The key idea behind SETGAN for an image generation task is for a given input image, we train a GAN on a remote server and use the trained model on edge devices. We use SinGAN, a single image unconditional generative model, that contains a pyramid of fully convolutional GANs, each responsible for learning the patch distribution at a different scale of the image. During the training process, we determine the optimal number of scales for a given input image and the energy constraint from target edge device. Results show that with the SETGAN's unique client-server based architecture, we were able to achieve 56% gain in energy for a loss of 3% to 12% SSIM accuracy. Also, with the parallel multi-scale training, we obtain around 4x gain in training time on the server.

In recent years, advances in image restoration have prominently featured Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs).Specifically, in seismic image denoising, the DnCNN model, rooted in CNN, employs residual learning combined with batch normalization for a streamlined approach to denoising.Alternatively, SRGAN, leveraging GAN and super-resolution techniques, trains its generative model using a perceptual loss function, accentuating perceptual disparities between generated and authentic images to boost visual quality.Recognizing the importance of preserving salient seismic data, the Constrained-DnCNN model has been introduced, refining seismic data interpretation.Additionally, the CR-SRGAN model, targeting super-resolution of artifacts and color restoration, deviates from conventional training datasets acquired via high-resolution image interpolation and downsampling.Upon comparative analysis, the DCGAN, amalgamating features of CNN and GAN, stands out for harnessing CNN's robust feature extraction.This bolsters the model's capacity to adeptly fit real data distributions.DCGAN's potential spans aspects like multimodal fusion, model interpretability, trustworthiness, and image restoration quality, marking it a promising research avenue.

Hyperspectral remote sensing image classification based on residual generative Adversarial Neural Networks

Thanks to the availability of large-scale data, deep Convolutional Neural Networks (CNNs) have witnessed success in various applications of computer vision. However, the performance of CNNs on Synthetic Aperture Radar (SAR) image classification is unsatisfactory due to the lack of well-labeled SAR data, as well as the differences in imaging mechanisms between SAR images and optical images. Therefore, this paper addresses the problem of SAR image classification by employing the Generative Adversarial Network (GAN) to produce more labeled SAR data. We propose special GANs for generating SAR images to be used in the training process. First, we incorporate the quadratic operation into the GAN, extending the convolution to make the discriminator better represent the SAR data; second, the statistical characteristics of SAR images are integrated into the GAN to make its value function more reasonable; finally, two types of parallel connected GANs are designed, one of which we call PWGAN, combining the Deep Convolutional GAN (DCGAN) and Wasserstein GAN with Gradient Penalty (WGAN-GP) together in the structure, and the other, which we call CNN-PGAN, applying a pre-trained CNN as a discriminator to the parallel GAN. Both PWGAN and CNN-PGAN consist of a number of discriminators and generators according to the number of target categories. Experimental results on the TerraSAR-X single polarization dataset demonstrate the effectiveness of the proposed method.

From real–time tumor classification to operative outcome prediction, applications of machine learning to neurosurgery are powerful. However, the translation of many of these applications are restricted by the lack of “big data” in neurosurgery. Important restrictions in patient privacy and sharing of imaging data reduce the diversity of the datasets used to train resulting models and therefore limit generalizability. Synthetic learning is a recent development in machine learning that generates synthetic data from real data and uses the synthetic data to train downstream models while preserving patient privacy. Such an approach has yet to be successfully demonstrated in the spine surgery domain. Spine radiographs were collected from the VinDR–SpineXR dataset, with 1470 labeled as abnormal and 2303 labeled as normal. A conditional generative adversarial network (GAN) was trained on the radiographs to generate a spine radiograph and normal/abnormal label. A modified conditional GAN (SpineGAN) was trained on the same task. A convolutional neural network (CNN) was trained using the real data to label abnormal radiographs. A CNN was trained to label abnormal radiographs using synthetic images from the GAN and in a separate experiment from SpineGAN. Using the real radiographs, an AUC of 0.856 was achieved in abnormality classification. Training on synthetic data generated by the standard GAN (AUC of 0.814) and synthetic data generated by our SpineGAN (AUC of 0.830) resulted in similar classifier performance. SpineGAN generated images with higher FID and lower precision scores, but with higher recall and increased performance when used for synthetic learning. The successful application of synthetic learning was demonstrated in the spine surgery domain for the classification of spine radiographs as abnormal or normal. A modified domain–relevant GAN is introduced for the generation of spine images, evidencing the importance of domain–relevant generation techniques in synthetic learning. Synthetic learning can allow neurosurgery to use larger and more diverse patient imaging sets to train more generalizable algorithms with greater patient privacy.

Crop yield estimation and prediction constitutes a key issue in agricultural management, particularly under the context of demographic pressure and climate change. Currently, the main challenge in estimating crop yields based on remotely sensed data and data-driven methods is how to cope with small datasets and the limited amount of annotated samples. In order to cope with small datasets and the limited amount of annotated samples and improve the accuracy of winter wheat yield estimation in the Guanzhong Plain, PR China, this study proposed a method of combining generative adversarial networks (GANs) and convolutional neural network (CNN) for comprehensive growth monitoring of winter wheat, in which the remotely sensed leaf area index (LAI), vegetation temperature condition index (VTCI) and meteorological data at four growth stages of winter wheat during 2012–2017 were generated as the inputs of multi-layer convolutional neural networks (CNNs), and GAN was employed to artificially increase the number of training samples. Then, a linear regression model between the simulated comprehensive growth monitoring (I) and the measured yields was established to estimate yields of winter wheat in the Guanzhong Plain pixel by pixel. The final results showed when GAN was used to double the size of the training samples, and the simulation values obtained by CNN based on augmented samples using GAN provided a better training (R2 = 0.95, RMSE = 0.05), validation (R2 = 0.54, RMSE = 0.16) and testing (R2 = 0.50, RMSE = 0.14) performance than that just using the original samples. The achieved best pixel-scale yield estimation accuracy of winter wheat (R2 = 0.50, RMSE = 591.46 kg/ha) in the Guanzhong Plain. These results showed that small samples can be enlarged by GAN, thus, more important features for reflecting the growth conditions and yields of winter wheat from the remotely sensed indices and meteorological indices can be extracted, and indicated that CNN accompanied with GAN could contribute a lot to the comprehensive growth monitoring and yield estimation of winter wheat and data augmentation methods are extremely useful for the application of small samples in deep learning.

The Generative Adversarial Network (GAN) is a state-of-the-art technique in the field of deep learning. A number of recent papers address the theory and applications of GANs in various fields of image processing. Fewer studies, however, have directly evaluated GAN outputs. Those that have been conducted focused on using classification performance, e.g., Inception Score (IS) and statistical metrics, e.g., Fr\'echet Inception Distance (FID). Here, we consider a fundamental way to evaluate GANs by directly analyzing the images they generate, instead of using them as inputs to other classifiers. We characterize the performance of a GAN as an image generator according to three aspects: 1) Creativity: non-duplication of the real images. 2) Inheritance: generated images should have the same style, which retains key features of the real images. 3) Diversity: generated images are different from each other. A GAN should not generate a few different images repeatedly. Based on the three aspects of ideal GANs, we have designed the Likeness Score (LS) to evaluate GAN performance, and have applied it to evaluate several typical GANs. We compared our proposed measure with two commonly used GAN evaluation methods: IS and FID, and four additional measures. Furthermore, we discuss how these evaluations could help us deepen our understanding of GANs and improve their performance.

Owing to their great success in unsupervised tasks, generative adversarial networks (GANs) are widely adopted for supervised (conditional) image-generation tasks such as in-painting. Unfortunately, designing an objective function using GANs is not trivial. For supervised tasks, the generator trained only with GAN loss does not yield the corresponding target for the given input because GAN is trained to match the data distribution, not to find the exact answer. Therefore, the loss function of a generator is often formulated by linearly combining supervised loss and GAN loss as similar to multi-task learning, expecting that each loss's weakness is complemented. Contrary to expectation, both losses cause a conflict in practice due to different optimum of each loss, yielding low objective and subjective image qualities. To address this problem, we empirically investigated the conflict caused by using a conventional GAN with pixel-wise losses; we then propose a novel (relativistic) accuracy-aware discriminator. Based on the proposed discriminator, we developed an accuracy-aware GAN (AAGAN) and proved its optimality under an ideal assumption. We then propose a relativistic accuracy-aware GAN (RAAGAN) by considering practical assumptions. Experimental results on supervised tasks demonstrated that the proposed schemes alleviated the competition between losses and outperformed conventional GANs in terms of both objective and subjective qualities.

Text sentiment analysis is one of the most important tasks in the field of public opinion monitoring, service evaluation and satisfaction analysis under network environments. Compared with the traditional Natural Language Processing analysis tools, convolution neural networks can automatically learn useful features from sentences and improve the performance of the affective analysis model. However, the original convolution neural network model ignores sentence structure information which is very important for text sentiment analysis. In this paper, we add piece-wise pooling to the convolution neural network, which allows the model to obtain the sentence structure. And the main features of different sentences are extracted to analyze the emotional tendencies of the text. At the same time, the user’s feedback involves many different fields, and there is less labeled data. In order to alleviate the sparsity of the data, this paper also uses the generative adversarial network to make common feature extractions, so that the model can obtain the common features associated with emotions in different fields, and improves the model’s Generalization ability with less training data. Experiments on different datasets demonstrate the effectiveness of this method.

1 - Congruence of deep learning in biomedical engineering: future prospects and challenges

Based on Global Cancer 2015 statistics, the lung cancer of all types constitutes 27% of overall cancers while 19.5% of cancer deaths are due to lung cancer. In lieu of this, an effective lung cancer screening test using Computed Tomography (CT) scan is crucial to detect cancer at the early stage. The interpretation of the CT images requires an advanced CAD system of high accuracy for instance, in classifying the lung nodules. Recently, Deep Learning method that is Convolution Neural Network (CNN) shows an outstanding success in lung nodules classification. However, the training of CNN requires a great number of images. Such a requirement is an issue in the case of medical images. Generative adversarial network (GAN) has been introduced to generate new image datasets for CNN training. Thus, the main objective of this study is to compare the performance of CNN architectures with and without the implementation of GAN for lung nodules classification in CT images. Here, the study used Conditional GAN (cGAN) to generate benign nodules images. The classification accuracy of the combined cGAN-CNN architecture was compared among CNN pretraining networks namely GoogleNet, ShuffleNet, DenseNet, and MobileNet based on classification accuracy, specificity, sensitivity, and AUC-ROC values. The experiment was tested on LIDC-IDRI database. The results showed cGAN-CNN architecture improves the overall classification accuracy as compared to CNN alone with the cGAN-ShuffleNet architecture performed the best, achieving 98.38% accuracy, 98.13% specificity, 100% sensitivity and AUC-ROC at 99.90%. Overall, the classification performance of CNN can be improved by integrating GAN architecture to mitigate the constraint of having a large medical image dataset, in this case, CT lung nodules images.

Background/Objectives: Accurate preoperative characterization of ovarian masses is essential for appropriate clinical management, particularly for borderline ovarian tumors (BOTs), which are less common and often difficult to distinguish from benign or malignant lesions on ultrasound. Although expert subjective ultrasound assessment achieves high diagnostic accuracy, limited availability of highly trained sonologists restricts its widespread application. Artificial intelligence-based approaches offer a potential solution; however, the low prevalence of BOTs restricts the development of robust deep learning models due to severe class imbalance. This study aimed to develop a Convolutional Neural Network (CNN)-based classifier enhanced with Generative Adversarial Networks (GANs) to improve the discrimination of ovarian masses as benign, malignant, or BOT using ultrasound images. Methods: A total of 3816 ultrasound images from 636 ovarian masses were retrospectively analyzed, including 390 benign lesions, 202 malignant tumors, and 44 BOTs. To address class imbalance, a Deep Convolutional GAN (DCGAN) was used to generate 2000 synthetic BOT images for data augmentation. A three-class ensemble CNN model integrating VGG16, ResNet50, and InceptionNetV3 architectures was developed. Performance was assessed on an independent test set and compared with a baseline model trained without DCGAN augmentation. Results: The incorporation of DCGAN-generated BOT images significantly enhanced classification performance. The BOT F1-score increased from 68.4% to 86.5%, while overall accuracy improved from 84.7% to 91.5%. For BOT identification, the final model achieved a sensitivity of 88.2% and specificity of 85.1%. Class-specific AUCs were 0.96 for benign lesions, 0.94 for malignant tumors, and 0.91 for BOTs. Conclusions: DCGAN-based augmentation effectively expands limited ultrasound datasets and improves CNN performance, particularly for BOT detection. This approach demonstrates potential as a decision support tool for preoperative assessment of ovarian masses.