Speech Super Resolution Generative Adversarial Network

Super-resolution reconstruction of pneumocystis carinii pneumonia images based on generative confrontation network

In this paper, a generative adversarial network based on SMPL model is used to reconstruct three-dimensional (3D) human mesh from a single image. The generative adversarial network consists of two parts (generative network and adversarial network). The generative network is made up of a deep residual network and a multilayer perception network. The goal is to generate suitable SMPL model parameters and camera parameters which fit the 2D and 3D position annotation of human skeleton joints. The adversarial network judges whether the shape and pose parameters generated by the generative network match the real human body through a discriminator. The purpose is to guide the generative network output parameters that conform to the laws of human kinematics. A geometric prior of the 3D joint angle limitation of the human body is introduced into the network to further constrain the generated human pose. Experiments verify the effectiveness of the algorithm.

Generative adversarial network (GAN) has been widely applied to produce super-resolution (SR) image with real perception and texture details. In most existing SR approaches, the training objective typically measures a pixel-wise average distance between the SR and high-resolution (HR) images. However, as the degradation function of different images from HR to low resolution (LR) is generally different, optimizing such metrics often leads to certain unpleasant artificial traces. Unlike the prevalent GAN inversion methods that require expensive image-specific optimization at runtime, we present an alternative formulation by directly leveraging latent representation produced by a pretrained AutoEncoder. We call this improved method reduce dimension super-resolved GAN (RD-SRGAN). RD-SRGAN first obtains the latent feature representation of LR image by a pretrained AutoEncoder as input to the generator network. This process not only reduces noise effects but also decreases the overall computational complexity. On the other hand, the residual between the ground truth and the produced images replaces the produced images as input to the discriminator network, and a 2D zero mean Gaussian noise with controllable low variance replaces the real images as another input to the discriminator network. By leveraging the feature representation and properties of the 2D zero mean Gaussian noise, we restrict the optimization space to produce an SR image. Therefore, the residual of the generated SR images tend to approximates to a Gaussian noise, which introduces useless deviation information as little as possible. Experimental results show that RD-SRGAN can benefit from these strategies and achieve improved fidelity and naturalness comparison to existing methods. Switching the pretrained AutoEncoder allows the method to deal with images from diverse categories, e.g., remote sensing satellite imaging, medical imaging, and astronomy.

The detection performance of tree crowns in forest environment has not been satisfactory compared to common objects, especially using aerial RGB imagery. Previous methods regarding Individual Tree Crown Detection (ITCD) utilizes different data sources to improve the detection rate due to the noisy image. Image enhancement methods such as super-resolution provide a solution to the noisy image by reconstructing the image using the low-resolution image. Generative Adversarial Network (GAN)-based model has shown success in super-resolution techniques. However, the GAN-based model created artefacts that may hinder the accuracy of the detection. In this paper, a noise-cancelling GAN-based model is proposed by averaging the weights of a compressed image and non-compressed image. The proposed method forces the network to discriminate the noise to generate a more photorealistic image. This method is inspired by super-resolution GAN (SRGAN) architecture with Residual Dense Network as the generator network. A two-stage object detection RetinaNet model is then used to detect the individual tree crowns in a sequential fashion. Extensive experiments have been conducted on a self-assembled tree crown dataset which showed the proposed model is more superior than a non-enhanced model with 0.6017 and 0.5908 respectively. Based on the results of the proposed method, the super-resolution technique can be used in conjunction with object detection algorithm to improve the detection in ITCD to improve the detection rate.

Recovering images from highly undersampled measurements has a wide range of applications across imaging sciences. State-of-the-art analytics however are not aware of the image perceptual quality, and demand iterative algorithms that incur significant computational overhead. To sidestep these hurdles, this paper brings forth a novel compressive imaging framework using deep neural networks that approximates a low-dimensional manifold of images using generative adversarial networks. To ensure the images are consistent with the measurements a recurrent GAN (RGAN) architecture is deployed that consists of multiple alternative blocks of generator networks and affine projection, which is then followed by a discriminator network to score the perceptual quality of the generated images. A deep residual network with skip connections is used for the generator, while the discriminator is a multilayer perceptron. Experiments performed with real-world contrast enhanced MRI data corroborate the diagnostic quality of the retrieved images relative to state-of-the-art CS schemes. In addition, it achieves about two-orders of magnitude faster reconstruction.

The terahertz (THz) metamaterial sensor design is typically complex and requires substantial expertise in physics. To simplify this process, we propose a novel reverse design model based on an improved conditional generative adversarial network that integrates self-attention generative adversarial network and Wasserstein generative adversarial network (WGAN) networks, and is referred to as the self-attention conditional Wasserstein GAN (SACW-GAN) model. By using the target response of the sensor as the input to the generator network, and incorporating labeling information, an attention mechanism, and the Wasserstein distance, we achieve effective reverse design of THz metamaterial sensors. The simulation results demonstrate the model's high performance, with spectral and image accuracies of 95% and 97%, respectively. This deep learning approach offers new perspectives and methodologies for the reverse design and application of THz metamaterial sensors, significantly advancing the field.

This paper introduces a novel automotive radar interference mitigation approach using a generative adversarial network (GAN). Instead of tackling the mutual interference in the time domain, a generative adversarial network is trained and used to recover the complex signal in the frequency domain, namely on the complex range profile obtained after the fast Fourier transform of fast-time samples (RFFT spectrum). It is shown that by employing the gated convolution and an attention mechanism, the generator network has the ability to learn the amplitude and phase information for missing data from the remaining signal. Experimental results show that the proposed method can provide a remarkable improvement in signal-to-interference-plus-noise ratio (SINR) and preserves its robustness in severely disturbed scenarios that are much more complex than the training dataset.

Generative adversarial network (GAN) has been applied for low-dose CT images to predict normal-dose CT images. However, the undesired artifacts and details bring uncertainty to the clinical diagnosis. In order to improve the visual quality while suppressing the noise, in this paper, we mainly studied the two key components of deep learning based low-dose CT (LDCT) restoration models—network architecture and adversarial loss, and proposed a disentangled noise suppression method based on GAN (DNSGAN) for LDCT. Specifically, a generator network, which contains the noise suppression and structure recovery modules, is proposed. Furthermore, a multi-scaled relativistic adversarial loss is introduced to preserve the finer structures of generated images. Experiments on simulated and real LDCT datasets show that the proposed method can effectively remove noise while recovering finer details and provide better visual perception than other state-of-the-art methods.

Generative adversarial networks (GANs) with gaming abilities have been widely applied in image generation. However, gamistic generators and discriminators may reduce the robustness of the obtained GANs in image generation under varying scenes. Enhancing the relation of hierarchical information in a generation network and enlarging differences of different network architectures can facilitate more structural information to improve the generation effect for image generation. In this paper, we propose an enhanced GAN via improving a generator for image generation (EIGGAN). EIGGAN applies a spatial attention to a generator to extract salient information to enhance the truthfulness of the generated images. Taking into relation the context account, parallel residual operations are fused into a generation network to extract more structural information from the different layers. Finally, a mixed loss function in a GAN is exploited to make a tradeoff between speed and accuracy to generate more realistic images. Experimental results show that the proposed method is superior to popular methods, i.e., Wasserstein GAN with gradient penalty (WGAN-GP) in terms of many indexes, i.e., Frechet Inception Distance, Learned Perceptual Image Patch Similarity, Multi-Scale Structural Similarity Index Measure, Kernel Inception Distance, Number of Statistically-Different Bins, Inception Score and some visual images for image generation.

In recent years, deep neural networks have demonstrated high efficiency in solving the problems associated with image processing and analysis. The latest studies in generative adversarial neural networks open up broad prospects for solving problems in image style transfer, cross-domain adaptation, and the generation of new data for a target probability distribution. The ability of generative networks to implement complex and weakly formalized transformations allows the information systems based on generative adversarial technologies to solve problems in machine learning in a manner similar to human creative thinking. This paper presents the results of an experimental study on the design, training and deployment of an intelligent control system for a CNC laser engraver based on the generative adversarial network. In order to train deep models, large amounts of data are required. Therefore, one of the main challenges in this experimental study is a lack of labelled training dataset. The proposed control system for the laser engraver based on the deep generative model allows decreasing the time of the manufacturing process.

Optimizing the discriminator in Generative Adversarial Networks (GANs) to completion in the inner training loop is computationally prohibitive, and on finite datasets would result in overfitting. To address this, a common update strategy is to alternate between k optimization steps for the discriminator D and one optimization step for the generator G. This strategy is repeated in various GAN algorithms where k is selected empirically. In this paper, we show that this update strategy is not optimal in terms of accuracy and convergence speed, and propose a new update strategy for networks with Wasserstein GAN (WGAN) group related loss functions (e.g. WGAN, WGAN-GP, Deblur GAN, and Super resolution GAN). The proposed update strategy is based on a loss change ratio comparison of G and D. We demonstrate that the proposed strategy improves both convergence speed and accuracy.

Segmentation of breast cancer tumor plays an important role in identifying the location of the tumor, to know the shape of tumor and hence the stage of breast cancer. This paper deals with the segmentation of tumor from whole mammographic mass images using Generative Adversarial Network (GAN). A mini dataset was considered with mammograms and their corresponding ground truth images. Pre-processing like image format conversion, enhancement, pectoral muscle removal and resizing was performed on raw mammogram images. GANs have two neural nets called generative and discriminative networks that compete against each other to obtain the segmentation output. PIX2PIX is a conditional GAN variant which has U-Net as the Generator network and a simple deep neural net as the discriminator. The input to the network was pair of pre-processed mass image and the associated ground truth. A binary image with highlighted tumor was obtained as output. The performance of GAN was evaluated by plotting Generator and discriminator loss. The segmented output was compared with corresponding ground truth. Metrics like Jaccard index, Jaccard distance and Dice-coefficient were calculated. A Dice-coefficient and Jaccard index of 90% and 88.38% was achieved. In future, higher accuracy could be achieved by involving larger dataset to make the system robust.

Many machine learning models require huge datasets to get trained, to make correct predictions, or to increase their accuracy, that's where generative adversarial network (GAN) comes into the picture. GAN is a deep learning model, which can be used to create artificial data. It consists of two neural networks that cooperate in a way that resembles a game: a generator network and a discriminator network. Within the suggested model in this work, a dense motion network-based first-order motion model for image animation is developed. Here a trained GAN extracts the face landmarks from the driving video and develops the embedding model to create the synthesis video using the dedicated module to prepare the Deepfakes, employing key point detectors as a baseline. Lastly, an approach that makes use of dense motion networks to increase the effectiveness of a collection of GAN generators is provided. With the help of the sequel driving combination of driving video with the source image, the given results produce the augmented animation video. Hence, it is tried to implement a lighter model which consists of modules, with the approximate same accuracy when compared with other implementations of GAN. This work has a wide range of applications, including doubling dataset counts with a small number of sources, creating real-time backgrounds and characters for the gaming and animation industries using CG platforms, translating clothes, predicting videos, creating 3D objects, etc.

Generating High-Resolution Climate Prediction through Generative Adversarial Network

Manual segmentation of brain lesions from Magnetic Resonance Images (MRI) is cumbersome and introduces errors due to inter-rater variability. This paper introduces a semi-supervised technique for detection of brain lesion from MRI using Generative Adversarial Networks (GANs). GANs comprises of a Generator network and a Discriminator network which are trained simultaneously with the objective of one bettering the other. The networks were trained using non lesion patches (n=13,000) from 4 different MR sequences. The network was trained on BraTS dataset and patches were extracted from regions excluding tumor region. The Generator network generates data by modeling the underlying probability distribution of the training data, (PData). The Discriminator learns the posterior probability P (Label Data) by classifying training data and generated data as “Real” or “Fake” respectively. The Generator upon learning the joint distribution, produces images/patches such that the performance of the Discriminator on them are random, i.e. P (Label Data = GeneratedData) = 0.5. During testing, the Discriminator assigns posterior probability values close to 0.5 for patches from non lesion regions, while patches centered on lesion arise from a different distribution (PLesion) and hence are assigned lower posterior probability value by the Discriminator. On the test set (n=14), the proposed technique achieves whole tumor dice score of 0.69, sensitivity of 91% and specificity of 59%. Additionally the generator network was capable of generating non lesion patches from various MR sequences.