Hyperspectral Imagery Spatial Super-Resolution Using Generative Adversarial Network

Generative Adversarial Network is the topic of interest in today’s research in the field of image processing and computer vision. A basic GAN model was introduced by Ian Goodfellow et al. in 2014. After that advancement in the field of research in GAN models has been application specific. In computer vision and image to image translation GANs are playing very effective role either in the case of face detection and recognition or in image resolution enhancement and image augmentation. This paper represents a concise overview of various GAN models along with their features and applications. Pix2Pix and conditional GAN models work upon paired datasets while other models like cycle GAN, discover GAN, dual GAN, info GAN, deep convolutional GAN etc. work upon unpaired datasets. Various image datasets which are commonly used for training of generator and discriminator networks are also discussed in this paper. Since partial mode collapse is a common problem to occur during training process for all models, therefore various normalization techniques are also preferred during the training of generator and discriminator networks. As the advancements in GAN models are increasing at a very fast rate, soon these models will also be preferred in commercial applications.

In this paper, a generative adversarial network based on SMPL model is used to reconstruct three-dimensional (3D) human mesh from a single image. The generative adversarial network consists of two parts (generative network and adversarial network). The generative network is made up of a deep residual network and a multilayer perception network. The goal is to generate suitable SMPL model parameters and camera parameters which fit the 2D and 3D position annotation of human skeleton joints. The adversarial network judges whether the shape and pose parameters generated by the generative network match the real human body through a discriminator. The purpose is to guide the generative network output parameters that conform to the laws of human kinematics. A geometric prior of the 3D joint angle limitation of the human body is introduced into the network to further constrain the generated human pose. Experiments verify the effectiveness of the algorithm.

Segmentation of breast cancer tumor plays an important role in identifying the location of the tumor, to know the shape of tumor and hence the stage of breast cancer. This paper deals with the segmentation of tumor from whole mammographic mass images using Generative Adversarial Network (GAN). A mini dataset was considered with mammograms and their corresponding ground truth images. Pre-processing like image format conversion, enhancement, pectoral muscle removal and resizing was performed on raw mammogram images. GANs have two neural nets called generative and discriminative networks that compete against each other to obtain the segmentation output. PIX2PIX is a conditional GAN variant which has U-Net as the Generator network and a simple deep neural net as the discriminator. The input to the network was pair of pre-processed mass image and the associated ground truth. A binary image with highlighted tumor was obtained as output. The performance of GAN was evaluated by plotting Generator and discriminator loss. The segmented output was compared with corresponding ground truth. Metrics like Jaccard index, Jaccard distance and Dice-coefficient were calculated. A Dice-coefficient and Jaccard index of 90% and 88.38% was achieved. In future, higher accuracy could be achieved by involving larger dataset to make the system robust.

Many machine learning models require huge datasets to get trained, to make correct predictions, or to increase their accuracy, that's where generative adversarial network (GAN) comes into the picture. GAN is a deep learning model, which can be used to create artificial data. It consists of two neural networks that cooperate in a way that resembles a game: a generator network and a discriminator network. Within the suggested model in this work, a dense motion network-based first-order motion model for image animation is developed. Here a trained GAN extracts the face landmarks from the driving video and develops the embedding model to create the synthesis video using the dedicated module to prepare the Deepfakes, employing key point detectors as a baseline. Lastly, an approach that makes use of dense motion networks to increase the effectiveness of a collection of GAN generators is provided. With the help of the sequel driving combination of driving video with the source image, the given results produce the augmented animation video. Hence, it is tried to implement a lighter model which consists of modules, with the approximate same accuracy when compared with other implementations of GAN. This work has a wide range of applications, including doubling dataset counts with a small number of sources, creating real-time backgrounds and characters for the gaming and animation industries using CG platforms, translating clothes, predicting videos, creating 3D objects, etc.

In order to enhance the visual effects of the reconstructed images in the single image super-resolution, and solve the problem of instability of the training phase of super-resolution with generative adversary network (SRGAN), a new super-resolution model is built with more depth and width, and the corresponding super-resolution algorithm is proposed. The network structure of the original SRGAN is modified, so that deeper and wider convolutional networks can be used with high efficiency. Furthermore, the loss function is employed to balance the training of the generative network and the discriminator network, and solve the instability problem in the training phase. The training images for the network are obtained from the database of ImageNet on the web as other researchers. Experimental results show that images reconstructed by the proposed algorithm has better visual effects than those by the original SRGAN. In addition, the objective measures of PSNR and SSIM of the reconstructed images have also been improved.

Existence of haze significantly degrades visual quality and hence negatively affects the performance of visual surveillance, video analysis, and human–machine interaction. To remove haze from a visual signal, in this paper, we propose a generative adversarial network for visual haze removal called HRGAN. HRGAN consists of a generator network and a discriminator network. A unified network jointly estimating transmission maps, atmospheric light, and haze-free images (called UNTA) is proposed as the generator network of HRGAN. Instead of being optimized by minimizing the pixel-wise loss, HRGAN is optimized by minimizing a novel loss function consisting of pixel-wise loss, perceptual loss, and adversarial loss produced by a discriminator network. Classical model-based image dehazing algorithms consist of three separate stages: 1) estimating transmission map; 2) estimating atmospheric light; and 3) restoring haze-free image by using an atmospheric scattering model to process the transmission map and atmospheric light. Such a separate scheme is not guaranteed to achieve optimal results. On the contrary, UNTA performs transmission map estimation and atmospheric light estimation simultaneously to obtain joint optimal solutions. The experimental results on both synthetic and real-world image databases demonstrate that HRGAN outperforms the state-of-the-art algorithms in terms of both effectiveness and efficiency.

Image style transformation techniques are an important research area in the field of computer vision, aiming to combine the content of one image with the artistic style of another image to create new images with the content of the input image and the style of the artistic style image. The development of this field has benefited from the rapid development of deep learning algorithms, especially the application of techniques such as Convolutional Neural Networks (CNN) and Generative Adversarial Networks (GAN). At the heart of image style transformation is how to capture both the content and artistic style of the input image. This is typically achieved by using different levels of activation of the convolutional neural network as content and style representations. The content representation captures the object and structural information in the image, while the style representation captures the texture and colour information of the image. To capture the style of an image, a Gram matrix is typically used to measure the correlation between features at a particular level. This matrix representation allows us to capture texture information and colour distribution, thus enabling style migration. The goal of the image style transformation model is to minimise both content loss and style loss to produce synthetic images. Content loss typically uses Euclidean distance to measure the difference between content representations, while style loss involves the distance between Gram matrices. GANs have become an important tool in image style transformation. The generator network is responsible for generating synthetic images and the discriminator network evaluates the realism of the generated images. Through adversarial training, the generator and discriminator networks constantly compete to improve the quality of the generated images. Finally, this study highlights the potential applications of image style conversion techniques in the fields of art creation, image editing and virtual reality, and proposes directions for future research on deep learning algorithms and underlying principles to further improve the efficiency and quality of image style conversion techniques. The continuous development of this field will bring new opportunities for image processing and creative applications.

As a promising generative modeling method, Generative Adversarial Networks are a deep-learning-based generative model, in which two networks, namely the generative network and the discriminant network, contest with each other in a game during which the generator “learns” and get trained to be able to fool the discriminator of believing a specific image is (superficially) authentic. It is widely used to generate pictures based on existing data. Based on the MNIST data set, this paper studies and analyzes the application of Generative Adversarial Networks in the generation of handwritten digital images. In addition, by studying the Deep Convolutional Generative Adversarial Network and the Conditional Generation Adversarial Network model, we compared the connections and differences between these three models in training the MNIST dataset to generate handwritten digits algorithms, and gave an optimization method for generating handwritten digits.

Image dehazing has always been a challenging topic in image processing. The development of deep learning methods, especially the generative adversarial networks (GAN), provides a new way for image dehazing. In recent years, many deep learning methods based on GAN have been applied to image dehazing. However, GAN has two problems in image dehazing. Firstly, For haze image, haze not only reduces the quality of the image but also blurs the details of the image. For GAN network, it is difficult for the generator to restore the details of the whole image while removing the haze. Secondly, GAN model is defined as a minimax problem, which weakens the loss function. It is difficult to distinguish whether GAN is making progress in the training process. Therefore, we propose a guided generative adversarial dehazing network (GGADN). Different from other generation adversarial networks, GGADN adds a guided module on the generator. The guided module verifies the network of each layer of the generator. At the same time, the details of the map generated by each layer are strengthened. Network training is based on the pre-trained VGG feature model and L1-regularized gradient prior which is developed by new loss function parameters. From the dehazing results of synthetic images and real images, the proposed method is better than the state-of-the-art dehazing methods.

The hyperspectral image (HSI) classification applications become widespread in multiple applications. Therefore, the accurate classification of ground features using HSIs is an important research topic that has received a lot of interest. However, the conventional approaches failed to classify the pixels perfectly, which resulted in reduced accuracy. Thus, this work is focused on implementation of HSI classification using generative adversarial networks (HCGAN-Net). Initially, Gabor filtering is applied on HSIs for elimination of different types of noises, which also extracts the spatial features. Then, probabilistic principal component analysis (PCA) is applied to perform the dimensionality reduction operation, which also selects the best features using probability dependencies between spatial features. Then, spatial features are extracted for each spectral band and forms the hybrid spatial-spectral features. Finally, HCGAN-Net is used to performs the HSI classification operation using trained spatial-spectral features. The simulation results show that the proposed HCGAN-Net resulted in superior subjective and objective performance as compared to state of art HSI classifiers on various datasets.

Manual segmentation of brain lesions from Magnetic Resonance Images (MRI) is cumbersome and introduces errors due to inter-rater variability. This paper introduces a semi-supervised technique for detection of brain lesion from MRI using Generative Adversarial Networks (GANs). GANs comprises of a Generator network and a Discriminator network which are trained simultaneously with the objective of one bettering the other. The networks were trained using non lesion patches (n=13,000) from 4 different MR sequences. The network was trained on BraTS dataset and patches were extracted from regions excluding tumor region. The Generator network generates data by modeling the underlying probability distribution of the training data, (PData). The Discriminator learns the posterior probability P (Label Data) by classifying training data and generated data as “Real” or “Fake” respectively. The Generator upon learning the joint distribution, produces images/patches such that the performance of the Discriminator on them are random, i.e. P (Label Data = GeneratedData) = 0.5. During testing, the Discriminator assigns posterior probability values close to 0.5 for patches from non lesion regions, while patches centered on lesion arise from a different distribution (PLesion) and hence are assigned lower posterior probability value by the Discriminator. On the test set (n=14), the proposed technique achieves whole tumor dice score of 0.69, sensitivity of 91% and specificity of 59%. Additionally the generator network was capable of generating non lesion patches from various MR sequences.

Retinal vessel segmentation is of great significance for assisting doctors in diagnosis of ophthalmological diseases such as diabetic retinopathy, macular degeneration and glaucoma. This article proposes a new retinal vessel segmentation algorithm using generative adversarial learning with a large receptive field. A generative network maps an input retinal fundus image to a realistic vessel image while a discriminative network differentiates between images drawn from the database and the generative network. Firstly, the proposed generative network combines shallow features with the upsampled deep features to assemble a more precise vessel image. Secondly, the residual module in the proposed generative and discriminative networks can effectively help deep nets easy to optimize. Moreover, the dilated convolutions in the proposed generative network effectively enlarge the receptive field without increasing the amount of computations. A number of experiments are conducted on two publicly available datasets (DRIVE and STARE) achieving the segmentation accuracy rates of 95.63% and 96.84%, and the average areas under the ROC curve of 98.12% and 98.53%. Performance results show that the proposed automatic retinal vessel segmentation algorithm outperforms state‐of‐the‐art algorithms in many validation metrics. The proposed algorithm can not only detect small tiny blood vessels but also capture large‐scale high‐level semantic vessel features.

In recent years, the integration of convolutional neural networks (CNNs) and Transformers has significantly advanced hyperspectral image (HSI) classification by jointly capturing local and global features. However, most existing methods primarily focus on the fusion of spectral–spatial features while neglecting the complementary information contained in frequency domain features. To address this issue, we propose a spatial frequency cross-attention fusion network (SFCFNet) that jointly models spectral, spatial, and frequency-domain features for HSI classification. The framework consists of three core modules: first, the multiscale spectral spatial feature learning (MSSFL) module extracts joint spectral spatial features using multi-scale 3D and 2D convolutions. Next, the triple-branch representation module (TBRM) employs three branches to capture global spatial features of large-scale structures, local spatial features of fine grained textures, and multi-scale frequency features based on Haar wavelet decomposition, providing complementary multi domain representations for subsequent deep fusion. Finally, the dual-domain feature cross-attention fusion module (DFCF) achieves effective fusion of spatial structures and frequency domain textures, enhancing the model's ability to separate complex backgrounds from fine-grained targets and thereby improving classification performance. Compared with other methods, SFCFNet achieves higher overall accuracy (OA) on the Salinas, Houston2013, WHU-Hi-LongKou, and Xuzhou datasets, reaching 99.05%, 98.07%, 98.76%, and 98.18%, respectively.

The science of deep learning has undergone a revolution thanks to the invention of generative adversarial networks (GANs), a kind of deep learning design that can produce synthetic data that is equivalent to real-world data. GANs have recently been employed for a number of images processing operations, including photo colorization. The storing of grayscale photos, which require less storage space and can then be colored, and the restoration of old photographs are two of the key uses for image colorization. Although there are numerous color choices that can be employed with a specific grayscale image, this problem is challenging. Deep learning has been used in recent advances to make an effort to overcome this issue. In addition to the generator network, the Pix2Pix architecture also has a discriminator network that tries to tell bogus from real color images. Many datasets, including the CIFAR-10 and COCO datasets, have shown that the suggested method produces high-quality colorized images that closely resemble the ground truth images. Overall, this study shows how well GANs–and specifically the Pix2Pix architecture–can be used to colorize images. According to the research, this technology might be applied to a number of activities, including the processing of images and videos, digital restoration, and aesthetic representation.

Generative adversarial networks (GANs) with gaming abilities have been widely applied in image generation. However, gamistic generators and discriminators may reduce the robustness of the obtained GANs in image generation under varying scenes. Enhancing the relation of hierarchical information in a generation network and enlarging differences of different network architectures can facilitate more structural information to improve the generation effect for image generation. In this paper, we propose an enhanced GAN via improving a generator for image generation (EIGGAN). EIGGAN applies a spatial attention to a generator to extract salient information to enhance the truthfulness of the generated images. Taking into relation the context account, parallel residual operations are fused into a generation network to extract more structural information from the different layers. Finally, a mixed loss function in a GAN is exploited to make a tradeoff between speed and accuracy to generate more realistic images. Experimental results show that the proposed method is superior to popular methods, i.e., Wasserstein GAN with gradient penalty (WGAN-GP) in terms of many indexes, i.e., Frechet Inception Distance, Learned Perceptual Image Patch Similarity, Multi-Scale Structural Similarity Index Measure, Kernel Inception Distance, Number of Statistically-Different Bins, Inception Score and some visual images for image generation.