3D-Mask-GAN:Unsupervised Single-View 3D Object Reconstruction

Generative adversarial networks (GANs) have proven useful for several medical imaging tasks, including image reconstruction and stochastic object model generation. Thus far, most of the work with GANs has been constrained to twodimensional images. Considering that medical imaging data are often inherently three-dimensional (3D), a 3D GAN would be a more principled way to synthesize realistic volumes. Training a 3D GAN is both computationally and memory intensive. However, prior work has not considered the anisotropic nature of many medical imaging systems. In this paper, the SlabGAN is proposed to reduce the inefficiencies associated with training a 3D GAN. The SlabGAN uses the progressive GAN architecture extended to 3D, but removes the requirement of the three dimensions being equal sizes. This permits the generation of anisotropic 3D volumes with large <i>x </i>and<i> y</i> dimensions. The SlabGAN is trained on MRI brain images from the fastMRI dataset to generate images of dimension 256&times;256&times;16. The <i>x</i> and <i>y</i> dimensions of these images are comparable to previously published results while requiring significantly fewer computational resources to generate. The trained SlabGAN is applicable to tasks such as 3D medical image reconstruction and thin-slice MR super resolution.

Recent advances in convolutional neural networks have shown promising results in 3D shape completion. But due to GPU memory limitations, these methods can only produce low-resolution outputs. To inpaint 3D models with semantic plausibility and contextual details, we introduce a hybrid framework that combines a 3D Encoder-Decoder Generative Adversarial Network (3D-ED-GAN) and a Longterm Recurrent Convolutional Network (LRCN). The 3DED- GAN is a 3D convolutional neural network trained with a generative adversarial paradigm to fill missing 3D data in low-resolution. LRCN adopts a recurrent neural network architecture to minimize GPU memory usage and incorporates an Encoder-Decoder pair into a Long Shortterm Memory Network. By handling the 3D model as a sequence of 2D slices, LRCN transforms a coarse 3D shape into a more complete and higher resolution volume. While 3D-ED-GAN captures global contextual structure of the 3D shape, LRCN localizes the fine-grained details. Experimental results on both real-world and synthetic data show reconstructions from corrupted models result in complete and high-resolution 3D objects.

As a new unsupervised learning algorithm framework, generative adversarial networks(GAN) have been favored by more and more researchers, and it has become a research hotspot now. GAN is inspired by the two-person zero-sum game theory in game theory. Its unique adversarial training idea can generate high-quality samples and has more powerful feature learning and feature expression capabilities than traditional machine learning algorithms. At present, GAN has achieved remarkable success in the field of computer vision, especially in the field of sample generation. Every year, a large number of GAN-related research papers are produced, reflecting the fiery degree of research on GAN model. Aiming at the hot model of GAN, first introduce the research status of GAN; then introduce the theory and framework of GAN, which analyzes in detail why the gradient disappears and the mode collapses during the training of GAN; then discussed some typical GAN improvement models, and summarized their theoretical improvements, advantages, limitations, application scenarios and implementation costs; Finally, the application results of GAN in data generation, image super-resolution, and image style conversion are shown, and the current challenges and future research directions of GAN are discussed.

Generative Adversarial Networks (GANs) have received a great deal of attention due in part to recent success in generating original, high-quality samples from visual domains. However, most current methods only allow for users to guide this image generation process through limited interactions. In this work we develop a novel GAN framework that allows humans to be “in-the-loop” of the image generation process. Our technique iteratively accepts relative constraints of the form “Generate an image more like image A than image B”. After each constraint is given, the user is presented with new outputs from the GAN, informing the next round of feedback. This feedback is used to constrain the output of the GAN with respect to an underlying semantic space that can be designed to model a variety of different notions of similarity (e.g. classes, attributes, object relationships, color, etc.). In our experiments, we show that our GAN framework is able to generate images that are of comparable quality to equivalent unsupervised GANs while satisfying a large number of the constraints provided by users, effectively changing a GAN into one that allows users interactive control over image generation without sacrificing image quality.

Computer-Aided-Diagnosis (CADx) systems assist radiologists with identifying and classifying potentially malignant pulmonary nodules on chest CT scans using morphology and texture-based (radiomic) features. However, radiomic features are sensitive to differences in acquisitions due to variations in dose levels and slice thickness. This study investigates the feasibility of generating a normalized scan from heterogeneous CT scans as input. We obtained projection data from 40 low-dose chest CT scans, simulating acquisitions at 10%, 25% and 50% dose and reconstructing the scans at 1.0mm and 2.0mm slice thickness. A 3D generative adversarial network (GAN) was used to simultaneously normalize reduced dose, thick slice (2.0mm) images to normal dose (100%), thinner slice (1.0mm) images. We evaluated the normalized image quality using peak signal-to-noise ratio (PSNR), structural similarity index (SSIM) and Learned Perceptual Image Patch Similarity (LPIPS). Our GAN improved perceptual similarity by 35%, compared to a baseline CNN method. Our analysis also shows that the GAN-based approach led to a significantly smaller error (p-value <; 0.05) in nine studied radiomic features. These results indicated that GANs could be used to normalize heterogeneous CT images and reduce the variability in radiomic feature values.

3D SOC-Net: Deep 3D reconstruction network based on self-organizing clustering mapping

Many robotic tasks require accurate shape models in order to properly grasp or interact with objects. However, it is often the case that sensors produce incomplete 3D models due to several factors such as occlusion or sensor noise. To address this problem, we propose a semi-supervised method that can recover the complete the shape of a broken or incomplete 3D object model. We formulated a hybrid of 3D variational autoencoder (VAE) and generative adversarial network (GAN) to recover the complete voxelized 3D object. Furthermore, we incorporated a separate classifier in the GAN framework, making it a three player game instead of two which helps stabilize the training of the GAN as well as guides the shape completion process to follow the object class labels. Our experiments show that our model produces 3D object reconstructions with high-similarity to the ground truth and outperforms several baselines in both quantitative and qualitative evaluations.

Recently, cyber-attacks to smart energy grid has become a critical subject for Energy System Operators (ESOs). To keep the energy grid cyber-secured, the attacker's behavior, resources and goals must be modeled properly. Then, the counter-measurement actions can be designed based on the attacker's model. In this paper, a new zero-sum game based on the Generative Adversarial Networks (GANs) is presented. The attacker to energy smart grid pursues two objects. The first goal is to be undetected by the system defender, and the second goal is to make profit via its False Data Injection (FDI) into the system. On the defender hand, the ESO needs to detect FDI using fast and reliable models. Thus, each party tries to defeat the other part. GANs are deep layer networks which consist of two rivals networks: the generator and the discriminator. The Generator Network (GN) aims to create the data similar to real data (plays the attacker role) and the Discriminator Network (DN) wants to properly detect whether the data is real or faked by the GN (plays the system defender role). In this paper, a new algorithm is presented to model both the attacker and ESO in the GAN frame work. Finally, A five-bus smart grid case is considered to show the effectiveness of the presented algorithm.

Hyperspectral remote sensing image classification based on residual generative Adversarial Neural Networks

Both JPEG restoration and anti-forensics remove the artifacts left by JPEG compression, and recover the JPEG compressed image. However, how to restore the high-frequency details of a JPEG compressed image for JPEG restoration and how to deceive the existing JPEG compression detectors without sacrificing visual quality in JPEG anti-forensics remain challenging. To address these issues, a framework of generative adversarial networks (GAN) with novel loss functions for JPEG restoration and anti-forensics (JRA-GAN) is proposed to allow a JPEG compressed image to be translated into a reconstructed one. Since JPEG compression causes impairment to high-frequency components, an alternating current (AC)-component loss function that measures the loss of AC components is proposed in JRA-GAN to recover these components. To prevent forensic detection, a calibration loss function is also introduced in JRA-GAN to mitigate the variance gap in the high-frequency subbands between a generated image and its calibrated version. Our experimental results demonstrate that the proposed JPEG restoration method outperforms existing methods in terms of image quality, and the JPEG anti-forensic scheme achieves better visual quality and anti-forensic performance that is comparable to the existing state-of-the-art anti-forensic methods. Our code is available in this page: https://github.com/wujianyuan/JRG-GAN .

Computer vision is one of the hottest research fields in deep learning. The emergence of generative adversarial networks (GANs) provides a new method and model for computer vision. The idea of GANs using the game training method is superior to traditional machine learning algorithms in terms of feature learning and image generation. GANs are widely used not only in image generation and style transfer but also in the text, voice, video processing, and other fields. However, there are still some problems with GANs, such as model collapse and uncontrollable training. This paper deeply reviews the theoretical basis of GANs and surveys some recently developed GAN models, in comparison with traditional GAN models. The applications of GANs in computer vision include data enhancement, domain transfer, high-quality sample generation, and image restoration. The latest research progress of GANs in artificial intelligence (AI) based security attack and defense is introduced. The future development of GANs in computer vision is also discussed at the end of the paper with possible applications of AI in computer vision.

A novel generative adversarial network for estimation of trip travel time distribution with trajectory data

&lt;p&gt;High data collection costs and complicated data access regulations increase the demand for synthetic data. Generative Adversarial Networks (GANs) are a novel generative framework with great potential for high quality synthetic data generation. GANs formulate the true distribution of data implicitly, and the success of GANs are often measured based on the similarity of generated data to this true distribution. GANs were originally designed to work with continuous data. However, many important real-world datasets such as medical images involve discontinuous distributions. GAN training for discontinuous distributions is relatively more challenging, as the training procedure often suffers from instability and mode collapse issues. This dissertation focuses on designing novel GAN architectures to generate representative synthetic image data, and proposes new structures to alleviate GANs' mode collapse issue. As part of this thesis, novel applications of image data generation with GANs have been also investigated for important problems arising in the telecommunication industry and medical domain. Specifically, we first explore various GAN structures to generate engineered electromagnetic surfaces. We consider the continuous approximation of the data and explore the capabilities of feed-forward and convolutional GANs for synthetic data generation. Next, we introduce a novel GAN architecture to address the problem of mode collapse in GAN training. The proposed structure incorporates a third network that penalizes the generator for generating low diversity samples. Lastly, we study the challenging problem of object generation in 3D space using GANs, and we propose extensions to existing 3D GAN structures to generate connected 3D volumes. Additionally, we explore a more challenging version of this 3D volume generation problem by generating connected volumes packed with spheres. This research has applications in radiosurgery treatment planning, and the proposed 3D GAN structure can help generate rare, unseen 3D tumor volumes and information on how to treat these tumors. Accordingly, our analysis contributes to overcoming data scarcity issues (e.g., due to privacy considerations) for an important practical problem in the medical domain.&lt;/p&gt;

MADRAS-NET: A deep learning approach for detecting and classifying android malware using Linknet

Generative Adversarial Network is the topic of interest in today’s research in the field of image processing and computer vision. A basic GAN model was introduced by Ian Goodfellow et al. in 2014. After that advancement in the field of research in GAN models has been application specific. In computer vision and image to image translation GANs are playing very effective role either in the case of face detection and recognition or in image resolution enhancement and image augmentation. This paper represents a concise overview of various GAN models along with their features and applications. Pix2Pix and conditional GAN models work upon paired datasets while other models like cycle GAN, discover GAN, dual GAN, info GAN, deep convolutional GAN etc. work upon unpaired datasets. Various image datasets which are commonly used for training of generator and discriminator networks are also discussed in this paper. Since partial mode collapse is a common problem to occur during training process for all models, therefore various normalization techniques are also preferred during the training of generator and discriminator networks. As the advancements in GAN models are increasing at a very fast rate, soon these models will also be preferred in commercial applications.