Inverse Design of Digital Materials Using Corrected Generative Deep Neural Network and Generative Deep Convolutional Neural Network

Large amount of annotated images with rich variations are needed to train a deep network for detecting instance object in unstructured environment. Addressing the problem that the artificial acquisition and manual annotation is time-consuming, the generative deep deconvolutional neural network (GDDNE) to increase and diversify training data through the supervised learning strategy is created in this paper. Specifically, our network can not only generate with different styles such as shift, zoom, brightness and other superimposed transformations, but also interpolate generate the new ones between given viewpoints images in training samples. With 180 viewpoints realistic images in training samples: 30 rotation angles in plane and 6 angles of depression, our network can finally generated 1000 diversified viewpoint images and 21 kinds of data transformations for each instance object. Abundant experiments demonstrate that the remarkable performance of our generative network used in the generation task of large magnitude.

Deep generative neural networks have been used increasingly in computational chemistry for de novo design of molecules with desired properties. Many deep learning approaches employ reinforcement learning for optimizing the target properties of the generated molecules. However, the success of this approach is often hampered by the problem of sparse rewards as the majority of the generated molecules are expectedly predicted as inactives. We propose several technical innovations to address this problem and improve the balance between exploration and exploitation modes in reinforcement learning. In a proof-of-concept study, we demonstrate the application of the deep generative recurrent neural network enhanced by several novel technical tricks to designing experimentally validated potent inhibitors of the epidermal growth factor (EGFR). The proposed technical solutions are expected to substantially improve the success rate of finding novel bioactive compounds for specific biological targets using generative and reinforcement learning approaches.

Synthesis of a six-bar mechanism for generating knee and ankle motion trajectories using deep generative neural network

Breast dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is a sensitive imaging technique critical for breast cancer diagnosis. However, the administration of contrast agents poses a potential risk. This can be avoided if contrast-enhanced MRI can be obtained without using contrast agents. Thus, we aimed to generate T1-weighted contrast-enhanced MRI (ceT1) images from pre-contrast T1 weighted MRI (preT1) images in the breast. We proposed a generative adversarial network to synthesize ceT1 from preT1 breast images that adopted a local discriminator and segmentation task network to focus specifically on the tumor region in addition to the whole breast. The segmentation network performed a related task of segmentation of the tumor region, which allowed important tumor-related information to be enhanced. In addition, edge maps were included to provide explicit shape and structural information. Our approach was evaluated and compared with other methods in the local (n = 306) and external validation (n = 140) cohorts. Four evaluation metrics of normalized mean squared error (NRMSE), Pearson cross-correlation coefficients (CC), peak signal-to-noise ratio (PSNR), and structural similarity index map (SSIM) for the whole breast and tumor region were measured. An ablation study was performed to evaluate the incremental benefits of various components in our approach. Our approach performed the best with an NRMSE 25.65, PSNR 54.80 dB, SSIM 0.91, and CC 0.88 on average, in the local test set. Performance gains were replicated in the validation cohort. We hope that our method will help patients avoid potentially harmful contrast agents. Clinical and Translational Impact Statement-Contrast agents are necessary to obtain DCE-MRI which is essential in breast cancer diagnosis. However, administration of contrast agents may cause side effects such as nephrogenic systemic fibrosis and risk of toxic residue deposits. Our approach can generate DCE-MRI without contrast agents using a generative deep neural network. Thus, our approach could help patients avoid potentially harmful contrast agents resulting in an improved diagnosis and treatment workflow for breast cancer.

Utterance Style Transfer Using Deep Models

Deep learning based process monitoring methods are attracting increasing research attention in recent years, which generally assume that the process variables are uniformly sampled. In practice, however, the process data are generally collected at multiple different rates, resulting in structurally-incomplete training data. Under such circumstances, how to build effective deep models to fully mine the multirate sampled data has become a constraint in achieving better process monitoring performance. In this paper, a sequentially-adaptive deep variational model is designed in which the knowledge that existed in variables with different rates is comprehensively extracted through deep generative neural networks. The multirate samples are first divided into multiple data blocks in which each block is collected at a uniform rate. A deep generative model is then constructed to model the uncertain data distribution and extract probabilistic feature representations considering the slowness principle. To restrain the small data problem in the blocks with slow rates, a sequentially-adaptation strategy is designed to adapt the knowledge from the fast blocks with sufficient training data and enhance the overall modeling performance. The effectiveness is demonstrated through a real-world industrial thermal power plant case.

The regulatory role of the serotonergic system on conscious perception can be investigated perturbatorily with psychedelic drugs such as N,N-Dimethyltryptamine. There is increasing evidence that the serotonergic system gates prior (endogenous) and sensory (exogenous) information in the construction of a conscious experience. Using two generative deep neural networks as examples, we discuss how such models have the potential to be, firstly, an important medium to illustrate phenomenological visual effects of psychedelics—besides paintings, verbal reports and psychometric testing—and, secondly, their utility to conceptualize biological mechanisms of gating the influence of exogenous and endogenous information on visual perception.

Deep attention and generative neural networks for nonintrusive load monitoring

A deep learning aided optimization algorithm for the design of flat thin-film multilayer optical systems is developed. The authors introduce a deep generative neural network, based on a variational autoencoder, to perform the optimization of photonic devices. This algorithm allows one to find a near-optimal solution to the inverse design problem of creating an anti-reflective grating, a fundamental problem in material science. As a proof of concept, the authors demonstrate the method's capabilities for designing an anti-reflective flat thin-film stack consisting of multiple material types. We designed and constructed a dielectric stack on silicon that exhibits an average reflection of 1.52 %, which is lower than other recently published experiments in the engineering and physics literature. In addition to its superior performance, the computational cost of our algorithm based on the deep generative model is much lower than traditional nonlinear optimization algorithms. These results demonstrate that advanced concepts in deep learning can drive the capabilities of inverse design algorithms for photonics. In addition, the authors develop an accurate regression model using deep active learning to predict the total reflectivity for a given optical system. The surrogate model of the governing partial differential equations can then be broadly used in the design of optical systems and to rapidly evaluate their behavior.

Three-dimensional (3D) imaging with structured light is crucial in diverse scenarios, ranging from intelligent manufacturing and medicine to entertainment. However, current structured light methods rely on projector–camera synchronization, limiting the use of affordable imaging devices and their consumer applications. In this work, we introduce an asynchronous structured light imaging approach based on generative deep neural networks to relax the synchronization constraint, accomplishing the challenges of fringe pattern aliasing, without relying on any a priori constraint of the projection system. To overcome this need, we propose a generative deep neural network with U-Net-like encoder–decoder architecture to learn the underlying fringe features directly by exploring the intrinsic prior principles in the fringe pattern aliasing. We train within an adversarial learning framework and supervise the network training via a statistics-informed loss function. We demonstrate that by evaluating the performance on fields of intensity, phase, and 3D reconstruction. It is shown that the trained network can separate aliased fringe patterns for producing comparable results with the synchronous one: the absolute error is no greater than 8 μm, and the standard deviation does not exceed 3 μm. Evaluation results on multiple objects and pattern types show it could be generalized for any asynchronous structured light scene.

Conversational agents are getting more popular and applied in a wide range of practical application areas. The main task of these agents is not only to generate context-appropriate responses to a given query but also to make the conversation human-like. Thanks to the ability of deep learning based models in natural language modeling, recent studies have made progress in designing conversational agents that can provide more semantically accurate responses. However, the naturalness in such conversation setting has not been given adequate attention in these studies. This paper aims to incorporate both important criteria of accuracy and naturalness of conversation in developing a new model for conversational agents. To this end, inspired by the idea of Turing test and the idea of adversarial learning strategy, we propose to design a model based on generative deep neural networks that interestingly allow to generate accurate responses optimized by the mechanics of imitating human-generated conversations. Experimental results demonstrate that the proposed models produce more natural and accurate responses, yielding significant gains in BLEU scores.

This thesis tackles fine-grained image recognition, the task of sub-category or species classification. It explores general methods to improve fine-grained image classification including the use of generative models and deep convolutional neural networks leading to novel models such as a Mixture of deep convolution neural networks. This work led to 9 peer reviewed publications and a Best Paper Award.

The mid-infrared spectra of galaxies are rich in features such as the Polycyclic Aromatic Hydrocarbon (PAH) and silicate dust features which give valuable information about the physics of galaxies and their evolution. For example they can provide information about the relative contribution of star formation and accretion from a supermassive black hole to the power output of galaxies. However, the mid-infrared spectra are currently available for a very small fraction of galaxies that have been detected in deep multi-wavelength surveys of the sky. In this paper we explore whether Deep Generative Network methods can be used to reconstruct mid-infrared spectra in the 5–35μm range using the limited multi-wavelength photometry in ∼20 bands from the ultraviolet to the submillimeter which is typically available in extragalactic surveys. For this purpose we use simulated spectra computed with a combination of radiative transfer models for starbursts, active galactic nucleus (AGN) tori and host galaxies. We find that our method using Deep Generative Networks, namely Generative Adversarial Networks and Generative Latent Optimization models, can efficiently produce high quality reconstructions of mid-infrared spectra in ∼ 60% of the cases. We discuss how our method can be improved by using more training data, photometric bands, model parameters or by employing other generative networks.

The use of is widespread in the biological domain, and has also been used extensively by armed forces around the world in order to make visual detection and classification of objects of military interest more difficult. The recent advent of ever more autonomous military agents raises the questions of whether can have a similar effect on autonomous agents as it has on human agents, and if so, what kind of will be effective against such adversaries. In previous works, we have shown that image classifiers based on deep neural networks can be confused by patterns generated by generative networks (GANs). Specifically, we trained a classifier to distinguish between two ship types, military and civilian. We then used a GAN to generate patterns that, when overlaid on parts of military vessels (frigates), made the classifier confuse the modified frigates with civilian vessels. We termed such patterns adversarial camouflage (AC) since these patterns effectively the frigates with respect to the classifier. The type of attack described in our previous work is a so-called white box attack. This term describes attacks that are devised given full knowledge of the classifier under attack. This is as opposed to black box attacks, which describe attacks on unknown classifiers. In our context, the ultimate goal is to design a GAN that is capable of black box attacks, in other words: a GAN that will generate AC that has effect across a wide range of neural network classifiers. In the current work, we study techniques to improve the robustness of our GAN-based approach by investigating whether a GAN can be trained to fool a selection of several neural network-based classifiers, or reduce the confidence of the classifications to a degree which makes them unreliable. Our results indicate that it is indeed possible to weaken a wider range of neural network classifiers by training the generator on several classifiers.

With the rapid development of neural networks, many deep learning-based image processing tasks have shown outstanding performance. In this paper, we describe a unified deep learning-based approach for image image denoising. The proposed method is composed of deep convolutional neural and conditional generative adversarial networks. For the discriminator network, we present a new network architecture with bi-skip connections to address hard training and details losing issues. In the generative network, a objective optimization is derived to solve the problem of common conditions being non-identical. Through extensive experiments on image denoising task on both qualitative and quantitative criteria, we demonstrate that our proposed method performs favorably against current state-of-the-art approaches.