Image Denoising Using Deep CGAN With Bi-Skip Connections

Low-level image processing is mainly concerned with extracting descriptions (that are usually represented as images themselves) from images. With the rapid development of neural networks, many deep learning-based low-level image processing tasks have shown outstanding performance. In this paper, we describe a unified deep learning based approach for low-level image processing, in particular, image denoising, image deblurring, and compressed image restoration. The proposed method is composed of deep convolutional neural and conditional generative adversarial networks. For the discriminator network, we present a new network architecture with bi-skip connections to address hard training and details losing issues. In the generative network, a multi-objective optimization is derived to solve the problem of common conditions being non-identical. Through extensive experiments on three low-level image processing tasks on both qualitative and quantitative criteria, we demonstrate that our proposed method performs favorably against all current state-of-the-art approaches.

Due to the real working conditions, the collected mechanical fault datasets are actually limited and always highly imbalanced, which restricts the diagnosis accuracy and stability. To solve these problems, we present an imbalanced fault diagnosis method based on the generative model of conditional-deep convolutional generative adversarial network (C-DCGAN) and provide a study in detail. Deep convolutional generative adversarial network (DCGAN), based on traditional generative adversarial networks (GAN), introduces convolutional neural network into the training for unsupervised learning to improve the effect of generative networks. Conditional generative adversarial network (CGAN) is a conditional model obtained through introducing conditional extension into GAN. C-DCGAN is a combination of DCGAN and CGAN. In C-DCGAN, based on the feature extraction ability of convolutional networks, through the structural optimization, conditional auxiliary generative samples are used as augmented data and applied in machine fault diagnosis. Two datasets (Bearing dataset and Planetary gear box dataset) are carried out to validate. The simulation experiments showed that the improved performance is mainly due to the generated signals from C-DCGAN to balance the dataset. The proposed method can deal with imbalanced fault classification problem much more effectively. This model could improve the accuracy of fault diagnosis and the generalization ability of the classifier in the case of small samples and display better fault diagnosis performance.

Electric vehicle (EV) charging behaviors are important for load forecasting, but charging behavior data collection is time-consuming and labor-intensive. To solve this problem, an improved generative adversarial network (GAN) based charging behavior data generation method for EVs was proposed. In this method, convolutional neural network is used as the structure, Wasserstein distance is used as the loss function of the neural network, and conditional labels are added to make the generator learn the probability distribution mapping relationship that meets the corresponding conditions. These methods finally construct the EV charging behavior data generation network. In the end, this paper compares this method with traditional data generation algorithm synthetic minority oversampling technique (SMOTE) and conditional generative adversarial network (CGAN). The result shows that this method can effectively learn the distribution rule of small sample data and effectively expand the data scale.

In this paper, a deep learning (DL) framework was proposed to predict the taxi-passenger demand while the spatial, the temporal, and external dependencies were considered simultaneously. The proposed DL framework combined a modified density-based spatial clustering algorithm with noise (DBSCAN) and a conditional generative adversarial network (CGAN) model. More specifically, the modified DBSCAN model was applied to produce a number of sub-networks considering the spatial correlation of taxi pick-up events in the road network. And the CGAN model, fed with the historical taxi passenger demand and other conditional information, was capable to predict the taxi-passenger demands. The proposed CGAN model was made up with two long short-term memory (LSTM) neural networks, which are termed as the generative network ${G}$ and the discriminative network ${D}$ , respectively. Adversarial training process was conducted to the two LSTMs. In the numerical experiment, different model layouts were compared. It was found that different network layouts provided reasonable accuracy. With limited training data, more LSTM layers in the generator network resulted in not only higher accuracy, but also more difficulties in training. Comparisons were also conducted between the proposed prediction model and four typical approaches, including the moving average method, the autoregressive integrated moving method, the neural network model, and the LSTM neural network model. The comparison results showed that the proposed model outperformed all the other methods. And the repeated experiment indicated that the proposed CGAN model provided significant better predictions than the LSTM model did. Future research was recommended to include more datasets for testing the model and more information for improving predictive performance.

Generative networks are effective tools for digital materials (DM) inverse design. However, the optimization performance of generative networks is restricted by the increasing discrepancy between the optimized input and the prescribed input domain as the design loop increases. Herein, a correction technique is incorporated into generative deep neural network (GDNN) and generative deep convolutional neural network (GDCNN). The correction is performed by pulling the machine learning (ML)‐optimized inputs back to the prescribed domain at certain interval during the optimization process instead of only postprocessing at the end. A DM system with two phases, i.e., the matrix phase and the pore phase, is used for the structural design. The datasets are produced using a numerical model and describe the relationship between the material structure and the elastic modulus and tensile strength. The results show that the optimization effectiveness of corrected GDNN/GDCNN significantly improves given the fact that more structures converge to the best structures and fewer nonrepetitive structures are left after optimization, which helps to search for the best structures and decreases the computational burden when verifying the ML‐recommended structures. The corrected GDNN and GDCNN also manage to find structures with higher tensile strength in the new inverse design.

Image style transformation techniques are an important research area in the field of computer vision, aiming to combine the content of one image with the artistic style of another image to create new images with the content of the input image and the style of the artistic style image. The development of this field has benefited from the rapid development of deep learning algorithms, especially the application of techniques such as Convolutional Neural Networks (CNN) and Generative Adversarial Networks (GAN). At the heart of image style transformation is how to capture both the content and artistic style of the input image. This is typically achieved by using different levels of activation of the convolutional neural network as content and style representations. The content representation captures the object and structural information in the image, while the style representation captures the texture and colour information of the image. To capture the style of an image, a Gram matrix is typically used to measure the correlation between features at a particular level. This matrix representation allows us to capture texture information and colour distribution, thus enabling style migration. The goal of the image style transformation model is to minimise both content loss and style loss to produce synthetic images. Content loss typically uses Euclidean distance to measure the difference between content representations, while style loss involves the distance between Gram matrices. GANs have become an important tool in image style transformation. The generator network is responsible for generating synthetic images and the discriminator network evaluates the realism of the generated images. Through adversarial training, the generator and discriminator networks constantly compete to improve the quality of the generated images. Finally, this study highlights the potential applications of image style conversion techniques in the fields of art creation, image editing and virtual reality, and proposes directions for future research on deep learning algorithms and underlying principles to further improve the efficiency and quality of image style conversion techniques. The continuous development of this field will bring new opportunities for image processing and creative applications.

Self-mixing interferometry (SMI) is superior to other laser interferometry methods due to its simplicity and compactness. However, SMI signals are often complex and difficult to process due to interference in the form of variations in the effective reflectivity of the target, noisy signals, complex signal shapes, and other dependencies. Deep neural networks have been a very popular area of research in computer artificial intelligence in recent years, allowing more implicit features to be uncovered than traditional shallow machine learning. It has been shown that convolutional and back-propagation neural networks can be used for SMI signal processing. There are also studies that have used machine learning genetic algorithms for absolute distance measurement. Based on the above, this study used convolutional neural networks to form a deep neural network for absolute distance measurement based on SMI technology. We first trained the deep convolutional neural network at different feedback strengths. The results of the Convolutional Neural Network (CNN) model showed a coefficient of determination of 0.9987. which is consistent with the required model. The trained network was then used to estimate absolute distances with and without the addition of noise. The comparison proves that the proposed method is noise-proof and has high adaptability for measurements under different conditions.

The terahertz (THz) metamaterial sensor design is typically complex and requires substantial expertise in physics. To simplify this process, we propose a novel reverse design model based on an improved conditional generative adversarial network that integrates self-attention generative adversarial network and Wasserstein generative adversarial network (WGAN) networks, and is referred to as the self-attention conditional Wasserstein GAN (SACW-GAN) model. By using the target response of the sensor as the input to the generator network, and incorporating labeling information, an attention mechanism, and the Wasserstein distance, we achieve effective reverse design of THz metamaterial sensors. The simulation results demonstrate the model's high performance, with spectral and image accuracies of 95% and 97%, respectively. This deep learning approach offers new perspectives and methodologies for the reverse design and application of THz metamaterial sensors, significantly advancing the field.

Aiming at the problems of low signal-to-noise ratio, low resolution and low illumination in low illumination environment, this paper proposes a low illumination images enhancement method based on generative adversarial networks with self attention mechanism. Firstly, the DenseNet framework is used to build the generator network, and the self attention mechanism module and constraint conditions are introduced. At the same time, the average discriminator is used to improve the traditional binary discriminator. Secondly, the low illumination images is introduced into the generator network to generate the illumination enhancement images, and then the discriminator network is used to supervise the enhancement effect of the generator on the low illumination images. Through the game between the generator and the discriminator, the network weight is continuously optimized, and finally the generator has better enhancement effect on the low illumination images. Experimental results show that, compared with the existing mainstream methods, this method not only has obvious advantages in brightness enhancement and clarity restoration of low illumination images, but also has significant advantages in objective evaluation indexes of images quality such as peak signal-to-noise ratio and structural similarity.KeywordsLow illumination images enhancementConditional generative adversarial networksDenseNetSelf attention mechanismAverage discriminator

Non-intrusive load monitoring (NILM) provides fine-grained consumption information at the appliance level by analyzing the terminal voltage and total current measured. It shows prospective applications in demand side management, such as demand response, energy efficiency, and home energy management system. However, most cutting-edge NILM models have a critical assumption that switching events are triggered by known appliances in the training set, which may be unrealistic. In reality, new appliances are constantly being added, reducing the effectiveness of current methods for load monitoring. In this paper, we propose a conditional generative adversarial network (CGAN) to correctly classify all the known appliances while simultaneously detecting unknown ones using the V-I trajectory features. We integrate variational autoencoder and capsule networks in the generator network, the capsule features of the same known appliance class are forced to match a pre-defined Gaussian and a group of Gaussian priors (one for each appliance class) as the posterior distribution approximately. Furthermore, using an additional encoder network maps the generated V-I trajectory to its latent representation, minimizing the distance between the latent representations and the feature vectors of the generator aids in learning the data distribution for known appliance samples during the training process. In these ways, we can improve the ability of the model to detect unknown appliances. Experimental results on two public datasets demonstrate the effectiveness and superiority of our method.

Deep convolutional neural networks have achieved great success in recent years. They have been widely used in various applications such as optical and SAR image scene classification, object detection and recognition, semantic segmentation, and change detection. However, deep neural networks rely on large-scale high-quality training data, and can only guarantee good performance when the training and test data are independently sampled from the same distribution. Deep convolutional neural networks are found to be vulnerable to subtle adversarial perturbations. This adversarial vulnerability prevents the deployment of deep neural networks in security-sensitive applications such as medical, surveillance, autonomous driving and military scenarios. This paper first presents a holistic view of security issues for deep convolutional neural network-based image recognition systems. The entire information processing chain is analyzed regarding safety and security risks. In particular, poisoning attacks and evasion attacks on deep convolutional neural networks are analyzed in detail. The root causes of adversarial vulnerabilities of deep recognition models are also discussed. Then, we give a formal definition of adversarial robustness and present a comprehensive review of adversarial attacks, adversarial defense, and adversarial robustness evaluation. Rather than listing existing research, we focus on the threat models for the adversarial attack and defense arms race. We perform a detailed analysis of several representative adversarial attacks on SAR image recognition models and provide an example of adversarial robustness evaluation. Finally, several open questions are discussed regarding recent research progress from our workgroup. This paper can be further used as a reference to develop more robust deep neural network-based image recognition models in dynamic adversarial scenarios.

The accuracy and speed of a single image super-resolution using a convolutional neural network is often a problem in improving finer texture details when using large enhancement factors. Some recent studies have focused on minimal mean square error, resulting in a high peak signal to noise ratio. Generally, although the peak signal to noise ratio has a high value, the output image is less detailed. This shows that the determination of super-resolution is not optimal. Conditional Generative Adversarial Network based on Boundary Equilibrium Generative Adversarial Network, by combining Mean Square Error Loss and GAN Loss as a loss function to optimize the super-resolution model and produce super-resolution images. Also, the generator network is designed with skip connection architecture to increase convergence speed and strengthen feature distribution. Image quality value parameters used in this study are Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index (SSIM). The results showed the highest image quality values using dataset validation were 26.55 for PSNR values and 0.93 for SSIM values. The highest image quality values using the testing dataset are 24.56 for the PSNR value and 0.91 for the SSIM value.

Subject motion in MRI remains an unsolved problem; motion during image acquisition may cause blurring and artifacts that severely degrade image quality. In this work, we approach motion correction as an image-to-image translation problem, which refers to the approach of training a deep neural network to predict an image in 1 domain from an image in another domain. Specifically, the purpose of this work was to develop and train a conditional generative adversarial network to predict artifact-free brain images from motion-corrupted data. An open source MRI data set comprising T2 *-weighted, FLASH magnitude, and phase brain images for 53 patients was used to generate complex image data for motion simulation. To simulate rigid motion, rotations and translations were applied to the image data based on randomly generated motion profiles. A conditional generative adversarial network, comprising a generator and discriminator networks, was trained using the motion-corrupted and corresponding ground truth (original) images as training pairs. The images predicted by the conditional generative adversarial network have improved image quality compared to the motion-corrupted images. The mean absolute error between the motion-corrupted and ground-truth images of the test set was 16.4% of the image mean value, whereas the mean absolute error between the conditional generative adversarial network-predicted and ground-truth images was 10.8% The network output also demonstrated improved peak SNR and structural similarity index for all test-set images. The images predicted by the conditional generative adversarial network have quantitatively and qualitatively improved image quality compared to the motion-corrupted images.

Hybrid short-term load forecasting using CGAN with CNN and semi-supervised regression

Interpreting seismic data requires the characterization of a number of key elements such as the position of faults and main reflections, presence of structural bodies, and clustering of areas exhibiting a similar amplitude versus angle response. Manual interpretation of geophysical data is often a difficult and time-consuming task, complicated by lack of resolution and presence of noise. In recent years, approaches based on convolutional neural networks have shown remarkable results in automating certain interpretative tasks. However, these state-of-the-art systems usually need to be trained in a supervised manner, and they suffer from a generalization problem. Hence, it is highly challenging to train a model that can yield accurate results on new real data obtained with different acquisition, processing, and geology than the data used for training. In this work, we introduce a novel method that combines generative neural networks with a segmentation task in order to decrease the gap between annotated training data and uninterpreted target data. We validate our approach on two applications: the detection of diffraction events and the picking of faults. We show that when transitioning from synthetic training data to real validation data, our workflow yields superior results compared to its counterpart without the generative network.