STaDA: Style Transfer as Data Augmentation

Portrait style transfer using deep convolutional neural networks and facial segmentation

Recently, style transfer based on the convolutional neural network has achieved remarkable results. In this paper, we extend the original neural style transfer algorithm to ameliorate the instability in the reconstruction of certain structural information, and improve the ghosting artefacts in the background of image which with low texture and homogeneous areas. For that end, we adopt zigzag learning strategy: The model parameters are optimized to an intermediate target firstly, then let the model converge to the final goal. We show the zigzag learning to multi-sample model which is fabricated from resampling the style input and to loss function that is split into two sections. And also, we demonstrate experimentally the effectiveness of the proposed algorithm and provide its theoretical analysis. Finally we show how to integrate the zigzag learning strategy in fast neural style transfer framework.

Style transfer is an optimization technique used to create a new image out of a content image and a style image (as in an artistic work by a well-known painter) as a result of blending, the content image is altered to be painted in the same style as the content image but to resemble it of the style image. The essential method for creating style transfer is the convolutional neural network that allows for transfer (CNN). This paper will analyse key methods for performing style transfer photos and quickly compare multiple models and their outcomes. We primarily contrast one pioneering technique, developed by Leon Gatys in 2015, with the most recent technology in the High-Resolution Network from 2019 for the transfer of photorealistic style. Gatys's Neural Algorithm of Artistic Style produces excellent results, but the resulting data doesn't maintain the content image's characteristics and the fact that the paper gives no concise description of the internal mechanisms gram matrix. The enhancement is the conversion to a photorealistic style transfer from Gatys' neural style transfer. It aids in maintaining the structure and defining characteristics within the content image. Both methods make use of VGG, a network that was trained using the ImageNet dataset for conducting image categorization. We made our own data set with five types or groups and used transfer learning in a VGG network to do a style transfer using transfer learning.

The development of Convolutional Neural Networks (CNN) and Generative Adversarial Networks (GAN) has catalysed the emergence of the Neural Style Transfer (NST) field. NST elucidates the computer's comprehension of artistic elements, holding paramount significance in both academic and practical domains. In this paper, we introduce a novel system termed "Content-Adaptive Style Transfer." This system incorporates two additional techniques: object segmentation and image fusion, augmenting the customizability of NST. Our experimental results underscore its capability to execute style transfer exclusively on the foreground, leaving the background unaffected, or to modify the background style while preserving the original foreground. These innovations promise more rational outcomes in specific scenarios.

Availability of limited training remote sensing datasets is one of the problems in deep learning, as deep architectures require a large number of training samples for proper training. In this paper, we present a technique for data augmentation based on a spectral indexed generative adversarial network to train deep convolutional neural networks. This technique uses the spectral characteristic of multispectral (MS) images to support data augmentation in order to generate realistic training samples with respect to each land-use and land-cover class. The impact of multispectral remote sensing data generated through the spectral indexed GAN are evaluated through classification experiments. Experimental results obtained on the classification of the Sentinel-2 Eurosatallband datasets show that data augmentation through spectral indexed GAN enhances the main accuracy metrics.

The deep learning method is widely used in computer vision tasks with large-scale annotated datasets. However, obtaining such datasets in most directions of the vision based nondestructive testing (NDT) field is very challenging. Data augmentation is proved as an efficient way of dealing with the lack of large-scale annotated datasets. In this paper, we propose a CycleGAN-based extra-supervised (CycleGAN-ES) model to generate synthetic NDT images, where the ES is used to ensure that the bidirectional mapping is learned for corresponding labels and defects. Furthermore, we show the effectiveness of using the synthesized images to train deep convolutional neural networks (DCNNs) for defect recognition. In the experiments, we extract a number of x-ray welding images with both defect and no defects from the published GDXray dataset, and CycleGAN-ES is used to generate the synthetic defect images based on a small number of extracted defect images and manually drawn labels that are used as a content guide. For quality verification of the synthesized defect images, we use a high-performance classifier pretrained using a big dataset to recognize the synthetic defects and show the comparability of the performances of classifiers trained using synthetic defects and real defects, respectively. To present the effectiveness of using the synthesized defects as an augmentation method, we train and evaluate the performances of DCNN for defect recognition with or without the synthesized defects.

Data augmentation (DA) increases the diversity of training data to improve the model generalization ability. Most of the DA methods focus on image classification or object detection. Directly using the existing DA methods on change detection tasks not only falls short of fully exploring the specificity of the change image pairs but also leads to longer training times. In this article, we first propose a mask-guided mixing (MGM) DA for change detection, which mixes the change regions of the current training sample based on prediction results and labels to generate high-quality samples with more positive samples. We then propose a new reinforcement learning (RL)-based Adaptive DA method, AdaAug+, to adaptively select the optimal DA policy for the training samples. An actor selects the best augmentation operation from the operation set according to the image pair. The augmented image pairs make it easier for the change detector to learn the optimal parameters and improve the final detection performance. To reduce the training time, we identify and remove the redundant training samples during the training process by our redundancy searching policy. We have conducted various experiments on four remote sensing change detection datasets with different change detectors. The experimental results demonstrate that AdaAug+ achieves promising performance compared to the state-of-the-art DA methods and requires less training time.

Deep convolutional neural network training enrichment using multi-view object-based analysis of Unmanned Aerial systems imagery for wetlands classification

Breast cancer is a fatal disease that affects millions of women worldwide. The number of cases of breast cancer has risen over time. Although it is difficult to prevent this disease, the survival rate can be improved with early detection and proper treatment planning of the disease. Thanks to breakthroughs in deep learning, the progress of computer-assisted diagnosis (CAD) of breast cancer has seen a lot of improvement. Deep neural networks have advanced to the point where their diagnostic capabilities are approaching those of a human specialist. Mammograms are crucial radiological images used to detect breast cancer at its early stage. Mammogram image scaling at the input layer is required for training deep convolutional neural networks (DCNNs) directly on high-resolution images. This could result in the loss of crucial information for discovering medical abnormalities. Instead of developing an image classifier, the idea is to create a patch classifier. The technique for extracting abnormal patches from mammography images is proposed in this research. Patches are extracted from a benchmark and publicly available MIAS dataset. These patches are then used to train deep learning classifiers such as VGG-16, ResNet-50, and EfficientNet-B7. With the patches already included in the CBIS-DDSM dataset, we contrast the results of the MIAS patches. EfficientNet-B7 on CBIS-DDSM patches produced good results (92% accuracy) when compared to other classifiers such as VGG-16 and ResNet-50. We also discovered that ResNet-50 is demonstrated to be quite robust on both datasets.

A problem with convolutional neural networks (CNNs) is that they require large datasets to obtain adequate robustness; on small datasets, they are prone to overfitting. Many methods have been proposed to overcome this shortcoming with CNNs. In cases where additional samples cannot easily be collected, a common approach is to generate more data points from existing data using an augmentation technique. In image classification, many augmentation approaches utilize simple image manipulation algorithms. In this work, we propose some new methods for data augmentation based on several image transformations: the Fourier transform (FT), the Radon transform (RT), and the discrete cosine transform (DCT). These and other data augmentation methods are considered in order to quantify their effectiveness in creating ensembles of neural networks. The novelty of this research is to consider different strategies for data augmentation to generate training sets from which to train several classifiers which are combined into an ensemble. Specifically, the idea is to create an ensemble based on a kind of bagging of the training set, where each model is trained on a different training set obtained by augmenting the original training set with different approaches. We build ensembles on the data level by adding images generated by combining fourteen augmentation approaches, with three based on FT, RT, and DCT, proposed here for the first time. Pretrained ResNet50 networks are finetuned on training sets that include images derived from each augmentation method. These networks and several fusions are evaluated and compared across eleven benchmarks. Results show that building ensembles on the data level by combining different data augmentation methods produce classifiers that not only compete competitively against the state-of-the-art but often surpass the best approaches reported in the literature.

Practical application of deep learning based non-intrusive load monitoring (NILM) system requires the deep neural network model to generalize on new unseen data. Existing NILM solutions are not suitable for real-world application due to their poor disaggregation accuracy on new unseen data. In order to address this problem, this paper presents a NILM algorithm that uses data augmentation to generate synthetic data for training deep convolutional neural network models for each target appliance. Proposed data augmentation technique works by combining on and off-durations of a target appliance from various datasets, and forms a unified and comprehensive synthetic aggregate and sub-meter profiles. Apart from proposed algorithm, this paper also proposes an evaluation approach that relies on total predicted energy and ground-truth energy of an appliance to provide detailed insights about total overlapping energy, missing energy and extra energy predicted by the algorithm. Comparison results on our proposed evaluation approach showed that proposed disaggregation algorithm was able to predict energy that was 60% overlapping with ground truth energy and 36% energy was extra. Overall results showed that overlapping energy was 2.5 times more, and extra-predicted energy was 60% less than state-of-the-art algorithms in unseen test cases.

The lack of training samples remains one of the major obstacles in applying convolutional neural networks (CNNs) to the hyperspectral image (HSI) classification. In this letter, the accurate classification of HSI with limited training samples is investigated. Due to the advantages of minimizing the distance between samples in the same class and maximizing the distance between samples in different classes, siamese CNN is used for HSI classification with limited training samples. After that, to improve the classification performance, data augmentation is investigated for siamese CNN-based HSI classification. Specifically, pair data augmentation based on CutMix is proposed to generate the training pairs of the same or different classes and the new generated training pairs are used to train siamese CNN. Traditional data augmentation methods simply generate new training samples. It is not proper when data augmentation methods keep the loss function and change the content of the input at the same time. Therefore, a soft-loss-based siamese CNN, which changes its loss according to the coupled replacement data augmentation, is proposed to further address the HSI classification with limited training samples. In the experimental part, on two widely used hyperspectral datasets, the influences of different training samples and window sizes are discussed, and the experimental results reveal that the proposed soft-augmentation-based siamese CNN provides competitive results with limited training samples compared with state-of-the-art methods.

The image classification is a classical problem of image processing, computer vision and machine learning fields. In this paper we study the image classification using deep learning with Neural style transfer that has been a high risk application for deep learning, make attention from and advertising the effectiveness to both the academic prisons and the general public. However, we have found by removal experiments that optimizing an image in the way neural style transfer does, we can even factor out the deepness (multiple layers of exchange linear and nonlinear transformations) all together and have neural style transfer working to a certain range. We present the VGG-19 Artistic Type Neural Algorithm, which can convert and recombine the image quality and style of natural images. This algorithm allows us to produce new images of high perceptual quality that combine the content of an arbitrary photograph with the appearance of numerous well known portrait. Our results provide new insights into the deep image representations learned by Convolutional Neural Networks (CNN) and demonstrate their potential for high level image synthesis and manipulation.

In today's world, Neural Style Transfer (NST) has become a trendsetting term. NST combines two pictures, a content picture and a reference image in style (such as the work of a renowned painter) in a way that makes the output image look like an image of the material, but rendered with the form of a reference picture. However, there is no study using the artwork or painting of Bangladeshi painters. Bangladeshi painting has a long history of more than two thousand years and is still being practiced by Bangladeshi painters. This study generates NST stylized image on Bangladeshi paintings and analyzes the human point of view regarding the aesthetic preference of NST on Bangladeshi paintings. To assure our study's acceptance, we performed qualitative human evaluations on generated stylized images by 60 individual humans of different age and gender groups. We have explained how NST works for Bangladeshi paintings and assess NST algorithms, both qualitatively \& quantitatively. Our study acts as a pre-requisite for the impact of NST stylized image using Bangladeshi paintings on mobile UI/GUI and material translation from the human perspective. We hope that this study will encourage new collaborations to create more NST related studies and expand the use of Bangladeshi artworks.

<span id="docs-internal-guid-cdb76bbb-7fff-978d-961c-e21c41807064"><span>During the last few years, deep learning achieved remarkable results in the field of machine learning when used for computer vision tasks. Among many of its architectures, deep neural network-based architecture known as convolutional neural networks are recently used widely for image detection and classification. Although it is a great tool for computer vision tasks, it demands a large amount of training data to yield high performance. In this paper, the data augmentation method is proposed to overcome the challenges faced due to a lack of insufficient training data. To analyze the effect of data augmentation, the proposed method uses two convolutional neural network architectures. To minimize the training time without compromising accuracy, models are built by fine-tuning pre-trained networks VGG16 and ResNet50. To evaluate the performance of the models, loss functions and accuracies are used. Proposed models are constructed using Keras deep learning framework and models are trained on a custom dataset created from Kaggle CAT vs DOG database. Experimental results showed that both the models achieved better test accuracy when data augmentation is employed, and model constructed using ResNet50 outperformed VGG16 based model with a test accuracy of 90% with data augmentation & 82% without data augmentation.</span></span>