Feature transforms for image data augmentation

A problem with Convolutional Neural Networks (CNNs) is that they require large datasets to obtain adequate robustness; on small datasets, they are prone to overfitting. Many methods have been proposed to overcome this shortcoming with CNNs. In cases where additional samples cannot easily be collected, a common approach is to generate more data points from existing data using an augmentation technique. In image classification, many augmentation approaches utilize simple image manipulation algorithms. In this work, we build ensembles on the data level by adding images generated by combining fourteen augmentation approaches, three of which are proposed here for the first time. These novel methods are based on the Fourier Transform (FT), the Radon Transform (RT) and the Discrete Cosine Transform (DCT). Pretrained ResNet50 networks are finetuned on training sets that include images derived from each augmentation method. These networks and several fusions are evaluated and compared across eleven benchmarks. Results show that building ensembles on the data level by combining different data augmentation methods produce classifiers that not only compete competitively against the state-of-the-art but often surpass the best approaches reported in the literature.

Convolutional Neural Network (CNN) tend to have better results on large data sets and poor performance on small data sets, so the data augmentation is crucial for a CNN to get better performance based on the dataset with limited size. In this paper, Deep Convolution Generative Adversarial Network (DCGAN) was used to augment data to make the AlexNet perform better on an image classification task with small data sets. AlexNet was trained on a small anime face training set with only 160 samples to determine whether the anime face was male or female, and then tested its accuracy on a test set with 240 sample. Then, a pre-trained DCGAN was transferred to train on the male and female training sets respectively. And 2 DCGANs were obtained, one could generate male cartoon faces and another could generate female cartoon faces. The images generated by DCGANs were put in train set, which was used to train AlexNet again and the result was recorded. Other data augmentation methods such as cutout, cutmix and Noise Injection were compared as well. Finally, it is found that AlexNet has the best performance when using the DCGAN augmentation method, which can significantly improve the verification accuracy of the model.

Successful training of convolutional neural networks (CNNs) requires a substantial amount of data. With small datasets, networks generalize poorly. Data Augmentation techniques improve the generalizability of neural networks by using existing training data more effectively. Standard data augmentation methods, however, produce limited plausible alternative data. Generative Adversarial Networks (GANs) have been utilized to generate new data and improve the performance of CNNs. Nevertheless, data augmentation techniques for training GANs are underexplored compared to CNNs. In this work, we propose a new GAN architecture for augmentation of chest X-rays for semi-supervised detection of pneumonia and COVID-19 using generative models. We show that the proposed GAN can be used to effectively augment data and improve classification accuracy of disease in chest X-rays for pneumonia and COVID-19. We compare our augmentation GAN model with Deep Convolutional GAN and traditional augmentation methods (rotate, zoom, etc.) on two different X-ray datasets and show our GAN-based augmentation method surpasses other augmentation methods for training a GAN in detecting anomalies in X-ray images.

Crop yield estimation and prediction constitutes a key issue in agricultural management, particularly under the context of demographic pressure and climate change. Currently, the main challenge in estimating crop yields based on remotely sensed data and data-driven methods is how to cope with small datasets and the limited amount of annotated samples. In order to cope with small datasets and the limited amount of annotated samples and improve the accuracy of winter wheat yield estimation in the Guanzhong Plain, PR China, this study proposed a method of combining generative adversarial networks (GANs) and convolutional neural network (CNN) for comprehensive growth monitoring of winter wheat, in which the remotely sensed leaf area index (LAI), vegetation temperature condition index (VTCI) and meteorological data at four growth stages of winter wheat during 2012–2017 were generated as the inputs of multi-layer convolutional neural networks (CNNs), and GAN was employed to artificially increase the number of training samples. Then, a linear regression model between the simulated comprehensive growth monitoring (I) and the measured yields was established to estimate yields of winter wheat in the Guanzhong Plain pixel by pixel. The final results showed when GAN was used to double the size of the training samples, and the simulation values obtained by CNN based on augmented samples using GAN provided a better training (R2 = 0.95, RMSE = 0.05), validation (R2 = 0.54, RMSE = 0.16) and testing (R2 = 0.50, RMSE = 0.14) performance than that just using the original samples. The achieved best pixel-scale yield estimation accuracy of winter wheat (R2 = 0.50, RMSE = 591.46 kg/ha) in the Guanzhong Plain. These results showed that small samples can be enlarged by GAN, thus, more important features for reflecting the growth conditions and yields of winter wheat from the remotely sensed indices and meteorological indices can be extracted, and indicated that CNN accompanied with GAN could contribute a lot to the comprehensive growth monitoring and yield estimation of winter wheat and data augmentation methods are extremely useful for the application of small samples in deep learning.

This study introduces a new data augmentation method to improve the corruption robustness of the convolutional neural network (CNN). Many data augmentation methods have been studied to reduce overfitting and to improve the generalization ability of CNNs. One of the most widely used data augmentation techniques is Random Erasing which erases a random rectangle region of an image. Most of the augmentation methods are applied in the spatial domain, but augmentation techniques in the frequency domain are less studied. In this study, the image is processed with 2D discrete Fourier transform (DFT), and then Random Erasing is applied in the frequency domain. Finally, the inversed image generated with inverse DFT is used as an input. As a result, the proposed method leads to the robustness improvement of the model against common corruptions.

Rotated target recognition is a challenge for Convolutional Neural Networks (CNN), and the current solution is to make CNN rotational invariant through data augmentation. However, data augmentation makes CNN easy to overfit small scale sonar image datasets, and increases its numbers of parameters and training time. This paper proposes to recognize rotated targets of sonar images using a novel CNN with Rotated Inputs (RICNN), which doesn’t need data augmentation. During training, RICNN was trained with sonar images of targets only at one orientation, which avoid it to learn multiple rotated versions of the same targets, and reduces both number of parameters and training time of CNN. During testing, RICNN calculated classification scores for each test image and its all-possible rotated versions. The max of these classification scores were used to simultaneously estimate the category and orientation of each target. Besides, to improve the generalization of RICNN on imbalanced sonar datasets, this paper also designs an imbalanced data sampler. Experiments on a self-made small, imbalanced sonar image rotated target recognition dataset show that the improved RICNN achieves 4.25% higher classification accuracy than data augmentation, and reduces the number of parameters and training time to 2.25% and 19.2% of that of data augmentation method. Moreover, RICNN achieves comparable orientation estimation accuracy with a CNN orientation regressor trained with data augmentation. Codes, dataset are publicly available.

Deep learning has recently been applied to automatically classify the modulation categories of received radio signals without manual experience. However, training deep learning models requires massive volume of data. An insufficient training data will cause serious overfitting problem and degrade the classification accuracy. To cope with small dataset, data augmentation has been widely used in image processing to expand the dataset and improve the robustness of deep learning models. However, in wireless communication areas, the effect of different data augmentation methods on radio modulation classification has not been studied yet. In this paper, we evaluate different data augmentation methods via a state-of-the-art deep learning-based modulation classifier. Based on the characteristics of modulated signals, three augmentation methods are considered, i.e., rotation, flip, and Gaussian noise, which can be applied in both training phase and inference phase of the deep learning algorithm. Numerical results show that all three augmentation methods can improve the classification accuracy. Among which, the rotation augmentation method outperforms the flip method, both of which achieve higher classification accuracy than the Gaussian noise method. Given only 12.5% of training dataset, a joint rotation and flip augmentation policy can achieve even higher classification accuracy than the baseline with initial 100% training dataset without augmentation. Furthermore, with data augmentation, radio modulation categories can be successfully classified using shorter radio samples, leading to a simplified deep learning model and shorter the classification response time.

The success of training deep Convolutional Neural Networks (CNNs) heavily depends on a significant amount of labelled data. Recent research has found that neural style transfer algorithms can apply the artistic style of one image to another image without changing the latter's high-level semantic content, which makes it feasible to employ neural style transfer as a data augmentation method to add more variation to the training dataset. The contribution of this paper is a thorough evaluation of the effectiveness of the neural style transfer as a data augmentation method for image classification tasks. We explore the state-of-the-art neural style transfer algorithms and apply them as a data augmentation method on Caltech 101 and Caltech 256 dataset, where we found around 2% improvement from 83% to 85% of the image classification accuracy with VGG16, compared with traditional data augmentation strategies. We also combine this new method with conventional data augmentation approaches to further improve the performance of image classification. This work shows the potential of neural style transfer in computer vision field, such as helping us to reduce the difficulty of collecting sufficient labelled data and improve the performance of generic image-based deep learning algorithms.

Optical character recognition is a technology that turns texts and scanned documents into digital formats. The practical applications of OCR systems face a lot of challenges because of heterogeneity in scripts, font styles, and different quality images arising during their usage. The proposed work attempts to improve the robustness of OCR models built for the Hindi language by addressing the deficiencies inherent in the widely used IIIT-HW-Dev dataset. Toward this end, two methods of data augmentation are proposed: 1) synthetic images with half characters and conjuncts for complex words, and 2) various image degradations to simulate the real world. The augmented training set is used to train the two deep architectures, namely Convolutional Recurrent Neural Networks (CRNNs) and ResNet-50. Comparisons and in-depth experiments further reveal the merits of the suggested data augmentation techniques: the top performance obtained by the CRNN model exceeds current approaches. On the test set, CRNN achieved a Character Error Rate (CER) of 2.14% and Word Error Rate (WER) of 7.96%, much beyond the CER of 3.27% and WER of 11.82% achieved by the ResNet-50 model. Qualitatively, there was robustness in the case of complex words and images that were also deteriorated. The augmented dataset and trained models can be downloaded by anyone to promote more research in Hindi OCR. So, developing robust OCR systems for complex scripts starts from variability-rich and representative training data along with specific architectures, such as the CRNN architecture.

This study presents an improved data augmentation technique for the classification of remote sensing (RS) scenes. The method is called Quality-based Sample Selection (QSS) data augmentation technique. It is based on generating a large number of samples using geometric transformations and then selecting the best ones based on a quality criterion. Sample images are generated online, i.e. during training of the convolutional neural network (CNN) model. For each training sample image, a few images are generated randomly using a set of geometric transformations. Then, the generated images are passed to the CNN model being trained, and the predicted probabilities are used to evaluate the quality of the images. The motivation behind this research is that we aim to augment the training set with new images that will benefit the learning of the CNN model. Therefore, an objective method to evaluate the quality of the new images is the CNN model itself. The image with the best criteria score is used to augment the training set. Several quality criteria based on the prediction probabilities of the CNN model are considered including entropy, breaking-ties and our own proposed criteria called True Label Probability (TLP). QSS is tested on five common RS scene classification datasets: UCMerced, Optimal31, RSSCN7, AID and NWPU-RS45. It has outperformed all previous methods except for one case involving NWPU-RS45 dataset. For the 10%–90% train–test split, QSS achieved 94.51, 81.72, 89.89, 94.09 and 93.98, respectively, for the mentioned datasets. While for the 20%–80% train–test split, it achieved 97.09, 88.37, 94.01, 95.71 and 94.71, respectively, for the listed datasets. Thus, this work presents a novel data augmentation method that uses data with higher quality instead of randomly selected. The paper proves experimentally that the proposed method improves the classification accuracy of RS scene classification.

With global warming, increasingly severe forest fires have caused serious damage to the Earth’s ecological environment. Remote sensing (RS) technology has become an important means of forest fire monitoring owing to its unique advantages of multiple cycles and wide coverage. In smoke scene detection, the current simple mixed sample data augmentation (MSDA) methods lead to the loss of key objects in images, and training the samples whose images do not match the labels is the main reason for the decline in classifier accuracy. We propose a Class Activation Map-based Mixing method (CAMMix), a new MSDA method. In contrast to previous data augmentation (DA) methods, CAMMix can select the region and degree of mixing throughout the significance map. With the aid of an auxiliary classifier, CAMMix generates a mixed mask with class significance, such that the newly generated mixed sample distribution is close to the original data distribution. We also propose an intervention for the coverage method to further prevent the loss of smoke objects. On a smoke dataset (USTC_SmokeRS), CAMMix achieves the best accuracy of 94.95% and 83% for 64% and 8% training set respectively. It results that CAMMix and variants provides high classification accuracy and preserves key object information, outperforming the Mixup, CutMix, ResizeMix and FMix.

(Aim) Breast cancer is the most common cancer in women and the second most common cancer worldwide. With the rapid advancement of deep learning, the early stages of breast cancer development can be accurately detected by radiologists with the help of artificial intelligence systems. (Method) Based on mammographic imaging, a mainstream clinical breast screening technique, we present a diagnostic system for accurate classification of breast abnormalities based on ResNet-50. To improve the proposed model, we created a new data augmentation framework called SCDA (Scaling and Contrast limited adaptive histogram equalization Data Augmentation). In its procedure, we first conduct the scaling operation to the original training set, followed by applying contrast limited adaptive histogram equalisation (CLAHE) to the scaled training set. By stacking the training set after SCDA with the original training set, we formed a new training set. The network trained by the augmented training set, was coined as ResNet-SCDA-50. Our system, which aims at a binary classification on mammographic images acquired from INbreast and MINI-MIAS, classifies masses, microcalcification as "abnormal", while normal regions are classified as "normal". (Results) We present the first attempt to use the image contrast enhancement method as the data augmentation method, resulting in an averaged 98.55 percent specificity and 92.83 percent sensitivity, which gives our best model an overall accuracy of 95.74 percent. (Conclusion) Our proposed method is effective in classifying breast abnormality.

<span id="docs-internal-guid-cdb76bbb-7fff-978d-961c-e21c41807064"><span>During the last few years, deep learning achieved remarkable results in the field of machine learning when used for computer vision tasks. Among many of its architectures, deep neural network-based architecture known as convolutional neural networks are recently used widely for image detection and classification. Although it is a great tool for computer vision tasks, it demands a large amount of training data to yield high performance. In this paper, the data augmentation method is proposed to overcome the challenges faced due to a lack of insufficient training data. To analyze the effect of data augmentation, the proposed method uses two convolutional neural network architectures. To minimize the training time without compromising accuracy, models are built by fine-tuning pre-trained networks VGG16 and ResNet50. To evaluate the performance of the models, loss functions and accuracies are used. Proposed models are constructed using Keras deep learning framework and models are trained on a custom dataset created from Kaggle CAT vs DOG database. Experimental results showed that both the models achieved better test accuracy when data augmentation is employed, and model constructed using ResNet50 outperformed VGG16 based model with a test accuracy of 90% with data augmentation & 82% without data augmentation.</span></span>

Preliminary geological mapping with convolution neural network using statistical data augmentation on a 3D model

Convolutional Neural Networks (CNN) can achieve excellent computer-assisted diagnosis with a good amount of data. However, there is still a growing demand for specific data and information for training Machine Learning models, either for classification or other tasks such as segmentation. Towards this, the Data Augmentation (DA) technique can handle the small medical imaging dataset problem by generating artificial training data. In this context, Generative Adversarial Networks (GANs) can synthesize realistic images to increase the number of images in a dataset. Therefore, to maximize the DA efficiency in a CNNbased tumor classification task, we propose using non-extensive Gabor filters as a convolutional layer kernel initializer. Our proposal has been tested in the BraTS15 dataset and results show that CNN with an additional q-Gabor layer can achieve an average accuracy 3.65% better than CNN with Gabor and 5.03% better than the default model when trained with artificial images (data augmentation).