Class Activation Map-Based Data Augmentation for Satellite Smoke Scene Detection

Data augmentation (DA) increases the diversity of training data to improve the model generalization ability. Most of the DA methods focus on image classification or object detection. Directly using the existing DA methods on change detection tasks not only falls short of fully exploring the specificity of the change image pairs but also leads to longer training times. In this article, we first propose a mask-guided mixing (MGM) DA for change detection, which mixes the change regions of the current training sample based on prediction results and labels to generate high-quality samples with more positive samples. We then propose a new reinforcement learning (RL)-based Adaptive DA method, AdaAug+, to adaptively select the optimal DA policy for the training samples. An actor selects the best augmentation operation from the operation set according to the image pair. The augmented image pairs make it easier for the change detector to learn the optimal parameters and improve the final detection performance. To reduce the training time, we identify and remove the redundant training samples during the training process by our redundancy searching policy. We have conducted various experiments on four remote sensing change detection datasets with different change detectors. The experimental results demonstrate that AdaAug+ achieves promising performance compared to the state-of-the-art DA methods and requires less training time.

Deep learning has recently been applied to automatically classify the modulation categories of received radio signals without manual experience. However, training deep learning models requires massive volume of data. An insufficient training data will cause serious overfitting problem and degrade the classification accuracy. To cope with small dataset, data augmentation has been widely used in image processing to expand the dataset and improve the robustness of deep learning models. However, in wireless communication areas, the effect of different data augmentation methods on radio modulation classification has not been studied yet. In this paper, we evaluate different data augmentation methods via a state-of-the-art deep learning-based modulation classifier. Based on the characteristics of modulated signals, three augmentation methods are considered, i.e., rotation, flip, and Gaussian noise, which can be applied in both training phase and inference phase of the deep learning algorithm. Numerical results show that all three augmentation methods can improve the classification accuracy. Among which, the rotation augmentation method outperforms the flip method, both of which achieve higher classification accuracy than the Gaussian noise method. Given only 12.5% of training dataset, a joint rotation and flip augmentation policy can achieve even higher classification accuracy than the baseline with initial 100% training dataset without augmentation. Furthermore, with data augmentation, radio modulation categories can be successfully classified using shorter radio samples, leading to a simplified deep learning model and shorter the classification response time.

Intelligent fault diagnosis is an important method in rotating machinery fault diagnosis and equipment health management. To deal with co-frequency vibration faults, a type of typical fault in rotating machinery, this paper proposes a fault diagnosis method based on the stacked autoencoder (SAE) and ensembled ResNet-SVM. Furthermore, the time- and frequency-domain features of several co-frequency vibration faults are summarized based on the mechanism analysis and calculated using actual vibration data. To realize and validate the high-precision diagnosis method of rotating equipment with co-frequency faults proposed in this study, the following three criteria are required: First, to improve the effectiveness and robustness of the ensembled model and the sliding window using data augmentation, adding noise, autoencoder (AE) and SAE methods are analyzed in terms of principle and practical effects. Second, ResNet is used as the feature extractor for the ensembled ResNet-SVM model. Feature extraction is carried out twice, and the extracted co-frequency fault features are more comprehensive. Finally, the data augmentation method and ensemble ResNet-SVM are combined for fault diagnosis and compared with other methods. The experimental results show that the accuracy of the proposed method can exceed 99.9%.

The rapid serial visual presentation-based brain–computer interface (RSVP-BCI) system achieves the recognition of target images by extracting event-related potential (ERP) features from electroencephalogram (EEG) signals and then building target classification models. Currently, how to reduce the training and calibration time for classification models across different subjects is a crucial issue in the practical application of RSVP. To address this issue, a zero-calibration (ZC) method termed Attention-ProNet, which involves meta-learning with a prototype network integrating multiple attention mechanisms, was proposed in this study. In particular, multiscale attention mechanisms were used for efficient EEG feature extraction. Furthermore, a hybrid attention mechanism was introduced to enhance model generalization, and attempts were made to incorporate suitable data augmentation and channel selection methods to develop an innovative and high-performance ZC RSVP-BCI decoding model algorithm. The experimental results demonstrated that our method achieved a balance accuracy (BA) of 86.33% in the decoding task for new subjects. Moreover, appropriate channel selection and data augmentation methods further enhanced the performance of the network by affording an additional 2.3% increase in BA. The model generated by the meta-learning prototype network Attention-ProNet, which incorporates multiple attention mechanisms, allows for the efficient and accurate decoding of new subjects without the need for recalibration or retraining.

Combinations of data augmentation methods and deep learning architectures for automatic pancreas segmentation on CT images are proposed and evaluated. Images from a public CT dataset of pancreas segmentation were used to evaluate the models. Baseline U-net and deep U-net were chosen for the deep learning models of pancreas segmentation. Methods of data augmentation included conventional methods, mixup, and random image cropping and patching (RICAP). Ten combinations of the deep learning models and the data augmentation methods were evaluated. Four-fold cross validation was performed to train and evaluate these models with data augmentation methods. The dice similarity coefficient (DSC) was calculated between automatic segmentation results and manually annotated labels and these were visually assessed by two radiologists. The performance of the deep U-net was better than that of the baseline U-net with mean DSC of 0.703–0.789 and 0.686–0.748, respectively. In both baseline U-net and deep U-net, the methods with data augmentation performed better than methods with no data augmentation, and mixup and RICAP were more useful than the conventional method. The best mean DSC was obtained using a combination of deep U-net, mixup, and RICAP, and the two radiologists scored the results from this model as good or perfect in 76 and 74 of the 82 cases.

Data augmentation is widely utilized as an effective technique to enhance the generalization performance of deep models. However, data augmentation may inevitably introduce distribution shifts and noises, which significantly constrain the potential and deteriorate the performance of deep networks. To this end, we propose a novel information-preserving framework, namely IPF-RDA, to enhance the robustness of data augmentations in this paper. IPF-RDA combines the proposal of (i) a new class-discriminative information estimation algorithm that identifies the points most vulnerable to data augmentation operations and corresponding importance scores; And (ii) a new information-preserving scheme that preserves the critical information in the augmented samples and ensures the diversity of augmented data adaptively. We divide data augmentation methods into three categories according to the operation types and integrate these approaches into our framework accordingly. After being integrated into our framework, the robustness of data augmentation methods can be enhanced and their full potential can be unleashed. Extensive experiments demonstrate that although being simple, IPF-RDA consistently improves the performance of numerous commonly used state-of-the-art data augmentation methods with popular deep models on a variety of datasets, including CIFAR-10, CIFAR-100, Tiny-ImageNet, CUHK03, Market1501, Oxford Flower, and MNIST, where its performance and scalability are stressed.

A problem with convolutional neural networks (CNNs) is that they require large datasets to obtain adequate robustness; on small datasets, they are prone to overfitting. Many methods have been proposed to overcome this shortcoming with CNNs. In cases where additional samples cannot easily be collected, a common approach is to generate more data points from existing data using an augmentation technique. In image classification, many augmentation approaches utilize simple image manipulation algorithms. In this work, we propose some new methods for data augmentation based on several image transformations: the Fourier transform (FT), the Radon transform (RT), and the discrete cosine transform (DCT). These and other data augmentation methods are considered in order to quantify their effectiveness in creating ensembles of neural networks. The novelty of this research is to consider different strategies for data augmentation to generate training sets from which to train several classifiers which are combined into an ensemble. Specifically, the idea is to create an ensemble based on a kind of bagging of the training set, where each model is trained on a different training set obtained by augmenting the original training set with different approaches. We build ensembles on the data level by adding images generated by combining fourteen augmentation approaches, with three based on FT, RT, and DCT, proposed here for the first time. Pretrained ResNet50 networks are finetuned on training sets that include images derived from each augmentation method. These networks and several fusions are evaluated and compared across eleven benchmarks. Results show that building ensembles on the data level by combining different data augmentation methods produce classifiers that not only compete competitively against the state-of-the-art but often surpass the best approaches reported in the literature.

Discriminative feature constraints via supervised contrastive learning for few-shot forest tree species classification using airborne hyperspectral images

This study presents an improved data augmentation technique for the classification of remote sensing (RS) scenes. The method is called Quality-based Sample Selection (QSS) data augmentation technique. It is based on generating a large number of samples using geometric transformations and then selecting the best ones based on a quality criterion. Sample images are generated online, i.e. during training of the convolutional neural network (CNN) model. For each training sample image, a few images are generated randomly using a set of geometric transformations. Then, the generated images are passed to the CNN model being trained, and the predicted probabilities are used to evaluate the quality of the images. The motivation behind this research is that we aim to augment the training set with new images that will benefit the learning of the CNN model. Therefore, an objective method to evaluate the quality of the new images is the CNN model itself. The image with the best criteria score is used to augment the training set. Several quality criteria based on the prediction probabilities of the CNN model are considered including entropy, breaking-ties and our own proposed criteria called True Label Probability (TLP). QSS is tested on five common RS scene classification datasets: UCMerced, Optimal31, RSSCN7, AID and NWPU-RS45. It has outperformed all previous methods except for one case involving NWPU-RS45 dataset. For the 10%–90% train–test split, QSS achieved 94.51, 81.72, 89.89, 94.09 and 93.98, respectively, for the mentioned datasets. While for the 20%–80% train–test split, it achieved 97.09, 88.37, 94.01, 95.71 and 94.71, respectively, for the listed datasets. Thus, this work presents a novel data augmentation method that uses data with higher quality instead of randomly selected. The paper proves experimentally that the proposed method improves the classification accuracy of RS scene classification.

Data augmentation methods enrich datasets with augmented data to improve the performance of neural networks. Recently, automated data augmentation methods have emerged, which automatically design augmentation strategies. The existing work focuses on image classification and object detection, whereas we provide the first study on semantic image segmentation and introduce two new approaches: SmartAugment and SmartSamplingAugment. SmartAugment uses Bayesian Optimization to search a rich space of augmentation strategies and achieves new state-of-the-art performance in all semantic segmentation tasks we consider. SmartSamplingAugment, a simple parameter-free approach with a fixed augmentation strategy, competes in performance with the existing resource-intensive approaches and outperforms cheap state-of-the-art data augmentation methods. Furthermore, we analyze the impact, interaction, and importance of data augmentation hyperparameters and perform ablation studies, which confirm our design choices behind SmartAugment and SmartSamplingAugment. Lastly, we will provide our source code for reproducibility and to facilitate further research.

Pedestrian detection plays a crucial role in autonomous driving by identifying the position, size, orientation, and dynamic features of pedestrians in images or videos, assisting autonomous vehicles in making better decisions and controls. It’s worth noting that the performance of pedestrian detection models largely depends on the quality and diversity of available training data. Current datasets for autonomous driving have limitations in terms of diversity, scale, and quality. In recent years, numerous studies have proposed the use of data augmentation strategies to expand the coverage of datasets, aiming to maximize the utilization of existing training data. However, these data augmentation methods often overlook the diversity of data scenarios. To overcome this challenge, in this paper, we propose a more comprehensive method for data augmentation, based on image descriptions and diffusion models. This method aims to cover a wider range of scene variations, including different weather conditions and lighting situations. We have designed a classifier to select data samples for augmentation, followed by extracting visual features based on image captions and converting them into high-level semantic information as textual descriptions for the corresponding samples. Finally, we utilize diffusion models to generate new variants. Additionally, we have designed three modification patterns to increase diversity in aspects such as weather conditions, lighting, and pedestrian poses within the data. We conducted extensive experiments on the KITTI dataset and in real-world environments, demonstrating that our proposed method significantly enhances the performance of pedestrian detection models in complex scenarios. This meticulous consideration of data augmentation will notably enhance the applicability and robustness of pedestrian detection models in actual autonomous driving scenarios.

A Method of Data Augmentation for Classifying Road Damage Considering Influence on Classification Accuracy

Low-resource languages often face the problem of insufficient data, which leads to poor quality in machine translation. One approach to address this issue is data augmentation. Data augmentation involves creating new data by transforming existing data through methods such as flipping, cropping, rotating, and adding noise. Traditionally, pseudo-parallel corpora are generated by randomly replacing words in low-resource language machine translation. However, this method can introduce ambiguity, as the same word may have different meanings in different contexts. This study proposes a new approach for low-resource language machine translation, which involves generating pseudo-parallel corpora by replacing phrases. The performance of this approach is compared with other data augmentation methods, and it is observed that combining it with other data augmentation methods further improves performance. To enhance the robustness of the model, R-Drop regularization is also used. R-Drop is an effective method for improving the quality of machine translation. The proposed method was tested on Chinese–Kazakh (Arabic script) translation tasks, resulting in performance improvements of 4.99 and 7.7 for Chinese-to-Kazakh and Kazakh-to-Chinese translations, respectively. By combining the generation of pseudo-parallel corpora through phrase replacement with the application of R-Drop regularization, there is a significant advancement in machine translation performance for low-resource languages.

The success of training deep Convolutional Neural Networks (CNNs) heavily depends on a significant amount of labelled data. Recent research has found that neural style transfer algorithms can apply the artistic style of one image to another image without changing the latter's high-level semantic content, which makes it feasible to employ neural style transfer as a data augmentation method to add more variation to the training dataset. The contribution of this paper is a thorough evaluation of the effectiveness of the neural style transfer as a data augmentation method for image classification tasks. We explore the state-of-the-art neural style transfer algorithms and apply them as a data augmentation method on Caltech 101 and Caltech 256 dataset, where we found around 2% improvement from 83% to 85% of the image classification accuracy with VGG16, compared with traditional data augmentation strategies. We also combine this new method with conventional data augmentation approaches to further improve the performance of image classification. This work shows the potential of neural style transfer in computer vision field, such as helping us to reduce the difficulty of collecting sufficient labelled data and improve the performance of generic image-based deep learning algorithms.

Data augmentation has been broadly applied in training deep-learning models to increase the diversity of data. This study ingestigates the effectiveness of different data augmentation methods for deep-learningbased human intention prediction when only limited training data is available. A human participant pitches a ball to nine potential targets in our experiment. We expect to predict which target the participant pitches the ball to. Firstly, the effectiveness of 10 data augmentation groups is evaluated on a single-participant data set using RGB images. Secondly, the best data augmentation method (i.e., random cropping) on the single-participant data set is further evaluated on a multi-participant data set to assess its generalization ability. Finally, the effectiveness of random cropping on fusion data of RGB images and optical flow is evaluated on both single- and multi-participant data sets. Experiment results show that: 1) Data augmentation methods that crop or deform images can improve the prediction performance; 2) Random cropping can be generalized to the multi-participant data set (prediction accuracy is improved from 50% to 57.4%); and 3) Random cropping with fusion data of RGB images and optical flow can further improve the prediction accuracy from 57.4% to 63.9% on the multi-participant data set.