Evaluation of Transfer Learning Algorithms Using Different Base Learners

The lack of reliable and up-to-date data in developing countries is a major obstacle to sustainable development. In Morocco, where groundwater withdrawals by farmers are very intensive and informal, maps describing and monitoring the extension of irrigated areas are scarce and labor-intensive to obtain. In this paper a novel transfer learning algorithm is proposed to map irrigated areas at different stages of an agricultural cycle from Landsat 8 images. The results obtained displays satisfactory performance over traditional machine learning algorithms. On a small dataset, we initially tested three well known deep learning architectures (SegNet, DenseNet and Unet). The results obtained were not satisfactory. So, to get high performance, we rely on a transfer learning architecture combining UNet with ResNet50 backbone (trained on 2012 ILSVRC ImageNet dataset) as a baseline after a phase where different configurations were tested. In the first part of this study, we compared the use of three optimization methods: Adam and two variants of Stochastic Gradient Descent (SGD) associated with two techniques (Cyclical Learning Rate and Warm Restart) to find the optimal learning rate and then test the impact of data augmentation on the overall accuracies. Data augmentation had improved the overall accuracy for the three methods. Adam based method from 94% to 97% with mean IoU of 0,79 (for all land cover classes) and 0,86 for irrigated areas class. For SGD based methods, the overall accuracy had increased from 91% to 94% with mean IoU of 0,75 (for all land cover classes) and 0,82 for irrigated areas class. As we are interested in having irrigated areas maps at different key periods of the agricultural cycle, we also explored, in the second part of this study, the temporal generalization of the best model.

This paper presents a high-spatial-resolution dataset with remote sensing images of the Brazilian Cerrado for land use and land cover classification. The Biome Cerrado Dataset (CerraData) is a large database created from 150 scenes of the CBERS-4A satellite. Images were created by merging the near-infrared, green, and blue bands. Moreover, pan-sharpening was performed between all the scenes and their respective panchromatic bands, resulting in a final spatial resolution of two meters. A total of 2.5 million tiles of 256x256 pixels were derived from these scenes. From this total, 50 thousand tiles were labeled. We also conducted a few-shot learning experiment considering a training set with only 100 samples, 11 deep neural networks (DNNs), and two traditional machine learning (ML) algorithms, i.e., support vector machine (SVM) and random forest (RF). Results show that the DNN DenseNet-161 was the best model but its performance can be improved if it is used only as a feature extractor, leaving the classification task for the traditional ML algorithms. However, by decreasing the size of the training set, smarter approaches are needed. The labeled subset of CerraData as well as the source code we developed to support this study are available on-line: https://github.com/ai41uc/CerraData-code-data.

An exhaustive measurement of re-sampling detection in lossy compressed images using deep learning approach

Feature mining for encrypted malicious traffic detection with deep learning and other machine learning algorithms

Deep learning is a subfield of machine learning and deep neural architectures can extract high level features automatically without handcraft feature engineering unlike traditional machine learning algorithms. In this paper, we propose a method, which combines feature extraction layers of a convolutional neural network with traditional machine learning algorithms, such as, support vector machine, gradient boosting machines, and random forest. All of the proposed hybrid models and the above mentioned machine learning algorithms are trained on three different datasets: MNIST, Fashion-MNIST, and CIFAR-10. Results show that the proposed hybrid models are more successful than traditional models while they are being trained from raw pixel values. In this study, we empower traditional machine learning algorithms for classification using feature extraction ability of deep neural network architectures and we are inspired by transfer learning methodology to this.

The existing identification methods of transmission tower safety state in goafs have some defects, such as large sample data scale, and the identification accuracy is significantly affected by some time-varying parameters. To solve this problem, this paper proposes an identification method based on improved fuzzy theory and transfer learning. Firstly, the image feature data of transmission tower in goafs is extracted according to the YOLOv3 algorithm which belongs to one-stage target detection algorithm, and the image feature data is preprocessed by fuzzy algorithm. Secondly, a deep convolution neural network framework is proposed for security state recognition, and a multi-stage migration learning algorithm is used to complete network training. Finally, the results show that compared with the traditional machine learning algorithm without migration learning training, the proposed algorithm improves the accuracy and precision by more than 4.5% and 3.8%, respectively, which verifies the effectiveness and superiority of the proposed algorithm.

Recent advancements in computer technology have precipitated a shift towards virtual environments, accelerated by the COVID-19 pandemic. Cybercriminals have capitalized on this trend, transitioning their activities to exploit vulnerabilities in cyberspace. Malicious software (malware) has emerged as a preferred tool for launching cyber-attacks, continually evolving with sophisticated obfuscation and packing techniques to evade detection. Traditional machine learning (ML) algorithms, once effective in identifying malware, are now struggling to keep pace with these advancements. In response, deep learning (DL) algorithms offer a promising solution, leveraging their ability to discern intricate patterns and correlations within data. This study proposes a novel hybrid deep-learning-based architecture, integrating two pre-trained network models to enhance classification accuracy. Through extensive evaluation on datasets including Malimg, Microsoft BIG 2015, and Malevis, the proposed method demonstrates significant improvements in accuracy, outperforming existing ML-based malware detection methods in the literature. Specifically, the proposed method achieves an impressive accuracy of 97.78% on the Malimg dataset, underscoring its effectiveness in combating sophisticated malware variants. Keywords — Malware, malware classification, malware detection, malware variants, deep neural networks, transfer learning, deep learning.

Recently deep learning has shown a improved performance than machine learning in many of the areas like pattern recognition, image classification computer vision, video segmentation and many more. But out of all these areas, disease classification is one of the major area in which deep learning has shown a remarkable performance than the traditional machine learning algorithms especially in the area of image recognition. Machine learning algorithms are not enough capable to handle the image so in this work we will apply the deep learning approach on the Alzheimer's disease dataset for performing the early detection and classification of the disease and this has done through using neuroimaging data. Previous work done in this area was based on traditional machine learning algorithm and they have used stacked auto encoder (SAC) for dimensionality reduction and they have achieved a classification accuracy of 83.7% during the prediction from initial symptom to final development of Alzheimer's disease. The deep learning algorithm ResNet which is implemented in this paper has shown a classification accuracy of 93% and this is also achieved without applying any dimensionality reduction approach and this has been considered as the best predictive rate on the neuroimaging data till now. The applied ResNet is the improved ResNet and the comparison of both the Resnet models are shown in this work. This deep learning application will also be useful for other types of disease classification like cancer, diabetics, etc. Keyword : ResNet, mild cognitive impairments (MCI), ADNI, ReLU, Residual Block, Convolutions.

Transfer learning, which aims to reuse knowledge in different domains, has achieved great success in many scenarios via minimizing domain discrepancy and enhancing feature discriminability. However, there are seldom practical determination methods for measuring the transferability among domains. In this paper, we bring forward a novel meta-transfer feature method (MetaTrans) for this problem. MetaTrans is used to train a model to predict performance improvement ratio from historical transfer learning experiences, and can consider both the Transferability between tasks and the Discriminability emphasized on targets. We apply this method to both shallow and deep transfer learning algorithms, providing a detail explanation for the success of specific transfer learning algorithms. From experimental studies, we find that different transfer learning algorithms have varying dominant factor deciding their success, so we propose a multi-task learning framework which can learn both common and specific experience from historical transfer learning results. The empirical investigations reveal that the knowledge obtained from historical experience can facilitate future transfer learning tasks.

Traditionally, the classification of seed defects mainly relies on the characteristics of color, shape, and texture. This method requires repeated extraction of a large amount of feature information, which is not efficiently used in detection. In recent years, deep learning has performed well in the field of image recognition. We introduced convolutional neural networks (CNNs) and transfer learning into the quality classification of seeds and compared them with traditional machine learning algorithms. Experiments showed that deep learning algorithm was significantly better than the machine learning algorithm with an accuracy of 95% (GoogLeNet) vs. 79.2% (SURF+SVM). We used three classifiers in GoogLeNet to demonstrate that network accuracy increases as the depth of the network increases. We used the visualization technology to obtain the feature map of each layer of the network in CNNs and used the heat map to represent the probability distribution of the inference results. As an end-to-end network, CNNs can be easily applied for automated seed manufacturing.

Dictionary learning is an efficient knowledge representation method that can learn the essential features of data. Traditional dictionary learning methods are difficult to obtain nonlinear information when processing large-scale and high-dimensional datasets. While most dictionary learning algorithms are based on the assumption that the training data and test data have the same feature distribution, which is not always true in practical applications. To address the above problems, we propose the Kernel Fisher Dictionary Transfer Learning (KFDTL) algorithm. First, we map each sample to high-dimensional space through kernel mapping and use any dictionary learning algorithm to learn the essential features. Then, the feature-based transfer learning method is performed to predict the labels of the target samples. This method includes three main contributions: (1) KFDTL constructs a discriminative Fisher embedding model to make the same class samples have similar coding coefficients; (2) Based on the relationship between profiles and atoms, KFDTL constructs an adaptive model that adapts source domain samples to target domain samples; (3) The kernel method is used to efficiently solve nonlinear problems. Experiments on a large number of public image datasets have proved the effectiveness of the proposed method. The source code of the proposed method is available at https://github.com/zzfan3/KFDTL .

Abstract: Bridge vortex induced vibration (VIV) is a resonance phenomenon caused by the periodic shedding of vortices generated by natural wind passing through bridges. Bridge VIVs will not only cause fatigue damage to the structures but also affect driving safety for the passing vehicles. With the popularization of structural health monitoring (SHM) systems, machine learning technology is widely used in the field of vortex induced vibration identification for long-span bridges, due to its intelligence, real-time performance, and sensitivity to data. However, although traditional machine learning algorithms can identify the VIV based on the response data history of long-span bridges collected by SHM systems, they are difficult to apply to bridges which do not have historical vortex induced vibration data. Therefore, this paper proposes an adaptive transfer learning method for identifying VIV in the main girder of long-span suspension bridges. The proposed method can identify VIV without VIV history data of the target bridge. Results show that it can well identify VIVs at the earlier stage based on the SHM datasets of two long-span suspension bridges, verifying its effectiveness and generalization ability.

Kiwifruit has a high decay rate, in part because quality changes during storage cannot be easily monitored in real time. In order to better monitor the shelf life of kiwifruit and understand the quality changing process during storage, internal quality indexes such as hardness, respiratory intensity and TSS(Total Soluble Solid) were considered into the prediction models. The prediction models were constructed based on BPNN (Back Propagation Neural Network), Random Forest (RF) and XGBoost (eXtreme Gradient Boosting) respectively. And transfer learning algorithm was used to construct the quality prediction models with BPNN, RF, and XGBoost algorithms as the base learner. In the experiments, sample data were augmented by adding Gaussian noise, which effectively prevented the model from over-fitting. The experimental results showed that the prediction accuracy of each index based on transfer learning was better than that of individual BPNN, RF and XGBoost. Moreover, the average prediction accuracy of the models based on transfer learning was 96.2%, and that of respiratory intensity was as high as 99.4%. Therefore transfer learning can be used to effectively analyze and predict changes of kiwifruit quality indexes during storage.

Protein lysine acetylation is an important post- translational modification (PTM) that takes an important part in many cellular processes, such as metabolism, differentiation and replication. However, the current lysine acetylation site identification mainly adopts experimental detection methods which are time-consuming and inefficient. Therefore, several shallow machine learning algorithms were applied for protein lysine acetylation site prediction. However, traditional machine learning algorithms are not as accurate as deep learning algorithms. As a special kind of recurrent neural network (RNN), long short-term memory (LSTM) can not only process sequence data but also solve the vanishing gradient problem. Therefore, this paper proposes a method based on LSTM to predict lysine acetylation sites. In addition, two traditional machine learning algorithms k-nearest neighbors (KNN) and support vector machine (SVM) were adopted for training as comparison. Through experimental verification, the prediction method based on LSTM is superior to the other two based on traditional machine learning algorithms, and the final prediction accuracy of LSTM model can reach up to 81.52%.

One of the most important threats to today’s civilization is terrorism. Terrorism not only disturbs the law and order situations in a society but also affects the quality of lives of humans and makes them suppressed physically and emotionally and deprives them of enjoying life. The more the civilizations have advanced, the more the people are working towards exploring different mechanisms to protect the mankind from terrorism. Different techniques have been used as counterterrorism to protect the lives of individuals in society and to improve the quality of life in general. Machine learning methods have been recently explored to develop techniques for counterterrorism based on artificial intelligence (AI). Since deep learning has recently gained more popularity in machine learning domain, in this paper, these techniques are explored to understand the behavior of terrorist activities. Five different models based on deep neural network (DNN) are created to understand the behavior of terrorist activities such as is the attack going to be successful or not? Or whether the attack is going to be suicide or not? Or what type of weapon is going to be used in the attack? Or what type of attack is going to be carried out? Or what region is going to be attacked? The models are implemented in single-layer neural network (NN), five-layer DNN, and three traditional machine learning algorithms, i.e., logistic regression, SVM, and Naïve Bayes. The performance of the DNN is compared with NN and the three machine learning algorithms, and it is demonstrated that the performance in DNN is more than 95% in terms of accuracy, precision, recall, and F1-Score, while ANN and traditional machine learning algorithms have achieved a maximum of 83% accuracy. This concludes that DNN is a suitable model to be used for predicting the behavior of terrorist activities. Our experiments also demonstrate that the dataset for terrorist activities is big data; therefore, a DNN is a suitable model to process big data and understand the underlying patterns in the dataset.