Trustworthy Transfer Learning: A Survey

A novel framework of knowledge transfer system for construction projects based on knowledge graph and transfer learning

Transfer learning aims to extract the knowledge from a label-rich source domain to enhance the predictive model of a target domain. Previous methods achieve knowledge transfer by detecting a shared low-dimensional feature representation from source domain to target domain. Along this line, many algorithms, e.g., dual transfer learning (DTL), triplex transfer learning (TRi-TL) etc., have been proposed and widely used for text classification. However, we argue that it is difficult for models to distinguish exactly the common concepts or identical concepts across different domains through the existing algorithms, even though source and target domains are related but different. So we propose to use the similar feature clusters as knowledge transfer, that is, we only guarantee the approximate similarity of common word clusters across different domains, rather than the exactly same. Based on the above assumption, the derived association matrices between word clusters and document classes should be slightly different to account for the word clusters variations. To take the above assumptions into account, we propose a novel Nonnegative Matrix Tri-Factorization based transfer learning by linking similar feature clusters (LSF-TL) for sentiment classification, in which an approximate constraint between similar word clusters matrices is added to allow differences while keeping the knowledge transferring function. Besides, LSF-TL also provides the same approximate constraint for the derived clusters association matrices. Then we employ an iterative updating algorithm with sound theoretical proof to find the local optimal solution. Last, we evaluate our method by conducting extensive experiments on Amazon product reviews. The results show that our approach achieves better classification accuracy than the state-of-the-art methods for both Cross-lingual sentiment classification(CLSC) and Cross-lingual cross-domain sentiment classification(CLCDSC) tasks.

Timely and accurate crop mapping is crucial for providing essential data support for agricultural production management. Reliable ground truth samples form the foundation for crop mapping using remote sensing imagery, a task that presents significant challenges in regions with limited sample availability. To address this issue, this study evaluates instance-based transfer learning methods, using the Hexi Corridor as a case study to explore crop mapping strategies in areas with scarce samples. High-confidence pixels from the United States Cropland Data Layer (CDL), along with high-density time series data derived from Sentinel-1, Sentinel-2, and Landsat-8 satellite imagery, as well as key vegetation indices, were selected as training samples for the source domain. Various algorithms, including Random Forest (RF), Extreme Gradient Boosting (XGBoost), and TrAdaBoost, were employed to transfer knowledge from the source domain to the target domain for crop type mapping. The results demonstrated that during the transfer learning process using only source domain data—without utilizing any target domain data—the overall classification accuracy reached 73.88%, with optimal accuracies for maize and alfalfa at 88.97% and 85.23%, respectively. As target domain data were gradually incorporated, the total accuracy for all models ranged from 0.77 to 0.92, with F1-scores ranging from 0.76 to 0.92, showing a consistent improvement in model performance. This study highlights the feasibility of employing transfer learning for crop mapping in the Hexi Corridor, demonstrating its potential to reduce labeling costs for target domain samples and providing a valuable reference for crop mapping in regions with limited sample availability.

The primary purpose of this investigation was to assess the level of human immunodeficiency virus/acquired immune deficiency syndrome (HIV/AIDS) knowledge of high school adolescents in Kenya. As a subsidiary, objective of the investigation was to determine whether there was transfer of knowledge between boys and girls regarding their sexual behaviour. The results showed that adolescent high school students have a high level of knowledge of HIV/AIDS and that such knowledge transfers to the sexual relationship between boys and girls. The sample of the study consisted of 157 participants attending two high schools in Nairobi. Participants were administered a questionnaire based on HIV/AIDS transmission, infection and prevention. The method of data analysis used were frequencies, percentage and the chi-squared test. The study concluded that, even though the level of HIV/AIDS knowledge was high, there were identified misconceptions which would further justify the continued promotion of public education regarding HIV/AIDS. Unlike similar studies, this study found a correlation between knowledge and the transfer of this knowledge to change of behaviour. This study shows that public education does produce dividends in the prevention of HIV/AIDS transmission and the spread of the epidemic worldwide. Key words: Avoidance, epidemic, infection, invulnerability, learning transfer, misconception, physical contact, transmission, human immunodeficiency virus/acquired immune deficiency syndrome (HIV/AIDS).

One of the challenges faced by machine learning in human activity recognition systems is the different distributions of the training and test samples. Transfer learning constitutes a solution to this problem. On the other hand, to perform transfer learning, it is necessary to have access to the original dataset. However, access to the dataset to implement the transfer learning algorithms results in a privacy breach. To deal with this challenge, this paper presents semi-supervised and unsupervised scenarios for privacy-preserving transfer learning in centralized and distributed manner. In the proposed distributed algorithms, it is not necessary to share the original data among the clients to implement the transfer learning algorithms. Instead, the transfer learning process can be fulfilled without having the original datasets. PPSETR and PPUSTR algorithms transfer the knowledge while preserving the privacy of the datasets on the client side. In contrast, PPDSETR and PPDUSTR algorithms provide the privacy protection of the distributed data on both the client and server sides. The proposed semi-supervised algorithms reduce the recognition error rate by 20.58% and the unsupervised algorithms decrease the recognition error rate by about 15.97% while these algorithms considerably preserve the privacy.

Remote sensing data represent one of the most important sources for automized yield prediction. High temporal and spatial resolution, historical record availability, reliability, and low cost are key factors in predicting yields around the world. Yield prediction as a machine learning task is challenging, as reliable ground truth data are difficult to obtain, especially since new data points can only be acquired once a year during harvest. Factors that influence annual yields are plentiful, and data acquisition can be expensive, as crop-related data often need to be captured by experts or specialized sensors. A solution to both problems can be provided by deep transfer learning based on remote sensing data. Satellite images are free of charge, and transfer learning allows recognition of yield-related patterns within countries where data are plentiful and transfers the knowledge to other domains, thus limiting the number of ground truth observations needed. Within this study, we examine the use of transfer learning for yield prediction, where the data preprocessing towards histograms is unique. We present a deep transfer learning framework for yield prediction and demonstrate its successful application to transfer knowledge gained from US soybean yield prediction to soybean yield prediction within Argentina. We perform a temporal alignment of the two domains and improve transfer learning by applying several transfer learning techniques, such as -SP, BSS, and layer freezing, to overcome catastrophic forgetting and negative transfer problems. Lastly, we exploit spatio-temporal patterns within the data by applying a Gaussian process. We are able to improve the performance of soybean yield prediction in Argentina by a total of 19% in terms of RMSE and 39% in terms of compared to predictions without transfer learning and Gaussian processes. This proof of concept for advanced transfer learning techniques for yield prediction and remote sensing data in the form of histograms can enable successful yield prediction, especially in emerging and developing countries, where reliable data are usually limited.

Graph convolutional networks (GCNs) have achieved impressive results in many medical scenarios involving graph node classification tasks. However, there are difficulties in transfer learning for graph representation learning and graph network models. Most GNNs work only in a single domain and cannot transfer the learned knowledge to other domains. Coronary Heart Disease (CHD) is a high-mortality disease, and there are non-public and significant differences in CHD datasets for current research, which makes it difficult to perform unified transfer learning. Therefore, in this paper, we propose a novel adversarial domain-adaptive multichannel graph convolutional network (DAMGCN) that can perform graph transfer learning on cross-domain tasks to achieve cross-domain medical knowledge transfer on different CHD datasets. First, we use a two-channel GCN model for feature aggregation using local consistency and global consistency. Then, a uniform node representation is generated for different graphs using an attention mechanism. Finally, we provide a domain adversarial module to decrease the discrepancies between the source and target domain classifiers and optimize the three loss functions in order to accomplish source and target domain knowledge transfer. The experimental findings demonstrate that our model performs best on three CHD datasets, and its performance is greatly enhanced by graph transfer learning.

In the domain of credit risk assessment lenders may have limited or no data on the historical lending outcomes of credit applicants. Typically this disproportionately affects Micro, Small, and Medium Enterprises (MSMEs), for which credit may be restricted or too costly, due to the difficulty of predicting the Probability of Default (PD). However, if data from other related credit risk domains is available Transfer Learning may be applied to successfully train models, e.g., from the credit card lending and debt consolidation (CD) domains to predict in the small business lending domain. In this article, we report successful results from an approach using transfer learning to predict the probability of default based on the novel concept of Progressive Shift Contribution (PSC) from source to target domain. Toward real-world application by lenders of this approach, we further address two key questions. The first is to explain transfer learning models, and the second is to adjust features when the source and target domains differ. To address the first question, we apply Shapley values to investigate how and why transfer learning improves model accuracy, and also propose and test a domain adaptation approach to address the second. These results show that adaptation improves model accuracy in addition to the improvement from transfer learning. We extend this by proposing and testing a combined strategy of feature selection and adaptation to convert values of source domain features to better approximate values of target domain features. Our approach includes a strategy to choose features for adaptation and an algorithm to adapt the values of these features. In this setting, transfer learning appears to improve model accuracy by increasing the contribution of less predictive features. Although the percentage improvements are small, such improvements in real world lending could be of significant economic importance.

Argumentation research often addresses the nature of argumentation as a complex set of procedures that can aid learning of complex scientific knowledge and practices. Thus, studies might look at transfer of knowledge and processes to specific science domains or to other types of procedural knowledge, such as teacher implementation in the elementary classroom or attention to argumentation procedures. This study explored the extent of transfer of the procedures and knowledge of argumentation to preservice teachers’ lesson plans. Participants were four African American female preservice graduate students enrolled in an elementary science methods course. The students engaged in argumentation exercises in science from segments of the IDEAS project [Osborne, J., Erduran, S., & Simon, S. (2004). Ideas, evidence and argument in science (IDEAS) project. University of London Press], then participated in a socioscientific crime scene investigation. Students’ capstone unit plans were analysed to explore the extent of transfer of argumentation knowledge and procedures to students’ lessons. Schema theory as an analytic framework shows the nature of psychological and architectural features consistent with students’ abstractions and attention to argumentation. The research has implications for understanding the importance of considering transfer of learning in relation to students’ prior knowledge, especially when it comes to understanding the connections that African American students make to science pedagogy and content.

O objetivo deste artigo é analisar os múltiplos papéis da socialização na transferência de conhecimento tácito entre uma empresa multinacional (EMN) brasileira e a sua subsidiária localizada em um país menos desenvolvido. A despeito dos avanços da literatura sobre transferência de conhecimento em EMNs e sobre o papel da socialização nesse processo, dois aspectos – um conceitual e outro empírico – têm sido negligenciados: (i) a inefabilidade do conhecimento tácito e (ii) a transferência de conhecimento para países menos desenvolvidos caracterizados por baixa capacidade absortiva. Realizou-se o estudo de caso do Cine Group, empresa brasileira do setor de audiovisual que, em 2009, instalou uma subsidiária em Maputo (Moçambique). Os resultados evidenciam que, em razão da inefabilidade do conhecimento tácito e do contexto de baixa capacidade absortiva da subsidiária, a socialização pode ter papéis distintos, que vão da transferência de conhecimento à mudança comportamental, passando pela aprendizagem conjunta.

Backdoor attacks are a serious security threat to open-source and outsourced development of computational systems based on deep neural networks (DNNs). In particular, the transferability of backdoors is remarkable; that is, they can remain effective after transfer learning is performed. Given that transfer learning from natural images is widely used in real-world applications, the question of whether backdoors can be transferred from neural models pretrained on natural images involves considerable security implications. However, this topic has not been evaluated rigorously in prior studies. Hence, in this study, we configured backdoors in 10 representative DNN models pretrained on a natural image dataset, and then fine-tuned the backdoored models via transfer learning for four real-world applications, including pneumonia classification from chest X-ray images, emergency response monitoring from aerial images, facial recognition, and age classification from images of faces. Our experimental results show that the backdoors generally remained effective after transfer learning from natural images, except for small DNN models. Moreover, the backdoors were difficult to detect using a common method. Our findings indicate that backdoor attacks can exhibit remarkable transferability in more realistic transfer learning processes, and highlight the need for the development of more advanced security countermeasures in developing systems using DNN models for sensitive or mission-critical applications.

We introduce the novel problem of inter-robot transfer learning for perceptual classification of objects, where multiple heterogeneous robots communicate and transfer learned object models consisting of a fusion of multiple object properties. Unlike traditional transfer learning, there can be severe differences in the data distributions, resulting from differences in sensing, sensory processing, or even representations, that each robot uses to learn. Furthermore, only some properties may overlap between the two robots. We show that in such cases, the abstraction of raw sensory data into an intermediate representation can be used not only to aid learning, but also the transfer of knowledge. Further, we utilize statistical metrics, learned during an interactive process where the robots jointly explore the environment, to determine which underlying properties are shared between the robots. We demonstrate results in a visual classification task where objects are represented via a combination of properties derived from different modalities: color, texture, shape, and size. Using our methods, two heterogeneous robots utilizing different sensors and representations are able to successfully transfer support vector machine (SVM) classifiers among each other, resulting in speedups during learning.

This paper present a systematic overview of barriers and solutions of e learning in knowledge management (KM) and knowledge transfer (KT) with more focus on organizations. The review paper also discusses KT in organizational settings and KT in the field of e learning. Here, an e-learning initiative shows adaptive solutions to overcome knowledge transfer barriers. The paper concludes that the flow of knowledge depends on people and the social environment they operate in. Therefore E-Learning environments support knowledge transfer not only by helping learners to make sense of content, but also by enabling communication, both among learners and between learners and tutors.

This paper proposes a transfer learning method for an object detection model using a genetic algorithm to solve the difficulty of the conventional transfer learning of deep learning-based object detection models. The genetic algorithm of the proposed method can select the re-learning layers automatically in the transfer learning process instead of a trial-and-error selection of the conventional method. Transfer learning was performed using the EfficientDet-d0 model pre-trained on the COCO dataset and the Global Wheat Head Detection (GWHD) dataset, and experiments were conducted to compare fine-tuning and the proposed method. Using the training data and the validation data of the GWHD, we compare the mean average precision (mAP) of the models trained by the conventional and the proposed methods, respectively, on the test data of the GWHD. It is confirmed that the model trained by the proposed method has higher performance than the model trained by the conventional method. The average of mAP of the proposed method, which automatically selects the re-learning layer (≈0.603), is higher than the average of mAP of the conventional method (≈0.594). Furthermore, the standard deviation of results obtained by the proposed method is smaller than that of the conventional method, and it shows the stable learning process of the proposed method.

Transfer learning refers to the transfer of knowledge or information from a relevant source task to a target task. However, most existing works assume both tasks are sampled from a stationary task distribution, thereby leading to the sub-optimal performance for dynamic tasks drawn from a non-stationary task distribution in real scenarios. To bridge this gap, in this paper, we study a more realistic and challenging transfer learning setting with dynamic tasks, i.e., source and target tasks are continuously evolving over time. We theoretically show that the expected error on the dynamic target task can be tightly bounded in terms of source knowledge and consecutive distribution discrepancy across tasks. This result motivates us to propose a generic meta-learning framework L2E for modeling the knowledge transferability on dynamic tasks. It is centered around a task-guided meta-learning problem with a group of meta-pairs of tasks, based on which we are able to learn the prior model initialization for fast adaptation on the newest target task. L2E enjoys the following properties: (1) effective knowledge transferability across dynamic tasks; (2) fast adaptation to the new target task; (3) mitigation of catastrophic forgetting on historical target tasks; and (4) flexibility in incorporating any existing static transfer learning algorithms. Extensive experiments on various image data sets demonstrate the effectiveness of the proposed L2E framework.