Progressive transfer learning Unifies Multi-Corpus EEG for robust and scalable Imagined-Speech decoding

Active Learning is a machine learning and data mining technique that selects the most informative samples for labeling and uses them as training data; it is especially useful when there are large amount of unlabeled data and labeling them is expensive. Recently, batch-mode active learning, where a set of samples are selected concurrently for labeling, based on their collective merit, has attracted a lot of attention. The objective of batch-mode active learning is to select a set of informative samples so that a classifier learned on these samples has good generalization performance on the unlabeled data. Most of the existing batch-mode active learning methodologies try to achieve this by selecting samples based on certain criteria. In this article we propose a novel criterion which achieves good generalization performance of a classifier by specifically selecting a set of query samples that minimize the difference in distribution between the labeled and the unlabeled data, after annotation. We explicitly measure this difference based on all candidate subsets of the unlabeled data and select the best subset. The proposed objective is an NP-hard integer programming optimization problem. We provide two optimization techniques to solve this problem. In the first one, the problem is transformed into a convex quadratic programming problem and in the second method the problem is transformed into a linear programming problem. Our empirical studies using publicly available UCI datasets and two biomedical image databases demonstrate the effectiveness of the proposed approach in comparison with the state-of-the-art batch-mode active learning methods. We also present two extensions of the proposed approach, which incorporate uncertainty of the predicted labels of the unlabeled data and transfer learning in the proposed formulation. In addition, we present a joint optimization framework for performing both transfer and active learning simultaneously unlike the existing approaches of learning in two separate stages, that is, typically, transfer learning followed by active learning. We specifically minimize a common objective of reducing distribution difference between the domain adapted source, the queried and labeled samples and the rest of the unlabeled target domain data. Our empirical studies on two biomedical image databases and on a publicly available 20 Newsgroups dataset show that incorporation of uncertainty information and transfer learning further improves the performance of the proposed active learning based classifier. Our empirical studies also show that the proposed transfer-active method based on the joint optimization framework performs significantly better than a framework which implements transfer and active learning in two separate stages.

Dynamic multiobjective optimization problems (DMOPs) aim to optimize multiple (often conflicting) objectives that are changing over time. Recently, there are a number of promising algorithms proposed based on transfer learning methods to solve DMOPs. However, it is very challenging to reduce the negative effect in transfer learning and find more effective transferred solutions. To fill this research gap, this article proposes a clustering-based transfer (CBT) learning method to solve DMOPs. When the environment changes, two novel operations (clustering-based selection (CBS) and CBT) are used to guide knowledge transfer. Specifically, CBS aims to find a population with nondominated solutions and dominated solutions as the training data for the new environment. Then, CBT further collects the previous Pareto-optimal solutions and some noise solutions as the training data for the previous environment. Two training data sets from different environments are, respectively, divided into multiple clusters and transfer learning is conducted on two similar clusters with high probability to reduce the negative effect, which can train an accurate prediction model to identify the promising solutions for the new environment. Empirical studies have been conducted on 14 benchmark DMOPs and one real-life path planning problem of unmanned air/ground vehicles, which validate the effectiveness of our proposed method. Especially, our method can significantly reduce negative transfer on 12 out of 14 cases when compared with direct transfer learning.

The aim of this study was to investigate the transfer of learning of professional competences from vocational colleges to the workplace context in vocational education. Concretely, the study examined the relations between the professional competences learned at school and the use and further development of those competences at the workplace during the practice module in vocational education programmes in Spain. The study analysed individual, educational design and work environment factors that facilitate or hinder transfer of learning. The study took a multidimensional approach to professional competences by observing four dimensions: hard-specific, hard-generic, interpersonal and intrapersonal. The data collection drew from a questionnaire administered to students of vocational education programmes (N = 379) after they had completed the practice module scheme. A moderation analysis was applied to observe the effect of individual, educational design and workplace factors on the relation between the competences learned at school and those used at the workplace. The findings showed that compulsory placement, school-work alignment, work-based supervisor, access to resources and task complexity are significant moderators between the competences learned at school and those used at the workplace. The study highlights the coherence between the school and workplaces and work environment characteristics as factors conducive to transfer of learning.

Deep neural networks (DNNs), e.g., convolutional neural network (CNN), are able to learn effective features from wafer maps for dimensional reduction and feature extraction. However, very large image data are needed to train DNNs to obtain high generalization performance. It is still a difficult task due to the lack of sufficient labeled images with various defects. This article proposes a semisupervised deep transfer learning algorithm called joint feature and label adversarial network (JFLAN). JFLAN uses CNNs to extract transferable features of wafer maps and then introduces a multilayer domain adaptation and pseudolabel learning block based on the generative adversarial network (GAN). This effectively reduces the distribution discrepancy and the among-class distance of the transferable features. Finally, JFLAN transfers knowledge from wafer image source data collected offline and then achieves the goal of significantly improved accuracy of wafer defect recognition and realizes online adaptive defect recognition. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The defect recognition on wafer maps plays a key role to recognize fault sources of the semiconductor manufacturing processes. Transfer learning is able to use the existing labeled data to assist in the classification of unlabeled data and is very effective to solve the problem of small samples and nonstationary generalization errors. In particular, the infusion of adversarial learning in transfer learning will provide a new idea for deep feature learning. This article provides a novel method based on transfer learning to implement wafer map defect recognition (WMDR) to quickly identify defect root causes for yield enhancement. This article provides a novel way for quality control of semiconductor manufacturing processes based on transfer and adversarial learning.

Deep learning has emerged as a powerful tool in various domains, revolutionising machine learning research. However, one persistent challenge is the scarcity of labelled training data, which hampers the performance and generalisation of deep learning models. To address this limitation, researchers have developed innovative methods to overcome data scarcity and enhance deep model learning capabilities. Two prevalent techniques that have gained significant attention are transfer learning and self-supervised learning. Transfer learning leverages knowledge learned from pre-training on a large-scale dataset, such as ImageNet, and applies it to a target task with limited labelled data. This approach allows models to benefit from the learned representations and effectively transfer knowledge to new tasks, resulting in improved learning performance and generalisation. On the other hand, self-supervised learning focuses on training models using pretext tasks that do not require manual annotation, allowing them to learn valuable representations from large amounts of unlabelled data. These learned representations can then be fine-tuned for downstream tasks, mitigating the need for extensive labelled data. In recent years, transfer and self-supervised learning have found applications in various fields, including medical image processing, video recognition, and natural language processing. These approaches have demonstrated remarkable achievements, enabling breakthroughs in areas such as disease diagnosis, object recognition, and language understanding. However, while these methods offer numerous advantages, they also have limitations. For example, transfer learning may face domain mismatch issues between the pre-training and target domains, while self-supervised learning requires careful design of pretext tasks to ensure meaningful representations. This review paper explores the recent applications of these pre-training methods in various fields within the past three years. It delves into the advantages and limitations of each approach, assesses the performance of models employing these techniques, and identifies potential directions for future research. By providing a comprehensive review of current pre-training methods, this article offers guidance for selecting the best technique for specific deep learning applications to address the data scarcity issue.

Granular transfer learning using type-2 fuzzy HMM for text sequence recognition

A new transfer learning strategy is proposed for classification in this work, based on fully connected neural networks. The transfer learning process consists in a training phase of the neural network on a source dataset. Then, the last two layers are retrained using a different small target dataset. Clustering techniques are also applied in order to determine the most suitable data to be used as target. A preliminary study has been conducted to train and test the transfer learning proposal on the classification problem of phenology forecasting, by using up to sixteen different parcels located in Spain. The results achieved are quite promising and encourage conducting further research in this field, having led to a \(7.65\%\) of improvement with respect to other three different strategies with both transfer and non-transfer learning models.

We propose a novel unsupervised transfer learning framework that utilises unlabelled auxiliary data to quantify and select the most relevant transferrable knowledge for recognising a target object class from the background given very limited training target samples. Unlike existing transfer learning techniques, our method does not assume that auxiliary data are labelled, nor the relationships between target and auxiliary classes are known a priori. Our unsupervised transfer learning is formulated by a novel kernel adaptation transfer (KAT) learning framework, which aims to (a) extract general knowledge about how more structured objects are visually distinctive from cluttered background regardless object class, and (b) more importantly, perform selective transfer of knowledge extracted from the auxiliary data to minimise negative knowledge transfer suffered by existing methods. The effectiveness and efficiency of the proposed approach is demonstrated by performing one-class object recognition (object vs. background) task using the Caltech256 dataset.

Recent studies on urolithiasis detection using deep learning have demonstrated promising accuracy; however, most rely on large-scale labeled imaging datasets. In clinical practice, only limited and partially labeled computed tomography (CT) scans are typically available, restricting the generalizability of conventional supervised models. This study aimed to propose a data-efficient framework for accurate stone detection from a small CT dataset by integrating self-supervised learning (SSL) and transfer learning (TL). A total of 100 abdominal CT scans were analyzed and labeled as stone present or normal by expert radiologists. To learn generalizable feature representations from limited data, a SimCLR-based SSL framework with a ResNet50 backbone was employed. During the SSL stage, the model learned from augmented image pairs without labels to maximize similarity between positive pairs and minimize similarity between negatives. The pretrained encoder was subsequently fine-tuned using labeled data in the TL stage, with the lower layers frozen and higher blocks optimized using a linear classifier. Model training was performed with 5-fold cross-validation, and performance was evaluated using accuracy, precision, recall, F1-score, and area under the receiver operating characteristic curve (AUC). The proposed SSL+TL model achieved the best performance (AUC, 0.95; F1-score, 0.91), significantly outperforming both the random initialization and TL-only models. These findings indicate that SSL pretraining effectively learns robust and transferable representations even with limited data. The proposed framework demonstrates the feasibility of artificial intelligence-based urolithiasis detection in small-data clinical environments. Combining SSL and TL alleviates data scarcity and provides a foundation for developing generalizable and resource-efficient diagnostic models for urological imaging.

Two inductive knowledge transfer approaches - multitask learning (MTL) and Feature Net (FN) - have been used to build predictive neural networks (ASNN) and PLS models for 11 types of tissue-air partition coefficients (TAPC). Unlike conventional single-task learning (STL) modeling focused only on a single target property without any relations to other properties, in the framework of inductive transfer approach, the individual models are viewed as nodes in the network of interrelated models built in parallel (MTL) or sequentially (FN). It has been demonstrated that MTL and FN techniques are extremely useful in structure-property modeling on small and structurally diverse data sets, when conventional STL modeling is unable to produce any predictive model. The predictive STL individual models were obtained for 4 out of 11 TAPC, whereas application of inductive knowledge transfer techniques resulted in models for 9 TAPC. Differences in prediction performances of the models as a function of the machine-learning method, and of the number of properties simultaneously involved in the learning, has been discussed.

Transfer learning using computational intelligence: A survey

The concept of transfer learning has received a great deal of concern and interest throughout the last decade. Selecting an ideal representational framework for instances of various domains to minimize the divergence among source and target domains is a fundamental research challenge in representative transfer learning. The domain adaptation approach is designed to learn more robust or higher-level features, required in transfer learning. This paper presents a novel transfer learning framework that employs a marginal probability-based domain adaptation methodology followed by a deep autoencoder. The proposed frame adapts the source and target domain by plummeting distribution deviation between the features of both domains. Further, we adopt the deep neural network process to transfer learning and suggest a supervised learning algorithm based on encoding and decoding layer architecture. Moreover, we have proposed two different variants of the transfer learning techniques for classification, which are termed as (i) Domain adapted transfer learning with deep autoencoder-1 (D-TLDA-1) using the linear regression and (ii) Domain adapted transfer learning with deep autoencoder-2 (D-TLDA-2) using softmax regression. Simulations have been conducted with two popular real-world datasets: ImageNet datasets for image classification problem and 20_Newsgroups datasets for text classification problem. Experimental findings have established and the resulting improvements in accuracy measure of classification shows the supremacy of the proposed D-TLDA framework over prominent state-of-the-art machine learning and transfer learning approaches.

Purpose– This article aims to develop a conceptual framework delineating the key dimension of the learning organization which significantly influences learning transfer.Design/methodology/approach– The conceptual framework was developed by analyzing previous studies and synthesizing the results associated with the following four relationships: the learning organization and learning transfer; the learning organization and organizational performance; the Learning Transfer System Inventory (LTSI) and learning transfer; and the Dimensions of Learning Organization Questionnaire (DLOQ) and the LTSI.Findings– This paper developed the learning transfer framework of the learning organization, which emphasizes the significance of leadership for learning on facilitating learning transfer in the learning organization.Research limitations/implications– First, empirical-analytical research needs to be accumulated to verify the conceptual framework developed in the present paper. Second, instruments to measure the learning organization need to incorporate the learning transfer concept. Last, organizations need to focus on leadership for learning to increase learning transfer if the organization does not have enough resources to develop all dimensions of the learning organization.Originality/value– This article contributes to the literature by producing a conceptual framework that isolates leadership for learning as a key dimension of the learning organization that specifically influences learning transfer. The pivotal dimension of leadership for learning was extracted through investigating the numerous empirical-analytical studies.

This study identified how informal learning affects learner self-efficacy and learning transfer in college students in the Department of Beauty, and analyzed whether learner self-efficacy plays a mediating role in the relationship between informal learning and learning transfer. Based on domestic and international prior studies, the theoretical background for informal learning, learner self-efficacy, and learning transfer was reviewed, and based on this, research models and hypotheses were derived. Of a total of 180 survey responses, 174 copies were used for the final analysis, excluding 6 copies that were unfaithful. As a result of the study, it was found that external exploration and self-reflection among the sub-factors of informal learning had a significant effect on learner self-efficacy, and it was also confirmed that learner self-efficacy also had a significant effect on learning transfer. In addition, it was found that informal learning has a direct effect on learning transfer, and learner self-efficacy has a mediating effect in the relationship between informal learning and learning transfer. These results suggest that college students majoring in beauty are actively using informal learning, and such learning experiences are effectively transferred to the actual beauty field. Therefore, in the field of beauty education, it is necessary to establish an evaluation and support system based on informal learning to reinforce learner self-efficacy and learning transfer.

Society is facing a paradigm shift where general education is moving towards lifelong education. Massive open online course (MOOC) is the leading online education with large-scale open classes. South Korea has localised 1,340 MOOCs known as K-MOOCs. However, K-MOOCs are not being managed effectively, and are generally low-level lectures compared to MOOCs. The variance-based structural equation modelling (SEM) using the partial least squares (PLS) model was applied for data analysis. Online surveys and analysis of 224 respondents studied 14 hypotheses and analysed eight factors to improve K-MOOC operation in South Korea. Findings indicated 'authentic learning' positively affects 'learning transfer'; 'gamification', 'teaching presence' and 'reputation' positively affect 'satisfaction' and 'commitment'. 'Learning transfer' and 'satisfaction' impacts 'continuous use intention'. Furthermore, 'commitment' positively affects 'satisfaction', and 'continuous use intention'. This research suggests that these factors must be taken into account when constructing teaching methods and educational content to improve K-MOOCs continuous usage.