Easy-to-Hard Domain Adaptation With Human Interaction for Hyperspectral Image Classification

Domain Adaptation (DA) methods are usually carried out by means of simply reducing the marginal distribution differences between the source and target domains, and subsequently using the resultant trained classifier, namely source classifier, for use in the target domain. However, in many cases, the true predictive distributions of the source and target domains can be vastly different especially when their class distributions are skewed, causing the issues of sample selection bias in DA. Hence, DA methods which leverage the source labeled data may suffer from poor generalization in the target domain, resulting in negative transfer. In addition, we observed that many DA methods use either a source classifier or a linear combination of source classifiers with a fixed weighting for predicting the target unlabeled data. Essentially, the labels of the target unlabeled data are spanned by the prediction of these source classifiers. Motivated by these observations, in this paper, we propose to construct many source classifiers of diverse biases and learn the weight for each source classifier by directly minimizing the structural risk defined on the target unlabeled data so as to heal the possible sample selection bias. Since the weights are learned by maximizing the margin of separation between opposite classes on the target unlabeled data, the proposed method is established here as Maximal Margin Target Label Learning (MMTLL), which is in a form of Multiple Kernel Learning problem with many label kernels. Extensive experimental studies of MMTLL against several state-of-the-art methods on the Sentiment and Newsgroups datasets with various imbalanced class settings showed that MMTLL exhibited robust accuracies on all the settings considered and was resilient to negative transfer, in contrast to other counterpart methods which suffered significantly in prediction accuracy.

The ability of supervised image analysis methods to provide state-of-the-art performance is limited by availability of high-quality labeled data in large quantities. Domain adaptation approaches propose a solution to this problem by leveraging quality labeled information from auxiliary data sources. In this work, we use adversarial learning for domain adaptation for remote sensing applications. First, we approached the problem of unavailable target domain labels with unsupervised domain adaptation and then extended our method for semisupervised domain adaptation to use a few available labels as well. We are using adversarial learning to extract discriminative target domain features that are aligned with source domain. We test our framework for two very different applications of remote sensing imagery, multiclass classification in hyperspectral images and semantic segmentation in large scale satellite images. For hyperspectral image analysis two datasets were used: the University of Houston shadow data was used for quantifying the efficacy of our approach to varying illumination, and the Botswana data was used to quantify the efficacy of our approach under multitemporal spectral shifts. Multisensor high-resolution images from National Agriculture Imagery Program and SpaceNet-Rio datasets were used as the source and target for the task of building extraction for large scale semantic segmentation based domain adaptation.

In the absence of the labeled samples in a domain referred to as target domain, Domain Adaptation (DA) techniques come in handy. Generally, DA techniques assume there are available source domains that share similar predictive function with the target domain. Two core challenges of DA typically arise, variance that exists between source and target domains, and the inherent source hypothesis bias. In this paper, we first propose a Stability Transfer criterion for selecting relevant source domains with low variance. With this criterion, we introduce a TARget learning Assisted by Source Classifier Adaptation (TARASCA) method to address the two core challenges that have impeded the performances of DA techniques. To verify the robustness of TARASCA, extensive experimental studies are carried out with comparison to several state-of-the-art DA methods on the real-world Sentiment and Newsgroups datasets, where various settings for the class ratios of the source and target domains are considered.

Transfer learning and domain adaptation (DA) methods have been utilized in bearing prognostic and health management, but most of the current DA methods do not take into account the feature scale change of degraded features when aligning the feature distribution, and these methods are more suitable for the classification problem, which is more robust to the feature scale change. However, they perform poorly in regression problems. In addition, most of the remaining useful life (RUL) prediction methods require preprocessing such as statistical feature extraction on the signal, which makes the prediction process complicated. To solve the above problems, a DA method based on the representation subspace distance (RSD) is proposed for predicting the bearing RUL under different operating conditions. First, the proposed convolutional neural network (CNN) self-attention (SA) long short term memory network model is utilized to extract the deep features from the original signal, which overcomes the limitations of the CNN in extracting time series. Then, the RSD in the Riemannian geometry of the Grassmann manifold is proposed as a domain transfer loss to learn domain invariant features. The modified method can align the feature distribution of the source domain and the target domain without changing the feature scale. At the same time, the bases mismatch penalization is introduced to avoid destroying the semantic information of the features in the process of domain alignment. Finally, the effectiveness of the proposed method is verified by experiments on four types of transfer tasks, and its superiority is also demonstrated by comparison with other advanced methods.

In this paper, we propose a new unsupervised domain adaptation method for document classification. We address the problem of domain adaptation for document classification where the source and target domains do not differ significantly and there is no labeled data in the target domain. In this case, we can use conventional semi-supervised learning. Thus, we use the naive Bayes-based expectation-maximization method (NBEM) which is very effective for document classification. However, NBEM does not utilize the difference between a source domain and a target domain. We combine NBEM with the feature weighted method for domain adaptation, referred to as “self-training feature weight” (STFW). Our proposed method alternately uses NBEM and STFW to gradually improve document classification precision for a target domain. This method significantly outperforms the conventional unsupervised methods for domain adaptation.

Although Deep Convolutional Neural Network (DCNN) has been admittedly witnessed as an enormous success in a wide range of applications, most of them require sufficient annotations with time-consuming and labor-exhausting efforts. Existing domain adaptation (DA) approaches delve into designing an effective loss module to minimize the distribution gap between the source and target domains. However, few studies pay attention to improve the backbone or network architecture for DA issues. In this paper, we propose a new backbone for DA specially, i.e., Transferable ResNet (TransResNet). TransResNet remedies the residual block in ResNet, separating source and target input features and highlighting more transferable channels in each block. It can be easily applied to all kinds of DA methods, without adding any extra learning parameters. We conduct substantial experiments on two general DA datasets and embed TransResNet into two seminal DA methods, including DANN and CDAN. Experimental results demonstrate TransResNet improves the transferability of the architecture, indicating that it is a great substitute for ResNet as a network backbone in DA issues.

Building segmentation for remote sensing images usually requires pixel-level labels which is difficult to collect when the images are in low resolution and quality. Recently, weakly supervised semantic segmentation methods have achieved promising performance, which only rely on image-level labels for each image. However, buildings in remote sensing images tend to present regular structures. The lack of supervision information may result in the ambiguous boundaries. In this paper, we propose a new weakly supervised network for refined building segmentation by mining the cross-domain structure affinity (CDSA) from multi-source remote sensing images. CDSA integrates the ideas of weak supervision and domain adaptation, where a pixel-level labeled source domain and an image-level labeled target domain are required. The target of CDSA is to learn a powerful segmentation network on the target domain with the guidance of source domain data. CDSA mainly consists of two branches, the structure affinity module (SAM) and the spatial structure adaptation (SSA). In brief, SAM is developed to learn the structure affinity of the buildings from source domain, and SSA infuses the structure affinity to the target domain via a domain adaptation approach. Moreover, we design an end-to-end network structure to simultaneously optimize the SAM and SSA. In this case, SAM can receive pseudosupervised information from SSA, and in turn provide a more accurate affinity matrix for SSA. In the experiments, our model can achieve an IoU score at 57.87% and 79.57% for the WHU and Vaihingen data sets. We compare CDSA with several state-of-the-art weakly supervised and domain adaptation methods, and the results indicate that our method presents advantages on two public data sets.

Active multi-kernel domain adaptation for hyperspectral image classification

Recently, transfer learning, and especially domain adaptation (DA), has been introduced to the cross-scene remote sensing image scene classification task. The existing DA approaches for the remote sensing image built on the hypothesis that the source domain and target domain share identical category spaces. However, in a new region, novel categories may appear. Open-set domain adaptation (OSDA) methods is proposed to solve such problems. Existing OSDA methods need to determine the thresholds or other hyper-parameters (such as number of unknown sample) to detect novel classes. In this paper, an end-to-end open-set domain adaptation for cross scene classification (OSDA_CSC) framework is proposed to solve these problems. In this framework, to overcome the different feature distributions caused by the different class spaces and the feature discriminability decrease in the remote sensing OSDA scenario, we propose an unknown-class detector using a one-vs-all network considering class distances to find the unknown classes in the target domain. We also introduce target domain neighbourhood clustering loss to improve the target domain feature discriminability. The proposed method can improve cross-scene classification and automatically label the unknown class samples in the target domain.

Cross-database micro-expression recognition (CDMER) is one of recently emerging and interesting problem in micro-expression analysis. CDMER is more challenging than the conventional micro-expression recognition (MER), because the training and testing samples in CDMER come from different micro-expression databases, resulting in inconsistency of the feature distributions between the training and testing sets. In this paper, we contribute to this topic from three aspects. First, we establish a CDMER experimental evaluation protocol aiming to allow the researchers to conveniently work on this topic and evaluate their proposed methods under the same standard. Second, we conduct benchmark experiments by using NINE state-of-the-art domain adaptation (DA) methods and SIX popular spatiotemporal descriptors for investigating CDMER problem from two different perspectives. Third, we propose a novel DA method called region selective transfer regression (RSTR) to deal with the CDMER task. The overall superior performance of RSTR over the state-of-the-art DA methods demonstrates that taking into consideration the facial local region information used in RSTR contributes to developing effective DA methods for dealing with CDMER problem.

Deep multi-Wasserstein unsupervised domain adaptation

In this paper, we propose a new framework called domain adaptation machine (DAM) for the multiple source domain adaption problem. Under this framework, we learn a robust decision function (referred to as target classifier) for label prediction of instances from the target domain by leveraging a set of base classifiers which are prelearned by using labeled instances either from the source domains or from the source domains and the target domain. With the base classifiers, we propose a new domain-dependent regularizer based on smoothness assumption, which enforces that the target classifier shares similar decision values with the relevant base classifiers on the unlabeled instances from the target domain. This newly proposed regularizer can be readily incorporated into many kernel methods (e.g., support vector machines (SVM), support vector regression, and least-squares SVM (LS-SVM)). For domain adaptation, we also develop two new domain adaptation methods referred to as FastDAM and UniverDAM. In FastDAM, we introduce our proposed domain-dependent regularizer into LS-SVM as well as employ a sparsity regularizer to learn a sparse target classifier with the support vectors only from the target domain, which thus makes the label prediction on any test instance very fast. In UniverDAM, we additionally make use of the instances from the source domains as Universum to further enhance the generalization ability of the target classifier. We evaluate our two methods on the challenging TRECIVD 2005 dataset for the large-scale video concept detection task as well as on the 20 newsgroups and email spam datasets for document retrieval. Comprehensive experiments demonstrate that FastDAM and UniverDAM outperform the existing multiple source domain adaptation methods for the two applications.

Easy Domain Adaptation for cross-subject multi-view emotion recognition

This study suggests a new domain adaptation paradigm that can address potential data-privacy issues. Despite promising results of existing domain adaptation methods, they have a strong constraint where the source and target samples are accessible during a training phase. However, direct usage of source samples possibly causes data-privacy issues especially when each label of source domain acts as an individual's identifier such as biometric information. To address data-privacy problems in conventional domain adaptation, we propose privacy-preserving domain adaptation (PPDA). Our main hypothesis is that if we train our target model initialized from a pre-trained source model in a self-learning manner, we can successfully transfer knowledge from a labeled source domain to an unlabeled target domain. In our preliminary study, we observe that target samples with low self-entropy measured from the pre-trained source model achieves sufficiently high accuracy. From this key observation, we first select the reliable samples based on self-entropy and define them as class prototypes. We then assign pseudo labels to the target samples through the similarity between target samples and class prototypes. To further reduce the uncertainty of the pseudo labeling process, we also introduce a sample-level reweighting scheme. Surprisingly, our PPDA model outperforms conventional domain adaptation methods on public datasets even though we do not directly access any source data.

Population-based structural health monitoring (PBSHM) involves utilising knowledge from one set of structures in a population and applying it to a different set, such that predictions about the health states of each member in the population can be performed and improved. Central ideas behind PBSHM are those of knowledge transfer and mapping. In the context of PBSHM, knowledge transfer involves using information from a structure, defined as a source domain, where labels are known for a given feature, and mapping these onto the unlabelled feature space of a different, target domain structure. If the mapping is successful, a machine learning classifier trained on the transformed source domain data will generalise to the unlabelled target domain data; i.e. a classifier built on one structure will generalise to another, making Structural Heath Monitoring (SHM) cost-effective and applicable to a wide range of challenging industrial scenarios. This process of mapping features and labels across source and target domains is defined as domain adaptation, a subcategory of transfer learning. However, a key assumption in conventional domain adaptation methods is that there is consistency between the feature and label spaces. This means that the features measured from one structure must be the same dimension as the other (i.e. the same number of spectral lines of a transmissibility), and that labels associated with damage locations, classification and assessment, exist on both structures. These consistency constraints can be restrictive, limiting to which types of population domain adaptation can be applied. This paper, therefore, provides a mathematical underpinning for when domain adaptation is possible in a structural dynamics context, with reference to topology of a graphical representation of structures. By defining when conventional domain adaptation is applicable in a structural dynamics setting, approaches are discussed that could overcome these consistency restrictions. This approach provides a general means for performing transfer learning within a PBSHM context for structural dynamics-based features.KeywordsPopulation-based structural health monitoringTransfer learningDomain adaptation