A Segmentation Map Difference-Based Domain Adaptive Change Detection Method

Cross-database micro-expression recognition (CDMER) is one of recently emerging and interesting problem in micro-expression analysis. CDMER is more challenging than the conventional micro-expression recognition (MER), because the training and testing samples in CDMER come from different micro-expression databases, resulting in inconsistency of the feature distributions between the training and testing sets. In this paper, we contribute to this topic from three aspects. First, we establish a CDMER experimental evaluation protocol aiming to allow the researchers to conveniently work on this topic and evaluate their proposed methods under the same standard. Second, we conduct benchmark experiments by using NINE state-of-the-art domain adaptation (DA) methods and SIX popular spatiotemporal descriptors for investigating CDMER problem from two different perspectives. Third, we propose a novel DA method called region selective transfer regression (RSTR) to deal with the CDMER task. The overall superior performance of RSTR over the state-of-the-art DA methods demonstrates that taking into consideration the facial local region information used in RSTR contributes to developing effective DA methods for dealing with CDMER problem.

Deep multi-Wasserstein unsupervised domain adaptation

ABSTRACTIn this article, we propose a novel difference image (DI) creation method for unsupervised change detection in multi-temporal multi-spectral remote-sensing images based on deep learning theory. First, we apply deep belief network to learn local and high-level features from the local neighbour of a given pixel in an unsupervised manner. Second, a back propagation algorithm is improved to build a DI based on selected training samples, which can highlight the difference on changed regions and suppress the false changes on unchanged regions. Finally, we get the change trajectory map using simple clustering analysis. The proposed scheme is tested on three remote-sensing data sets. Qualitative and quantitative evaluations show its superior performance compared to the traditional pixel-level and texture-level-based approaches.

Remote sensing image change detection (RSICD) is an essential measure for monitoring the earth’s surface changes. In recent years, the explosive growth of very high-resolution (VHR) satellite sensors and the booming innovations in deep learning technology have significantly boosted RSICD development. However, most of the current RSICD models focus on locating accurate change areas while ignoring the efficiency of their method, which limits the practical application of RSICD models, especially for large-scale and emergency RSICD tasks. In this paper, we propose an Efficient Multi-scale-fusion Change Detection Network (EMS-CDNet) for bi-temporal RSICD tasks. Our EMS-CDNet pays more attention to the model’s inference speed and the accuracy-efficiency trade-off rather than only pursuing detection accuracy. We designed a multi-scale fusion module for EMS-CDNet, which adopts multi-scale and multi-branch operations to extract multi-scale features simultaneously and aggregate features at different feature levels. In addition to EMS-CDNet’s ability to achieve sufficient feature extraction, the multi-scale image input within the designed module alleviates the influence of image registration errors in practical applications, thereby strengthening EMS-CDNet’s value for practical RSICD tasks. We also integrated a novel partition unit in EMS-CDNet to lighten the model while maintaining the detection ability of small targets, thus shortening its processing time without a severe accuracy decrease. We conducted experiments on two state-of-the-art (SOTA) public RSICD datasets and our own collected dataset. The public datasets were utilized to comparatively measure the overall accuracy and efficiency measurement of EMS-CDNet, and the dataset of images we collected was used to observe EMS-CDNet’s performance under the influence of image registration errors. Our experimental results show that EMS-CDNet achieved a better accuracy-efficiency trade-off than the SOTA public datasets methods. For example, EMS-CDNet reduced the inference time by about 33% while maintaining identical detection accuracy to CLNet (the optimal method among the comparison methods). Furthermore, EMS-CDNet achieved higher accuracy on our collected dataset, with an F1 of 74% and mIoU of 0.806, demonstrating its robustness to image registration errors and showing its value for practical RSICD applications.

Although Deep Convolutional Neural Network (DCNN) has been admittedly witnessed as an enormous success in a wide range of applications, most of them require sufficient annotations with time-consuming and labor-exhausting efforts. Existing domain adaptation (DA) approaches delve into designing an effective loss module to minimize the distribution gap between the source and target domains. However, few studies pay attention to improve the backbone or network architecture for DA issues. In this paper, we propose a new backbone for DA specially, i.e., Transferable ResNet (TransResNet). TransResNet remedies the residual block in ResNet, separating source and target input features and highlighting more transferable channels in each block. It can be easily applied to all kinds of DA methods, without adding any extra learning parameters. We conduct substantial experiments on two general DA datasets and embed TransResNet into two seminal DA methods, including DANN and CDAN. Experimental results demonstrate TransResNet improves the transferability of the architecture, indicating that it is a great substitute for ResNet as a network backbone in DA issues.

Transfer learning and domain adaptation (DA) methods have been utilized in bearing prognostic and health management, but most of the current DA methods do not take into account the feature scale change of degraded features when aligning the feature distribution, and these methods are more suitable for the classification problem, which is more robust to the feature scale change. However, they perform poorly in regression problems. In addition, most of the remaining useful life (RUL) prediction methods require preprocessing such as statistical feature extraction on the signal, which makes the prediction process complicated. To solve the above problems, a DA method based on the representation subspace distance (RSD) is proposed for predicting the bearing RUL under different operating conditions. First, the proposed convolutional neural network (CNN) self-attention (SA) long short term memory network model is utilized to extract the deep features from the original signal, which overcomes the limitations of the CNN in extracting time series. Then, the RSD in the Riemannian geometry of the Grassmann manifold is proposed as a domain transfer loss to learn domain invariant features. The modified method can align the feature distribution of the source domain and the target domain without changing the feature scale. At the same time, the bases mismatch penalization is introduced to avoid destroying the semantic information of the features in the process of domain alignment. Finally, the effectiveness of the proposed method is verified by experiments on four types of transfer tasks, and its superiority is also demonstrated by comparison with other advanced methods.

Remote sensing image change detection is the detection process of determining the surface change area and change feature type for the same image area from multiple time series remote sensing data. It is the core technical means of land use change detection and land cover change detection, and it is also the key research field of remote sensing applied science. In view of the problems of time-consuming, labor-intensive, and low detection accuracy in the past image change detection methods, researchers in related fields have proposed more and more cutting-edge remote sensing image change detection methods. This article first describes the development of remote sensing image change detection methods, then explains the conventional processing flow of change detection and the conventional methods of change monitoring, and finally discusses the research progress of more mainstream change detection methods.

Domain Adaptation (DA) methods are usually carried out by means of simply reducing the marginal distribution differences between the source and target domains, and subsequently using the resultant trained classifier, namely source classifier, for use in the target domain. However, in many cases, the true predictive distributions of the source and target domains can be vastly different especially when their class distributions are skewed, causing the issues of sample selection bias in DA. Hence, DA methods which leverage the source labeled data may suffer from poor generalization in the target domain, resulting in negative transfer. In addition, we observed that many DA methods use either a source classifier or a linear combination of source classifiers with a fixed weighting for predicting the target unlabeled data. Essentially, the labels of the target unlabeled data are spanned by the prediction of these source classifiers. Motivated by these observations, in this paper, we propose to construct many source classifiers of diverse biases and learn the weight for each source classifier by directly minimizing the structural risk defined on the target unlabeled data so as to heal the possible sample selection bias. Since the weights are learned by maximizing the margin of separation between opposite classes on the target unlabeled data, the proposed method is established here as Maximal Margin Target Label Learning (MMTLL), which is in a form of Multiple Kernel Learning problem with many label kernels. Extensive experimental studies of MMTLL against several state-of-the-art methods on the Sentiment and Newsgroups datasets with various imbalanced class settings showed that MMTLL exhibited robust accuracies on all the settings considered and was resilient to negative transfer, in contrast to other counterpart methods which suffered significantly in prediction accuracy.

Dynamic transfer partial least squares for domain adaptive regression

Hyperspectral images (HSI) change detection (CD) has become a powerful tool to analyze the sublte surface changes. However, the application of HSI CD is constrained by the limited availability of homogeneous HSIs. HSI-RGB multimodal CD address these limitations by collaboratively utilizing multi-source data. Although multimodal CD methods have achieved encouraging results, their performance often relies on the assumption that the training and test samples have similar distributions. Recently, some domain adaptive CD methods have been introduced. However, the additional modality differences in cross-domain multimodal CD pose challenges to existing domain adaptation techniques. To address these challenges, we propose a cycle-based frequency disentanglement diffusion model with self-training for cross-domain HSI-RGB multimodal CD, which explores a frequency-domain diffusion-driven self-training mechanism to enhance consistency in change representations across different modalities and domains. Specifically, a cyclic frequency domain disentanglement-based modality-domain alignment diffusion network is proposed to achieve modality and domain alignment within a unified diffusion framework. Subsequently, a curriculum-learning based self-training dual-domain CD network is designed to process the aligned images, which leverages pseudo-label reliability to ensure stable transfer of prior knowledge while exploits complementary features across modalities for collaborative CD. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art approaches in cross-domain multimodal CD tasks.

Remote sensing image change detection (RS-CD) has made impressive progress with the help of deep learning techniques. Small object change detection (SoCD) still faces many challenges. On the one hand, when the scale of changing objects varies greatly, deep learning models with overall accuracy as the optimization goal tend to focus on large object changes and ignore small object changes to some extent. On the other hand, the RS-CD model based on deep convolutional networks needs to perform multiple spatial pooling operations on the feature map to obtain deep semantic features, which leads to the loss of small object feature-level information in the local space. Therefore, we propose a Siamese transformer change detection network with a multiscale window via an adaptive fusion strategy (SWaF-Trans). To solve the problem of ignoring small object changes, we compute self-attention in windows of different scales to model changing objects at the corresponding scales and establish semantic information links through a moving window mechanism to capture more comprehensive small object features in small-scale windows, thereby enhancing the feature representation of multiscale objects. To fuse multiscale features and alleviate the problem of small object feature information loss, we propose a channel-related fusion mechanism to model the global correlation between channels for display and adaptively adjust the fusion weights of channels to enable the network to capture more discriminative features of interest and reduce small object feature information loss. Experiments on the CDD and WHU-CD datasets show that SWaF-Trans exceeds eight advanced baseline methods, with absolute F1 scores as high as 97.10% and 93.90%, achieving maximum increases of 2% and 5.6%, respectively, compared to the baseline methods.

Summary Introduction : There has been a rapid development of deep learning (DL) models for medical imaging. However, DL requires a large labeled dataset for training the models. Getting large-scale labeled data remains a challenge, and multi-center datasets suffer from heterogeneity due to patient diversity and varying imaging protocols. Domain adaptation (DA) has been developed to transfer the knowledge from a labeled data domain to a related but unlabeled domain in either image space or feature space. DA is a type of transfer learning (TL) that can improve the performance of models when applied to multiple different datasets. Objective : In this survey, we review the state-of-the-art DL-based DA methods for medical imaging. We aim to summarize recent advances, highlighting the motivation, challenges, and opportunities, and to discuss promising directions for future work in DA for medical imaging. Methods : We surveyed peer-reviewed publications from leading biomedical journals and conferences between 2017-2020, that reported the use of DA in medical imaging applications, grouping them by methodology, image modality, and learning scenarios. Results : We mainly focused on pathology and radiology as application areas. Among various DA approaches, we discussed domain transformation (DT) and latent feature-space transformation (LFST). We highlighted the role of unsupervised DA in image segmentation and described opportunities for future development. Conclusion : DA has emerged as a promising solution to deal with the lack of annotated training data. Using adversarial techniques, unsupervised DA has achieved good performance, especially for segmentation tasks. Opportunities include domain transferability, multi-modal DA, and applications that benefit from synthetic data.

With the increase in the amount of remote sensing images (RSIs), deep learning (DL) has been used to the change detection (CD) task in remote sensing field and achieved good results. However, most existing methods do not take full advantage of the temporal dependence of the multi-temporal images. In this paper, we propose a novel method for CD, namely SNN-LSTM (a deep Siamese neural network (SNN) with convolutional Long Short-Term Memory (ConvLSTM) and channel attention module (CAM)), especially for capturing and representing spatial-temporal information effectively. It mainly contains three parts. First, a network based on Siamese convolutional architecture is designed to extract multi-level features. Then, a ConvLSTM block is introduced to further obtain time dependency of multi-temporal RSIs, and spatial information is also extracted simultaneously. Finally, CAM blocks are used to refine the extracted multi-level features, enhance the feature of changes, and eventually generate change map. The experiments are conducted on LEVIR-CD dataset, both visual results and quantitative assessment prove that the proposed method outperforms several state-of-the-art methods.

With the enhancement of air-based and space-based perception capabilities, space-aeronautics incorporation and integration is growing in importance. Full domain awareness is crucial for integrated perception systems, in which domain adaptation is one of the key problems in improving the performance of cross-domain perception. Deep learning is currently an advanced technique for complex inverse synthetic aperture radar (ISAR) object recognition. However, the training procedure needs many annotated samples, which is insufficient for certain targets, such as aircraft. Few-shot learning provides a new approach to solving the above problem by transferring useful knowledge from other domains, such as optical images from satellites. Nevertheless, it fails to fully consider the domain shift between the source and target domains, generally neglecting the transferability of training samples in the learning process. Consequently, it produces suboptimal recognition accuracy. To address the composite problems mentioned above, we propose a domain adaptive few-shot learning method from satellites to an ISAR called S2I-DAFSL for aircraft recognition tasks. Furthermore, unlike conventional domain adaptation methods that directly align the distributions, the attention transferred importance-weighting network (ATIN) is proposed to improve the transferability in the domain adaptation procedure. Compared with state-of-the-art methods, it shows that the proposed method achieves better performance, increasing the accuracy and effectiveness of classification, which is more suitable for cross-domain few-shot ISAR aircraft recognition tasks.

In real-world hyperspectral image (HSI) classification, the limited annotated examples usually lead to the insufficiently trained classifier, which further generates low classification accuracy. To overcome this challenge, Domain Adaptation (DA) methods have been developed to transfer learnable knowledge from the external HSIs with sufficient labeled examples ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., the source domain) to the interested HSI with scarce labeled examples ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., the target domain). Conventional DA approaches often pseudo-label the examples with high classification confidence and then incorporate them in the training process. However, due to the significant domain gap, relying solely on the confident examples may not be adequate to achieve satisfactory performance. Therefore, this paper proposes an Interactive Easy-to-Hard Domain Adaptation method (IEH-DA) to arrange the adaptation process so that the “easy” examples are adapted ahead of the “hard” ones. In an early stage, the easy examples with high pseudo-labeling confidence are selected for the adversarial learning based DA. In a later stage, the “hard” examples with high informativity are further selected, and they are interactively labeled by human expert to provide accurate supervision information for adaptation. As a result, the examples in target domain are used in an easy-to-hard way, which forms a curriculum sequence for orderly model training. Extensive experiments conducted on typical public datasets demonstrate that IEH-DA outperforms other state-of-the-art DA methods for HSI classification.