- Research Article
30
- 10.1016/j.patcog.2021.108436
Source data-free domain adaptation for a faster R-CNN
- Nov 19, 2021
- Pattern Recognition
- Lin Xiong + 4 more +4
Source data-free domain adaptation for a faster R-CNN
Most of the existing unsupervised domain adaptive object detection methods are based on twostage object detectors such as Faster-RCNN because the region proposal network(RPN) module, which onestage detectors don’t have, can help the model achieve instance-level local domain adaption. Although this type of methods have good detection accuracy, the real-time performance cannot meet the requirements of scenarios such as autonomous driving. To this end, we propose a self-attention domain adaptive network suitable for the one-stage detector YOLOv7, which performs the self-attention to extract the position information related to the foreground target as RPN to do instance-level local domain adaptation. At the same time, we also propose a local self-attention augmentation method to suppress the attention noise so that the model can pay attention to the foreground target accurately. Considering the fact that the detection ability of model is from weak to robust during training, we apply coarse-to-fine alignment that global domain adaptation first aligns the the global feature coarsely and transferring to local domain adaptation gradually to finely align the local feature of the foreground target. Our proposed method is validated on Cityspaces, Foggy Cityspaces, KITTI and Sim10K datasets, outperforming existing domain adaptive object detection methods.
Source data-free domain adaptation for a faster R-CNN
Source data-free domain adaptation for a faster R-CNN
Multilevel Heterogeneous Domain Adaptation Method for Remote Sensing Image Segmentation
Due to more abundant data sources, more various objects of interest, and more time-consuming annotations, there is a large amount of out-of-distribution (OOD) data in the remote sensing field, on which the performance of high-accuracy image segmentation models trained under ideal experimental conditions generally degrades dramatically. Domain adaptation (DA) consequently comes into being, which aims to learn the predictor for the label-scarce target domain of interest with the help of the label-sufficient source domain in the presence of the distribution difference, namely, domain shift, between the two domains. However, the off-the-shelf DA methods for image segmentation not only struggle to cope with the more complex domain shift problems in remote sensing imagery but also almost cannot process heterogeneous data directly without information loss. While the current heterogeneous DA methods mostly still rely on some supervision information from the target domain, which is typically inaccessible in the real world. To overcome these drawbacks, we propose the multilevel heterogeneous unsupervised DA (UDA) method, termed MHDA, which unifies the instance-level DA based on cycle consistency, the feature-level DA based on contrastive learning, and the decision-level DA based on task consistency into a framework to more effectively handle the complex domain shift and heterogeneous data. After that, extensive DA experiments are conducted on the International Society for Photogrammetry and Remote Sensing (ISPRS) dataset, the BigCity dataset constructed by ourselves, and the Wuhan University (WHU) dataset, to explore the effect of each module in MHDA, the necessity of heterogeneous DA, and the effectiveness of multilevel DA. And the results demonstrate that MHDA can achieve superior performance on the remote sensing image segmentation task, compared with several state-of-the-art DA methods.
Read moreSemi-supervised domain adaptation for segmentation models on different monitoring settings
Semi-supervised domain adaptation for segmentation models on different monitoring settings
Does BERT need domain adaptation for clinical negation detection?
Classifying whether concepts in an unstructured clinical text are negated is an important unsolved task. New domain adaptation and transfer learning methods can potentially address this issue. We examine neural unsupervised domain adaptation methods, introducing a novel combination of domain adaptation with transformer-based transfer learning methods to improve negation detection. We also want to better understand the interaction between the widely used bidirectional encoder representations from transformers (BERT) system and domain adaptation methods. We use 4 clinical text datasets that are annotated with negation status. We evaluate a neural unsupervised domain adaptation algorithm and BERT, a transformer-based model that is pretrained on massive general text datasets. We develop an extension to BERT that uses domain adversarial training, a neural domain adaptation method that adds an objective to the negation task, that the classifier should not be able to distinguish between instances from 2 different domains. The domain adaptation methods we describe show positive results, but, on average, the best performance is obtained by plain BERT (without the extension). We provide evidence that the gains from BERT are likely not additive with the gains from domain adaptation. Our results suggest that, at least for the task of clinical negation detection, BERT subsumes domain adaptation, implying that BERT is already learning very general representations of negation phenomena such that fine-tuning even on a specific corpus does not lead to much overfitting. Despite being trained on nonclinical text, the large training sets of models like BERT lead to large gains in performance for the clinical negation detection task.
Read moreAdversarial unsupervised domain adaptive inland vessel detection method
Aiming at the problems that the target detection accuracy of inland watercraft is reduced, and the difficulty of labeling samples is increased due to fog, according to the characteristics of inland water ship detection, an adversarial unsupervised domain adaptive method is proposed, using labeled sunny samples and unlabeled foggy samples, which can adaptively detect ship targets in foggy conditions. On the basis of Faster-RCNN, a ship local feature domain discriminator based on full convolution and a ship global feature domain discriminator based on a hybrid attention mechanism are respectively constructed to realize the features of sunny and foggy ship images in multi-level domain space alignment, to better complete the adaptive detection from sunny ship data to foggy ship data. At the same time, using the idea of adversarial learning, a local feature adversarial loss function based on the least square method and a global feature adversarial loss based on cross entropy are designed to strengthen the adversarial training between feature extractor and domain discriminator. The experimental results on the inland ship data set show that only using the marked sunny ship data and the unmarked foggy ship data can adaptively detect ships in foggy weather, and the accuracy and real-time performance are significantly higher than those of the comparison model.
Read moreDiscriminative and Geometry-Aware Unsupervised Domain Adaptation.
Domain adaptation (DA) aims to generalize a learning model across training and testing data despite the mismatch of their data distributions. In light of a theoretical estimation of the upper error bound, we argue, in this article, that an effective DA method for classification should: 1) search a shared feature subspace where the source and target data are not only aligned in terms of distributions as most state-of-the-art DA methods do but also discriminative in that instances of different classes are well separated and 2) account for the geometric structure of the underlying data manifold when inferring data labels on the target domain. In comparison with a baseline DA method which only cares about data distribution alignment between source and target, we derive three different DA models for classification, namely, close yet discriminative DA (CDDA), geometry-aware DA (GA-DA), and discriminative and GA-DA (DGA-DA), to highlight the contribution of CDDA based on 1), GA-DA based on 2), and, finally, DGA-DA implementing jointly 1) and 2). Using both the synthetic and real data, we show the effectiveness of the proposed approach which consistently outperforms the state-of-the-art DA methods over 49 image classification DA tasks through eight popular benchmarks. We further carry out an in-depth analysis of the proposed DA method in quantifying the contribution of each term of our DA model and provide insights into the proposed DA methods in visualizing both real and synthetic data.
Read moreEnhanced Cross-Domain Dim and Small Infrared Target Detection via Content-Decoupled Feature Alignment
The detection and recognition of dim and small infrared (IR) targets across domains pose two formidable challenges: distributional discrepancies in samples and scarcity or absence of annotated instances in the target domain. While current unsupervised domain adaptive object detection methods can somewhat alleviate the performance degradation caused by these issues, they fail to address the differences in semantic content between the background environments in different application scenarios. This results in a semantic gap that impedes the algorithm’s adaptability. We propose a content-decoupled unsupervised domain adaptive method to mitigate the adverse impact of the semantic gap on domain adaptation. Specifically, we introduce an unsupervised task-guided branch in the one-stage detector that executes style transfer in real-time during training, directing the feature representation via shared parameters. A novel content-decoupled feature alignment module aligns semantically unrelated features in the source and target domains while preserving the intrinsic semantics of dim and small IR targets. The experiments on a cross-band IR dataset reveal that the proposed method enhances the mean average precision by 18.3% compared to the baseline detector without increasing computational burden.
Read moreDiscriminative distribution alignment for domain adaptive object detection
Discriminative distribution alignment for domain adaptive object detection
Cross-regional oil palm tree counting and detection via a multi-level attention domain adaptation network
Cross-regional oil palm tree counting and detection via a multi-level attention domain adaptation network
Attention Regularized Laplace Graph for Domain Adaptation.
In leveraging manifold learning in domain adaptation (DA), graph embedding-based DA methods have shown their effectiveness in preserving data manifold through the Laplace graph. However, current graph embedding DA methods suffer from two issues: 1). they are only concerned with preservation of the underlying data structures in the embedding and ignore sub-domain adaptation, which requires taking into account intra-class similarity and inter-class dissimilarity, thereby leading to negative transfer; 2). manifold learning is proposed across different feature/label spaces separately, thereby hindering unified comprehensive manifold learning. In this paper, starting from our previous DGA-DA, we propose a novel DA method, namely A ttention R egularized Laplace G raph-based D omain A daptation (ARG-DA), to remedy the aforementioned issues. Specifically, by weighting the importance across different sub-domain adaptation tasks, we propose the A ttention R egularized Laplace Graph for class aware DA, thereby generating the attention regularized DA. Furthermore, using a specifically designed FEEL strategy, our approach dynamically unifies alignment of the manifold structures across different feature/label spaces, thus leading to comprehensive manifold learning. Comprehensive experiments are carried out to verify the effectiveness of the proposed DA method, which consistently outperforms the state of the art DA methods on 7 standard DA benchmarks, i.e., 37 cross-domain image classification tasks including object, face, and digit images. An in-depth analysis of the proposed DA method is also discussed, including sensitivity, convergence, and robustness.
Read morePCLUDA: A Pseudo-Label Consistency Learning- Based Unsupervised Domain Adaptation Method for Cross-Domain Optical Remote Sensing Image Retrieval
Recent advances in deep learning have dramatically improved the performance of content-based remote sensing image retrieval (CBRSIR) with the same distribution of training set (source domain) and test set (target domain). In fact, their distributions are inconsistent in most cases, which can lead to a dramatic decrease in retrieval performance. Currently, some unsupervised domain adaptation (DA) methods for other remote sensing applications have been proposed to eliminate the inconsistency. However, the current unsupervised DA methods do not make full use of the target domain’s distribution characteristics when delineating its decision boundary. This tends to degrade the cross-domain retrieval performance. In this article, a pseudo-label consistency learning-based unsupervised DA method (PCLUDA) is proposed for cross-domain CBRSIR. Our PCLUDA method minimizes the difference in probability distribution between the target domain and its perturbed output by a pseudo-label self-training and consistency regularization strategy, followed by adjusting the target domain’s decision boundaries to the low-density region. Besides, minimize class confusion (MCC) is introduced to reduce negative transfer caused by large intraclass variance of RSIs. Two cross-domain datasets with 12 cross-domain scenarios are constructed based on six open access datasets to measure DA methods. Experimental results show that our PCLUDA method achieves superior retrieval performances with average retrieval precision improvement by 4.9%–32.3% compared with eight state-of-the-art DA approaches in complex cross-domain scenarios. Furthermore, other experimental results indicate that our PCLUDA can also reach optimal retrieval performances in different kinds of deep learning networks [i.e., vision transformer (ViT) and convolutional neural networks (CNNs)].
Read moreC³DA: A Universal Domain Adaptation Method for Scene Classification From Remote Sensing Imagery
Various remote sensing applications have widely used domain adaptation (DA) methods. Since it does not need to add human interpretation in the target domain, it can be used in cross-region, multi-temporal, and multi-sensor application scenarios. In order to further optimize the design of the loss function and better address the challenges of DA in remote sensing, in this paper, we propose a new universal DA method named C <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> DA for scene recognition of remote sensing images. It has a comprehensive C <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> criterion for recognizing the "unknown" classes by innovatively fusing confidence, consistency, and certainty of samples to make our network training more efficient. We evaluate the performance of our proposed method based on six transfer tasks on three remote sensing datasets. The evaluation results show that our proposed method achieves an average H-score of 58.44%, significantly higher than other SOTA universal DA methods with an average improvement of 2.32~29.43%. Compared to the baseline ResNet-50, it achieves up to 19.92% improvement, demonstrating that the proposed method outperforms in the universal DA scenario. In the future, we also plan to expand the application of this method to more scenarios.
Read moreIntegrated Generative-Model Domain-Adaptation for Object Detection under Challenging Conditions
Realistic object detection scenarios, most notably the detection of objects such as pedestrians and traffic signs by autonomous vehicles, present a real challenge for state-of-the-art methods. Domain shifts further aggravate this problem when an autonomous vehicle has to operate and detect objects under more challenging conditions (e.g., rainy weather). In this paper, we demonstrate that the performance of challenging object detection scenarios can be improved rather significantly using a novel integrated Generative-model based unsupervised training and Domain Adaptation (GDA) framework. In particular, we exploit unsupervised image-to-image translation to generate annotated visuals that are representatives of a target challenging domain. Then, we use these generated annotated visuals in addition to unlabeled target domain data to train a domain adaptive object detection method. We show that using this integrated approach outperforms both methods, unsupervised image translation, and domain adaptation, when they are used separately. We evaluate the performance of the proposed GDA framework using real visuals captured by vehicles driving under rainy weather. Our simulation results show that GDA can achieve significant improvements in the detection of pedestrians and traffic signs when tested under real challenging conditions in comparison with state-of-the-art methods.
Read moreGeometry-Insensitive RPN Prototypes for Domain Adaptive 3D Object Detection
The region proposal network (RPN) plays a critical role in object detection for a two-stage domain adaptive 3D object detector. However, current methods usually minimize the disparity between source and target domains by reducing the bias in intrinsic geometric information or by undertaking feature alignment according to the geometric disparity, but ignore the transferability of RPN related features and neglect the discriminability between foreground and background, resulting in generating low-quality RPN proposals. Thus we propose a novel domain adaptation method to distinguish the discriminability between foreground and background. It could implicitly avoid the geometric disparity of objects in feature alignment. Specifically, we first construct learnable and geometry-insensitive foreground RPN prototype and background RPN prototype. Then we enforce the foreground RPN features and background RPN features to align with the foreground RPN prototype and background RPN prototype, respectively. By this way, the distributional discrepancy is effectively decreased and the adaptability is promoted for existing 3D detectors. We demonstrate that our approach achieves promising results compared with other domain adaptation works on multiple cross-domain detection scenarios.
Read moreA Domain-Adaptive Tree-Crown Detection and Counting Method Based on Cascade Region Proposal Networks
Rapid and accurate tree-crown detection is significant to forestry management and precision forestry. In the past few decades, the development and maturity of remote sensing technology has created more convenience for tree-crown detection and planting management. However, the variability of the data source leads to significant differences between feature distributions, bringing great challenges for traditional deep-learning-based methods on cross-regional detection. Moreover, compared with other tasks, tree-crown detection has the problems of a poor abundance of objects, an overwhelming number of easy samples and the existence of a quantity of impervious background similar to the tree crown, which make it difficult for the classifier to learn discriminative features. To solve these problems, we apply domain adaptation (DA) to tree-crown detection and propose a DA cascade tree-crown detection framework with multiple region proposal networks, dubbed CAS-DA, realizing cross-regional tree-crown detection and counting from multiple-source remote sensing images. The essence of the multiple region proposal networks in CAS-DA is obtaining the multilevel features and enhancing deeper label classifiers gradually by filtering simple samples of source domain at an early stage. Then, the cascade structure is integrated with a DA object detector and the end-to-end training is realized through the proposed cascade loss function. Moreover, a filtering strategy based on the planting rules of tree crowns is designed and applied to filter wrongly detected trees by CAS-DA. We verify the effectiveness of our method in two different domain shift scenarios, including adaptation between satellite and drone images and cross-satellite adaptation. The results show that, compared to the existing DA methods, our method achieves the best average F1-score in all adaptions. It is also found that the performance between satellite and drone images is significantly worse than that between different satellite images, with average F1-scores of 68.95% and 88.83%, respectively. Nevertheless, there is an improvement of 11.88%~40.00% in the former, which is greater than 0.50%~5.02% in the latter. The above results prove that in tree-crown detection, it is more effective for the DA detector to improve the detection performance on the source domain than to diminish the domain shift alone, especially when a large domain shift exists.
Read more