Visual recognition, encompassing tasks such as image classification, object detection, and semantic segmentation, has been a fundamental challenge in computer vision research. It also serves as the foundation for numerous applications, including autonomous driving, remote sensing, and robotics. The emergence of deep learning has significantly advanced this field, enabling remarkable achievements through the use of end-to-end trainable deep neural networks (DNNs). However, this remarkable progress comes at the cost of requiring vast amounts of annotated training data, the collection of which is often extremely labor-intensive and time-consuming. This constraint presents a significant barrier to the scalability and adaptability of various deep learning approaches. Transfer learning, which aims to transfer the off-the-shelf learned knowledge from one or multiple related source domains to target domains, is a promising way to mitigate this constraint. In this thesis, we explore label-efficient visual recognition with various transfer learning techniques, where the objective is to learn effective visual representations for target domains without any annotations by transferring the off-the-shelf knowledge learned in different but related source domains. Specifically, we investigate three representative types of transfer learning problems, including unsupervised domain adaptation, black-box unsupervised domain adaptation and test-time prompt tuning. As one of the most typical transfer learning problems, unsupervised domain adaptation aims to adapt a visual recognition model that is trained on labeled source-domain data to perform well on a related but unlabeled target-domain data. The primary challenge is to mitigate the distribution differences between the source and target domains, which are also referred to as domain gap or domain discrepancy, such that the model can generalize well on the target domain. In this thesis, we propose three innovative unsupervised domain adaptation techniques designed to efficiently transfer knowledge from labeled source domains to unlabeled target domains across a range of visual recognition tasks. Specifically, we explore diverse approaches to align domains from multiple perspectives (e.g., data or feature space alignment), thereby reducing the domain gap and enabling to perform well on unlabeled target domain. Black-box unsupervised domain adaptation aims to learn visual recognition model for target domain with the initial predictions of target data obtained by black-box source-domain model. In another word, during the adaptation process, only the model predictions of target data are available while either source data or source models are inaccessible. Compared to conventional unsupervised domain adaptation, black-box unsupervised domain adaptation offers notable advantages in terms of data privacy and flexibility, allowing adaptation to different target models regardless of the architecture or details of the source-trained black-box model. In this thesis, we analyze the challenges in black-box domain adaptation and design a novel approach for effectively exploiting the initial predictions of target data, leading to better adapted model on unlabeled target data. We further explore label-efficient visual recognition in the recent era of vision-language models. Despite the impressive zero-shot generalization capabilities of vision-language models, they always face unneglectable gaps while applied to target datasets, i.e., a target dataset typically features unique image styles and text formats tailored to its specific tasks. To circumvent this limitation, we explore an effective transfer learning approach for vision-language models, i.e., test-time prompt tuning, where the prompts are optimized as learnable vectors during the test stage without requiring any annotations for target data. Specifically, we examine the limitation of existing test-time prompt tuning methods, and propose a novel approach that could effectively exploit the learned knowledge in vision-language models and learn prompts for target data on-the-fly. Extensive experiments conducted over various visual recognition benchmarks indicate that our proposed methods achieve superior performance over target datasets, enabling to largely mitigate the dependence of labeled data across various visual recognition tasks.
Read more