Multi-label Few-shot Learning for Sound Event Recognition

Few-Shot Learning (FSL) aims to improve a model’s generalization capability in low data regimes. Recent FSL works have made steady progress via metric learning, meta learning, representation learning, etc. However, FSL remains challenging due to the following longstanding difficulties. 1) The seen and unseen classes are disjoint, resulting in a distribution shift between training and testing. 2) During testing, labeled data of previously unseen classes is sparse, making it difficult to reliably extrapolate from labeled support examples to unlabeled query examples. To tackle the first challenge, we introduce Hybrid Consistency Training to jointly leverage two types of consistency: 1) interpolation consistency, which interpolates hidden features to imposes linear behavior locally, and 2) data augmentation consistency, which learns robust embeddings against sample variations. As for the second challenge, we use unlabeled examples to iteratively normalize features and adapt prototypes, as opposed to commonly used one-time update, for more reliable prototype-based transductive inference. We show that our method generates a 2% to 5% improvement over the state-of-the-art methods with similar backbones on five FSL datasets and, more notably, a 7% to 8% improvement for more challenging cross-domain FSL.

Revisiting metric learning for few-shot image classification

TST_MFL: Two-stage training based metric fusion learning for few-shot image classification

Coarse-to-fine pseudo supervision guided meta-task optimization for few-shot object classification

Self-supervised learning (SSL) techniques have recently been integrated into the few-shot learning (FSL) framework and have shown promising results in improving the few-shot image classification performance. However, existing SSL approaches used in FSL typically seek the supervision signals from the global embedding of every single image. Therefore, during the episodic training of FSL, these methods cannot capture and fully utilize the local visual information in image samples and the data structure information of the whole episode, which are beneficial to FSL. To this end, we propose to augment the few-shot learning objective with a novel self-supervised Episodic Spatial Pretext Task (ESPT). Specifically, for each few-shot episode, we generate its corresponding transformed episode by applying a random geometric transformation to all the images in it. Based on these, our ESPT objective is defined as maximizing the local spatial relationship consistency between the original episode and the transformed one. With this definition, the ESPT-augmented FSL objective promotes learning more transferable feature representations that capture the local spatial features of different images and their inter-relational structural information in each input episode, thus enabling the model to generalize better to new categories with only a few samples. Extensive experiments indicate that our ESPT method achieves new state-of-the-art performance for few-shot image classification on three mainstay benchmark datasets. The source code will be available at: https://github.com/Whut-YiRong/ESPT.

Few-shot learning aims to learn a classifier to recognize unseen classes with limited labeled examples. The scarcity of training data remains a challenging problem in few-shot classification. Many few-shot learning methods focus on the structure of meta-network models and there is often no dedicated study of meta-knowledge representation. In this paper, a meta-learning strategy is introduced and a meta-network NAM Net is proposed. Taking the advantage of 'learning to learn', the model acquires meta-knowledge via few-shot classification tasks and applies it to new few-shot scenarios. Specifically, the meta-learner learns the best parameters of the feature extraction and similarity metric modules. The distance between the support feature and query feature is obtained by a learnable metric function, which leads to the classification result. The base learner migrates the meta-knowledge to the target class to perform classification in a new few-shot episode. Moreover, the Normalization-based Attention Module is adapted to feature extractor to enhance meta-knowledge representation. Compared with few-shot learning benchmarks, NAM Net is effective and achieves higher accuracy in both 5-way 1-shot and 5-shot classification tasks.

Few-shot learning(FSL) aims to generalize model to novel categoeries by few labelled samples, which is challenging for machine. Large-scaled pretrained models, especially vision transformers achieve excellent performances benefiting from numerous and diverse data. Researchers have exploited pretrained models in few-shot classification by simply updating the whole parameters and finetuning on few samples. In this paper, we explore two methods: vision prompt tuning and a reparameterization method called ‘scaling&&shift’ to leverage pretrained models in few-shot classification. Vision prompt tuning is for vision transformer only and we first evaluate the method in few-shot setting. ‘Scaling&&shift’ is originally applied in convolution neural networks(CNN). We extend it to vision transformer. The two methods are evaluated on standard benchmarks such as miniImageNet, CUB, CIFAR-FS, clipart and sketch. The results show that ‘scaling&&shift’ reaches the same level compared to updating the whole parameters. Vision prompt tuning is 0%~5% lower than updating the whole parameters over five datasets while it has quite smaller amount of parameters updated.

Few-shot learning (FSL) is designed to equip models with the capability to quickly adapt to new, unseen domains in open-world scenarios. However, there is a notable discrepancy between the multitude of new concepts encountered in the open world and the limited scale of existing FSL studies, which focus predominantly on a small number of novel classes. This limitation hinders the practical implementation of FSL in real-world situations. To address this issue, we introduce a novel problem called Few-Shot Learning with Many Novel Classes (FSL-MNC), which expands the number of novel classes more than 500 times compared to traditional FSL settings. This new challenge presents two main difficulties: increased computational load during meta-training and reduced classification accuracy due to the larger number of classes during meta-testing. To tackle these problems, we introduce the Simple Hierarchy Pipeline (SHA-Pipeline). In response to the inefficiency of traditional Episode Meta-Learning (EML) protocols, we redesign a more efficient meta-training strategy to manage the increased number of novel classes. Moreover, to distinguish distinct semantic features across a broad array of novel classes, we effectively reconstruct and utilize class hierarchy information during meta-testing. Our experiments demonstrate that the SHA-Pipeline substantially outperforms both the ProtoNet baseline and current leading alternatives across various numbers of novel classes.

Due to a lack of labeled samples, deep learning methods generally tend to have poor classification performance in practical applications. Few-shot learning (FSL), as an emerging learning paradigm, has been widely utilized in hyperspectral image (HSI) classification with limited labeled samples. However, the existing FSL methods generally ignore the domain shift problem in cross-domain scenes and rarely explore the associations between samples in the source and target domain. To tackle the above issues, a graph-based domain adaptation FSL (GDAFSL) method is proposed for HSI classification with limited training samples, which utilizes the graph method to guide the domain adaptation learning process in a uniformed framework. First, a novel deep residual hybrid attention network (DRHAN) is designed to extract discriminative embedded features efficiently for few-shot HSI classification. Then, a graph-based domain adaptation network (GDAN), which combines graph construction with domain adversarial strategy, is proposed to fully explore the domain correlation between source and target embedded features. By utilizing the fully explored domain correlations to guide the domain adaptation process, a domain invariant feature metric space is learned for few-shot HSI classification. Comprehensive experimental results conducted on three public HSI datasets demonstrate that GDAFSL is superior to the state-of-the-art with a small sample size.

The goal of few-shot image classification is to learn a classifier that can be well generalized to the unseen classes with a few available labeled samples. One major challenge for few-shot learning is how to conduct effective image representation for support and query images. Recently, local region-based image representation and metric learning approaches have been demonstrated effectively for few-shot classification problem. However, existing approaches generally conduct representations of image regions individually which thus lack of considering the rich spatial/structural relationships among image regions. In this paper, we propose to bridge the individual regions and exploit the structural contexts among regions via a novel Region-Graph Transformer (RGTransformer). In RGTransformer, each region aggregates the information from its neighboring regions and thus can obtain context-aware feature representations for regions. Using the proposed RGTransformer, we propose an effective metric learning model for few-shot image classification. We evaluate the proposed method on four benchmark datasets and experimental results demonstrate the effectiveness and advantages of the proposed RGTransformer.

Most graph-network-based meta-learning approaches model instance-level relation of examples. We extend this idea further to explicitly model the distribution-level relation of one example to all other examples in a 1-vs-N manner. We propose a novel approach named distribution propagation graph network (DPGN) for few-shot learning. It conveys both the distribution-level relations and instance-level relations in each few-shot learning task. To combine the distribution-level relations and instance-level relations for all examples, we construct a dual complete graph network which consists of a point graph and a distribution graph with each node standing for an example. Equipped with dual graph architecture, DPGN propagates label information from labeled examples to unlabeled examples within several update generations. In extensive experiments on few-shot learning benchmarks, DPGN outperforms state-of-the-art results by a large margin in 5% $\sim$ 12% under supervised setting and 7% $\sim$ 13% under semi-supervised setting. Code will be released.

Recent advancements have significantly improved the efficiency and effectiveness of deep learning methods for image-based remote sensing tasks. However, the requirement for large amounts of labeled data can limit the applicability of deep neural networks to existing remote sensing datasets. To overcome this challenge, few-shot learning has emerged as a valuable approach for enabling learning with limited data. While previous research has evaluated the effectiveness of few-shot learning methods on satellite-based datasets, little attention has been paid to exploring the applications of these methods to datasets obtained from Unmanned Aerial Vehicles (UAVs), which are increasingly used in remote sensing studies. In this review, we provide an up-to-date overview of both existing and newly proposed few-shot classification techniques, along with appropriate datasets that are used for both satellite-based and UAV-based data. We demonstrate few-shot learning can effectively handle the diverse perspectives in remote sensing data. As an example application, we evaluate state-of-the-art approaches on a UAV disaster scene dataset, yielding promising results. Furthermore, we highlight the significance of incorporating explainable AI (XAI) techniques into few-shot models. In remote sensing, where decisions based on model predictions can have significant consequences, such as in natural disaster response or environmental monitoring, the transparency provided by XAI is crucial. Techniques like attention maps and prototype analysis can help clarify the decision-making processes of these complex models, enhancing their reliability. We identify key challenges including developing flexible few-shot methods to handle diverse remote sensing data effectively. This review aims to equip researchers with an improved understanding of few-shot learning’s capabilities and limitations in remote sensing, while pointing out open issues to guide progress in efficient, reliable and interpretable data-efficient techniques.

SCL: Self-supervised contrastive learning for few-shot image classification

Few-shot learning aims to enable machines to recognize unseen novel classes using limited samples akin to human capabilities. Metric learning is a crucial approach to addressing this challenge, with its performance primarily dependent on the effectiveness of feature extraction and prototype computation. This article introduces an Adaptive Prototype few-shot image classification method based on Feature Pyramid (APFP). APFP employs a novel feature extraction method called FResNet, which builds upon the ResNet architecture and leverages a feature pyramid structure to retain finer details. In the 5-shot scenario, traditional methods for computing average prototypes exhibit limitations due to the typically diverse and uneven distribution of samples, where simple means may inadequately reflect such diversity. To address this issue, APFP proposes an Adaptive Prototype method (AP) that dynamically computes class prototypes of the support set based on the similarity between support set samples and query samples. Experimental results demonstrate that APFP achieves 67.98% and 85.32% accuracy in the 5-way 1-shot and 5-way 5-shot scenarios on the MiniImageNet dataset, respectively, and 84.02% and 94.44% accuracy on the CUB dataset. These results indicate that the proposed APFP method addresses the few-shot learning problem.

Few-shot learning aims to learn a classifier that classifies unseen classes well with limited labeled samples. Existing meta learning-based works, whether graph neural network or other baseline approaches in few-shot learning, has benefited from the meta-learning process with episodic tasks to enhance the generalization ability. However, the performance of meta-learning is greatly affected by the initial embedding network, due to the limited number of samples. In this paper, we propose a novel Multi-granularity Recurrent Attention Graph Neural Network (MRA-GNN), which employs Multi-granularity graph to achieve better generalization ability for few-shot learning. We first construct the Local Proposal Network (LPN) based on attention to generate local images from foreground images. The intra-cluster similarity and the inter-cluster dissimilarity are considered in the local images to generate discriminative features. Finally, we take the local images and original images as the input of multi-grained GNN models to perform classification. We evaluate our work by extensive comparisons with previous GNN approaches and other baseline methods on two benchmark datasets (i.e., miniImageNet and CUB). The experimental study on both of the supervised and semi-supervised few-shot image classification tasks demonstrates the proposed MRA-GNN significantly improves the performances and achieves the state-of-the-art results we know.