Match them up: visually explainable few-shot image classification

Self-supervised learning (SSL) techniques have recently been integrated into the few-shot learning (FSL) framework and have shown promising results in improving the few-shot image classification performance. However, existing SSL approaches used in FSL typically seek the supervision signals from the global embedding of every single image. Therefore, during the episodic training of FSL, these methods cannot capture and fully utilize the local visual information in image samples and the data structure information of the whole episode, which are beneficial to FSL. To this end, we propose to augment the few-shot learning objective with a novel self-supervised Episodic Spatial Pretext Task (ESPT). Specifically, for each few-shot episode, we generate its corresponding transformed episode by applying a random geometric transformation to all the images in it. Based on these, our ESPT objective is defined as maximizing the local spatial relationship consistency between the original episode and the transformed one. With this definition, the ESPT-augmented FSL objective promotes learning more transferable feature representations that capture the local spatial features of different images and their inter-relational structural information in each input episode, thus enabling the model to generalize better to new categories with only a few samples. Extensive experiments indicate that our ESPT method achieves new state-of-the-art performance for few-shot image classification on three mainstay benchmark datasets. The source code will be available at: https://github.com/Whut-YiRong/ESPT.

Revisiting metric learning for few-shot image classification

TST_MFL: Two-stage training based metric fusion learning for few-shot image classification

The goal of few-shot image classification is to learn a classifier that can be well generalized to the unseen classes with a few available labeled samples. One major challenge for few-shot learning is how to conduct effective image representation for support and query images. Recently, local region-based image representation and metric learning approaches have been demonstrated effectively for few-shot classification problem. However, existing approaches generally conduct representations of image regions individually which thus lack of considering the rich spatial/structural relationships among image regions. In this paper, we propose to bridge the individual regions and exploit the structural contexts among regions via a novel Region-Graph Transformer (RGTransformer). In RGTransformer, each region aggregates the information from its neighboring regions and thus can obtain context-aware feature representations for regions. Using the proposed RGTransformer, we propose an effective metric learning model for few-shot image classification. We evaluate the proposed method on four benchmark datasets and experimental results demonstrate the effectiveness and advantages of the proposed RGTransformer.

Our generation has seen the boom and ubiquitous advent of Internet connectivity. Adversaries have been exploiting this omnipresent connectivity as an opportunity to launch cyber attacks. As a consequence, researchers around the globe devoted a big attention to data mining and machine learning with emphasis on improving the accuracy of intrusion detection system (IDS). In this paper, we present a few-shot deep learning approach for improved intrusion detection. We first trained a deep convolutional neural network (CNN) for intrusion detection. We then extracted outputs from different layers in the deep CNN and implemented a linear support vector machine (SVM) and 1-nearest neighbor (1-NN) classifier for few-shot intrusion detection. few-shot learning is a recently developed strategy to handle situation where training samples for a certain class are limited. We applied our proposed method to the two well-known datasets simulating intrusion in a military network: KDD 99 and NSL-KDD. These datasets are imbalanced, and some classes have much less training samples than others. Experimental results show that the proposed method achieved better performances than the state-of-the-art on those two datasets.

Coarse-to-fine pseudo supervision guided meta-task optimization for few-shot object classification

Few-shot learning aims to learn a classifier that classifies unseen classes well with limited labeled samples. Existing meta learning-based works, whether graph neural network or other baseline approaches in few-shot learning, has benefited from the meta-learning process with episodic tasks to enhance the generalization ability. However, the performance of meta-learning is greatly affected by the initial embedding network, due to the limited number of samples. In this paper, we propose a novel Multi-granularity Recurrent Attention Graph Neural Network (MRA-GNN), which employs Multi-granularity graph to achieve better generalization ability for few-shot learning. We first construct the Local Proposal Network (LPN) based on attention to generate local images from foreground images. The intra-cluster similarity and the inter-cluster dissimilarity are considered in the local images to generate discriminative features. Finally, we take the local images and original images as the input of multi-grained GNN models to perform classification. We evaluate our work by extensive comparisons with previous GNN approaches and other baseline methods on two benchmark datasets (i.e., miniImageNet and CUB). The experimental study on both of the supervised and semi-supervised few-shot image classification tasks demonstrates the proposed MRA-GNN significantly improves the performances and achieves the state-of-the-art results we know.

Due to a lack of labeled samples, deep learning methods generally tend to have poor classification performance in practical applications. Few-shot learning (FSL), as an emerging learning paradigm, has been widely utilized in hyperspectral image (HSI) classification with limited labeled samples. However, the existing FSL methods generally ignore the domain shift problem in cross-domain scenes and rarely explore the associations between samples in the source and target domain. To tackle the above issues, a graph-based domain adaptation FSL (GDAFSL) method is proposed for HSI classification with limited training samples, which utilizes the graph method to guide the domain adaptation learning process in a uniformed framework. First, a novel deep residual hybrid attention network (DRHAN) is designed to extract discriminative embedded features efficiently for few-shot HSI classification. Then, a graph-based domain adaptation network (GDAN), which combines graph construction with domain adversarial strategy, is proposed to fully explore the domain correlation between source and target embedded features. By utilizing the fully explored domain correlations to guide the domain adaptation process, a domain invariant feature metric space is learned for few-shot HSI classification. Comprehensive experimental results conducted on three public HSI datasets demonstrate that GDAFSL is superior to the state-of-the-art with a small sample size.

SCL: Self-supervised contrastive learning for few-shot image classification

Few-shot learning aims to learn a classifier to recognize unseen classes with limited labeled examples. The scarcity of training data remains a challenging problem in few-shot classification. Many few-shot learning methods focus on the structure of meta-network models and there is often no dedicated study of meta-knowledge representation. In this paper, a meta-learning strategy is introduced and a meta-network NAM Net is proposed. Taking the advantage of 'learning to learn', the model acquires meta-knowledge via few-shot classification tasks and applies it to new few-shot scenarios. Specifically, the meta-learner learns the best parameters of the feature extraction and similarity metric modules. The distance between the support feature and query feature is obtained by a learnable metric function, which leads to the classification result. The base learner migrates the meta-knowledge to the target class to perform classification in a new few-shot episode. Moreover, the Normalization-based Attention Module is adapted to feature extractor to enhance meta-knowledge representation. Compared with few-shot learning benchmarks, NAM Net is effective and achieves higher accuracy in both 5-way 1-shot and 5-shot classification tasks.

Learning from the limited labeled examples, denoted as few-shot, has been successfully applied into the image detection and classification. However, their ability to learn new concepts quickly is limited. Meta-learning is one approach to address this issue, by enabling the network to learn how to learn. However, most of metric-based meta-learning few-shot models low recognition accuracy and due to the large difference in data distribution in few-shot learning, the importance of model feature extraction is more reflected. Especially, some existing methods only utilize high-level semantic feature, thus ignore the underlying information that is essential for few-shot learning. In addition, these methods adopted the fixed distance functions for feature measurement, thus they cannot fully integrate features. To solve these problems, an end-to-end, few-shot classification model (AMBR) with the adaptive multi-branch feature fusion attention is proposed in this paper. The multi-path feature residual module is designed to extract and fuse the features for making full use of the underlying information. Then, while utilizing the metric module to learn the metric between the support set and query set features, the attention module is employed to enhance the learned features. Finally, a non-linear classifier is proposed for image classification. We evaluate our method on two benchmark datasets, namely Omniglot and Mini-imagenet. Experimental results demonstrate our proposed AMBR algorithm achieves the excellent performance in these two datasets through a shallow feature extraction network, and is superior to the state-of-the-art.

Few-shot learning (FSL) is designed to equip models with the capability to quickly adapt to new, unseen domains in open-world scenarios. However, there is a notable discrepancy between the multitude of new concepts encountered in the open world and the limited scale of existing FSL studies, which focus predominantly on a small number of novel classes. This limitation hinders the practical implementation of FSL in real-world situations. To address this issue, we introduce a novel problem called Few-Shot Learning with Many Novel Classes (FSL-MNC), which expands the number of novel classes more than 500 times compared to traditional FSL settings. This new challenge presents two main difficulties: increased computational load during meta-training and reduced classification accuracy due to the larger number of classes during meta-testing. To tackle these problems, we introduce the Simple Hierarchy Pipeline (SHA-Pipeline). In response to the inefficiency of traditional Episode Meta-Learning (EML) protocols, we redesign a more efficient meta-training strategy to manage the increased number of novel classes. Moreover, to distinguish distinct semantic features across a broad array of novel classes, we effectively reconstruct and utilize class hierarchy information during meta-testing. Our experiments demonstrate that the SHA-Pipeline substantially outperforms both the ProtoNet baseline and current leading alternatives across various numbers of novel classes.

In many real-world problems, collecting a large number of labeled samples is infeasible. Few-shot learning (FSL) is the dominant approach to address this issue, where the objective is to quickly adapt to novel categories in presence of a limited number of samples. FSL tasks have been predominantly solved by leveraging the ideas from gradient-based meta-learning and metric learning approaches. However, recent works have demonstrated the significance of powerful feature representations with a simple embedding network that can outperform existing sophisticated FSL algorithms. In this work, we build on this insight and propose a novel training mechanism that simultaneously enforces equivariance and invariance to a general set of geometric transformations. Equivariance or invariance has been employed standalone in the previous works; however, to the best of our knowledge, they have not been used jointly. Simultaneous optimization for both of these contrasting objectives allows the model to jointly learn features that are not only independent of the input transformation but also the features that encode the structure of geometric transformations. These complementary sets of features help generalize well to novel classes with only a few data samples. We achieve additional improvements by incorporating a novel self-supervised distillation objective. Our extensive experimentation shows that even without knowledge distillation our proposed method can outperform current state-of-the-art FSL methods on five popular benchmark datasets.

The classification of marine plankton images is of great significance in ecological studies and environmental monitoring. In practical applications, plankton image classification faces several challenges, including sample imbalance, distinguishing between-class and within-class differences, and recognizing fine-grained features. To address these issues, we propose a few-shot self-supervised transfer learning (FSTL) framework. In FSTL, we design a new loss function that incorporates both supervised and self-supervised learning. The core of FSTL is a hybrid learning objective that integrates self-supervised contrastive learning for robust feature representation. Operating within a transfer learning paradigm, FSTL effectively adapts knowledge from head-classes to boost the few-shot classification performance on tail-classes. We applied FSTL to two datasets, in which plankton images were collected from Daya Bay and provided by the Woods Hole Oceanographic Institution (WHOI) datasets respectively. The experimental results demonstrated that our method showed better adaptability in the classification of plankton images. The findings of this study not only apply to the classification of plankton images but also offer the potential for classifying small-sample categories within long-tailed datasets.

Few-shot learning(FSL) aims to generalize model to novel categoeries by few labelled samples, which is challenging for machine. Large-scaled pretrained models, especially vision transformers achieve excellent performances benefiting from numerous and diverse data. Researchers have exploited pretrained models in few-shot classification by simply updating the whole parameters and finetuning on few samples. In this paper, we explore two methods: vision prompt tuning and a reparameterization method called ‘scaling&&shift’ to leverage pretrained models in few-shot classification. Vision prompt tuning is for vision transformer only and we first evaluate the method in few-shot setting. ‘Scaling&&shift’ is originally applied in convolution neural networks(CNN). We extend it to vision transformer. The two methods are evaluated on standard benchmarks such as miniImageNet, CUB, CIFAR-FS, clipart and sketch. The results show that ‘scaling&&shift’ reaches the same level compared to updating the whole parameters. Vision prompt tuning is 0%~5% lower than updating the whole parameters over five datasets while it has quite smaller amount of parameters updated.