Few-Shot Image Classification Along Sparse Graphs

Few-shot learning (FSL) is of great significance to the field of machine learning. The ability to learn and generalize using a small number of samples is an obvious distinction between artificial intelligence and humans. In the FSL domain, most graph neural networks (GNNs) focus on transferring labeled sample information to an unlabeled query sample, ignoring the important role of semantic information during the classification process. Our proposed method embeds semantic information of classes into a GNN, creating a word embedding distribution propagation graph network (WPGN) for FSL. We merge the attention mechanism with our backbone network, use the Mahalanobis distance to calculate the similarity of classes, select the Funnel ReLU (FReLU) function as the activation function of the Transform layer, and update the point graph and word embedding distribution graph. In extensive experiments on FSL benchmarks, compared with the baseline model, the accuracy of the WPGN on the 5-way-1/2/5 shot tasks increased by 9.03, 4.56, and 4.15%, respectively.

Unlike in the field of visual scene recognition, where tremendous advances have taken place due to the availability of very large datasets to train deep neural networks, inference from medical images is often hampered by the fact that only small amounts of data may be available. When working with very small dataset problems, of the order of a few hundred items of data, the power of deep learning may still be exploited by using a pre-trained model as a feature extractor and carrying out classic pattern recognition techniques in this feature space, the so-called few-shot learning problem. However, medical images are highly complex and variable, making it difficult for few-shot learning to fully capture and model these features. To address these issues, we focus on the intrinsic characteristics of the data. We find that, in regimes where the dimension of the feature space is comparable to or even larger than the number of images in the data, dimensionality reduction is a necessity and is often achieved by principal component analysis or singular value decomposition (PCA/SVD). In this paper, noting the inappropriateness of using SVD for this setting we explore two alternatives based on discriminant analysis (DA) and non-negative matrix factorization (NMF). Using 14 different datasets spanning 11 distinct disease types we demonstrate that at low dimensions, discriminant subspaces achieve significant improvements over SVD-based subspaces and the original feature space. We also show that at modest dimensions, NMF is a competitive alternative to SVD in this setting. The implementation of the proposed method is accessible via the following link.

MetAdapt: Meta-learned task-adaptive architecture for few-shot classification

Nearest neighbor (NN) search computations are at the core of many applications such as few-shot learning, classification, and hyperdimensional computing. As such, efficient hardware support for NN search is highly desired. In-memory computing using emerging devices offers attractive solutions for NN search. Solutions based on ternary content-addressable memories (TCAMs) offer high energy and latency improvements for NN search at the expense of accuracy. In this work, we propose a novel distance function that can be natively evaluated with multi-bit content-addressable memories (MCAMs) based on ferroelectric FETs (FeFETs) to perform a single-step, in-memory NN search. We evaluate the efficacy of FeFET MCAMs in the context of few-shot learning applications with different datasets. As an example, we achieve a 78.54% accuracy for a 5-way, 5-shot classification task for the mini-ImageNet dataset (only 1.5% lower than software-based implementations) when using a 3-bit MCAM for NN search. We consider the effects of FeFET threshold voltage variations on the application accuracy and analyze the area and search energy requirements of FeFET MCAMs for accurate operations. Our results indicate that MCAMs require 2× lower area and search energy than TCAMs to achieve the same accuracy. Furthermore, we experimentally demonstrate a 2-bit implementation of FeFET MCAM using AND arrays from GLOBALFOUNDRIES to further validate the design concept.

Few-shot learning aims to recognize novel categories from just a few labeled instances. Existing metric learning-based approaches perform classifications by nearest neighbor search in the embedding space. The embedding function is a deep neural network and usually shared by all novel categories. However, these brute approaches lack a fast adaptation mechanism like meta-learning when dealing with novel categories. To tackle this, we present a novel instance-level embedding adaptation mechanism, aiming at rapidly adapting embedding deep features to improve their generalization ability in recognizing novel categories. To this end, we design an Attention Adaptation Module to pull a query instance and its corresponding class center as close as possible. Note that, each query instance is pulled closer to its corresponding class center before performing nearest neighbor classifications. This instance-level reduction of intra-class distance increases the probability of correct classifications, and thus improves the generalization ability to embed deep features and promoting the performance. The extensive experiments are conducted on two benchmark datasets: miniImageNet and CUB. Our approach yields very promising results on both datasets. In addition, in a realistic cross-domain evaluation setting, our method also achieves the-state-of-the-art performance.

The k Nearest Neighbors (KNN) algorithm has been widely applied in various supervised learning tasks due to its simplicity and effectiveness. However, the quality of KNN decision making is directly affected by the quality of the neighborhoods in the modeling space. Efforts have been made to map data to a better feature space either implicitly with kernel functions, or explicitly through learning linear or nonlinear transformations. However, all these methods use pre-determined distance or similarity functions, which may limit their learning capacity. In this paper, we propose a novel deep learning architecture, which is called the Deep Similarity-Enhanced K Nearest Neighbors (DSE-KNN), to learn an optimized similarity function of the data directly towards the goal of optimizing the KNN decision making. In other words, the type of similarity function that is used in our method is not pre-determined but rather learned to map data to a high-dimensional feature space where the accuracy of the KNN decision making is maximized. Experimental results show that DSE-KNN outperforms other common machine learning methods on classifying different types of disease datasets and predicting daily price direction of different stock ETFs.

Deep neural networks are vulnerable to adversarial attacks and hard to interpret because of their black-box nature. The recently proposed invertible network is able to accurately reconstruct the inputs to a layer from its outputs, thus has the potential to unravel the black-box model. An invertible network classifier can be viewed as a two-stage model: (1) invertible transformation from input space to the feature space; (2) a linear classifier in the feature space. We can determine the decision boundary of a linear classifier in the feature space; since the transform is invertible, we can invert the decision boundary from the feature space to the input space. Furthermore, we propose to determine the projection of a data point onto the decision boundary, and define explanation as the difference between data and its projection. Finally, we propose to locally approximate a neural network with its first-order Taylor expansion, and define feature importance using a local linear model. We provide the implementation of our method: https://github.com/juntang-zhuang/explain_invertible.

Building a good feature space is essential for the metric-based few-shot algorithms to recognize a novel class with only a few samples. The feature space is often built by Convolutional Neural Networks (CNNs). However, CNNs primarily focus on local information with the limited receptive field, and the global information generated by distant pixels is not well used. Meanwhile, having a global understanding of the current task and focusing on distinct regions of the same sample for different queries are important for the few-shot classification. To tackle these problems, we propose the Cross Non-Local Neural Network (CNL) for capturing the long-range dependency of the samples and the current task. CNL extracts the task-specific and context-aware features dynamically by strengthening the features of the sample at a position via aggregating information from all positions of itself and the current task. To reduce losing important information, we maximize the mutual information between the original and refined features as a constraint. Moreover, we add a task-specific scaling to deal with multi-scale and task-specific features extracted by CNL. We conduct extensive experiments for validating our proposed algorithm, which achieves new state-of-the-art performances on two public benchmarks.

Few-shot learning (FSL), aiming to extract new knowledge from very small amount of labeled samples, has attracted noticeable attentions recently. However, most of existing methods often fail when facing huge domain shift between seen and unseen classes. We think this should be attributed to the episode strategy which ignore utilizing support samples to induct the test classes. So in this paper, for the first time, we propose a bilevel episode strategy (BL-ES) to train a inductive graph network (IGN) that learn to both comparison and induction. Specifically, first, outer episodes in BL-ES simulate the cross-domain few-shot tasks constantly, while inner episodes learn to drive IGN to induct the common features of test classes. Then, the propsoed IGN captures the correlation among all samples to update meta points of each category in induction module. Finally, we introduce a geometrical constraint term utilizing meta points into the training loss, to update the nodes and edges in feature space. This way improves the robustness of training process. Extensive experiments show that our framework outperforms the state-of-the-art FSL alternatives, and are more suitable for real-world applications.

Personalized federated learning (PFL) learns a personalized model for each client in a decentralized manner, where each client owns private data that are not shared and data among clients are non-independent and identically distributed (i.i.d.) However, existing PFL solutions assume that clients have sufficient training samples to jointly induce personalized models. Thus, existing PFL solutions cannot perform well in a few-shot scenario, where most or all clients only have a handful of samples for training. Furthermore, existing few-shot learning (FSL) approaches typically need centralized training data; as such, these FSL methods are not applicable in decentralized scenarios. How to enable PFL with limited training samples per client is a practical but understudied problem. In this article, we propose a solution called personalized federated few-shot learning (pFedFSL) to tackle this problem. Specifically, pFedFSL learns a personalized and discriminative feature space for each client by identifying which models perform well on which clients, without exposing local data of clients to the server and other clients, and which clients should be selected for collaboration with the target client. In the learned feature spaces, each sample is made closer to samples of the same category and farther away from samples of different categories. Experimental results on four benchmark datasets demonstrate that pFedFSL outperforms competitive baselines across different settings.

Few-shot learning (FSL) aims to recognize new categories given limited training samples. The core challenge is to avoid overfitting to the minimal data while ensuring good generalization to novel classes. One mainstream method employs prototypes from visual feature extractors as classifier weight and the performance depends on the quality of the prototype. Since different categories may have similar visual features, the visual prototype has limitations. This is because existing methods only learn a simple visual feature extractor during the pre-training stage but neglect the importance of a well-developed feature space for the prototype. We introduce the Semantic Enhanced Visual Prototype framework (SEVpro) to address this issue. SEVpro refines prototype learning from the pre-training stage and serves as a versatile plug-and-play framework for all prototype-based FSL methods. Specifically, we enhance prototype discriminability by transforming semantic embeddings into the visual space, aiding in separating categories with similar visual features. For novel class learning, we leverage knowledge from base classes and incorporate semantic information to elevate prototype quality further. Meanwhile, extensive experiments on FSL benchmarks and ablation studies demonstrate the superiority of our proposed SEVpro for FSL.

The core challenge of few-shot learning is the serious overfitting problem on new tasks because of the scarcity of labeled data. Self-supervision learning can mine powerful supervision signals from the data itself to enhance the generalization performance of the model. Thus a rotation self-supervision module has been directly integrated to a few-shot learning network to alleviate the overfitting problem. However, due to the level difference or convergence speed difference in the loss function for each task, the overall model can be alternately dominated or biased by a certain task during the training stages, which has disadvantages for the main task performance. Therefore, we design a network architecture with auxiliary task learning speed equalization (LSENet) for few-shot learning. The overall model improves the generalization capability using an auxiliary task. In addition, we design a speed equalization module to constrain the decline rate of the two tasks to achieve balanced learning. Our method alleviates the overfitting problem of few-shot learning and greatly improves classification accuracy. Extensive experiments are conducted on benchmark datasets to demonstrate the effectiveness of our method.

Active learning is one form of supervised machine learning. In supervised learning, a set of labeled samples is passed to a learning algorithm for training a classifier. However, labeling large amounts of training samples can be costly and error-prone. Active learning deals with the development of algorithms that interactively select a subset of the available unlabeled samples for labeling, and aims at minimizing the labeling effort while maintaining classification performance. The key challenge for the development of so-called active learning strategies is the balance between exploitation and exploration: On the one hand, the estimated decision boundary needs to be refined in feature space regions where it has already been established, while, on the other hand, the feature space needs to be scanned carefully for unexpected class distributions. In this thesis, two approaches to active learning are presented that consider these two aspects in a novel way. In order to lay the foundations for the first one, it is proposed to express the uncertainty in class prediction of a classifier at a test point in terms of a second-order distribution. The mean of this distribution corresponds to the common estimate of the posterior class probabilities and thus is related to the distance of the test point to the decision boundary, whereas the spread of the distribution indicates the degree of exploration in the corresponding region of feature space. This allows for the evaluation of the utility of labeling a yet unlabeled point with respect to classifier improvement in a principled way and leads to a completely novel approach to active learning. The proposed strategy is then implemented and evaluated based on kernel density classification. The generic active learning strategy can be combined with any other classifier, but it performs best if the derived second-order distributions are sufficiently good approximations to the sampling distribution. Although second-order distributions for random forests are derived in this thesis, they do not approximate sufficiently well the sampling distribution and mainly allow only for the relative comparison of prediction uncertainty between test points. In order to combine the state of the art classification performance of random forests with the principal ideas of the first active learning approach, a related second approach for random forests is derived. It is, in addition, tailored to the demands in industrial optical inspection: bag-wise labeling with weak labels and strongly imbalanced classes. Moreover, an outlier detection scheme based on random forests is derived that is used by the proposed active learning algorithm. Finally, a new computational scheme for Gaussian process classification is presented. It is compared to two standard methods in geostatistics, both with respect to theoretical consistency and practical performance. The method evolved as a by-product when considering using Gaussian process models for active learning.

In this paper we introduce an extension of unsupervised nearest neighbors for embedding patterns into continuous latent spaces of arbitrary dimensionality with stochastic sampling. Distances in data space are employed as standard deviation for Gaussian sampling in latent space. Neighborhoods are preserved with the nearest neighbor data space reconstruction error. Similar to the previous unsupervised nearest neighbors (UNN) variants this approach is an iterative method that constructs a latent embedding by selecting the position with the lowest error. Further, we introduce kernel functions for computing the data space reconstruction error in a feature space that allows to better handle non-linearities. Experimental studies show that kernel unsupervised nearest neighbors (KUNN) is an efficient method for embedding high-dimensional patterns.

Bootstrapping search margin-based nearest neighbor method for qualitative spectroscopic analysis.