SAPENet: Self-Attention based Prototype Enhancement Network for Few-shot Learning

Few-shot object detection in sonar images faces several critical challenges, including severe image noise interference, low contrast, and variations in object scales. Dual-branch meta-learning approaches for few-shot object detection typically utilize a support branch to extract class prototypes, which are subsequently fused with query branch features to enhance detection performance. However, existing methods commonly suffer from imprecise class prototype representations, and the fixed receptive field in prototype assignment mechanisms fails to effectively accommodate the scale variability of underwater targets. To address these issues, we propose the Integrated Discriminative Prototype Distillation module, which partitions the support feature maps and employs a set of learnable embeddings to extract integrated class prototypes. This design enables diverse feature queries to focus on distinct local regions within the support features, while incorporating global residual information to achieve a balanced representation between local details and global contextual semantics. Subsequently, the Scale-Aware Prototype Assignment module is employed to perform multiscale matching between the integrated class prototypes and the query feature maps, thereby enhancing the expressiveness of feature representations. Finally, high-level features are fused via an adaptive weighting strategy, effectively enhancing class feature discriminability while suppressing noise-induced bias. Experimental results demonstrate that the proposed method achieves superior performance compared to state-of-the-art approaches on three publicly available sonar data sets, validating its effectiveness in few-shot object detection tasks for sonar images.

Contemporary deep face recognition techniques predominantly utilize the Softmax loss function, designed based on the similarities between sample features and class prototypes. These similarities can be categorized into four types: in-sample target similarity, in-sample non-target similarity, out-sample target similarity, and out-sample non-target similarity. When a sample feature from a specific class is designated as the anchor, the similarity between this sample and any class prototype is referred to as in-sample similarity. In contrast, the similarity between samples from other classes and any class prototype is known as out-sample similarity. The terms target and non-target indicate whether the sample and the class prototype used for similarity calculation belong to the same identity or not. The conventional Softmax loss function promotes higher in-sample target similarity than in-sample non-target similarity. However, it overlooks the relation between in-sample and out-sample similarity. In this paper, we propose Global Cross-Entropy loss (GCE), which promotes 1) greater in-sample target similarity over both the in-sample and out-sample non-target similarity, and 2) smaller in-sample non-target similarity to both in-sample and out-sample target similarity. In addition, we propose to establish a bilateral margin penalty for both in-sample target and non-target similarity, so that the discrimination and generalization of the deep face model are improved. To bridge the gap between training and testing of face recognition, we adapt the GCE loss into a pairwise framework by randomly replacing some class prototypes with sample features. We designate the model trained with the proposed Global Cross-Entropy loss as GFace. Extensive experiments on several public face benchmarks, including LFW, CALFW, CPLFW, CFP-FP, AgeDB, IJB-C, IJB-B, MFR-Ongoing, and MegaFace, demonstrate the superiority of GFace over other methods. Additionally, GFace exhibits robust performance in general visual recognition task.

Source-free domain adaptation with Class Prototype Discovery

In this paper, we equip Prototypical Networks (PNs) with a novel dissimilarity measure to enable discriminative feature normalization for few-shot learning. The embedding onto the hypersphere requires no direct normalization and is easy to optimize. Our theoretical analysis shows that the proposed dissimilarity measure, denoted the Squared root of the Euclidean distance and the Norm distance (SEN), forces embedding points to be attracted to its correct prototype, while being repelled from all other prototypes, keeping the norm of all points the same. The resulting SEN PN outperforms the regular PN with a considerable margin, with no additional parameters as well as with negligible computational overhead.

Accurate identification of pathogenic bacteria from images taken under light microscopy is an important role in clinical diagnosis and microbiology. Traditional deep learning methods such as Convolutional Neural Networks (CNNs) have achieved excellent performance but are based on enormous, well-annotated datasets that may not be readily available. Absence of labeled images of bacteria is one of the key challenges in real-world applications, where class imbalance and diversity of samples affect generalization. Recent years have witnessed few-shot learning being a potential paradigm for mitigating data paucity by enabling models to learn well from sparse samples. Among numerous few-shot approaches, prototypical networks have proven useful for image classification with sparse data. This paper overviews the progress in bacterial identification from light microscopy images, reviews existing classical and deep learning methods, and highlights how prototypical networks solve the shortcomings of conventional models through few-shot learning. The review concludes with open problems and future directions, including integrating fewshot learning into web-based, scalable diagnostic platforms for efficient pathogen detection.

Few-shot learning in image classification aims at learning a model to identify novel categories from a few training samples. Many few-shot learning methods have been proposed to improve the model performance. However, they still suffer from data dependencies, weak feature representation ability, and poor performance on complex tasks. To address these three issues, we propose a global and local attention-based multiscale prototypical network (called GLAP), which includes a feature extraction network and a multiscale classification head. The feature extraction network is stacked by four global attention modules (GAMs) and four local attention modules (LAMs). On the one hand, each GAM learns the global context information in the whole image by calculating the global attention map of all pixels, thereby extracting general features to alleviate data dependencies. On the other hand, each LAM learns the local correspondences between features extracted by the corresponding GAM and features extracted by pre-trained ResNet50 at different scales, which combines high-level semantic features with low-level visual features to enhance feature representation ability. Afterward, the multiscale classification head is designed, which adopts a multiscale metric-based meta-learning (MMML) training method. MMML first calculates the category prototypes of four scales for metric classification, with the aim of generating intermediate predictions of four scales. Then, MMML generates the classification result by combining the intermediate predictions of four scales, thus improving the model performance on complex tasks. Extensive experiments on three benchmark datasets demonstrate that GLAP performs better than state-of-the-art few-shot learning methods.

Recent advancements have significantly improved the efficiency and effectiveness of deep learning methods for image-based remote sensing tasks. However, the requirement for large amounts of labeled data can limit the applicability of deep neural networks to existing remote sensing datasets. To overcome this challenge, few-shot learning has emerged as a valuable approach for enabling learning with limited data. While previous research has evaluated the effectiveness of few-shot learning methods on satellite-based datasets, little attention has been paid to exploring the applications of these methods to datasets obtained from Unmanned Aerial Vehicles (UAVs), which are increasingly used in remote sensing studies. In this review, we provide an up-to-date overview of both existing and newly proposed few-shot classification techniques, along with appropriate datasets that are used for both satellite-based and UAV-based data. We demonstrate few-shot learning can effectively handle the diverse perspectives in remote sensing data. As an example application, we evaluate state-of-the-art approaches on a UAV disaster scene dataset, yielding promising results. Furthermore, we highlight the significance of incorporating explainable AI (XAI) techniques into few-shot models. In remote sensing, where decisions based on model predictions can have significant consequences, such as in natural disaster response or environmental monitoring, the transparency provided by XAI is crucial. Techniques like attention maps and prototype analysis can help clarify the decision-making processes of these complex models, enhancing their reliability. We identify key challenges including developing flexible few-shot methods to handle diverse remote sensing data effectively. This review aims to equip researchers with an improved understanding of few-shot learning’s capabilities and limitations in remote sensing, while pointing out open issues to guide progress in efficient, reliable and interpretable data-efficient techniques.

Facial skin type analysis is a critical task in several fields, including dermatology, cosmetics, and biometrics, and has been the subject of significant research in recent years. Traditional facial skin type analysis approaches rely on large, labeled datasets, which can be time-consuming and costly to collect. This study proposes a novel few-shot learning (FSL) approach for facial skin type analysis that can accurately classify skin types with limited labeled data. A diverse dataset of facial images with varying skin tones and conditions was curated. The proposed approach leverages pre-trained deep neural networks and an FSL algorithm based on prototypical networks (PNs) and matching networks (MNs) to address the challenge of limited labeled data. Importantly, this study has significant implications for improving access to dermatological care, especially in underserved populations, as many individuals are unaware of their skin type, which can lead to ineffective or even harmful skincare practices. Our approach can help individuals quickly determine their skin type and develop a personalized skincare routine based on their unique skin characteristics. The results of our experiments demonstrate the effectiveness of the proposed approach. PNs achieved the highest accuracy in the 2-way, 10-shot, 15-query scenario with an accuracy of 95.78 ± 2.79%, while MNs achieved the highest accuracy of 90.33 ± 4.10% in the 2-way, 5-shot, 10-query scenario. In conclusion, this study highlights the potential of FSL and deep neural networks to overcome the limitations of traditional approaches to facial skin analysis, offering a promising avenue for future research in this field.

Facial skin type analysis is a critical task in several fields, including dermatology, cosmetics, and biometrics, and has been the subject of significant research in recent years. Traditional facial skin type analysis approaches rely on large, labeled datasets, which can be time-consuming and costly to collect. This study proposes a novel few-shot learning (FSL) approach for facial skin type analysis that can accurately classify skin types with limited labeled data. A diverse dataset of facial images with varying skin tones and conditions was curated. The proposed approach leverages pre-trained deep neural networks and an FSL algorithm based on prototypical networks (PNs) and matching networks (MNs) to address the challenge of limited labeled data. Importantly, this study has significant implications for improving access to dermatological care, especially in underserved populations, as many individuals are unaware of their skin type, which can lead to ineffective or even harmful skincare practices. Our approach can help individuals quickly determine their skin type and develop a personalized skincare routine based on their unique skin characteristics. The results of our experiments demonstrate the effectiveness of the proposed approach. PNs achieved the highest accuracy in the 2-way, 10-shot, 15-query scenario with an accuracy of 95.78 ± 2.79%, while MNs achieved the highest accuracy of 90.33 ± 4.10% in the 2-way, 5-shot, 10-query scenario. In conclusion, this study highlights the potential of FSL and deep neural networks to overcome the limitations of traditional approaches to facial skin analysis, offering a promising avenue for future research in this field.

Our generation has seen the boom and ubiquitous advent of Internet connectivity. Adversaries have been exploiting this omnipresent connectivity as an opportunity to launch cyber attacks. As a consequence, researchers around the globe devoted a big attention to data mining and machine learning with emphasis on improving the accuracy of intrusion detection system (IDS). In this paper, we present a few-shot deep learning approach for improved intrusion detection. We first trained a deep convolutional neural network (CNN) for intrusion detection. We then extracted outputs from different layers in the deep CNN and implemented a linear support vector machine (SVM) and 1-nearest neighbor (1-NN) classifier for few-shot intrusion detection. few-shot learning is a recently developed strategy to handle situation where training samples for a certain class are limited. We applied our proposed method to the two well-known datasets simulating intrusion in a military network: KDD 99 and NSL-KDD. These datasets are imbalanced, and some classes have much less training samples than others. Experimental results show that the proposed method achieved better performances than the state-of-the-art on those two datasets.

Few-shot learning aims to learn generalized knowledge from limited data, which is a challenging study. Metric-based few-shot learning is the most widely studied direction, and most metric-based methods focus on how to learn a more robust feature embedding space or distance metric for classification. Few-shot learning is trained with a large number of tasks. Most existing methods do not realize that the role of the same sample in different learning tasks may be different, which will affect the effectiveness of classification. This paper proposes a task-adaptive feature adjustment method to obtain new features with task context information and more discrimination to solve this problem. We also introduce the Sphere manifold into few-shot learning, use the manifold metric instead of the Euclidean metric, and deduce the gradient optimization on the manifold. We propose a novel few-shot learning method with an excellent performance by combining these two innovations. We conduct several experiments on the few-shot learning benchmark datasets to demonstrate the advancement of our methods. The experimental results have significantly improved over the baseline and are highly competitive with the mainstream and effective methods.

Recent progress in machine learning has been mainly due to the availability of large amounts of annotated data used for training complex models with deep architectures. Annotating this training data becomes burdensome and creates a major bottleneck in maintaining machine-learning databases. Moreover, these trained models fail to generalize to new categories or new varieties of the same categories. This is because new categories or new varieties have data distribution different from the training data distribution. To tackle these problems, this thesis proposes to develop a family of transfer-learning techniques that can deal with different training (source) and testing (target) distributions with the assumption that the availability of annotated data is limited in the testing domain. This is done by using the auxiliary data-abundant source domain from which useful knowledge is transferred that can be applied to data-scarce target domain. This transferable knowledge serves as a prior that biases target-domain predictions and prevents the target-domain model from overfitting. Specifically, we explore structural priors that encode relational knowledge between different data entities, which provides more informative bias than traditional priors. The choice of the structural prior depends on the information availability and the similarity between the two domains. Depending on the domain similarity and the information availability, we divide the transfer learning problem into four major categories and propose different structural priors to solve each of these sub-problems. This thesis first focuses on the unsupervised-domain-adaptation problem, where we propose to minimize domain discrepancy by transforming labeled source-domain data to be close to unlabeled target-domain data. For this problem, the categories remain the same across the two domains and hence we assume that the structural relationship between the source-domain samples is carried over to the target domain. Thus, graph or hyper-graph is constructed as the structural prior from both domains and a graph/hyper-graph matching formulation is used to transform samples in the source domain to be closer to samples in the target domain. An efficient optimization scheme is then proposed to tackle the time and memory inefficiencies associated with the matching problem. The few-shot learning problem is studied next, where we propose to transfer knowledge from source-domain categories containing abundantly labeled data to novel categories in the target domain that contains only few labeled data. The knowledge transfer biases the novel category predictions and prevents the model from overfitting. The knowledge is encoded using a neural-network-based prior that transforms a data sample to its corresponding class prototype. This neural network is trained from the source-domain data and applied to the target-domain data, where it transforms the few-shot samples to the novel-class prototypes for better recognition performance. The few-shot learning problem is then extended to the situation, where we do not have access to the source-domain data but only have access to the source-domain class prototypes. In this limited information setting, parametric neural-network-based priors would overfit to the source-class prototypes and hence we seek a non-parametric-based prior using manifolds. A piecewise linear manifold is used as a structural prior to fit the source-domain-class prototypes. This structure is extended to the target domain, where the novel-class prototypes are found by projecting the few-shot samples onto the manifold. Finally, the zero-shot learning problem is addressed, which is an extreme case of the few-shot learning problem where we do not have any labeled data in the target domain. However, we have high-level information for both the source and target domain categories in the form of semantic descriptors. We learn the relation between the sample space and the semantic space, using a regularized neural network so that classification of the novel categories can be carried out in a common representation space. This same neural network is then used in the target domain to relate the two spaces. In case we want to generate data for the novel categories in the target domain, we can use a constrained generative adversarial network instead of a traditional neural network. Thus, we use structural priors like graphs, neural networks and manifolds to relate various data entities like samples, prototypes and semantics for these different transfer learning sub-problems. We explore additional post-processing steps like pseudo-labeling, domain adaptation and calibration and enforce algorithmic and architectural constraints to further improve recognition performance. Experimental results on standard transfer learning image recognition datasets produced competitive results with respect to previous work. Further experimentation and analyses of these methods provided better understanding of machine learning as well.

Existing Few-Shot Learning (FSL) methods predominantly focus on developing different types of sophisticated models to extract the transferable prior knowledge for recognizing novel classes, while they almost pay less attention to the feature learning part in FSL which often simply leverage some well-known CNN as the feature learner. However, feature is the core medium for encoding such transferable knowledge. Feature learning is easy to be trapped in the over-fitting particularly in the scarcity of the training data, and thereby degenerates the performances of FSL. The handcrafted features, such as Histogram of Oriented Gradient (HOG) and Local Binary Pattern (LBP), have no requirement on the amount of training data, and used to perform quite well in many small-scale data scenarios, since their extractions involve no learning process, and are mainly based on the empirically observed and summarized prior feature engineering knowledge. In this paper, we intend to develop a general and simple approach for generally boosting FSL via exploiting such prior knowledge in the feature learning phase. To this end, we introduce two novel handcrafted feature regression modules, namely HOG and LBP regression, to the feature learning parts of deep learning-based FSL models. These two modules are separately plugged into the different convolutional layers of backbone based on the characteristics of the corresponding handcrafted features to guide the backbone optimization from different feature granularity, and also ensure that the learned feature can encode the handcrafted feature knowledge which improves the generalization ability of feature and alleviate the over-fitting of the models. Three recent state-of-the-art FSL approaches are leveraged for examining the effectiveness of our method. Extensive experiments on miniImageNet, CIFAR-FS and FC100 datasets show that the performances of all these FSL approaches are well boosted via applying our method on all three datasets. Our codes and models have been released.

Existing cross-domain few-shot hyperspectral image (HSI) classification methods based meta-learning face two main issues: one is that the optimization of class prototypes is insufficient with a small number of samples, resulting in poor balance between inter-class discriminability and intra-class compactness; the other is the lack of dynamic coordination of different fine-grained domain distributions, leading to limited domain adaptability. To address these issues, this paper proposes an adversarial domain adaptive network based on dynamic class prototype and graph convolution (DPGC) with two core strategies to improve the classification accuracy of cross-domain few-shot HSI classification. The first strategy, dynamic class prototype, optimizes the position distribution of class prototypes by adjusting the weights of real and pseudo-labels, thereby widening inter-class distances and compressing intra-class distances. The second strategy, graph-based domain adversarial strategy, extracts local and global domain features and forces the feature extractor to generate domain-independent features through adversarial training, mitigating performance degradation caused by domain distribution differences. Experiments show that DPGC achieves the best classification performance on three hyperspectral datasets and the code is publicly available at https://github.com/tiwchui/DPGC.

A pattern-based simulation technique using wavelet analysis is proposed for the simulation (wavesim) of categorical and continuous variables. Patterns are extracted by scanning a training image with a template and then storing them in a pattern database. The dimension reduction of patterns in the pattern database is performed by wavelet decomposition at certain scale and the approximate sub-band is used for pattern database classification. The pattern database classification is performed by the k-means clustering algorithm and classes are represented by a class prototype. For the simulation of categorical variables, the conditional cumulative density function (ccdf) for each class is generated based on the frequency of the individual categories at the central node of the template. During the simulation process, the similarity of the conditioning data event with the class prototypes is measured using the L 2-norm. When simulating categorical variables, the ccdf of the best matched class is used to draw a pattern from a class. When continuous variables are simulated, a random pattern is drawn from the best matched class. Several examples of conditional and unconditional simulation with two- and three- dimensional data sets show that the spatial continuity of geometric features and shapes is well reproduced. A comparative study with the filtersim algorithm shows that the wavesim performs better than filtersim in all examples. A full-field case study at the Olympic Dam base metals deposit, South Australia, simulates the lithological rock-type units as categorical variables. Results show that the proportions of various rock-type units in the hard data are well reproduced when similar to those in the training image; when rock-type proportions between the training image and hard data differ, the results show a compromise between the two.