From patch, sample to domain: Capture geometric structures for few-shot learning

In Hyperspectral Image (HSI) classification, acquiring large quantities of high-quality labeled samples is typically costly and impractical. Traditional deep learning methods are limited in such scenarios due to their dependence on sample quantities. To address this challenge, researchers have turned to Few-Shot Learning (FSL). Although existing FSL methods improve classification performance by enhancing domain invariance through domain adaptation, they often overlook the critical issue of high inter-class similarity and large intra-class variability. Moreover, during domain alignment, features of different categories within the same domain may become confused. To address these issues, this paper proposes a novel Domain-Invariant Few-Shot Contrastive Learning (DIFSCL) method, which combines domain adaptation and contrastive learning strategies to not only learn domain-invariant features but also significantly enhance inter-class discriminability. Based on this, we further design a multi-scale adaptive attention mechanism for a hyperspectral feature extraction network to more effectively extract and optimize generalized features within the DIFSCL framework, significantly improving intra-class consistency and inter-class discriminative capability of features. Experimental results on three widely used HSI datasets demonstrate that our method significantly outperforms existing techniques in few-shot classification tasks.

Accurate identification of pathogenic bacteria from images taken under light microscopy is an important role in clinical diagnosis and microbiology. Traditional deep learning methods such as Convolutional Neural Networks (CNNs) have achieved excellent performance but are based on enormous, well-annotated datasets that may not be readily available. Absence of labeled images of bacteria is one of the key challenges in real-world applications, where class imbalance and diversity of samples affect generalization. Recent years have witnessed few-shot learning being a potential paradigm for mitigating data paucity by enabling models to learn well from sparse samples. Among numerous few-shot approaches, prototypical networks have proven useful for image classification with sparse data. This paper overviews the progress in bacterial identification from light microscopy images, reviews existing classical and deep learning methods, and highlights how prototypical networks solve the shortcomings of conventional models through few-shot learning. The review concludes with open problems and future directions, including integrating fewshot learning into web-based, scalable diagnostic platforms for efficient pathogen detection.

Few-shot learning for surgical phase recognition: Performance and generalization in cholecystectomy.

Few-shot learning aims to learn a classifier to recognize unseen classes with limited labeled examples. The scarcity of training data remains a challenging problem in few-shot classification. Many few-shot learning methods focus on the structure of meta-network models and there is often no dedicated study of meta-knowledge representation. In this paper, a meta-learning strategy is introduced and a meta-network NAM Net is proposed. Taking the advantage of 'learning to learn', the model acquires meta-knowledge via few-shot classification tasks and applies it to new few-shot scenarios. Specifically, the meta-learner learns the best parameters of the feature extraction and similarity metric modules. The distance between the support feature and query feature is obtained by a learnable metric function, which leads to the classification result. The base learner migrates the meta-knowledge to the target class to perform classification in a new few-shot episode. Moreover, the Normalization-based Attention Module is adapted to feature extractor to enhance meta-knowledge representation. Compared with few-shot learning benchmarks, NAM Net is effective and achieves higher accuracy in both 5-way 1-shot and 5-shot classification tasks.

Limited sample collection in complex forest stand is a serious challenge for automatic high-precision classification of multiple tree species. Few-shot learning (FSL) have significant advantages in handling small-sample problems, especially in transferring domain-invariant feature representations/meta knowledge from the source domain (SD) to the target dataset (TD). However, domain shift, significant differences in ground categories between SD and TD, is commonly present in cross domain FSL. A novel TSDCFSL framework is proposed, which decouples domain-invariant feature transfer from domain specific feature extraction in a two-stage manner to solve the small-sample and domain shift problems simultaneously. Its core contribution is to align the global and related subdomain distributions of SD and TD via local maximum mean discrepancy in the pre-training, thereby enhancing the domain-invariant feature representation, which is beneficial to extracting domain specific features using FSL supervised contrastive learning, and data augmentation in the fine-tuning to improve the discriminability of TD samples. Five hyperspectral datasets including three SDs and two TDs are adopted to conduct thorough experiments. Taking the XiongAn as SD, the training set contains 5 samples per class on each TD, the overall accuracy (OA) of the optimal model on GaoFeng-A and GaoFeng-B is 90.74% and 90.02%, which is better than the SD of Chikusei and HanChuan. As the training set increases to 20 samples per class, the OA reaches the highest 98.51% and 95.26% on GaoFeng-A and GaoFeng-B, respectively. The outstanding results indicate that our TSDCFSL has great potential in training high-precision tree species classification models.

Detecting Autism Spectrum Disorder (ASD) using Eye Tracking (ET) datasets is a challenging task and has been a long-standing problem. Recently, there has been a trend of developing ASD diagnosis models based on machine learning (ML), especially deep learning techniques. In this paper, we show that these existing methods still struggle to make accurate diagnoses in few-shot learning (FSL) settings, where the data available for training is limited in amount and imbalanced in nature. To address this challenge, we propose a model, named ConASD, for effective diagnosis of ASD under the FSL setting. The proposed model is a two-stage framework: it first trains an encoder for ET images using supervised contrastive learning, followed by fine-tuning a classifier for final diagnosis. With the contrastive learning strategy, the pre-trained encoder can better capture the discriminative features of the eye-tracking images, even with limited training data, and ultimately leads to better diagnosis accuracy and better generalization to unseen data. We evaluate the proposed ConASD model using two real-world ET datasets. The results demonstrate that ConASD outperforms existing approaches, particularly in few-shot scenarios, by up-to 7% improvement in terms of F1 scores. The results in this paper highlight the potential of using contrastive learning as a powerful tool, particularly in real-world medical scenarios where class imbalance is frequent and the data is limited.

The core challenge of few-shot learning is the serious overfitting problem on new tasks because of the scarcity of labeled data. Self-supervision learning can mine powerful supervision signals from the data itself to enhance the generalization performance of the model. Thus a rotation self-supervision module has been directly integrated to a few-shot learning network to alleviate the overfitting problem. However, due to the level difference or convergence speed difference in the loss function for each task, the overall model can be alternately dominated or biased by a certain task during the training stages, which has disadvantages for the main task performance. Therefore, we design a network architecture with auxiliary task learning speed equalization (LSENet) for few-shot learning. The overall model improves the generalization capability using an auxiliary task. In addition, we design a speed equalization module to constrain the decline rate of the two tasks to achieve balanced learning. Our method alleviates the overfitting problem of few-shot learning and greatly improves classification accuracy. Extensive experiments are conducted on benchmark datasets to demonstrate the effectiveness of our method.

As one of the most common sports injuries, lateral ankle sprains often lead to chronic lateral ankle instability (CLAI), which may require ankle lateral stabilization surgery to enhance ankle stability. Postoperative rehabilitation is crucial for patients to regain pre-injury sports capabilities, yet traditional rehabilitation methods are time-consuming and costly, relying heavily on subjective and objective clinical assessments. Therefore, this study has developed a wearable intelligent telerehabilitation device designed to offer cost-effective postoperative rehabilitation progress evaluations for CLAI patients. The developed device integrates a portable sensor system including pressure insoles and inertial measurement units (IMUs) to capture gait and biomechanical data, and an evaluation algorithm employing few-shot learning model to enhance model performance with small datasets. The system was trained and tested using gait data collected from 102 patients, labelled by a professional clinician with over 20 years of surgical experiences through subjective self-reported fuctions, physical evaluation, and objective examination. In the test stage, the system demonstrated an accuracy of 0.89, recall of 0.88, specificity of 0.89, and an overall F1 score of 0.90, initially fulfills the clinical requirements. Compared to traditional machine learning models, the few-shot learning approach improved accuracy by at least 0.17 and the F1-score by 0.13. The device's cost-effectiveness, ease of use, and high repeatability make it a promising tool for at-scale both clinical and home-based rehabilitation. Besides, this study creatively introduced the few-shot learning method to the field of rehabilitation, offering a solution to address the challenge of limited high-quality data in rehabilitation studies, promoting the development of intelligent healthcare.

In Defence and Security, we are often interested in rare events and occurrences where we only have a few examples. This presents a problem for traditional machine learning approaches which typically require thousands of examples per class to learn an effective classifier. Few-shot learning techniques look to use tasks comprising of a small set of labelled images and a novel, unlabelled example, to classify the novel example. These are typically considered as N-way k-shot problems, where we have N distinct classes, with only k labelled examples per class. At the Defence Science and Technology Laboratory (Dstl) in the UK, we are looking to understand the application of few-shot learning techniques to Defence and Security problems, particularly on imagery datasets. In this paper we discuss the application of few-shot learning approaches from literature to Defence and Security problems and discuss meta-learning, one of the key types of approaches to few-shot learning. We also present experimentation on meta-learning models, baselined against a transfer learned ResNet, across a range of few-shot tasks with differing data proportions on the miniImageNet and Caltech-USCD Birds datasets. This experimentation looks to improve our understanding of the behaviour of these few-shot models on a range of data limited problems. We identify a number of challenges that require further research before few-shot approaches can be effectively applied to Defence and Security problems and further research that could increase the range of Defence and Security problems to which few-shot approaches could be applied.

Few-Shot Learning (FSL) algorithms have made substantial progress in learning novel concepts with just a handful of labelled data. To classify query instances from novel classes encountered at test-time, they only require a support set composed of a few labelled samples. FSL benchmarks commonly assume that those queries come from the same distribution as instances in the support set. However, in a realistic setting, data distribution is plausibly subject to change, a situation referred to as Distribution Shift (DS). The present work addresses the new and challenging problem of Few-Shot Learning under Support/ Query Shift (FSQS) i.e., when support and query instances are sampled from related but different distributions. Our contributions are the following. First, we release a testbed for FSQS, including datasets, relevant baselines and a protocol for a rigorous and reproducible evaluation. Second, we observe that well-established FSL algorithms unsurprisingly suffer from a considerable drop in accuracy when facing FSQS, stressing the significance of our study. Finally, we show that transductive algorithms can limit the inopportune effect of DS. In particular, we study both the role of Batch-Normalization and Optimal Transport (OT) in aligning distributions, bridging Unsupervised Domain Adaptation with FSL. This results in a new method that efficiently combines OT with the celebrated Prototypical Networks. We bring compelling experiments demonstrating the advantage of our method. Our work opens an exciting line of research by providing a testbed and strong baselines. Our code is available at https://github.com/ebennequin/meta-domain-shift.

Spectroscopic analysis and modeling face challenges due to the high-dimensional and heterogeneous nature of spectral data, compounded by complex noise interference. Traditional spectral processing methods, including feature engineering, spectral preprocessing, and data augmentation, are limited by subjective bias, spurious feature artifacts, and physical inconsistencies. Knowledge distillation (KD) facilitates the optimization of raw spectral information by transferring dark knowledge to a student. To achieve further model compression, (self-) KD (self-KD) involves discarding the large teacher model and its redundant branches. However, this may hinder the establishment of a robust knowledge flow during training, especially in the context of few-shot learning scenarios. These limitations constrain the generalization capability and practical utility of spectroscopic analytical models. To mitigate these issues, we propose an adaptive feedback cross-loop self-KD method to perform preserving spectral information optimization with satisfactory robustness. First, feature space reduction and spectral dataset expansion are automatically achieved by introducing preserving odd-even decomposition while retaining the basic physical and chemical profiles. Then, we introduce a cross-loop architecture with four-teachers to adaptively adjust spectral information during training. The effectiveness of the proposed method is demonstrated through extensive experiments on Infrared (N = 500), Ultraviolet-visible (N = 180), and Raman (N = 2,500) datasets. Our approach significantly mitigates the information degradation caused by spectral processing, thereby alleviating the overfitting issue in spectral few-shot learning. Notably, on the Raman dataset, we achieve a comparable classification accuracy of 96.8% using only 2000 spectra, while the previous result was achieved with a reference dataset of 72 000 spectra. Compared to spectral processing and previous state-of-the-art KD methods, our simple yet powerful method boosts the performance of modern convolutional neural networks across a comprehensive set of metrics, including accuracy, universality, and robustness. The codes are available at https://github.com/wanna-dd/PAD.

Enhancing engine reliability with machine learning techniques on spark plug deposition using green alcohol blend fuels on gasoline engine

With the successful development of deep learning techniques in recent years, deep neural networks have achieved excellent results in both computer vision and natural language processing by relying on large-scale datasets but still face significant challenges in solving the problem of learning from few-shot. Inspired by the ability of humans to learn to recognize objects as a way to simulate the cognitive process of learning from a small sample size, few-shot learning is a hot topic of research in deep neural networks today. It is also a significant and challenging problem. This paper first introduces the research background and definition of few-shot learning, introduces the relevant models, and summarizes and analyzes the common approaches to the problem of few-shot learning based on deep neural networks at the present stage, which are divided into four types: data augmentation, model fine-tuning, metric learning and meta-learning. Finally, popular datasets for few-shot learning are described, the paper is concluded and future research directions are discussed.

Prognostics and Health Management (PHM) is vital for complex engineering systems, yet its data-driven solutions are often hampered by the "small data problem"—a scarcity of labeled fault data in industrial settings. This limitation restricts the training and generalization of machine learning models and is compounded by varying operational conditions that reduce the relevance of historical data and pre-trained models. This research introduces a research framework to tackle these small data challenges in PHM. The primary objective is to develop a robust and adaptable PHM methodology by enhancing and synergistically integrating similarity-based Few-Shot Learning (FSL) with large-scale pre-trained time-series models. The research will focus on two main thrusts. First, it aims to improve the generalization capabilities of FSL frameworks by addressing limitations such as noise robustness, domain shift adaptability, and generalization to novel faults across diverse PHM domains. This involves developing noise-robust feature extraction, integrating domain adaptation techniques, and exploring expressive similarity metrics. Second, the study will investigate the effective adaptation of state-of-the-art pre-trained time-series models (e.g., TimesNet) for PHM tasks under data scarcity, focusing on efficient fine-tuning and synergistic integration with the enhanced FSL approaches. The author's prior success in a PHM data challenge using a similarity-based method for spacecraft systems provides preliminary validation. This research is expected to deliver an enhanced PHM framework for high-accuracy diagnostics with limited data, contributing generalized FSL models, systematic methods for leveraging pre-trained models in PHM, and advancing the practical deployment of intelligent PHM solutions.

In this paper, we investigate the feasibility of applying few-shot learning algorithms to a speech task. We formulate a user-defined scenario of spoken term classification as a few-shot learning problem. In most few-shot learning studies, it is assumed that all the N classes are new in a N-way problem. We suggest that this assumption can be relaxed and define a N+M-way problem where N and M are the number of new classes and fixed classes respectively. We propose a modification to the Model-Agnostic Meta-Learning (MAML) algorithm to solve the problem. Experiments on the Google Speech Commands dataset show that our approach outperforms the conventional supervised learning approach and the original MAML.