Few-Shot Emotion Recognition in Conversation with Sequential Prototypical Networks

Emotion recognition in conversation (ERC) faces two major challenges: biased predictions and poor calibration. Classifiers often disproportionately favor certain emotion categories, such as neutral, due to the structural complexity of classifiers, the subjective nature of emotions, and imbalances in training datasets. This bias results in poorly calibrated predictions where the model’s predicted probabilities do not align with the true likelihood of outcomes. To tackle these problems, we introduce the application of conformal prediction (CP) into ERC tasks. CP is a distribution-free method that generates set-valued predictions to ensure marginal coverage in classification, thus improving the calibration of models. However, inherent biases in emotion recognition models prevent baseline CP from achieving a uniform conditional coverage across all classes. We propose a novel CP variant, class spectrum conformation, which significantly reduces coverage bias in CP methods. The methodologies introduced in this study enhance the reliability of prediction calibration and mitigate bias in complex natural language processing tasks.

In this paper, we equip Prototypical Networks (PNs) with a novel dissimilarity measure to enable discriminative feature normalization for few-shot learning. The embedding onto the hypersphere requires no direct normalization and is easy to optimize. Our theoretical analysis shows that the proposed dissimilarity measure, denoted the Squared root of the Euclidean distance and the Norm distance (SEN), forces embedding points to be attracted to its correct prototype, while being repelled from all other prototypes, keeping the norm of all points the same. The resulting SEN PN outperforms the regular PN with a considerable margin, with no additional parameters as well as with negligible computational overhead.

Accurate identification of pathogenic bacteria from images taken under light microscopy is an important role in clinical diagnosis and microbiology. Traditional deep learning methods such as Convolutional Neural Networks (CNNs) have achieved excellent performance but are based on enormous, well-annotated datasets that may not be readily available. Absence of labeled images of bacteria is one of the key challenges in real-world applications, where class imbalance and diversity of samples affect generalization. Recent years have witnessed few-shot learning being a potential paradigm for mitigating data paucity by enabling models to learn well from sparse samples. Among numerous few-shot approaches, prototypical networks have proven useful for image classification with sparse data. This paper overviews the progress in bacterial identification from light microscopy images, reviews existing classical and deep learning methods, and highlights how prototypical networks solve the shortcomings of conventional models through few-shot learning. The review concludes with open problems and future directions, including integrating fewshot learning into web-based, scalable diagnostic platforms for efficient pathogen detection.

Abstract—An Emotion Recognition-Based Activity Recommendation System aims at providing users with adequate activity recommendations based on emotional states using the latest developments within the scope of emotion recognition technology. This project would apply speech emotion recognition techniques, specifically focusing on the most current state of-the-art methods in Triangular Region Cut-Mix augmentation for the enhancement of accuracy of emotion classification while preserving audio spectrogram information related to key emotions. Furthermore, it involves a dual learning framework integrating Emotion Recognition in Conversation and Emotional Response Generation for a richer emotional analysis. This system identifies the user’s emotions from the speech input and suggests the appropriate activities to engage the users, which is of prime interest in terms of mental wellness and betterment in the user’s experience. Some of the potential enhancements would be multimodal integration, adaptive data augmentation, and real-time detection to give more extensive and interactive recommendations. That way, there would be a highly developed user-centered solution that could help most areas, from mental support and therapy to entertainment programs individually designed based on user interest. Index Terms—Speech Emotion Recognition (SER), Triangular Region Cut-Mix, Emotion Recognition in conversation(ERC) Emotional Response Generation(ERG), Multimodal Integration Transfer Learning, Data Augmentation, Natural Language Processing (NLP) Activity Recommendation , Personalized Recommendations.

Multimodal emotion recognition in conversation (MERC) involves predicting the emotion category of a conversation on the basis of textual, acoustic, and visual modalities. Information from these diverse modalities can reinforce each other to enhance the accuracy of emotion prediction. However, some information modalities may be absent in real-world applications and information from various modalities may be difficult to integrate. Therefore, a suitable strategy is required to compensate for missing modalities by using information from the available modalities and prioritizing important information. Consequently, this study developed a graph convolutional network (GCN) model with a feature compensation module and dual-channel second-order pooling module for MERC. This model initially uses a GCN to compensate for missing features by aggregating features corresponding to the same utterance node. Subsequently, it applies dual-channel second-order pooling to sift through and integrate all features. Empirical evaluations of the proposed model against other baseline models on two benchmark datasets, namely the IEMOCAP and MELD datasets, indicated that the proposed model outperformed the other models; this finding underscores the effectiveness of the proposed model in recognizing the emotions expressed in multimodal dialogue data.

Multimodal Emotion Recognition in Conversation MERC holds significant importance in Natural Language Processing due to its broad range of applications. However, existing methods still face challenges in addressing the highly imbalanced class problem and extracting robust representations for complex conversational scenarios. which leads to a decrease in the generalization ability of the model and an inability to effectively recognize minority emotion classes. To address these challenges, a cause-based supervised contrastive learning framework with adversarial sample-label CaSCLA is proposed in this paper. Specifically, we employ a modality balancing technique to fuse the multimodal features, which are then fed into a novel causal-aware network to effectively capture the underlying causal relationships within dialogues. Besides, a supervised contrastive learning with adversarial sample-label method is proposed to alleviate the class imbalance problem by learning label representations and optimizing the similarity between sample features and label embeddings. Furthermore, CaSCLA applies an adversarial samples training strategy, constructing additional positive sample-label pairs to enhance the diversity of the data and increase the robustness of the model. Extensive experiments on the IEMOCAP and MELD benchmark datasets demonstrate that CaSCLA achieves competitive performance.

Facial skin type analysis is a critical task in several fields, including dermatology, cosmetics, and biometrics, and has been the subject of significant research in recent years. Traditional facial skin type analysis approaches rely on large, labeled datasets, which can be time-consuming and costly to collect. This study proposes a novel few-shot learning (FSL) approach for facial skin type analysis that can accurately classify skin types with limited labeled data. A diverse dataset of facial images with varying skin tones and conditions was curated. The proposed approach leverages pre-trained deep neural networks and an FSL algorithm based on prototypical networks (PNs) and matching networks (MNs) to address the challenge of limited labeled data. Importantly, this study has significant implications for improving access to dermatological care, especially in underserved populations, as many individuals are unaware of their skin type, which can lead to ineffective or even harmful skincare practices. Our approach can help individuals quickly determine their skin type and develop a personalized skincare routine based on their unique skin characteristics. The results of our experiments demonstrate the effectiveness of the proposed approach. PNs achieved the highest accuracy in the 2-way, 10-shot, 15-query scenario with an accuracy of 95.78 ± 2.79%, while MNs achieved the highest accuracy of 90.33 ± 4.10% in the 2-way, 5-shot, 10-query scenario. In conclusion, this study highlights the potential of FSL and deep neural networks to overcome the limitations of traditional approaches to facial skin analysis, offering a promising avenue for future research in this field.

Facial skin type analysis is a critical task in several fields, including dermatology, cosmetics, and biometrics, and has been the subject of significant research in recent years. Traditional facial skin type analysis approaches rely on large, labeled datasets, which can be time-consuming and costly to collect. This study proposes a novel few-shot learning (FSL) approach for facial skin type analysis that can accurately classify skin types with limited labeled data. A diverse dataset of facial images with varying skin tones and conditions was curated. The proposed approach leverages pre-trained deep neural networks and an FSL algorithm based on prototypical networks (PNs) and matching networks (MNs) to address the challenge of limited labeled data. Importantly, this study has significant implications for improving access to dermatological care, especially in underserved populations, as many individuals are unaware of their skin type, which can lead to ineffective or even harmful skincare practices. Our approach can help individuals quickly determine their skin type and develop a personalized skincare routine based on their unique skin characteristics. The results of our experiments demonstrate the effectiveness of the proposed approach. PNs achieved the highest accuracy in the 2-way, 10-shot, 15-query scenario with an accuracy of 95.78 ± 2.79%, while MNs achieved the highest accuracy of 90.33 ± 4.10% in the 2-way, 5-shot, 10-query scenario. In conclusion, this study highlights the potential of FSL and deep neural networks to overcome the limitations of traditional approaches to facial skin analysis, offering a promising avenue for future research in this field.

Understanding emotions in conversation is a challenging task, as the sentences often have an implied meaning that is not generally understood in isolation. Efficient use of contextual information is essential for emotion recognition in conversations. Many published datasets provide contextual information for situations such as text-based online messaging, chatbots, and movie dialogues. However, such dialogue-based datasets are collected by selecting ideal conversational situations and thus do not include many variations in dialogue length and number of participants. Therefore, such datasets may not be applicable for emotion recognition in text-based movie transcripts, where scenes contain variations in the number of speakers and length of spoken sentences. We present a conversation dataset based on the Korean television show transcripts to analyze the emotions in presence of scene context. The Korean Drama Scene Transcript dataset for Emotion Recognition (KD-EmoR) is a text-based conversation dataset. We analyze three classes of complex emotions: euphoria, dysphoria, and neutral, in the scenes of a television drama to build a publicly available dataset for further research. We developed a context-aware deep learning model to classify emotions using the speaker-level context and scene context and achieved an F1-score of 0.63 on the proposed dataset.

Plant and crop production plays a vital role in the overall development of any country. Sugarcane is one of the most produced crops in Australia. Sugarcane plants are vulnerable to various leaf diseases. Early leaf disease detection is crucial for agricultural productivity. Traditional machine learning often relies on handcrafted features, struggles with high-dimensional data, and has scalability issues. Deep learning models overcomes these problems, still there are some challenges, such as the overfitting issue, the need for a large dataset, and less adaptability to new classes. This research addresses the challenge of limited dataset availability of sugarcane leaf diseases by employing few shot learning (FSL) and episodic training. This research proposes a Prototypical Network (PN) based few-shot learning technique to classify sugarcane leaf diseases. The proposed technique uses pretrained deep learning models for feature extraction, such as MobileNet, ResNet, EfficientNet, and PN based FSL to learn from fewer training examples. The experimental results show that the PN with a backbone of ResNet achieved the highest test accuracy of 87 %. The analysis of the experimental results indicates that PN based few shot learning with deep learning models as backbone outperforms standalone pre-trained deep learning models while using limited number of sugarcane images.

SAPENet: Self-Attention based Prototype Enhancement Network for Few-shot Learning

IntroductionKnee injuries, especially Anterior Cruciate Ligament (ACL) tears and meniscus tears, are becoming increasingly common and can severely restrict mobility and quality of life. Early diagnosis is essential for effective treatment and for preventing long-term complications such as knee osteoarthritis. While deep learning approaches have shown promise in identifying knee injuries from MRI scans, they often require large amounts of labeled data, which can be both scarce and privacy-sensitive.MethodsThis paper analyses a hybrid methodology that integrates few-shot learning with federated learning for the diagnosis of knee injuries using MRI scans. The proposed model used a 3DResNet50 architecture as the backbone to enhance both feature extraction and embedding representation. A combined Centralized and Federated Few-Shot Learning Framework is analysed to leverage episodic-intermittent training strategy based on Prototypical Networks. The model is trained incorporating Stochastic Gradient Descent (SGD), Cross-Entropy Loss, and a MultiStep Learning Rate scheduler to enhance few-shot classification. This model also addressed the challenge of limited annotated data ensuring patient data privacy through distributed learning across multiple regions.ResultsThe models performance was evaluated on the MRNet dataset for multi-label classification. In the centralized setting, the model achieved accuracies of 85.3% on axial views, 82.1% on sagittal views, and 71% on coronal views. The propose work attained accuracies as 83% (axial), 83.9% (sagittal), and 65% (coronal), demonstrating the framework’s effectiveness across different learning configurations.DiscussionThe proposed method outperforms in diagnostic accuracy, generalization across MRI planes, and patient privacy via federated learning. However, it faces limitations, including lower coronal view performance and high computational demands due to its complex architecture.

ABSTRACTBreast cancer (BC) detection based on mammogram images is still an open issue, particularly when there is little annotated data. Combining few‐shot learning (FSL) with transfer learning (TL) has been identified as a potential solution to overcome this problem due to its ability to learn from a few examples while producing robust features for classification. The objective of this study is to use and analyze FSL integrated with TL to enhance the classification accuracy and generalization ability in a limited dataset. The proposed approach integrates the FSL models (prototypical networks, matching networks, and relation networks) with the TL procedures. The models are trained using a small set of samples with annotation and can be assessed using various performance metrics. The models were trained and compared to the TL and the state‐of‐the‐art methods regarding accuracy, precision, recall, F1‐score, and area under the ROC curve (AUC). The models proved to be effective when integrated, and the relation networks model was the most accurate, with an accuracy of 95.6% and an AUC of 0.970. The models provided higher accuracy, recall, and F1‐scores, especially in the case of discerning between normal, benign, and malignant cases, as compared to TL traditional techniques and the various recent state‐of‐the‐art techniques. This integrated approach gives high efficiency, accuracy, and scalability to the whole BC detection process, and it has potential for further medical imaging domains. Future research will explore hyperparameter tuning and incorporating electronic health record systems to enhance diagnostic precision and individualized care.

E-commerce in indonesia is expected to keep expanding due to the high consumer spending habits of the indonesian population. Online transactions can be conducted through applications or websites accessible via smartphones, with tokopedia being one of the most prominent e-commerce platforms. The scale used in this study is the likert scale. The population in this study is tokopedia users in surabaya. Taking into account suggestion number 3 on determining the number of samples by Hair et al. (2010), total sample taken was 160. In collecting the data, the researcher distributed questionnaires by using google form to 160 users of tokopedia in surabaya city. The result of the study showed that after sales services and customer ratings and reviews had positive and significant affect toward customer purchase intention. Besides, after sales services and customer ratings and reviews had positive and significant affect toward customer loyalty. After sales services and customer ratings and reviews had positive and significant affect toward customer satisfaction. After sales services and customer ratings and reviews has a significant indirect effect on customer purchase intention through customer satisfaction. After sales services and and customer ratings and reviews has a significant indirect effect on customer loyalty through customer satisfaction.

Few-Shot Learning (FSL) has gained significant attention in fault diagnosis due to its ability to classify faults with limited labeled data. Metric learning-based approaches, such as prototypical networks, have demonstrated effectiveness in this domain by computing class prototypes from support examples. However, these methods often rely on high-dimensional embeddings, which can lead to computational inefficiencies and overfitting, particularly in low-data regimes. This work introduces the Regularized Embedding Memory Book (REMB), a novel module designed to improve prototype estimation and enhance classification accuracy in FSL for fault diagnosis. The REMB module constructs a memory book of embeddings for each class and refines prototype computation by selecting the most representative embeddings. Additionally, we incorporate a set of regularization techniques to improve generalization and robustness, particularly in noisy environments. We evaluate REMB on four benchmark datasets commonly used in fault diagnosis: CWRU, JNU, PU, and an additional dataset with varying training sample sizes. The experimental results demonstrate that REMB improves classification performance, particularly in low-data scenarios, and enhances model calibration. Furthermore, we analyze the impact of noise adaptation and the contribution of different regularization terms through an ablation study. Our findings suggest that the proposed approach provides a robust and efficient solution for few-shot fault diagnosis, outperforming conventional metric learning methods.