Enhancing Few-shot Learning Performance with Boosting on Transformers: Experiments on Sentiment Analysis Tasks

Prognostics and Health Management (PHM) is vital for complex engineering systems, yet its data-driven solutions are often hampered by the "small data problem"—a scarcity of labeled fault data in industrial settings. This limitation restricts the training and generalization of machine learning models and is compounded by varying operational conditions that reduce the relevance of historical data and pre-trained models. This research introduces a research framework to tackle these small data challenges in PHM. The primary objective is to develop a robust and adaptable PHM methodology by enhancing and synergistically integrating similarity-based Few-Shot Learning (FSL) with large-scale pre-trained time-series models. The research will focus on two main thrusts. First, it aims to improve the generalization capabilities of FSL frameworks by addressing limitations such as noise robustness, domain shift adaptability, and generalization to novel faults across diverse PHM domains. This involves developing noise-robust feature extraction, integrating domain adaptation techniques, and exploring expressive similarity metrics. Second, the study will investigate the effective adaptation of state-of-the-art pre-trained time-series models (e.g., TimesNet) for PHM tasks under data scarcity, focusing on efficient fine-tuning and synergistic integration with the enhanced FSL approaches. The author's prior success in a PHM data challenge using a similarity-based method for spacecraft systems provides preliminary validation. This research is expected to deliver an enhanced PHM framework for high-accuracy diagnostics with limited data, contributing generalized FSL models, systematic methods for leveraging pre-trained models in PHM, and advancing the practical deployment of intelligent PHM solutions.

The massive growth in user reviews on the online travel agent (OTA) website can be automatically processed using sentiment analysis to understand consumer satisfaction or feedback. Sentiment analysis is commonly implemented as a sentiment classification task by applying classical machine learning and deep learning algorithms. However, implementing both strategies has a significant challenge in providing a reliable labeled dataset since labeling is time-consuming and highly resource-intensive. Therefore, this study aims to compare the performance of two learning methods: semi-supervised learning (SSL) and few-shot learning (FSL), since there is still no direct, controlled comparison between both methods. SSL is a learning method that builds a generalized classification model as a refined model based on automatically generated additional labeled data. In contrast, FSL is a learning method that enables a generalized pre-trained model to predict unlabeled data using only a few labeled samples per class. This study evaluates the self-training method on SSL, and the implemented FSL algorithm is Sentence Transformer Fine-Tuning (SETFIT). The results show that implementing FSL (employing only 16 labeled training samples) outperforms SSL with an accuracy improvement of 9.5%. The implementation of SETFIT is very promising as a solution to overcome the limited amount of labeled data in the classification task. Moreover, SETFIT is more adaptable to various low-resource language domains than other, more data-intensive learning approaches.

Modality translation-based multimodal sentiment analysis under uncertain missing modalities

The advent of pre-trained language models has directed a new era of Natural Language Processing (NLP), enabling us to create powerful language models. Among these models, Transformer-based models like BERT have grown in popularity due to their cutting-edge effectiveness. However, these models heavily rely on resource-intensive languages, forcing other languages into multilingual models(mBERT). The two fundamental challenges with mBERT become significantly more challenging in a resource-constrained language like Bangla. It was trained on a limited and organized dataset and contained weights for all other languages. Besides, current research on other languages suggests that a language-specific BERT model will exceed multilingual ones. This paper introduces Bangla-BERT^a, a monolingual BERT model for the Bangla language. Despite the limited data available for NLP tasks in Bangla, we perform pre-training on the largest Bangla language model dataset, BanglaLM, which we constructed using 40 GB of text data. Bangla-BERT achieves the highest results in all datasets and vastly improves the state-of-the-art performance in binary linguistic classification, multilabel extraction, and named entity recognition, outperforming multilingual BERT and other previous research. The pre-trained model is assessed against several non-contextual models such as Bangla fasttext and word2vec the downstream tasks. Finally, this model is evaluated by transfer learning based on hybrid deep learning models such as LSTM, CNN, and CRF in NER, and it is observed that Bangla-BERT outperforms state-of-the-art methods. The proposed Bangla-BERT model is assessed by using benchmark datasets, including Banfakenews, Sentiment Analysis on Bengali News Comments, and Cross-lingual Sentiment Analysis in Bengali. Finally, it is concluded that Bangla-BERT surpasses all prior state-of-the-art results by 3.52%, 2.2%, and 5.3%.

Purpose: This paper provides a comprehensive review of current methods, progress, and open challenges in Sentiment Analysis (SA)—a computational discipline for extracting and interpreting opinions, sentiments, and emotions from text. It aims to identify the limitations of existing approaches and explore pathways toward more generalizable, explainable, and ethically robust sentiment analysis models. Design/Methodology/Approach: The study synthesizes insights from a tertiary review (Lighart et al., 2021), which aggregates findings from 14 systematic literature reviews and mapping studies, and a domain-specific review (Sweta, 2024) focusing on SA in educational contexts. A thematic analysis integrates methodological evolution, major challenges, and emerging trends to produce a consolidated overview of the field’s trajectory. Findings: Results indicate a clear shift from lexicon-based and traditional machine learning approaches to deep learning and transformer-based architectures (e.g., LSTM, CNN, BERT, GPT). Despite significant progress, persistent issues remain—such as domain and language dependency, contextual subtlety (sarcasm, irony), data imbalance, lack of explainability, and ethical concerns regarding privacy. New trends, including cross-domain transfer learning, explainable AI (XAI), multimodal sentiment analysis, and real-time processing, hold promise for overcoming these barriers. Research Limitations/Implications: The review highlights the need for standardized datasets, cross-lingual benchmarks, and interdisciplinary collaboration between NLP researchers and domain experts to enhance model robustness and ethical compliance. Originality/Value: By integrating high-level evidence from tertiary research with practical insights from domain-specific studies, this paper outlines the current landscape and future directions for building generalizable, transparent, and ethically grounded sentiment analysis systems.

Audio sentiment analysis is a rapidly evolving field that seeks to uncover emotional insights from audio signals, with broad applications in technology, healthcare, and user experience design. This paper presents a detailed review of recent advancements, highlighting key techniques, ongoing challenges, and real-world applications that illustrate the potential of this technology. The Literature Review provides an integrated analysis of significant findings across various studies, identifying core trends and unique contributions. Finally, the Conclusion explores promising future research directions, laying the groundwork for continued innovation in extracting sentiment from sound. Keywords Audio Sentiment Analysis, Sentiment Classification, Deep Learning Techniques, Feature Extraction, Transfer Learning Ensemble Methods, Multi-Task Learning, Challenges in Sentiment Analysis, Applications of Sentiment Analysis, Literature Review of Audio Sentiment Analysis

Since 2020, when the coronavirus epidemic was at its peak, the Indonesian Ministry of Health's social media accounts have been constantly followed by a big number of individuals. The Indonesian Ministry of Health account is a fantastic resource for social media users, particularly Twitter users. The Republic of Indonesia's Ministry of Health's Twitter account publishes a wide range of content at random. As a result, it is usually difficult for Twitter users to determine the type of information provided by the Ministry of Health of the Republic of Indonesia. The positive and negative responses of Twitter users to material released by the Indonesian Ministry of Health's Twitter account are frequently noted. The decision tree algorithm is tree-based, similar to the extreme gradient boosting method (XGBoost). The extreme gradient boosting approach has been successfully implemented with high performance in the classification process. This classification is separated into two primary categories: general and essential information categorization and sentiment analysis, which is classified into three classes: positive, neutral, and negative. Both the classification work and the sentiment analysis produced outstanding accuracy levels. Based on 2243 tweets, an accuracy rate of 89.35% has been achieved for classification, supported by a precision of 88.76% and a recall value of 88.58% when using 80 data training and 20 data testing. Similarly, the maximum accuracy in sentiment analysis was achieved utilizing the same 80-20 data partitioning, with a 91.22% accuracy rate. Using 304 comments data, accuracy was calculated to be 89.17% and recall was calculated to be 89.06%. It's worth noting that an 80-20 split for training and testing consistently produced the best results for both the sentiment analysis and classification tasks.

Sentiment analysis, a critical task in natural language processing, aims to automatically identify and classify the sentiment expressed in textual data. Aspect-level sentiment analysis focuses on determining sentiment at a more granular level, targeting specific aspects or features within a piece of text. In this paper, we explore various techniques for sentiment analysis, including traditional machine learning approaches and state-of-the-art deep learning models. Additionally, deep learning techniques has been utilized to identifying and extracting specific aspects from text, addressing aspect-level ambiguity, and capturing nuanced sentiments for each aspect. These datasets are valuable for conducting aspect-level sentiment analysis. In this article, we explore a language model based on pre-trained deep neural networks. This model can analyze sequences of text to classify sentiments as positive, negative, or neutral without explicit human labeling. To evaluate these models, data from Twitter's US airlines sentiment database was utilized. Experiments on this dataset reveal that the BERT, RoBERTA and DistilBERT model outperforms than the ML based model in accuracy and is more efficient in terms of training time. Notably, our findings showcase significant advancements over previous state-of-the-art methods that rely on supervised feature learning, bridging existing gaps in sentiment analysis methodologies. Our findings shed light on the advancements and challenges in sentiment analysis, offering insights for future research directions and practical applications in areas such as customer feedback analysis, social media monitoring, and opinion mining.

Prompt-based tuning for pre-trained language models (PLMs) has shown its effectiveness in few-shot learning. Typically, prompt-based tuning wraps the input text into a cloze question. To make predictions, the model maps the output words to labels via a verbalizer, which is either manually designed or automatically built. However, manual verbalizers heavily depend on domain-specific prior knowledge and human efforts, while finding appropriate label words automatically still remains challenging.In this work, we propose the prototypical verbalizer (ProtoVerb) which is built directly from training data. Specifically, ProtoVerb learns prototype vectors as verbalizers by contrastive learning. In this way, the prototypes summarize training instances and are able to enclose rich class-level semantics. We conduct experiments on both topic classification and entity typing tasks, and the results demonstrate that ProtoVerb significantly outperforms current automatic verbalizers, especially when training data is extremely scarce. More surprisingly, ProtoVerb consistently boosts prompt-based tuning even on untuned PLMs, indicating an elegant non-tuning way to utilize PLMs. Our codes are avaliable at https://github.com/thunlp/OpenPrompt.

Documents with complex sentiment expressions generally pose great challenges in sentiment analysis. This paper proposes a statistical framework to improve sentiment classification within multiscale sentences or paragraphs. A Set of Sentiment Parts (SSP) is first introduced to express sentiment features in different contexts of varying scales. A statistic combination is then determined by analyzing canonical correlations on Riemannian manifolds. A metric learning method is designed to keep the orthogonality within Riemannian point pairs. The nearest neighbor (NN) method is finally used to classify sentiments of SSP. Promising results on various sentiment analysis data sets demonstrate the effectiveness of the proposed method.

Nowadays, with the rapid development of social networks, community-oriented Web sentiment analysis technology has gradually become a hot topic in the field of data mining. Being concise and flexible, Chinese microblog poses new challenges for sentiment analysis. This paper proposes an approach to classify Chinese microblog sentiments into positive and negative by the plain Naive Bayes (NB) and Support Vector Machine (SVM). Based on data preprocessing, sentiment lexicon construction, combining element of users’ comments, this research posit this Comments-attached Microblog Sentiment Classification, which is a novel method of attaching microblog users’ comments to the target microblog in order to improve the accuracy of sentiment classification. The experiment proves the vitality of this method and the advancement of the indecency from the way of language expressions.KeywordsChinese Microblogsentiment classificationmachine learninguser’s comments

Under the background of the gradual development and popularization of mobile Internet information technology, this paper realizes network public opinion monitoring and emotion analysis based on the deep learning method, aiming at the research needs of people's ideological changes and emotional trends. Aiming at the shortcomings of sentiment dictionaries or machine learning methods in sentiment analysis tasks, this paper builds a sentiment classification model based on deep learning methods. First, the current main text preprocessing methods are introduced, and then a sentiment classification model, BCBL, is proposed, combining BERT, CNN, and Bi LSTM. Compared with traditional models, BCBL can better complete text sentiment classification tasks on standard datasets. Next, in view of the problem that BCBL does not consider the distribution of vocabulary weights, an attention mechanism is introduced to improve BCBL, and then the BCBL-Att model is proposed. Set up multiple sets of comparative experiments again and find that the classification effect and overall performance of BCBL-Att on standard datasets are better than BCBL, indicating that BCBL-Att has more advantages in text sentiment classification tasks.

A comprehensive survey on sentiment analysis: Approaches, challenges and trends

The rapid growth of Internet-based applications, such as social media platforms and blogs, has resulted in comments and reviews concerning day-to-day activities. Sentiment analysis is the process of gathering and analyzing people’s opinions, thoughts, and impressions regarding various topics, products, subjects, and services. People’s opinions can be beneficial to corporations, governments, and individuals for collecting information and making decisions based on opinion. However, the sentiment analysis and evaluation procedure face numerous challenges. These challenges create impediments to accurately interpreting sentiments and determining the appropriate sentiment polarity. Sentiment analysis identifies and extracts subjective information from the text using natural language processing and text mining. This article discusses a complete overview of the method for completing this task as well as the applications of sentiment analysis. Then, it evaluates, compares, and investigates the approaches used to gain a comprehensive understanding of their advantages and disadvantages. Finally, the challenges of sentiment analysis are examined in order to define future directions.

Everyone in this rapid evolution generation makes use of online resources. There is a huge amount of textual content available online from several sources, including social media apps, e-commerce sites, movie recommendation system, etc. Everyone in the modern society provides a variety of reviews for a specific product or item. An effective and smart technique is required that can analyze and provide the polarity of this textual data. since it is impossible to manually review such a vast volume of user-generated data. For this user-generated data, automatic sentiment classification is possible using a variety of tools and methods. Opinion mining and sentiment classification benefit from the dependable and accuracy of machine learning. Moreover, the sentiment analysis and evaluation process are fraught with difficulties. These complexities make it difficult to interpret sentiments accurately and figure out the right sentiment polarity. This article describes an overview on sentiment analysis(SA) applications, challenges, and issues encountered by researchers, with the goal of providing a systematic review on sentiment analysis and related areas for future research.