Korean Drama Scene Transcript Dataset for Emotion Recognition in Conversations

Context information brings new opportunities for efficient and effective system resource management of mobile devices. In this work, we focus on the use of context information to achieve energy-efficient, ubiquitous wireless connectivity. Our field-collected data show that the energy cost of network interfaces poses a great challenge to ubiquitous connectivity, despite decent availability of cellular networks. We propose to leverage the complementary strengths of Wi-Fi and cellular interfaces by automatically selecting the most efficient one based on context information. We formulate the selection of wireless interfaces as a statistical decision problem. The challenge is to accurately estimate Wi-Fi network conditions without powering up the network interface. We explore the use of different context information, including time, history, cellular network conditions, and device motion, to statistically estimate Wi-Fi network conditions with negligible overhead. We evaluate several context-based algorithms for the estimation and prediction of current and future network conditions. Simulations using field-collected traces show that our network estimation algorithms can improve the average battery lifetime of a commercial mobile phone for an ECG reporting application by 40 percent, very close to the estimated theoretical upper bound of 42 percent. Furthermore, our most effective algorithm can predict Wi-Fi availability for one and ten hours into the future with 95 and 90 percent accuracy, respectively.

Emotion recognition in conversation (ERC) faces two major challenges: biased predictions and poor calibration. Classifiers often disproportionately favor certain emotion categories, such as neutral, due to the structural complexity of classifiers, the subjective nature of emotions, and imbalances in training datasets. This bias results in poorly calibrated predictions where the model’s predicted probabilities do not align with the true likelihood of outcomes. To tackle these problems, we introduce the application of conformal prediction (CP) into ERC tasks. CP is a distribution-free method that generates set-valued predictions to ensure marginal coverage in classification, thus improving the calibration of models. However, inherent biases in emotion recognition models prevent baseline CP from achieving a uniform conditional coverage across all classes. We propose a novel CP variant, class spectrum conformation, which significantly reduces coverage bias in CP methods. The methodologies introduced in this study enhance the reliability of prediction calibration and mitigate bias in complex natural language processing tasks.

Emotion Recognition in Conversations (ERC) is the task of identifying the emotions of utterances from speakers in a conversation. The major challenge of ERC is how to aggregate useful contextual information from multiple utterances. Existing research in this area has mainly focused on (1) the context modeling of utterances and (2) the interaction between speakers in conversation. Another concern with ERC is that emphatically enhancing context modeling might over contextualize surrounding information and neglect the valuable influence of context-free semantic information from the utterance. In this work, we present the cross-level attention to preserve context-free semantics of utterances and prevent over-contextualizing. Additionally, multiparty attention masks are introduced to better model complex speaker interactions by separating the conversation into speakers and others. The proposed methods are integrated with the transformer-based architecture resulting in the Cross-Level Attention with Multi-Party mask model (CLAMP). The experimental results indicate that CLAMP empirically achieves competitive performance on the IEMOCAP dataset. The proposed method presents and improvement on context utilization when adding conversation utterances. Additionally the contribution of the proposed components is demonstrated in an ablation study.

Modeling conversational context is an essential step for emotion recognition in conversations. Existing works still suffer from insufficient utilization of local context information and remote context information. This article designs a hypergraph neural network, namely HNN-ERC, to better utilize local and remote contextual information. HNN-ERC combines the recurrent neural network with the conventional hypergraph neural network to strengthen connections between utterances and make each utterance receive information from other utterances better. The proposed model has empirically achieved state-of-the-art results on three benchmark datasets, demonstrating the effectiveness and superiority of the new model.

Emotion recognition in conversations is an important step in various virtual chatbots which require opinion-based feedback, like in social media threads, online support, and many more applications. Current emotion recognition in conversations models face issues like: (a) loss of contextual information in between two dialogues of a conversation, (b) failure to give appropriate importance to significant tokens in each utterance, (c) inability to pass on the emotional information from previous utterances. The proposed model of Advanced Contextual Feature Extraction (AdCOFE) addresses these issues by performing unique feature extraction using knowledge graphs, sentiment lexicons and phrases of natural language at all levels (word and position embedding) of the utterances. Experiments on emotion recognition in conversations datasets show that AdCOFE is beneficial in capturing emotions in conversations.

Emotion recognition in conversations is an important step in various virtual chatbots which require opinion-based feedback, like in social media threads, online support, and many more applications. Current emotion recognition in conversations models face issues like: (a) loss of contextual information in between two dialogues of a conversation, (b) failure to give appropriate importance to significant tokens in each utterance, (c) inability to pass on the emotional information from previous utterances. The proposed model of Advanced Contextual Feature Extraction (AdCOFE) addresses these issues by performing unique feature extraction using knowledge graphs, sentiment lexicons and phrases of natural language at all levels (word and position embedding) of the utterances. Experiments on emotion recognition in conversations datasets show that AdCOFE is beneficial in capturing emotions in conversations.

Multi-modal graph context extraction and consensus-aware learning for emotion recognition in conversation

Abstract—An Emotion Recognition-Based Activity Recommendation System aims at providing users with adequate activity recommendations based on emotional states using the latest developments within the scope of emotion recognition technology. This project would apply speech emotion recognition techniques, specifically focusing on the most current state of-the-art methods in Triangular Region Cut-Mix augmentation for the enhancement of accuracy of emotion classification while preserving audio spectrogram information related to key emotions. Furthermore, it involves a dual learning framework integrating Emotion Recognition in Conversation and Emotional Response Generation for a richer emotional analysis. This system identifies the user’s emotions from the speech input and suggests the appropriate activities to engage the users, which is of prime interest in terms of mental wellness and betterment in the user’s experience. Some of the potential enhancements would be multimodal integration, adaptive data augmentation, and real-time detection to give more extensive and interactive recommendations. That way, there would be a highly developed user-centered solution that could help most areas, from mental support and therapy to entertainment programs individually designed based on user interest. Index Terms—Speech Emotion Recognition (SER), Triangular Region Cut-Mix, Emotion Recognition in conversation(ERC) Emotional Response Generation(ERG), Multimodal Integration Transfer Learning, Data Augmentation, Natural Language Processing (NLP) Activity Recommendation , Personalized Recommendations.

Emotion Recognition in Conversation (ERC) is becoming increasingly popular due to the accessibility of an enormous measure of openly accessible conversational information. Moreover, it has potential applications in opinion mining, social media and the health care domain. In this paper, we propose a novel Context and Knowledge Enriched Transformer Framework (CKETF) in which we interpret the contextual information from the utterances using a pre-trained Bidirectional Encoder Representations from Transformers (BERT) model and leverage additive attention based hierarchical transformer for encoding the knowledge sentences. Experiments on the knowledge-grounded Topical Chat dataset shows that both context and external knowledge are important for conversational emotion recognition. We demonstrate through extensive experiments and analysis that our proposed model significantly outperforms the current state-of-the-art methods.

Multimodal emotion recognition in conversation (MERC) involves predicting the emotion category of a conversation on the basis of textual, acoustic, and visual modalities. Information from these diverse modalities can reinforce each other to enhance the accuracy of emotion prediction. However, some information modalities may be absent in real-world applications and information from various modalities may be difficult to integrate. Therefore, a suitable strategy is required to compensate for missing modalities by using information from the available modalities and prioritizing important information. Consequently, this study developed a graph convolutional network (GCN) model with a feature compensation module and dual-channel second-order pooling module for MERC. This model initially uses a GCN to compensate for missing features by aggregating features corresponding to the same utterance node. Subsequently, it applies dual-channel second-order pooling to sift through and integrate all features. Empirical evaluations of the proposed model against other baseline models on two benchmark datasets, namely the IEMOCAP and MELD datasets, indicated that the proposed model outperformed the other models; this finding underscores the effectiveness of the proposed model in recognizing the emotions expressed in multimodal dialogue data.

Multimodal Emotion Recognition in Conversation MERC holds significant importance in Natural Language Processing due to its broad range of applications. However, existing methods still face challenges in addressing the highly imbalanced class problem and extracting robust representations for complex conversational scenarios. which leads to a decrease in the generalization ability of the model and an inability to effectively recognize minority emotion classes. To address these challenges, a cause-based supervised contrastive learning framework with adversarial sample-label CaSCLA is proposed in this paper. Specifically, we employ a modality balancing technique to fuse the multimodal features, which are then fed into a novel causal-aware network to effectively capture the underlying causal relationships within dialogues. Besides, a supervised contrastive learning with adversarial sample-label method is proposed to alleviate the class imbalance problem by learning label representations and optimizing the similarity between sample features and label embeddings. Furthermore, CaSCLA applies an adversarial samples training strategy, constructing additional positive sample-label pairs to enhance the diversity of the data and increase the robustness of the model. Extensive experiments on the IEMOCAP and MELD benchmark datasets demonstrate that CaSCLA achieves competitive performance.

DialogueLLM: Context and emotion knowledge-tuned large language models for emotion recognition in conversations.

Several recent studies on dyadic human-human interactions have been done on conversations without specific business objectives. However, many companies might benefit from studies dedicated to more precise environments such as after sales services or customer satisfaction surveys. In this work, we place ourselves in the scope of a live chat customer service in which we want to detect emotions and their evolution in the conversation flow. This context leads to multiple challenges that range from exploiting restricted, small and mostly unlabeled datasets to finding and adapting methods for such context. We tackle these challenges by using Few-Shot Learning while making the hypothesis it can serve conversational emotion classification for different languages and sparse labels. We contribute by proposing a variation of Prototypical Networks for sequence labeling in conversation that we name ProtoSeq. We test this method on two datasets with different languages: daily conversations in English and customer service chat conversations in French. When applied to emotion classification in conversations, our method proved to be competitive even when compared to other ones.

Can Large Language Models Generate Effective Datasets for Emotion Recognition in Conversations?

The need for improving the robustness and the ability to adapt to different operational conditions is a key requirement for a wider deployment of robots in many application domains. To this end, uses of contextual information have proven to be effective in many robotic applications. In this paper, we present a case study on the use of contextual information for rescue robots. The goal is to show that contextual information can be used to improve the system performance, by dynamically tailoring the functionalities of the robot to the specific features of the situation at hand. The case study includes several experiments, both in simulation and on a real robot, which address two core tasks in rescue applications: navigation and mapping.