• Home
  • Search
  • Multimodal Sentiment Analysis With Mutual Information-Based Disentangled Representation Learning
  • Cite Icon10
  • https://doi.org/10.1109/taffc.2025.3529732Copy DOI Icon

Multimodal Sentiment Analysis With Mutual Information-Based Disentangled Representation Learning

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Multimodal sentiment analysis seeks to utilize various types of signals to identify underlying emotions and sentiments. A key challenge in this field lies in multimodal representation learning, which aims to develop effective methods for integrating multimodal features into cohesive representations. Recent advancements include two notable approaches: one focuses on decomposing multimodal features into modality-invariant and -specific components, while the other emphasizes the use of mutual information to enhance the fusion of modalities. Both strategies have demonstrated effectiveness and yielded remarkable results. In this paper, we propose a novel learning framework that combines the strengths of these two approaches, termed mutual information-based disentangled multimodal representation learning. Our approach involves estimating different types of information during feature extraction and fusion stages. Specifically, we quantitatively assess and adjust the proportions of modality-invariant, -specific, and -complementary information during feature extraction. Subsequently, during fusion, we evaluate the amount of information retained by each modality in the fused representation. We employ mutual information or conditional mutual information to estimate each type of information content. By reconciling the proportions of these different types of information, our approach achieves state-of-the-art performance on popular sentiment analysis benchmarks, including CMU-MOSI and CMU-MOSEI.

Similar Papers
  • Research Article

A Transfer Learning-Based Text-Centric Model for Multimodal Sentiment Analysis

  • Apr 09, 2025
  • International Journal of Computational and Experimental Science and Engineering
  • Shaowei Yi +1
  • Research Article
  • Citations11

Multimodal sentiment analysis based on disentangled representation learning and cross-modal-context association mining

  • Nov 24, 2024
  • Neurocomputing
  • Zuhe Li +8
  • Research Article

Deep Learning for Multimodal Sentiment Analysis Integrating Text, Audio, and Video

  • Feb 16, 2025
  • International Journal of Recent Development in Engineering and Technology
  • Pallavi Suryavanshi
  • PDF
  • Research Article
  • Citations35

HMTL: Heterogeneous Modality Transfer Learning for Audio-Visual Sentiment Analysis

  • Jan 01, 2020
  • IEEE Access
  • Sanghyun Seo +2
  • Conference Article
  • Citations1

A Method with Universal Transformer for Multimodal Sentiment Analysis

  • Sep 23, 2022
  • Hao Ai +3
  • PDF
  • Research Article
  • Citations36

Multimodal Sentiment Analysis Representations Learning via Contrastive Learning with Condense Attention Fusion

  • Mar 01, 2023
  • Sensors (Basel, Switzerland)
  • Huiru Wang +4
  • PDF
  • Research Article
  • Citations16

Feature Extraction Network with Attention Mechanism for Data Enhancement and Recombination Fusion for Multimodal Sentiment Analysis

  • Aug 24, 2021
  • Information
  • Qingfu Qi +2
  • Research Article
  • Citations22

Integrating color cues to improve multimodal sentiment analysis in social media

  • Aug 09, 2023
  • Engineering Applications of Artificial Intelligence
  • Jieyu An +1
  • Research Article

KD-MSA: A Multimodal Implicit Sentiment Analysis Approach Based on KAN and Asymmetric Contribution-Aware Dynamic Fusion

  • Aug 28, 2025
  • Symmetry
  • Zhiyuan Hou +4
  • Conference Article
  • Citations18

The MuSe 2024 Multimodal Sentiment Analysis Challenge: Social Perception and Humor Recognition

  • Oct 28, 2024
  • Shahin Amiriparian +10
  • Research Article
  • Citations38

Multimodal sentiment analysis based on multi-layer feature fusion and multi-task learning

  • Jan 16, 2025
  • Scientific Reports
  • Yujian Cai +5
  • PDF
  • Research Article
  • Citations5

Multimodal Sentiment Analysis Method Based on Hierarchical Adaptive Feature Fusion Network

  • Jan 12, 2024
  • International Journal on Semantic Web and Information Systems
  • Huchao Zhang
  • PDF
  • Research Article
  • Citations5

Context-Dependent Multimodal Sentiment Analysis Based on a Complex Attention Mechanism

  • Aug 20, 2023
  • Electronics
  • Lujuan Deng +4
  • PDF
  • Research Article
  • Citations54

Multimodal Sentiment Analysis Using Multi-tensor Fusion Network with Cross-modal Modeling

  • Nov 21, 2021
  • Applied Artificial Intelligence
  • Xueming Yan +3
  • Research Article
  • Citations1

Multi Modal Image Text Sentiment Analysis Method for Social Media

  • Dec 23, 2025
  • Highlights in Science, Engineering and Technology
  • Zelin Wang
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.