• Home
  • Search
  • RDMMFET: Representation of Dense Multimodality Fusion Encoder Based on Transformer
  • https://doi.org/10.1155/2021/2662064Copy DOI Icon

RDMMFET: Representation of Dense Multimodality Fusion Encoder Based on Transformer

Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Similar Papers
Abstract

Visual question answering (VQA) is the natural language question-answering of visual images. The model of VQA needs to make corresponding answers according to specific questions based on understanding images, the most important of which is to understand the relationship between images and language. Therefore, this paper proposes a new model, Representation of Dense Multimodality Fusion Encoder Based on Transformer, for short, RDMMFET, which can learn the related knowledge between vision and language. The RDMMFET model consists of three parts: dense language encoder, image encoder, and multimodality fusion encoder. In addition, we designed three types of pretraining tasks: masked language model, masked image model, and multimodality fusion task. These pretraining tasks can help to understand the fine-grained alignment between text and image regions. Simulation results on the VQA v2.0 data set show that the RDMMFET model can work better than the previous model. Finally, we conducted detailed ablation studies on the RDMMFET model and provided the results of attention visualization, which proves that the RDMMFET model can significantly improve the effect of VQA.

Loading PDF

Similar Papers
  • Research Article

VTFCGNet: a novel cross-modal reasoning network integrating Fourier self-attention and graph attention for visual text question answering

  • Feb 01, 2026
  • Neural Computing and Applications
  • Yujie Huo +3
  • Research Article
  • Citations36

Answer Questions with Right Image Regions: A Visual Attention Regularization Approach

  • Mar 04, 2022
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Yibing Liu +6
  • Research Article
  • Citations503

Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

  • Sep 11, 2018
  • International Journal of Computer Vision
  • Yash Goyal +5
  • Research Article

An Adaptive Multimodal Fusion Network Based on Multilinear Gradients for Visual Question Answering

  • Dec 24, 2024
  • Electronics
  • Chengfang Zhao +3
  • Research Article
  • Citations32

Word-to-region attention network for visual question answering

  • Aug 02, 2018
  • Multimedia Tools and Applications
  • Liang Peng +6
  • Research Article
  • Citations4

Multi-stage reasoning on introspecting and revising bias for visual question answering

  • Oct 08, 2024
  • ACM Transactions on the Web
  • L An-An +9
  • Research Article
  • Citations8

Attacking VQA Systems via Adversarial Background Noise

  • Aug 01, 2020
  • IEEE Transactions on Emerging Topics in Computational Intelligence
  • Akshay Chaturvedi +1
  • Conference Article

Dual-Stream Semantic-Aware Fusion for Robust Visual Question Answering

  • Dec 19, 2025
  • Shivshanker Singh +1
  • Book Chapter
  • Citations1

Second Order Enhanced Multi-glimpse Attention in Visual Question Answering

  • Jan 01, 2021
  • Qiang Sun +2
  • Conference Article
  • Citations2558

Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

  • Jul 01, 2017
  • Yash Goyal +4
  • Research Article

Debiasing Visual Question Answering via Ensemble Gradient Detection and Iterative Attention Forgetting

  • Dec 18, 2025
  • Informatica
  • Qiuying Han +4
  • Conference Article
  • Citations16

A Deep Learning-Based Bengali Visual Question Answering System

  • Dec 17, 2022
  • Mahamudul Hasan Rafi +5
  • Research Article
  • Citations78

Rich Visual Knowledge-Based Augmentation Network for Visual Question Answering

  • Sep 17, 2020
  • IEEE Transactions on Neural Networks and Learning Systems
  • Liyang Zhang +6
  • Research Article

Visual Question Answering Model Using Transformer-Based Multi-Head Cross Attention Network with Optimized Gold Rush Octave Convolution Network

  • Oct 28, 2025
  • International Journal of Image and Graphics
  • J Jinu Sophia +2
  • Research Article
  • Citations91

Exploring Human-Like Attention Supervision in Visual Question Answering

  • Apr 27, 2018
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Tingting Qiao +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.