• Home
  • Search
  • Reasoning on the Relation: Enhancing Visual Representation for Visual Question Answering and Cross-Modal Retrieval
  • Cite Icon93
  • https://doi.org/10.1109/tmm.2020.2972830Copy DOI Icon

Reasoning on the Relation: Enhancing Visual Representation for Visual Question Answering and Cross-Modal Retrieval

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Cross-modal analysis has become a promising direction for artificial intelligence. Visual representation is crucial for various cross-modal analysis tasks that require visual content understanding. Visual features which contain semantical information can disentangle the underlying correlation between different modalities, thus benefiting the downstream tasks. In this paper, we propose a Visual Reasoning and Attention Network (VRANet) as a plug-and-play module to capture rich visual semantics and help to enhance the visual representation for improving cross-modal analysis. Our proposed VRANet is built based on the bilinear visual attention module which identifies the critical objects. We propose a novel Visual Relational Reasoning (VRR) module to reason about pair-wise and inner-group visual relationships among objects guided by the textual information. The two modules enhance the visual features at both relation level and object level. We demonstrate the effectiveness of the proposed VRANet by applying it to both Visual Question Answering (VQA) and Cross-Modal Information Retrieval (CMIR) tasks. Extensive experiments conducted on VQA 2.0, CLEVR, CMPlaces, and MS-COCO datasets indicate superior performance comparing with state-of-the-art work.

Similar Papers
  • Research Article
  • Citations14

Cross-modal learning with prior visual relation knowledge

  • Jun 16, 2020
  • Knowledge-Based Systems
  • Jing Yu +4
  • Research Article
  • Citations71

Multimodal feature fusion by relational reasoning and attention for visual question answering

  • Aug 19, 2019
  • Information Fusion
  • Weifeng Zhang +4
  • Research Article
  • Citations9

Deep Adversarial Cascaded Hashing for Cross-Modal Vessel Image Retrieval

  • Jan 01, 2023
  • IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
  • Jiaen Guo +1
  • Research Article
  • Citations143

Fine-Grained Visual Textual Alignment for Cross-Modal Retrieval Using Transformer Encoders

  • Nov 12, 2021
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Nicola Messina +5
  • Research Article

VTFCGNet: a novel cross-modal reasoning network integrating Fourier self-attention and graph attention for visual text question answering

  • Feb 01, 2026
  • Neural Computing and Applications
  • Yujie Huo +3
  • Research Article
  • Citations7

Multi-Granularity Semantic Information Integration Graph for Cross-Modal Hash Retrieval

  • Jan 01, 2024
  • IEEE Access
  • Zhichao Han +3
  • Research Article
  • Citations26

Cross-Modality Retrieval by Joint Correlation Learning

  • Apr 30, 2019
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Shuo Wang +3
  • Research Article
  • Citations472

The multi-modal fusion in visual question answering: a review of attention mechanisms.

  • May 30, 2023
  • PeerJ Computer Science
  • Siyu Lu +5
  • Research Article
  • Citations7

Remote Sensing Cross-Modal Text-Image Retrieval Based on Attention Correction and Filtering

  • Jan 31, 2025
  • Remote Sensing
  • Xiaoyu Yang +5
  • Research Article

ADMM-based online spectral hashing with elastic metric for cross-modal retrieval

  • Mar 04, 2026
  • Journal of King Saud University Computer and Information Sciences
  • Han Zhang +3
  • Conference Article

A cross-modal multimedia retrieval method based on semi-supervised learning and category information alignment

  • Nov 01, 2021
  • Baiqiang Gan +1
  • PDF
  • Research Article
  • Citations4

VITR: Augmenting Vision Transformers with Relation-Focused Learning for Cross-modal Information Retrieval

  • Oct 18, 2024
  • ACM Transactions on Knowledge Discovery from Data
  • Yan Gong +2
  • Research Article

NeSyVQA: Neurosymbolic Visual Question Answering with Knowledge-Enriched Scene Graphs

  • Jan 01, 2025
  • IEEE Access
  • M Jaleed Khan +2
  • Research Article
  • Citations20

A framework for visual question answering with the integration of scene-text using PHOCs and fisher vectors

  • Nov 07, 2021
  • Expert Systems with Applications
  • Himanshu Sharma +1
  • Research Article
  • Citations15

Incomplete Cross-Modal Retrieval with Deep Correlation Transfer

  • Jan 11, 2024
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Dan Shi +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.