• Home
  • Search
  • Beyond Words: Exploring Co-Attention with BERT in Visual Question Answering
  • Cite Icon2
  • https://doi.org/10.1109/incet61516.2024.10593261Copy DOI Icon

Beyond Words: Exploring Co-Attention with BERT in Visual Question Answering

  • May 24, 2024
  • Vibhashree B S +4 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

The Visual Question Answering project described in this work uses a multi-modal approach that combines Bidirectional Encoder Representations from Transformers for complex natural language question understanding, You Only Look Once for robust object detection, and a Hierarchical Co-Attention Mechanism for efficient integration of textual and visual data. The goal is to develop a thorough Visual Question Answering system that can respond to inquiries about visual information with precision and contextual relevance. While Bidirectional Encoder Representations from Transformers captures subtle question semantics, the You Only Look Once model makes efficient object detection possible. In order to handle the complexities of multi-modal information processing, the Hierarchical Co-Attention Mechanism improves the fusion of local and global characteristics. The evaluation criteria include co-attention fusion scores, object detection metrics, Natural Language Processing metrics for Bidirectional Encoder Representations from Transformers, and overall Visual Question Answering accuracy, among other pertinent dimensions. By taking into account both response correctness and the efficiency of individual components, the study seeks to obtain a sophisticated view of the system's performance. Open-ended evaluation criteria allow for a comprehensive assessment because of the Visual Question Answering assignment's varied nature. The results show that the system can intelligently answer to questions regarding visual content and demonstrate its potential for a wide range of practical applications. The project is open-sourced, encouraging community involvement and collaboration to push the boundaries of multi-modal question answering systems even farther.

Similar Papers
  • Conference Article
  • Citations2

VQAR: Review on Information Retrieval Techniques based on Computer Vision and Natural Language Processing

  • Mar 01, 2019
  • Shivangi Modi +1
  • PDF
  • Research Article
  • Citations86

MedFuseNet:\xa0An\xa0attention-based multimodal deep learning model for visual question answering in the medical domain

  • Oct 06, 2021
  • Scientific Reports
  • Dhruv Sharma +2
  • Research Article
  • Citations1

Counting in Visual Question Answering: Methods, Datasets, and Future Work

  • Oct 20, 2023
  • International Journal of Image and Graphics
  • Tesfayee Meshu Welde +1
  • Book Chapter
  • Citations2

A VQA System for Medical Image Classification Using Transfer Learning

  • Jan 01, 2021
  • C Dhanush +2
  • Research Article
  • Citations49

Medical knowledge-based network for Patient-oriented Visual Question Answering

  • Dec 21, 2022
  • Information Processing & Management
  • Jian Huang +7
  • Book Chapter
  • Citations2

A-VQA: Attention Based Visual Question Answering Technique for Handling Improper Count of Occluded Object

  • Jan 01, 2021
  • Shivangi Modi +1
  • Book Chapter
  • Citations1

Visual Question Answering System Using Co-attention Model

  • Jan 01, 2022
  • D Karthika Renuka +5
  • PDF
  • Research Article
  • Citations2

Visual Question Answering for Intelligent Interaction

  • Jul 06, 2022
  • Mobile Information Systems
  • Panpan Gao +4
  • Book Chapter
  • Citations3

Optimal Image Feature Ranking and Fusion for Visual Question Answering

  • Sep 09, 2020
  • Sruthy Manmadhan +1
  • Conference Article
  • Citations1

A Visual Question Answering System using YOLO Model

  • Dec 01, 2021
  • Soumya Priyadarsini Panda +1
  • Abstract

Understanding the diagnosis and management of external eye disease: Gurwood AS: Br J Optom Disp 1995;3:155–163

  • May 01, 1996
  • International Contact Lens Clinic
  • Research Article
  • Citations45

A survey of methods, datasets and evaluation metrics for visual question answering

  • Oct 15, 2021
  • Image and Vision Computing
  • Himanshu Sharma +1
  • Conference Article

Development of Counting Based Visual Question Answering System by Using Transformer and Pyramid Networks with Hybrid Deep Learning Model

  • Aug 05, 2023
  • Nigisha S +1
  • Conference Article

Flexible Sentence Analysis Model for Visual Question Answering Network

  • Sep 19, 2018
  • Wei Deng +3
  • Research Article
  • Citations12

Visual Question Answering using Transformer Architectures: Applying Transformer Models to Improve Performance in VQA Tasks

  • Mar 31, 2022
  • Journal of Artificial Intelligence & Cloud Computing
  • Vedant Singh
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.