• Home
  • Search
  • ConceptBert: Concept-Aware Representation for Visual Question Answering
  • Cite Icon150
  • https://doi.org/10.18653/v1/2020.findings-emnlp.44Copy DOI Icon

ConceptBert: Concept-Aware Representation for Visual Question Answering

  • Jan 1, 2020
  • François Gardères +3 more
Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Visual Question Answering (VQA) is a challenging task that has received increasing attention from both the computer vision and the natural language processing communities. A VQA model combines visual and textual features in order to answer questions grounded in an image. Current works in VQA focus on questions which are answerable by direct analysis of the question and image alone. We present a concept-aware algorithm, ConceptBert, for questions which require common sense, or basic factual knowledge from external structured content. Given an image and a question in natural language, ConceptBert requires visual elements of the image and a Knowledge Graph (KG) to infer the correct answer. We introduce a multi-modal representation which learns a joint Concept-Vision-Language embedding inspired by the popular BERT architecture. We exploit ConceptNet KG for encoding the common sense knowledge and evaluate our methodology on the Outside Knowledge-VQA (OK-VQA) and VQA datasets.

Loading PDF

Similar Papers
  • Book Chapter
  • Citations49

Visual Question Answering Using Deep Learning: A Survey and Performance Analysis

  • Jan 01, 2021
  • Yash Srivastava +3
  • Conference Article
  • Citations2

A Survey on Visual Question Answering

  • Oct 01, 2021
  • Mrinal Banchhor +1
  • Conference Article
  • Citations59

Visual Question Answering as Reading Comprehension

  • Jun 01, 2019
  • Hui Li +3
  • Research Article
  • Citations23

Dual Attention and Question Categorization-Based Visual Question Answering

  • Feb 01, 2023
  • IEEE Transactions on Artificial Intelligence
  • Aakansha Mishra +2
  • Research Article
  • Citations503

Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

  • Sep 11, 2018
  • International Journal of Computer Vision
  • Yash Goyal +5
  • Conference Article
  • Citations3

NLP Meets Vision for Visual Interpretation - A Retrospective Insight and Future directions

  • May 20, 2021
  • Ahmed Jamshed +1
  • Research Article
  • Citations71

Multimodal feature fusion by relational reasoning and attention for visual question answering

  • Aug 19, 2019
  • Information Fusion
  • Weifeng Zhang +4
  • Conference Article
  • Citations2

VQAR: Review on Information Retrieval Techniques based on Computer Vision and Natural Language Processing

  • Mar 01, 2019
  • Shivangi Modi +1
  • Research Article

Incorporating Verb Semantic Information in Visual Question Answering Through Multitask Learning Paradigm

  • Jun 01, 2020
  • International Journal of Semantic Computing
  • Mehrdad Alizadeh +1
  • Conference Article
  • Citations2

CarParkingVQA: Visual Question Answering application on Car parking occupancy detection

  • Nov 24, 2022
  • Vikash Hela +2
  • Book Chapter

Multi-view Visual Question Answering Dataset for Real Environment Applications

  • Jan 01, 2020
  • Yue Qiu +3
  • Book Chapter

Multimodal Attention for Visual Question Answering

  • Nov 02, 2018
  • Lorena Kodra +1
  • Research Article
  • Citations297

Visual question answering: Datasets, algorithms, and future challenges

  • Jun 13, 2017
  • Computer Vision and Image Understanding
  • Kushal Kafle +1
  • Research Article
  • Citations42

Knowledge-Routed Visual Question Reasoning: Challenges for Deep Representation Embedding.

  • Jan 01, 2021
  • IEEE Transactions on Neural Networks and Learning Systems
  • Qingxing Cao +4
  • Dissertation
  • Citations1

A Convolutional Neural Network Based Approach For Visual Question Answering

  • Apr 01, 2018
  • Lavanya Abhinaya Koduri
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.