• Home
  • Search
  • NuScenes-QA: A Multi-Modal Visual Question Answering Benchmark for Autonomous Driving Scenario
  • Cite Icon127
  • https://doi.org/10.1609/aaai.v38i5.28253Copy DOI Icon

NuScenes-QA: A Multi-Modal Visual Question Answering Benchmark for Autonomous Driving Scenario

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

We introduce a novel visual question answering (VQA) task in the context of autonomous driving, aiming to answer natural language questions based on street-view clues. Compared to traditional VQA tasks, VQA in autonomous driving scenario presents more challenges. Firstly, the raw visual data are multi-modal, including images and point clouds captured by camera and LiDAR, respectively. Secondly, the data are multi-frame due to the continuous, real-time acquisition. Thirdly, the outdoor scenes exhibit both moving foreground and static background. Existing VQA benchmarks fail to adequately address these complexities. To bridge this gap, we propose NuScenes-QA, the first benchmark for VQA in the autonomous driving scenario, encompassing 34K visual scenes and 460K question-answer pairs. Specifically, we leverage existing 3D detection annotations to generate scene graphs and design question templates manually. Subsequently, the question-answer pairs are generated programmatically based on these templates. Comprehensive statistics prove that our NuScenes-QA is a balanced large-scale benchmark with diverse question formats. Built upon it, we develop a series of baselines that employ advanced 3D detection and VQA techniques. Our extensive experiments highlight the challenges posed by this new task. Codes and dataset are available at https://github.com/qiantianwen/NuScenes-QA.

Similar Papers
  • Research Article

VTFCGNet: a novel cross-modal reasoning network integrating Fourier self-attention and graph attention for visual text question answering

  • Feb 01, 2026
  • Neural Computing and Applications
  • Yujie Huo +3
  • Research Article
  • Citations30

Improving Automatic VQA Evaluation Using Large Language Models

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Oscar Mañas +2
  • Book Chapter
  • Citations49

Visual Question Answering Using Deep Learning: A Survey and Performance Analysis

  • Jan 01, 2021
  • Yash Srivastava +3
  • PDF
  • Conference Article
  • Citations68

Sunny and Dark Outside?! Improving Answer Consistency in VQA through Entailed Question Generation

  • Jan 01, 2019
  • Arijit Ray +4
  • Conference Article
  • Citations4

Incorporating 3D Information Into Visual Question Answering

  • Sep 01, 2019
  • Yue Qiu +3
  • Research Article
  • Citations15

MobiVQA

  • Jul 04, 2022
  • Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
  • Qingqing Cao +3
  • Research Article
  • Citations19

Achieving Human Parity on Visual Question Answering

  • Apr 04, 2023
  • ACM Transactions on Information Systems
  • Ming Yan +11
  • Conference Article
  • Citations48

Coarse-to-Fine Reasoning for Visual Question Answering

  • Jun 01, 2022
  • Binh X Nguyen +5
  • Book Chapter
  • Citations16

Learning Convolutional Text Representations for Visual Question Answering

  • May 07, 2018
  • Zhengyang Wang +1
  • Conference Article
  • Citations227

OCR-VQA: Visual Question Answering by Reading Text in Images

  • Sep 01, 2019
  • Anand Mishra +3
  • Conference Article
  • Citations2558

Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

  • Jul 01, 2017
  • Yash Goyal +4
  • Conference Article
  • Citations211

Visual Question Generation as Dual Task of Visual Question Answering

  • Jun 01, 2018
  • Yikang Li +6
  • Research Article
  • Citations29

OpenViVQA: Task, dataset, and multimodal fusion models for visual question answering in Vietnamese

  • Jul 04, 2023
  • Information Fusion
  • Nghia Hieu Nguyen +3
  • Conference Article
  • Citations51

Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models

  • Oct 01, 2021
  • Linjie Li +3
  • Book Chapter
  • Citations5

Neural Networks for Detecting Irrelevant Questions During Visual Question Answering

  • Jan 01, 2020
  • Mengdi Li +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.