• Home
  • Search
  • A Metamorphic Testing Approach for Assessing Question Answering Systems
  • Open Access IconOpen Access
  • Cite Icon12
  • https://doi.org/10.3390/math9070726Copy DOI Icon

A Metamorphic Testing Approach for Assessing Question Answering Systems

Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Question Answering (QA) enables the machine to understand and answer questions posed in natural language, which has emerged as a powerful tool in various domains. However, QA is a challenging task and there is an increasing concern about its quality. In this paper, we propose to apply the technique of metamorphic testing (MT) to evaluate QA systems from the users’ perspectives, in order to help the users to better understand the capabilities of these systems and then to select appropriate QA systems for their specific needs. Two typical categories of QA systems, namely, the textual QA (TQA) and visual QA (VQA), are studied, and a total number of 17 metamorphic relations (MRs) are identified for them. These MRs respectively focus on some characteristics of different aspects of QA. We further apply MT to four QA systems (including two APIs from the AllenNLP platform, one API from the Transformers platform, and one API from CloudCV) by using all of the MRs. Our experimental results demonstrate the capabilities of the four subject QA systems from various aspects, revealing their strengths and weaknesses. These results further suggest that MT can be an effective method for assessing QA systems.

Loading PDF

Similar Papers
  • Research Article

Bridging the Gap Between Question and Answer: A Comprehensive Analysis on Medical Question Answering Systems

  • Feb 25, 2025
  • Journal of Information Systems Engineering and Management
  • Keerthana Sm
  • Book Chapter
  • Citations1

Automated Question Generation and Answer Verification Using Visual Data

  • Jan 01, 2020
  • Shrey Nahar +4
  • Conference Article

QAQA-SS: An Improved Fuzzing Approach with Seed Scheduling Based on the UCB Algorithm for QA Systems

  • Jul 01, 2024
  • Jinfu Chen +4
  • Conference Article
  • Citations6

An Ontology-Driven Question Answering System For Computer Network Module

  • Dec 02, 2021
  • M I M Nowshad +1
  • Research Article

A textual question answering and handwritten answer evaluation system for hindi language

  • Oct 09, 2024
  • International Journal of Knowledge-based and Intelligent Engineering Systems
  • Khushboo Khurana +4
  • PDF
  • Research Article
  • Citations2

Visual Question Answering for Intelligent Interaction

  • Jul 06, 2022
  • Mobile Information Systems
  • Panpan Gao +4
  • Conference Article
  • Citations73

Metamorphic testing of RESTful web APIs

  • May 27, 2018
  • Sergio Segura +3
  • Conference Article
  • Citations5

Bottom-up Integration Testing with the Technique of Metamorphic Testing

  • Oct 01, 2014
  • Xiaoyuan Xie +3
  • Conference Article

Ask George Washington on Negotiated Texts with Quill Q&A

  • Nov 01, 2018
  • Omer Gunes +3
  • Research Article

DLRCNeg: Deep Learning based Reading Comprehension by handling Negation

  • Sep 05, 2019
  • Proceedings of ADCOM
  • Felicia Lilian J +2
  • Research Article
  • Citations13

Challenges and opportunities for Arabic question-answering systems: current techniques and future directions

  • Oct 20, 2023
  • PeerJ Computer Science
  • Asmaa Alrayzah +2
  • Research Article
  • Citations40

Theoretical and Empirical Analyses of the Effectiveness of Metamorphic Relation Composition

  • Jul 21, 2020
  • IEEE Transactions on Software Engineering
  • Kun Qiu +3
  • Conference Article
  • Citations2

Beyond Words: Exploring Co-Attention with BERT in Visual Question Answering

  • May 24, 2024
  • Vibhashree B S +4
  • Research Article
  • Citations25

Question answering systems for health professionals at the point of care-a systematic review.

  • Feb 16, 2024
  • Journal of the American Medical Informatics Association : JAMIA
  • Gregory Kell +8
  • Book Chapter
  • Citations1

Towards Open Ended and Free Form Visual Question Answering: Modeling VQA as a Factoid Question Answering Problem

  • Jan 01, 2021
  • Abhishek Narayanan +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.