• Home
  • Search
  • QAQA-SS: An Improved Fuzzing Approach with Seed Scheduling Based on the UCB Algorithm for QA Systems
  • https://doi.org/10.1109/qrs-c63300.2024.00089Copy DOI Icon

QAQA-SS: An Improved Fuzzing Approach with Seed Scheduling Based on the UCB Algorithm for QA Systems

  • Jul 1, 2024
  • Jinfu Chen +4 more
Show More
  • Abstract
  • Literature Map
  • References
  • Similar Papers
Abstract

Question Answering (QA) system is a type of artificial intelligence application designed to comprehend natural language questions and provide accurate, meaningful natural language answers. However, there are errors in the QA system, which can cause inconvenience for users and even lead to economic losses and security issues. In response to these challenges, QAQA showed up. QAQA designs several metamorphic relations (MRs) and achieves precise automatic testing through semantic-guided search and enhanced test oracles. However, QAQA fairly selects each seed (initial test input) in sequential order and assigns them a fixed number of mutations, which leaves many seeds that could produce quality test inputs neglected. This paper proposes an improved version named QAQA-SS (QAQA-Seed Schedule), where we enhance QAQA with a seed scheduling module and a fuzzing framework featuring iterative feedback. We combine the Upper Confidence Bound (UCB) algorithm with perplexity to devise a novel seed scheduling algorithm. QAQA-SS selects superior seeds and allocates appropriate mutation counts (energy) to generate more error-inducing test inputs within a limited energy budget. Additionally, we introduce a text paraphrasing operation to improve the performance of finding errors in the QA system. We conducted experiments following the experimental setup of QAQA on three datasets: BoolQ, SQuAD2, and NarrativeQA. Our experimental results reveal a noteworthy enhancement in the error discovery rate for QAQA-SS, achieving 81,777 error discoveries compared to 64,354 error discoveries with QAQA.

Similar Papers
  • PDF
  • Research Article
  • Citations12

A Metamorphic Testing Approach for Assessing Question Answering Systems

  • Mar 28, 2021
  • Mathematics
  • Kaiyi Tu +2
  • Conference Article
  • Citations6

An Ontology-Driven Question Answering System For Computer Network Module

  • Dec 02, 2021
  • M I M Nowshad +1
  • Research Article
  • Citations11

Building a deep learning-based QA system from a CQA dataset

  • Jun 15, 2023
  • Decision Support Systems
  • Sol Jin +4
  • Research Article
  • Citations361

Core techniques of question answering systems over knowledge bases: a survey

  • Sep 25, 2017
  • Knowledge and Information Systems
  • Dennis Diefenbach +3
  • Conference Article

Ask George Washington on Negotiated Texts with Quill Q&A

  • Nov 01, 2018
  • Omer Gunes +3
  • Research Article
  • Citations13

Challenges and opportunities for Arabic question-answering systems: current techniques and future directions

  • Oct 20, 2023
  • PeerJ Computer Science
  • Asmaa Alrayzah +2
  • PDF
  • Research Article
  • Citations13

BERT+vnKG: Using Deep Learning and Knowledge Graph to Improve Vietnamese Question Answering System

  • Jan 01, 2020
  • International Journal of Advanced Computer Science and Applications
  • Truong H V Phan +1
  • PDF
  • Research Article
  • Citations3

A Knowledge-Based Sense Disambiguation Method to Semantically Enhanced NL Question for Restricted Domain

  • Oct 31, 2021
  • Information
  • Ammar Arbaaeen +1
  • Research Article
  • Citations9

PerAnSel: A Novel Deep Neural Network-Based System for Persian Question Answering

  • Jul 18, 2022
  • Computational Intelligence and Neuroscience
  • Jamshid Mozafari +3
  • Research Article

DLRCNeg: Deep Learning based Reading Comprehension by handling Negation

  • Sep 05, 2019
  • Proceedings of ADCOM
  • Felicia Lilian J +2
  • Research Article
  • Citations25

Question answering systems for health professionals at the point of care-a systematic review.

  • Feb 16, 2024
  • Journal of the American Medical Informatics Association : JAMIA
  • Gregory Kell +8
  • Research Article
  • Citations7

The effect of answer validation on the performance of Question-Answering systems

  • Sep 07, 2018
  • Expert Systems with Applications
  • Alvaro Rodrigo +2
  • Research Article
  • Citations108

Research and Reviews in Question Answering System

  • Jan 01, 2013
  • Procedia Technology
  • Sanjay K Dwivedi +1
  • Research Article
  • Citations130

Hybrid query expansion using lexical resources and word embeddings for sentence retrieval in question answering

  • Dec 03, 2019
  • Information Sciences
  • Massimo Esposito +4
  • Research Article
  • Citations8

Linguistic and semantic passage retrieval strategies for question answering

  • Jan 03, 2011
  • ACM SIGIR Forum
  • Matthew W Bilotti
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.