• Home
  • Search
  • Fault Detection Effectiveness of Metamorphic Relations Developed for Testing Supervised Classifiers
  • Open Access IconOpen Access
  • Cite Icon25
  • https://doi.org/10.1109/aitest.2019.00019Copy DOI Icon

Fault Detection Effectiveness of Metamorphic Relations Developed for Testing Supervised Classifiers

  • Apr 1, 2019
  • Prashanta Saha +1 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

In machine learning, supervised classifiers are used to obtain predictions for unlabeled data by inferring prediction functions using labeled data. Supervised classifiers are widely applied in domains such as computational biology, computational physics and healthcare to make critical decisions. However, it is often hard to test supervised classifiers since the expected answers are unknown. This is commonly known as the oracle problem and metamorphic testing (MT) has been used to test such programs. In MT, metamorphic relations (MRs) are developed from intrinsic characteristics of the software under test (SUT). These MRs are used to generate test data and to verify the correctness of the test results without the presence of a test oracle. Effectiveness of MT heavily depends on the MRs used for testing. In this paper we have conducted an extensive empirical study to evaluate the fault detection effectiveness of MRs that have been used in multiple previous studies to test supervised classifiers. Our study uses a total of 709 reachable mutants generated by multiple mutation engines and uses data sets with varying characteristics to test the SUT. Our results reveal that only 14.8% of these mutants are detected using the MRs and that the fault detection effectiveness of these MRs do not scale with the increased number of mutants when compared to what was reported in previous studies.

Similar Papers
  • Conference Article
  • Citations7

Automated identification of metamorphic test scenarios for an ocean-modeling application

  • Aug 01, 2020
  • Dilip J Hiremath +3
  • Research Article
  • Citations4

Metamorphic Relations Based Test Oracles for Image Processing Applications

  • Jan 01, 2016
  • International Journal of Software Innovation
  • Tahir Jameel +2
  • Research Article
  • Citations112

Metamorphic Testing of RESTful Web APIs

  • Nov 01, 2018
  • IEEE Transactions on Software Engineering
  • Sergio Segura +3
  • Research Article
  • Citations111

Predicting metamorphic relations for testing scientific software: a machine learning approach using graph kernels

  • Nov 16, 2015
  • Software Testing, Verification and Reliability
  • Upulee Kanewala +2
  • Conference Article
  • Citations19

Quality assurance of bioinformatics software

  • May 27, 2018
  • Madhusudan Srinivasan +3
  • Conference Article
  • Citations103

Using machine learning techniques to detect metamorphic relations for programs without test oracles

  • Nov 01, 2013
  • Upulee Kanewala +1
  • Research Article
  • Citations7

Metamorphic testing of the NUIT code based on burnup time

  • Dec 19, 2020
  • Annals of Nuclear Energy
  • Meng Li +7
  • Conference Article
  • Citations13

Security Assurance with Metamorphic Testing and Genetic Algorithm

  • Aug 01, 2010
  • Guowei Dong +4
  • PDF
  • Research Article
  • Citations9

A Genetic Algorithm-Based Approach for Composite Metamorphic Relations Construction

  • Dec 10, 2019
  • Information
  • Zhenglong Xiang +2
  • Conference Article
  • Citations2

Metamorphic Testing on Nuclide Inventory Tool

  • Aug 04, 2020
  • Meng Li +3
  • Conference Article
  • Citations11

Achievements and Challenges of Metamorphic Testing

  • Dec 01, 2013
  • Zhan-Wei Hui +1
  • Research Article
  • Citations8

A template-based approach to describing metamorphic relations

  • May 20, 2017
  • Sergio Segura +3
  • Conference Article
  • Citations9

On Applying Metamorphic Testing: An Empirical Study on Academic Search Engines

  • May 01, 2019
  • Stevao Alves De Andrade +5
  • PDF
  • Research Article
  • Citations174

An innovative approach for testing bioinformatics programs using metamorphic testing

  • Jan 19, 2009
  • BMC Bioinformatics
  • Tsong Yueh Chen +3
  • Research Article
  • Citations11

MT4WS: an automated metamorphic testing system for web services

  • Jan 01, 2016
  • International Journal of High Performance Computing and Networking
  • Chang Ai Sun +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.