• Home
  • Search
  • Testing convolutional neural network based deep learning systems: a statistical metamorphic approach.
  • Cite Icon3
  • https://doi.org/10.7717/peerj-cs.2658Copy DOI Icon

Testing convolutional neural network based deep learning systems: a statistical metamorphic approach.

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Machine learning technology spans many areas and today plays a significant role in addressing a wide range of problems in critical domains, i.e., healthcare, autonomous driving, finance, manufacturing, cybersecurity, etc. Metamorphic testing (MT) is considered a simple but very powerful approach in testing such computationally complex systems for which either an oracle is not available or is available but difficult to apply. Conventional metamorphic testing techniques have certain limitations in verifying deep learning-based models (i.e., convolutional neural networks (CNNs)) that have a stochastic nature (because of randomly initializing the network weights) in their training. In this article, we attempt to address this problem by using a statistical metamorphic testing (SMT) technique that does not require software testers to worry about fixing the random seeds (to get deterministic results) to verify the metamorphic relations (MRs). We propose seven MRs combined with different statistical methods to statistically verify whether the program under test adheres to the relation(s) specified in the MR(s). We further use mutation testing techniques to show the usefulness of the proposed approach in the healthcare space and test two CNN-based deep learning models (used for pneumonia detection among patients). The empirical results show that our proposed approach uncovers 85.71% of the implementation faults in the classifiers under test (CUT). Furthermore, we also propose an MRs minimization algorithm for the CUT, thus saving computational costs and organizational testing resources.

Similar Papers
  • Conference Article
  • Citations5

Bottom-up Integration Testing with the Technique of Metamorphic Testing

  • Oct 01, 2014
  • Xiaoyuan Xie +3
  • Conference Article
  • Citations79

An Automatic Testing Approach for Compiler Based on Metamorphic Testing Technique

  • Nov 01, 2010
  • Qiuming Tao +3
  • Research Article
  • Citations9

How can non-technical end users effectively test their spreadsheets?

  • Oct 28, 2014
  • Information Technology & People
  • Pak-Lok Poon +3
  • PDF
  • Research Article
  • Citations174

An innovative approach for testing bioinformatics programs using metamorphic testing

  • Jan 19, 2009
  • BMC Bioinformatics
  • Tsong Yueh Chen +3
  • Conference Article
  • Citations14

Bug or not Bug? Analysing the Reasons Behind Metamorphic Relation Violations

  • Mar 01, 2023
  • Alejandra Duque-Torres +3
  • Conference Article
  • Citations10

Testing Graph Searching Based Path Planning Algorithms by Metamorphic Testing

  • Dec 01, 2019
  • Jiantao Zhang +4
  • PDF
  • Research Article
  • Citations12

A Metamorphic Testing Approach for Assessing Question Answering Systems

  • Mar 28, 2021
  • Mathematics
  • Kaiyi Tu +2
  • Research Article

Boosting Metamorphic Testing: A General Metamorphic Specification Language and A Supporting System

  • Feb 07, 2026
  • ACM Transactions on Software Engineering and Methodology
  • Chang-Ai Sun +3
  • Conference Article
  • Citations73

Metamorphic testing of RESTful web APIs

  • May 27, 2018
  • Sergio Segura +3
  • Research Article
  • Citations40

Theoretical and Empirical Analyses of the Effectiveness of Metamorphic Relation Composition

  • Jul 21, 2020
  • IEEE Transactions on Software Engineering
  • Kun Qiu +3
  • Conference Article
  • Citations1

An Approach for Iteratively Generating Adequate Tests in Metamorphic Testing: A Case Study

  • Jun 01, 2016
  • Junhua Ding +1
  • Conference Article
  • Citations8

Measurable Metrics for Qualitative Guidelines of Metamorphic Relation

  • Jul 01, 2015
  • Zhan-Wei Hui +4
  • Conference Article
  • Citations22

The impact of source test case selection on the effectiveness of metamorphic testing

  • May 14, 2016
  • Arlinta Christy Barus +4
  • Conference Article
  • Citations1

2nd International Workshop on Metamorphic Testing (MET 2017)

  • May 01, 2017
  • Laura L Pullum +4
  • Research Article
  • Citations5

Effectiveness of Teaching Metamorphic Testing, Part II

  • Jan 01, 2013
  • Columbia Academic Commons (Columbia University)
  • Kunal Swaroop Mishra +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.