• Home
  • Search
  • Analyzing inter-reader variability affecting deep ensemble learning for COVID-19 detection in chest radiographs.
  • Open Access IconOpen Access
  • Cite Icon66
  • https://doi.org/10.1371/journal.pone.0242301Copy DOI Icon

Analyzing inter-reader variability affecting deep ensemble learning for COVID-19 detection in chest radiographs.

  • Nov 12, 2020
  • PLOS ONE
  • Sivaramakrishnan Rajaraman +4 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Data-driven deep learning (DL) methods using convolutional neural networks (CNNs) demonstrate promising performance in natural image computer vision tasks. However, their use in medical computer vision tasks faces several limitations, viz., (i) adapting to visual characteristics that are unlike natural images; (ii) modeling random noise during training due to stochastic optimization and backpropagation-based learning strategy; (iii) challenges in explaining DL black-box behavior to support clinical decision-making; and (iv) inter-reader variability in the ground truth (GT) annotations affecting learning and evaluation. This study proposes a systematic approach to address these limitations through application to the pandemic-caused need for Coronavirus disease 2019 (COVID-19) detection using chest X-rays (CXRs). Specifically, our contribution highlights significant benefits obtained through (i) pretraining specific to CXRs in transferring and fine-tuning the learned knowledge toward improving COVID-19 detection performance; (ii) using ensembles of the fine-tuned models to further improve performance over individual constituent models; (iii) performing statistical analyses at various learning stages for validating results; (iv) interpreting learned individual and ensemble model behavior through class-selective relevance mapping (CRM)-based region of interest (ROI) localization; and, (v) analyzing inter-reader variability and ensemble localization performance using Simultaneous Truth and Performance Level Estimation (STAPLE) methods. We find that ensemble approaches markedly improved classification and localization performance, and that inter-reader variability and performance level assessment helps guide algorithm design and parameter optimization. To the best of our knowledge, this is the first study to construct ensembles, perform ensemble-based disease ROI localization, and analyze inter-reader variability and algorithm performance for COVID-19 detection in CXRs.

Similar Papers
  • Abstract
  • Citations3

Consensus Guidelines for Delineation of Clinical Target Volume for Intensity-Modulated Pelvic Radiation Therapy in Postoperative Treatment of Endometrial and Cervical Cancer- An Update

  • Sep 01, 2014
  • International Journal of Radiation Oncology*Biology*Physics
  • W Small +16
  • Research Article
  • Citations7

Level set framework of multi-atlas label fusion with applications to magnetic resonance imaging segmentation of brain region of interests and cardiac left ventricles

  • Apr 01, 2017
  • Digital Medicine
  • Zhaoxuan Gong +8
  • Conference Article
  • Citations1

CO-RCNet: Optimized ResCapsNet for COVID-19 Detection from chest CT scan images

  • Feb 10, 2023
  • Zhihua Li +2
  • PDF
  • Research Article
  • Citations16

Automated segmentation of magnetic resonance bone marrow signal: a feasibility study

  • Feb 02, 2022
  • Pediatric Radiology
  • Elisabeth Von Brandis +10
  • Research Article
  • Citations23

MFBCNNC: Momentum factor biogeography convolutional neural network for COVID-19 detection via chest X-ray images

  • Sep 15, 2021
  • Knowledge-Based Systems
  • Junding Sun +4
  • Research Article
  • Citations24

Retroperitoneal sarcoma (RPS) high risk gross tumor volume boost (HR GTV boost) contour delineation agreement among NRG sarcoma radiation and surgical oncologists.

  • May 28, 2015
  • Annals of Surgical Oncology
  • Elizabeth H Baldini +14
  • Research Article
  • Citations27

GEC-ESTRO (ACROP)–ABS–CBG Consensus Brachytherapy Target Definition Guidelines for Recurrent Endometrial and Cervical Tumors in the Vagina

  • Oct 01, 2022
  • International Journal of Radiation Oncology*Biology*Physics
  • Mitchell Kamrava +18
  • Research Article
  • Citations196

Consensus Contouring Guidelines for Postoperative Completely Resected Cavity Stereotactic Radiosurgery for Brain Metastases

  • Oct 04, 2017
  • International Journal of Radiation Oncology*Biology*Physics
  • Hany Soliman +11
  • Research Article
  • Citations11

Variability of gross tumour volume delineation: MRI and CT based tumour and lymph node delineation for lung radiotherapy

  • Dec 08, 2021
  • Radiotherapy and Oncology
  • Shivani Kumar +6
  • Research Article

Validation of QyScore’s® fully automated quantitative image segmentation tools against expert manual gold‐standards

  • Dec 01, 2023
  • Alzheimer's & Dementia
  • Elizabeth Gordon +4
  • Research Article
  • Citations21

Consensus Delineation Guidelines for Pelvic Lymph Node Radiation Therapy of Prostate Cancer: On Behalf of the Francophone Group of Urological Radiation Therapy (GFRU)

  • Jul 26, 2023
  • International Journal of Radiation Oncology*Biology*Physics
  • Olivier De Hertogh +12
  • PDF
  • Research Article

Evaluating the relationship between contouring variability and modelled treatment outcome for prostate bed radiotherapy

  • Apr 03, 2024
  • Physics in Medicine & Biology
  • Viet Le Bao +12
  • Research Article
  • Citations28

Automated detection of COVID-19 from X-ray images using CNN and Android mobile

  • Aug 12, 2021
  • Research on Biomedical Engineering
  • Kanij Fatema Bushra +2
  • PDF
  • Research Article
  • Citations40

An Efficient Deep Learning Method for Detection of COVID-19 Infection Using Chest X-ray Images.

  • Dec 30, 2022
  • Diagnostics
  • Soumya Ranjan Nayak +4
  • Research Article
  • Citations49

Deep CNN models for predicting COVID-19 in CT and x-ray images.

  • Apr 21, 2021
  • Journal of medical imaging (Bellingham, Wash.)
  • Ahmad Chaddad +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.