• Home
  • Search
  • Can Ground Truth Label Propagation from Video Help Semantic Segmentation?
  • Open Access IconOpen Access
  • Cite Icon44
  • https://doi.org/10.1007/978-3-319-49409-8_66Copy DOI Icon

Can Ground Truth Label Propagation from Video Help Semantic Segmentation?

  • Jan 1, 2016
  • Siva Karthik Mustikovela +2 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

For state-of-the-art semantic segmentation task, training convolutional neural networks (CNNs) requires dense pixelwise ground truth (GT) labeling, which is expensive and involves extensive human effort. In this work, we study the possibility of using auxiliary ground truth, so-called pseudo ground truth (PGT) to improve the performance. The PGT is obtained by propagating the labels of a GT frame to its subsequent frames in the video using a simple CRF-based, cue integration framework. Our main contribution is to demonstrate the use of noisy PGT along with GT to improve the performance of a CNN. We perform a systematic analysis to find the right kind of PGT that needs to be added along with the GT for training a CNN. In this regard, we explore three aspects of PGT which influence the learning of a CNN: (i) the PGT labeling has to be of good quality; (ii) the PGT images have to be different compared to the GT images; (iii) the PGT has to be trusted differently than GT. We conclude that PGT which is diverse from GT images and has good quality of labeling can indeed help improve the performance of a CNN. Also, when PGT is multiple folds larger than GT, weighing down the trust on PGT helps in improving the accuracy. Finally, We show that using PGT along with GT, the performance of Fully Convolutional Network (FCN) on Camvid data is increased by \(2.7\,\%\) on IoU accuracy. We believe such an approach can be used to train CNNs for semantic video segmentation where sequentially labeled image frames are needed. To this end, we provide recommendations for using PGT strategically for semantic segmentation and hence bypass the need for extensive human efforts in labeling.

Similar Papers
  • Research Article
  • Citations19

Erase then grow: Generating correct class activation maps for weakly-supervised semantic segmentation

  • Apr 30, 2021
  • Neurocomputing
  • Yanwen Chong +3
  • Research Article
  • Citations52

A regularized convolutional neural network for semantic image segmentation

  • Feb 10, 2020
  • Analysis and Applications
  • Fan Jia +2
  • PDF
  • Research Article
  • Citations46

Parallel Fully Convolutional Network for Semantic Segmentation

  • Dec 30, 2020
  • IEEE Access
  • Jian Ji +5
  • Research Article
  • Citations42

Weakly-supervised object detection via mining pseudo ground truth bounding-boxes

  • Jul 05, 2018
  • Pattern Recognition
  • Yongqiang Zhang +4
  • PDF
  • Research Article

Semantic Segmentation of 3D Medical Images with 3D Convolutional Neural Networks

  • Apr 01, 2020
  • CLEI Electronic Journal
  • Alejandra Márquez Herrera +2
  • PDF
  • Research Article
  • Citations6

A top-down manner-based DCNN architecture for semantic image segmentation.

  • Mar 24, 2017
  • PLOS ONE
  • Kai Qiao +4
  • Research Article
  • Citations72

High-Resolution Aerial Images Semantic Segmentation Using Deep Fully Convolutional Network With Channel Attention Mechanism

  • Sep 01, 2019
  • IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
  • Haifeng Luo +4
  • Conference Article
  • Citations4

Improving Semantic Segmentation of 3D Medical Images on 3D Convolutional Neural Networks

  • Sep 01, 2019
  • Alejandra Marquez Herrera +2
  • Conference Article
  • Citations225

A Relation-Augmented Fully Convolutional Network for Semantic Segmentation in Aerial Scenes

  • Jun 01, 2019
  • Lichao Mou +2
  • Research Article
  • Citations5

Revisiting Domain-Adaptive Semantic Segmentation via Knowledge Distillation.

  • Jan 01, 2024
  • IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
  • Seongwon Jeong +3
  • Conference Article
  • Citations8

Spatial Relational Reasoning in Networks for Improving Semantic Segmentation of Aerial Images

  • Jul 01, 2019
  • Lichao Mou +2
  • Conference Article

Weakly-Supervised Estimation of Auroral Motion Field Via Iterative Pseudo Ground Truth Learning

  • Sep 23, 2022
  • Qian Wang +2
  • Conference Article
  • Citations8

CYLINDRICAL TRANSFORM: 3D SEMANTIC SEGMENTATION OF KIDNEYS WITH LIMITED ANNOTATED IMAGES

  • Nov 01, 2018
  • Hojjat Salehinejad +4
  • Book Chapter
  • Citations1

Design Pseudo Ground Truth with Motion Cue for Unsupervised Video Object Segmentation

  • Jan 01, 2019
  • Ye Wang +6
  • PDF
  • Research Article
  • Citations16

Precise Extraction of Buildings from High-Resolution Remote-Sensing Images Based on Semantic Edges and Segmentation

  • Aug 05, 2021
  • Remote Sensing
  • Liegang Xia +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.