• Home
  • Search
  • Self-supervised multi-task learning for semantic segmentation of urban scenes
  • https://doi.org/10.1117/12.2600194Copy DOI Icon

Self-supervised multi-task learning for semantic segmentation of urban scenes

  • Sep 12, 2021
  • Jonathan Gonzalez-Santiago +2 more
Show More
  • Abstract
  • Literature Map
  • References
  • Similar Papers
Abstract

The task of semantic segmentation plays a vital role in the analysis of remotely sensed imagery. Currently, this task is mainly solved using supervised pre-training, where very Deep Convolutional Neural Networks (DCNNs) are trained on large annotated datasets for mostly solving a classification problem. They are useful for many visual recognition tasks but heavily depend on the amount and quality of the annotations to learn a mapping function for predicting on new data. Motivated by the plethora of data generated everyday, researchers have come up with alternatives such as Self-Supervised Learning (SSL). These methods play a deciding role in boosting the progress of deep learning without the need of expensive labeling. They entirely explore the data, find supervision signals and solve a challenge known as Pretext Task (PTT) to learn robust representations. Thereafter, the learned features are transferred to resolve the so-called Downstream Task (DST), which can represent a group of computer vision applications such as classification or object detection. The current work explores the conception of a DCNN and training strategy to jointly predict on multiple PTTs in order to learn general visual representations that could lead more accurate semantic segmentations. The first Pretext Task is Image Colorization (IC) that identifies different objects and related parts present in a grayscale image to paint those areas with the right color. The second task is Spatial Context Prediction (SCP), which captures visual similarity across images to discover the spatial configuration of patches generated out of an image. The DCNN architecture is constructed considering each particular objective of the Pretext Tasks. It is subsequently trained and its acquired knowledge is transferred into a SSL trunk network to build a Fully Convolutional Network (FCN) on top of it. The FCN with SSL trunk learns a compound of features through fine-tuning to ultimately predict the semantic segmentation. With the aim of evaluating the quality of the learned representations, the performance of the trained model will be compared with inference results of a FCN-ResNet101 architecture pre-trained on ImageNet. This comparison employs the F1-Score as quality metric. Experiments show that the method is capable of achieving general feature representations that can definitely be employed for semantic segmentation purposes.

Similar Papers
  • Conference Article
  • Citations4

Self-supervised Learning Based on Max-tree Representation for Medical Image Segmentation

  • Jul 18, 2022
  • Qian Tang +2
  • Conference Article
  • Citations2

A Novel Multi-Task Self-Supervised Representation Learning Paradigm

  • May 28, 2021
  • Control theory & applications
  • Yinggang Li +5
  • Research Article
  • Citations20

Self-supervised visual learning in the low-data regime: A comparative evaluation

  • Mar 01, 2025
  • Neurocomputing
  • Sotirios Konstantakos +6
  • Research Article
  • Citations8

Self-supervised learning for heterogeneous graph via structure information based on metapath

  • May 12, 2023
  • Applied Soft Computing
  • Shuai Ma +2
  • Research Article
  • Citations12

Self-supervised learning improves robustness of deep learning lung tumor segmentation models to CT imaging differences.

  • Dec 05, 2024
  • Medical physics
  • Jue Jiang +2
  • Conference Article
  • Citations4

Conditional Independence for Pretext Task Selection in Self-Supervised Speech Representation Learning

  • Aug 30, 2021
  • Salah Zaiem +2
  • PDF
  • Research Article
  • Citations16

A Generic Self-Supervised Learning (SSL) Framework for Representation Learning from Spectral–Spatial Features of Unlabeled Remote Sensing Imagery

  • Nov 03, 2023
  • Remote Sensing
  • Xin Zhang +1
  • Research Article
  • Citations16

Pretext Tasks Selection for Multitask Self-Supervised Audio Representation Learning

  • Oct 01, 2022
  • IEEE Journal of Selected Topics in Signal Processing
  • Salah Zaiem +3
  • Research Article
  • Citations45

Volumetric white matter tract segmentation with nested self-supervised learning using sequential pretext tasks.

  • Apr 30, 2021
  • Medical Image Analysis
  • Qi Lu +2
  • Book Chapter
  • Citations7

CaSS: A Channel-Aware Self-supervised Representation Learning Framework for Multivariate Time Series Classification

  • Jan 01, 2022
  • Yijiang Chen +4
  • Conference Article
  • Citations64

Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting

  • Jun 01, 2021
  • Ayan Kumar Bhunia +5
  • Book Chapter

MInF: Multi-Band Invariant Feature Learning for Efficient Inertial Navigation

  • Oct 21, 2025
  • Frontiers in artificial intelligence and applications
  • Yan Li +1
  • PDF
  • Conference Article
  • Citations3

Organoids Segmentation using Self-Supervised Learning: How Complex Should the Pretext Task Be?

  • Nov 09, 2023
  • Asmaa Haja +2
  • Research Article

Self-supervised learning for stroke lesion segmentation on CT: a new pretext task for neuroimaging.

  • Jul 14, 2025
  • Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference
  • Juliette Moreau +6
  • Conference Article
  • Citations2

Point Cloud Reconstruction Is Insufficient to Learn 3D Representations

  • Oct 28, 2024
  • Weichen Xu +6
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.