• Home
  • Search
  • Point Cloud Reconstruction Is Insufficient to Learn 3D Representations
  • Cite Icon2
  • https://doi.org/10.1145/3664647.3680890Copy DOI Icon

Point Cloud Reconstruction Is Insufficient to Learn 3D Representations

  • Oct 28, 2024
  • Weichen Xu +6 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

This paper revisits the development of generative self-supervised learning in 2D images and 3D point clouds in autonomous driving. In 2D images, the pretext task has evolved from low-level to high-level features. Inspired by this, through explore model analysis, we find that the gap in weight distribution between self-supervised learning and supervised learning is substantial when employing only low-level features as the pretext task in 3D point clouds. Low-level features represented by PoInt Cloud reconsTruction are insUfficient to learn 3D REpresentations (dubbed PICTURE). To advance the development of pretext tasks, we propose a unified generative self-supervised framework. Firstly, high-level features are demonstrated to exhibit semantic consistency with downstream tasks. We utilize the high-level features as an additional pretext task to enhance the understanding of semantic information during the pre-training. Next, we propose inter-class and intra-class discrimination-guided masking (I2Mask) based on the attributes of the high-level features, adaptively setting the masking ratio for each superclass. On Waymo and nuScenes datasets, we achieve 75.13% mAP and 72.69% mAPH for 3D object detection, 79.4% mIoU for 3D semantic segmentation, and 18.4% mIoU for occupancy prediction. Extensive experiments have demonstrated the effectiveness and necessity of high-level features.

Similar Papers
  • Research Article
  • Citations3

Mutual information-driven self-supervised point cloud pre-training

  • Nov 20, 2024
  • Knowledge-Based Systems
  • Weichen Xu +6
  • Conference Article
  • Citations2

A Novel Multi-Task Self-Supervised Representation Learning Paradigm

  • May 28, 2021
  • Control theory & applications
  • Yinggang Li +5
  • Research Article
  • Citations45

Volumetric white matter tract segmentation with nested self-supervised learning using sequential pretext tasks.

  • Apr 30, 2021
  • Medical Image Analysis
  • Qi Lu +2
  • Research Article
  • Citations16

Pretext Tasks Selection for Multitask Self-Supervised Audio Representation Learning

  • Oct 01, 2022
  • IEEE Journal of Selected Topics in Signal Processing
  • Salah Zaiem +3
  • Research Article
  • Citations8

Self-supervised learning for heterogeneous graph via structure information based on metapath

  • May 12, 2023
  • Applied Soft Computing
  • Shuai Ma +2
  • Research Article
  • Citations20

Self-supervised visual learning in the low-data regime: A comparative evaluation

  • Mar 01, 2025
  • Neurocomputing
  • Sotirios Konstantakos +6
  • PDF
  • Research Article
  • Citations16

A Generic Self-Supervised Learning (SSL) Framework for Representation Learning from Spectral–Spatial Features of Unlabeled Remote Sensing Imagery

  • Nov 03, 2023
  • Remote Sensing
  • Xin Zhang +1
  • Conference Article
  • Citations4

Self-supervised Learning Based on Max-tree Representation for Medical Image Segmentation

  • Jul 18, 2022
  • Qian Tang +2
  • Conference Article
  • Citations4

Conditional Independence for Pretext Task Selection in Self-Supervised Speech Representation Learning

  • Aug 30, 2021
  • Salah Zaiem +2
  • Research Article

Self-supervised learning for stroke lesion segmentation on CT: a new pretext task for neuroimaging.

  • Jul 14, 2025
  • Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference
  • Juliette Moreau +6
  • PDF
  • Conference Article
  • Citations3

Organoids Segmentation using Self-Supervised Learning: How Complex Should the Pretext Task Be?

  • Nov 09, 2023
  • Asmaa Haja +2
  • Research Article
  • Citations6

Self-Supervised Video Representation Learning by Serial Restoration With Elastic Complexity

  • Jan 01, 2024
  • IEEE Transactions on Multimedia
  • Ziyu Chen +2
  • Conference Article
  • Citations64

Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting

  • Jun 01, 2021
  • Ayan Kumar Bhunia +5
  • Conference Article

Self-supervised multi-task learning for semantic segmentation of urban scenes

  • Sep 12, 2021
  • Jonathan Gonzalez-Santiago +2
  • PDF
  • Research Article
  • Citations2

PatchLoc: Embedded Patch Localization Pretext Task for Tumor Segmentation in Medical Images

  • Jan 01, 2024
  • IEEE Access
  • Ramchandra Cheke +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.