• Home
  • Search
  • Deeply Supervised Block-Wise Neural Architecture Search.
  • Cite Icon6
  • https://doi.org/10.1109/tnnls.2023.3347542Copy DOI Icon

Deeply Supervised Block-Wise Neural Architecture Search.

  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Neural architecture search (NAS) has shown great promise in automatically designing neural network models. Recently, block-wise NAS has been proposed to alleviate deep coupling problem between architectures and weights existed in the well-known weight-sharing NAS, by training the huge weight-sharing supernet block-wisely. However, the existing block-wise NAS methods, which resort to either supervised distillation or self-supervised contrastive learning scheme to enable block-wise optimization, take massive computational cost. To be specific, the former introduces an external high-capacity teacher model, while the latter involves supernet-scale momentum model and requires a long training schedule. Considering this, in this work, we propose a resource-friendly deeply supervised block-wise NAS (DBNAS) method. In the proposed DBNAS, we construct a lightweight deeply-supervised module after each block to enable a simple supervised learning scheme and leverage ground-truth labels to indirectly supervise optimization of each block progressively. Besides, the deeply-supervised module is specifically designed as structural and functional condensation of the supernet, which establishes global awareness for progressive block-wise optimization and helps search for promising architectures. Experimental results show that the DBNAS method only takes less than 1 GPU day to search out promising architectures on the ImageNet dataset with less GPU memory footprint than the other block-wise NAS works. The best-performing model among the searched DBNAS family achieves 75.6% Top-1 accuracy on ImageNet, which is competitive with the state-of-the-art NAS models. Moreover, our DBNAS family models also achieve good transfer performance on CIFAR-10/100, as well as two downstream tasks: object detection and semantic segmentation.

Similar Papers
  • Book Chapter
  • Citations292

Single-Path NAS: Designing Hardware-Efficient ConvNets in Less Than 4 Hours

  • Jan 01, 2020
  • Dimitrios Stamoulis +6
  • Book Chapter
  • Citations1

Comparative Analysis of Neural Architecture Search Methods for Classification of Cultural Heritage Sites

  • Jan 01, 2022
  • Sunil V Gurlahosur +8
  • Conference Article
  • Citations9

DATA: Domain-Aware and Task-Aware Self-supervised Learning

  • Jun 01, 2022
  • Qing Chang +6
  • Research Article
  • Citations5

Neural architecture search based on dual attention mechanism for image classification.

  • Jan 01, 2022
  • Mathematical Biosciences & Engineering
  • Cong Jin +3
  • Research Article
  • Citations5

ETNAS: An energy consumption task-driven neural architecture search

  • Nov 10, 2023
  • Sustainable Computing: Informatics and Systems
  • Dong Dong +5
  • Conference Article
  • Citations19

Efficient Neural Architecture Search for End-to-End Speech Recognition Via Straight-Through Gradients

  • Jan 19, 2021
  • Huahuan Zheng +2
  • PDF
  • Research Article
  • Citations5

Multi-Objective Evolutionary Neural Architecture Search with Weight-Sharing Supernet

  • Jul 15, 2024
  • Applied Sciences
  • Junchao Liang +4
  • Book Chapter
  • Citations1

Efficient Evolutionary Neural Architecture Search (NAS) by Modular Inheritable Crossover

  • Jan 01, 2020
  • Hao Tan +3
  • Research Article
  • Citations6

BossNAS Family: Block-Wisely Self-Supervised Neural Architecture Search.

  • May 01, 2025
  • IEEE transactions on pattern analysis and machine intelligence
  • Changlin Li +6
  • Research Article
  • Citations32

ReCNAS: Resource-Constrained Neural Architecture Search Based on Differentiable Annealing and Dynamic Pruning.

  • Feb 01, 2024
  • IEEE Transactions on Neural Networks and Learning Systems
  • Cheng Peng +3
  • Research Article
  • Citations22

EAT-NAS: elastic architecture transfer for accelerating large-scale neural architecture search

  • Aug 06, 2021
  • Science China Information Sciences
  • Jiemin Fang +7
  • Research Article
  • Citations18

Neural Architecture Search with Progressive Evaluation and Subpopulation Preservation

  • Oct 01, 2025
  • IEEE Transactions on Evolutionary Computation
  • Yu Xue +7
  • Book Chapter
  • Citations40

Resource Optimized Neural Architecture Search for 3D Medical Image Segmentation

  • Jan 01, 2019
  • Woong Bae +5
  • Research Article
  • Citations17

Remote Sensing Scene Classification Based on Attention-Enabled Progressively Searching

  • Jan 01, 2022
  • IEEE Transactions on Geoscience and Remote Sensing
  • Junge Shen +4
  • Research Article
  • Citations1726

AutoML: A survey of the state-of-the-art

  • Nov 24, 2020
  • Knowledge-Based Systems
  • Xin He +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.