• Home
  • Search
  • Medical Image Segmentation Approach via Transformer Knowledge Distillation
  • Cite Icon1
  • https://doi.org/10.1145/3596286.3596292Copy DOI Icon

Medical Image Segmentation Approach via Transformer Knowledge Distillation

  • Apr 28, 2023
  • Tian-Tian Zhang +3 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Numerous transformer-based medical image segmentation methods have been proposed and achieved good segmentation results. However, it is still a challenge to train and deploy transformer networks to mobile medical devices due to a large number of model parameters. To resolve the training and model parameter problems, in this paper, we propose a Transformer-based network for Medical Image Segmentation using Knowledge Distillation named MISTKD. The MISTKD consists of a teacher network and a student network. It achieves comparable performance to state-of-the-art transformer works using fewer parameters by employing the teacher network to train the student network. The training can be implemented by extracting the sequence in the teacher and student encoder networks during the training procedure. The losses between sequences are further calculated, thus the student network can learn from the teacher network. The experimental results on Synapse show that the proposed work achieves competitive performance using only one-eighth parameters.

Similar Papers
  • Research Article
  • Citations16

MC-DC: An MLP-CNN Based Dual-path Complementary Network for Medical Image Segmentation

  • Oct 05, 2023
  • Computer Methods and Programs in Biomedicine
  • Xiaoben Jiang +4
  • Research Article
  • Citations18

MH-Net: Model-data-driven hybrid-fusion network for medical image segmentation

  • Apr 19, 2022
  • Knowledge-Based Systems
  • Yunyun Yang +5
  • Research Article
  • Citations8

Segmenting medical images via explicit–implicit attention aggregation

  • Aug 30, 2023
  • Knowledge-Based Systems
  • Bangcheng Zhan +5
  • Research Article
  • Citations33

LCP-Net: A local context-perception deep neural network for medical image segmentation

  • Nov 06, 2020
  • Expert Systems with Applications
  • Dunlu Peng +3
  • Research Article
  • Citations24

MHSU-Net: A more versatile neural network for medical image segmentation

  • Jun 06, 2021
  • Computer Methods and Programs in Biomedicine
  • Hao Ma +2
  • Research Article
  • Citations3

Scale-wise discriminative region learning for medical image segmentation

  • Nov 22, 2023
  • Biomedical Signal Processing and Control
  • Jing Zhang +3
  • Research Article
  • Citations25

Explainable multi-module semantic guided attention based network for medical image segmentation

  • Oct 25, 2022
  • Computers in Biology and Medicine
  • Meghana Karri +2
  • Research Article
  • Citations9

Shape prior-constrained deep learning network for medical image segmentation

  • Jul 29, 2024
  • Computers in Biology and Medicine
  • Pengfei Zhang +2
  • PDF
  • Research Article
  • Citations55

SA-Net: A scale-attention network for medical image segmentation.

  • Apr 14, 2021
  • PLOS ONE
  • Jingfei Hu +5
  • Research Article
  • Citations14

FBCU-Net: A fine-grained context modeling network using boundary semantic features for medical image segmentation

  • Oct 03, 2022
  • Computers in biology and medicine
  • Mei Yu +5
  • PDF
  • Research Article
  • Citations58

A novel medical image segmentation approach by using multi-branch segmentation network based on local and global information synchronous learning

  • Apr 25, 2023
  • Scientific reports
  • Shangzhu Jin +4
  • Conference Article

ES-Unet: U-shaped image segmentation network based on Transformer and CNN

  • Dec 02, 2022
  • Rui Wang +3
  • Research Article
  • Citations11

MultiTrans: Multi-branch transformer network for medical image segmentation

  • Jun 08, 2024
  • Computer Methods and Programs in Biomedicine
  • Yanhua Zhang +5
  • Research Article

Deconver: A Deconvolutional Network for Medical Image Segmentation.

  • Jan 01, 2025
  • IEEE journal of biomedical and health informatics
  • Pooya Ashtari +6
  • Book Chapter
  • Citations23

High-Order Attention Networks for Medical Image Segmentation

  • Jan 01, 2020
  • Fei Ding +6
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.