• Home
  • Search
  • When Federated Learning Meets Knowledge Distillation
  • Cite Icon16
  • https://doi.org/10.1109/mwc.016.2300523Copy DOI Icon

When Federated Learning Meets Knowledge Distillation

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Federated learning (FL) has garnered significant attention in the Internet-of-things (IoT) domain due to its ability to facilitate collaborative learning among distributed privacy-sensitive devices without compromising their local data. However, FL's development and application are hindered by the heterogeneity, vulnerability, and limited computing and wireless communication resources of IoT systems. To solve the issues, knowledge distillation (KD), a technique that can transfer learned knowledge from one network to another in a quick and light manner, is incorporated into FL to enhance its usability and universality in IoT. KD-based FL can effectively achieve collaborative learning among resource-limited IoT devices that are heterogeneous in data distribution, model architectures, or quantity of resources, and offer enhanced privacy guarantees. Given the increasing adoption and benefits of KD in FL, it is essential to review KD-based FL schemes to identify the common methodologies and potential future directions. In this article, we examine the challenges associated with applying FL in IoT and sort out three key steps for incorporating KD into FL, along with the general workflow for KD-based FL. Furthermore, we conduct a comprehensive retrospective analysis of existing KD-based FL schemes and tease out their approaches to utilize KD to solve challenges, and then we compare them based on various design aspects. Based on our analysis and comparison, we shed light on several unresolved research questions that warrant further investigation.

Similar Papers
  • Research Article
  • Citations11

Knowledge Distillation in Federated Learning: A Survey on Long Lasting Challenges and New Solutions

  • Jan 01, 2025
  • International Journal of Intelligent Systems
  • Laiqiao Qin +3
  • Conference Article
  • Citations228

End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things

  • Sep 01, 2020
  • Yansong Gao +8
  • Research Article

A Hybrid‐Update Efficient Federated Learning Method Based on Multi‐Teacher Knowledge Distillation in the Internet of Things

  • Sep 28, 2025
  • Concurrency and Computation: Practice and Experience
  • Yang Lan +2
  • Research Article
  • Citations28

Federated learning vulnerabilities, threats and defenses: A systematic review and future directions

  • Sep 26, 2023
  • Internet of Things
  • Suzan Almutairi +1
  • Research Article
  • Citations9

PFedKD: Personalized Federated Learning via Knowledge Distillation Using Unlabeled Pseudo Data for Internet of Things

  • Jun 01, 2025
  • IEEE Internet of Things Journal
  • Hanxi Li +7
  • PDF
  • Research Article
  • Citations71

Federated Learning for the Internet-of-Medical-Things: A Survey

  • Dec 28, 2022
  • Mathematics
  • Vivek Kumar Prasad +10
  • Research Article
  • Citations38

Efficient Federated Learning for AIoT Applications Using Knowledge Distillation

  • Apr 15, 2023
  • IEEE Internet of Things Journal
  • Tian Liu +5
  • Research Article
  • Citations3

Human Immune System Inspired Security for Federated Learning-Empowered Internet of Things

  • Apr 14, 2025
  • ACM Transactions on Internet of Things
  • Aashma Uprety +2
  • Book Chapter
  • Citations6

Fusion of Federated Learning and 6G in Internet-of-Medical-Things: Architecture, Case Study and Emerging Directions

  • Jan 01, 2022
  • Ashwin Verma +4
  • Conference Article

FedScope-KD: Knowledge Distillation-Enhanced Federated Learning via Shared Composition and Personalized Exploration for Heat Emission Prediction in Additive Manufacturing

  • Jun 23, 2025
  • Rong Lei +2
  • Research Article
  • Citations528

Federated learning review: Fundamentals, enabling technologies, and future applications

  • Aug 26, 2022
  • Information Processing & Management
  • Syreen Banabilah +4
  • Research Article

Personalized User Models in a Real-world Edge Computing Environment: A Peer-to-peer Federated Learning Framework

  • Feb 07, 2025
  • Journal of Web Engineering
  • Xiangchi Song +3
  • Research Article
  • Citations78

On the Physical Layer Security of Federated Learning Based IoMT Networks.

  • Feb 01, 2023
  • IEEE Journal of Biomedical and Health Informatics
  • Junaid Ahmed +4
  • Research Article
  • Citations19

Knowledge Selection and Local Updating Optimization for Federated Knowledge Distillation With Heterogeneous Models

  • Jan 01, 2023
  • IEEE Journal of Selected Topics in Signal Processing
  • Dong Wang +3
  • Research Article
  • Citations9

Resource Efficient Federated LoRaWAN Architecture for Far-Edge IoT Applications

  • Jan 01, 2025
  • IEEE Access
  • Anna Triantafyllou +6
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.