• Home
  • Search
  • Training Heterogeneous Client Models using Knowledge Distillation in Serverless Federated Learning
  • Cite Icon6
  • https://doi.org/10.1145/3605098.3636015Copy DOI Icon

Training Heterogeneous Client Models using Knowledge Distillation in Serverless Federated Learning

  • Apr 8, 2024
  • Mohak Chadha +4 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Federated Learning (FL) is an emerging machine learning paradigm that enables the collaborative training of a shared global model across distributed clients while keeping the data decentralized. Recent works on designing systems for efficient FL have shown that utilizing serverless computing technologies, particularly Function-as-a-Service (FaaS) for FL, can enhance resource efficiency, reduce training costs, and alleviate the complex infrastructure management burden on data holders. However, existing serverless FL systems implicitly assume a uniform global model architecture across all participating clients during training. This assumption fails to address fundamental challenges in practical FL due to the resource and statistical data heterogeneity among FL clients. To address these challenges and enable heterogeneous client models in serverless FL, we utilize Knowledge Distillation (KD) in this paper. Towards this, we propose novel optimized serverless workflows for two popular conventional federated KD techniques, i.e., FedMD and FedDF. We implement these workflows by introducing several extensions to an open-source serverless FL system called FedLess. Moreover, we comprehensively evaluate the two strategies on multiple datasets across varying levels of client data heterogeneity using heterogeneous client models with respect to accuracy, fine-grained training times, and costs. Results from our experiments demonstrate that server-less FedDF is more robust to extreme non-IID data distributions, is faster, and leads to lower costs than serverless FedMD. In addition, compared to the original implementation, our optimizations for particular steps in FedMD and FedDF lead to an average speedup of 3.5x and 1.76x across all datasets.

Similar Papers
  • Conference Article
  • Citations11

FedLesScan: Mitigating Stragglers in Serverless Federated Learning

  • Dec 17, 2022
  • Mohamed Elzohairy +6
  • Conference Article
  • Citations7

Coalition Formation Game in the Cross-Silo Federated Learning System

  • Oct 01, 2022
  • Suhan Jiang +1
  • Conference Article
  • Citations13

FLAC: Federated Learning with Autoencoder Compression and Convergence Guarantee

  • Dec 04, 2022
  • Mahdi Beitollahi +1
  • Conference Article
  • Citations47

Federated Learning Attacks and Defenses: A Survey

  • Dec 17, 2022
  • Yao Chen +4
  • Conference Article
  • Citations2

Maverick: Personalized Edge-Assisted Federated Learning with Contrastive Training

  • Apr 22, 2025
  • Kaibin Wang +7
  • Conference Article
  • Citations1

Personalized Federated Learning with Optimized Contrastive Learning for Intrusion Detection System

  • Dec 21, 2024
  • Quan Hong Ngo +2
  • Conference Article
  • Citations3

Towards Robust and Bias-free Federated Learning

  • May 08, 2023
  • Ousmane Touat +1
  • PDF
  • Research Article
  • Citations50

Blockchain-Enabled: Multi-Layered Security Federated Learning Platform for Preserving Data Privacy

  • May 19, 2022
  • Electronics
  • Zeba Mahmood +1
  • Conference Article
  • Citations57

FedLess: Secure and Scalable Federated Learning Using Serverless Computing

  • Dec 15, 2021
  • Andreas Grafberger +4
  • Conference Article
  • Citations10

DSFL: Dynamic Sparsification for Federated Learning

  • Dec 27, 2022
  • Mahdi Beitollahi +2
  • Book Chapter
  • Citations160

CONTRA: Defending Against Poisoning Attacks in Federated Learning

  • Jan 01, 2021
  • Sana Awan +2
  • PDF
  • Research Article
  • Citations2

Leveraging local data sampling strategies to improve federated learning

  • Aug 29, 2024
  • International Journal of Data Science and Analytics
  • Christoph Düsing +2
  • Conference Article
  • Citations6

Blockchain-Based Decentralized Federated Learning

  • Sep 05, 2022
  • Ahmed Dirir +5
  • Conference Article
  • Citations2

Subjective Logic-based Decentralized Federated Learning for Non-IID Data

  • Jul 30, 2024
  • Agnideven Palanisamy Sundar +3
  • PDF
  • Research Article
  • Citations1

ARMOR: Differential Model Distribution for Adversarially Robust Federated Learning

  • Feb 07, 2023
  • Electronics
  • Yanting Zhang +5
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.