• Home
  • Search
  • Sparse Training for Federated Learning With Regularized Error Correction
  • Open Access IconOpen Access
  • Cite Icon3
  • https://doi.org/10.1109/jstsp.2024.3463505Copy DOI Icon

Sparse Training for Federated Learning With Regularized Error Correction

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Federated Learning (FL) is an emerging paradigm that allows for decentralized machine learning (ML), where multiple models are collaboratively trained in a privacy-preserving manner. However, training DNN models in FL systems face challenges such as elevated computational and communication costs in complex tasks. Sparse training schemes gain increasing attention in order to scale down the dimensionality of each client transmission. Specifically, sparsification with error correction methods is a promising technique, where only important updates are sent to the parameter server (PS) and the rest are accumulated locally. While error correction methods have shown to achieve a significant sparsification level without harming convergence, pushing sparsity further remains unresolved due to the staleness effect. In this paper, we propose a novel algorithm, dubbed Federated Learning with Accumulated Regularized Embeddings (FLARE), to overcome this challenge. FLARE presents a novel sparse training approach via accumulated pulling of the updated models with regularization on the embeddings in the FL process, providing a powerful solution to the staleness effect, and pushing sparsity to an exceptional level. Our theoretical analysis demonstrates that FLARE not only matches state-of-the-art performance in terms of convergence rate with time, but also achieves significant improvements in scalability with sparsity parameter. The empirical performance of FLARE is validated through extensive experiments on diverse and complex models, achieving a remarkable sparsity level (10 times and more beyond the current state-of-the-art) along with significantly improved accuracy. Additionally, an open-source software package has been developed for the benefit of researchers and developers in related fields.

Similar Papers
  • Research Article
  • Citations21

Revolutionizing healthcare data analytics with federated learning: A comprehensive survey of applications, systems, and future directions.

  • Jan 01, 2025
  • Computational and structural biotechnology journal
  • Nisha Thorakkattu Madathil +4
  • Research Article
  • Citations7

Accelerating Federated Learning with genetic algorithm enhancements

  • Jul 01, 2025
  • Expert Systems with Applications
  • Huanqing Zheng +4
  • PDF
  • Research Article
  • Citations47

Applicability of Deep Reinforcement Learning for Efficient Federated Learning in Massive IoT Communications

  • Feb 27, 2023
  • Applied Sciences
  • Prohim Tam +3
  • Research Article
  • Citations16

Accelerating Federated Learning via Parallel Servers: A Theoretically Guaranteed Approach

  • Oct 01, 2022
  • IEEE/ACM Transactions on Networking
  • Xuezheng Liu +6
  • PDF
  • Research Article
  • Citations49

Blockchain-Enabled: Multi-Layered Security Federated Learning Platform for Preserving Data Privacy

  • May 19, 2022
  • Electronics
  • Zeba Mahmood +1
  • Conference Article
  • Citations10

DSFL: Dynamic Sparsification for Federated Learning

  • Dec 27, 2022
  • Mahdi Beitollahi +2
  • Research Article
  • Citations15

Federated Learning Hyperparameter Tuning From a System Perspective

  • Aug 15, 2023
  • IEEE Internet of Things Journal
  • Huanle Zhang +6
  • Research Article
  • Citations43

Gradient Leakage Attacks in Federated Learning: Research Frontiers, Taxonomy, and Future Directions

  • Mar 01, 2024
  • IEEE Network
  • Haomiao Yang +5
  • Research Article

From privacy to trust in the agentic era: a taxonomy of challenges in trustworthy federated learning through the lens of trust report 2.0

  • Aug 01, 2026
  • Information Fusion
  • Nuria Rodríguez-Barroso +3
  • Research Article
  • Citations15

Contrastive encoder pre-training-based clustered federated learning for heterogeneous data

  • Jun 10, 2023
  • Neural Networks
  • Ye Lin Tun +4
  • Research Article
  • Citations14

Energy-Aware Incentive Mechanism for Hierarchical Federated Learning Using Water Filling Technique

  • Dec 01, 2024
  • IEEE Transactions on Industrial Informatics
  • Yangguang Cui +6
  • Book Chapter
  • Citations151

CONTRA: Defending Against Poisoning Attacks in Federated Learning

  • Jan 01, 2021
  • Sana Awan +2
  • Research Article
  • Citations10

Two-Stage Client Selection for Federated Learning Against Free-Riding Attack: A Multiarmed Bandits and Auction-Based Approach

  • Oct 15, 2024
  • IEEE Internet of Things Journal
  • Renhao Lu +9
  • Research Article
  • Citations7

FedDT: A Communication-Efficient Federated Learning via Knowledge Distillation and Ternary Compression

  • May 28, 2025
  • Electronics
  • Zixiao He +4
  • Research Article
  • Citations142

Toward On-Device Federated Learning: A Direct Acyclic Graph-Based Blockchain Approach.

  • Apr 01, 2023
  • IEEE Transactions on Neural Networks and Learning Systems
  • Mingrui Cao +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.