• Home
  • Search
  • Decentralized Deep Learning Using Momentum-Accelerated Consensus
  • Cite Icon16
  • https://doi.org/10.1109/icassp39728.2021.9414564Copy DOI Icon

Decentralized Deep Learning Using Momentum-Accelerated Consensus

  • Jun 6, 2021
  • Aditya Balu +5 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

We consider the problem of decentralized deep learning where multiple agents collaborate to learn from a distributed dataset. While several decentralized deep learning approaches exist, the majority consider a central parameter-server topology for aggregating the model parameters from the agents. However, such a topology may be inapplicable in networked systems such as ad-hoc mobile networks, field robotics, and power network systems where direct communication with the central parameter server may be inefficient. In this context, we propose and analyze a novel decentralized deep learning algorithm where the agents interact over a fixed communication topology (without a central server). Our algorithm is based on the heavy-ball acceleration method used in gradient-based optimization. We propose a novel consensus protocol where each agent shares with its neighbors its model parameters and gradient-momentum values during the optimization process. We consider nonconvex objective functions and theoretically analyze our algorithm’s performance. We present several empirical comparisons with competing decentralized learning methods to demonstrate the efficacy of our approach under different communication topologies.

Similar Papers
  • Research Article
  • Citations276

On global leader-following consensus of identical linear dynamic systems subject to actuator saturation

  • Dec 22, 2012
  • Systems & Control Letters
  • Ziyang Meng +2
  • Conference Article
  • Citations4

Observer design for consensus of general fractional-order multi-agent systems

  • Jun 01, 2014
  • Yang Li +4
  • Research Article
  • Citations64

Co-simulation of wireless networked control systems over mobile ad hoc network using SIMULINK and OPNET

  • Aug 01, 2009
  • IET Communications
  • M.S Hasan +3
  • Research Article
  • Citations14

A Federated Learning Framework for Brain Tumor Segmentation Without Sharing Patient Data

  • Jul 01, 2024
  • International Journal of Imaging Systems and Technology
  • Wei Zhang +4
  • Research Article

P3DL: A Privacy Preserving Personalized Distributed Learning Framework for EEG-Based Cognitive State Identification.

  • Jan 01, 2025
  • IEEE journal of biomedical and health informatics
  • Yu Ouyang +4
  • Research Article

Blockchain-Enabled Network Intrusion Detection System for Secure and Transparent Threat Logging

  • Jan 08, 2026
  • International Journal of Advanced Research in Science Communication and Technology
  • Piyush Katre Mrs N.A Inamdar +1
  • Conference Article
  • Citations5

Analysing credit based ARAN to detect selfish nodes in MANET

  • Aug 01, 2014
  • A.M Kurkure +1
  • Single Book
  • Citations7

Vehicular Social Networks

  • Mar 31, 2017
  • Athanasios V Vasilakos
  • Book Chapter
  • Citations1

Genetic-Algorithm-Based Optimization of Clustering in Mobile Ad Hoc Network

  • Jan 01, 2016
  • Koushik Majumder +3
  • Research Article
  • Citations18

Secure intrusion detection system routing protocol for mobile ad‐hoc network

  • Oct 16, 2021
  • Global Transitions Proceedings
  • Rajendra Prasad P +1
  • Conference Article

Load Management of Economic Power Dispatch using Optimization Algorithm

  • Jun 24, 2022
  • Ravindra M Malkar +2
  • Research Article
  • Citations2

Consensus of discrete-time multi-agent systems with consecutive packet losses in directed communication topology

  • Jan 12, 2023
  • Transactions of the Institute of Measurement and Control
  • Bing Chen +2
  • Conference Article

Routing System in Mobile Ad-Hoc Networks

  • Sep 01, 2006
  • A Minochkin +1
  • Conference Article
  • Citations2

A study of Multiagent System operation within dynamic ad hoc networks

  • Nov 01, 2008
  • Justin W Dean +2
  • Research Article

전술 모바일 애드 혹 네트워크를 위한 위치 기반의 신뢰성 제공을 위한 라우팅 방법

  • Nov 30, 2012
  • The Journal of Korea Information and Communications Society
  • Myungjune Youn +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.