• Home
  • Search
  • Exploring Decentralized Collaboration in Heterogeneous Edge Training
  • Cite Icon7
  • https://doi.org/10.1109/sec50012.2020.00069Copy DOI Icon

Exploring Decentralized Collaboration in Heterogeneous Edge Training

  • Nov 1, 2020
  • Xiang Chen +1 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Recent progress in deep learning techniques enabled collaborative edge training, which usually deploys identical neural network models globally on multiple devices for aggregating parameter updates over distributed data collection. However, as more and more heterogeneous edge devices are involved in practical training, the identical model deployment over collaborative edge devices cannot be guaranteed: On one hand, the weak edge devices with less computation resources may not catch up stronger ones' training progress, and appropriate local model training customization is necessary to balance the collaboration. On the other hand, a particular local edge device may have specific learning task preference, while the global identical model would exceed the practical local demand and cause unnecessary computation cost. Therefore, we explored the collaborative learning with heterogeneous convolutional neural networks (CNNs) in this work, expecting to address aforementioned real problems. Specifically, we proposed a novel decentralized collaborative training method by decoupling a training target CNN model into independently trainable sub-models correspond to a sub-set of learning tasks for each edge device. After sub-models are well-trained on edge nodes, the model parameters for individual learning tasks can be harvested from local models on every edge device and ensemble the global training model back to a single piece. Experiments demonstrate that, for the AlexNet and VGG on the CIFAR10, CIFAR100 and KWS dataset, our decentralized training method can save up to 11.8× less computation load while achieve central sever test accuracy.

Similar Papers
  • Conference Article
  • Citations435

Asynchronous Online Federated Learning for Edge Devices with Non-IID Data

  • Dec 10, 2020
  • Yujing Chen +3
  • Conference Article
  • Citations13

Towards Efficient Inference: Adaptively Cooperate in Heterogeneous IoT Edge Cluster

  • Jul 01, 2021
  • Xiang Yang +4
  • Conference Article
  • Citations2

Model Parallelism Optimization for Distributed DNN Inference on Edge Devices

  • Nov 24, 2023
  • Meng Wang +4
  • Research Article
  • Citations30

Self-aware distributed deep learning framework for heterogeneous IoT edge devices

  • Jul 14, 2021
  • Future Generation Computer Systems
  • Yi Jin +8
  • Conference Article
  • Citations27

EdgeFaaSBench: Benchmarking Edge Devices Using Serverless Computing

  • Jul 01, 2022
  • Kaustubh Rajendra Rajput +4
  • Conference Article
  • Citations5

FedPruNet: Federated Learning Using Pruning Neural Network

  • Jul 01, 2022
  • L Gowtham +2
  • Book Chapter
  • Citations1

A Hot/Cold Task Partition for Energy-Efficient Neural Network Deployment on Heterogeneous Edge Device

  • Jan 01, 2020
  • Jihe Wang +2
  • Conference Article
  • Citations7

Deep-Edge: An Efficient Framework for Deep Learning Model Update on Heterogeneous Edge

  • May 01, 2020
  • Anirban Bhattacharjee +6
  • Conference Article
  • Citations6

Towards Resource-aware DNN Partitioning for Edge Devices with Heterogeneous Resources

  • Dec 04, 2022
  • Muhammad Zawish +3
  • Conference Article

EdgeCPS-AI Knowledge Sharing Model for Supporting Computing Partition Services*

  • Oct 01, 2023
  • Young-Joo Kim +2
  • Research Article
  • Citations2

Motivating Learners in Multiorchestrator Mobile Edge Learning: A Stackelberg Game Approach

  • Jan 01, 2023
  • IEEE Canadian Journal of Electrical and Computer Engineering
  • Mhd Saria Allahham +3
  • Research Article
  • Citations40

Automated Exploration and Implementation of Distributed CNN Inference at the Edge

  • Apr 01, 2023
  • IEEE Internet of Things Journal
  • Xiaotian Guo +2
  • Conference Article

PCHEC: A Private Coded Computation Scheme For Heterogeneous Edge Computing

  • Oct 01, 2021
  • Jiqing Chang +5
  • Conference Article
  • Citations12

Parallel Detection for Efficient Video Analytics at the Edge

  • Dec 01, 2021
  • Yanzhao Wu +2
  • Conference Article
  • Citations72

Distributed Inference with Deep Learning Models across Heterogeneous Edge Devices

  • May 02, 2022
  • Chenghao Hu +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.