• Home
  • Search
  • Instance segmentation on distributed deep learning big data cluster
  • Cite Icon8
  • https://doi.org/10.1186/s40537-023-00871-9Copy DOI Icon

Instance segmentation on distributed deep learning big data cluster

Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Abstract Distributed deep learning is a promising approach for training and deploying large and complex deep learning models. This paper presents a comprehensive workflow for deploying and optimizing the YOLACT instance segmentation model as on big data clusters. OpenVINO, a toolkit known for its high-speed data processing and ability to optimize deep learning models for deployment on a variety of devices, was used to optimize the YOLACT model. The model is then run on a big data cluster using BigDL, a distributed deep learning library for Apache Spark. BigDL provides a high-level programming interface for defining and training deep neural networks, making it suitable for large-scale deep learning applications. In distributed deep learning, input data is divided and distributed across multiple machines for parallel processing. This approach offers several advantages, including the ability to handle very large data that can be stored in a distributed manner, scalability to decrease processing time by increasing the number of workers, and fault tolerance. The proposed workflow was evaluated on virtual machines and Azure Databricks, a cloud-based platform for big data analytics. The results indicated that the workflow can scale to large datasets and deliver high performance on Azure Databricks. This study explores the benefits and challenges of using distributed deep learning on big data clusters for instance segmentation. Popular distributed deep learning frameworks are discussed, and BigDL is chosen. Overall, this study highlights the practicality of distributed deep learning for deploying and scaling sophisticated deep learning models on big data clusters.

Loading PDF

Similar Papers
  • Research Article
  • Citations48

A Quantitative Survey of Communication Optimizations in Distributed Deep Learning

  • Dec 02, 2020
  • IEEE Network
  • Shaohuai Shi +5
  • Conference Article
  • Citations10

BAIPAS: Distributed Deep Learning Platform with Data Locality and Shuffling

  • Nov 01, 2017
  • Mikyoung Lee +3
  • PDF
  • Research Article
  • Citations24

Distributed Deep Learning and Energy-Efficient Real-Time Image Processing at the Edge for Fish Segmentation in Underwater Videos

  • Jan 01, 2022
  • IEEE Access
  • Mohammad Jahanbakht +3
  • Book Chapter

Distributed Deep Learning with Data Parallelism for Diabetic Retinopathy Classification

  • Jan 01, 2023
  • Sanskruti S Patil +5
  • Conference Article
  • Citations3

AccDP: Accelerated Data-Parallel Distributed DNN Training for Modern GPU-Based HPC Clusters

  • Dec 01, 2022
  • Nawras Alnaasan +4
  • Research Article
  • Citations1

DPro-SM – A distributed framework for proactive straggler mitigation using LSTM

  • Dec 10, 2023
  • Heliyon
  • Aswathy Ravikumar +1
  • PDF
  • Research Article
  • Citations31

Distributed Deep Reinforcement Learning: A Survey and a Multi-player Multi-agent Learning Toolbox

  • Jan 11, 2024
  • Machine Intelligence Research
  • Qiyue Yin +8
  • Research Article
  • Citations5

D3: Differential Testing of Distributed Deep Learning With Model Generation

  • Jan 01, 2025
  • IEEE Transactions on Software Engineering
  • Jiannan Wang +6
  • Book Chapter

Prospects of Deep Learning and Edge Intelligence in Agriculture

  • Jun 30, 2023
  • Ali Shaheen +1
  • PDF
  • Research Article
  • Citations19

A hybrid CNN and ensemble model for COVID-19 lung infection detection on chest CT scans.

  • Mar 09, 2023
  • PLOS ONE
  • Ahmed A Akl +3
  • PDF
  • Research Article
  • Citations24

Cross-institutional evaluation of deep learning and radiomics models in predicting microvascular invasion in hepatocellular carcinoma: validity, robustness, and ultrasound modality efficacy comparison

  • Oct 22, 2024
  • Cancer Imaging
  • Weibin Zhang +7
  • Research Article

Security and Privacy Preservation via Interference Tolerant Fast Convergence Zeroing Neural Network With Reptile Search Optimization Algorithm in Fog‐Cloud Computing

  • Apr 01, 2025
  • Transactions on Emerging Telecommunications Technologies
  • Pakkarisamy Janakiraman Sathish Kumar +3
  • PDF
  • Research Article
  • Citations2

Enhanced Sequence-to-Sequence Deep Transfer Learning for Day-Ahead Electricity Load Forecasting

  • May 20, 2024
  • Electronics
  • Vasileios Laitsos +4
  • Research Article
  • Citations4

A Novel Ant Colony Based DBN Framework to Analyze the Drug Reviews

  • Dec 08, 2021
  • International Journal of Intelligent Systems and Applications
  • Nazia Tazeen +1
  • Research Article
  • Citations9

Multimodal ultrasound deep learning to detect fibrosis in early chronic kidney disease

  • Oct 22, 2024
  • Renal Failure
  • Xiachuan Qin +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.