• Home
  • Search
  • Dependency-Aware and Resource-Efficient Scheduling for Heterogeneous Jobs in Clouds
  • Cite Icon34
  • https://doi.org/10.1109/cloudcom.2016.0032Copy DOI Icon

Dependency-Aware and Resource-Efficient Scheduling for Heterogeneous Jobs in Clouds

  • Dec 1, 2016
  • Jinwei Liu +1 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Data analytics frameworks shift towards larger degrees of parallelism. Efficient scheduling of data-parallel jobs (tasks) is critical for improving job performance such as response time, and resource utilization. It is an important challenge for large scale data analytics frameworks in which jobs are more complex and have diverse characteristics (e.g., diverse resource requirements). Prior work on scheduling cannot achieve low response time and high resource utilization simultaneously because they cannot accurately estimate the durations of tasks in the queue of a worker machine by using sampling-based approach (including sampling with late binding) for task placement, and thus they fail to place tasks at the best possible worker machine. Also, they do not sufficiently consider the diverse resource requirements of jobs (tasks) for placing tasks on worker machines. To address this challenge, we propose a Dependency-aware and Resource-efficient Scheduling (DRS) to achieve low response time and high resource utilization. DRS takes into account task dependency and assigns tasks that are independent of each other to different worker machines. Also, DRS considers tasks' resource requirements and packs complementary tasks whose resource demands on multiple resources are complementary to each other to increase the resource utilization. In addition, DRS uses the mutual reinforcement learning to estimate the task's waiting time (the duration of tasks in the queue of a worker), and assigns tasks to workers with the consideration of tasks' waiting time to reduce the response time. Extensive experimental results based on a real cluster and experiments using real-world Amazon EC2 cloud service show that DRS achieves low response time and high resource utilization compared to previous strategies.

Similar Papers
  • Research Article
  • Citations21

Healthcare resource utilization for chronic low back pain among high-utilizers

  • Dec 09, 2023
  • The spine journal : official journal of the North American Spine Society
  • Elayna P Kirsch +4
  • Conference Article
  • Citations6

An Adaptive Resource Allocation Approach Based on User Demand Forecasting for E-Healthcare Systems

  • May 16, 2022
  • Lin Zhang +5
  • Conference Article

Research on Automated Traffic Scheduling Technology for Dual Active Disaster Recovery Centers in Two Regions

  • Sep 03, 2025
  • Xi Zhang +5
  • Research Article
  • Citations46

Factors associated with high healthcare resource utilisation among COPD patients

  • Oct 08, 2012
  • Respiratory Medicine
  • Cayo García-Polo +12
  • PDF
  • Research Article
  • Citations5

Dynamic Logo Design System of Network Media Art Based on Convolutional Neural Network

  • May 31, 2022
  • Mobile Information Systems
  • Zhi Yang
  • Conference Article
  • Citations3

Fault-tolerance with dual module redundancy for dynamic reconfigurable FPGAs

  • Dec 01, 2011
  • Qi Zhong Zhou +3
  • Research Article
  • Citations4

Clinical low vision resource usage prediction.

  • Jul 01, 1994
  • Optometry and vision science : official publication of the American Academy of Optometry
  • David M Dilts +2
  • Abstract

S0106 Imaging Resource Utilization in the Surveillance of Presumed Branched Duct Intraductal Papillary Mucinous Neoplasm (BD-IPMN)

  • Oct 01, 2020
  • American Journal of Gastroenterology
  • Muhammad N Yousaf +11
  • Research Article
  • Citations25

Decentralized saddle-point dynamics solution for optimal power flow of distribution systems with multi-microgrids

  • Jun 06, 2019
  • Applied Energy
  • Chong Tang +4
  • Research Article
  • Citations18

Dynamic inter-SLA resource sharing in path-oriented differentiated services networks

  • Jun 01, 2006
  • IEEE/ACM Transactions on Networking
  • Yu Cheng +1
  • Research Article
  • Citations7

The association between urgency level and hospital admission, mortality and resource utilization in three emergency department triage systems: an observational multicenter study

  • May 01, 2025
  • Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine
  • Marit E Van Wegen +4
  • Conference Article
  • Citations4

OMBM: Optimized Memory Bandwidth Management for Ensuring QoS and High Server Utilization

  • Sep 01, 2017
  • Hanul Sung +3
  • Abstract
  • Citations1

Characteristics of Sickle Cell Patients with Frequent ED Visits and Hospitalizations: Demographic, Clinical, Laboratory and Psycho-Social Aspects

  • Dec 02, 2016
  • Blood
  • Kyle Michael Kidwell +11
  • Research Article
  • Citations5

PiPar: Pipeline parallelism for collaborative machine learning

  • Jul 02, 2024
  • Journal of Parallel and Distributed Computing
  • Zihan Zhang +4
  • Research Article
  • Citations7

Assessing Mental Health and Its Association With Income and Resource Utilization in Old-Old Chinese in Hong Kong

  • Mar 01, 2005
  • The American Journal of Geriatric Psychiatry
  • James X Zhang +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.