• Home
  • Search
  • Scheduling shared scans of large data files
  • Cite Icon77
  • https://doi.org/10.14778/1453856.1453960Copy DOI Icon

Scheduling shared scans of large data files

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

We study how best to schedule scans of large data files, in the presence of many simultaneous requests to a common set of files. The objective is to maximize the overall rate of processing these files, by sharing scans of the same file as aggressively as possible, without imposing undue wait time on individual jobs. This scheduling problem arises in batch data processing environments such as Map-Reduce systems, some of which handle tens of thousands of processing requests daily, over a shared set of files. As we demonstrate, conventional scheduling techniques such as shortest-job-first do not perform well in the presence of cross-job sharing opportunities. We derive a new family of scheduling policies specifically targeted to sharable workloads. Our scheduling policies revolve around the notion that, all else being equal, it is good to schedule nonsharable scans ahead of ones that can share IO work with future jobs, if the arrival rate of sharable future jobs is expected to be high. We evaluate our policies via simulation over varied synthetic and real workloads, and demonstrate significant performance gains compared with conventional scheduling approaches.

Similar Papers
  • Book Chapter
  • Citations6

Stochastic Project Network Scheduling Technique for Construction Projects Using GERT

  • Jan 01, 2022
  • K Pregina +1
  • Conference Article
  • Citations19

Adaptive Metric-Aware Job Scheduling for Production Supercomputers

  • Sep 01, 2012
  • Wei Tang +3
  • Research Article
  • Citations36

A hierarchical multiprocessor bandwidth reservation scheme with timing guarantees

  • Jul 09, 2009
  • Real-Time Systems
  • Hennadiy Leontyev +1
  • Research Article
  • Citations9

Fixed-priority scheduling to reduce both the dynamic and leakage energy on variable voltage processors

  • Jan 01, 2009
  • International Journal of Embedded Systems
  • Gang Quan +3
  • Conference Article
  • Citations26

Exploiting inter-thread temporal locality for chip multithreading

  • Apr 01, 2010
  • Jiayuan Meng +2
  • Research Article

Development of an Improved Deep Reinforcement Learning Framework for Spectrum and Resource Allocation in 5G Networks

  • Feb 24, 2026
  • Iconic Research and Engineering Journals
  • Joel D Kuruye +1
  • Conference Article

Automating Bridge Construction Scheduling Data with BIM and Machine Learning

  • Jan 01, 2024
  • Israel Simon Mariaca Clavel +2
  • Research Article
  • Citations13

Deep Reinforcement Learning for Dynamic Task Scheduling in Edge-Cloud Environments

  • Nov 19, 2024
  • International journal of electrical and computer engineering systems
  • D Mamatha Rani +2
  • Conference Article
  • Citations9

Review and survey of mass memories

  • Jan 01, 1963
  • L C Hobbs
  • Research Article
  • Citations3

Delay-Optimal Probabilistic Scheduling for Low-Complexity Wireless Links With Fixed Modulation and Coding: A Cross-Layer Design

  • Oct 01, 2016
  • IEEE Transactions on Vehicular Technology
  • Xiang Chen +1
  • Conference Article
  • Citations25

Dynamic Resource Provisioning and Scheduling with Deadline Constraint in Elastic Cloud

  • Apr 01, 2013
  • Guan Le +2
  • Research Article
  • Citations254

Scheduling for Cellular Federated Edge Learning With Importance and Channel Awareness

  • Aug 04, 2020
  • IEEE Transactions on Wireless Communications
  • Jinke Ren +5
  • Book Chapter
  • Citations9

Scheduling MapReduce Jobs and Data Shuffle on Unrelated Processors

  • Jan 01, 2015
  • Dimitris Fotakis +4
  • Research Article
  • Citations5

Orbital Complications and Cost Analysis of Enucleation by Myoconjunctival versus Conventional Techniques for Retinoblastoma

  • Jul 20, 2023
  • Ocular Oncology and Pathology
  • Leonardo Lando +3
  • Conference Article
  • Citations123

Data scheduling and SAR for Bluetooth MAC

  • May 15, 2000
  • M Kalia +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.