Critical Path Based Scheduling of Multiple Applications in Heterogeneous Distributed Computing

Heterogeneous Distributed Computing (HDC) system consists of Workstations (WSs) and Personal Computers (PCs). In HDC system, each WS/PC may have different processor and performance. In order to take advantage of this diversity of processing power of a system, an effective task partitioning, scheduling, and load balancing are needed to get the optimum performance. This paper examines the effectiveness of task partitioning and scheduling strategies for image (raytracing) processing application on HDC system. The static and dynamic/Run-time Task Scheduling (RTS) strategies are shown inadequate for balancing the load of HDC system. Two adaptive tasks scheduling strategies are proposed for HDC image computing system. These adaptive strategies are: i) Master Initiate Sub-task size (MIS) strategy (based on centralized resources management approach), and ii) Worker Initiated Sub-task size (WIS) strategy (based on semi-decentralized resources management approach). The measured results show that the adaptive (MIS & WIS) strategies dramatically improve the performance of HDC raytracing system and remedy the defects of static and RTS strategies. Performances of the all investigated strategies are evaluated on manager/master and workers model of HDC system. In performance comparisons of MIS and WIS strategies, the WIS strategy shows a slightly better speedup and scalability over MIS strategy.

Loosely coupled applications composed of a potentially very large number (from tens of thousands to even billions) of tasks are commonly used in high-throughput computing and many-task computing paradigms. To efficiently execute large-scale computations which can exceed the capability in a single type of computing resources within expected time, we should be able to effectively integrate resources from heterogeneous distributed computing (HDC) systems such as clusters, grids, and clouds. In this paper, we quantitatively analyze the performance of three different real scientific applications consisting of many tasks on top of HDC systems based on a partnership of distributed computing clusters, grids, and clouds to understand the application and resource characteristics, and show practical issues that normal scientific users can face during the course of leveraging these systems. Our experimental study shows that the performance of a loosely coupled application can be significantly affected by the characteristics of a HDC system, along with hardware specification of a node, and their impacts on the performance can vary widely depending on the resource usage pattern of each application. We then devise a preference-based scheduling algorithm that can reflect characteristics and resource usage patterns of various loosely coupled applications running on top of HDC systems from our experimental study. Our preference-based scheduling algorithm can allocate the resources from different HDC systems to loosely coupled applications based on the preferences of the applications for the HDC systems. We evaluate the overall system performance over various preference types, using trace-based simulations, which can be determined based on different factors such as CPU specifications and application throughputs. Our simulation results demonstrate the importance of understanding the application and resource characteristics on effective scheduling of loosely coupled applications on the HDC systems.

A New Method for Scheduling Divisible Data on a Heterogeneous Two-Levels Hierarchical System

Previously, DAG scheduling schemes used the mean (average) of computation or communication time in dealing with temporal heterogeneity. However, it is not optimal to consider only the means of computation and communication times in DAG scheduling on a temporally (and spatially) heterogeneous distributed computing system. In this paper, it is proposed that the second order moments of computation and communication times, such as the standard deviations, be taken into account in addition to their means, in scheduling “stochastic” DAGs. An effective scheduling approach which accurately estimates the earliest start time of each node and derives a schedule leading to a shorter average parallel execution time has been developed. Through an extensive computer simulation, it has been shown that a significant improvement (reduction) in the average parallel execution times of stochastic DAGs can be achieved by the proposed approach.

Over the past several years, a heterogeneous computing (HC) system has become more competative as a commercial computing platform than a homogeneous system. With the growing scale of HC systems, network failures become inevitable. To achieve high performance, communication reliability should be considered while designing reliability-aware task scheduling algorithms. In this paper, we propose a new algorithm called RMSR (Replication-based scheduling for Maximizing System Reliability), which incorporates task communication into system reliability. To maximize communication reliability, an improved algorithm which searches all optimal reliability communication paths for current tasks is proposed. During the task replication phase, the task reliability threshold is determined by users and each task has dynamic replicas. Our comparative studies based on randomly generated graphs show that our RMSR algorithm outperforms existing scheduling algorithms in terms of system reliability. Several factors affecting the performance are analyzed in the paper.

The task assignment problem is one of assigning tasks of a parallel program among the processors of a distributed computing system in order to reduce the job turnaround time and to increase the throughput of the system. Since the task assignment problem is known to be NP‐complete except in a few special situations, satisfactory suboptimal solutions obtainable in a reasonable amount of computation time are generally sought. In the paper we introduce a technique based on the problem‐space genetic algorithm (PSGA) for the static task assignment problem in both homogeneous and heterogeneous distributed computing systems to reduce the task turnaround time and to increase the throughput of the system by properly balancing the load and reducing the interprocessor communication time among processors. The PSGA based approach combines the power of genetic algorithms, a global search method, with a simple and fast problem‐specific heuristic to search a large solution space efficiently and effectively to find the best possible solution in an acceptable CPU time. Experimental results on test examples from the literature show considerable improvements in both the assignment cost and the CPU times over the previous work. The proposed scheme is also applied to a digital signal processing (DSP) system consisting of 119 tasks to illustrate its balancing properties and computational advantage on a large system. The proposed scheme offers 12–30% improvement in the assignment cost as compared to the previous best known results for the DSP example.

Finding an optimal solution to the problem of scheduling an application modeled by a directed acyclic graph (DAG) onto a distributed system is known to be NP-complete. The complexity of the problem increases when task scheduling is to be done in a heterogeneous computing system, where the processors in the network may not be identical and take different amounts of time to execute the same task. This paper introduces a performance effective task scheduling (PETS) algorithm for network of heterogeneous system, with complexity O(v+e) (p+log v), which provides optimal results for applications represented by DAGs. The performance of the algorithm is illustrated by comparing the schedule length, speedup, efficiency and the scheduling time with existing algorithms such as, heterogeneous earliest finish time (HEFT) and critical-path on a processor (CPOP) and levelized min time (LMT) reported in this paper. The comparison study based on both randomly generated graphs and graphs of some real applications shows that PETS algorithm substantially outperforms existing algorithms

article Free Access Share on Software support for heterogeneous computing Authors: Howard Jay Siegel Purdue University, West Lafayette, Indiana Purdue University, West Lafayette, IndianaView Profile , Henry G. Dietz Purdue University, West Lafayette, Indiana Purdue University, West Lafayette, IndianaView Profile , John K. Antonio Texas Tech University, Lubbock Texas Tech University, LubbockView Profile Authors Info & Claims ACM Computing SurveysVolume 28Issue 1March 1996 pp 237–239https://doi.org/10.1145/234313.234411Published:01 March 1996Publication History 21citation711DownloadsMetricsTotal Citations21Total Downloads711Last 12 Months67Last 6 weeks17 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

Executing complicated computations in parallel increases the speed of computing and brings user delight to the system. Decomposing the program into several small programs and running multiple parallel processors are modeled by Directed Acyclic Graph. Scheduling nodes to execute this task graph is an important problem that will speed up computations. Since task scheduling in this graph belongs to NP-hard problems, various algorithms were developed for node scheduling to contribute to quality service delivery. The present study brought a heuristic algorithm named looking ahead sequencing algorithm (LASA) to cope with static scheduling in heterogeneous distributed computing systems with the intention of minimizing the schedule length of the user application. In the algorithm proposed here, looking ahead is considered as a criterion for prioritizing tasks. Also, a property called Emphasized Processor has been added to the algorithm to emphasize the task execution on a particular processor. The effectiveness of the algorithm was shown on few workflow type applications and the results of the algorithm implementation were compared with two more heuristic and meta-heuristic algorithms.

This study considers a heterogeneous computing system and corresponding workload being investigated by the Extreme Scale Systems Center (ESSC) at Oak Ridge National Laboratory (ORNL). The ESSC is part of a collaborative effort between the Department of Energy (DOE) and the Department of Defense (DoD) to deliver research, tools, software, and technologies that can be integrated, deployed, and used in both DOE and DoD environments. The heterogeneous system and workload described here are representative of a prototypical computing environment being studied as part of this collaboration. Each task can exhibit a time-varying emph{importance} or emph{utility} to the overall enterprise. In this system, an arriving task has an associated priority and precedence. The priority is used to describe the importance of a task, and precedence is used to describe how soon the task must be executed. These two metrics are combined to create a utility function curve that indicates how valuable it is for the system to complete a task at any given moment. This research focuses on using time-utility functions to generate a metric that can be used to compare the performance of different resource schedulers in a heterogeneous computing system. The contributions of this paper are: (a) a mathematical model of a heterogeneous computing system where tasks arrive dynamically and need to be assigned based on their priority, precedence, utility characteristic class, and task execution type, (b) the use of priority and precedence to generate time-utility functions that describe the value a task has at any given time, (c) the derivation of a metric based on the total utility gained from completing tasks to measure the performance of the computing environment, and (d) a comparison of the performance of resource allocation heuristics in this environment

A notable requirement of heterogeneous parallel and distributed computing systems is to maximize their processing performance and agreed upon QoS. Lots of work in this field has been done to optimize the system performance by improving certain metrics such as reliability, robustness, security, and so on. However, most of them assume that systems are running without interruption all the time and seldom consider the system's intrinsic characteristics, such as failure rate, repair rate, and lifetime. In this paper, we study how to achieve high availability based on residual lifetime analysis for heterogeneous distributed computational systems with considering their essential features. First, we provide an availability model taking into account system's expected residual lifetime. Second, we propose an objective function about the model and develop a heuristic scheduling algorithm to maximize the availability with the makespan constraint. At last, we demonstrate these advantages through the extensive simulated experiments.

Efficient task scheduling is crucial to heterogeneous cluster performance. And various scheduling methods based on random search technique have been proposed for both homogeneous and heterogeneous cluster systems. However, most of these methods have high computational overhead and poor convergence. Invasive weed optimization algorithm (IWO) is a novel bionic intelligent optimization algorithm that has fast convergence rate and easier implementation than traditional genetic algorithm (GA) based algorithm. In this paper, an IWO task scheduling (IWOTS) algorithm is proposed for heterogeneous cluster system. To the best of our knowledge, this study is the first time to apply IWO to discrete task scheduling problems. Extensive simulation experiment results show that IWOTS generally exhibits outstanding convergence performance and could produce an optimal scheduling strategy.

There has been a recent increase of interest in heterogeneous computing systems, due partly to the fact that a single parallel architecture may not be adequate for exploiting all of a program's available parallelism. In some cases, heterogeneous systems have been shown to produce higher performance for lower cost than a single large machine. However, there has been only limited work on developing techniques and frameworks for partitioning and scheduling applications across the components of a heterogeneous system. In this paper we propose a general model for describing and evaluating heterogeneous systems that considers the degree of uniformity in the processing elements and the communication channels as a measure of the heterogeneity in the system. We also propose a class of dynamic scheduling algorithms for a heterogeneous computing system interconnected with an arbitrary communication network. These algorithms execute a novel optimization technique to dynamically compute schedules based on the potentially non‐uniform computation and communication costs on the processors of a heterogeneous system. A unique aspect of these algorithms is that they easily adapt to different task granularities, to dynamically varying processor and system loads, and to systems with varying degrees of heterogeneity. Our simulations are designed to facilitate the evaluation of different scheduling algorithms under varying degrees of heterogeneity. The results show improved performance for our algorithms compared to the performance resulting from existing scheduling techniques.

Ideally, a heterogeneous computing (HC) environment, is a well-orchestrated and coordinates suite of high-performance machines that provides support for computationally intensive applications with diverse computing requirements. Such an HC system includes a heterogeneous suite of machines, high-speed interconnections, interfaces, operating systems, communication protocols, and programming environments. HC is the effective use of these diverse hardware and software components to meet the distinct and varied computational requirements of a given application. Implicit in this concept of HC is the idea that subtasks with different machine architectural requirements are embedded in the applications executed by the HC system. Two types of HC systems, mixed-mode machines and mixed-machine systems, are discussed. The goals of and open problems in HC are overviewed.

It is the need of an era to store and process big data and its applications. To process these applications, it is inevitable to use heterogeneous distributed computing systems (HeDCS). The heterogeneous distributed systems facilitate scalability, an essential characteristic for big data processing. However, to implement the scalable model, it is essential to handle performance, efficiency, optimal resource utilization and several other key constraints. Scheduling algorithms play a vital role in achieving better performance and high throughput in heterogeneous distributed computing systems. Hence, selection of a proper scheduling algorithm, for the specific application, becomes a critical task. Selection of an appropriate scheduling algorithm in heterogeneous distributed computing systems require the consideration of various parameters like scheduling type, multi-core processors, and heterogeneity. The paper discusses broadly the hierarchical classification of scheduling algorithms implemented in heterogeneous distributed computing systems and presents a comparative study of these algorithms, thus providing an insight into the significance of various parameters that play a role in the selection of a scheduling algorithm.