Parallel algorithms for on-line dynamic data compression

For better utilization of computing resources, it is important to consider parallel programming environments in which the number of available processors varies at runtime. In this paper, we discuss runtime support for data parallel programming in such an adaptive environment. Executing data parallel-programs in an adaptive environment requires redistributing data when the number of processors changes, and also requires determining new loop bounds and communication patterns for the new set of processors. We have developed a runtime library to provide this support. We also present performance results for a multiblock Navier-Stokes solver run on a network of workstations using PVM for message passing. Our experiments show that if the number of processors is not varied frequently, the cost of data redistribution is not significant compared to the time required for the actual computations. >

We address the parallelization of the determination of all optimal solutions (DAOS) for the 1D array partitioning problem, an easy combinatorial optimization problem that may be solved by dynamic programming. It turns out that the designed DAOS algorithm is a polyhedral algorithm (PA), i.e. a FOR-loop nest with affine loop bounds, whose iteration space that has to be scanned may be of exponential cardinality in the array size hence too time consuming. Thus, after a cardinality reducing procedure leading to an improved PA (IPA), we propose to parallelize the IPA through a specific approach. This latter first consists in a dependence analysis within the IPA leading to detect that all its loops are parallel. We then apply a loop interchange on the IPA in order to have a new IPA whose first loop has the largest number of parallel iterations. Another alternative consists in moving to the first position the loop whose number of iterations is the closest to the number of available processors. We detail an analysis of the theoretical performances of the derived parallel IPA by proposing several versions fitting the number of available processors and reducing the overhead due to processor synchronizations.

This paper presents a static scheduler which selects, from a set of scheduling algorithms, a subset of those which carry out the best assignment of a DAG representing an application program. To make that selection, some characteristics of the DAG, a decision model, and the evaluation parameters for choosing the best solution provided by the selected scheduling algorithms are defined. The selection of the scheduling algorithms is based on five decision levels. At each level, a subset of scheduling algorithms is selected. The final decision is made by carrying out an intersection operation between the subsets. When the scheduler was tested with a series of DAGs having different characteristics, the scheduler's decision was right 95% of the time in those cases in which the number of available processors is known, and 75% of the time when the number of processors is unbounded.

We present an O((log log N)/sup 2/) -time algorithm for computing the distance transform of an N /spl times/ N binary image. Our algorithm is designed for the common concurrent read concurrent write parallel random access machine (CRCW PRAM) and requires O(N/sup 2+/spl epsi///log log N) processors, for any /spl epsi/ such that 0 < /spl epsi/ < 1. Our algorithm is based on a novel deterministic sampling scheme and can be used for computing distance transforms for a very general class of distance functions. We also present a scalable version of our algorithm when the number of processors is available p/sup 2+/spl epsi///log log p for some p < N. In this case, our algorithm runs in O((N/sup 2//p/sup 2/)+(N/p) log log p + (log log p)/sup 2/) time. This scalable algorithm is more practical since usually the number of available processors is much less than the size of the image.

We consider the problem of scheduling tasks on multiprocessor architectures in the presence of communication delays. Given a set of dependent tasks, the scheduling problem is to allocate the tasks to processors such that the pre-specified precedence constraints among the tasks are obeyed and certain cost-measures (such as the computation time) are minimized. Several cases of the scheduling problem have

Optimal schedules for d-D grid graphs with communication delays

A single integer linear programming model for optimally scheduling partitioned regular algorithms is presented. The herein presented methodology differs from existing methods in the following capabilities: 1) Not only constraints on the number of available processors and communication capabilities are taken into account, but also local memories and constraints on the size of available memories. 2) Different types of processors can be handled. 3) The size of the optimization model (number of integer variables) is independent of the size of the tiles to be executed. Hence, 4) the number of integer variables in the optimization model is greatly reduced such that problems of relevant size can be solved in practical execution time.

This paper is concerned with the grain-size problem; how can we partition a program into tasks (concurrent modules) for parallel processing. Multiprocessor scheduling problem has long been studied by a number of researchers, and several scheduling algorithms have also been proposed. However, the effect of parallel processing strictly attributes of the given tasks such as the topology of task graph (directed acyclic graph representing precedence relations among tasks) and the balance of grain-size (computational time of each task) as well as the available number of processors. In order to improve the solution of multiprocessor scheduling problem, this paper proposes an excellent method for restructuring a given task graph. Analyzing the topology of the task graph, adequate tasks are divided one by one for increasing the parallelism of the task graph. The effect of task division is evaluated by scheduling tasks, where communication delays between distinct processors are considered. As a result, an appropriate task graph could be derived from the initially given task graph with respect to the designated number of processors. The proposed method is successfully applied to the parallel processing of robot-arm control computation on a multiprocessor system.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

In this paper, we propose a duplication-based algorithm (IDSA) for the integrated scheduling of precedence related real-time tasks with hard and soft deadlines on a set of heterogeneous multiprocessors. We employ a priority-based scheme in which the hard subtasks are always allotted a higher priority as compared to the soft subtasks. IDSA utilizes selective subtask duplication that enables some subtasks to have earlier start times, which results in an increased number of subtasks (and hence task sets) finishing before their deadlines, thereby increasing the schedulability of a real-time application. Our algorithm is also shown to be capable of scheduling a real-time application even if the available number of processors is less than the required number of processors. Based on extensive simulation results, we observe that IDSA offers better success ratio as compared to the existing algorithms in the literature when communication is a dominant factor in the system.

The discretization, parallelization and performance of an implicit, unstructured, time‐dependent Computational Fluid Dynamics code is described. A detailed description is provided of the improvements made on second‐order accurate tools for spatial interpolation and gradient calculation to discretize the Navier–Stokes equations in an unstructured framework. The main goal in the development of the discretization tools was to ensure a scalable and accurate parallel code. The performance of the discretization tools has been validated using standard bench‐mark problems for non‐uniform, non‐orthogonal grids. Parallelization of the code is done within the PETSc framework using a single‐program‐multiple‐data (SPMD) parallelization model. The resulting parallel code is shown to scale linearly within the limit of the available number of processors. Copyright © 2006 John Wiley &amp; Sons, Ltd.

Clustering and scheduling of tasks for parallel implementation is a well researched problem. Several techniques have been presented in the literature to improve performance and reduce problem execution times. In this paper we present a novel approach where clustering and scheduling of tasks can be tuned to achieve maximal speedup or efficiency. The proposed scheme is based on the relation between the costs of computation and communication of task clusters. In this paper we show how clustering can be adapted to suit different architectures and number of available processors. The proposed efficient clustering and scheduling algorithm is flexible: the clustering and scheduling can be tuned to suit bounded or unbounded number of processors and/or parallel computing environment. Comparative studies indicate superior efficiency compared to most other schemes proposed in recent years.

Recent advances in the power of parallel computers have made them attractive for solving large computational problems. Scalable parallel programs are particularly well suited to Massively Parallel Processing (MPP) machines since the number of computations can be increased to match the available number of processors. Performance tuning can be particularly difficult for these applications since it must often be performed with a smaller problem size than that targeted for eventual execution. This research develops a performance prediction methodology that addresses this problem through symbolic analysis of program source code. Algebraic manipulations can then be performed on the resulting analytical model to determine performance for scaled up applications on different hardware architectures. >

Developing SPMD applications with load balancing

In recent multicore architectures, parallel computing with vectorization emerges for mathematical calculations and image processing by exploiting Intel Advanced Vector Extensions (AVX). In parallel computing, the performance of modern processors depends on many factors, such as the amount of memory storage, the size of caches, the number of available processors, and the programming methodologies for efficiency. Besides that, a hundred vector intrinsics supported by AVX can perform various calculations on floating-point operations that can be optimized by applying AVX-256 or AVX-512 registers. In this paper, fast recursive matrix multiplication is proposed by using AVX and OpenMP combined with the software prefetching method on block-wise matrix-matrix multiplication on shared memory. The proposed version applied prefetching which can attempt to reduce memory latencies by prefetching memory pages in advance before they have been used. As a result, when the matrix multiplication applied prefetching method for the size of 8192×8192, the execution time was reduced about 22% and the performance was enhanced by about 17% than without applying prefetching while testing on Intel core i7 processor.

When mapping a parallel program onto a parallel architecture, the number of available processors is usually less than the number of tasks in the program. This gives rise to clustering and intra-processor scheduling problems. We address these two problems for distributed-memory systems where programs are explicitly-parallel in nature. We show that existing models of program representation are insufficient to capture temporal behavior of such programs. We use a new temporal communication graph model that allows identification of overlap of communication with computation and inter-task parallelism. Clustering and intra-processor scheduling heuristics, attempting to minimize program completion time, are proposed using this model. Simulation results on random task graphs show 10-25% improvement in completion time over existing heuristics. >