A control structure for parallel processing

In a shared-memory parallel processing environment there are a number of inherent hardware and software bottlenecks. One of these bottlenecks is the exclusive accesses that are needed to the shared variables within a critical section. The time cost of synchronization of processes which require access to the critical section can be minimized by using optimal scheduling methods. This paper presents a set of pre-run-time optimal scheduling algorithms for accessing the critical sections by the processes which are spawned at FORK operation and are terminated at the associated JOIN operation. >

The adaptability of parallel processes is regularly limited by the execution of critical section. Effective management of locking algorithms shows significant changes in performance. Round robin delegation algorithm can be used to assign lock to boost performance. In this algorithm when more than one process requires a same file then each process is given a specific time for which it holds the lock on that file. All other threads delegate the work to the current working thread and continue its execution. The working thread after completing its execution, checks if it still has time to do the work delegated to it. If so the working thread will take the work and after execution it will send back the results to the other threads. Proceeding with execution instead of waiting for the critical sections leads to increased parallelism and better versatility. Delegation reduces the overhead of acquiring locks, releasing locks.

Parallel processing has been proposed as a means of improving network protocol throughput. Several different strategies have been taken towards parallelizing protocols. A relatively popular approach is packet-level parallelism, where packets are distributed across processors.This paper provides an experimental performance study of packet-level parallelism on a contemporary shared-memory multiprocessor. We examine several unexplored areas in packet-level parallelism and investigate how various protocol structuring and implementation techniques can affect performance. We study TCP/IP and UDP/IP protocol stacks, implemented with a parallel version of the x-kernel running in user space on Silicon Graphics multiprocessors.Our results show that only limited packet-level parallelism can be achieved within a single connection under TCP, but that using multiple connections can improve available parallelism. We also demonstrate that packet ordering plays a key role in determining single-connection TCP performance, that careful use of locks is a necessity, and that selective exploitation of caching can improve throughput. We also describe experiments that compare parallel protocol performance on two generations of a parallel machine and show how computer architectural trends can influence performance.

We achieve very small runtime overhead: approximately a 1.2-10 times slowdown and moderate memory consumption. We demonstrate the effectiveness of Parallel Prophet in eight benchmarks in the Omp SCR and NAS Parallel benchmarks by comparing our predictions with actual parallelized code. Our simple memory model also identifies performance limitations resulting from the memory system contention. We present Parallel Prophet, which projects potential parallel speedup from an annotated serial program before actual parallelization. Programmers want to see how much speedup could be obtained prior to investing time and effort to write parallel code. With Parallel Prophet, programmers simply insert annotations that describe the parallel behavior of the serial program. Parallel Prophet then uses lightweight interval profiling and dynamic emulations to predict potential performance benefit. Parallel Prophet models many realistic features of parallel programs: unbalanced workload, multiple critical sections, nested and recursive parallelism, and specific thread schedulings and paradigms, which are hard to model in previous approaches. Furthermore, Parallel Prophet predicts speedup saturation resulting from memory and caches by onitoring cache hit ratio and bandwidth consumption in a serial program. We achieve very small runtime overhead: approximately a 1.2-10 times slowdown and moderate memory consumption. We demonstrate the effectiveness of Parallel Prophet in eight benchmarks in the OmpSCR and NAS Parallel benchmarks by comparing our predictions with actual parallelized code. Our simple memory model also identifies performance limitations resulting from memory system contention.

As the level of parallelism in manycore processors keeps increasing, providing efficient mechanisms for thread synchronization in concurrent programs is becoming a major concern. On cache-coherent shared-memory processors, synchronization efficiency is ultimately limited by the performance of the underlying cache coherence protocol. This paper studies how hardware support for message passing can improve synchronization performance. Considering the ubiquitous problem of mutual exclusion, we adapt two state-of-the-art solutions used on shared-memory processors, namely the server approach and the combining approach, to leverage the potential of hardware message passing. We propose HybComb, a novel combining algorithm that uses both message passing and shared memory features of emerging hybrid processors. We also introduce MP-Server, a straightforward adaptation of the server approach to hardware message passing. Evaluation on Tilera's TILE-Gx processor shows that MP-Server can execute contended critical sections with unprecedented throughput, as stalls related to cache coherence are removed from the critical path. HybComb can achieve comparable performance, while avoiding the need to dedicate server cores. Consequently, our queue and stack implementations, based on MP-Server and HybComb, largely outperform their most efficient pure-shared-memory counterparts.

Chapter 6 - Neural networks

Summary form only given. Database systems have long optimized for parallel execution; the research community has pursued parallel database machines since the early '80s, and several key ideas from that era underlie the design and success of commercial database engines today. Computer microarchitecture, however, has shifted drastically during the intervening decades. Until the end of the 20th century Moore's Law translated into single-processor performance gains; today's constraints in semiconductor technology cause transistor integration to increase the number of processors per chip roughly every two years. Chip multiprocessor, or multicore, platforms are commodity; harvesting scalable performance from the available raw parallelism, however, is increasingly challenging for conventional database servers running business intelligence and transaction processing workloads. A careful analysis of database performance scaling trends on future chip multiprocessors [7] demonstrates that current parallelism methods are of bounded utility as the number of processors per chip increases exponentially. Common sense is often contradicted; for instance, increasing on-chip cache size or aggressively sharing data among processors is often detrimental to performance [6]. When designing high performance database systems, there are tradeoffs between single-thread performance and scalability; as the number of hardware contexts grows, favoring scalability wins [5]. In order to transform a database storage manager from a single-threaded Atlas into a multi-threaded Lernaean Hydra which scales to infinity, substantial rethinking of fundamental constructs at all levels of the system is in order. Primitives such as the mechanism to access critical sections become crucial: spinning wastes cycles, while blocking incurs high overhead [3]. At the database processing level, converting concurrency into parallelism proves to be a challenging task, even for transactional workloads that are inherently concurrent. Typical obstacles are by-definition centralized operations, such as locking; we need to ensure consistency by decoupling transaction data access from process assignment, while adapting lessons from the first parallel database machines on the multicore platforms of the future [1][4]. Often, parallelism needs to be extracted from seemingly serial operations such as logging; extensive research in distributed systems proves to be very useful in this context [2]. At the query processing level, service-oriented architectures provide an excellent framework to exploit available parallelism. In this talk, I present lessons learned when trying to scale database workloads on chip multiprocessors. I discuss the tradeoffs and trends and outline the above research directions using examples from the StagedDB/CMP and ShoreMT projects at EPFL.

A parallel processing framework for the investigation of three dimensional subsurface geological problems is developed. The Floating Point Systems T-20 hypercube parallel computer system is used in the automatic correlation of petrophysical well logs. Parallel processing issues such as processor management, interprocessor communications management, memory management and algorithm development are discussed. Techniques are discussed to use spontaneous potential petrophysical logs to segregate subsurface strata into sand and shale segments. These segments are then used to generate sand/shale ratio logs for use in an automatic subsurface stratigraphic analysis system on a parallel machine. Phase one prepares petrophysical logs for stratigraphic analysis. The petrophysical logs are filtered and cleaned to remove any as salinity effects and improper tool responses. The spontaneous potential log is used to segregate each log into individual sand and shale beds and use these beds to create sand and shale databases which are used to create sand/shale ratio logs for later analysis. Phase two creates and solves a two dimensional matching probability matrix. The theoretical foundation is developed and several implementation issues are discussed. These issues include: (1) how to select data points, (2) how to compare data values from two petrophysical logs, (3) how to view the data, and (4) how to find the optimal set of strata observing several constraints imposed by the nature of the subsurface. Phase three developments parallel processing techniques to handle a multi-dimensional matching probability matrix. Each petrophysical log is treated as an axis of a multi-dimensional matrix with all points from one log being compared with all points from all other logs. A theoretical foundation is developed followed by implementation details. The data storage space required for this study was in excess of 10$\sp{56}$ bytes.

The application of parallel computers to reservoir simulation is studied with the aid of two recently introduced Multiple Instruction, Multiple Data (MIMD) computers. The problems posed by reservoir simulators are shown to be particularly well suited for such parallel computers* Fundamental parallel programming techniques using Fortran are presented for the Hetrogeneous Element Processor (HEP) produced by Denelcor and the iPSC Hypercube produced by Intel. These techniques are then applied to two of the most time consuming tasks in reservoir simulation: forming the matrix coefficients and producing a sparse matrix solution. The problem of parallel formation of sparse matrix coefficients is addressed for the single and multiphase cases using both black oil and compositional fluid models. Parallel sparse matrix solution is addressed by considering D4 ordered Gaussian elimination, and the multigrid, conjugate gradient, and Successive Over-Relaxation (SOR) methods. Three parallel algorithms based on SOR are developed to illustrate the possible diversity of parallel algorithms. The red-black and multicolored SOR methods which are often used on vector machines are shown to be easily adaptable to parallel MIMD machine. Matrix partitioning and a new method of iteration pipelining are also presented as parallel SOR methods. The various SOR algorithms, in parallel form, are mapped onto the HEP to illustrate the speedups currently possible on an MIMD machine. Projections are made on the future importance of these types of machines to reservoir simulation.

This application is motivated by a complex real-world scheduling problem found in the bottleneck workstation of the production line of an automotive safety glass manufacturing facility. The scheduling problem consists of scheduling jobs (glass parts) on a number of parallel batch processing machines (furnaces), assigning each job to a batch, and sequencing the batches on each machine. The two main objectives are to maximize the utilization of the parallel machines and to minimize the delay in the completion date of each job in relation to a required due date (specific for each job). Aside from the main objectives, the output batches should also produce a balanced workload on the parallel machines, balanced job due dates within each batch, and minimal capacity loss in the batches. The scheduling problem also considers a batch capacity constraint, sequence-dependent processing times, incompatible product families, additional resources, and machine capability. We propose a two-phase heuristic approach that combines exact methods with search heuristics. The first phase comprises a four-stage mixed-integer linear program for building the batches; the second phase is based on a Greedy Randomized Adaptive Search Procedure for sequencing the batches assigned to each machine. We conducted experiments on instances with up to 100 jobs built with real data from the manufacturing facility. The results are encouraging both in terms of computing time--5 min in average--and quality of the solutions--less than 10 % relative gap from the optimal solution in the first phase and less than 5 % in the second phase. Additional experiments were conducted on randomly generated instances of small, medium, and large size.

We proposed a parallel programming language based on Petri net, named PNPL. It consists of graphical and text components, where the text components are similar to programs written in Pascal and the graphical components form a Petri net which describes the flow of control in parallel processing. PNPL programs can be executed on multiprocessor machines controlled by a Petri net. This paper reports the implementation of an PNPL programming environment for workstations. The programming environment consists of an editor, a compiler and a simulator which includes a debugger. A Petri net is drawn using the graphic functions available in the editor, and the text components can be edited in multiple windows. The compiler translates PNPL programs into an intermediate C-language code and the simulator implemented using parallel processes on UNIX, and realizes the environment of the multiprocessor controlled by the Petri net. The debugger can trace the execution following the graph and text components, and can plot the movement of tokens in the Petri net graph. We can develop PNPL programs and evaluate the simulated hardware as if using the real machine. >

Some improved control statements for APL are presented, which the user can define and use without requiring either preprocessing or modification of the interpreter. A preprocessor is also given, however, in case improved execution time of programs using the control statements is desired.

Multicore machines are an extension of the single core personal computer that include several processors (cores) and a shared memory. As such, they are suitable for running parallel programs, including ParC, that use shared memory. Multicore machines replace single processor personal computers and are thus widely used. In this work we will use the term “core” instead of processor or CPU to indicate that several cores are packed in a single chip or actually in one die, functioning as a multi-processor machine. Basically, the parallelism available in multicore machines is used by the operating system to execute several unrelated processes in parallel (e.g., running two compilations and a web search on different cores). We will demonstrate that multicore machines can be used to execute parallel programs efficiently, programs that spawn many threads all communicating through shared memory. However, as an extension of a single core machine designed for personal computers, the mechanism using the shared memory is not as effective as it would have been had it been designed from scratch as a parallel machine. In fact, the shared memory of multicore machines is basically a simulation of shared memory over the single port memory module of the single core personal computer. Thus, it is important to understand how the shared memory of multicore machines works in order to determine how ParC can be implemented and used by multicore machines.KeywordsShared MemoryMain MemoryMemory ReferenceMemory ModuleCache LineThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Combinational logic synthesis is a very important phase of VLSI system design. But the logic synthesis process requires large computing times if near optimal quality of the logic network is desired. Parallel processing is fast becoming an attractive solution to reduce the computational time. Recently, researchers have started to investigate parallel algorithms for problems in logic synthesis and verification. Much of the work in parallel algorithms for CAD reported to date, however, suffers from a major limitation. The parallel algorithms proposed for the CAD applications are designed with a specific underlying parallel architecture in mind. Moreover, incompatibilities in programming environments also make it difficult to port these programs across different parallel machines. As a result, a parallel algorithm needs to be developed afresh for every target parallel architecture. The ongoing project of ProperCAD offers an attractive solution to that problem. It allows the development and implementation of a parallel algorithm on the CHARM runtime system such that it can be executed in all the parallel machines without any change in the program. In this paper, we describe a portable parallel algorithm for logic synthesis based on the Transduction method, called ProperSYN. This algorithm uses an asynchronous message driven data-flow model of computation, with no explicit synchronizing barriers separating different phases of parallel computation as used in many previously developed parallel algorithms. Our algorithm is therefore more scalable to large numbers of processors. The algorithm has been implemented and it runs on a variety of parallel machines. We present results on several benchmark circuits for shared memory MIMD machines like Sequent Symmetry and Encore Multimax, distributed memory MIMD machine like the Intel/860 hypercube and distributed processing systems like networks of SUN workstations.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Some radar image processing algorithms such as shape-from-shading are particularly compute-intensive and time consuming. If, in addition, a data set to be processed is large, then it may make sense to perform the processing of images on multiple workstations or parallel processing systems. We have implemented shape-from-shading, stereo matching, resampling, gridding and visualization of terrain models in such a manner that they execute either on parallel machines or on clusters of workstations. We were motivated by the large image data set from NASA's Magellan mission to planet Venus, but received additional inspiration from the European Union's Center for Earth Observation program (CEO) and Austria's MISSION initiative for distributed processing of remote sensing images on remote workstations, using publicly accessible algorithms. We have developed a multi-processor approach that we denote as CDIP for Concurrent and Distributed Image Processing. The speedup for image processing tasks increases nearly linearly with the number of processors, be they on a parallel machine or arranged in a cluster of distributed workstations. Our approach adds benefits for users of complex image processing algorithms: the efforts for code porting and code maintenance are reduced and the necessity for specialized parallel processing hardware is eliminated.