Code scheduling for multiple instruction stream architectures

9 - Parallel Processors

Horizon is a scalable shared-memory Multiple Instruction stream - Multiple Data stream (MIMD) computer architecture independently under study at the Supercomputing Research Center (SRC) and Tera Computer Company. It is composed of a few hundred identical scalar processors and a comparable number of memories, sparsely embedded in a three-dimensional nearest-neighbor network. Each processor has a horizontal instruction set that can issue up to three floating point operations per cycle without resorting to vector operations. Processors will each be capable of performing several hundred Million Floating Point Operations Per Second (FLOPS) in order to achieve an overall system performance target of 100 Billion (1011) FLOPS.This paper describes the architecture of the processor in the Horizon system. In the fashion of the Denelcor HEP, the processor maintains a variable number of Single Instruction stream - Single Data stream (SISD) processes, which are called instruction streams. Memory latency introduced by the large shared memory is hidden by switching context (instruction stream) each machine cycle. The processor functional units are pipelined to achieve high computational throughput rates; however, pipeline dependencies are hidden from user code. Hardware mechanisms manage the resources to guarantee anonymity and independence of instruction streams.

Multiple, high-level operators are assigned to a general operand. The operators are implemented with individual micro-processors and are "attached" dynamically to the memory location representing the "current" value of the operand. The operators then asynchronously use each new operand value as it is stored and perform their operations in parallel. The proposed architecture represents a true M I S D (Multiple Instruction Stream - Single Data Stream) computer. Its architecture can provide effective parallelism and reduced programming complexity for a large class of both numeric and non-numeric computer problems.

A parallelized point rowwise successive over-relaxation method on a multiprocessor

Algorithm for solving tridiagonal matrix problems in parallel

This paper presents a parallelization workflow in model-based development for a MIMD (Multiple Instruction streams, Multiple Data streams) processor with a vector accelerator. In the workflow, an Integer Linear Programming (ILP) formulation is proposed to deploy tasks extracted from the Simulink model. The proposed formulation can determine the number of processors that a task needs, and whether vectorization should be used to accelerate a task to reduce synchronization overhead and execution time. In the experiments, a MATLAB script was used to simulate the ILP formulation with randomly generated task graphs and evaluate its performance. Furthermore, we applied ILP task mapping to a real-world Model Predictive Control (MPC) application on DR1000C, a type of RISC-V MIMD processor with a vector accelerator. The maximum speedup of 10. 05x is obtained, which is promising for use in real-time embedded systems.

The classification of large amounts of data is a challenging problem that only a small number of classification algorithms can handle. In this paper we propose a Parallel Classification System based on an Ensemble of Mixture of Experts (PCEM). The system uses MIMD (Multiple Instruction and Multiple Data Stream) architecture, using a set of process that communicates via messages. PCEM is implemented using parallel schemes of traditional classifiers, for the mixture of experts, and using a parallel version of a Genetic Algorithm to implement a voting weighted criterion. The PCEM is a novel algorithm since it allows us to classify large amounts of data with low execution times and high performance measures, which makes it an excellent tool for in classification of large amounts of data. A series of tests were performed with well known databases that allowed us to measure how PCEM performs with many datasets and how well it does compared with other systems available.

The classification of large amounts of data is a challenging task in machine learning, which only some few classifiers can handle. In this paper we propose a Parallel Classifier based on Mixture of Experts (PCME)to handle this challenging task. The PCME is a novel algorithm since it allows us to classify large amounts of data with low execution times and performance measures (accuracy, lift, precision and recall) better than other classifiers. In previous work we saw that heterogeneous ensemble models were better but at a explores steep time cost. This work using parallel programming to overcome the time cost problem. We used the MIMD (Multiple Instruction and Multiple Data Stream) architecture, that uses a set of process that communicate via messages. PCME is implemented using parallel schemes of traditional classifiers, for the mixture of experts, and using also a parallel version of a Genetic Algorithm to implement a weighted voting criterion. We performed a series of tests with a set of databases that place the PCME as a very competitive classifier.

A PE (Processing Element) LSI for a DTW (Dynamic Time Warping) linear array processor has been designed. In designing this LSI, major effort has been focused on achieving regular data-flow among adjacent PEs maintaining pipelined operation in the array. A three data channel structure, a triple buffer structure and sophisticated control schemes make it possible for the designed LSI to carry out MIMD (Multiple Instruction and Multiple Data streams) and continuous pipelined DTW processing in sync with regular pattern input. Due to the high speed real time operation, and the versatile function of this PE-LSI, a high performance linear array processor can be constructed using a small number of PEs. The high speed operation of LSI is pursuited to achieve real time processing. Continuous speech recognition with an approximate vocabulary of 1000 words can be achieved using only 20 to 30 of these PE-LSIs.

The parallel implementation aspects of the Viterbi algorithm, which gives the maximum likelihood solution to decoding conventional codes, are studied. The interconnection of the trellis diagram of a binary convolutional code is similar to the shuffle exchange. This similarity is exploited to present a parallel version of the Viterbi algorithm. The proposed algorithm avoids tasks such as appending and exchanging the survivor lists that are considered pragmatic issues to be accommodated in implementing the Viterbi algorithm. This is done by introducing an additional back-tracking procedure which is almost identical to the forward path where branch and path metrics are computed. The parallel Viterbi algorithm is implemented on the connection machine which is a SIMD (single instruction stream and multiple data stream) parallel machine. >

The throughput of pipelined processors suffers from delays associated with instruction dependencies and memory latencies. Multithreaded architectures attempt to hide such delays by sharing the processor with multiple instruction streams. In this paper we develop a comprehensive analytic framework to quantitatively evaluate the performance of a wide spectrum of mulithreaded machines, ranging from those that are capable of switching threads every cycle to those that switch threads only on long delays. The models are validated against previously published simulation and modeling results, and then used to assess the performance potential of multithreading given current processor technology. >

The Multiple Associative Computing (MASC) parallelmodel is a generalization model of an AssociativeComputing (ASC) parallel model designed to supportmultiple ASC data parallel threads by using controlparallelism.

Parallel processing with minicomputers

EPIC architectures rely heavily on state-of-the-art compiler technology to deliver optimal performance while keeping hardware design simple. It is generally believed that an optimizing compiler has an enormous scheduling window to exploit instruction-level parallelism (ILP) since the compiler orchestrates the entire program. Many state-of-the-art compilers typically confine optimizations to loop boundaries (e.g. software pipelining, trace scheduling, and loop unrolling) and function boundaries (e.g. loop peeling, loop exchanges, invariant hoisting, and global optimizations). Although techniques such as function inlining and interprocedural optimizations can alleviate these constraints to a limited extent, loop and function boundaries are often the real scopes of the compiler scheduler. Several previous ILP studies have explored the limits of parallelism on dynamic superscalar machines; however, those results are not applicable to EPIC architectures since they rely on dynamic scheduling, not static code scheduling by the compiler, to reorder instructions. In this paper, we evaluate the limits in ILP obtained through compiler scheduling alone. We quantify these limits as more restrictive scheduling constraints are imposed-starting from inter-procedural code scheduling, to intra-procedural and finally to loop-confined code scheduling.

The cloud, rack-scale computing, and multi-core are the basis for today's computing platforms. Their intrinsic parallelism is a challenge for programmers, specially in areas lacking the necessary economies of scale in application/ code reuse because of the small number of potential users and frequently changing code and data. In this paper, based on an on-going collaboration with several projects in astrophysics, we present Pydron, a system to parallelize and execute sequential Python code on a cloud, cluster, or multi-core infrastructure. While focused on scientific applications, the solution we propose is general and provides a competitive alternative to moving the development effort to application specific platforms. Pydron uses semi-automatic parallelization and can parallelize with an API of only two decorators. Pydron also supports the scheduling and run-time management of the parallel code, regardless of the target platform. First experiences with real astrophysics data pipelines indicate Pydron significantly simplifies development without sacrificing the performance gains of parallelism at the machine or cluster level.