A combinatorial distributed architecture for Exascale computing

For better utilization of computing resources, it is important to consider parallel programming environments in which the number of available processors varies at runtime. In this paper, we discuss runtime support for data parallel programming in such an adaptive environment. Executing data parallel-programs in an adaptive environment requires redistributing data when the number of processors changes, and also requires determining new loop bounds and communication patterns for the new set of processors. We have developed a runtime library to provide this support. We also present performance results for a multiblock Navier-Stokes solver run on a network of workstations using PVM for message passing. Our experiments show that if the number of processors is not varied frequently, the cost of data redistribution is not significant compared to the time required for the actual computations. >

We propose a novel instruction compression scheme based on fine-grained programmable dictionaries. In its core is a compile-time region-based control flow analysis to selectively update the dictionary contents at runtime, minimizing the update overheads, while maximizing the beneficial use of the dictionary slots. Unlike in the previous work, our approach selects regions of instructions to compress at compile time and changes dictionary contents in a fine-grained manner at runtime with the primary goal of reducing the energy footprint of the processor instruction stream. The proposed instruction compression scheme is evaluated using RISC-V as an example instruction set architecture. The energy savings are compared to an instruction scratch pad and a filter cache as the next level storage. The method reduces instruction stream energy consumption up to 21 % and 5.5 % on average when compared to the RISC-V C extension with a 1% runtime overhead and a negligible hardware overhead. The previous state-of-the-art programmable dictionary compression method provides a slightly better compression ratio, but induces about 30 % runtime overhead.

Due to the need to support concurrent executions of versatile applications, the system complexity, in terms of the number of cores, is drastically increased from tens to hundreds or thousands of cores. These complex systems usually contain heterogeneous cores or processing elements such as different processor cores, memories, and several Silicon Intellectual Properties (SIPs). Network-on-chip (NoC) was proposed to provide scalability and higher throughput for these heterogeneous multi-core systems. However, general designs of NoC infrastructures for multi-core systems usually lack the flexibility to support different processing requirements such as performance, power, reliability, and response time. It is helpful if designers can provide a reconfigurable NoC design so that these requirements can be supported more easily. In this work, we take an existing reconfigurable NoC for example and discuss related hardware and software issues. Some issues such as the reconfiguration time overhead must be considered in the design of a reconfigurable NoC such that it can be used for heterogeneous multi-core systems.

A fast Fourier transform (FFT) algorithm is mapped onto a suggested processing element topology in order to demonstrate the utility of the systolic data flow machine (SDFM) approach. The SDFM is based on the partitioning of dataflow programs (graphs) into subgraphs that are small enough that they can be loaded into programmable systolic arrays, called processing elements. Mapability and performance criteria are suggested, such as the number of allocated primitive processors (in a systolic array) and the number of primitive processors and systolic arrays that are active at any one time. Conclusions about system attributes, such as the ratio of local to global communication, granularity, instruction execution and communication time, parallelism, processor utilization, are also presented. >

Because Richards' equation for describing unsaturated groundwater flow is highly nonlinear, it is challenging to test the accuracy and efficiency of parallel three-dimensional finite element groundwater programs used to model such activities as remediation of military sites and computing flows and pressures in levees broken by Hurricane Katrina. To aid in this testing, the author has derived steady-state and transient analytical solutions for unsaturated flow. With these solutions, both accuracy of the numerical solutions and the performance of the parallel versions of the computer programs may be easily tested. The other significant feature of these analytical solutions is that the amount of nonlinearity of the equation may be easily adjusted by a parameter a. This paper first presents the equations for the analytical solutions of steady-state and transient flow into a rectangular block of initially dry soil. It then uses the program FEMWATER as a benchmark to compute the numerical solutions corresponding to these analytical solutions. Accuracy results for various sizes of the mesh and numbers of processing elements (PEs) on the Cray XT3 are computed for different values of a. The number of nonlinear iterations for the steady-state problem is also reported. The result of the accuracy assessment is that as a is increased, the errors also increase significantly with even 512 PEs not being enough compute power to adequately solve the problem. Overall parallel performance and performance of the message passing interface calls for updating the ghost nodes and are also tested with results given. The result of this study is that scaling of a 1,000,000-element problem was good up to 256 PEs (64 PEs is the typical sweet spot for FEMWATER), and the performance of the ghost-node updating routine met or exceeded expectations

The load distribution among processing elements (PE) in the parallel computer system is one of the major factors affecting the performance of the entire system. This paper discusses the dynamic distribution scheme which distributes the load at the execution of the task. The major factor, which affects the performance in the dynamic load distribution scheme, is the information concerning the loading situation of each PE and the data transfer for distributing the load. This paper proposes a dynamic load distribution scheme, which reduces these overheads considerably by utilizing the network. The idea can be applied to both the circuit‐switching network and the packet‐switching network. Especially, the scheme based on the packet‐switching network is one of the load distribution schemes suited to the instruction‐level data‐driven computer, using the packet as the major communication means among PE's. Based on the proposal of the scheme, the performance of the system is evaluated for a typical configuration example for the parallel processing program. It is verified as a result that the proposed system operates very closely to the ideal dynamic load distribution system.

In the ray tracing method, each pixel can be computed independently. Therefore, when the ray tracing algorithm is implemented on a parallel computer, a group of pixels is usually assigned to each element processor. The element processor must have all the environment data and object data for the rendering. When there are too many objects for local memory, a distorted image will be generated. In solving this problem, to date, the necessary data had to be transmitted. However, when the number of element processors is increased, efficient rendering cannot be expected because of communication overheads. This paper proposes a hybrid parallel ray tracing algorithm that combines pixel-parallel and object-parallel ray tracing. As this algorithm inherits the advantages of pixel-parallel ray tracing, high efficiency can be achieved even on a parallel computer using a large number of element processors.

Conceptually, an SIMD machine has the capability to overlap operation of the control unit (CU) with the operation of the processing elements (PEs). Computation of a single program is most efficient when the work load of the CU and the PEs is balanced. Load balancing between the CU and PEs is accomplished by migrating certain computations (e.g., PE-common array index calculations, loop index variable manipulation) from the PEs to the CU and vice versa. The goal of this research is to develop some of the techniques needed for the automatic specification of CU/PE overlap at compile time. As a proof of concept, the ELP compiler has been modified to support experimentation with CU/PE overlap. >

Computational grids have recently attracted considerable attention as cost‐effective platforms for parallel processing. Generally, a parallel program carried out on such a platform can be characterized as a directed acyclic graph (DAG). To take full advantage of available resources in grids and achieve high computational performance, an effective DAG scheduling algorithm is indispensable. This study presents a novel constructive algorithm based on the Fast Ant System (FANT). The proposed algorithm, namely, the Fast Ant System for Task Matching and Scheduling (FANT‐TMS) algorithm, determines the scheduling order of all tasks in a parallel program and, then, allocates these tasks to appropriate processing elements in a computational grid, thereby minimizing total completion time of the parallel program. Performance of the FANTTMS algorithm is demonstrated by comparison with a genetic algorithm (GA)‐based scheduling technique in terms of overall schedule length of randomly generated DAGs. Experimental results demonstrate that the proposed algorithm outperforms the conventional GA‐based approach. Additionally, a novel local search strategy is devised that further enhances solution quality obtained with the FANT‐TMS algorithm.

A cooling control method based on a model predictive control (MPC) for a modular datacenter utilizing the fresh-air is proposed. The proposed method reduces the total energy consumption of information technology (IT) equipment and cooling facilities in the data center, while considering a relationship between energy-savings and the temperature information of IT equipment. This method based on MPC controls the central processing unit (CPU) temperature in servers by facility fans for cooling. To design the proposed method, it is developed a prediction model that represents the CPU temperature by the revolution speed of facility fans, the fresh-air temperature, utilization of servers, and other factors. Furthermore, the proposed control method is applied to the actual modular data center. The energy consumption of the proposed method is compared with that of a traditional method, which has controlled the temperature difference between the inlet and outlet of the server racks based on proportional integral (PI) control. Actual comparison experiments with traditional method are provided to validate effectiveness of the proposed method. The results show that the proposed method realizes energy-savings of more than 20% compared to the traditional control method in the actual modular datacenter.

An oil/water, 3-D fully implicit Reservoir Simulator has been implemented on the ICL Distributed Array Processor (DAP), by a joint BNOC/ICL project team. The DAP is a single instruction, multiple data stream machine with 4096 processors in parallel. The major computational areas of code have been implemented on the DAP, leaving I/O and well calculations as serial code on the host 2976 machine. Matrix assembly is inherently parallel and can be implemented easily and efficiently on the DAP with compact code. Table look-up, though not obviously parallel, has been coded using a very efficient parallel algorithm. The Linear Solver, of Line Gauss-Seidel type, has been chosen for its efficiency on the DAP, with odd and even reduction used for solution of the resulting tri-diagonal equations. Further research will be undertaken to increase the robustness of this solver. The initial implementation has a 1-1 mapping of active grid blocks to processors, thus allowing up to 4096 active grid blocks. It is planned to extend to much larger models, and to 3 phase.

Edge computing of reliable multimodal (2D RGB/3D RGB-Depth) data has a wide range of applications. However, many of currently reported visual processors cannot flexibly handle multimodal data, e.g., the visual streams of RGB-Depth data. The key challenge exists that these prior visual processors do not come with efficient and unified instruction set architecture (ISA) for both conventional and intelligent cognition on the 2D/3D multimodal sensory data. To fill such a gap, this paper proposes a programmable intelligent visual vector processor compatible with multimodal 2D/3D visual data processing ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1920\times 1080$ </tex-math></inline-formula> -pixel resolution). The processor consists of a reconfigurable processing element (PE) array, a memory access network flexibly configurable to be fine- or coarse-grained, and a high throughput I/O interface. The vectorial PE array with neighbor PE access increases the data reuse rate and parallel computation efficiency, and can implement both convolutional neural networks (CNNs) and conventional image processing algorithms. The proposed ISA is customized and optimally tailored targeting 2D/3D image processing from RGB/Time-of-Flight(ToF) raw data to intelligent inference results. The chip is fabricated in a 55-nm CMOS process. The experimental results showed that the area efficiency, peak performance, and peak throughput of our chip attained as high as 14.41GOPS/mm2, 409.6GOPS, and 9.6Gbps at 200MHz, respectively. The measured processing speeds of this chip on ToF depth reconstruction is 87fps ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$480\times 270$ </tex-math></inline-formula> ) or 31 fps( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1920\times 1080$ </tex-math></inline-formula> ),on 3D object classification is 219fps ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$256\times 256$ </tex-math></inline-formula> ), and on CNN-based 2D object tracking is 36fps ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$256\times 256$ </tex-math></inline-formula> ).

Animals live in an ever-changing environment to which they must continuously adapt. Adaptation in the nervous system occurs at every level, from ion channels and synapses to single neurons and whole networks. It operates in many different forms and on many time scales. Retinal adaptation, for example, permits us to adjust within minutes to changes of over eight orders of magnitude of brightness, from the dark of a moonless night to high noon. High-level memory—the storage and recognition of a person's face, for example—can also be seen as a specialized form of adaptation (see Squire, 1987). The ubiquity of adaptation in the nervous system is a radical but often underappreciated difference between brains and computers. With few exceptions, all modern computers are patterned according to the architecture laid out by von Neumann (1956). Here the adaptive elements—the random access memory (RAM)—are both physically and conceptually distinct from the processing elements, the central processing unit (CPU). Even proposals to incorporate massive amounts of so-called intelligent RAM (IRAM) directly onto any future processor chip fall well short of the degree of intermixing present in nervous systems (Kozyrakis et al., 1997). It is only within the last few years that a few pioneers have begun to demonstrate the advantages of incorporating adaptive elements at all stages of the computation into electronic circuits (Mead, 1990; Koch and Mathur, 1996; Diorio et al.,1996). For over a century (Tanzi, 1893; Ramón y Cajal, 1909, 1991), the leading hypothesis among both theoreticians and experimentalists has been that synoptic plasticity underlies most long-term behavioral plasticity. It has nevertheless been extremely difficult to establish a direct link between behavioral plasticity and its biophysical substrate, in part because most biophysical research is conducted with in vitro preparations in which a slice of the brain is removed from the organism, while behavior is best studied in the intact animal. In mammalian systems the problem is particularly acute, but combined pharmacological, behavioral, and genetic approaches are yielding promising if as yet incomplete results (Saucier and Cain, 1995; Cain, 1997; Davis, Butcher, and Morris, 1992; Tonegawa, 1995; McHugh et al., 1996; Rogan, Stäubli, LeDoux, 1997).

The number of processors is increasing, but the usefulness of parallel computation is not better leveraged due to the inflexibility of design and implementation for multiprocessor embedded system applications. A higher level abstraction (i.e., a parallel programming framework) can ease the programmers to define parallelism for tasks in an application but designers still face the complexity of mapping high-level requirements to the syntax and semantics of a parallel programming interface. Nevertheless, embedded system applications execute various periodic tasks that are carried out repeatedly within a certain time interval and these tasks may be able to run in parallel to utilize the system efficiently. Therefore, in this paper, we propose a parallel loop-based task construct approach to automate the design process of embedded system applications from AADL models for a parallel programming framework. To illustrate the applicability of our mechanism, we use a well-known parallel programming interface called OpenMP to demonstrate the automation process in mapping tasks over multiple processor cores. To ensure meeting high-level requirements of embedded system applications, we analyze the existing OpenMP scheduling mechanisms and propose a layer of adaptation. We show that our proposed adaptation layer facilitates a tighter execution time bound for time-sensitive tasks or a better throughput for tasks that require higher quality of service. Thus, the proposed design automation framework is applicable for a variety of applications with different quality of service (QoS) requirements preserved at the lower level.

State-of-the-art architecture description languages have been successfully used to model application-specific programmable architectures limited to particular control schemes. In this paper, we introduce a language and methodology that provide a framework for constructing and simulating a wider range of architectures. The framework exploits the fact that designers are often only concerned with data paths, not the instruction set and control. In the framework, each processing element is described in a structural language that only requires the specification of the data path and constraints on how it can be used. From such a description, the supported operations of the processing element are automatically extracted and a controller is generated. Various architectures are then realized by composing the processing elements. Furthermore, hardware descriptions and bit-true cycle-accurate simulators are automatically generated. Results show that our simulators are up to an order of magnitude faster than other reported simulators of this type and two orders of magnitude faster than equivalent Verilog simulations.