The MULTIPLUS/MULPLIX parallel processing environment

Sharing memory spaces between parallel processes is a common practice in a real-time simulation environment. In an environment where parallel processes can exist on the same machine or on different machines, it is a challenge to develop and maintain reusable software that interfaces with shared memory spaces. The two main problems involved in dealing with shared memory spaces are: 1) providing platform independent access to the memory space, and 2) providing to all processes a consistent view of the structure and content of the memory space. Object-oriented techniques were used to create a software architecture designed to manage shared memory spaces. Object-oriented design patterns were used to present client code with a conceptual model of shared memory spaces while concealing the underlying implementation details. The resulting architecture is largely computing platform independent, with platform specific code being isolated to a few selected classes. The design was implemented in C++ for the NASA Langley Standard Real-Time Simulation (LaSRS++) Application Framework. This paper provides an overview of the design and implementation of the LaSRS++ shared memory management software.

A generalized architecture is presented for reconfigurable m-ary tree structures, where m is any integer >1. The approach is based on a generalized multistage interconnection network (MIN), which is a generalization of the augmented shuffle-exchange MIN introduce by the authors previously (1990) for obtaining reconfigurable binary tree structures. The generalized architecture with m/sup k/ processing elements or nodes (where k is any integer >1) is implemented with a k-stage MIN. A single control code issued to the MIN establishes a distinct m-ary tree configuration among the nodes. The favorable features of the architecture include fast reconfiguration, simplified hardware in the nodes and the MIN, and simple routing control. The reconfigurability of the architecture is proved, and the results of the analysis are utilized to provide a procedure to synthesize the m-ary tree configuration that is generated for any given control code. Considerations for implementing the switching elements of the MIN are discussed.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

CHAPTER 4 - Processes, Shared Memory, and Simple Parallel Programs

GPUs provide high-bandwidth/low-latency on-chip shared memory and L1 cache to efficiently service a large number of concurrent memory requests. Specifically, concurrent memory requests accessing contiguous memory space are coalesced into warp-wide accesses. To support such large accesses to L1 cache with low latency, the size of L1 cache line is no smaller than that of warp-wide accesses. However, such L1 cache architecture cannot always be efficiently utilized when applications generate many memory requests with irregular access patterns especially due to branch and memory divergences that make requests uncoalesced and small. Furthermore, unlike L1 cache, the shared memory of GPUs is not often used in many applications, which essentially depends on programmers. In this article, we propose Elastic-Cache, which can efficiently support both fine- and coarse-grained L1 cache line management for applications with both regular and irregular memory access patterns to improve the L1 cache efficiency. Specifically, it can store 32- or 64-byte words in non-contiguous memory space to a single 128-byte cache line. Furthermore, it neither requires an extra memory structure nor reduces the capacity of L1 cache for tag storage, since it stores auxiliary tags for fine-grained L1 cache line managements in the shared memory space that is not fully used in many applications. To improve the bandwidth utilization of L1 cache with Elastic-Cache for fine-grained accesses, we further propose Elastic-Plus to issue 32-byte memory requests in parallel, which can reduce the processing latency of memory instructions and improve the throughput of GPUs. Our experiment result shows that Elastic-Cache improves the geometric-mean performance of applications with irregular memory access patterns by 104% without degrading the performance of applications with regular memory access patterns. Elastic-Plus outperforms Elastic-Cache and improves the performance of applications with irregular memory access patterns by 131%.

Parallel programming has been widely used in many scientific and technical areas to solve large problems. While general-purpose processors have rich infrastructure to support parallel programming on shared memory, such as coherent caches and synchronization libraries, parallel programming infrastructure for FPGAs is limited. Thus, development of FPGA-based parallel algorithms remains difficult. In this work, we seek to simplify parallel programming on FPGAs. We provide a set of easy-to-use declarative primitives to maintain coherency and consistency of accesses to shared memory resources. We propose a shared-memory service that automatically manages coherent caches on multiple FPGAs. Experimental results of a 2-dimensional heat transfer equation show that the shared memory service with our distributed coherent caches outperforms a centralized cache by 2.6x. To handle synchronization, we provide new lock and barrier primitives that leverage native FPGA communication capabilities and outperform traditional through-memory primitives by 1.8x.

In an often-cited paper, A. Karp [1] commented on the “sorry state” of parallel programming, lamenting the difficulties in programming and debugging parallel programs on existing programming models. Indeed, a variety of programming models have been proposed with not entirely satisfactory results. In this note, we focus on the world of massively parallel shared memory machines and examine programming models for these machines. A shared memory parallel computer is one in which the processors communicate by reading and writing data words to a shared memory. We consider shared memory parallel computers with the following characteristics. Each processor has some fast, relatively small, private memory (e.g. registers, cache). There is a large shared memory composed of a series of memory banks that are accessible by all processors via an interconnection network. Each processor has one such memory bank which is local to it, i.e. it can access the bank without going through the interconnection network. The access time to a memory bank for a processor depends in part on the lengths of the links traversed and the number of hops in the path from the processor to the bank. Special coprocessors take care of interprocessor communication: both programmers and processors are relieved of the burden of routing the parallel requests to shared memory, including handling intermediate hops of requests traveling to their destinations. In order to overcome the high latency of accessing global memory, the parallel computer supports the pipelining of global memory accesses through the network. This implies that the interconnection network has sufficient bandwidth to handle the multiple requests per processor.

Background: Multistage interconnection networks are being used in computer and communications. Multiprocessor architectures for parallel computing exercise these interconnection networks for connecting various processing elements and transfer data between sub-systems of a digital system. The vast diversity of the field poses an obstacle to realize different kinds of interconnection networks and their relationship. Methods: This paper consists of an extensive survey of multistage interconnection networks. Results: A broad classification of multistage interconnection networks based on network functionality, reliability and fault tolerance is presented in order to emphasize the important principles which differentiate the network architectures. For each class of network, significant results are given and the basic design principles are explained. Conclusion: The various multistage interconnection networks design provide high performance, availability, throughput, lower latency, less power consumption along with improved fault-tolerance and reliability. However, there is a rising demand for new fault-tolerant and reliable multistage interconnection networks.

Shared virtual memory, a technique for supporting a shared address space in software on parallel systems, has undergone a decade of research, with significant maturing of protocols and communication layers having now been achieved. We provide a survey of the key developments in this research, placing the multitrack flow of ideas and results obtained so far in a comprehensive new framework. Four major research tracks are covered: relaxed consistency models; protocol laziness; architectural support; and application-driven research. Several related avenues are also discussed, such as fine grained software coherence, software protocols across multiprocessor nodes, and performance scalability. We summarize comparative performance results from the literature, discuss their limitations, and identify lessons learned so far, key outstanding questions, and important directions for future research in this area.

Current trends in computing require dedicated chips to speed up computing performances. These dedicated chips or ASICs are designed for specific applications, however, the drawback in this approach is that IC (Integrated Circuits) functionality cannot be changed. In areas where H/W programmability and reconfigurability is required, FPGA is the solution to achieve this. System on Chip (SOC) FPGAs (Field Programmable Gate Arrays) include processor hard blocks connected to fabric through various interconnects. This architecture provides better performance and the ability to run Operating Systems (OS) like Linux <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">®</sup> that has direct access to programmable FPGA logic. This paper explains how to split the functions of an application code between hard block processor and FPGA fabric logic. OPUS is an open-source audio codec widely used in voice-over-IP (VoiP) applications. The codec execution can be accelerated by moving some of its functions to FPGA logic. One such function of OPUS code is implemented in FPGA logic using High-Level Synthesis (HLS). Shared DDR memory space is used to transfer the data between the processor and FPGA. In this paper, we summarize the reduction in execution time and processor load. The acceleration of OPUS codec allows processing of a greater number of audio channels in the SoC device compared to running the codec entirely on the processor.

Performance analysis of SPMD algorithms on a network of workstations with virtual shared memory

Shared virtual memory (SVM) systems, because of their software implementation, enable shared-memory programming at a low design and maintenance cost. Nevertheless, as hardware implementations become faster, their performance is still far from that achieved by distributed shared memory (DSM) systems. Nowadays, SVM systems use relaxed memory consistency models and multiple writer protocols as techniques to reduce latencies and false sharing, respectively. However, these techniques induce additional overhead that decreases system performance. We performed a study of workload behavior aimed at improving the design of SVM protocols. The work focused on the identification of the type of shared data patterns that can appear in the accesses to protected sections using semaphores. Most coherence actions in SVM systems are performed as a consequence of the write operations executed in critical sections, so we pay special attention to the write operations performed when multiple writers are allowed. As these write operations may present spatial locality, we also study the write patterns on shared pages with similar behaviour. Different software filters are applied in the instrumented parallel workloads selected to capture and classify the most common sharing patterns. This enables the recognition of those patterns in which coherence overhead can be reduced by modifying the coherence actions performed by the protocol. Despite the fact that the performance evaluation of new coherence solutions is not our main goal, the ideas presented to improve the behaviour of SVM systems can be implemented at a reasonable hardware/software cost.

Heterogeneous parallel computing can be considered a first step towards general purpose parallel machines. Different communication libraries are available for heterogeneous parallel computing. Most of them are based on the message passing model. The virtual shared memory communication model is now obtaining an increasing attention. Our point of view is that the availability of both models, following well defined criteria, provides an important contribution to the development of efficient heterogeneous parallel applications. We present HeterEL, a server for heterogeneous parallel computing using shared virtual memory and message passing communications. HeterEL is a building block for heterogeneous parallel programming, which interfaces message passing and shared virtual memory libraries in a transparent way. >

With the coming of the Internet of things(IoT) era and the development of semiconductor equipment, multicore processors have begun to be widely used in IoT devices to meet their requirements for powerful processing capabilities. Unlike desktop or server operating systems such as Linux, current embedded operating systems often do not support multi-core processors well. Tasks on different cores often require information exchange, known as inter-core communication, which significantly impacts the processing performance of multi-core operation systems. In this paper, we proposed an inter-core communication method based on signal transmission and shared memory, which is flexible and various types of data can be transferred efficiently. We have implemented and experimented with it on our own microkernel operating system named Mginkgo. The experimental results show that the average time to trigger an inter-core interrupt is about 0.093 microseconds. The average inter-core interrupt processing time is about 3.986 microseconds. And the communication time of the system for multi-core Inter-Process Communication(IPC) is about 18us, which is the same as that of single-core IPC. The inter-core communication method proposed in this paper achieves very low latency with almost no performance consumption and maintains the high performance of the whole system.

M-PVM is an implementation of PVM designed to work efficiently in parallel architectures supporting multithreading and the shared memory model. In particular, the current M-PVM implementation is running within MULPLIX, a Unix-like operating system designed to efficiently support parallel applications running on MUL TIPLUS, a distributed shared memory parallel computer under development at the Federal University of Rio de Janeiro. M-PVM is a parallel programming library built on top of the MULPLIX parallel programming primitives. Within M-PVM, PVM tasks are mapped onto MULPLIX threads. Two approaches have been adopted for the implementation ofthe message passing primitives. In the first approach a single copy of the message in memory is shared by all destination M-PVM tasks. The second approach replicates the message for every destination task but requires less synchronization. MPVM is not totally compatible with standard PVM but offers an environment which simplifies the portability of PVM applications to multithreaded shared memory platforms, in most cases, with performance improvements. Experimental results comparing the performance of M-PVM and PVM applications running on a 4-processor Sparcstation 20 under Solaris 2.5 are presented. These results show that M-PVM can produce speed-up gains in the range of 5 to 10% in relation to PVM.

Computer architectures are expected to change to support Exascale computing in the near future. As energy and cooling constraints limit increases in microprocessor clock speeds and number of cores, computer companies are turning to parallel programming. Nowadays, parallel programming is achieved by increasing the number of processing elements in processor cores, increasing the number of processor cores itself and complicated parallel programming where programmer has the responsibility of allocating memory and synchronizing the communication between the processing elements as well as processor cores. It becomes increasingly difficult and expensive to design and produce shared memory machines with ever increasing number of processors. Increase in the number of processors is a major disadvantage when it comes to energy consumption. In this work, we present a new architecture for processor design based on pairwise balanced combinatorial interconnection of processing and memory elements. The proposed processor uses two operand instructions, so that the set of executable machine instructions is partitioned by these pairs. This kind of partition allows parallel processing of data-independent instructions. Since this partition is done at the compile time, the architecture extracts the instruction level parallelism without run-time overheads. We analyze and confirm the performance improvements through simulations. The suggested combinatorial arrangement gives set of architectures with various degrees of performance enhancement.