Intra- and inter-processor hybrid performance modeling for MPSoC architectures

Solution of dense linear systems on an optimal systolic architecture

Chapter 1 describes the early work of von Neumann (1951) on cellular automata. This work was not accompanied by reductions to practice in hardware as it was impractical at that time to build machines having the millions of devices required. Only with the development of high-density integrated circuitry in the 1970s has this feat now been accomplished (Chapter 11). Therefore, during the 1950s cellular automata were emulated using the general-purpose computers which were available at that time. The work of Moore (1966) and Kirsch (1957) at the National Bureau of Standards, of Ulam (1962) at the Atomic Energy Commission, and Unger (1959) at Bell Telephone Laboratories are outstanding examples of this work. All of these workers simplified von Neumann’s 29-state processing element and concentrated their efforts on studying arrays of 2-state (binary) processing elements. In the 1960s a new trend began with the construction of the first cellular logic machines by one of the authors (Preston, 1961). These and other special-purpose machines emulated the cellular automaton by using a single high-speed processing element to operate sequentially on an array of binary data. With the introduction of the diff3 in the 1970s (Graham, and Norgren, 1980) cellular logic machines were manufactured having several processing elements. Then Sternberg (1981) introduced a pipelined cellular logic machine, called the Cytocomputer, having approximately 100 processing elements. The Cytocomputer was also the first cellular logic machine to include numerical (multi-state) processing elements in addition to binary processing elements, thus making the transition from high-speed special-purpose machines limited to bilevel data to a system which could manipulate multi-state data. This chapter concentrates on the evolution and architecture of the cellular logic machines which have been built over the past two decades, all of which handle bilevel data arrays. Machines of this kind are currently in wide use both for commercial and research purposes in image processing. Despite their limitation to bilevel data, they are also useful in graylevel image processing due to their ability to convert graylevel images to binary images by multiple thresholding and, after performing logical operations at these thresholds, to convert results to graylevel output by arithmetic summation (Chapter 2).

We have fabricated a Chip Multiprocessor prototype code-named Merlot to proof our novel speculative multithreading architecture. On Merlot, multiple threads provide wider issue window beyond ordinal instruction level parallel (ILP) processors like superscalar or VLIW. With the architecture, we estimate 3.0 times speedup against single processing elements (PE) on speech recognition code and IDCT code with four PEs. Merlot integrates on-chip devices, PCI interface, and SDRAM interfaces. We have encountered design issues of chip multiprocessor and SoC design. We have successfully run parallelized mpeg3 decoder on the first silicon with several software workarounds, thanks to functional verification environment including system modeling on RTL. However, bugs found in later stage of design have required larger manpower or delay of project. In this paper, we also discuss the methodology to improve functional verification coverage, and expect the solution in formal approaches.

Current application domains, like mobile robotics, or internet of things require high computational power associated with low energy consumption. Therefore, MPSoCs are widely used as an adequate platform for high performance embedded computation. Recently, the emergence of RISC-V instruction set architecture drives SoC designers to adopt it in the design of MPSoCs as a cost-free, modular processor and suitable to be implemented in different hardware platforms. Furthermore, these characteristics make the RISC-V an interesting candidate for an FPGA soft-core processor. In this paper, we present a modular hybrid memory system for a lightweight RISC-V based MPSoC architecture. The implementation of the hybrid memory consists of a global scratchpad on-chip shared memory for both instruction and data for the purpose of communication and synchronization between the processing elements. In addition to a tightly coupled memory associated with each processing element for low latency memory access for private computation. Moreover, the complete MPSoC architecture is scalable and configurable, in terms of the number of PEs, shared/private memory sizes and the number of memory mapped peripherals. A benchmarking environment is developed to evaluate the performance of the proposed hybrid memory system in terms of memory access latency and memory bandwidth and their impact on the computation time. The complete MPSoC architecture is implemented and tested on a Xilinx Zynq 7000 FPGA device.

In this thesis we describe a method of mapping one-dimensional and multidimensional filter algorithms onto systolic architectures using the z-domain approach. In this approach the filter algorithm is first transformed into its corresponding z-domain equivalent and recursive expressions similar to single assignment codes are derived using Horner's rule or other polynomial evaluation techniques. By obtaining different recursive expressions, different systolic structures can be derived. The characteristics of these structures can easily be deduced from the recursive expressions. The multidimensional filters derived are modular and hierarchical, i.e., the three-dimensional structures are obtained from the two-dimensional ones which are in turn obtained from one-dimensional structures.
\n
\nIn considering the design of any array processor, it is important to consider the design of the processing elements involved. The most important and demanding operation in these elements is the multiplication. Four different multipliers are designed in which the number of operations required to produce the desired result is reduced. The reduced number of operations along with the advantages of very-large-scale integration technology in terms of increased device density and faster switching make these multipliers potential candidates in high-speed signal processing applications. The first multiplier is an area-efficient multiplier that uses approximately 50% of the area of a full parallel multiplier. In this multiplier only the units yielding the most significant part of the product are used. In addition, a correction unit is incorporated to minimize the error resulting from circumventing the use of units yielding the least significant part of the product. The second multiplier is based on the modified octal Booth algorithm in which four-bit segments of the multiplier are scanned and corresponding operations effected on the multiplicand. The third multiplier is a diminished-1 multiplier that finds application in the Fermat number-theoretic transform. In this multiplier the use of a translator is circumvented and a novel technique for translation is incorporated in the multiplier structure. The fourth multiplier is one that performs an inner-product operation without the use of an accumulator thereby resulting in increased speed and reduced area.
\n
\nFinally we discuss the VLSI implementations of three of the multipliers mentioned above, a second-order digital filter, and a single processing element that can be used as a basic unit in designing one-dimensional and multidimensional digital filters. Some associated problems in digital-filter structure. viz., the quantization and overflow limit-cycle oscillations; have been taken into consideration and ways have been suggested for their elimination.

Unlike standard Convolutional Neural Networks (CNNs) with fully-connected layers, Fully Convolutional Neural Networks (FCN) are prevalent in computer vision applications such as object detection, semantic/image segmentation, and the most popular generative tasks based on Generative Adversarial Networks (GAN). In an FCN, traditional convolutional layers and deconvolutional layers contribute to the majority of the computation complexity. However, prior deep learning accelerator designs mostly focus on CNN optimization. They either use independent compute-resources to handle deconvolution or convert deconvolutional layers (Deconv) into general convolution operations, which arouses considerable overhead. To address this problem, we propose a unified fully convolutional accelerator aiming to handle both the deconvolutional and convolutional layers with a single processing element (PE) array. We re-optimize the conventional CNN accelerator architecture of regular 2D processing elements array, to enable it more efficiently support the data flow of deconvolutional layer inference. By exploiting the locality in deconvolutional filters, this architecture reduces the consumption of on-chip memory communication from 24.79 GB to 6.56 GB and improves the power efficiency significantly. Compared to prior baseline deconvolution acceleration scheme, the proposed accelerator achieves 1.3X -- 44.9X speedup and reduces the energy consumption by 14.6%-97.6% on a set of representative benchmark applications. Meanwhile, it keeps similar CNN inference performance to that of an optimized CNN-only accelerator with negligible power consumption and chip area overhead.

Internet-of-Things (IoT) opens a new world of possibilities for both personal and industrial applications. At the heart of an IoT device, the processor is the core component. Hence, as an open and free instruction set architecture RISC-V is gaining huge popularity for IoT. A large ecosystem is available around RISC-V, including various RTL implementations at one end and high-speed instruction set simulators (ISSs) at the other end. These ISSs facilitate functional verification of RTL implementations as well as early SW development to some extent. However, being designed predominantly for speed, they can hardly be extended to support further system-level use cases such as design space exploration, power/timing/performance validation or analysis of complex HW/SW interactions. In this paper, we propose and implement the first RISC-V based Virtual Prototype (VP) with the goal of filling this gap. We provide a RISC-V RV321M core, a PLIC-based interrupt controller and an essential set of peripherals together with SW debug capabilities. The VP is designed as extensible and configurable platform with a generic bus system and implemented in standard-compliant SystemC and TLM-2.0. The latter point is very important, since it allows to leverage cutting-edge SystemC-based modeling techniques needed for the mentioned use cases. Our VP allows a significantly faster simulation compared to RTL, while being more accurate than existing ISSs. Finally, our RISC-V VP is fully open source to help expanding the RISC-V ecosystem and stimulating further research and development.

Instruction Set Simulators (ISSes) are important tools for cross-platform software development. The simulation speed is a major concern and many approaches have been proposed to improve the performance of ISSes. A prevalent technique is compiled simulation, which translates target programs into host instructions. But orders of magnitude of speed deterioration is inevitable since the difference between target and host Instruction Set Architectures (ISAs) can be large. An alternative is to emulate the program without sticking to binary compatibility. The performance problem is solved by using native execution. However, these emulators either require a special programming language, or a given Application Programming Interface (API). Last but not least, it is not trivial to integrate an emulator into a system simulator (which provides devices, external memory, etc., that the embedded programmers do care). In this paper, we propose a fast and generic hybrid simulation approach using virtualization technique to accelerate simulation and simulator-based debugging of C programs. A novel virtual coprocessor (VCP) is introduced as a processing element which executes C functions at high speed. This approach is C89 compliant and compatible with third party libraries and platform dependent code. It is also retargetable and can be integrated with existing ISSes. Two different ISAs are supported at present: MIPS and mAgic DSP. The average execution speed of the coprocessor is about 100 million simulated instructions per second.

The efficient use of future MPSoCs with 1000 or more processor cores requires new means of resource-aware programming to deal with increasing imperfections such as process variation, fault rates, aging effects, and power as well as thermal problems. In this paper, we apply a new approach called invasive computing that enables an application programmer to spread computations to processors deliberately and on purpose at certain points of the program. Such decisions can be made depending on the degree of application parallelism and the state of the underlying resources such as utilization, load, and temperature. The introduced programming constructs for resource-aware programming are embedded into the parallel computing language X10 as developed by IBM using a library-based approach. Moreover, we show how individual heterogeneous MPSoC architectures may be modeled for subsequent functional simulation by defining compute resources such as processors themselves by lightweight threads that are executed in parallel together with the application threads by the X10 run-time system. Thus, the state changes of each hardware resource may be simulated including temperature, aging, and other useful monitor functionality to provide a first high-level programming test-bed for invasive computing.

SUMMARY Heterogeneous multi-core architectures with CPUs andaccelerators attract many attentions since they can achieve power-efficientcomputing in various areas from low-power embedded processing to high-performance computing. Since the optimal architecture is different fromapplication to application, finding the most suitable accelerator is very im-portant. In this paper, we propose an FPGA-based heterogeneous multi-core platform with custom accelerators for power-efficient computing. Us-ing the proposed platform, we evaluate several applications and accelera-tors to identify many key requirements of the applications and propertiesof the accelerators. Such an evaluation is very important to select and op-timize the most suitable accelerator according to the requirements of anapplication to achieve the best performance. key words: heterogeneous multicore processor, FPGA, Multimedia pro-cessing, High-performance-computing 1. Introduction Applications used in low-power embedded processing tohigh performance computing have different tasks such asdata-intensive tasks and control-intensive tasks. Therefore,optimal architecture is different from application to applica-tion. Heterogeneous multicore processing is proposed to ex-ecute applications power-efficiently. It uses different proces-sor cores such as CPU cores and accelerator cores as shownin Fig.1. If the tasks of an application are correctly allocatedto the most suitable processor cores, all the cores work to-gether to increase the overall performances.Examples of low-power heterogeneous multi-core pro-cessors are [1] and [2]. The former has multiple coresof CPUs and ALU arrays. The latter has multiple coresof CPUs, a micro-controller and SIMD (single-instructionmultiple-data) type processors. An example of a hetero-geneous high-performance computing is “Tianhe-1A” [3]which has Intel X5670 CPUs and NVDIA GPUs. Com-mercially available heterogeneous multicore processors arepartially programmable so that a part of the data path andcomputations of processing elements (PEs) can be changedto some extent. However, due to the wide variety of tasksand their different memory requirements, the programma-bility in commercially available processors is not enough toextract sufficient performance. Moreover, the programmingenvironments in various heterogeneous architectures such as

Transaction level modeling (TLM) improves the simulation performance by raising the abstraction level. In the TLM 2.0 standard based on OSCI SystemC, a single transaction can transfer a large data block. Due to such high abstraction, a great amount of information becomes invisible and thus timing accuracy can be degraded heavily. We present a methodology to accurately time such block transactions and achieve high simulation performance at the same time. First, before abstraction, a profiling process is performed on an instruction set simulator (ISS). Driver functions that implement the transfer of the data blocks are simulated. Several techniques are employed to trace the exact start and end of the driver functions as well as HW usages. Thus, a profile library of those driver functions can be constructed. Then, the application programs are host-compiled and use a single transaction to transfer a data block. A strategy is presented that efficiently estimates the timing of block transactions based on the profile library. It is the first method that takes into account caching effects that influence the timing of block transactions. Moreover, it ensures overall timing accuracy when integrated in other SW timing tools for full system simulation. Experimental results show that the block transactions are accurately timed, with average error less than 1%. At the same time, the simulation gain can be up to three orders of magnitude.

Internet-of-Things (IoT) opens a new world of possibilities for both personal and industrial applications. At the heart of an IoT device, the processor is the core component. Hence, as an open and free instruction set architecture RISC-V is gaining huge popularity for IoT. A large ecosystem is available around RISC-V, including various RTL implementations at one end and high-speed instruction set simulators (ISSs) at the other end. These ISSs facilitate functional verification of RTL implementations as well as early SW development to some extent. However, being designed predominantly for speed, they can hardly be extended to support further system-level use cases such as design space exploration, power/timing/performance validation or analysis of complex HW/SW interactions. In this paper, we propose and implement the first RISC-V based Virtual Prototype (VP) with the goal of filling this gap. We provide a RISC-V RV321M core, a PLIC-based interrupt controller and an essential set of peripherals together with SW debug capabilities. The VP is designed as extensible and configurable platform with a generic bus system and implemented in standard-compliant SystemC and TLM-2.0. The latter point is very important, since it allows to leverage cutting-edge SystemC-based modeling techniques needed for the mentioned use cases. Our VP allows a significantly faster simulation compared to RTL, while being more accurate than existing ISSs. Finally, our RISC-V VP is fully open source to help expanding the RISC-V ecosystem and stimulating further research and development.

Author index

Near-data in-memory processing research has been gaining momentum in recent years. Typical processing-in-memory architecture places a single or several processing elements next to a volatile memory, enabling processing without transferring data to the host CPU. The increased bandwidth to and from volatile memory leads to performance gain. However processing-in-memory does not alleviate von Neumann bottleneck for big data problems, where datasets are too large to fit in main memory. We present a novel processing-in-storage system based on Resistive Content Addressable Memory ReCAM. It functions simultaneously as a mass storage and as a massively parallel associative processor. ReCAM processing-in-storage resolves the bandwidth wall by keeping computation inside the storage arrays, without transferring it up the memory hierarchy. We show that ReCAM based processing-in-storage architecture may outperform existing processing-in-memory and accelerator based designs. ReCAM processing-in-storage implementation of Smith-Waterman DNA sequence alignment reaches a speedup of almost five over a GPU cluster. An implementation of in-storage inline data deduplication is presented and shown to achieve orders of magnitude higher throughput than traditional CPU and DRAM based systems.

With an increasing number of processing elements (PEs) integrated on a single chip, fault-tolerant techniques are critical to ensure the reliability of such complex systems. In current reconfigurable architectures, redundant PEs are utilized for fault tolerance. In the presence of faulty PEs, the physical topologies of various chips may be different, so the concept of virtual topology from network embedding problem has been used to alleviate the burden for the operating systems. With limited hardware resources, how to reconfigure a system into the most effective virtual topology such that the maximum repair rate can be reached presents a significant challenge. In this article, a new approach using a maximum flow (MF) algorithm is proposed for an efficient topology reconfiguration in reconfigurable architectures. In this approach, topology reconfiguration is converted into a network flow problem by constructing a directed graph; the solution is then found by using the MF algorithm. This approach optimizes the use of spare PEs with minimal impacts on area, throughput, and delay, and thus it significantly improves the repair rate of faulty PEs. In addition, it achieves a polynomial reconfiguration time. Experimental results show that compared to previous methods, the MF approach increases the probability to repair faulty PEs by up to 50% using the same redundant resources. Compared to a fault-free system, the throughput only decreases by less than 2.5% and latency increases by less than 4%. To consider various types of PEs in a practical application, a cost factor is introduced into the MF algorithm. An enhanced approach using a minimum-cost MF algorithm is further shown to be efficient in the fault-tolerant reconfiguration of heterogeneous reconfigurable architectures.