Accelerating Code Assembly: Exploiting Heterogeneous Computing Architectures for Optimization

SUMMARY Heterogeneous multi-core architectures with CPUs andaccelerators attract many attentions since they can achieve power-efficientcomputing in various areas from low-power embedded processing to high-performance computing. Since the optimal architecture is different fromapplication to application, finding the most suitable accelerator is very im-portant. In this paper, we propose an FPGA-based heterogeneous multi-core platform with custom accelerators for power-efficient computing. Us-ing the proposed platform, we evaluate several applications and accelera-tors to identify many key requirements of the applications and propertiesof the accelerators. Such an evaluation is very important to select and op-timize the most suitable accelerator according to the requirements of anapplication to achieve the best performance. key words: heterogeneous multicore processor, FPGA, Multimedia pro-cessing, High-performance-computing 1. Introduction Applications used in low-power embedded processing tohigh performance computing have different tasks such asdata-intensive tasks and control-intensive tasks. Therefore,optimal architecture is different from application to applica-tion. Heterogeneous multicore processing is proposed to ex-ecute applications power-efficiently. It uses different proces-sor cores such as CPU cores and accelerator cores as shownin Fig.1. If the tasks of an application are correctly allocatedto the most suitable processor cores, all the cores work to-gether to increase the overall performances.Examples of low-power heterogeneous multi-core pro-cessors are [1] and [2]. The former has multiple coresof CPUs and ALU arrays. The latter has multiple coresof CPUs, a micro-controller and SIMD (single-instructionmultiple-data) type processors. An example of a hetero-geneous high-performance computing is “Tianhe-1A” [3]which has Intel X5670 CPUs and NVDIA GPUs. Com-mercially available heterogeneous multicore processors arepartially programmable so that a part of the data path andcomputations of processing elements (PEs) can be changedto some extent. However, due to the wide variety of tasksand their different memory requirements, the programma-bility in commercially available processors is not enough toextract sufficient performance. Moreover, the programmingenvironments in various heterogeneous architectures such as

Computer architecture is transiting from the multicore era into the heterogeneous era in which heterogeneous architectures use on-chip networks to access shared resources and how a network is configured will likely have a significant impact on overall performance and power consumption. Recently, heterogeneous network on chip (NoC) has been proposed not only to achieve performance comparable to that of the NoCs with buffered routers but also to reduce buffer cost and energy consumption. However, heterogeneous NoC design for heterogeneous GPU-CPU architectures has not been studied in depth. This paper first evaluates the performance and power consumption of a variety of static hot-potato based heterogeneous NoCs with different buffered and bufferless router placements, which is helpful to explore the design space for heterogeneous GPU-CPU interconnection. Then it proposes Unidirectional Flow Control (UFC), a simple credit-based flow control mechanism for heterogeneous NoC in GPU-CPU architectures to control network congestion. UFC can guarantee that there are always unoccupied entries in buffered routers to receive flits coming from adjacent bufferless routers. Our evaluations show that when compared to hot-potato routing, UFC improves performance by an average of 14.1% with energy increased by an average of 5.3% only.

: The objective of the Integrated High Performance Distributed System (IHPDS) project is to design, implement, and demonstrate a distributed programming environment that will integrate and support very high bandwidth networks with heterogeneous computer architectures, including parallel and specialized processors, and that will support multiple programming models. The IHPDS project is developing the Photon distributed programming environment to meet the emerging needs of distributed application programmers. Photon provides an object-oriented programming model. This provides users with a consistent view of all services while insulating the user from the implementation details of those services. This enforcement of modularity and the separation of policy and mechanism make the object model a powerful way to build large-scale, distributed applications. Photon is a distributed computing environment that supports distributed application development across heterogeneous machine architectures and programming languages. It is particularly targeted to support the needs of high performance applications. Photon not only supports applications whose components are located within a single local area network, but also supports applications whose components are widely dispersed geographically.

The dramatic increase in the number of heterogeneous computing architectures (HCAs) which include CPUs, GPUs, TPUs, FPGAs, and new neuromorphic processors has transformed the high-performance computing (HPC) and artificial intelligence (AI) workloads. As the complexity of architecture increases there is corresponding uncertainty on reliability and performance checks, particularly when systems are incorporated with different processing elements, interconnections, and memory levels. Modern non-deterministic or non-stochastic heterogeneous environments are associated with traditional reliability models that feel great necessity to model nonlinear interdependencies between components. The current paper proposes an innovative AI-based modeling framework that would be capable of measuring system reliability and performance indicators in the context of HCAs. The suggested solution merges Deep Neural Networks (DNNs), Bayesian Inference Models, and the strategies of Reinforcement Learning (RL) to forecast the increase of the system reliability, the performance bottlenecks, and the mean time between failures (MTBF) at the system level. To train AI models that can capture the multi-domain interactions of computation, communication, and thermal dynamics, a large-scale simulation dataset was created with synthetic benchmarks, actual workloads (SPEC ACCEL, MLPerf, HPCG), unlikely fault profiles, and injected fault profiles to have AI models that are trained to simulate interactions in large-scale systems. The approach puts the feature engineering of runtime telemetry (e.g., power, temperature, utilization) and hardware counters (e.g., instruction-level parallelism, cache miss rates) to construct predictive and adaptive reliability estimators. As opposed to the traditional models, our AI-based system advances its state dynamically with online learning methods of training, allowing its internal state to identify fault and self-optimize it. Findings suggest that the suggested AI system can make accurate predictions in both system performance degradation and reliability estimates with a success level of 96.8 and 94.2 per cent respectively when operating in a heterogeneous environment. Comparative research with classical models of reliability, i.e. Markov and Weibull-based models reveal high gains on adaptability, precision and generalization. Moreover, the optimization using reinforcement learning provided a 1525 percentage ratio in task scheduling performance expressed in a limited thermal and power budget. This article adds to the increasing overlap between system modeling based on AI and the optimization of heterogeneous computing, sets the stage of the introduction of new intelligent reliability management of data centers, autonomous computing systems, and HPC infrastructure

We present a novel, linear programming (LP)-based scheduling algorithm that exploits heterogeneous multicore architectures such as CPUs and GPUs to accelerate a wide variety of proximity queries. To represent complicated performance relationships between heterogeneous architectures and different computations of proximity queries, we propose a simple, yet accurate model that measures the expected running time of these computations. Based on this model, we formulate an optimization problem that minimizes the largest time spent on computing resources, and propose a novel, iterative LP-based scheduling algorithm. Since our method is general, we are able to apply our method into various proximity queries used in five different applications that have different characteristics. Our method achieves an order of magnitude performance improvement by using four different GPUs and two hexa-core CPUs over using a hexa-core CPU only. Unlike prior scheduling methods, our method continually improves the performance, as we add more computing resources. Also, our method achieves much higher performance improvement compared with prior methods as heterogeneity of computing resources is increased. Moreover, for one of tested applications, our method achieves even higher performance than a prior parallel method optimized manually for the application. We also show that our method provides results that are close (e.g., 75 percent) to the performance provided by a conservative upper bound of the ideal throughput. These results demonstrate the efficiency and robustness of our algorithm that have not been achieved by prior methods. In addition, we integrate one of our contributions with a work stealing method. Our version of the work stealing method achieves 18 percent performance improvement on average over the original work stealing method. This result shows wide applicability of our approach.

Recent advances like deep learning algorithms or virtual reality applications require an amount of computational power in increasingly smaller devices never seen before. Heterogeneous architectures are seen as a solution to this problem, since they provide a significantly better performance per watt. However, their comparatively difficult integration and programming are a major drawback of this solution. The Heterogeneous System Architecture (HSA) Foundation accepted this problem and published a set of specifications to provide a uniform solution for different architectures. This has been successfully applied to GPUs and DSPs, but is insufficiently researched for FPGAs. However, the usage of reconfigurable logic can further increase the efficiency of embedded systems, while the hurdle of integration can be lowered by high-level synthesis (HLS). Therefore, a combination of the HSA specifications with HLS seems worthwhile to increase the heterogeneity and ease the programmability of FPGAs. This paper provides a suitability analysis of the HSA Intermediate Language (HSAIL) for HLS with suggestions for further improvements to the specification as well as an associated proof of concept. The results show that it is indeed possible to build such a hybrid HLS flow based on the HSA standard, where parts of the process can be shared with other classes of accelerators like GPUs. This implies that progress in the GPU sector like the available source languages can faster carry over to FPGAs and improve the general accessibility.

This paper presents the use of a Domain-Specific Language (DSL) for Software Defined Radio (SDR) in a Radar digital signal processing (DSP) using heterogeneous computing architectures (HCAs). These HCAs are a combinations of mul-ticore CPU, GPUs, and FPGAs. This DSL, which we named OptiSDR, uses a dataflow-like model of computations named parallel stream processing and a compiler guided optimization technique to provide optimized parallel executable patterns for a hybrid MCPU-GPU architecture. In this paper, we demonstrate the capabilities of OptiSDR in a Radar DSP case study, and show how OptiSDR achieves up to 50 times (50X) speed-up as compared to the hand-crafted Matlab and Octave scripts for the NetRAD pulse compression algorithm. We also show that, while hand-crafted CUDA-Qt implementation of the same pulse compression algorithm presents up to 2X speed-up against Op-tiSDR, this was minimized using platform-specific optimizations by efficiently utilizing available computing resources such as number of GPUs, memory, and number of cores of a MCPU.

Heterogeneous computing has emerged as a promising fit for scientific domains such as molecular dynamics simulations, bioinformatics, weather prediction. Such a computing paradigm includes x86 processors coupled with GPUs, FPGAs, DSPs or a coprocessor paradigm that takes advantage of all the cores and caches on a single die such as the Knights Landing. OpenACC, a high-level directive-based parallel programming model has emerged as a programming paradigm that can tackle the intensity of heterogeneity in architectures. Data-driven large scientific codes are increasingly using OpenACC, which makes it essential to analyze the accuracy of OpenACC compilers while they port code to various types of platforms. In response, we have been creating a validation suite to validate and verify the implementations of OpenACC features in conformance with the specification. The validation suite also provides a tool to compiler developers as a standard for the compiler to be tested against and to users and compiler developers alike in clarifying the OpenACC specification. This testsuite has been integrated into the harness infrastructure of the TITAN and Summitdev systems at Oak Ridge National Lab and is being used for production.

The advancements in modern computer architecture have made it possible for the simulation of large-scale problems that are of industrial application. However, it can be time-intensive to continually port scientific codes to the ever-evolving computer architectures. Therefore, it is important to develop computational tools that make it easy for researchers to port their existing scientific software or develop new software that can run optimally on current and future computer architectures from different vendors. Also, it often advantageous for scientific software to leverage fine-grained parallelism on homogeneous architectures that are comprised of multi-core CPUs and on heterogeneous computer architectures comprised of multi-core CPUs and GPUs. This paper demonstrates the incorporation of performance portability and data-oriented design paradigms in the phase-field modeling of microstructure evolution. To achieve this, we utilized MATAR, a C+ + software library that allows the straightforward creation and usage of multidimensional and multi-size dense or sparse matrix and array data structures that are also portable across disparate architectures using Kokkos. As a case study, we use the phase-field model for spinodal decomposition which numerically solves the Cahn-Hilliard equation using the semi-implicit Fourier spectral method. Performance portability across multi-core CPUs and different GPU architectures are investigated. Additionally, numerical accuracy and performance gains between single and double precision phase-field calculations are investigated.

It is our pleasure to introduce the technical program for the fifteenth edition of ASPLOS. The conference this year continues and reinforces the ASPLOS tradition of publishing innovative research spanning the boundaries of hardware, computer architecture, compilers, programming languages, operating systems, and distributed computing. We received 181 submissions this year, which was a record and a significant reversal of a trend from the last 3 years (158, 127 and 113 in 2006, 2008 and 2009, respectively). 19 submissions had PC members as co-authors. The most dominant theme, by far, was parallel processing: over half of the submissions were directly related to this topic, most of which mentioned multicore, manycore, SIMD, or heterogeneous architectures as a target. There were two key changes in the ASPLOS paper reviewing process this year, both of which have been used in other systems conferences: two rounds of reviews, and the use of an External Review Committee preselected by the Program Chair. Each paper was initially reviewed by two PC members and one ERC member. Based on these reviews, 91 papers were selected for a second round of reviews; see the Program Chair's Report on the ASPLOS Web site for the details on how these were selected. These papers received one or two more PC reviews plus one more ERC review. Together, these two changes held the average reviewing load per PC member to under 18 papers, while allowing them to spend more time on papers with a significant chance of being accepted. The PC met in person at the Chicago O'Hare Hilton for a one-day meeting on Oct. 30, 2009. 73 papers were discussed at the meeting. The PC chose 32 papers to be presented at the conference, for an overall acceptance rate of 17.7%. 6 of the 32 have PC members as co-authors. In another departure from previous years, all papers were assigned a shepherd to ensure that the final papers adequately addressed the concerns of the reviewers. The PC also awarded a Best Paper Award and nominated three papers for the CACM Research Highlights. If ASPLOS XV is successful, as we hope, numerous people should share the credit. Most of all, the technical program directly reflects the high quality of the submitted papers; we thank all the authors of the submissions for their effort. The members of the PC and ERC put in a very substantial effort for reviewing and shepherding, and for selecting the award papers. Mike Hind (IBM Research), in particular, also handled the review process and discussions for papers where the PC chair had conflicts, and provided valuable advice during the reviewing process. Seth Goldstein (CMU) once again organized and put his inimitable stamp on the ASPLOS Wild and Crazy Ideas session. We owe them all a debt of gratitude. The extensive logistics behind organizing this conference have been handled by the tireless effort of a large team of Pittsburgh volunteers: Shimin Chen (Intel), Allen Cheng (Pitt), Sangyeun Cho (Pitt), Franz Franchetti (CMU), Ken Mai (CMU), Todd Mowry (CMU), Onur Mutlu (CMU), Jun Yang (Pitt) and Youtao Zhang (Pitt). Each member of this team played an important role in making ASPLOS in Pittsburgh a success. In addition, the ASPLOS Steering Committee gave us timely and valuable guidance over the last year. We want to thank all members of the organizing committee and the steering committee for the time and attention they have invested in this conference.

Dynamic heterogeneous graph neural networks (DHGNNs) have been shown to be effective in handling the ubiquitous dynamic heterogeneous graphs. However, the existing DHGNNs are hand-designed, requiring extensive human efforts and failing to adapt to diverse dynamic heterogeneous graph scenarios. In this paper, we propose to automate the design of DHGNN, which faces two major challenges: 1) how to design the search space to jointly consider the spatial-temporal dependencies and heterogeneous interactions in graphs; 2) how to design an efficient search algorithm in the potentially large and complex search space. To tackle these challenges, we propose a novel Dynamic Heterogeneous Graph Attention Search (DHGAS) method. Our proposed method can automatically discover the optimal DHGNN architecture and adapt to various dynamic heterogeneous graph scenarios without human guidance. In particular, we first propose a unified dynamic heterogeneous graph attention (DHGA) framework, which enables each node to jointly attend its heterogeneous and dynamic neighbors. Based on the framework, we design a localization space to determine where the attention should be applied and a parameterization space to determine how the attention should be parameterized. Lastly, we design a multi-stage differentiable search algorithm to efficiently explore the search space. Extensive experiments on real-world dynamic heterogeneous graph datasets demonstrate that our proposed method significantly outperforms state-of-the-art baselines for tasks including link prediction, node classification and node regression. To the best of our knowledge, DHGAS is the first dynamic heterogeneous graph neural architecture search method.

Design space exploration for device and architectural heterogeneity in chip-multiprocessors

Modern heterogeneous high-end computing systems featuring many-core co-processors/accelerators pose tough challenges to parallelize and optimize real-world scientific codes. In this paper, we demonstrate highly scalable 3D Lattice Boltzmann multiphase flow simulations on the heterogeneous Tianhe-2 su-percomputer using MPI+OpenMP+SIMD. We highlight the use of the OpenMP4.5 accelerator programming model to collaborate CPUs and Intel Many Integrated Cores (MIC) co-processors, and present a range of optimizations to exploit hierarchical parallelism on the heterogeneous architecture. With SIMD-friendly data structures, computation reordering, cache blocking and auto-vectorization, we dramatically optimize the single-thread performance for typical LBM computational kernels on a CPU and MIC core. After implementation of shared-memory OpenMP threading, we further improve the OpenMP performance by enabling huge paging and OpenMP collapse clause on the thread-rich MIC architecture. To enhance the collaborative efficiency among intra-node CPUs and co-processors, we propose a flexible load balance model with heterogeneous domain decomposition for CPU-MIC task allocation, as well as asynchronous offloading to overlap operations of CPUs and multiple MICs. We effectively overlap all levels of CPU-MIC computation/communication and minimize the cost of halo exchanges as far as possible in large-scale scenario. 3D liquid and gases multi-phase cases simulating drop impact with gravity effect using D3Q19 Lattice Boltzmann discretization and Shan-Chen BGK collision model are presented, achieving a weak parallel efficiency of above 80% in going from 128 to 2048 compute nodes.

The demands on bandwidth, latency and energy efficiency are ever increasing in AI computing. Chiplets, connected by 2. 5D interconnect, promise a scalable platform to meet such needs. We present a pathfinding study to bridge AI algorithms with the chiplet architecture, covering in memory computing (IMC), network-on-package (NoP), and heterogeneous architecture. This study is enabled by our newly developed benchmarking tool, SIAM. We perform simulations on representative algorithms (DNNs, transformers and GCNs). Particular contributions include: (1) A roadmap of 2. 5D interconnect for technological exploration; (2) A generic mapping and optimization methodology that reveals various bandwidth needs in AI computing, where the evolution of 2.5D interconnect can or cannot support; (3) A big-little chiplet architecture that matches the non-uniform nature of AI algorithms and achieves >100× improvement in EDP. Overall, heterogeneous big-little chiplets with 2. 5D interconnect advance AI computing to the next level of data movement and computing efficiency.

The demands on software continues to increase through the constant addition of functionalities and high expectations from users. In particular, performance has been the focus in many projects with the goal of fulfilling complex and hard requirements across a variety of domains. One way to achieve satisfactory levels of performance is through heterogeneous computing, i.e., systems that contain more than one type of processing unit, such as CPUs, GPUs, and FPGAs. However, applications are typically designed to be executed on CPUs, and re-architecting software for execution on such heterogeneous hardware architectures entails several challenges that must be addressed. In this paper, we propose a framework that supports engineers in the process of making architectural decisions to re-architect software for execution on heterogeneous platforms. We present several relevant aspects that should be addressed in the process, along with suggestions on how to create design solutions using different existing approaches. The framework was developed based on multiple interactions with three industrial partners and evaluated through a computer vision application in the automotive domain.