Performance Modelling and Dynamic Scheduling on Heterogeneous-ISA Multi-core Architectures

Given the ever increasing demand for improved computational capabilities, heterogeneous-ISA multi-core architectures have emerged as a promising alternative to improve single-threaded performance. Such architectures comprise of multiple cores that differ not just in micro-architectural parameters but also in their Instruction Set Architectures (ISAs). Programs have affinity towards different ISAs during its execution based on nature of code and data input. To extract maximum performance gain, we need to ensure that at every point in the program's execution, the program runs on its best affine core with minimum migration overhead. In this letter, we propose a function-wise fine grained scheduling algorithm which schedules every function of a program to its most affined ISA. Results show that our function-based scheduler can achieve speedup of up to 22.9 percent over state-of-the-art in heterogeneous-ISA architectures.

We present a methodology for automatically generating an online scheduling process for an arbitrary objective with the help of evolution strategies. The scheduling problem comprises independent parallel jobs and multiple identical machines and occurs in many real massively parallel processing systems. The system owner defines the objective that may consider job waiting times and priorities of user groups. Our scheduling process is a variant of the simple and commonly used greedy scheduling algorithm in combination with a repeated sorting of the waiting queue. This sorting uses a criterion whose parameters are evolutionary optimized. We evaluate our new scheduling process with real workload data and compare it to the best offline solutions and to the online results of the standard EASY backfill algorithm. To this end, we partition the user of the workloads into groups and select an exemplary objective that prioritizes some of those groups over others

Heterogeneous processors that mix big high performance cores with small low power cores promise excellent single-threaded performance coupled with high multi-threaded throughput and higher performance-per-watt. A significant portion of the commercial multicore heterogeneous processors are likely to have a common instruction set architecture( ISA). However, due to limited design resources and goals, each core is likely to contain ISA extensions not yet implemented in the other core. Therefore, such heterogeneous processors will have inherent functional asymmetry at the ISA level and face significant software challenges. This paper analyzes the software challenges to the operating system and the application layer software on a heterogeneous system with functional asymmetry, where the ISA of the small and big cores overlaps. We look at the widely deployed Intel® Architecture and propose solutions to the software challenges that arise when a heterogeneous processor is designed around it. We broadly categorize functional asymmetries into those that can be exposed to application software and those that should be handled by system software. While one can argue that new software written should be heterogeneity-aware, it is important that we find ways in which legacy software can extract the best performance from heterogeneous multicore systems.

Dynamic Binary Translation (DBT) is the key technology behind cross-platform virtualization and allows software compiled for one Instruction Set Architecture (ISA) to be executed on a processor supporting a different ISA. Under the hood, DBT is typically implemented using Just-In-Time (JIT) compilation of frequently executed program regions, also called traces. The main challenge is translating frequently executed program regions as fast as possible into highly efficient native code. As time for JIT compilation adds to the overall execution time, the JIT compiler is often decoupled and operates in a separate thread independent from the main simulation loop to reduce the overhead of JIT compilation. In this paper we present two innovative contributions. The first contribution is a generalized trace compilation approach that considers all frequently executed paths in a program for JIT compilation, as opposed to previous approaches where trace compilation is restricted to paths through loops. The second contribution reduces JIT compilation cost by compiling several hot traces in a concurrent task farm. Altogether we combine generalized light-weight tracing, large translation units, parallel JIT compilation and dynamic work scheduling to ensure timely and efficient processing of hot traces. We have evaluated our industry-strength, LLVM-based parallel DBT implementing the ARCompact ISA against three benchmark suites (EEMBC, BioPerf and SPEC CPU2006) and demonstrate speedups of up to 2.08 on a standard quad-core Intel Xeon machine. Across short- and long-running benchmarks our scheme is robust and never results in a slowdown. In fact, using four processors total execution time can be reduced by on average 11.5% over state-of-the-art decoupled, parallel (or asynchronous) JIT compilation.

Physical and thermal restrictions hinder commensurate performance gains from the ever increasing transistor density. While multi-core scaling helped alleviate dimmed or dark silicon for some time, future processors will need to become more heterogeneous. To this end, single instruction set architecture (ISA) heterogeneous processors are a particularly interesting solution that combines multiple cores with the same ISA but asymmetric performance and power characteristics. These processors, however, are no free lunch for database systems. Mapping jobs to the core that fits best is notoriously hard for the operating system or a compiler. To achieve optimal performance and energy efficiency, heterogeneity needs to be exposed to the database system.

Heterogeneous systems with multiple different compute devices have come into common use recently, and the heterogeneity of the compute device is mainly reflected in three aspects: hardware architecture, instruction set architecture, and processing capability. Heterogeneous CPU-accelerator systems have attracted increasing attention especially. To make full use of multiple CPUs and accelerators to execute data-parallel applications, programmers may need to manually map computation and data to all available compute devices, which is tedious, error-prone, and difficult. Especially for some data-parallel applications, the inter-device communication could easily become the performance bottleneck of multi-device co-execution. Therefore, firstly, a runtime system is designed for supporting heterogeneous cooperative execution (HCE) of data-parallel applications, which can help programmers to automatically and efficiently map computation and data to multiple compute devices. Secondly, an incremental data transfer method is designed to avoid redundant data transfers between devices, and a three-way overlapping communication optimization method based on software pipelining is designed to effectively hide the inter-device communication overhead. Based on our previously proposed feedback-based dynamic and elastic task scheduling (FDETS) scheme and asynchronous-based dynamic and elastic task scheduling (ADETS) scheme, the modified FDETS that supports incremental data transfer and the modified ADETS that supports three-way overlapping communication optimization are proposed, which not only can effectively partition and balance the workload among multiple compute devices but also can significantly reduce data transfer overhead between devices. Thirdly, a prototype of the proposed runtime system is implemented, which provides a set of runtime APIs for task scheduling, device management, memory management, and transfer optimization. Our experimental results show that the communication overhead between devices is greatly reduced using the proposed inter-device communication optimization methods and the multi-device co-execution significantly outperforms the best single-device execution.

Edge computing provides a feasible technical solution to solve the problem of resource and energy supply constraints in terminal device deep learning applications. By migrating part of the deep learning application computing from the terminal to the cloud edge server, the computing pressure of the terminal device can be greatly relieved, and the energy consumption of the terminal can also be reduced to a certain extent. However, in the process of computing migration, the context switching loss of deep learning applications results in a long delay. This paper studies the problem of minimizing the completion time of deep learning applications under edge computing. We model the deep learning application as a task graph, and consider the constraints such as task switching delay, energy consumption and task dependency. Without changing the structure of the task graph, we transform it into a task scheduling-based application completion time minimization problem. In this paper, we first design a parallel scheduling algorithm based on greedy search. Considering the loss of context switching and the dependence of task data, we can allocate tasks to multiple processors for parallel processing by calculating the expected completion time of tasks, thus reducing the completion time of applications. Based on the experimental results, the greedy scheduling algorithm and comparison algorithm are analyzed and verified to optimize the completion time of different depth learning applications.

Real-time and embedded systems always have performance constraints. With conventional design techniques, these constraints are verified at the testing phase and the designer does no performance constraint estimation during the design. From the design information, it is possible to deduce a queueing network performance model, but its execution requires supplementary information concerning estimations of the execution times. Delays generated by logical locking and by resource use conflicts can be computed, via analytical or simulation techniques, from such a model. This paper presents experience acquired in the integration of design methods and the performance modelling techniques. >

In this paper, we consider scheduling in multicast switching networks, which aims to minimize the multicast latency for a set of multicast requests. Such a problem has been proved to be NP-complete. We propose a simple, fast greedy multicast scheduling algorithm and derive a lower bound and an upper bound on the performance of the algorithm. As can be seen, while a lower bound is fairly straightforward, the upper bound is much more difficult to obtain. By translating the multicast scheduling problem into a graph theory problem and employing a random graph approach, we are able to obtain a probabilistic upper bound on the performance of the multicast scheduling algorithm. Our analytical and simulation results show that the performance of the proposed multicast scheduling algorithm is quite close to the lower bound and is statistically guaranteed by the probabilistic upper bound.

Workloads with precedence constraints due to data dependencies are common in various applications. These workloads can be represented as directed acyclic graphs (DAG), and are often data-intensive, meaning that data loading cost is the dominant factor and thus cache misses should be minimized. We address the problem of parallel scheduling of a DAG of data-intensive tasks to minimize makespan. To do so, we propose greedy online scheduling algorithms that take load balancing, data dependencies, and data locality into account. Simulations and an experimental evaluation using an Apache Spark cluster demonstrate the advantages of our solutions.

As the smart cities emerged for more comfortable urban spaces, services, such as health, transportation, and so on, need to be promoted. In addition, the cloud computing provides flexible allocation, migration of services, and better security isolation; therefore, it is the infrastructure for the smart cities. Single instruction-set architecture (ISA) heterogeneous multi-core processors have higher performance per watt than their symmetric counterparts and are popular in current processors. In current cloud computing, which integrates a few fast out-of-order cores, coupled with a large number of simpler, slow cores, all cores expose the same ISA. The best way to leverage the effectiveness of these systems is to accelerate sequential CPU-bound threads using fast cores, and to improve the throughput of parallel memory-bound threads using slow cores. However, shared hardware resources, such as memory, respond to requests from all cores, which interfere with each other, leading to both low speed for fast cores and low throughput for slow cores. In this paper, we propose a dynamic resource partitioning (DRP) method for single-ISA heterogeneous multi-cores, which partitions the shared resources according to both threads' requirements for the shared resources and the performance of their running cores. The key principle is to profile both threads' resource characteristics at run-time and the performance of the cores that the threads are running on to estimate demands for resources. Then, we use the estimation to direct our resource partitioning. Moreover, we integrate our DRP with current memory scheduling policies to improve the system performance further for the two methods being orthogonal.

In today’s technological landscape, embedded and IoT devices face escalating demands for performance and power efficiency in inference tasks employing Convolution Neural Networks (CNNs). Various methods have emerged to improve performance, including quantization and the adoption of fixed-point formats instead of floating-point formats. RISC-V Instruction Set Architecture (ISA) being open-source and with a modular specification poses as a suitable platform for targeting such applications for embedded processing. The RISC-V’s standard P-extension represents a significant advancement in integer-based computation, offering new ways for achieving higher speedups through integer-packed SIMD instructions. This work presents the first RISC-V processor based on RV32IM base ISA with an optional support of P-extension Version 0.9.11-draft-20211209 called the P-CORE. The P-CORE implements P-extension ISA for RV32 specification. The implemented processor is evaluated for Xilinx Zynq Ultrascale+ FPGA using different test case functions; matrix multiplication, convolution, max-pooling, and fully connected. A CNN used for detecting handwritten digits called Lenet-5 is also used as a test case to evaluate the P-CORE. The evaluation also presents Design Space Exploration (DSE) for P-extension by exploiting the level of data-level-parallelism; SIMD8 (8-bit (4-way)) and SIMD16 (16-bit (2-way)). Our investigation demonstrates that with the P-extension activated, CNN algorithm computations can achieve speedups of up to 17x for max-pooling, 7x for matrix multiplication, and 4.8x for fully connected networks, accompanied by similar enhancements in power efficiency. The processor achieved a maximum performance of 335.1 MIOPS and 490.6 MIOPS/W efficiency for matrix multiplication using -O3 compiler optimization.

Considering a heterogeneous network where both macro base station (BS) and small cell (SC) nodes are equipped with massive number of antennas, this paper studies the performance for multiple-input multiple-output downlinks when the macro and small cells share the same spectrum, and hence interfere with each other. Suppose that the large-scale antenna arrays at both macro BS and SC nodes employ maximum-ratio transmission (MRT) or zero-forcing transmission (ZFT) precoding, and transmit data streams to served users simultaneously. A new pilot reuse pattern among SCs is proposed for channel estimation. Taking into account imperfect channel state information (CSI), capacity lower bounds for MRT and ZFT are derived, respectively, in closed-form expressions involving only statistical CSI, followed by asymptotic analyses for massive arrays under specific power scaling laws. Subsequently, two user scheduling algorithms, greedy scheduling and asymptotical scheduling algorithm (ASA), are proposed based on derived capacity lower bounds and asymptotic analyses, respectively. ASA is demonstrated to be a near optimal in the asymptotic regime and has low complexity. Finally, the derived closed-form expressions are verified to be accurate predictors of the system performance by Monte Carlo simulations. Numerical results demonstrate the effectiveness of the asymptotic analysis and proposed user scheduling schemes.

Multiple-input multiple-output (MIMO) wireless communications systems have attracted a lot of attentions, because of their potential for high gain in channel capacity. Recently, attention has shifted to multiuser MIMO (MU-MIMO) systems. In the schemes for downlink MU-MIMO system, dirty paper coding (DPC) and block diagonalization (BD) algorithms can provide higher sum rate capacity of MU-MIMO systems; however, exhaustive user scheduling for them may cause prohibitive computation complexity. The greedy scheduling algorithms were proposed for DPC and for BD; and proportionally fair (PF) scheduling for MU-MIMO was also discussed in the previous works. However, the discussions in where mainly based on the system model with equal average signal to noise power ratio (SNR) users. In this paper, we focus on the user scheduling for the downlink of MU-MIMO systems with DPC transmission scheme, where each user is assumed to have only one receive antenna, which will simplify the analysis and is expected to give more insight of PF scheduling. We also assume each individual user has different average transmit SNR in the system. We modify the conventional user scheduling algorithm for DPC to modified Gram-Schmidt orthogonalization (M-GSO) based algorithm and extend it into PF scheduling. We also propose a modified PF metric corresponding to the unequal SNR users system model, that increases the sum rate capacity of MU-MIMO system.

This study investigates the scheduling problem for a single-cell downlink distributed antenna systems (DASs) with a massive number of remote access units (RAUs). To reduce signalling overhead under limited backhaul capacity, the authors make use of local long-term channel state information (CSI) in coordinated scheduling design. They first derive the ergodic rate expressions for both the single RAU transmission and the cooperative RAU transmission modes as functions of long-term CSI. Then greedy scheduling algorithms (GSAs) aiming for the maximum ergodic sum rate for the massive DAS using long-term CSI are proposed. To mitigate the intra-cell interference, a two-stage GSA with hybrid transmission mode is devised. Asymptotic analysis reveals that as the number of RAUs goes to infinity, intra-cell interference can be effectively mitigated. Simulation results verify the analysis and demonstrate that the two-stage GSA exhibits a higher ergodic sum-rate.