Control speculation in multithreaded processors through dynamic loop detection

Hardware multithreading is a common approach for tolerating memory latency by utilizing idle cycles and avoiding CPU stalling. Nowadays, multithreading architectures are commonly used across many processors and various embedded edge devices to improve performance. This work suggests a new multithreading in-order pipeline microarchitecture design for RISC-V and proposes an efficient event-based issue scheduling algorithm. The proposed scheduling algorithm is based on the unique RISC-V ISA that enables decoding of the instruction type in an early stage of the pipeline. The RISC-V-based multithreading architecture is evaluated using a dedicated software simulator. Simulation results show that the proposed algorithm outperforms the classical Round Robin and the coarse grain algorithms. The proposed architecture is evaluated using the standard MiBench benchmark and other common applications, demonstrating pipeline utilization improvement of up to about 26% in terms of IPC using four threads.

A simultaneous multithreading multicore processor is proposed to accelerate object recognition for 720p HD video streams. The multithreading architecture with Q-learning based dynamic resource management enables concurrent processing of 8 region-of-interests with 5-stage fine grained recognition pipeline outperforming previous object recognition processors with 342GOPS computing power. In addition, the dynamic resource management contributes to increase of energy efficiency by applying the on-line learning DVFS and dynamic tile allocation based on task variance and hardware utilization to achieve 9.6mJ/frame with 1280×720 pixel image. It achieves 2.72× throughput and 3.7× energy efficiency compared to previous recognition processors.

The throughput of pipelined processors suffers from delays associated with instruction dependencies and memory latencies. Multithreaded architectures attempt to hide such delays by sharing the processor with multiple instruction streams. In this paper we develop a comprehensive analytic framework to quantitatively evaluate the performance of a wide spectrum of mulithreaded machines, ranging from those that are capable of switching threads every cycle to those that switch threads only on long delays. The models are validated against previously published simulation and modeling results, and then used to assess the performance potential of multithreading given current processor technology. >

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> Multithreading and multicore processing are powerful ways to take advantage of parallelism in applications in order to boost a system's performance. However, exploring sufficient parallelism and achieving data locality with low communication overhead are still important research issues in embedded multithreading/multicore design. This paper introduces the design of a fast data switching mechanism between multilevel storage structures in a new multicore architecture. This paper makes several contributions to the development of contemporary sophisticated multimedia applications with advanced standards such as H.264. The first contribution, <emphasis emphasistype="italic">collaborative-multithreading</emphasis>, tightly unifies reduced instruction set computer and collaborative multithreading digital signal processing (DSP) in order to exploit high parallelism to provide sufficient computing power to applications. Each collaborative thread of our DSP is constructed by a heterogeneous-simultaneously multithreading single instruction, multiple data structure, and four media processing cores, which is connected by a fast switch for providing a fast data exchange mechanism among correlative streams on a thread-level basis. Our second contribution is <emphasis emphasistype="italic">one-stop streaming processing</emphasis>, which aims to keep data in the system for as long as possible until it is no longer needed, thus making data more efficient to access. Our third contribution is a <emphasis emphasistype="italic">chunk threading programming model</emphasis>, including a thread management library and threading communication directives for reducing data communication and synchronization overhead. By a combination of coarse-grained and fine-grained threading, programmers can choose various threading levels based on the amount of data exchange in a program. With our proposed techniques and an appropriate programming model, we can reduce processing time by 54.9% in H.264 video encoding (common intermediate format video at 16.574 f/s) with the 1-virtual independent and streaming processing by open collaborative multithreading configuration, compared to the Texas Instruments C62 core that owns 8 function units. We realize our design as a prototype by chip implementation, and fabricate it as a chip based on the Taiwan Semiconductor Manufacturing Company Ltd. 0.13 <formula formulatype="inline"><tex Notation="TeX">$\mu {\rm m}$</tex></formula> process. The die size of the processor core is 16.12 <formula formulatype="inline"><tex Notation="TeX">${\rm mm}^{2}$</tex></formula>, including 414 k logic transistors and 34.4 kB of on-chip static random access memory. The processor runs at 180 MH0z/1.2-V and consumes 245 mW by postsimulation results. </para>

In multiprocessor systems, the gap between peak and effective performance has getting larger. To cope with this performance gap, it is important to use multigrain parallelism in addition to ordinary loop level parallelism. Also, effective use of memory hierarchy is important for the performance improvement of multiprocessor systems because the speed gap between processors and memories is getting larger. This paper describes coarse grain task parallel processing that uses parallelism among macro-tasks like loops and subroutines considering cache optimization using data localization scheme. The proposed scheme is implemented on OSCAR automatic multigrain parallelizing compiler. OSCAR compiler generates OpenMP FORTRAN program realizing the proposed scheme from a sequential FORTRAN77 program. Its performance is evaluated on IBM RS6000 SP 604e High Node 8 processors SMP machine using SPEC95fp tomcatv, swim, mgrid. In the evaluation, the proposed coarse grain task parallel processing scheme with cache optimization gives us up to 1.3 times speedup on 1PE, 4.7 times speedup on 4PE and 8.8 times speedup on 8PE compared with a sequential processing time.

Multithreaded processors in their different organizations (simultaneous, coarse grain and fine grain) have been shown as effective architectures to reduce the issue waste. On the other hand, retiring instructions from the pipeline in an out-of-order fashion helps to unclog the ROB when a long latency instruction reaches its head. This further contributes to maintain a higher utilization of the available issue bandwidth. In this paper, we evaluate the impact of retiring instructions out of order on different multithreaded architectures and different instruction fetch policies, using the recently proposed Validation Buffer microarchitecture as baseline out-of-order commit technique. Experimental results show that, for the same performance, out-of-order commit permits to reduce multithread hardware complexity (e.g., fine grain multithreading with a lower number of supported threads).

The instruction-level parallelism found in a conventional instruction stream is limited. Studies have shown the limits of processor utilization even for today's superscalar microprocessors. One solution is the additional utilization of more coarse-grained parallelism. The main approaches are the (single) chip multiprocessor and the multithreaded processor which optimize the throughput of multiprogramming workloads rather than single-thread performance. The chip multiprocessor integrates two or more complete processors on a single chip. Every unit of a processor is duplicated and used independently of its copies on the chip. In contrast, the multithreaded processor is able to pursue two or more threads of control in parallel within the processor pipeline. Unused instruction slots, which arise from pipelined execution of single-threaded programs by a contemporary microprocessor, are filled by instructions of other threads within a multithreaded processor. The execution units are multiplexed between the threads in the register sets. Underutilization of a superscalar processor due to missing instruction-level parallelism can be overcome by simultaneous multithreading, where a processor can issue multiple instructions from multiple threads each cycle. Simultaneous multithreaded processors combine the multithreading technique with a wide-issue superscalar processor such that the full issue bandwidth is utilized by potentially issuing instructions from different threads simultaneously. This survey paper explains and classifies the various multithreading techniques in research and in commercial microprocessors and compares multithreaded processors with chip multiprocessors.

This paper presents an integrated compiler, runtime control, and hardware solution to improve inter-thread communication and synchronization in a multithreaded processor architecture. Multithreading improves processor utilization by exploiting more parallelism. The improvement in utilization, however, is hindered by interthread communication and synchronization problems, which incur extra communication overhead and thus degrade the performance of the system. In this paper, we propose efficient inter-thread communication and synchronization schemes based on a superscalar DLX processor with multithreading functionality. Compiler, runtime control, and hardware support used in the schemes are discussed. Simulations are presented to show the effectiveness of the proposed schemes.

This paper presents a multithreaded processor architecture for massively parallel computers, and presents RICA-1 multithreaded processor based on it. RICA-1 provides fairly efficient mechanisms of message reception, thread invocation, message generation/transmission and synchronization. It provides simple communication pipelines. Computation pipelines and communication pipelines are highly fused within a processor architecture. Parallel primitives such as remote memory access, remote procedure call and synchronizations are efficiently performed by RICA-1. The first version of RICA-1 is being implemented by a CMOS standard cell chip with 200 K random gates and 13 KB internal RAMs. It is packed into the 527 pin CPGA package and will operate with a 50 MHz clock in August 1996. In RICA-1, communication pipelines and the RISC-type execution pipelines are highly fused by a simple sequencer and three sets of register files substantially reduces the thread switch overhead.

The research about processor development was made to analyze the reasons of transition from increasing clock speed (gigahertz) to multi-threaded architecture. Main goal is to find out how the technological limits which is related to energy consumption, heat dispersion and CMOS upscaling principles, affected the CPU progression model.Key results: Assumed, that the yearly average productivity rate in single threaded mode was 11.2 percent, and in multi-threaded it was 25.2 percent. The performance parity between multi core and single core processors increased from 2.1 multiplication in 2007 to 19.6 multiplication in 2025. The major technological reasons are being limited in 3.8-4.0 gigahertz clock speed, exponential rise in power consumption, and the impossibility of further size reduction made multi core technology the only was forward for scaling.Conclusions: The trend towards multi thread processors is due to the fundamental limitations of silicon technology. The trend towards multi thread processors is due to the fundamental limitations of silicon technology. Multi-threaded processors give a significant improvement in computing capacity while keeping energy consumption in a reasonable level and accepting the requirements of modern applications. This tendency is predicted to continue into the 2030s.

Real-time execution of applications is one of key requirements for Cyber-Physical Systems (CPS) that integrate computational and physical elements for our social infrastructure, such as robotics, transportation, and consumer appliances. In such real-time systems, a task must be executed so as not to violate given time constraints. Moreover, it is desirable that the execution time of the task is predictable precisely. When Out-of-Order (OoO) execution is adopted for real-time systems to enhance the performance, it is much difficult to predict execution time because of the feature of OoO execution. In order to deal with this problem, various schemes were proposed such as IPC control mechanism of Responsive Multithreaded (RMT) Processor. RMT Processor is a real-time microprocessor adopting simultaneous multithreading (SMT) architecture with OoO execution. Its IPC control mechanism which tries to adjust the number of instruction commits to meet a given target IPC. The IPC control scheme can be implemented not only on RMT Processor but also on various processors and can improve the predictability of execution time. However, if an error between target and actual IPCs is observed, it cannot cancel the error in the next control window, which is used in the control mechanism. Since such uncorrected errors are accumulated in the successive control window, the predictability of the execution time is degraded gradually. To overcome this problem, in this paper, we propose a thread speed control scheme for real-time microprocessors. This scheme is based on the IPC control mechanism on RMT Processor. Our proposed thread speed control scheme calculates an error between reference and actual IPCs, then it dynamically updates the reference IPC of the next control window in order to cancel the past errors. Our proposed scheme is designed and implemented on RMT Processor. The simulation results show that the error is reduced to 2.60 × 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-5</sup> % in case that four threads are executed simultaneously.

Computational Fluid Dynamics (CFD) is an increasingly important application domain for computational scientists. In this paper, we propose and analyze optimizations necessary to run CFD simulations consisting of multi-billion-cell mesh models on large processor systems. Our investigation leverages the general industrial Navier-Stokes open-source CFD application, Code_Saturne, developed by Electricité de France (EDF). Our work considers emerging processor features such as many-core, Symmetric Multi-threading (SMT), Single Instruction Multiple Data (SIMD), Transactional Memory, and Thread Level Speculation. Initially, we have targeted per-node performance improvements by reconstructing the code and data layouts to optimally use multiple threads. We present a general loop transformation that will enable the compiler to generate OpenMP threads effectively with minimal impact to overall code structure. A renumbering scheme for mesh faces is proposed to enhance thread-level parallelism and generally improve data locality. Performance results on IBM Blue Gene/P supercomputer and Intel Xeon Westmere cluster are included.

Speculative multithreading is a technique that has been used to improve single thread performance. Speculative multithreading architectures for Chip multiprocessors (CMPs) have been extensively studied. But there have been relatively few studies on the design of speculative multithreading for simultaneous multithreading (SMT) processors. The current SMT based designs – IMT [9] and DMT [2] use load/store queue (LSQ) to perform dependence checking. Since the size of the LSQ is limited, this design is suitable only for small threads. In this paper we present a novel cache-based architecture support for speculative simultaneous multithreading which can efficiently handle larger threads. In our architecture, the associativity in the cache is used to buffer speculative values. Our 4-thread architecture can achieve about 15% speedup when compared to the equivalent superscalar processors and about 3% speedup on the average over the LSQ-based architectures, however, with a less complex hardware. Also our scheme can perform 14% better than the LSQ-based scheme for larger threads.

In this paper, we propose a multithreaded processor architecture which improves machine throughput. In our processor architecture, instructions from different threads (not a single thread) are issued simultaneously to multiple functional units, and these instructions can begin execution unless there are functional unit conflicts. This parallel execution scheme greatly improves the utilization of the functional unit. Simulation results show that by executing two and four threads in parallel on a nine-functional-unit processor, a 2.02 and a 3.72 times speed-up, respectively, can be achieved over a conventional RISC processor.Our architecture is also applicable to the efficient execution of a single loop. In order to control functional unit conflicts between loop iterations, we have developed a new static code scheduling technique. Another loop execution scheme, by using the multiple control flow mechanism of our architecture, makes it possible to parallelize loops which are difficult to parallelize in vector or VLIW machines.

Efficient resource sharing algorithm for physical register file in simultaneous multi-threading processors