A simultaneous multithreading heterogeneous object recognition processor with machine learning based dynamic resource management

This paper presents the first analysis of operating system execution on a simultaneous multithreaded (SMT) processor. While SMT has been studied extensively over the past 6 years, previous research has focused entirely on user-mode execution. However, many of the applications most amenable to multithreading technologies spend a significant fraction of their time in kernel code. A full understanding of the behavior of such workloads therefore requires execution and measurement of the operating system, as well as the application itself.To carry out this study, we (1) modified the Digital Unix 4.0d operating system to run on an SMT CPU, and (2) integrated our SMT Alpha instruction set simulator into the SimOS simulator to provide an execution environment. For an OS-intensive workload, we ran the multithreaded Apache Web server on an 8-context SMT. We compared Apache's user- and kernel-mode behavior to a standard multiprogrammed SPECInt workload, and compared the SMT processor to an out-of-order superscalar running both workloads. Overall, our results demonstrate the microarchitectural impact of an OS-intensive workload on an SMT processor and provide insight into the OS demands of the Apache Web server. The synergy between the SMT processor and Web and OS software produced a greater throughput gain over superscalar execution than seen on any previously examined workloads, including commercial databases and explicitly parallel programs.

Speculative multithreading is a technique that has been used to improve single thread performance. Speculative multithreading architectures for Chip multiprocessors (CMPs) have been extensively studied. But there have been relatively few studies on the design of speculative multithreading for simultaneous multithreading (SMT) processors. The current SMT based designs – IMT [9] and DMT [2] use load/store queue (LSQ) to perform dependence checking. Since the size of the LSQ is limited, this design is suitable only for small threads. In this paper we present a novel cache-based architecture support for speculative simultaneous multithreading which can efficiently handle larger threads. In our architecture, the associativity in the cache is used to buffer speculative values. Our 4-thread architecture can achieve about 15% speedup when compared to the equivalent superscalar processors and about 3% speedup on the average over the LSQ-based architectures, however, with a less complex hardware. Also our scheme can perform 14% better than the LSQ-based scheme for larger threads.

Simultaneous Multithreading (SMT) processors achieve high processor throughput at the expense of single-thread performance. This paper investigates resource allocation policies for SMT processors that preserve, as much as possible, the single-thread performance of designated threads, while still permitting other threads to share resources. Since background threads on such an SMT machine have a near-zero performance impact on foreground threads, we refer to the background threads as transparent threads. Transparent threads are ideal for performing low-priority or non-critical computations, with applications in process scheduling, subordinate multithreading, and on-line performance monitoring.To realize transparent threads, we propose three mechanisms for maintaining the transparency of background threads: slot prioritization, background thread instruction-window partitioning, and background thread flushing. In addition, we propose three mechanisms to boost background thread performance without sacrificing transparency: aggressive fetch partitioning, foreground thread instruction-window partitioning, and foreground threadflushing. We implement our mechanisms on a detailed simulator of an SMT processor, and evaluate them using 8 benchmarks, including 7 from the SPEC CPU2000 suite. Our results show when cache and branch predictor interference are factored out, background threads introduce less than 1% performance degradation on the foreground thread. Furthermore, maintaining the transparency of background threads reduces their throughput by only 23% relative to an equal priority scheme.To demonstrate the usefulness of transparent threads, we study Transparent Software Prefetching (TSP), an implementation of software data prefetching using transparent threads. Due to its near-zero overhead, TSP enables prefetch instrumentation for all loads in a program, eliminating the need for profiling. TSP, without any profile information, achieves a 9.52% gain across 6 SPEC benchmarks, whereas conventional software prefetching guided by cache-miss profiles increases performance by only 2.47%.

This paper proposes a cycle accounting architecture for Simultaneous Multithreading (SMT) processors that estimates the execution times for each of the threads had they been executed alone, while they are running simultaneously on the SMT processor. This is done by accounting each cycle to either a base, miss event or waiting cycle component during multi-threaded execution. Single-threaded alone execution time is then estimated as the sum of the base and miss event components; the waiting cycle component represents the lost cycle count due to SMT execution. The cycle accounting architecture incurs reasonable hardware cost (around 1KB of storage) and estimates single-threaded performance with average prediction errors around 7.2% for two-program workloads and 11.7% for four-program workloads. The cycle accounting architecture has several important applications to system software and its interaction with SMT hardware. For one, the estimated single-thread alone execution time provides an accurate picture to system software of the actually consumed processor cycles per thread. The alone execution time instead of the total execution time (timeslice) may make system software scheduling policies more effective. Second, a new class of thread-progress aware SMT fetch policies based on per-thread progress indicators enable system software level priorities to be enforced at the hardware level.

Symbiotic job scheduling improves simultaneous multithreading (SMT) processor performance by coscheduling jobs that have “compatible” demands on the processor's shared resources. Existing approaches however require a sampling phase, evaluate a limited number of possible coschedules, use heuristics to gauge symbiosis, are rigid in their optimization target, and do not preserve system-level priorities/shares. This article proposes probabilistic job symbiosis modeling, which predicts whether jobs will create positive or negative symbiosis when coscheduled without requiring the coschedule to be evaluated. The model, which uses per-thread cycle stacks computed through a previously proposed cycle accounting architecture, is simple enough to be used in system software. Probabilistic job symbiosis modeling provides six key innovations over prior work in symbiotic job scheduling: (i) it does not require a sampling phase, (ii) it readjusts the job coschedule continuously, (iii) it evaluates a large number of possible coschedules at very low overhead, (iv) it is not driven by heuristics, (v) it can optimize a performance target of interest (e.g., system throughput or job turnaround time), and (vi) it preserves system-level priorities/shares. These innovations make symbiotic job scheduling both practical and effective. Our experimental evaluation, which assumes a realistic scenario in which jobs come and go, reports an average 16% (and up to 35%) reduction in job turnaround time compared to the previously proposed SOS (sample, optimize, symbios) approach for a two-thread SMT processor, and an average 19% (and up to 45%) reduction in job turnaround time for a four-thread SMT processor.

Efficient resource sharing algorithm for physical register file in simultaneous multi-threading processors

An integrated autonomous control mechanism to optimize sharing of multiple resources in Simultaneous Multi-Threading processors

Today's general-purpose processors are increasingly using multithreading in order to better leverage the additional on-chip real estate available with each technology generation. Simultaneous multi-threading (SMT) was originally proposed as a large dynamic superscalar processor with monolithic hardware structures shared among all threads. Inters hyper-threaded Pentium 4 processor partitions the queue structures among two threads, demonstrating more balanced performance by reducing the hoarding of structures by a single thread. IBM's Power5 processor is a 2-way chip multiprocessor (CMP) of SMT processors, each supporting 2 threads, which significantly reduces design complexity and can improve power efficiency. This paper examines processor partitioning options for larger numbers of threads on a chip. While growing transistor budgets permit four and eight-thread processors to be designed, design complexity, power dissipation, and wire scaling limitations create significant barriers to their actual realization. We explore the design choices of sharing, or of partitioning and distributing, the front end (instruction cache, instruction fetch, and dispatch), the execution units and associated state, as well as the L1 Dcache banks, in a clustered multi-threaded (CMT) processor. We show that the best performance is obtained by restricting the sharing of the L1 Dcache banks and the execution engines among threads. On the other hand, significant sharing of the front-end resources is the best approach. When compared against large monolithic SMT processors, a CMT processor provides very competitive IPC performance on average, 90-96% of that of partitioned SMT while being more scalable and much more power efficient. In a CMP organization, the gap between SMT and CMT processors shrinks further, making a CMP of CMT processors a highly viable alternative for the future

DLL-conscious instruction fetch optimization for SMT processors

Summary form only given. Speculative executions help modern processors to expose independent instructions on the fly and accordingly exploit more instruction-level parallelism. However, when incorrect speculations occur, useless work is performed for those incorrectly speculated instructions. This lowers a sustained performance and leads to a significant waste of power. Unlike superscalar processors, simultaneous multithreading (SMT) processors can concurrently execute multiple threads. Thus, they have a chance to control speculative executions by deliberately choosing threads from which instructions will be fetched at each cycle, considering the dynamic characteristics of running threads. We present an efficient front-end mechanism, called SAFE-T (speculation-aware front-end throttling), for scheduling threads in SMT processors. It involves thread prioritizing and throttling; priority given to a thread can be overridden when that thread seems to suffer from an excessive amount of incorrect speculations, therefore preventing instructions from being fetched. Simulation results show that our policy provides an average reduction of 41.6% in the number of wrong-path instructions and improves the instruction throughput by up to 14.5%. A cost-effective implementation for the proposed policy is shown as well.

This paper presents Threaded Multi-Path Execution (TME), which exploits existing hardware on a Simultaneous Multi-threading (SMT) processor to speculatively execute multiple paths of execution. When there are fewer threads in an SMT processor than hardware contexts, threaded multi-path execution uses spare contexts to fetch and execute code along the less likely path of hard-to-predict branches.This paper describes the hardware mechanisms needed to enable an SMT processor to efficiently spawn speculative threads for threaded multi-path execution. The Mapping Synchronization Bus is described, which enables the spawning of these multiple paths. Policies are examined for deciding which branches to fork, and for managing competition between primary and alternate path threads for critical resources. Our results show that TME increases the single program performance of an SMT with eight thread contexts by 14%-23% on average, depending on the misprediction penalty for programs with a high misprediction rate.

Currently,fetch policies in Simultaneous Multithreading(SMT)processors almost focus on overall performance optimization,and provide no control over how individual threads are executed.A novel fetch policy called CPIT(Controlling Performance of Individual Thread)is proposed to control the execution of a particular thread in SMT processors.Results show that for more than 94% of all cases measured,CPIT can control the execution and consequently achieve the desired performance for a given thread.For the failing cases,the average variance is within 1.25%.Furthermore,CPIT does not sacrifice overall performance of SMT processors severely.Compared to fetch policies orienting towards performance maximization such as ICOUNT,the average degradation of overall performance is not more than 3% and the degradation of threads other than the given thread in performance is only 1.75%.

Dynamic Resource Management (DRM) allows dynamically changing the resources assigned to a job during its execution. This flexibility can improve productivity in completed jobs per unit of time, resource utilization rate, and energy consumption, among other metrics. Despite its benefits, DRM remains complex to implement, which is one of the reasons it has not yet been widely adopted in production HPC systems. To address this challenge, we design, implement, and evaluate a methodology aimed at improving the programmability of DRM for iterative algorithm-based applications, while maintaining a generic and flexible foundation. Specifically, we interface the Dynamic Management of Resources API (DMR-API), an application-level abstraction layer that simplifies the adoption of dynamic resources—particularly in classical iteration-based HPC applications—with the Dynamic Processes with PSets (DPP) approach, which provides a set of generic design principles for dynamic resource management in high-performance parallel programming models. This integration makes DRM more accessible for iteration-based HPC applications through the DMR-API, while preserving the generality and flexibility of DPP at the system software level to support a wider range of HPC application types. Our results show that DRM can be effectively leveraged in HPC environments with minimal coding effort, unlocking the benefits of dynamic resource allocation for job throughput and system utilization.

Distributed middleware technology is emerging as a flexible promising solution for network resource management and QoS (quality of service) control in a distributed environment. The concept of dynamic bandwidth reservation (DBR) algorithm additionally allocates the terminated bandwidth for an ongoing call is one of the promising methods to reduce the new call blocking and handoff call dropping probabilities. Dynamic resource management (DRM) framework concentrates on design and developing a framework for allocating shared resources to support multiple QoS and to provide probabilistic assurances of service in wireless networks. Existing methods and architecture do not guarantee on resource allocation and management in mobile, wireless and ad hoc networks. When a mobile node (MN) moves from its home network (HN) to foreign network (FN), the minimal required resources and services can be provided locally in the current network where it is connected presently. Dynamic resource management (DRM) framework provides an application level simulator for wireless network with new QoS management concept. DRM framework comprises of a set of programmable distributed objects and distributed middleware. DRM framework is constructed using distributed objects and distributed middleware. Distributed object carry applications across networks and allow them to connect to local server with FN in the same way as they had moved from the HN. The framework must dynamically accommodate multimedia calls with different bandwidth requirements in response to network traffic and it supports mobility management. The utility of DRM framework describes how the required resources are reserved based on the user priority option in an adaptive manner. The proposed DBR algorithm guarantees the required bandwidth during call setup and handoff without any flow rejection.

Teaching resources in higher education are an important part of the talent cultivation system of higher education institutions. Dynamic management and optimal allocation of teaching resources are the focus of the construction of each school, which reflects the teaching, research, and management level of the school to a certain extent. In this paper, after combining the current situation and management problems of teaching resources in six schools in City A and putting forward constructive opinions, we further use data envelopment analysis to study the efficiency of teaching resource allocation in School H in City A and put forward the method of resource optimization.The input redundancy rate of School X1 in School H during the three years of 2021-2023 is 30.25%, 44.15%, and 55.26%, respectively. The data shows that the school has an excess of teacher inputs and there is a shortage of resources for teachers. For the current dynamic management and resource allocation problems of teaching resources in A city school, this paper argues that the management of school teaching resources is a long-term and complex organic whole, and needs to be reformed in terms of structure, resource integration and management, and to construct a high-quality and efficient management model. In terms of resource optimization and allocation, schools can optimize the talent resource allocation mechanism and other aspects to promote the full allocation of teaching resources in higher education.