Highly Concurrent Latency-tolerant Register Files for GPUs

Register files are a key data storage unit that impacts instruction throughput for graphics processing units (GPUs). Typically, GPU register files are quite large to accommodate many concurrent threads and are implemented using the same SRAM technology as the on-chip cache. We propose contextrf, a new register file architecture that efficiently leverages register files with nonuniform access characteristics, including hybrid SRAM/DRAM (S/D) and spintronic domain-wall memories (DWMs). Contextrf allows greater-capacity register files to be implemented in the same area within the GPU, with reduced power consumption. We also propose contextPreRF, a hardware preswitch scheme to hide switching delays—as soon as a register request is queued, the nonuniform access memories containing the corresponding register are sent a preemptive switch request. Thus, our scheme transparently hides the penalties of switching between register contexts. After replacing the register file SRAM with S/D, we can reduce energy by 37%, with a 1.4% average performance drop. Employing DWM, we reduce register file energy by 74%, with a 0.4% average performance penalty. For the denser DWM, we model converting the saved area into additional registers, cache, and shared memory—this improves performance by 13.5% over the baseline SRAM register file.

High power consumption and slow access of enlarged and multiported register files make it difficult to design high performance superscalar processors. The clustered architecture, where the conventional monolithic register file is partitioned into several smaller register files, is expect to overcome the register file issues. In the clustered architecture, the more a monolithic register file is partitioned, the lower power and faster access register files can be realized. However, the partitioning causes losses of IPC (instructions per clock cycle) due to communication among register files. Therefore, degree of partitioning has a strong impact on the trade-off between power consumption and performance. In addition, the organization of partitioned register files also affects the trade-off. In this paper, we attempt to investigate appropriate degrees of partitioning and organizations of partitioned register files in a clustered architecture to assess the trade-off. From the results of execute-driven simulation, we find that the organization of register files and the degree of partitioning have a strong impact on the IPC, and the configuration with non-consistent register files can make use of the partitioned resources more effectively. From the results of register file access time and energy modeling, we find that the configurations with the highly partitioned non-consistent register file organization can receive benefit of the partitioning in terms of operating frequency and access energy of register files. Further, we examine relationship between IPS (instructions per second) and the product of IPC and operating frequency of register files. The results suggest that highly partitioned non-consistent configurations tends to gain more advantage in performance and power.

The increasing adoption of graphics processing units (GPUs) for high-performance computing raises the reliability challenge, which is generally ignored in traditional GPUs. GPUs usually support thousands of parallel threads and require a sizable register file. Such large register file is highly susceptible to soft errors and power-hungry. Although ECC has been adopted to register file in modern GPUs, it causes considerable power overhead, which further increases the power stress. Thus, an energy-efficient soft-error protection mechanism is more desirable. Besides its extremely low leakage power consumption, resistive memory (e.g., spin-transfer torque RAM) is also immune to the radiation induced soft errors due to its magnetic field based storage. In this article, we propose to LEverage reSistive memory to enhance the Soft-error robustness and reduce the power consumption (LESS) of registers in the General-Purpose computing on GPUs (GPGPUs). Since resistive memory experiences longer write latency compared to SRAM, we explore the unique characteristics of GPGPU applications to obtain the win-win gains: achieving the near-full soft-error protection for the register file, and meanwhile substantially reducing the energy consumption with negligible performance degradation. Our experimental results show that LESS is able to mitigate the registers soft-error vulnerability by 86% and achieve 61% energy savings with negligible (e.g., 1%) performance degradation.

In modern microprocessors, the access time of register file becomes a critical part in total delay. Instruction level or thread level parallelism improves instructions per cycle (IPC) by executing multiple instructions in one cycle. Such multiple instructions need to read or write data from/to register files simultaneously. To satisfy that, register file with sufficient ports should be designed. However, the area and access time of register file with large ports will increase sharply. Duplicated register file (DupRF) architecture can reduce access time by distributing read ports. In this paper, we propose a new kind of DupRF architecture for embedded simultaneous multithreading (SMT) microprocessor and estimate the effect with respect to the area and access time. Especially, we measure the product of area and access time as computation cost. For a SMT microprocessor with 6 threads, 64-bit data-width and 6 function units, 3-duplicate register file architecture can reduce access time by 12.61% with a slight increase of computation cost by 3.35% compared with the central register file architecture

To achieve high compute performance, graphics processing units (GPUs) provide a large register file and a large number of execution units. However, these design components consume a large portion of the total dynamic power in the GPU, particularly for general purpose applications. In this paper, we present a low-cost gating scheme to reduce dynamic power consumption in the register file and execution units without impacting performance. The scheme proposed dynamically exploit frequent found data value of zeros within and across registers in order to gate off register file reads and writes as well as execution units. We find that on general purpose applications from Rodinia, our low-cost gating scheme can reduce register file reads and writes on average by 35% and 40%, respectively. The register file and execution unit dynamic power are reduced on average by 19% and 13%, respectively. The reduction in total GPU dynamic power achieved is ranging from 3% to 19% with 8% on average with no performance loss.

Graphics processing units(GPUs) have been increasingly used to accelerate general purpose computations. By exploiting massive thread-level parallelism (TLP), GPUs can achieve high throughput as well as memory latency hiding. As a result, a very large register file (RF) is typically required to enable fast and low-cost context switching between tens of thousands of active threads. However, RF resource is still insufficient to enable all thread level parallelism and the lack of RF resources can hurt performance by limiting the occupancy of GPU threads. Moreover, if the available RF capacity can not fit the requirement of a thread block, GPU needs to fetch some variables from local memory which may lead to long memory access latencies. By observing that a large percentage of computed results actually have fewer significant bits compared to the full width of a 32-bit register for many GPGPU applications, we propose a GPU register packing scheme to dynamically exploit narrowwidth operands and pack multiple operands into a single fullwidth register. By using dynamically register packing, more RF space is available which allows GPU to enable more TLP through assigning additional thread blocks on SMs (Streaming Multiprocessors) and thus improve performance. The experimental results show that our GPU register packing scheme can achieve up to 1.96X speedup and 1.18X on average.

In response to the remarkable increase in 3D applications in consumer electronics devices in recent years, graphics processing units (GPUs) have become widely available on mobile devices. These GPUs typically use hardware multithreaded shaders to improve their throughputs for real-time rendering, but they depend on duplicate register files to maintain the context of each hardware thread, increasing power consumption. However, the register usage of shading programs is often relatively low, which causes many registers to remain unused, thus wasting power. Long latency memory operations can also consume unnecessary power to activate registers. This study proposes a low-power register file with multiple power modes to reduce the power consumption of the register file. This study also presents an adaptive thread scheduling mechanism to achieve a tradeoff between the power consumption of the register file and frames per second (FPS). Results show that the average performance degradation from the proposed low-power register file is only 0.62%. The proposed adaptive thread scheduling has average under prediction ratio of 3.32%. The leakage reduction of the proposed low-power register file is 74.80%. This reduction can be improved to 81.49%, 82.22%, and 84.28% with adaptive thread scheduling at frame rates of 30, 25, and 20, respectively.

To facilitate efficient context switches, GPUs usually employ a large-capacity register file to accommodate a massive amount of context information. However, the large register file introduces high power consumption, flowing to high leakage power SRAM cells. Emerging non-volatile STT-RAM memory has recently been studied as a potential replacement to alleviate the leakage challenge when constructing register files on GPUs. Unfortunately, due to the long write latency and high energy consumption associated with write operations in STT-RAM, simply replacing SRAM with STTRAM for register files would incur non-trivial performance overhead and only bring marginal energy benefits.

As embedded processors being widely used in specific application domains, such as communications, multimedia, and networking, the register file has contributed a substantial budget in embedded processor energy consumption due to its long working time for the data intensive computations and the large switching capacitance. It is found that 25% of registers can account for 83% of register file accessing time during many embedded application execution. This fact motivates us to reduce the register file power consumption by partitioning the registers to different regions according to their usage pattern. The most frequently used registers are put in the hot part, and the cold part of register file is rarely accessed. We employ the register file bitline splitting and the drowsy register cell techniques in our design to reduce the overall accessing power of the register file. We propose a novel approach to partition the register file in a way so that the largest power saving can be achieved. We formulate the register file partitioning process into a graph partitioning problem, and apply an effective algorithm to obtain the optimal result. We evaluate our algorithm on MiBench applications, and an average saving of 43.6% in the register file access power consumption over the original non-partitioned register file is achieved for the SimpleScalar PISA system.

A low-level software-based fault tolerance approach to detect SEUs in GPUs' register files

Many difficulties have confronted the system-on-chip (SoC) industry during the previous ten years. These difficulties will likely become more significant as more and more applications revolve on the Internet of Everything (IoE). These issues include reducing power consumption for greater energy efficiency, overcoming numerous causes of variation to ensure reliable performance, and reducing design area to save money and boost integration. As a result, chip designers encounter various hurdles when attempting to create stable structures with complicated angles of capability while preserving a small die size and power expenditure. Memory components are among the most critical circuit components on every chip. They account for the majority of the chip's size and power consumption, which has an impact on the overall efficacy and reliability of the chip. These consist of a diverse multitude of serial elements in logic paths, caches, register files, and large memory arrays. In contemporary synchronized CMOS circuits, sequence elements are indispensable components. In fact, they can be responsible for up to 50% of the average number of cells in a semiconductor. We propose a novel methodology aimed at enhancing the reliability of pulsed latches while ensuring that there is no substantial decline in efficiency, area, or power consumption. Furthermore, given that sequential elements can be utilized in creating compact register files, the implementation of pulsed latches in register files is reviewed and compared with other conventional implementations, including static random-access memory (SRAM) and flip-flops. Also shown are new implementations of multiport register files, which are highly beneficial for many applications. The suggested approach has been proven to considerably decrease the enormous excess in area, power consumption, and latency that is typically associated with conventional methods of designing multiport register files

The register file access time is one of the critical delays in current superscalar processors. Its impact on processor performance is likely to increase in future processor generations, as they are expected to increase the issue width (which implies more register ports) and the size of the instruction window (which implies more registers), and to use some kind of multithreading. Under this scenario, the register file access time could be a dominant delay and a pipelined implementation would be desirable to allow for high clock rates. However, a multi-stage register file has severe implications for processor performance (e.g. higher branch misprediction penalty) and complexity (more levels of bypass logic). To tackle these two problems, in this paper we propose a register file architecture composed of multiple banks. In particular we focus on a multi-level organization of the register file, which provides low latency and simple bypass logic. We propose several caching policies and prefetching strategies and demonstrate the potential of this multiple-banked organization. For instance, we show that a two-level organization degrades IPC by 10% and 2% with respect to a non-pipelined single-banked register file, for SpecInt95 and SpecFP95 respectively, but it increases performance by 87% and 92% when the register file access time is factored in.

The increasing number of complex jobs scheduled to execute on embedded systems has increased the importance of fast response times in job scheduling and task switching on embedded processors. This paper addresses the issue of reducing context-switching overhead. We present a novel register file architecture, the paged register file (pRF), that comprises two novel mechanisms for reducing context-switching latency related to the hardware context: the valid-annotated register file and semishadowing. The valid-annotated register file contains valid bits to annotate the used registers, which can be used to reduce register accesses in context switching. Moreover, the utilization of the register file is increased by semishadowing, which allows two threads to share the same register file. Experimental results obtained on the XEEMU platform with the benchmarks, DSPstone, show that support for our proposed pRF design reduces the frequency of context-switching by around 24% and reduces the amount of register movements by 46.9%.

Very Long Instruction Word (VLIW) processors are very popular in embedded and mobile computing domain. Use of VLIW processors range from Digital Signal Processors (DSPs) found in a plethora of communication and multimedia devices to Graphics Processing Units (GPUs) used in gaming and high performance computing devices. The advantage of VLIWs is their low complexity and low power design which enable high performance at a low cost. Scalability of VLIWs is limited by the scalability of register file ports. It is not viable to have a VLIW processor with a single large register file because of area and power consumption implications of the register file. Clustered VLIW solve the register file scalability issue by partitioning the register file into multiple clusters and a set of functional units that are attached to register file of that cluster. Using a clustered approach, higher issue width can be achieved while keeping the cost of register file within reasonable limits. Several commercial VLIW processors have been designed using the clustered VLIW model. VLIW processors can be used to run a larger set of applications. Many of these applications have a good Lnstruction Level Parallelism (ILP) which can be efficiently utilized. However, several applications, specially the ones that are control code dominated do not exibit good ILP and the processor is underutilized. Cache misses is another major source of resource underutiliztion. Multithreading is a popular technique to improve processor utilization. Interleaved MultiThreading (IMT) hides cache miss latencies by scheduling a different thread each cycle but cannot hide unused instructions slots. Simultaneous MultiThread (SMT) can also remove ILP under-utilization by issuing multiple threads to fill the empty instruction slots. However, SMT has a higher implementation cost than IMT. The thesis presents Cluster-level Simultaneous MultiThreading (CSMT) that supports a limited form of SMT where VLIW instructions from different threads are merged at a cluster-level granularity. This lowers the hardware implementation cost to a level comparable to the cheap IMT technique. The more complex SMT combines VLIW instructions at the individual operation-level granularity which is quite expensive especially in for a mobile solution. We refer to SMT at operation-level as OpSMT to reduce ambiguity. While previous studies restricted OpSMT on a VLIW to 2 threads, CSMT has a better scalability and upto 8 threads can be supported at a reasonable cost. The thesis proposes several other techniques to further improve CSMT performance. In particular, Cluster renaming remaps the clusters used by instructions of different threads to reduce resource conflicts. Cluster renaming is quite effective in reducing the issue-slots under-utilization and significantly improves CSMT performance.The thesis also proposes: a hybrid between IMT and CSMT which increases the number of supported threads, heterogeneous instruction merging where some instructions are combined using SMT and CSMT rest, and finally, split-issue, a technique that allows to launch partially an instruction making it easier to be combined with others.

Modern General Purpose Graphic Processing Unit (GPGPU) demands a large Register File (RF), which is typically organized into multiple banks to support the massive parallelism. Although heavy banking benefits RF throughput, its associated area and energy costs with diminishing performance gains greatly limit future RF s-caling. In this paper, we propose an improved RF design with a bank stealing technique, which enables a high RF throughput with compact area. By deeply investigating the GPGPU microarchitecture, we identify the deficiency in the state-of-the-art RF designs as the bank conflict problem, while the majority of conflicts can be eliminated leveraging the fact that the highly-banked RF oftentimes experiences under-utilization. This is especially true in GPGPU where multiple ready warps are available at the scheduling stage with their operands to be wisely coordinated. Our lightweight bank stealing technique can opportunistically fill the idle banks for better operand service, and the average GPGPU performance can be improved under smaller energy budget with significant area saving, which makes it promising for sustainable RF scaling.