Optimizing power efficiency for 3D stacked GPU-in-memory architecture

Near-Data Processing (NDP) modifies the traditional computer system design by placing logic near the memory, bringing computation to the data. One NDP approach places such elements on the logic layer of 3D-stacked memories to quickly access data while avoiding reliance on narrow buses and better accessing the parallelism these devices offer. However, NDP architectures often fail to fully leverage available memory resources. In this work, we propose adding an instruction buffer to a common NDP design with large vector instructions. This modification allows the NDP to fetch instruction operands out of program order and delegates some responsibility regarding precise exceptions to the near-data device. Our results show our modifications cause a reduction in execution time of up to 28% while consuming up to 25% less energy.

As computation becomes increasingly limited by data movement and energy consumption, exploiting locality throughout the memory hierarchy becomes critical to continued performance scaling. Moving computation closer to memory presents an opportunity to reduce both energy and data movement overheads. We explore the use of 3D die stacking to move memory-intensive computations closer to memory. This approach to processing in memory addresses some drawbacks of prior research on in-memory computing and is commercially viable in the foreseeable future.Because 3D stacking provides increased bandwidth, we study throughput-oriented computing using programmable GPU compute units across a broad range of benchmarks, including graph and HPC applications. We also introduce a methodology for rapid design space exploration by analytically predicting performance and energy of in-memory processors based on metrics obtained from execution on today's GPU hardware. Our results show that, on average, viable PIM configurations show moderate performance losses (27%) in return for significant energy efficiency improvements (76\% reduction in EDP) relative to a representative mainstream GPU at 22nm technology. At 16nm technology, on average, viable PIM configurations are performance competitive with a representative mainstream GPU (7% speedup) and provide even greater energy efficiency improvements (85\% reduction in EDP).

Memory-bound sparse gathering, caused by irregular random memory accesses, has become an obstacle in several on-demand applications such as embedding lookup in recommendation systems. To reduce the amount of data movement, and thereby better utilize memory bandwidth, previous studies have proposed near-data processing (NDP) solutions. The issue of prior work, however, is that they either minimize data movement effectively at the cost of limited memory parallelism or try to improve memory parallelism (up to a certain degree) but cannot successfully decrease data movement, as prior proposals rely on spatial locality (an optimistic assumption) to utilize NDP. More importantly, neither approach proposes a solution for gathering data from random memory addresses; rather they just offload operations to NDP. We propose an effective solution for sparse gathering, an efficient near-memory intelligent reduction (Fafnir) tree, the leaves of which are all the ranks in a memory system, and the nodes gradually apply reduction operations while data is gathered from any rank. By using such an overall tree, Fafnir does not rely on spatial locality; therefore, it minimizes data movement by performing entire operations at NDP and fully benefits from parallel memory accesses in parallel processing at NDP. Further, Fafnir offers other advantages such as using fewer connections (because of the tree topology), eliminating redundant memory accesses without using costly and less effective caching mechanisms, and being applicable to other domains of sparse problems such as scientific computing and graph analytics. To evaluate Fafnir, we implement it on an XCVU9P Xilinx FPGA and in 7 nm ASAP ASIC. Fafnir looks up the embedding tables up to 21.3× more quickly than the state-of-the-art NDP proposal. Furthermore, the generic architecture of Fafnir allows running classic sparse problems using the same 1.2 mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> hardware up to 4.6× more quickly than the state of the art.

Genome Assembly is an important Big Data analytics which involves massive computations for similarity searches on sequence databases. Being major component of runtime, similarity searches require careful design for scalable performance. MinHash Sketching is an extensively used data structure in Long-read genome assembly pipelines, which involves generating, randomizing and minimizing a set of hashes for all the k-mers in genome sequences. Compute-hungry MinHash sketch processing on commercially available multi-threaded CPUs suffer from the limited bandwidth of the L1-cache, which causes the CPUs to stall. Near-Data Processing (NDP) is an emerging trend in data-bound Big Data analytics to harness the low-latency, highbandwidth available within the Dual In-line Memory Modules (DIMMs). While NDP architectures have generally been utilized for memory-bound computations, MinHash sketching is a potential application that can gain massive throughput by exploiting memory Banks as higher bandwidth L1-cache.In this work, we propose MSIM, a distributed, highly parallel and efficient hardware-software co-design for accelerating MinHash Sketch processing on light-weight components placed on the DRAM hierarchy. Multiple ASIC-based Processing Engines (PEs) placed at the bank-group-level in MSIM provide highparallelism for low-latency computations. The PEs sequentially access data from all Banks within their bank-group with the help of a dedicated Address calculator, which utilizes an optimal data mapping scheme. The PEs are controlled by a custom Arbiter, which is directly activated by the host CPU using general DDR commands, without requiring any modification to the memory controller or the DIMM standard buses. MSIM requires limited area and power overheads, while displaying up-to 384.9x speedup and 1088.4x energy reduction compared to the baseline multithreaded software solution in our experiments. MSIM achieves 4.26x speedup over high-end GPU, while consuming 26.4x lesser energy. Moreover, MSIM design is highly scalable and extendable in nature.

As applications become more data-intensive, issues like von Neumann's bottleneck and the memory wall became more apparent since data movement is the main source of inefficiency in computer systems. Looking to mitigate this issue, Near-Data Processing (NDP) moves computation from the processor to the memory, thus reducing the data movement required by many data-intensive workloads. In this paper, we look to database query operators, common targets of NDP research as database systems often need to deal with large amounts of data. We investigate the migration of most time-consuming database operators to Vector-In-Memory Architecture (VIMA), a novel 3D-stacked memory-based NDP architecture. We consider the selection, projection, and bloom join database query operators, commonly used by data analytics applications, comparing VIMA to a high-performance x86 baseline. Our results show speedups of up to 8× for selection, 6× for projection, and 16× for join while consuming up to 99% less energy. To the best of our knowledge, these results outperform the state-of-the-art for these operators on NDP platforms.

Most general-purpose distributed storage systems are not designed with near data processing (NDP) in mind. They do not respect semantic data boundaries when writing data, for example splitting a record across servers. This reduces NDP effectiveness by requiring data collation before computation. While semantic data awareness and NDP functions can be retroactively added to existing distributed storage, it is often complex and difficult to accomplish in practice. We propose sharing storage system layout information with data writers so they can adjust data layouts to prevent data alignment issues regardless of the underlying architectures. By doing so, we can simplify NDP implementation by reducing the need for data reassembly, and reduce the need for complex storage system or application extensions. We demonstrate a hinting mechanism on both HDFS with computational block storage and an erasure coded MinIO deployment, reducing data movement by up to 99% when querying CSV data with NDP co-located with the stored data. This was accomplished purely with client side data alignment, no modifications to the server side write paths, and no inter-node collation of data.

Practical Mechanisms for Reducing Processor–Memory Data Movement in Modern Workloads

This paper presents a 2:1 low power switched-capacitor DC---DC converter designed in $$180$$180 nm Standard CMOS technology. The converter operates from $$1.8$$1.8 V input and delivers a $$0.8$$0.8 V power supply with $$50$$50 mA load current. Simplicity and power efficiency improvement is the main goal of this design. The achieved efficiency of the converter is more than $$80\,\%$$80%. Also, high power efficiency, high current delivery, small size, low output voltage ripples are some main properties of the proposed DC---DC converters. The converter occupies about an area of $$1$$1 mm$$^2$$2. Besides, switching frequency of the switched-capacitor DC-DC converter has been increased to reduce output voltage ripples. Maximum output voltage ripple is about $$21$$21 mV. Power saving in gate driver stage is the method used in this design to improve the power efficiency. Also, a little changes applied in non-overlapping clock signal generator to gain a better power efficiency. During the design procedure, the main power loss sources have been explained and their solutions have been presented.

The performance and scalability of modern data-intensive systems processing large datasets are limited by unnecessary data movement. Even though near-data processing (NDP) can provably reduce data transfers and increase performance, at present, NDP is utilized primarily in read-only settings. Near-data execution of data-intensive modification operations is currently infeasible due to the lack of transactional consistency and the absence of practicable low-latency synchronization mechanisms between the host database engine and the NDP-engine on smart storage. In this paper, we introduce Update NDP as an approach to offloading modifications to computational storage with transactional guarantees in an NDP database system called neoDBMS . To ensure consistency, we introduce a low-latency shared lock table between the host and computational storage, based on novel cache-coherent interconnects . We also introduce a novel locking protocol that seamlessly integrates the shared lock table within the lock manager of the host NDP-engine. To handle failure recovery, while preserving high and robust performance, we introduce novel extended locking and logging mechanisms that allow the host and computational storage to perform useful work during log-movement. Our evaluation indicates that in-storage modifications in neoDBMS in mixed workload settings are ≥ 6.52 × faster than host-only executions and exhibit robust performance due to lower data movement and better resource utilization.

In recent years, a system using renewable energy has attracted attention, and power converters with high power efficiency and low cost are required. Dual Active Bridge (DAB) DC- DC converters are attracting attention because of their high power-efficiency due to Zero Voltage Switching (ZVS) operation over a wide load range. However, when the voltage input/output ratio is different from the turns ratio of the transformer, there is a disadvantage that the ZVS operation cannot be achieved at light loads, and the power efficiency drops. Therefore, this paper proposes a digital control method that can improve power efficiency at light loads regardless of the voltage input/output conversion ratio. In the experiment, it was confirmed that the power efficiency at light loads could be improved by about 10% when the input/output ratio is 200V / 150V, 150V / 100V, and 125V / 75 V.

This paper presents a new full-swing BiNMOS gate with improved power efficiency, superior power consumption with supply voltages down to 1.2V and significantly reduced layout area. The new circuit was compared with bootstrapped-based BiCMOS logic styles, and the standard CMOS logic style. Low-power methodology was used for the optimization of all gates. Tests have been performed at various power supply voltages (0.9V-3V), with various output load capacitances (0.1pF-1pF), at 50MHz frequency and 27°C temperature. At 1.2V and output loads 0.1pF to 0.81pF, the new circuit has up to 2.8 times better power efficiency than the best bootstrapped BiCMOS style reported and consumes 20%-49% less switching power. Under optimal conditions (V dd = 1.6V), the new design has up to 37% higher power efficiency than conventional CMOS logic for loads 0.4pF to 1pF and up to 112% better power efficiency than the best reported BiCMOS style for output loads 0.1pF-0.95pF. Test circuits have been designed and fabricated with 0.8 µm BiCMOS technology with V thn = 0.8 V and V thp = -0.9 V.

Improving power efficiency (PE) and reducing roll-off are of significant importance for the commercialization of white organic light-emitting diodes (WOLEDs) in consideration of energy conservation. Herein, record-beating PE of 130.7 lm W−1 and outstanding external quantum efficiency (EQE) of 31.1% are achieved in all-fluorescence two-color WOLEDs based on a simple sandwich configuration of emitting layer consisting of sky-blue and orange delayed fluorescence materials. By introducing a red fluorescence dopant, all-fluorescence three-color WOLEDs with high color rendering index are constructed based on an interlayer sensitization configuration, furnishing ultrahigh PE of 110.7 lm W−1 and EQE of 30.8%. More importantly, both two-color and three-color WOLEDs maintain excellent PEs at operating luminance with smaller roll-offs than the reported state-of-the-art WOLEDs, and further device optimization realizes outstanding comprehensive performances of low driving voltages, large luminance, high PEs and long operational lifetimes. The underlying mechanisms of the impressive device performances are elucidated by host-tuning effect and electron-trapping effect, providing useful guidance for the development of energy-conserving all-fluorescence WOLEDs.

Data access costs dominate the execution times of most parallel applications and they are expected to be even more important in the future. To address this, recent research has focused on Near Data Processing (NDP) as a new paradigm that tries to bring computation to data, instead of bringing data to computation (which is the norm in conventional computing). This paper explores the potential of compiler support in exploiting NDP in the context of emerging many core systems. To that end, we propose a novel compiler algorithm that partitions the computations in a given loop nest into sub computations and schedules the resulting sub computations on different cores with the goal of reducing the distance-to-data on the on-chip network. An important characteristic of our approach is that it exploits NDP while taking advantage of data locality. Our experiments with 12 multi-threaded applications running on a state of-the-art commercial many core system indicate that the proposed compiler-based approach significantly reduces data movements on the on-chip network by taking advantage of NDP, and these benefits lead to an average execution time improvement of 18.4%. CCS CONCEPTS •Computer systems organization → Multi-core architectures;

Recently, the bi-directional dc-dc converter has been focused on because of the huge demand for diversification of power supply network including battery. The dual active bridge (DAB) dc-dc converter is one of the most popular circuits for bi-directional applications because of its simple structure. However, power efficiency at light load is the intrinsic problem of a bi-directional DAB DC-DC converter. In this paper, the simple solution with digital operation for the problem is proposed and experiments are performed with 1kW system. This method can reduce a switching surge without other circuits such as snubber and improve power efficiency at light load. Therefore it can reduce loss of switching surge, and improve power efficiency. From the results, 37% maximum power efficiency improvement at light load is confirmed. Furthermore, this method is capable for control in the conventional method in the heavy load range. Consequently, it is possible to reduce the switching surge and realize high power efficiency in a wide load range.

Approximate nearest neighbor search (ANNS) in high-dimensional vector spaces has become increasingly crucial in database and machine learning applications. Most previous ANNS algorithms require TB-scale memory to store indices of billion-scale datasets, making their deployment extremely expensive for high-performance ANNS services. The emerging SmartSSD technology offers an opportunity to achieve scalable ANNS via near data processing (NDP). However, there remain several challenges to directly adopt existing ANNS algorithms on multiple SmartSSDs. In this paper, we present SmartANNS, a SmartSSD-empowered billion-scale ANNS solution based on a hierarchical indexing methodology. We propose several novel designs to achieve near-linear scaling with multiple SmartSSDs. First, we propose a “host CPUs + SmartSSDs” cooperative architecture incorporated with hierarchical indices to significantly reduce data accesses and computations on SmartSSDs. Second, we propose dynamic task scheduling based on optimized data layout to achieve both load balancing and data reusing for multiple SmartSSDs. Third, we further propose a learning-based shard pruning algorithm to eliminate unnecessary computations on SmartSSDs. We implement SmartANNS using Samsung’s commercial SmartSSDs. Experimental results show that SmartANNS can improve query per second (QPS) by up to 10.7 × compared with the state-of-the-art SmartSSD-based ANNS solution–CSDANNS. Moreover, SmartANNS can achieve near-linear performance scalability for large-scale datasets using multiple SmartSSDs.