The recent advancement of the natural language processing (NLP) models is the result of the ever-increasing model size and datasets. Most of these modern NLP models adopt the Transformer based model architecture, whose main bottleneck is exhibited in the self-attention mechanism. As the computation required for self-attention increases rapidly as the model size gets larger, self-attentions have been the main challenge for deploying NLP models. Consequently, there are several prior works which sought to address this bottleneck, but most of them suffer from significant design overheads and additional training requirements. In this work, we propose HAMMER, hardware-friendly approximate computing solution for self-attentions employing mean-redistribution and linearization, which effectively increases the performance of self-attention mechanism with low overheads. Compared to previous state-of-the-art self-attention accelerators, HAMMER improves performance by <inline-formula><tex-math notation="LaTeX">$1.2-1.6\times$</tex-math></inline-formula> and energy efficiency by <inline-formula><tex-math notation="LaTeX">$1.2-1.5\times$</tex-math></inline-formula>.

Recent CPU microarchitectural attacks utilize contention over the NoC to mount covert and side-channel attacks on multicore CPUs and leak information from victim applications. We propose NoIR, a dynamic LLC slice selection mechanism using slice remapping to obfuscate interconnect contention patterns. NoIR reduces contention variance by 92.18% and mean IPC degradation due to cache invalidation is limited to 7.38% for SPEC CPU 2017 benchmarks for a 1000-access threshold. While previous defenses focused on redesigning the NoC and routing algorithms, we show that a top-down system-level approach can significantly raise the bar for a NoC security vulnerability with minimal modifications to the NoC hardware.

Recent DNA <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pre-alignment</i> filter designs employ DRAM for storing the reference genome and its associated meta-data. However, DRAM incurs increasingly high energy consumption of background and refresh energy as devices scale. To overcome this problem, this paper explores a design with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">racetrack memory</i> (RTM)–an emerging non-volatile memory that promises higher storage density, faster access latency, and lower energy consumption. Multi-bit storage cells in RTM are inherently sequential and thus require data placement strategies to mitigate the performance and energy impacts of shifting during data accesses. We propose a near-memory pre-alignment filter with a novel data mapping and several shift reduction strategies designed explicitly for RTM. On a set of four input genomes from the 1000 Genome Project, our approach improves performance and energy efficiency by 68% and 52%, respectively, compared to the state-of-the-art DRAM-based architecture.

Data-intensive applications and throughput-oriented processors demand more memory bandwidth. Memory compression can provide more data beyond physical limits, yet new data types and smaller block sizes are challenging. This paper presents a novel and lightweight memory compression framework, Multi-Prediction Compression (MPC), to increase the effective memory bandwidth. Based on multiple prediction models and data-driven algorithm tuning, MPC can provide 31.7% better compression than state-of-the-art (SOTA) algorithms for 32B blocks. Moreover, MPC is hardware-friendly and scalable to support a growing number of data patterns.

Designing a graph processing system that can scale to graph sizes that are orders of magnitude larger than what is possible on a single accelerator requires a careful codesign of accelerator memory bandwidth and capacity, the interconnect bandwidth between accelerators, and the overall system architecture. We present a high-level bottleneck-analysis model for design and evaluation of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">scalable</i> and balanced accelerators for graph processing. We show several applications of this model including how to choose the right mix of different memory types, network topology, network bisection bandwidth, and system-level architecture to match the access patterns and capacity requirements of different data structures for a given graph and a performance target.

Sorting is an important kernel that requires many passes on data, where each pass imposes significant data movement overhead. Processing in memory (PIM) can reduce this data movement overhead while providing high parallelism. The radix sorting algorithm is scalable and can exploit PIM's parallelism. However, this algorithm is inefficient for current PIM-based accelerators for three reasons: (i) requiring a large intermediate array per processing unit, wasting capacity, (ii) requiring a prefix-sum operation across all the large intermediate arrays, imposing performance overhead, and (iii) requiring significant random accesses, which are costly in PIM. In this paper, we propose an algorithm and hardware co-optimization for sorting that enable every group of processing elements to cooperatively share and generate an intermediate array, reducing the capacity overhead of intermediate arrays and performance overhead of the prefix-sum operation. To prevent the shared array from becoming a bottleneck due to random accesses, we eliminate random accesses by adding a local sorting step to the radix sorting and providing efficient hardware support for this step. On average, our hardware/algorithm optimizations, Pulley, deliver 20× speedup compared to Bonsai, an FPGA-based sorting accelerator, and 13× speedup compared to IMC, an in-logic-layer-based sorting accelerator.

Significant innovation has been made in the development of public-key cryptography that is able to withstand quantum attacks, known as post-quantum cryptography (PQC). This paper focuses on the development of an efficient PQC hardware implementation. Specifically, an implementation of the binary Ring-learning-with-errors (BRLWE)-based encryption scheme, a promising lightweight PQC suitable for resource-constrained applications, is proposed. The paper first develops the mathematical formulation to present the proposed algorithmic process. The corresponding hardware accelerators are then described in detail. Finally, comparisons with previous implementations are provided to demonstrate the superior performance of the proposed design. For instance, the proposed low-complexity accelerator has 34.7% less area-delay product (ADP) than the state-of-the-art design for <inline-formula><tex-math notation="LaTeX">$n=256$</tex-math></inline-formula> in the field-programmable gate array (FPGA) platform. Apart from the efficiency of the hardware architectures, the proposed design also has a complete input/output processing setup, and thus is feasible for emerging lightweight applications.

Advancements in modern deep learning have shown that deeper networks with larger datasets can achieve state of the art results in many different tasks. As networks become deeper, the memory requirement of neural network training proves to be the primary bottleneck of single-machine training. In this letter, we first study the characteristics of neural network weight, gradient, feature map, gradient map, and optimizer state distributions for some popular neural network architectures. Our investigation shows that the majority of the data structures used by neural networks can have their value distributions be approximated with normal distributions. We then introduce Smart Quantization (SmaQ), a quantization scheme that exploits this observed normal distribution to quantize the data structures. Our dynamic quantization method calculates the sampled mean and standard deviation of tensors and quantizes each tensor element to 6 or 8 bits based on the z-score of that value. Our scheme reduces the memory usage during training by up to 6.7x with minor losses in accuracy.

Modern data centers run web-scale applications on tens of thousands of servers, generating tens of thousands of Remote Procedure Calls (RPCs) to backend services for each incoming user request. Tail latency, due to a small fraction of randomly slow RPCs, decreases the performance of these incoming requests, degrades users' quality of experience, and limits disaggregation (applications' ability to scale across a data center). We argue that current approaches to improve tail latency (especially, those bounding computation time) are insufficient, even with (reconfigurable-) hardware accelerators. Instead, to chop off the tail, datacenter services should dynamically trade correctness (or result quality) for timeliness, providing bounded latency with near-ideal accuracy. In this paper, we discuss how the increasing prevalence of machine learning (including search techniques like approximate nearest neighbor and PageRank), perceptual algorithms (like computational photography and image/video caching), and natural language processing lets modern hardware accelerators make these dynamic correctness tradeoffs while improving users' quality of experience.

As Moore's Law nears its end, we are searching for alternative technologies and architectures to further increase performance. Cryogenic computing has gained considerable attention in the last couple years, due to the highly ideal performance of CMOS circuits at very low temperatures. While cryogenic operation can provide impressive performance benefits, it introduces a new trade-off space which must be examined. Additionally, it does not eliminate current bottlenecks for performance, such as the memory wall. Processing-in-Memory architectures present an interesting opportunity. They are suitable to operate at cryogenic temperatures, enable lower cooling costs via extreme energy efficiency, and enable compute and memory capabilities in this regime with relatively minor adjustments to the architecture.