Web memory hierarchy learning and research environment

Among the beyond-complementary metal-oxide- semiconductor (CMOS) devices being explored, ferroelectric field-effect transistors (FeFETs) are considered as one of the most promising. FeFETs are being studied by all major semiconductor manufacturers, and experimentally, FeFETs are making rapid progress. FeFETs also stand out with the unique hysteretic Ids-Vgs characteristic that allows a device to function as both a switch and a nonvolatile (NV) storage element. We exploit this FeFET property to build two categories of fine-grained logic-in-memory (LiM) circuits: 1) ternary content addressable memory (TCAM) which integrates efficient and compact logic/processing elements into various levels of memory hierarchy; 2) basic logic function units for constructing larger and more complex LiM circuits. Two writing schemes (with and without negative supply voltages respectively) for FeFETs are introduced in our LiM designs. The resulting designs are compared with existing LiM approaches based on CMOS, magnetic tunnel junctions (MTJs), resistive random access memories (ReRAMs), ferrorelectric tunnel junctions (FTJs), etc., that afford the same circuit-level functionality. Simulation results show that FeFET-based NV TCAMs offer lower area overhead than MTJ (79%) and CMOS (42% less) equivalents, as well as better search energy-delay products (EDPs) than TCAM designs based on MTJ (149×), ReRAM (1.7×), and CMOS (1.3×) in array evaluations. NV FeFET-based LiM basic circuit blocks are also more efficient than functional equivalents based on MTJs in terms of propagation delay (4.2×) and dynamic power (2.5×). A case study for an FeFET-based LiM accumulator further demonstrates that by employing FeFET as both a switch and an NV storage element, the FeFET-based accumulator can save area (36%) and power consumption (40%) when compared with a conventional CMOS accumulator with the same structure.

This paper presents a simple dictionary structure designed for a hierarchical memory. The proposed data structure is cache oblivious and locality preserving. A cache-oblivious data structure has memory performance optimized for all levels of the memory hierarchy even though it has no memory-hierarchy-specific parameterization. A locality-preserving dictionary maintains elements of similar key values stored close together for fast access to ranges of data with consecutive keys.The data structure presented here is a simplification of the cache-oblivious B-tree of Bender, Demaine, and Farach-Colton. Like the cache-oblivious B-tree, this structure supports search operations using only O(logBN) block operations at a level of the memory hierarchy with block size B. Insertion and deletion operations use O(logBN + log2N/B) amortized block transfers. Finally, the data structure returns all k data items in a given search range using O(logBN + k/B) block operations.This data structure was implemented and its performance was evaluated on a simulated memory hierarchy. This paper presents the results of this simulation for various combinations of block and memory sizes.

Due to the increasing gap between processor speed and memory access time, a large fraction of a program's execution time is spent in accesses to the various levels in the memory hierarchy. Hence, cache-aware programming is of prime importance. For efficiently utilizing the memory subsystem, many architecture-specific characteristics must be taken into account: cache size, replacement strategy, access latency, number of memory levels, etc.In this paper, we present a simulator for the accurate performance prediction of sequential and parallel programs on shared memory systems. It assists the programmer in locating the critical parts of the code that have the greatest impact on the overall performance. Our simulator is based on the Latency-of-Data-Access Model, that focuses on the modeling of the access times to different memory levels.We describe the design of our simulator, its configuration and its usage in an example application.

The paper presents a performance model that can be used to optimally distribute computations over heterogeneous computers. This model is application-centric representing the speed of each computer by a function of the problem size. This way it takes into account the processor heterogeneity, the heterogeneity of memory structure, and the memory limitations at each level of memory hierarchy. A problem of optimal partitioning of ann-element set overpheterogeneous processors using this performance model is formulated, and its efficient solution of the complexity O(p3× log2n) is given.

SummaryOpenACC is a directive‐based programming model which allows programmers to write graphic processing unit (GPU) programs by simply annotating parallel loops. However, OpenACC has poor support for irregular nested parallel loops, which are natural choices to express nested parallelism. We propose PFACC, a programming model similar to OpenACC. PFACC directives can be used to annotate parallel loops and to guide data movement between different levels of memory hierarchy. Parallel loops can be arbitrarily nested or be placed inside functions that would be (possibly recursively) called in other parallel loops. The PFACC translator translates C programs with PFACC directives into CUDA programs by inserting runtime iteration‐sharing and memory allocation routines. The PFACC runtime iteration‐sharing routine is a two‐level mechanism. Thread blocks dynamically organize loop iterations into batches and execute the batches in a depth‐first order. Different thread blocks share iterations among one another with an iteration‐stealing mechanism. PFACC generates CUDA programs with reasonable memory usage because of the depth‐first execution order. The two‐level iteration‐sharing mechanism is implemented purely in software and fits well with the CUDA thread hierarchy. Experiments show that PFACC outperforms CUDA dynamic parallelism in terms of performance and code size on most benchmarks.

In this paper, we investigate methods for improving the hit rates in the first level of memory hierarchy. Particularly, we propose victim cache structures to reduce the number of accesses to more power consuming structures such as level 2 caches. We compare the proposed victim cache techniques to increasing the associativity or the size of the level 1 data cache and show that the enhanced victim cache technique yield better energy-delay and energy-delay-area products. We also propose techniques that predict the hit/miss behavior of the victim cache accesses and bypass the victim cache when a miss can be determined quickly. We report simulation results obtained from SimpleScalar/ARM modeling a representative Network Processor architecture. The simulations show that the victim cache is able to reduce the energy consumption by as much as 17.6% (8.6% on average) while reducing the execution time by as much as 8.4% (3.7% on average) for a set of representative applications.

Petascale computing will require coordinating the actions of 100,000+ processors, and directing the flow of data between up to six levels of memory hierarchy and along channels that differ by over a factor of 100 in bandwidth. Amdahl's law requires that petascale applications have less than 0.001% sequential or replicated work in order to be at least 50% efficient. These are profound challenges for all but the most regular or embarrassingly parallel applications, yet we also demand that not just bigger and better, but fundamentally new science. In this presentation I will discuss how we are attempting to confront simultaneously the complexities of petascale computation while increasing our scientific productivity. I hope that I can convince you that our development of MADNESS (multiresolution adaptive numerical scientific simulation) is not as crazy as it sounds.

Electronic health records (EHRs) provide a potentially valuable source of information for pharmacovigilance. However, adverse drug events (ADEs), which can be encoded in EHRs with specific diagnosis codes, are heavily under-reported. To provide more accurate estimates for drug safety surveillance, machine learning systems that are able to detect ADEs could be used to identify and suggest missing ADE-specific diagnosis codes. A fundamental consideration when building such systems is how to represent the EHR data to allow for accurate predictive modeling. In this study, two types of clinical code are used to represent drugs and diagnoses: the Anatomical Therapeutic Chemical Classification System (ATC) and the International Statistical Classification of Diseases and Health Problems (ICD). More specifically, it is investigated whether their hierarchical structure can be exploited to improve predictive performance. The use of random forests with feature sets that include only the original, low-level, codes is compared to using random forests with feature sets that contain all levels in the hierarchies. An empirical investigation using thirty datasets with different ADE targets is presented, demonstrating that the predictive performance, in terms of accuracy and area under ROC curve, can be significantly improved by exploiting codes on all levels in the hierarchies, compared to using only the low-level encoding. A further analysis is presented in which two strategies are employed for adding features level-wise according to the concept hierarchies: top-down, starting with the highest abstraction levels, and bottom-up, starting with the most specific encoding. The main finding from this subsequent analysis is that predictive performance can be kept at a high level even without employing the more specific levels in the concept hierarchies.

Modern architectures are characterized by deeper levels of memory hierarchy, often explicitly addressable. Optimizing applications for such architectures requires careful management of the data movement across all these levels. In this paper, we focus on the problem of mapping tensor contractions to memory hierarchies with more than two levels, specifically addressing placement of memory allocation and data movement statements, choice of loop fusions, and tile size selection. Existing algorithms to find an integrated solution to this problem even for two-level memory hierarchies have been shown to be expensive. We improve upon this work by focusing on the first-order cost components, simplifying the analysis required and reducing the number of candidates to be evaluated. We have evaluated our framework on a cluster of GPUs. Using five candidate tensor contraction expressions, we show that fusion at multiple levels improves performance, and our framework is effective in determining profitable transformations.

Ubiquitous embedded systems are revolutionizing our daily lives. Whole systems on a chip deliver unprecedented computation power at ever decreasing costs. However, their complexity makes their design with traditional RTL-based flows extremely challenging. Complexity in such systems arises not only from the diversity of the technologies, from RF front-ends to baseband DSP software, that must be integrated on-chip, but also from the fact that such systems must be increasingly built from parts that have been designed separately and using different tools and flows. High abstraction levels and component reuse are essential. Often such systems are highly networked and rely on sophisticated communication mechanisms. Architectural design and performance analysis of such networked system-chips is a crucial part of the embedded system design process.Two basic methods for tackling the growing complexity of system-on-chip design are emerging. Formal specification and design methods, and platform-based design. Formal methods are essential to capture precisely the designer intent at the highest possible level of abstraction, and to guide him down towards an implementation. Design errors are reduced by the use of synthesis techniques, as well as by the availability of a variety of validation techniques. Moreover, the orthogonalization of concerns (e.g., computation vs. communication, function vs. architecture) promotes extensive reuse of both parts of an application (derivative designs) and of previously used architectures.A platform in this context is the precise specification of a high-level implementation target that supports both a variety of lower level implementations, and a variety of higher-level applications. A classical example is the PC, that has supported an explosive growth of both applications and implementations. In embedded systems-on-chip platform clearly need to be application-specific. A typical platform includes one or more processors, memory, one or more bus hierarchy levels, and peripherals devoted to application-specific functions (e.g., MPEG decoding, filtering, A/D conversion, ...). Mapping to a platform involves identifying which portions of the application will be implemented on each computation and communication resource. Communication must be refined from high-level primitives (e.g., FIFOs) down to platform primitives (e.g., interrupts, memory buffers, ...). Functionality must be synthesized and scheduled based on platform-specific cost, memory and performance constraints.In this tutorial we will cover both aspects in detail, illustrating first the languages and Models of Computation available to the system-level designer to capture precisely and unambiguously the requirements. We will discuss for what application domain and platform each language is most appropriate, focusing mostly on platform-independent languages and MOCs, since they support the greatest freedom in mapping choice. We will then discuss how the architecture of the platform, and the services it offers to the application designer, can also be formally and compactly captured and specified. We will show how the mapping paradigm can be used to select an implementation for the functional blocks and their communication, and how simulation and implementation methods can be derived automatically. In particular, we will describe how software estimation and synthesis for reactive real-time systems can be competitive with hand design, while retaining the ease of re-use typical of high-level specifications.In the next part of the tutorial, we will establish the importance of on-chip communication architectures in determining the performance of System-on-Chips (SoCs). We will track several on-chip communication architectures that are in use today, including different types of bus architectures, token ring architectures, and crossbar switches. We will analyze the performance of the existing communication architectures under different classes of on-chip communication traffic. We will also describe novel on-chip communication architectures that are being developed to satisfy the growing needs of fast and concurrent on-chip communication in very high-performance systems like network processors. Next, we will describe efficient methodologies that can be used to analyze and design customized on-chip communication architectures for new application-specific platforms, as well as optimally map an application's communication requirements to the communication architecture present in an existing platform. We will also discuss reconfigurable on-chip communication architectures that allow runtime adaptation of the communication protocols to changing communication demands of the platform. Besides performance, we will analyze the energy consumed by on-chip communication architectures, and describe techniques to minimize such energy consumption. Lastly, we will analyze the impact of deep sub-micron technologies on SoC communication, and discuss the needs for development of noise-aware on-chip communication.

Current heterogeneous systems integrate generalpurpose Central Processing Units (CPUs), Graphics Processing Units (GPUs), and Neural Processing Units (NPUs). The efficient use of such systems requires a significant programming effort to distribute computation and synchronize across devices, which usually involves using Atomic Memory Operations (AMOs). Arm recently launched a floating-point Atomic Memory Operations (FAMOs) extension to perform atomic updates on floating-point data types specifically. This work characterizes and models heterogeneous architectures to understand how floating-point AMOs impact graph, Machine Learning (ML), and high-performance computing (HPC) workloads. Our analysis shows that many AMOs are performed on floating-point data, which modern systems execute using inefficient compare-and-swap (CAS) constructs. Therefore, replacing CASbased constructs with FAMOs can improve a wide range of workloads. Moreover, we analyze the trade-offs of executing FAMOs at different memory hierarchy levels, either in private caches (near) or remotely in shared caches (far). We have extended the widely used AMBA CHI protocol to evaluate such FAMO support on a simulated chiplet-based heterogeneous architecture. While near FAMOs achieve an average $1.34 \times$ speed-up, far FAMOs reach an average $1.58 \times$ speed-up. We conclude that FAMOs can bridge the gap between CPU architecture and accelerators and enabling synchronization in key application domains.

SummaryMemory is of central importance to the research of high‐level cognitive behaviors of humans, where the plastic synapse contributes to the memory formation. Due to the controllable conductance and high‐density integration, the memristor acts as a promising candidate of artificial synapse. In terms of memory retention time, memory contains short‐term memory (STM) and long‐term memory (LTM). Additionally, the STM can be changed to the LTM via repeated learning, and this process is called memory consolidation. A threshold‐type memristor combined with basic operational circuits is used to design a memory circuit that can emulate various memory hierarchy levels containing the STM, the LTM, and the STM‐to‐LTM transition. Threshold firing behavior can also be emulated because of the threshold characteristic of the memristor. Furthermore, a floating emulator circuit that consists of several off‐the‐shelf devices is developed to mimic the electrical behaviors of the threshold‐type memristor, and its hardware circuit implementation is presented. Finally, both simulation and experimental results validate the feasibility and effectiveness of the proposed memory circuit.

The need for energy efficiency continues to grow for many classes of processors, including those for which performance remains vital. Data cache is crucial for good performance, but it also represents a significant portion of the processor's energy expenditure. We describe the implementation and use of a tagless access buffer (TAB) that greatly improves data access energy efficiency while slightly improving performance. The compiler recognizes memory reference patterns within loops and allocates these references to a TAB. This combined hardware/software approach reduces energy usage by (1) replacing many level-one data cache (L1D) accesses with accesses to the smaller, more power-efficient TAB; (2) removing the need to perform tag checks or data translation lookaside buffer (DTLB) lookups for TAB accesses; and (3) reducing DTLB lookups when transferring data between the L1D and the TAB. Accesses to the TAB occur earlier in the pipeline, and data lines are prefetched from lower memory levels, which result in a small performance improvement. In addition, we can avoid many unnecessary block transfers between other memory hierarchy levels by characterizing how data in the TAB are used. With a combined size equal to that of a conventional 32-entry register file, a four-entry TAB eliminates 40% of L1D accesses and 42% of DTLB accesses, on average. This configuration reduces data-access related energy by 35% while simultane-ously decreasing execution time by 3%.

Transactional Memory (TM) is an emerging technology which simplifies the concurrency control in a parallel program. In this paper we propose Quick TM, a new hardware transactional memory (HTM) architecture. It incorporates three features to address known bottlenecks in the existing HTM architectures. First, we propose hardware-only dynamic detection of true-shared variables. Our result shows that true-shared variables account for only about 20% in the commit set of any transaction. Rest can be completely disregarded from the commit phase. This shortens every commit phase drastically resulting in a significant overall speed-up. Second, we keep both the speculative and the last committed version local to each processor. This benefits when a transaction is repeated in a loop. The processor request gets satisfied from the L1 data cache(L1D) itself. Furthermore, since both the versions are locally maintained, the commit action involves only broadcast of addresses. Third, we have proposed a mechanism to address overflow in transactions. In our proposal, each processor continues to run transactions even if one processor has overflown its L1D. Our technique eliminates the stall of a thread even if it conflicts with the overflown transaction. Overflown transaction commits in-place and periodically broadcasts its write set addresses, termed “partial commit”. This gradually reduces conflicts and allows other threads to progress towards commit. Moreover, the technique does not require any additional hardware at any memory hierarchy level beyond L1. Quick TM outperforms the state-of-the-art scalable HTM architecture, Scalable-TCC, on average by 20% in the latest TM benchmark suite STAMP. It outperforms the original TCC proposal with serialized commit by 28% on average. Maximum speed-up achieved in these two cases are 43% and 67% respectively. Our proposal handles transaction overflow gracefully and outperforms the current overflow-aware HTM proposal, One TM-concurrent by 12% on average.

Matrix computations are both fundamental and ubiquitous in computational science and its vast application areas. Along with the development of more advanced computer systems with complex memory hierarchies, there is a continuing demand for new algorithms and library software that efficiently utilize and adapt to new architecture features. This article reviews and details some of the recent advances made by applying the paradigm of recursion to dense matrix computations on today's memory-tiered computer systems. Recursion allows for efficient utilization of a memory hierarchy and generalizes existing fixed blocking by introducing automatic variable blocking that has the potential of matching every level of a deep memory hierarchy. Novel recursive blocked algorithms offer new ways to compute factorizations such as Cholesky and QR and to solve matrix equations. In fact, the whole gamut of existing dense linear algebra factorization is beginning to be reexamined in view of the recursive paradigm. Use of recursion has led to using new hybrid data structures and optimized superscalar kernels. The results we survey include new algorithms and library software implementations for level 3 kernels, matrix factorizations, and the solution of general systems of linear equations and several common matrix equations. The software implementations we survey are robust and show impressive performance on today's high performance computing systems.