A prefetch-aware memory system for data access patterns in multimedia applications

The DRAM-based main memory performance is lagging far behind the performance of modern processors. Achieving the highest possible performance of memory-bound applications on modern processors has been increasingly limited by the average DRAM memory access latency. The employed page policy, also referred to as a row-buffer management policy, has a significant impact on DRAM performance. Modern DRAM-based main memory systems have mainly two-page policies; Closed Page Autoprecharge (CPA) policy and Open Page (OP) policy. The optimal page policy is an application-dependent, hence a static page policy utilization may not always be the perfect choice to provide the lowest possible average DRAM memory access latency for the memory requests. In this paper, a dynamic page policy is proposed, in which the decision to switch between the OP policy and the CPA policy is made at run-time for each memory request through monitoring the memory access characteristics of the application instructions. The proposed approach aims at improving the DRAM bandwidth utilization and reducing the average DRAM memory access latency by lowering the number of time-consuming memory operations. The results obtained from this study revealed significant average Last Level Cache (LLC) miss penalty reduction of 25.8% when utilizing effectively the proposed approach compared to the baseline in the evaluated workloads.

To achieve higher memory bandwidth in network-based multiprocessor architectures, multiple dynamic random access memories can be accessed simultaneously. In such architectures, not only resource utilization and latency are the critical issues but also a reordering mechanism is required to deliver the response transactions of concurrent memory accesses in-order. In this paper, we present a memory-efficient on-chip network architecture to cope with these issues efficiently. Each node of the network is equipped with a novel network interface (NI) to deal with out-of-order delivery, and a priority-based router to decrease the network latency. The proposed NI exploits a streamlined reordering mechanism to handle the in-order delivery and utilizes the advance extensible interface transaction-based protocol to maintain compatibility with existing intellectual property cores. To improve the memory utilization and reduce the memory latency, an optimized memory controller is integrated in the presented NI. Experimental results with synthetic test cases demonstrate that the proposed on-chip network architecture provides significant improvements in average network latency (16%), average memory access latency (19%), and average memory utilization (22%).

Using multiple SDRAMs in MPSoCs and NoCs to increase memory parallelism is very common nowadays. In-order delivery, resource utilization, and latency are the most critical issues in such architectures. In this paper, we present a novel network interface architecture to cope with these issues efficiently. The proposed network interface exploits a resourceful reordering mechanism to handle the in-order delivery and to increase the resource utilization. A brilliant memory controller is efficiently integrated into this network interface to improve the memory utilization and reduce both memory and network latencies. In addition, to bring compatibility with existing IP cores the proposed network interface utilizes AXI transaction based protocol. Experimental results with synthetic test cases demonstrate that the proposed architecture gives significant improvements in average network latency (12%), average memory access latency (19%), and average memory utilization (22%).

Big data applications demand a better memory performance. Data Locality has been the focus of reducing data access delay. Data access concurrency, however, has become prevalent in modern memory systems in recent years. How to extend existing locality-based performance optimization to consider data concurrency becomes a timely issue facing the researchers and practitioners in the field of computing, especially in the field of big data computing. In this study, we introduce the concept and definition of Concurrency-aware data access Locality (CaL), which, as its name states, extends the concept of locality by considering concurrency. Compared to the conventional concept of locality, CaL accurately reflects the combined impact of data access locality and concurrency in modern memory systems and is very effective for data intensive applications. The value of CaL can be quantitatively measured directly by performance counters in mainstream commercial processors and is practically feasible. Two theoretical results are presented to reveal the relationships between CaL and existing memory system performance metrics of memory accesses per cycle (APC), average memory access time (AMAT), and memory bandwidth (B). In this way, we provide a methodology to use existing locality-based optimization methods directly or in combination with data concurrency optimizations, to improve the value of CaL and to improve the performance of a memory system. To demonstrate the practical value of CaL, we conduct four case studies to illustrate the power of concurrency-aware locality optimization. Compared with the conventional locality based optimization, the CaL-aware design has achieved significant performance improvement. It achieved a 3.12-fold speedup on K-means, which is a widely-used data analytic kernel from the big data benchmarks.

In this paper we present a high performance and low power media cache structure using a dynamic fetching mechanism. The proposed cache consists of three parts, i.e., a dual direct mapped cache, a fully associative spatial buffer, and a dynamic fetch unit. When a cache miss occurs, the dynamic fetch controller generates fetch signals for one of three block sizes (e.g., 64-byte, 128-byte, or 192-byte) depending on information that is kept on recent block access patterns. Simulation results show that the proposed cache can achieve better performance than a 2-way or 4-way set associative cache with twice as much space. Also, compared with a victim cache, the average memory access time is improved by about 15% on media applications. It is also shown that power consumption of the proposed cache is around 60% lower than other cache systems.

On-chip distributed memory system has become an attractive solution for massive parallel memory accesses found in future many-core processors. However, increasing number of on-chip cores and memory controllers inevitably introduce many remote memory accesses, which generate a large amount of on-chip traffic and put great pressure on the interconnection. This paper tries to optimize on-chip memory access traffic via runtime thread migration. We first analyze memory access behaviors in multi-threaded applications and find that the memory access targets and volumes are similar during short periods, which makes runtime prediction feasible. But the memory access targets exhibit great mobility during long periods, motivating us to dynamically move threads towards the data. Based on these observations, we propose a novel low-cost and distributed thread migration algorithm which adjusts thread placement in chains based on benefit estimation. We present details of the workflow, including the trigger and arbitration of migration requests and the procedures to determine the migration chains. Simulation results show that our algorithm achieves system performance speedup of 11.5 % and reduces average memory access latency by 11.0 %. It can find a few but effective thread migrations to optimize on-chip memory access traffic with acceptable hardware and runtime overheads.

Cache ways are fixed in traditional caches.This paper proposes an algorithm to have variable number of ways in set associative caches.The cache is assumed to be fast registers.A free register to a set is allocated on cold miss.The least recently used policy is used to replace a way in case of no free registers.This algorithm results in variable number of ways for mapped sets.Simulations were performed with SPEC2K benchmarks on the proposed model.An improvement of 3% is seen in average memory access time.

Traditional memory performance metrics, such as average memory access time (AMAT), are designed for sequential data accesses and can prove misleading for contemporary cache technologies that increasingly rely on access concurrency. C-AMAT, a new performance metric, accounts for concurrency at both the component and system levels for modern memory design.

One of the main challenges for computer architects is how to hide the high average memory access latency from the processor. In this context, Hybrid Memory Cubes (HMCs) can provide substantial energy and bandwidth improvements compared to traditional memory organizations. However, it is not clear how this reduced average memory access latency will impact the LLC. For applications with high cache miss ratios, the latency to search for the data inside the cache memory will impact negatively on the performance. The importance of this overhead depends on the memory access latency. In this paper, we present an evaluation of the L3 cache importance on a high performance processor using HMC also exploring chip area tradeoffs between the cache size and number of processor cores. We show that the high bandwidth provided by HMC memories can eliminate the need for L3 caches, removing hardware and making room for more processing power. Our evaluations show that performance increased 37% and the EDP improved 12% while maintaining the same original chip area in a wide range of parallel applications, when compared to DDR3 memories.

Data Grid deals with the movement of massive amount of data among storage elements. One of the challenges in Data Grids is how to reduce the average data access time by replicating data. In this article, a method for dynamic data replication called RBAAC (Replica Based Average Access Cost) is proposed which, removes files with least cost of average access time when encountering insufficient space for replication. Simulation results show that the proposed method, improved the average access time to files and network bandwidth use.

This paper proposes a combined frame memory architecture which is smaller in size and is potential in reducing power consumption compared to the most commonly used ping-pong frame memory. The combined frame memory maps both reference frame data and current frame data onto one single frame memory instead of two in ping-pong architecture. Together with the characteristic of high percentage of MB with zero-valued MV and no residual, the combined frame memory architecture is evaluated to be able to reduce not only the memory size, but also the average energy consumption and memory access latency for applications like surveillance, video telephony, and videoconferencing. According to the statistics and analysis results the proposed combined frame memory architecture memory size is only 57% compared to ping-pong architecture. The proposed combined frame memory architecture can reduce up to 83% of average latency and 39% of average power consumption compared to ping-pong frame memory architecture.

DRAM latency is a major bottleneck for many applications in modern computing systems. In this work, we rigorously characterize the effects of reducing DRAM access latency on 282 state-of-the-art LPDDR4 DRAM modules. As found in prior work on older DRAM generations (DDR3), we show that regions of LPDDR4 DRAM modules can be accessed with latencies that are significantly lower than manufacturer-specified values without causing failures. We present novel data that 1) further supports the viability of such latency reduction mechanisms and 2) exposes a variety of new cases in which access latencies can be effectively reduced. Using our observations, we propose a new low-cost mechanism, Solar-DRAM, that 1) identifies failure-prone regions of DRAM at reduced latency and 2) robustly reduces average DRAM access latency while maintaining data correctness, by issuing DRAM requests with reduced access latencies to non-failure-prone DRAM regions. We evaluate Solar-DRAM on a wide variety of multi-core workloads and show that for 4-core homogeneous workloads, Solar-DRAM provides an average (maximum) system performance improvement of 4.31% (10.87%) compared to using the default fixed DRAM access latency.

The efficiency of modern microprocessors is extremely sensitive towards the structure and memory access pattern of programs to be executed. This is caused by memory hierarchies which were introduced to reduce average memory access times. In this paper, we consider embedded Runge-Kutta (RK) methods for the solution of ordinary differential equations arising from space discretization problems for partial differential equations and study their efficient implementation on modern microprocessors. Different program variants with different execution orders and storage schemes are investigated. In particular, we explore how the potential parallelism in the stage vector computation can be exploited in a pipelining approach in order to improve the locality behavior of the RK implementations. Experiments show that this results in efficiency improvements on several recent processors.

In modern computer architectures, access latency varies considerably between different levels in the memory hierarchy. Consequently, applications with data access patterns that don't reuse much data in fast levels of the hierarchy incur additional delays. To improve the performance of complex, data-intensive applications, developers need tools that help them understand the causes of poor memory hierarchy utilization. While most performance tools associate metrics with functions or statements, in this paper we explore data-centric analyses that associate metrics not only with data accesses but also with data objects themselves. Our contributions are three-fold. First, we propose several refinements to existing data-centric techniques that enable accurate and low-overhead measurements. Second, we combine data-centric analysis with call path profiling; this combination of techniques relates inefficient access patterns back to data objects across complete dynamic call chains. Third, we developed a graphical user interface that gracefully presents our analysis results using a multiplicity of views, which helps users identify problematic accesses and data structures. We demonstrate the utility of our approach by showing how our tool identifies problematic data access patterns in several HPC applications and a pair of the SPEC CPU2006 benchmarks.

: Future CMPs will have more cores and greater onchip cache capacity. The on-chip cache can either be divided into separate private L2 caches for each core, or treated as a large shared L2 cache. Private caches provide low hit latency but low capacity, while shared caches have higher hit latencies but greater capacity. Victim replication was previously introduced as a way of reducing the average hit latency of a shared cache by allowing a processor to make a replica of a primary cache victim in its local slice of the global L2 cache. Although victim replication performs well on multithreaded and single-threaded codes, it performs worse than the private scheme for multiprogrammed workloads where there is little sharing between the different programs running at the same time. In this paper, we propose victim migration, which improves on victim replication by adding an additional set of migration tags on each node which are used to implement an exclusive cache policy for replicas. When a replica has been created on a remote node, it is not also cached on the home node, but only recorded in the migration tags. This frees up space on the home node to store shared global lines or replicas for the local processor. We show that victim migration performs better than private, shared, and victim replication schemes across a range of single threaded, multithreaded, and multiprogrammed workloads, while using less area than a private cache design. Victim migration provides a reduction in average memory access latency of up to 10% over victim replication.