Linebacker

Hierarchical cache-aided transmission cooperation in 5G user-centric network: Performance analysis and design insights

Long latency of memory operation is a prominent performance bottleneck in graphics processing units (GPUs). The small data cache that must be shared across dozens of warps (a collection of threads) creates significant cache contention and premature data eviction. Prior works have recognized this problem and proposed warp throttling which reduces the number of active warps contending for cache space. In this paper we discover that individual load instructions in a warp exhibit four different types of data locality behavior: (1) data brought by a warp load instruction is used only once, which is classified as streaming data (2) data brought by a warp load is reused multiple times within the same warp, called intra-warp locality (3) data brought by a warp is reused multiple times but across different warps, called inter-warp locality (4) and some data exhibit both a mix of intra- and inter-warp locality. Furthermore, each load instruction exhibits consistently the same locality type across all warps within a GPU kernel. Based on this discovery we argue that cache management must be done using per-load locality type information, rather than applying warp-wide cache management policies. We propose Access Pattern-aware Cache Management (APCM), which dynamically detects the locality type of each load instruction by monitoring the accesses from one exemplary warp. APCM then uses the detected locality type to selectively apply cache bypassing and cache pinning of data based on load locality characterization. Using an extensive set of simulations we show that APCM improves performance of GPUs by 34% for cache sensitive applications while saving 27% of energy consumption over baseline GPU.

Recent studies have shown that moderate to high data redundancy exists in primary storage systems, such as VM-based, enterprise and HPC storage systems, which indicates that the data deduplication technology can be used to effectively reduce the write traffic and storage space in such environments. However, our experimental studies reveal that applying data deduplication to primary storage systems will cause space contention in main memory and data fragmentation on disks. This is in part because applying data deduplication introduces significant index memory overhead to the existing system and in part because a file or block is split into multiple small data chunks that are often located in non-sequential locations on disks after deduplication. This fragmentation of data can cause a subsequent read operation to invoke many disk I/O requests, thus leading to performance degradation. The existing primary data deduplication schemes, such as iDedup[1], are to leverage spatial locality in that they only select the large requests to deduplicate and exclude the small requests (e.g., 4KB, 8KB or less) because the latter only account for a tiny fraction of the storage capacity requirement[2]. Moreover, these schemes tend to overlook the importance of cache management, leading them to manage the index cache and the read cache separately. However, previous workload studies on primary storage systems have revealed that small I/O requests dominate in the primary storage systems (more than 50%) and are at the root of the system performance bottleneck. Furthermore, the accesses in primary storage systems exhibit obvious I/O burstiness. The existing primary-storage data deduplication schemes fail to consider these workload characteristics in primary storage systems from the performance's perspective. We argue that, primary-storage data deduplication schemes should take the workload characteristics of primary storage into the design considerations. To address the two problems and take the primary-storage workload characteristics into considerations, we propose a Performance-Oriented I/O Deduplication approach, POD, to improving the I/O performance of primary storage systems in the Cloud. POD takes a two-pronged deduplication approach to improve primary storage systems, a request-based I/O and data deduplication scheme, called Select-Dedupe, aimed at alleviating data fragmentation and an adaptive memory management scheme, called iCache, to ease the main memory contention. More specifically, the former takes the workload characteristics of small-I/O-request domination into the design considerations. It deduplicates all the write requests if their write data is already stored sequentially on disks, including the small write requests that would otherwise be excluded from by the capacity-oriented deduplication schemes. For other write requests, Select-Dedupe does not deduplicate their redundant write data to maintain the performance of the subsequent read requests to these data. iCache takes the I/O burstiness characteristics into the design considerations. It dynamically adjusts the cache space between the index cache and the read cache according to the workload characteristics, and swaps these data between memory and backend storage devices accordingly. During the write-intensive bursty periods, iCache enlarges the index cache size and shrinks the read cache size to detect much more redundant write requests, thus improving the write performance. The read cache size is enlarged to cache more hot read data to improve the read performance during the read-intensive bursty periods. The prototype of the POD scheme is implemented as an embedded module at the block-device level with the fixed-size chunking method. Preliminary evaluations driven by the real traces conducted on our lightweight POD prototype implementation show that POD significantly outperforms iDedup in improving the performance of primary storage systems in the Cloud.

In recent years, with the rapid development of virtual reality (VR), 360-degree panoramic video has grown up to become a popular Internet service. Streaming 360-degree video on today's Internet is challenging due to its ultra-high bandwidth and ultra-low motion-to-photon latency requirements. Although some VR solutions use tiling technology and Scalable Video Coding(SVC) encoding for 360-degree video transmission to reduce bandwidth consumption, it is still being challenging to transmit 360-degree video streams from remote content servers due to network latency. Caching popular video content at the edge of the network can decrease network latency and bandwidth consumption by reducing the number of future requests that have to be sent all the way to remote content servers. Because of the limited cache size, it is essential to decide which content to cache and how to replace the cache when the cache size is insufficient. In this paper, we propose an edge cache replacement strategy named Size-Popularity-Layer-FoV strategy(SPLF) for tile-based 360-degree video streaming using SVC. To improve caching performance, every video chunks' cache value is estimated according to its popularity, size, SVC layer and whether it is in FoV. Simulation results manifest that the proposed strategy can not only improve the hit rate and video quality compared to various baseline algorithms, but also decrease the playback stalls and the duration of rebuffering.

Developing FPGA implementations with an input specification in a high-level programming language such as C/C++ or OpenCL allows for a substantially shortened design cycle compared to a design entry at register transfer level. This work targets high-level synthesis (HLS) implementations that process large amounts of data and therefore require access to an off-chip memory. We leverage the customizability of the FPGA on-chip memory to automatically construct a multi-cache architecture in order to enhance the performance of the interface between parallel functional units of the HLS core and an external memory. Our focus is on automatic cache sizing. Firstly, our technique determines and uses up unused left-over block RAM resources for the construction of on-chip caches. Secondly, we devise a high-level cache performance estimation based on the memory access trace of the program. We use this memory trace to find a heterogeneous configuration of cache sizes, tailored to the application's memory access characteristic, that maximizes the performance of the multi-cache system subject to an on-chip memory resource constraint. We evaluate our technique with three benchmark implementations on an FPGA board and obtain a reduction in execution latency of up to 2× (1.5× on average) when compared to a one-size-fits-all cache sizing. We also quantify the impact of our automatically generated cache system on the overall energy consumption of the implementation.

Rapid growth in network services and vast usage of Internet worldwide have led to an increase in network traffic and created bottleneck over the internet. Such traffic results in an increase of access time of web documents, server response latency, reduced network bandwidth and slow response time for popular websites. Web cache is an essential optimization technique used to reduce response time and improve performance. However due to its limited size and cost of cache comparable to other storage devices, cache replacement algorithm is used to determine and evict page when the cache is full to create space for new pages. Several algorithms had been introduced and their performances are important in producing high speed web caching. However, their performances are not well optimized. This work proposes a hybrid method that optimize cache replacement algorithm using Naive Bayes (NB) based approach. Naive Bayes is an intelligent method that depends on Bayes’ probability theory integrated with Adaptive Weight Ranking Policy (AWRP) via dynamic aging factor to improve the response time and network performance. From the pseudocode of the proposed algorithm, it is observed that the complexity of the algorithm is O(n)which is linear, hence the response time is considergood. Performance evaluations based on hit rate and byte hit rate for new the method over conventional methods with real data will be conducted for validation and verification.

Caching systems using the Least Recently Used (LRU) principle have now become ubiquitous. A fundamental question for these systems is whether the cache space should be pooled together or divided to serve multiple flows of data item requests in order to minimize the miss probabilities. In this paper, we show that there is no straight yes or no answer to this question, and depends on complex combinations of critical factors, including, e.g., request rates, overlapped data items across different request flows, data item popularities and their sizes. To this end, we characterize the performance of multiple flows of data item requests under resource pooling and separation when the cache size is large. Analytically we show that it is asymptotically optimal to jointly serve multiple flows if their data item sizes and popularity distributions are similar, and their arrival rates do not differ significantly; the self-organizing property of LRU caching automatically optimizes the resource allocation among them asymptotically. Otherwise, separating these flows could be better, e.g., when data sizes vary significantly. We also quantify critical points beyond which resource pooling is better than separation for each of the flows when the overlapped data items exceed certain levels. These results provide new insights on the performance of caching systems.

Caching systems using the Least Recently Used (LRU) principle have now become ubiquitous. A fundamental question for these systems is whether the cache space should be pooled together or divided to serve multiple flows of data item requests in order to minimize the miss probabilities. In this paper, we show that there is no straight yes or no answer to this question, and depends on complex combinations of critical factors, including, e.g., request rates, overlapped data items across different request flows, data item popularities and their sizes. To this end, we characterize the performance of multiple flows of data item requests under resource pooling and separation when the cache size is large. Analytically we show that it is asymptotically optimal to jointly serve multiple flows if their data item sizes and popularity distributions are similar, and their arrival rates do not differ significantly; the self-organizing property of LRU caching automatically optimizes the resource allocation among them asymptotically. Otherwise, separating these flows could be better, e.g., when data sizes vary significantly. We also quantify critical points beyond which resource pooling is better than separation for each of the flows when the overlapped data items exceed certain levels. These results provide new insights on the performance of caching systems.

Caching systems using the Least Recently Used (LRU) principle have now become ubiquitous. A fundamental question for these systems is whether the cache space should be pooled together or divided to serve multiple flows of data item requests in order to minimize the miss probabilities. In this paper, we show that there is no straight yes or no answer to this question, depending on complex combinations of critical factors, including, e.g., request rates, overlapped data items across different request flows, data item popularities and their sizes. To this end, we characterize the performance of multiple flows of data item requests under resource pooling and separation for LRU caching when the cache size is large. Analytically, we show that it is asymptotically optimal to jointly serve multiple flows if their data item sizes and popularity distributions are similar and their arrival rates do not differ significantly; the self-organizing property of LRU caching automatically optimizes the resource allocation among them asymptotically. Otherwise, separating these flows could be better, e.g., when data sizes vary significantly. We also quantify critical points beyond which resource pooling is better than separation for each of the flows when the overlapped data items exceed certain levels. Technically, for a broad class of heavy-tailed distributions we derive the asymptotic miss probabilities of multiple flows of requests with varying data item sizes in a shared LRU cache space. It also validates the characteristic time approximation under certain conditions. These results provide new insights on improving the performance of caching systems.

Immersive video creates a unique experience by enabling viewers to freely control the viewing direction and has been regarded as a killer application of 5G. Huge bandwidth requirement and delay-sensitive are two major challenges faced by immersive video transmissions. Edge caching is believed as a promising scheme to solve these challenges. This paper proposes a cache algorithm specialized for immersive video service. First, we explore the relationship between quantity of video versions and users' watching quality by actual users' immersive video watching traces through which we have found it is different for different video frames. Based on this observation, we propose a user behavior aware immersive video caching algorithm to jointly optimize the video version design and cache space allocation based on users' watching behavior history. Numerical experiment driven by actual users' watching traces shows that our scheme is superior to baseline scheme under different cache sizes. Specifically, it can improve the average MO by at least 2.57%, and the improvement percentage can even reach 36.14% under some specific conditions.

Shared last level cache has been widely used in modern multicore processors. However, uncontrolled cache sharing on multicore leads to more serious cache pollution than that on single-core processor. A process with weak locality can evict strong locality data sets that belong to other concurrent ones. Processes in multiprocessing environment always affect each other on multicore systems with shared last level cache. Prior approaches either partition shared cache in process level to reduce inter-process cache contention, or isolate the non-temporal memory accesses in order to accelerate single application execution. Process-based cache partitioning may make intra-process cache pollution more serious and have great impact on single process performance. In this work, we take an alternative view to explore physical page layout optimization by combining process-based cache partitioning and pollute region isolation for improving the shared last level cache utilization on multicore systems. Our proposed approach includes three steps. The first step determines the cache sizes of co-scheduled applications and the second step recognizes weak-locality regions of each application on different cache size configurations. Lastly, the third step customizes the physical page layout to partition cache space among concurrent processes and set up global pollute buffer for mapping pollute regions into a small slice of shared last level cache. Our approach is directly used in commercial multicore systems without any additional hardware requirement. Our experimental results show that in comparison with default Linux memory management scheme, our approach improves performance by 26.73% on average. Even compared to the process-based cache partitioning RapidMRC, our approach further eliminates the harmful effect of non-reusable data, and system performance is also improved by 5.63% on average.

The formalism underlying EASYMAP: A precompiler for refinement-based exploration of hierarchical data organizations

SummaryTo boost input‐output performance, operating systems employ a kernel‐managed caching space called the buffer cache or page cache. Given the limited size of a buffer cache, an effective cache manager is required to decide which blocks should be evicted from the cache. Previous cache managers use historical information to make replacement decisions. However, existing approaches are unable to maximize performance since they rely on limited historical information. Motivated by the limitations of existing solutions, this paper proposes a novel manager called the Pattern‐assisted Adaptive Recency Caching (PARC) manager. PARC simultaneously uses the historical information of recency, frequency, and access patterns to estimate the locality strengths of blocks and, upon a cache miss, evicts the block with the least strength. Specifically, PARC exploits the reference regularities exhibited in past input‐output behaviors to actively and rapidly adapt the recency and frequency information of blocks so as to precisely distinguish blocks with long‐ and short‐term utility. Through comprehensive simulations on a variety of traces of different access patterns, we show that PARC is robust since, except for random workloads where the performance of each cache manager is similar, PARC always outperforms existing solutions.

As the popularity of WebVR applications rises in recent years, a great disparity exists between the huge 3D content on servers and the limited cache capacity on clients. In order to alleviate server loading, peer-to-peer (P2P)-based 3D content distribution has been proposed recently. However, given the limited cache sizes at clients, how to maintain and replace the cache content effectively becomes an important issue. We present a progressive scene replacement mechanism (PSRM) in this paper to support interactive walkthrough in P2P-based large-scale WebVR worlds. First, we define a new metric called preservation degree, based on both the visual attention paid by the user and the content's potential relevance for sharing. Second, cached objects are replaced progressively with ascending order in terms of their preservation degrees. Experimental results have shown that PSRM enables users with limited cache to walkthrough a large-scale WebVR world with high visual quality; while download requests handled by the servers are reduced significantly.

Modern CPUs operate at GHz frequencies, but the latencies of memory accesses are still relatively large, in the order of hundreds of cycles. Deeper cache hierarchies with larger cache sizes can mask these latencies for codes with good data locality and reuse, such as structured dense matrix computations. However, cache hierarchies do not necessarily benefit sparse scientific computing codes, which tend to have limited data locality and reuse. We therefore propose a new memory architecture with a load miss predictor (LMP), which includes a data bypass cache and a predictor table, to reduce access latencies by determining whether a load should bypass the main cache hierarchy and issue an early load to main memory. Our architecture uses the L2 (and lower caches) as a victim cache for data removed from our bypass cache. We use cycle-accurate simulations, with SimpleScalar and Wattch to show that our LMP improves the performance of sparse codes, our application domain of interest, on average by 14%, with a 13.6% increase in power. When the LMP is used with dynamic voltage and frequency scaling (DVFS), performance can be improved by 8.7% with system power savings of 7.3% and energy reduction of 17.3% at 1800 MHz relative to the base system at 2000 MHz. Alternatively our LMP can be used to improve the performance of SPEC benchmarks by an average of 2.9 % at the cost of 7.1 % increase in average power.