Competitive prefetching for concurrent sequential I/O

As the speed gap between CPU and external memory widens, memory latency has become the dominant performance bottleneck in modern applications. Closely connected are caches which play an important role in reducing the average memory latency. The way data is accessed strongly influences cache performance. Numerous multimedia algorithms operating on data such as images and videos, perform processing over rectangular regions of pixels. If this and other data access patterns are properly exploited, significant performance improvements can be achieved. This paper proposes a prefetch-aware memory system that exploits 2D, stride and sequential data access patterns in multimedia applications. It aims at reducing the average memory access latency, lowering the number of memory accesses and utilizing the bandwidth efficiently. Our results reveal significant average memory access time (AMAT) reduction of 21.2% when utilizing effectively the proposed approach compared to the baseline in the evaluated workloads.

Memory latency and bandwidth are progressing at a much slower pace than processor performance. In this paper, we describe and evaluate the performance of three variations of a hardware function unit whose goal is to assist a data cache in prefetching data accesses so that memory latency is hidden as often as possible. The basic idea of the prefetching scheme is to keep track of data access patterns in a reference prediction table (RPT) organized as an instruction cache. The three designs differ mostly on the timing of the prefetching. In the simplest scheme (basic), prefetches can be generated one iteration ahead of actual use. The lookahead variation takes advantage of a lookahead program counter that ideally stays one memory latency time ahead of the real program counter and that is used as the control mechanism to generate the prefetches. Finally the correlated scheme uses a more sophisticated design to detect patterns across loop levels. These designs are evaluated by simulating the ten SPEC benchmarks on a cycle-by-cycle basis. The results show that 1) the three hardware prefetching schemes all yield significant reductions in the data access penalty when compared with regular caches, 2) the benefits are greater when the hardware assist augments small on-chip caches, and 3) the lookahead scheme is the preferred one cost-performance wise.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Software transactional memory (STM) has been proposed as a promising programming paradigm for shared memory multi-threaded programs as an alternative to conventional lock based synchronization primitives. Typical STM implementations employ a conflict detection scheme, which works with uniform access granularity, tracking shared data accesses either at word/cache line or at object level. It is well known that a single fixed access tracking granularity cannot meet the conflicting goals of reducing false conflicts without impacting concurrency adversely. A fine grained granularity while improving concurrency can have an adverse impact on performance due to lock aliasing, lock validation overheads, and additional cache pressure. On the other hand, a coarse grained granularity can impact performance due to reduced concurrency. Thus, in general, a fixed or uniform granularity access tracking (UGAT) scheme is application-unaware and rarely matches the access patterns of individual application or parts of an application, leading to sub-optimal performance for different parts of the application(s). In order to mitigate the disadvantages associated with UGAT scheme, we propose a Variable Granularity Access Tracking (VGAT) scheme in this paper. We propose a compiler based approach wherein the compiler uses inter-procedural whole program static analysis to select the access tracking granularity for different shared data structures of the application based on the application's data access pattern. We describe our prototype VGAT scheme, using TL2 as our STM implementation. Our experimental results reveal that VGAT-STM scheme can improve the application performance of STAMP benchmarks from 1.87% to up to 21.2%.

We propose a prefetch cache sizing module for use with any sequential prefetching scheme and evaluate its impact on the hit rate. Disk array caches perform sequential prefetching by loading data contiguous to I/O request data into the array cache. If the I/O workload has sequential locality, then data prefetched in response to sequential accesses in the workload will receive hits. Different schemes prefetch different data, so the prefetch cache size requirement varies. Moreover, the proportion of sequential and random requests in the workload and their interleaving pattern affects the size requirement. If the cache is too small, then prefetched data would get evicted from the cache before a request for the data arrives, thus lowering the hit rate. If the cache is too large, then valuable cache space is wasted. We present a simple sizing module that can be added to any prefetching scheme to ensure that the prefetch cache size is adequately matched to the requirement of the prefetching scheme on a dynamic workload comprising multiple streams. We analytically compute the maximal hit rate achievable by popular prefetching schemes and through simulations, show that our sizing module maintains the prefetch cache at a size that nearly achieves this maximal hit rate.

We study the replication transition problem (RTP) in the Grid systems. Most distributed systems replicate data to increase data access efficiency. A replication strategy dictates where the replicas are stored in respond to data access pattern, and a good strategy can improve data access efficiency. However, the access pattern in a distributed system is constantly changing. As a result a good replication strategy must evolve accordingly. The replication transition problem is to seek an efficient transition from one replication strategy to another in order to cope with the dynamic data access pattern. This paper focuses on the RTP problem for Grid systems in four communication models that have different communication capabilities, i.e., whether message forwarding is allowed and whether network capacity is uniform among different links. We show that there exists a polynomial time algorithm that provides optimal solution for the RTP problem when forwarding is not allowed and the communication links are uniform. We also propose heuristic algorithms for solving variants of the RTP problem and conduct experiments to evaluate their performances. The experimental results indicate that our proposed heuristics are very effective.

Large scientific collaborations often have multiple scientists accessing the same set of files while doing different analyses, which create repeated accesses to the large amounts of shared data located far away. These data accesses have long latency due to distance and occupy the limited bandwidth available over the wide-area network. To reduce the wide-area network traffic and the data access latency, regional data storage caches have been installed as a new networking service. To study the effectiveness of such a cache system in scientific applications, we examine the Southern California Petabyte Scale Cache for a high-energy physics experiment. By examining about 3TB of operational logs, we show that this cache removed 67.6% of file requests from the wide-area network and reduced the traffic volume on wide-area network by 12. 3TB (or 35.4%) an average day. The reduction in the traffic volume (35.4%) is less than the reduction in file counts (67.6%) because the larger files are less likely to be reused. Due to this difference in data access patterns, the cache system has implemented a policy to avoid evicting smaller files when processing larger files. We also build a machine learning model to study the predictability of the cache behavior. Tests show that this model is able to accurately predict the cache accesses, cache misses, and network throughput, making the model useful for future studies on resource provisioning and planning.

As deep learning technologies increasingly influence various aspects of human life, the emerging paradigm of Artificial Intelligence of Things (AIoT) is gaining significant attention. AIoT, which relies on large datasets and complex models, poses significant challenges to the limited resources of mobile systems. Many solutions have been proposed to address these challenges, including computation offloading to edge or cloud servers. In this article, we demonstrate that analyzing data access patterns and managing them efficiently are also crucial for further enhancing the performance of AIoT systems. Specifically, we propose an efficient write buffer management scheme tailored to mobile AIoT workloads. Our approach is based on an extensive analysis of data access patterns, revealing that the conventional buffer cache architectures and algorithms are inefficient for AIoT workloads due to access characteristics such as write dominance and long repetitive loops. To address these issues, we adopt a non-volatile write buffer at the storage layer and judiciously manage write loops. The key contributions of our study are as follows: 1) Despite relying solely on limited flush information at the storage layer, our scheme achieves performance comparable to full loop detection at the host system. 2) Our scheme requires no modifications to host OS semantics or buffer cache interfaces, ensuring high adaptability. 3) Experimental results demonstrate that our scheme significantly reduces storage traffic, outperforms representative algorithms such as LRU, LFU, LIRS, CLOCK-Pro, and LeCaR, and enhances data access efficiency for AIoT workloads.

Understanding the inherent system characteristics is crucial to the design and optimization of cloud storage system, and few studies have systematically investigated its data characteristics and access patterns. This paper presents an analysis of file system snapshot and five-month access trace of a campus cloud storage system that has been deployed on Tsinghua campus for three years. The system provides online storage and data sharing services for more than 19,000 students and 500 student groups. We report several data characteristics including file size and file type, as well as some access patterns, including read/write ratio, read-write dependency and daily traffic. We find that there are many differences between cloud storage system and traditional file systems: our cloud storage system has larger file sizes, lower read/write ratio, and smaller set of active files than those of a typical traditional file system. With a trace-driven simulation, we find that the cache efficiency can be improved by 5 times using the guidance from our observations.

The performance achieved by an application is limited by architectural features such as program data access and processing patterns. Parallelization approaches exhibit dissimilar performance and have a direct impact in application execution time. Additionally, developing parallel code involves additional complexity and productivity for programmers to accelerate or rewrite the program. In this paper, we present a comparative performance evaluation of a CPU, GPU, and many-core (Xeon Phi KNL) architectures for the Non-Local Means filter. We asses the effect of different data access and processing patterns in two computational optimizations developed for the aforementioned filter. We follow a top-down approach in terms of the parallelization approach chosen, starting from performance primitives as a first step to give easy drop-in acceleration and then compiler directives with frameworks such as OpenMP and OpenACC as an intermediate step to map computing tasks to the underlying hardware. Results show that both libraries and directives are effective at accelerating code with a combination of both being necessary to overcome performance bottlenecks.

In modern computer architectures, access latency varies considerably between different levels in the memory hierarchy. Consequently, applications with data access patterns that don't reuse much data in fast levels of the hierarchy incur additional delays. To improve the performance of complex, data-intensive applications, developers need tools that help them understand the causes of poor memory hierarchy utilization. While most performance tools associate metrics with functions or statements, in this paper we explore data-centric analyses that associate metrics not only with data accesses but also with data objects themselves. Our contributions are three-fold. First, we propose several refinements to existing data-centric techniques that enable accurate and low-overhead measurements. Second, we combine data-centric analysis with call path profiling; this combination of techniques relates inefficient access patterns back to data objects across complete dynamic call chains. Third, we developed a graphical user interface that gracefully presents our analysis results using a multiplicity of views, which helps users identify problematic accesses and data structures. We demonstrate the utility of our approach by showing how our tool identifies problematic data access patterns in several HPC applications and a pair of the SPEC CPU2006 benchmarks.

Most video streaming traffic is delivered over HTTP using standard web servers. While traditional web server workloads consist of requests that are primarily for small files that can be serviced from the file system cache, HTTP video streaming workloads often service a long tail of large infrequently requested videos. As a result, optimizing disk accesses is critical to obtaining good server throughput. In this article we explore serialized, aggressive disk prefetching, a technique that can be used to improve the throughput of HTTP streaming video web servers. We identify how serialization and aggressive prefetching affect performance, and, based on our findings, we construct and evaluate Libception, an application-level shim library that implements both techniques. By dynamically linking against Libception at runtime, applications are able to transparently obtain benefits from serialization and aggressive prefetching without needing to change their source code. In contrast to other approaches that modify applications, make kernel changes, or attempt to optimize kernel tuning, Libception provides a portable and relatively simple system in which techniques for optimizing I/O in HTTP video streaming servers can be implemented and evaluated. We empirically evaluate the efficacy of serialization and aggressive prefetching both with and without Libception, using three web servers (Apache, nginx, and the userver) running on two operating systems (FreeBSD and Linux). We find that, by using Libception, we can improve streaming throughput for all three web servers by at least a factor of 2 on FreeBSD and a factor of 2.5 on Linux. Additionally, we find that with significant tuning of Linux kernel parameters, we can achieve similar performance to Libception by globally modifying Linux’s disk prefetch behaviour. Finally, we demonstrate Libception’s ability to reduce the completion time of a microbenchmark involving two applications competing for disk resources.

High availability for both data and applications is rapidly becoming a business requirement. While database systems support recovery, providing high database availability, applications may still lose work because of server outages. When a server crashes, any volatile state associated with the application's database session is lost and the application may require an operator-assisted restart. This exposes server failures to end-users and always degrades application availability. Our Phoenix/ODBC system supports persistent database sessions that can survive a database crash without the application being aware of the outage, except for possible timing considerations. This improves application availability and eliminates the application programming needed to cope with database crashes. Phoenix/ODBC requires no changes to the database system, data access routines or applications. Hence, it can be deployed in any application that uses ODBC to access a database. Further, our generic approach can be exploited for a variety of data access protocols. In this paper, we describe the design of Phoenix/ODBC and introduce an extension to optimize the response time and to reduce overhead for OLTP workloads. We present a performance evaluation using the TPC-C and TPC-H benchmarks that demonstrate Phoenix/ODBC's extra overhead is modest.

Various air indexing and data scheduling schemes for wireless broadcast of spatial data have been developed for energy efficient query processing. The existing schemes are not effective when the clients' data access patterns are skewed to some items. It is because the schemes are based on flat broadcast that does not take the popularity of the data items into consideration. In this paper, thus, we propose a data scheduling scheme letting the popular items appear more frequently on the channel, and grid-based distributed index for non-flat broadcast (GDIN) for window query processing. The proposed GDIN allows quick and energy efficient processing of window query, matching the clients' linear channel access pattern and letting the clients access only the queried data items. The simulation results show that the proposed GDIN significantly outperforms the existing schemes in terms of access time, tuning time, and energy efficiency.

A lot of research has been done on answering single-item queries, only a few have looked at answering multiple-item queries in data broadcast systems. The few that did, have proposed approaches that are less responsive to changes in the query queue. It is not immediately clear how single-item scheduling algorithms will perform when used in answering pull-based multiple-item queries. This paper investigates the performance of existing single-item scheduling algorithms in answering multiple-item queries in pull-based data broadcast systems. We observed that Longest Wait First, a near-optimal single-item data scheduling algorithm, has been used in environments where users' data access pattern is skewed. This paper also investigates the performance of Longest Wait First under various user access patterns. We propose $\mathcal{Q}$LWF: an online data broadcast scheduling algorithm for answering multiple-item queries in pull-based data broadcast systems. For the purpose of comparison with $\mathcal{Q}$LWF, we adapted existing pull single-item algorithm, push single-item algorithm, and push multiple-item algorithm to answer multiple-item queries in pull environments. Results from extensive sets of experiments show that $\mathcal{Q}$LWF has a superior performance compared with the adapted algorithms.

Auction is an effective mechanism to distribute spectrum resources. Although many privacy-preserving auction schemes for spectrum allocation have been proposed, none of them is able to perform practical spectrum auctions while ensuring enough security for bidders' private information, such as geo-locations, bid values, and data access patterns. To address this problem, we propose SLISA, a lightweight auction framework which enables an efficient spectrum allocation without revealing anything but the auction outcome, i.e., the winning bidders and their clearing prices. We present contributions on two fronts. First, as a foundation of our design, we adopt a Shuffle-then-Compute strategy to build a series of secure sub-protocols based on lightweight cryptographic primitives (e.g., additive secret sharing and basic garbled circuits). Second, we improve an advanced spectrum auction mechanism to make it data-oblivious, such that data access patterns can be hidden. Meanwhile, the modified protocols adapt to our elaborate building blocks without affecting its validity and security. We formally prove the security of all protocols under a semi-honest adversary model, and demonstrate performance improvements compared with state-of-the-art works through extensive experiments.