DELTA: Validate GPU Memory Profiling with Microbenchmarks

Accurately predicting program behaviors (e.g., memory locality, method calling frequency) is fundamental for program optimizations and runtime adaptations. Despite decades of remarkable progress, prior studies have not systematically exploited the use of program inputs, a deciding factor of program behaviors, to help in program dynamic optimizations. Triggered by the strong and predictive correlations between program inputs and program behaviors that recent studies have uncovered, the dissertation work aims to bring program inputs into the focus of program behavior analysis and program dynamic optimization, cultivating a new paradigm named input-centric program behavior analysis and dynamic optimization.;The new optimization paradigm consists of three components, forming a three-layer pyramid. at the base is program input characterization, a component for resolving the complexity in program raw inputs and extracting important features. In the middle is input-behavior modeling, a component for recognizing and modeling the correlations between characterized input features and program behaviors. These two components constitute input-centric program behavior analysis, which (ideally) is able to predict the large-scope behaviors of a program's execution as soon as the execution starts. The top layer is input-centric adaptation, which capitalizes on the novel opportunities created by the first two components to facilitate proactive adaptation for program optimizations.;This dissertation aims to develop this paradigm in two stages. In the first stage, we concentrate on exploring the implications of program inputs for program behaviors and dynamic optimization. We construct the basic input-centric optimization framework based on of line training to realize the basic functionalities of the three major components of the paradigm. For the second stage, we focus on making the paradigm practical by addressing multi-facet issues in handling input complexities, transparent training data collection, predictive model evolvement across production runs. The techniques proposed in this stage together cultivate a lifelong continuous optimization scheme with cross-input adaptivity.;Fundamentally the new optimization paradigm provides a brand new solution for program dynamic optimization. The techniques proposed in the dissertation together resolve the adaptivity-proactivity dilemma that has been limiting the effectiveness of existing optimization techniques. its benefits are demonstrated through proactive dynamic optimizations in Jikes RVM and version selection using IBM XL C Compiler, yielding significant performance improvement on a set of Java and C/C++ programs. It may open new opportunities for a broad range of runtime optimizations and adaptations. The evaluation results on both Java and C/C++ applications demonstrate the new paradigm is promising in advancing the current state of program optimizations.

Random Access Memory plays a crucial role in data processing in various computing systems for faster applications. A processor always fetches the data from the main memory for processing, sometimes the same data may be required for the processing and need to be fetched repeatedly from the main memory results time consuming. Cache memory supports faster data storage and access by storing instances of programs and data which is routinely accessed by the processor. Therefore, the cache memory is used to store recently fetched data and when a processor requests to fetch the data the controller will first check the data in cache, if the fetched data is there in the cache then it will be fetched otherwise the controller will fetch the data from the main memory. Therefore, the cache memory is useful for improving speed of processing without wasting much time for fetching the repeated data. This paper deals with various structures of single ended S-RAM cells and proposes a novel three transistor S -RAM cell design Using Tanner 45nm Technology.

Two performance gaps in the memory hierarchy, between CPU cache and main memory, and main memory and mass storage, will become increasingly severe bottlenecks for computing-system performance. Although it is necessary to increase memory capacity to fill these gaps, power also increases when conventional volatile memories are used. A new nonvolatile memory for this purpose has been anticipated. Storage class memory is used to fill the second gap. Many candidates exist: ReRAM, PRAM, and 3D-cross point type with resistive change RAM. However, nonvolatile last level cache (LLC) is used to fill the first gap. Advanced STT-MRAM has achieved sub-4ns read and write accesses with perpendicular magnetic tunnel junctions (p-MTJ) [1–2]. Furthermore, mature integration processes have been developed and 8Mb STT-MRAM with sub-5ns operation has shown high reliability [3]. Moreover, because of its non-volatility, STT-MRAM can reduce operation energy by more than 81% compared to SRAM for cache [1]. This paper presents STT-MRAM-based last level cache memory (LLC) including MRAM memory core, peripherals and cache logic circuits, using novel power optimization with high-speed power gating (HS-PG),considering processor architectures and cache memory accesses. The STT-MRAM-based cache has high reliability to reduce the write-error rate with novel write-verify-write. Furthermore, a read-modify-write scheme is implemented to reduce active power without penalty. Figure 7.2.1 presents a block diagram of a 4Mb STT-MRAM based cache.

Summary form only given. An approach towards an information theoretic based analysis and design of cache memories is presented. Computer systems usually have a slow main memory and a faster cache memory, usually much smaller than the main memory because of its cost. A somewhat similar situation is present also in Internet servers. A miss happens whenever an item is not found in the cache memory. If so, the item is fetched from the main memory and placed in the cache. A hit is obtained whenever the item is found in the cache. Usually the cost of a miss is several times that of a hit. The goal is to find strategies for the many to one mapping of addresses of the main memory to the cache memory, as well as the replacement strategies. Usual replacement strategies are least recently used, LRU, random replacement, RR, etc. The main goal is to obtain strategies that will optimize the running time of the program under execution. Since almost all programs use branch instructions and loops, some of the information theoretic approaches previously introduced consider the prediction of the result of a branch based on its past behavior. Here a different approach is considered. In particular the opposite case is analysed, i.e. a linear loop that is executed indefinitely. In particular a combined random replacement and least recently used strategy is analyzed. It is shown that this model is equivalent to the one of classifying N objects in M classes with at least c objects in each class, and that this problem gives a generalization of the Ehrenfests' urn model used in statistical thermodynamics in connection with the Boltzmann H-theorem.

On the effectiveness of phase based regression models to trade power and performance using dynamic processor adaptation

The performance monitoring of computer systems is a complex affair, made even more challenging by the increasing gap between hardware and software. Methods that collect and feed data to performance analysis can usually be classified into one of two groups. Methods in the first group (e.g., software instrumentation), can be accurate but have unacceptable overheads and offer no insight into the interaction of the software and the hardware. Methods in the second group (e.g., hardware supported) run in real time and connect code to the response in the architecture, at a considerable decrease in accuracy, with a high degree of specificity to the hardware and therefore with limited actionable information made available. In this work, we focus on the latter group - the collection and analysis of raw performance data sourced from hardware Performance Monitoring Units, built into the majority of processors. The periodic collection of samples (Event Based Sampling), containing code locations, for instance, can be triggered on events such as retired instructions or cache misses. We quantify accuracy improvements to existing methods, propose a better performing method of performance data collection and a new architecture-agnostic method of performance data analysis. In our experiments, we analyze compiled, large-scale production workloads, industry standard benchmarks, and micro-benchmarks reproducing specific architectural behaviors. First, we focus on performance data collection methods and their accuracy, when establishing instruction retirement rates. These vary from least to most advanced and we examine the low level factors influencing accuracy. In particular, we employ a method based on Last Branch Records, a hardware facility present on some Intel architectures, which allows for the sampling of short records of last taken branches leading up to the sample. We compare the most commonly used method, its improved derivatives and the Last Branch Record based method. With respect to the first method, we observe accuracy improvements of up to 18x on synthetic benchmarks and 15x on real applications. Second, to further improve accuracy, we propose a new collection method named Hybrid Basic Block Profiling, fusing those based on Last Branch Records and the widely used Event Based Sampling. We apply simple machine learning techniques to determine the operating criteria. As a demonstration of capability, our method and a profiling tool are used to generate Instruction Mixes, which traditionally require good quality data at the most detailed level of granularity - basic blocks. Compared to software instrumentation, we observe an improvement in runtime of up to 76x, while keeping instruction attribution errors at 2.1% on average. In our third contribution we describe an architecture-agnostic performance analysis methodology called Hierarchical Cycle Accounting. The user is presented with a navigable hierarchy of microarchitectural issues, expressed in a metric that is universal and simple to compare - core cycles. We develop a reference implementation for the Intel Ivy Bridge microarchitecture and provide pointers for implementations on other processor architectures. Our approach and tool are successfully used by non-experts to improve and vectorize complex scientific code. Experts benefit as well - by being able to pinpoint software issues undetected by other methods, and identifying a bug in a family of Intel processors.

The Personal Software Process (PSP) is used by software engineers to gather and analyze data about their work. Published studies typically use data collected using the PSP to draw quantitative conclusions about its impact upon programmer behavior and product quality. However, our experience using PSP in both industrial and academic settings revealed problems both in collection of data and its later analysis. We hypothesized that these two kinds of data quality problems could make a significant impact upon the value of PSP measures. To test this hypothesis, we built a tool to automate the PSP and then examined 89 projects completed by ten subjects using the PSP manually in an educational setting. We discovered 1539 primary errors and categorized them by type, subtype, severity, and age. To examine the collection problem we looked at the 90 errors that represented impossible combinations of data and at other less concrete anomalies in Time Recording Logs and Defect Recording Logs. To examine the analysis problem we developed a rule set, corrected the errors as far as possible, and compared the original and corrected data. This resulted in significant differences for measures such as yield and the cost-performance ratio, confirming our hypothesis. Our results raise questions about the accuracy of manually collected and analyzed PSP data, indicate that integrated tool support may be required for high quality PSP data analysis, and suggest that external measures should be used when attempting to evaluate the impact of the PSP upon programmer behavior and product quality.

Cache memory hierarchies are used to buffer those portions of main memory with the most frequent use by the CPU. As cache memory is very costly, good design techniques must consider small cache sizes maintaining high levels of use (hit ratio) and ease of implementation. The memory replacement policy is important in maintaining a high hit ratio. Most replacement policies used are easily implemented when the main memory has fixed page locations. A new cache algorithm using a variable page configuration is explained in terms of program behaviour.

Nonvolatile random access memory (NVRAM) is becoming a popular alternative as the memory and storage medium in battery-powered embedded systems because of its fast read/write performance, byte-addressability, and nonvolatility. A well-known example is phase-change memory (PCM) that has much longer life expectancy and faster access performance than NAND flash. When NVRAM is considered as both main memory and storage in battery-powered embedded systems, existing page cache mechanisms have too many unnecessary data movements between main memory and storage. To tackle this issue, we propose the concept of “union page cache,” to jointly manage data of the page cache in both main memory and storage. To realize this concept, we design a partial page cache strategy that considers both main memory and storage as its management space. This strategy can eliminate unnecessary data movements between main memory and storage without sacrificing the data integrity of file systems. A series of experiments was conducted on an embedded platform. The results show that the proposed strategy can improve the file accessing performance up to 85.62% when PCM used as a case study.

Writing specifications about program behavior is hard. Writing specifications about non-functional effects such as resource usage is often even harder. If manually instrumenting the program is not an option, programmers have to rely on comment-based specification languages like JML to introduce ghost variables and other fairly abstract concepts that are complicated and hard to maintain. Even worse, most static analysis tools nowadays operate on bit- or bytecode and cannot process those type of specifications To address this problem, we propose a library-based specification formalism for time complexity in Java. The approach is inspired by the success of assertion libraries like CodeContracts and Guava. Our library provides a set of methods and enumerated types that allow the user to write complexity assertions in a 'human' readable form and without instrumenting the code. On the backend, we provide a bytecode rewriting tool that uses these assertions to automatically instrument the program with counter variables. The transformed program can then be checked via testing or off-the-shelf automated static analyzers

Main memory contains transient information for all resident applications. However, if memory chip contents survives power-off, e.g., via freezing DRAM chips, sensitive data such as passwords and keys can be extracted. Main memory persistence will soon be the norm as recent advancements in MRAM and FeRAM position non-volatile memory technologies for widespread deployment in laptop, desktop, and embedded system main memory. Unfortunately, the same properties that provide energy efficiency, tolerance against power failure, and instant-on'' power-up also subject systems to offline memory scanning. In this paper, we propose a memory encryption control unit (MECU) that provides memory confidentiality during system suspend and across reboots. The MECU encrypts all memory transfers between the processor-local level 2 cache and main memory to ensure plaintext data is never written to the persistent medium. The MECU design is outlined and performance and security trade-offs considered. We evaluate a MECU-enhanced architecture using the SimpleScalar hardware simulation framework on several hardware benchmarks. This analysis shows the majority of memory accesses are delayed by less than 1 ns, with higher access latencies (caused by resume state reconstruction) subsiding within 0.25 seconds of a system resume. In effect, the MECU provides zero-cost steady state memory confidentiality for non-volatile main memory.

The exponential increase in multicore processor (CMP) cache sizes accompanied by growing on-chip wire delays make it difficult to implement traditional caches with a single, uniform access latency. Non-Uniform Cache Architecture (NUCA) designs have been proposed to address this problem. A NUCA divides the whole cache memory into smaller banks and allows banks nearer a processor core to have lower access latencies than those further away, thus mitigating the effects of the cache's internal wires. Determining the best placement for data in the NUCA cache at any particular moment during program execution is crucial for exploiting the benefits that this architecture provides. Dynamic NUCA (D-NUCA) allows data to be mapped to multiple banks within the NUCA cache, and then uses data migration to adapt data placement to the program's behavior. Although the standard migration scheme is effective in moving data to its optimal position within the cache, half the hits still occur within non-optimal banks. This paper reduces this number by anticipating data migrations and moving data to the optimal banks in advance of being required. We introduce a prefetcher component to the NUCA cache that predicts the next memory request based on the past. We develop a realistic implementation of this prefetcher and, furthermore, experiment with a perfect prefetcher that always knows where the data resides, in order to evaluate the limits of this approach. We show that using our realistic data prefetching to anticipate data migrations in the NUCA cache can reduce the access latency by 15% on average and achieve performance improvements of up to 17%.

GPUs, due to their massively-parallel computing architectures, provide high performance for data-parallel applications. However, existing GPU simulators are too slow to enable architects to quickly evaluate their hardware designs and software analysis studies. Sampled simulation methodologies are one common way to speed up CPU simulation. However, GPUs apply drastically different execution models that challenge the sampled simulation methods designed for CPU simulations. Recent GPU sampled simulation methodologies do not fully take advantage of the GPU’s special architecture features, such as limited types of basic blocks or warps. Moreover, these methods depend on up-front analysis via profiling tools or functional simulation, making them difficult to use.

Widespread use of microarray technology that led to highly complex datasets often is addressing similar or related biological questions. In translational medicine research is often based on measurements that have been obtained at different points in time. However, the researcher looks at them as a progression over time. If a biological stimulus shows an effect on a particular gene that is reversed over time, this would show, for instance, as a peak in the gene's temporal expression profile. Our program SPOT helps researchers find these patterns in large sets of microarray data. We created the software tool using open-source platforms and the Semantic Web tool Protégé-OWL.

This research proposes a phase-change memory (PCM) based main memory system with an effective combination of a superblock-based adaptive buffering structure and its associated set divisible last-level cache (LLC). To achieve high performance similar to that of dynamic random-access memory (DRAM) based main memory, the superblock-based adaptive buffer (SABU) is comprised of dual DRAM buffers, i.e., an aggressive superblock-based pre-fetching buffer (SBPB) and an adaptive sub-block reusing buffer (SBRB), and a set divisible LLC based on a cache space optimization scheme. According to our experiment, the longer PCM access latency can typically be hidden using our proposed SABU, which can significantly reduce the number of writes over the PCM main memory by 26.44%. The SABU approach can reduce PCM access latency up to 0.43 times, compared with conventional DRAM main memory. Meanwhile, the average memory energy consumption can be reduced by 19.7%.