Code cache management schemes for dynamic optimizers

A dynamic optimizer is a runtime software system that groups a program's instruction sequences into traces, optimizes those traces, stores the optimized traces in a software-based code cache, and then executes the optimized code in the code cache. To maximize performance, the vast majority of the program's execution should occur in the code cache and not in the different aspects of the dynamic optimization system. In the past, designers of dynamic optimizers have used the SPEC2000 benchmark suite to justify their use of simple code cache management schemes. In this paper, we show that the problem and importance of code cache management changes dramatically as we move from SPEC2000, with its relatively small number of dynamically generated code traces, to large interactive Windows applications. We also propose and evaluate a new cache management algorithm based on generational code caches that results in an average miss rate reduction of 18% over a unified cache, which translates into 19% fewer instructions spent in the dynamic optimizer. The algorithm categorizes code traces based on their expected lifetimes and group traces with similar lifetimes together in separate storage areas. Using this algorithm, short-lived code traces can easily be removed from a code cache without introducing fragmentation and without suffering the performance penalties associated with evicting long-lived code traces.

Both dynamic binary translation systems and optimization systems store the translated or optimized code in the software maintained code cache for reuse. The performance of the code cache is crucial. Translated code is usually organized as code blocks in the code cache and each code block transfer control to the next one through a control transfer instruction. As the target address of a control transfer instruction is in the form of its source program counter, the conventional code cache system has to check the address mapping table for the translated target address to find the required target code block, which will cause considerable performance degradation. Control transfer instructions can be divided into two categories as direct control transfer instructions and indirect control transfer instructions. For indirect control transfer instructions, the target address is hold in the register or memory element whose content can be changed during the execution of the program. It is difficult to chain the indirect control transfer instructions with a fixed translated target address through pure software approaches. A novel indirect control transfer chaining approach is proposed in this paper. The principle of the technique is to insert custom chaining instructions into the translated code block while translating the indirect control transfer instructions and execute those chaining instructions to implement dynamical chaining. Some special hardware and software assists are proposed in this paper. Evaluation of the proposed approach is conducted on a code cache simulator. Experiment results show that our hardware assisted indirect control transfer instruction chaining approach can improve the performance of the code cache system dramatically.Keywordscode cacheindirect control transfer instructiondynamic chainingsimulator

Most dynamic binary translators (DBT) and optimizers (DBO) target binary traces, i.e. frequently executed paths, as code regions to be translated and optimized. Code region formation is the most important first step in all DBTs and DBOs. The quality of the dynamically formed code regions determines the extent and the types of optimization opportunities that can be exposed to DBTs and DBOs, and thus, determines the ultimate quality of the final optimized code. The Next-Executing-Tail (NET) trace formation method used in HP Dynamo is an early example of such techniques. Many existing trace formation schemes are variants of NET. They work very well for most binary traces, but they also suffer a major problem: the formed traces may contain a large number of early exits that could be branched out during the execution. If this happens frequently, the program execution will spend more time in the slow binary interpreter or in the unoptimized code regions than in the optimized traces in code cache. The benefit of the trace optimization is thus lost. Traces/regions with frequently taken early-exits are called delinquent traces/regions. Our empirical study shows that at least 8 of the 12 SPEC CPU2006 integer benchmarks have delinquent traces. In this paper, we propose a light-weight region formation technique called Early-Exit Guided Region Formation (EEG) to improve the quality of the formed traces/regions. It iteratively identifies and merges delinquent regions into larger code regions. We have implemented our EEG algorithm in two LLVM-based multi-threaded DBTs targeting ARM and IA32 instruction set architecture (ISA), respectively. Using SPEC CPU2006 benchmark suite with reference inputs, our results show that compared to an NET-variant currently used in QEMU, a state-of-the-art retargetable DBT, EEG can achieve a significant performance improvement of up to 72% (27% on average), and to 49% (23% on average) for IA32 and ARM, respectively.

Most dynamic binary translators (DBT) and optimizers (DBO) target binary traces, i.e. frequently executed paths, as code regions to be translated and optimized. Code region formation is the most important first step in all DBTs and DBOs. The quality of the dynamically formed code regions determines the extent and the types of optimization opportunities that can be exposed to DBTs and DBOs, and thus, determines the ultimate quality of the final optimized code. The Next-Executing-Tail (NET) trace formation method used in HP Dynamo is an early example of such techniques. Many existing trace formation schemes are variants of NET. They work very well for most binary traces, but they also suffer a major problem: the formed traces may contain a large number of early exits that could be branched out during the execution. If this happens frequently, the program execution will spend more time in the slow binary interpreter or in the unoptimized code regions than in the optimized traces in code cache. The benefit of the trace optimization is thus lost. Traces/regions with frequently taken early-exits are called delinquent traces/regions. Our empirical study shows that at least 8 of the 12 SPEC CPU2006 integer benchmarks have delinquent traces.

In this paper, we propose a static cache management policy for RAID architecture which uses an SSD as a disk cache. Because SSD has several drawbacks, dynamic cache management scheme such as LRU can cause serious problem i.e., too much write operations can be occurred in order to contain most recently used data. Frequent write operations on SSD not only slow down the performance but also shorten the lifetime. For this reason, we propose a static cache management policy which takes popularity and size into account. Popular data of some period is copied into SSD and too large files are excluded since too large files can lower the hit ratio. We used real trace to evaluate our scheme and the results showed that write counts are reduced to 1/37 of LRU policy as we desired. Unfortunately, the average hit ratio was lower than LRU policy but when popular data is similar between two periods, our scheme showed better hit ratio than LRU. This implies that our scheme can be used on web server which follows zipf’s distribution and has smooth changing of popular data.

Non-Uniform Cache Access (NUCA) architectures provide a potential solution to reduce the average latency for the last-level-cache (LLC), where the cache is organized into per-core local and remote partitions. Recent research has demonstrated the benefits of cooperative cache sharing among local and remote partitions. However, ignoring cache access patterns of concurrently executing applications sharing the local and remote partitions can cause inter-partition contention that reduces the overall instruction throughput. We propose a dynamic cache management scheme for LLC in NUCA-based architectures, which reduces inter-partition contention. Our proposed scheme provides efficient cache sharing by adapting migration, insertion, and promotion policies in response to the dynamic requirements of the individual applications with different cache access behaviors. Our adaptive cache management scheme allows individual cores to steal cache capacity from remote partitions to achieve better resource utilization. On average, our proposed scheme increases the performance (instructions per cycle) by 28% (minimum 8.4%, maximum 75%) compared to a private LLC organization.

The user-generated podcasting service over mobile opportunistic networks can facilitate the user generated content dissemination while humans are on the move. However, in such a distributed and dynamic network environment, the design of efficient content forwarding and cache management schemes are challenging due to the lack of global podcast channel popularity information at each individual node. We design a distributed reputation system at each node for estimating the global channel popularity information which is significant for forwarding and cache management decision. Our simulation result shows that, compare to history-based rank scheme, our reputation system can significantly improve system performance under community-based random way-point (C-RWP) mobility model and localized channel popularity distribution.

Data broadcast has been considered a promising way of information dissemination to a massive number of users in a wireless communication environment. Reducing user-waiting time is a major problem in developing a data broadcast system. There are two approaches for this problem; One is to design a broadcast schedule at the server side which reduces the mean response time, and the other is to utilize a local cache at the user side which may respond to a user request instantly. Though these two approaches were addressed separately in the literature, they may be taken jointly for better performance. The performance of system with joint approach depends on several factors such as broadcast schedule, cache size, cache management strategy, etc. In this paper we analyze response time in a data broadcast system with joint approach in which information items are structurally related with each other as in WWW. Based on the worst-case assumption, we derive a lower bound on the system performance for a given set of broadcast schedule, cache size, and cache management strategy. This result will be of help for designing and developing a data broadcast system. We support our analysis by carrying out an extensive simulation on some interesting proposed broadcast schedules and cache management strategies.

The power systems of the future - smart grid - will see an increase in both renewable energy sources and load demand increasing the need for fast dynamic reconfiguration of system parameters. Advanced simulation capabilities are needed to support future planning and to implement real-time dynamic optimization. To achieve this, intelligent algorithms are needed for prediction and system monitoring along with advanced hardware to provide speed. FASTSIM capabilities allows for fast prediction of how a proposed reconfiguration will affect the system permitting implementation of radical reconfigurations without fear of disturbing vital system settings since the proposed solution will be checked before being implemented. Power systems will need to be continuously monitored for power losses and system contingencies. A combination of system states prediction, dynamic load flow, system optimization, and solution stability checking will allow for a more reliable, affordable and clean power grid. This paper presents a dynamic optimization method based on adaptive critic designs for the smart grid environment.

With modern technologies, it is possible to create annotated interactive non-linear videos (a form of hypervideo) for the Web. These videos have a non-linear structure of linked scenes to which additional information (other media like images, text, audio, or additional videos) can be added. A variety of user interactions—like in- and between-scene navigation or zooming into additional information—are possible in players for this type of video. Like linear video, quality of experience (QoE) with annotated hypervideo experiences is tied to the temporal consistency of the video stream at the client end—its flow. Despite its interactive complexity, users expect this type of video experience to flow as seamlessly as simple linear video. However, the added hypermedia elements bog playback engines down. Download and cache management systems address the flow issue, but their effectiveness is tied to numerous questions respecting user requirements, computational strategy, and evaluative metrics. In this work, we (a) define QoE metrics, (b) examine structural and behavioral patterns of interactive annotated non-linear video, (c) propose download and cache management algorithms and strategies, (d) describe the implementation of an evaluative simulation framework, and (e) present the algorithm test results.

We present a position-based neighbour prefetch algorithm for client-server based walkthrough. This algorithm predicts which neighbouring viewcell will be needed in the near future according to the current view position of the client and prefetches the pre-computed potential visible set (PVS) for that viewcell to the client-side cache. Our algorithm has a low memory requirement for the client-side cache. With the same cache size, our algorithm offers a much better performance than the simple neighbour prefetch method. We also provide a delta-transmission algorithm to optimize the prefetch transmission procedure by avoiding transmitting those geometries that are already in the client-side cache. A cache management and entry replacement strategy is also proposed to maintain the client-side cache.

Dynamic optimization has the potential to adapt the program's behavior at run-time to deliver performance improvements over static optimization. Dynamic optimization systems usually perform their optimization in series with the application's execution. This incurs overhead which reduces the benefit of dynamic optimization, and prevents some aggressive optimizations from being performed. In this paper we propose a new dynamic optimization framework called Trident. Concurrent with the program's execution, the framework uses hardware support to identify optimization opportunities, and uses spare threads on a multithreaded processor to perform dynamic optimizations for these optimization events. We evaluate the benefit of using Trident to guide code layout, basic compiler optimizations, and value specialization. Our results show that using Trident with these optimizations achieves an average 20% speedup, and is complementary with other memory latency tolerant techniques, such as prefetching.

The estimation of time varying parameters in nonlinear dynamic systems remains as an open problem. In this work, we propose a new approach to addressing this problem by using dynamic optimization. The system is described by a rigorous model which is a set of nonlinear differential and algebraic equations with unknown parameters. The sequential quadratic programming and the orthogonal collocation methods are used to solve the dynamic nonlinear optimization problem. An efficient procedure is presented for calculation of the sensitivity coefficients, gradients and optimization. The effectiveness of the approach is illustrated by estimating tray efficiencies of a pilot distillation column.

An enhanced scatter search method for the global dynamic optimization of nonlinear processes using the control vector parametrization (CVP) approach is presented. Sharing some features of the scatter search metaheuristic, this new method presents a simpler but more effective design which helps to overcome typical difficulties of nonlinear dynamic systems optimization such as noise, flat areas, nonsmoothness, and/or discontinuities. This new algorithm provides a good balance between robustness and efficiency in the global phase, and couples a local search procedure to accelerate the convergence to optimal solutions. Its application to four multimodal dynamic optimization problems, compared with other state-of-the-art global optimization algorithms, including an advanced scatter search design, proves its efficiency and robustness, showing a very good scalability.

Recent advancements in flash-based solid state drive (SSD) make it a highly desirable storage device, especially for data-intensive applications. There are significant more SSDs used in data centers and high performance computing systems. SSDs perform one or two orders better than traditional hard disk drives generally. However, the performance of random writes on SSDs, especially small random writes, is still largely limited due to the garbage collection (GC) process. Existing work tried to utilize the on-device RAM as a write cache to improve the write performance, however directly utilizing it as a normal write cache under utilizes the RAM cache. In this poster, we present our initial study of a cache management scheme that hides the GC latency.