Dynamic profiling and trace cache generation

Dynamic program optimization is increasingly important for achieving good runtime performance. A key issue is how to select which code to optimize. One approach is to dynamically detect traces, long sequences of instructions spanning multiple methods, which are likely to execute to completion. Traces are easy to optimize and have been shown to be a good unit for optimization. The paper reports on a new approach for dynamically detecting, creating and storing traces in a Java virtual machine. We first describe four important criteria for a successful trace strategy: good instruction stream coverage, low dispatch rate, cache stability, and optimizability of traces. We then present our approach based on branch correlation graphs. A branch correlation graph stores information about the correlation between pairs of branches, as well as additional state information. We present the complete design for an efficient implementation of the system, including a detailed discussion of the trace cache and profiling mechanisms. We have implemented an experimental framework to measure the traces generated by our approach in a direct-threaded Java VM (SableVM) and we present experimental results to show that the traces we generate meet the design criteria.

For the last decade, the Java Virtual Machine (JVM) has been a popular platform to host languages other than Java. Language implementation frameworks like Truffle allow the implementation of dynamic languages such as JavaScript or Ruby with competitive performance and completeness. However, statically typed languages are still rare under Truffle. We present Sulong, an LLVM IR interpreter that brings all LLVM-based languages including C, C++, and Fortran in one stroke to the JVM. Executing these languages on the JVM enables a wide area of future research, including high-performance interoperability between high-level and low-level languages, combination of static and dynamic optimizations, and a memory-safe execution of otherwise unsafe and unmanaged languages.

The trace cache is a recently proposed solution to achieving high instruction fetch bandwidth by buffering and reusing dynamic instruction traces. This work presents a new block-based trace cache implementation that can achieve higher IPC performance with more efficient storage of traces. Instead of explicitly storing instructions of a trace, pointers to blocks constituting a trace are stored in a much smaller trace table. The block-based trace cache renames fetch addresses at the basic block level and stores aligned blocks in a block cache. Traces are constructed by accessing the replicated block cache using block pointers from the trace table. Performance potential of the block-based trace cache is quantified and compared with perfect branch prediction and perfect fetch schemes. Comparing to the conventional trace cache, the block-based design can achieve higher IPC, with less impact on cycle time.Results: Using the SPECint95 benchmarks, a 16-wide realistic design of a block-based trace cache can improve performance 75% over a baseline design and to within 7% of a baseline design with perfect branch prediction. With idealized trace prediction, it is shown the block-based trace cache with an 1K-entry block cache achieves the same performance of the conventional trace cache with 32K entries.

As process technology continues to shrink, a large number of bitcells in on-chip caches is expected to be faulty. The number of defective cells varies from die-to-die, wafer-to-wafer, and in the field of application depends on the run-time operating conditions (e.g., supply voltage and frequency). Those trends necessitate i) to study fault-tolerant (FT) cache mechanisms in a wide spectrum of fault-probabilities and ii) to devise appropriate FT techniques that must be able to adapt their FT capacity to the volume of defective locations of the target faulty caches. It is well known that keeping the cache capacity, block size and the volume of defective cells constant, the average number of misses, due to faulty cells, decreases as the associativity of the cache increases. To this end, we propose DARCA, a Defect-Aware Reconfigurable Cache Architecture, which is equipped with the ability of dynamically varying its associativity according to the volume of defective cells. To keep the hardware overhead very small, as the associativity of the cache is multiplied by a power of two, its block size is divided by the same number. Since almost all contemporary processors use prefetching, we also applied DARCA to prefetch-assisted caches. By performing cycle-accurate simulations for the SPEC2006 benchmarks assuming a wide range of fault-probabilities, we show that DARCA compares favorably against several known FT cache mechanisms with respect to the performance loss caused by defective cells.

Server virtualization has enabled resource consolidation and has minimized the need for additional and expensive hardware. Server virtualization has been widely deployed in a lot of organizations, because of the attractive benefits it offers like minimal hardware, cost effective resource consolidation, easier management and maintenance of hardware resources and so on. Many cloud service providers have offloaded processing capabilities to virtual servers in their data centers instead of using multiple physical servers which are very expensive these days. When such a virtualized server setup is connected to disk arrays, latency of data access becomes prominent. In order to minimize the response time of application requests on the virtual servers and make efficient utilization of cache, a dynamic caching mechanism is proposed by the authors on the hypervisor. The caching mechanism proposed in the research is different, in the sense that, it is an isolated, unshared and dynamic cache, where caching is done on the hypervisor instead of the virtual machines. The dynamic cache allocation algorithm enables thin provisioning of the hypervisor cache and leads to efficient cache utilization. Experimental results obtained using the thin provisioning algorithm look promising in terms of reducing the response time and efficiently utilizing the available cache.

Live migration of Java Virtual Machines (JVMs) consumes significant amounts of time and resources, imposing relevant application performance overhead. This problem is specially hard when memory modified by applications changes faster than it can be transferred through the network (to a remote host). Current solutions to this problem resort to several techniques which depend on high-speed networks and application throttling, require lots of CPU time to compress memory, or need explicit assistance from the application. We propose a novel approach, Garbage Collector (GC) assisted JVM Live Migration for Java Server Applications (ALMA). ALMA makes a snapshot to be migrated containing a minimal amount of application state, by taking into account the amount of reachable memory (i.e. live data) detected by the GC. The main novelty of ALMA is the following: ALMA analyzes the JVM heap looking for regions in which a collection phase is advantageous w.r.t. the network bandwidth available (i.e. it pays to collect because a significant amount of memory will not be part of the snapshot). ALMA is implemented on OpenJDK 8 and extends CRIU (a Linux disk-based process checkpoint/restore tool) to support process live migration over the network. We evaluate ALMA using well-known JVM performance benchmarks (SPECjvm2008 and DaCapo), and by comparing it to other previous approaches. ALMA shows very good performance results.

In big data mining analysis, the data records in the dataset are randomly retrieved. The distributed storage modes, such as BigTable, HBase, provide the cache policy for file blocks in retrieval operations. Since these records are scattered in different file blocks, the block cache does not have a high hit rate. To deal with the above problem, we propose an LRU-based double queue K-frequency cache method (DLK). The method presents a double queue storage structure, applying different storage and eviction rules for the data with varying access frequency (i.e., high/low access frequency). While the method divides the memory into data area and list area and adopts different data structure to reduce the time of data retrieval and data processing. The experimental results show that proposed method can reduce retrieval time by 30% with the cache mechanism. Compared with existing methods DLK can improve the hit rate by 60.1% and reduce the retrieval time by 43.5%. While applying in smaller cache capacity, our method outperforms other algorithms.

Microprocessors have turned to multicore, i.e. multiple processor cores, along with some levels of on-chip caches and interconnection networks, integrated on a singe chip. However, it brings challenges on how to program these processors effectively and efficiently, which is known as the ldquoWallrdquo. This paper proposes a systematic approach to attack problem. We describe an extension of Java programming language with dataflow paradigm and transactional memory. Our approach alleviates the difficulties of parallel programming by providing a higher level of abstraction and relieving programmers of low-level threading and locking details. We also describe the design of a runtime system to support and optimize for the extension. We have implemented a prototype based on Apache Harmony DRL Virtual Machine. Preliminary experimental results on a 16-core SMP machine show that our approach achieves reasonable scalability and can adapt to the variance of available hardware resources.

The rapidly growing mobile market has stimulated the demand for more and more complex custom applications for embedded mobile devices, for example, smart phones. To manage this complexity and, at the same time, to keep the time to market small, advanced software engineering methods have to be applied. Aspect-oriented programming (AOP) provides advanced modularization and abstraction mechanisms. The main advantages of this concept are improved maintainability, reusability, and extensibility of applications. Furthermore, dynamic AOP can be used to implement the dynamic adaptation of mobile device applications to changing contexts, like the location. However, the overhead introduced by the additional abstraction mechanisms limits the applicability to embedded mobile devices because they have limited resources (CPU, memory) compared to desktop PCs. To overcome this problem, we present a set of optimizations that significantly reduce the overhead of common AOP mechanisms and, finally, make AOP applicable for embedded mobile devices. The foundation of our work is a thorough analysis of the overhead that is typically generated by the realization of AOP mechanisms. The key idea of our approach is a deep integration of AOP mechanisms into the virtual machine. To this end, we shift mechanisms like the registration of activated aspects to the level of the JVM. Furthermore, we optimize the execution of AOP programs by introducing caching mechanisms and specialized bytecode instructions that are tailored for the execution of AOP mechanisms. Moreover, we analyze AOP-specific semantic code properties in order to develop optimizations that utilize these AOP-specific semantic information and that exploit typical AOP usage schemes. In addition to the AOP optimizations, we realize an efficient dynamic aspect deployment mechanism. We apply our optimizations to the Java-based aspect-oriented programming language ObjectTeams [HHM07] by extending the extremely small and portable JamVM [Lou] Java virtual machine. To evaluate our approach, we execute micro benchmarks, investigate the effect of our optimizations on a real-world application, and finally discuss the transferability of our optimizations to other approaches. Our evaluation shows a considerable performance gain for the aspect activation and the aspect execution of ObjectTeams. In particular, we demonstrate that our optimizations improve the performance of commonly used AOP mechanisms by up to 90%. At the same time, we reduce the code size of the adapted classes, which is also important for small devices. Finally, with our case study, namely the OTPong game application, we show that our approach is capable of significantly optimizing the execution time of real-word applications. Our main contribution is a significant reduction of the overhead of high-level AOP constructs, which is also demonstrated by the results of our experiments. The success of the optimizations provides evidence that advanced high-level abstraction techniques like AOP can be efficiently used in embedded mobile devices. Furthermore, our work shows that efficient dynamic aspect deployment can be supported on the level of the JVM. This substantially enhances the dynamic capabilities of ObjectTeams.

The high performance implementation of Java Virtual Machines (JVM) and just-in-time (JIT) compilers is directed toward adaptive compilation optimizations on the basis of online runtime profile information. This paper describes the design and implementation of a dynamic optimization framework in a production-level Java JIT compiler. Our approach is to employ a mixed mode interpreter and a three level optimizing compiler, supporting quick, full, and special optimization, each of which has a different set of tradeoffs between compilation overhead and execution speed. a lightweight sampling profiler operates continuously during the entire program's exectuion. When necessary, detailed information on runtime behavior is collected by dynmiacally generating instrumentation code which can be installed to and uninstalled from the specified recompilation target code. Value profiling with this instrumentation mechanism allows fully automatic code specialization to be performed on the basis of specific parameter values or global data at the highest optimization level. The experimental results show that our approach offers high performance and a low code expansion ratio in both program startup and steady state measurements in comparison to the compile-only approach, and that the code specialization can also contribute modest performance improvement

A dynamic real-time optimization (D-RTO) methodology has been developed and applied to a batch reactor where polymer grafting reactions take place. The objective is to determine the on-line reactor temperature profile that minimizes the batch time while meeting terminal constraints on the overall conversion rate and grafting efficiency. The methodology combines a constrained dynamic optimization method and a moving horizon state estimator within a closed-loop control. The results show very good performances in terms of state estimation, constraints fulfillment and computation load.

In the real word, optimization problems in multi-objective optimization (MOP) and dynamic optimization can be seen everywhere. During the last decade, among various swarm intelligence algorithms for multi-objective optimization problems, glowworm swarm optimization (GSO) and bacterial foraging algorithm (BFO) have attracted increasing attention from scholars. Although many scholars have proposed improvement strategies for GSO and BFO to keep a good balance between convergence and diversity, there are still many problems to be solved carefully. In this paper, a new coupling algorithm based on GSO and BFO (MGSOBFO) is proposed for solving dynamic multi-objective optimization problems (dMOP). MGSOBFO is proposed to achieve a good balance between exploration and exploitation by dividing into two parts. Part I is in charge of exploitation by GSO and Part II is in charge of exploration by BFO. At the same time, the simulation binary crossover (SBX) and polynomial mutation are introduced into the MGSOBFO to enhance the convergence and diversity ability of the algorithm. In order to show the excellent performance of the algorithm, we experimentally compare MGSOBFO with three algorithms on the benchmark function. The results suggests that such a coupling algorithm has good performance and outperforms other algorithms which deal with dMOP.

Collection data structures in standard libraries of programming languages are designed to excel for the average case by carefully balancing memory footprint and runtime performance. These implicit design decisions and hard-coded trade-offs do constrain users from using an optimal variant for a given problem. Although a wide range of specialized collections is available for the Java Virtual Machine (JVM), they introduce yet another dependency and complicate user adoption by requiring specific Application Program Interfaces (APIs) incompatible with the standard library.A product line for collection data structures would relieve library designers from optimizing for the general case. Furthermore, a product line allows evolving the potentially large code base of a collection family efficiently. The challenge is to find a small core framework for collection data structures which covers all variations without exhaustively listing them, while supporting good performance at the same time.We claim that the concept of Array Mapped Tries (AMTs) embodies a high degree of commonality in the sub-domain of immutable collection data structures. AMTs are flexible enough to cover most of the variability, while minimizing code bloat in the generator and the generated code. We implemented a Data Structure Code Generator (DSCG) that emits immutable collections based on an AMT skeleton foundation. The generated data structures outperform competitive hand-optimized implementations, and the generator still allows for customization towards specific workloads.

This chapter describes methods aimed at alleviating some of the complexities associated with sensor-based system design through the use of computer-aided design techniques. It presents a dynamic profiling and optimization platform (DPOP) capable of observing application-level behavior and dynamically tuning the underlying platform accordingly. Dynamic optimization relies upon accurate profiling results collected at runtime. There exists much research in the area of dynamic optimizations; however, most previous work has focused on the processor or memory in computer systems. Within the DPOP environment, dynamically profiling a sensor-based application requires profiling methods to be incorporated within each node to monitor the execution behavior for individual sensor nodes. The chapter shows the per-node dynamic optimization process, which is orchestrated by the dynamic optimization controller. The process consists of two operating modes: the one-shot mode, wherein the sensor-node operating state is directly determined, and the improvement mode, wherein the operating state is iteratively improved using an online optimization algorithm.

The voltage profile on a distribution circuit is becoming more difficult to maintain as more dynamic generation, such as solar, is connected. A high penetration of solar generation can lead to increased voltage deviations which, in turn, can increase the number of operations on expensive distribution automation equipment. This paper analyzed a new approach to mitigate voltage fluctuations. The voltage fluctuations studied in this paper were not from a renewable generation source, but instead were from the day to day dynamic load flow on the studied circuit. However, the performance of the technology studied in this paper gives an indication of how the technology operates with a dynamic voltage profile. The revolutionary technology was able to provide voltage support at different locations on the circuit by using dynamic reactive power injection. Results from a field installation proved that the secondary installed, voltage-controlled capacitors could not only support low voltage, but could also reduce the number of voltage deviations on the circuit.