Resource Contention Detection in Virtualized Environments

OF THE DISSERTATION Application-Specific Memory Subsystems by Joseph G. Wingbermuehle Doctor of Philosophy in Computer Science Washington University in St. Louis, 2015 Professor Roger D. Chamberlain, Chair The disparity in performance between processors and main memories has led computer architects to incorporate large cache hierarchies in modern computers. These cache hierarchies are designed to be general-purpose in that they strive to provide the best possible performance across a wide range of applications. However, such a memory subsystem does not necessarily provide the best possible performance for a particular application. Although general-purpose memory subsystems are desirable when the work-load is unknown and the memory subsystem must remain fixed, when this is not the case a custom memory subsystem may be beneficial. For example, in an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA) designed to run a particular application, a custom memory subsystem optimized for that application would be desirable. In addition, when there are tunable parameters in the memory subsystem, it may make sense to change these parameters depending on the application being run. Such a situation arises today with FPGAs and, to a lesser extent, GPUs, and it is plausible that general-purpose computers will begin to support greater flexibility in the memory subsystem in the future. In this dissertation, we first show that it is possible to create application-specific memory subsystems that provide much better performance than a general-purpose memory subsystem. In addition, we show a way to discover such memory subsystems automatically using a superoptimization technique on memory address traces gathered from applications. This allows one to generate a custom memory subsystem with little effort. xi We next show that our memory subsystem superoptimization technique can be used to optimize for objectives other than performance. As an example, we show that it is possible to reduce the number of writes to the main memory, which can be useful for main memories with limited write durability, such as flash or Phase-Change Memory (PCM). Finally, we show how to superoptimize memory subsystems for streaming applications, which are a class of parallel applications. In particular, we show that, through the use of ScalaPipe, we can author and deploy streaming applications targeting FPGAs with superoptimized memory subsystems. ScalaPipe is a domain-specific language (DSL) embedded in the Scala programming language for generating streaming applications that can be implemented on CPUs and FPGAs. Using the ScalaPipe implementation, we are able to demonstrate actual performance improvements using the superoptimized memory subsystem with applications implemented in hardware.

Shared resource contention has been a major performance issue for CMPs. In this paper, we tackle the power contention problem in power constrained CMPs by considering and treating power as a first-class shared resource. Power contention occurs when multiple processes compete for power, and leads to degraded system performance. In order to solve this problem, we develop a shared resource contention-aware scheduling algorithm that mitigates the contention for power and the shared memory subsystem at the same time. The proposed scheduler improves system performance by balancing the shared resource usage among scheduling groups. Evaluation results across a variety of multiprogrammed workloads show performance improvements over a state-of-the-art scheduling policy which only considers memory subsystem contention.

VM3: Measuring, modeling and managing VM shared resources

Horizontally Partitioned Caches (HPCs) are a promising architectural feature to reduce the energy consumption of the memory subsystem. However, the energy reduction obtained using HPC architectures is very sensitive to the HPC parameters. Therefore it is very important to explore the HPC design space and carefully choose the HPC parameters that result in minimum energy consumption for the application. However, since in HPC architectures, the compiler has a significant impact on the energy consumption of the memory subsystem, it is extremely important to include compiler while deciding the HPC design parameters. While there has been no previous approaches to HPC design exploration, existing cache design space exploration methodologies do not include the compiler effects during DSE. In this paper, we present a Compiler-in- the-Loop (CIL) Design Space Exploration (DSE) methodology to explore and decide the HPC design parameters. Our experimental results on HP iPAQ h4300-like memory subsystem running benchmarks from the MiBench suite demonstrate that CIL DSE can discover HPC configurations with up to 80% lesser energy consumption than the HPC configuration in the iPAQ. In contrast, tradition simulation-only exploration can discover HPC design parameters that result in only 57% memory subsystem energy reduction. Finally our hybrid CIL DSE heuristic saves 67% of the exploration time as compared to the exhaustive exploration, while providing maximum possible energy savings on our set of benchmarks.

In this work, we present an initial step towards enabling TT scheduling on a real COTS multicore platform P4080. It takes into account inter-core interferences in the on-chip network and the memory sub-system. We propose an approach comprising a runtime mechanism and an offline phase. For the runtime mechanism, we propose two servers running on each core-processing time server and memory access server implemented using built-in hardware monitors. Jointly, the two servers on each core, enforce slot-level offline computed server budget reservations, thereby limiting the maximum inter-core interferences introduced and experienced by each task considering different inter-core interference latencies. In the offline phase, we propose a procedure that can be used by any offline scheduler to compute the bound on variability in execution time of each task while allowing different slot-level memory access server budget reservations. We also did a preliminary bare-metal implementation of our proposed runtime mechanism on a real COTS multicore platform P4080. Overall, our proposed method facilitates integration of COTS multicore platforms in TT systems, while maintaining features of TT architecture like slot-level determinism, clock synchronization, etc.

While multicore processors improve overall chip throughput and hardware utilization, resource sharing among the cores leads to unpredictable performance for the individual threads running on a multicore processor. Unpredictable per-thread performance becomes a problem when considered in the context of multicore scheduling: system software assumes that all threads make equal progress, however, this is not what the hardware provides. This may lead to problems at the system level such as missed deadlines, reduced quality-of-service, non-satisfied service-level agreements, unbalanced parallel performance, priority inversion, unpredictable interactive performance, etc. This article proposes a hardware-efficient per-thread cycle accounting architecture for multicore processors. The counter architecture tracks per-thread progress in a multicore processor, detects how inter-thread interference affects per-thread performance, and predicts the execution time for each thread if run in isolation. The counter architecture captures the effects of additional conflict misses due to cache sharing as well as increased latency for other memory accesses due to resource and bandwidth contention in the memory subsystem. The proposed method accounts for 74.3% of the interference cycles, and estimates per-thread progress within 14.2% on average across a large set of multi-program workloads. Hardware cost is limited to 7.44KB for an 8-core processor, a reduction by almost 10× compared to prior work while being 63.8% more accurate. Making system software progress aware improves fairness by 22.5% on average over progress-agnostic scheduling.

The assumptions and procedures of engineering design are fundamentally different from those of business applications and result in quite different needs with regard to database system requirements. One important difference is in the assumptions regarding integrity. In business, it is assumed that a database has complete integrity during all quiescent periods. In engineering design, in contrast, integrity does not exist for a database until design is almost complete. In a previous paper, we developed a revised definition of transaction that explicitly allows partial integrity. A management scheme, called entity state transaction management, was proposed that deals with both the partial ordering of transactions and the contention for shared information resources. The extension of this technique for application to large engineering problems is explored in this paper, with the development of a significant example.

A topology-aware load balancing algorithm for clustered hierarchical multi-core machines

Cloud computing is a consolidation environment that hosts heterogeneous applications. Currently, clouds use virtualization technologies to provide an isolated execution environment and agile resource provisioning. Furthermore, live migration is widely exploited to achieve energy saving. However, consolidation and live migration can cause resource contention and utilization overhead that influence application performance. In this paper, we implement a network model and a memory subsystem model into CloudSim simulator. Then, we present a thorough analysis of the High Performance Computing NPB suite including memory bandwidth demand, communication patterns between processes, and migration overhead. To study the performance of the NPB Benchmark suite, we simulate the behaviour of these benchmarks including multi-thread communication via shared memory, multi-process communication via network. Thus, this allows implementing efficient VMs scheduling and resource provisioning policies. The results show that using CPU utilization as a trigger to perform management action such as VM migration is misleading and might aggravate application performance. Furthermore, we present a power model including the utilization of CPU, memory-bus, and network. This model provides a precise estimation of power consumption.

Virtual machine (VM) migration is a technique for transferring the execution state of a VM from one physical host to another. While VM migration is critical for load balancing, consolidation, and server maintenance in virtualized data centers, it can also increase security risks. During VM migration, an attacker with sufficient privileges can compromise a VM by modifying its memory contents during transit to subvert its applications or the guest operating system. One could maintain dedicated, and presumably more secure, control networks to carry the migration traffic, but at significant hardware and administrative complexity. Alternatively, one could encrypt the migration traffic, which eliminates the need for dedicated control networks, but might introduce performance overheads. To date, there has been no systematic study of how encryption affects VM migration, especially in high-bandwidth low-delay networks that are common within data centers. In this paper, we present a study of the impact of AES and 3DES encryption algorithms on two widely used live VM migration approaches -- pre-copy and post-copy. Our key findings are as follows. The encryption algorithm used can have a significant impact on the total migration time. The impact of encryption on downtime varies with the type of the migration technique. The overhead of encryption also depends upon the relative speeds of source and target machines. Finally, an application's performance within a VM during encrypted migration varies with the type of the application and the migration mechanism.

This research aims to apply Service Oriented Architecture (SOA) to mobile applications to improve the flexibility, interoperability, and scalability of the system. SOA is an approach that enables better integration between different components in an application architecture. Mobile applications, as an integral part of everyday life, require high flexibility and adaptability to changing user needs. The research method used involves application requirements analysis, SOA architecture design, and implementation on an existing mobile application. Through the implementation of SOA, resource management and service distribution become more efficient, making it easy to add or change features without affecting the entire system. Test results show improved application performance and responsiveness, along with the ability to easily integrate services from multiple sources. In addition, the SOA architecture provides flexibility in the development and maintenance of mobile applications, accelerating the development process and minimizing the impact of changes. This research contributes to the development of mobile applications by introducing the SOA approach as an architectural foundation. The implementation of SOA in mobile applications is expected to guide developers in creating systems that are more resilient, adaptive, and easy to integrate with rapidly evolving software environments.

Network processors (NPs), a type of multicore, multithread system that exploits system-on-chip techniques, usually adopt a distributed, shared memory hierarchy to scale up network processing. Network applications running on NP often display distinct characteristics on memory subsystem compared with traditional processors with multilevel cache. As security, especially cryptography, has become indispensable in today's networks, we present a benchmark suite called NPCryptBench. It is the first benchmark specially designed to evaluate cryptographic performance on NPs, particularly that of processor and memory subsystem. By running NPCryptBench on Intel IXP network processors, we provide quantitative evidence that performance bottlenecks under such a highly parallel environment reside in memory hierarchy and prolonged access latency, instead of limited computing resources described in recent literature. To alleviate this memory-wall effect, we suggest several software optimizations, which averagely boost the benchmark throughput by 145% on Intel IXP2800. Our analysis and optimizations are also suitable for other NPs with similar architectures.

Over the decades, robotics technology has acquired sufficient advancement through the progression of 5G Internet, Artificial Intelligence (AI), Internet of Things (IoT), Cloud, and Edge Computing. Though nowadays, Cobot and Service Oriented Architecture (SOA) supported robots with edge computing paradigms have achieved remarkable performances in diverse applications, the existing SOA robotics technology fails to develop a multi-domain expert with high performing robots and demands improvement to Service-Oriented Brain, SOB (including AI model, driving service application and metadata) enabling robot for deploying brain and a new computing model with more scalability and flexibility. In this paper, instead of focusing on SOA and Robot as a Service (RaaS) model, we propose a novel computing architecture, addressed as Brainware Computing, for driving multiple domain-specific brains one-at-a-time in a single hardware robot according to the service, addressed as Brain as a Service (BaaS). In Brainware Computing, each robot can install and remove the virtual machine, which contains SOB and operating applications from the nearest edge cloud. Secondly, we provide an extensive explanation of the scope and possibilities of Brainware Computing. Finally, we demonstrate several challenges and opportunities and then concluded with future research directions in the field of Brainware Computing.

One of the key decisions made by both MapReduce and HPC cluster management frameworks is the placement of jobs within a cluster. To make this decision, they consider factors like resource constraints within a node or the proximity of data to a process. However, they fail to account for the degree of collocation on the cluster's nodes. A tight process placement can create contention for the intra-node shared resources, such as shared caches, memory, disk, or network bandwidth. A loose placement would create less contention, but exacerbate network delays and increase cluster-wide power consumption. Finding the best job placement is challenging, because among many possible placements, we need to find one that gives us an acceptable trade-off between performance and power consumption. We propose to tackle the problem via multi-objective optimization. Our solution is able to balance conflicting objectives specified by the user and efficiently find a suitable job placement.

Emerging applications today operate on increasingly larger data sets, and as a result, the memory subsystem is prone to be a bottleneck in bandwidth and capacity. Many of these applications store data based on real world phenomena which often have continuity properties and can be approximated by mathematical functions. In this work, we view computer memory abstractly as a set of functions that map each address input to a data output. We propose Approximate Algebraic Memory (A2M), a specialized memory model that uses finite degree polynomials to approximate ranges of memory content with the desired properties. Specifically, A2M uses dedicated hardware to derive and store polynomial coefficients rather than memory data. In error resilient workloads, the benefits of this design is threefold: (1) high ratio memory compression, (2) high bandwidth accesses, and (3) direct computation on memory. We evaluate A2M’s potential as an on-chip structure for general-purpose processors, and as a specialized read-only memory for neural network accelerators. Our results show that for CPU workloads, A2M yields minimal error (< 1%) at a fixed compression ratio of 16, and improves performance by 11.3% on average.