Improving distributed workload performance by sharing both CPU and memory resources

Job migrations and network RAM are two approaches for effectively using global memory resources in a workstation cluster, aimed at reducing page faults in each local workstation and improving the overall performance of cluster computing. Using either remote executions or pre-emptive migrations, a load-sharing system is able to migrate a job from a workstation without sufficient memory space to a lightly loaded workstation with a large idle memory space for the migrated job. In a network RAM system, if a job cannot find sufficient memory space for its working sets, it utilizes idle memory space from other workstations in the cluster through remote paging. Conducting trace-driven simulations, we have compared the performance and tradeoffs of the two approaches and their impacts on job execution time and cluster scalability. Job migration-based load-sharing schemes are able to balance executions of jobs in a cluster well, while network RAM is able to satisfy data-intensive jobs which may not be migratable by sharing all the idle memory resources in a cluster. A network RAM cluster of workstations is scalable only if the network is sufficiently fast. We propose an improved load-sharing scheme by combining job migrations with network RAM for cluster computing. This scheme uses remote execution to initially allocate a job to the most lightly loaded workstation and, if necessary, network RAM to provide a larger memory space for the job than would be available otherwise. The improved scheme has the merits of both job migrations and network RAM. Our experiments show its effectiveness and scalability for cluster computing.

AbstractÐIn the past decade, the exponential growth in commodity CPU's speed has far outpaced advances in memory latency. A second trend is that CPU performance advances are not only brought by increased clock rate, but also by increasing parallelism inside the CPU. Current database systems have not yet adapted to these trends and show poor utilization of both CPU and memory resources on current hardware. In this paper, we show how these resources can be optimized for large joins and translate these insights into guidelines for future database architectures, encompassing data structures, algorithms, cost modeling, and implementation. In particular, we discuss how vertically fragmented data structures optimize cache performance on sequential data access. On the algorithmic side, we refine the partitioned hash-join with a new partitioning algorithm called radix-cluster, which is specifically designed to optimize memory access. The performance of this algorithm is quantified using a detailed analytical model that incorporates memory access costs in terms of a limited number of parameters, such as cache sizes and miss penalties. We also present a calibration tool that extracts such parameters automatically from any computer hardware. The accuracy of our models is proven by exhaustive experiments conducted with the Monet database system on three different hardware platforms. Finally, we investigate the effect of implementation techniques that optimize CPU resource usage. Our experiments show that large joins can be accelerated almost an order of magnitude on modern RISC hardware when both memory and CPU resources are optimized. Index TermsÐMain-memory databases, query processing, memory access optimization, decomposed storage model, join algorithms, implementation techniques. 1

Load balancing in a cluster system has been investigated extensively, mainly focusing on the effective usage of global CPU and memory resources. However, if a significant portion of applications running in the system is I/O-intensive, traditional load balancing policies that focus on CPU and memory usage may cause the system performance to decrease substantially. To solve this problem, a new I/O-aware load-balancing scheme with preemptive job migration is presented to sustain the high performance of a cluster with a diverse set of workload conditions. The proposed scheme dynamically detects I/O load imbalance on nodes of a cluster, and determines whether to preempt some running jobs on overloaded nodes and migrate them to other less- or under-loaded nodes. Besides balancing I/O load, the scheme takes into account both CPU and memory load sharing in clusters, thereby maintaining the same level of performance as existing schemes when I/O load is low or well balanced. Results from a trace-driven simulation show that, compared to the existing approaches that only consider I/O with nonpreemptive job migrations, the proposed schemes achieve the improvement in mean slowdown by up to a factor of 10.

Hard Disk Drive (HDD) based storage systems in high performance computing (HPC), such as Lustre, are being confronted with great challenges from massive parallel data-intensive applications, and becomes one of the most critical factors that influence applications performance. In this paper, we analyze the performance of Lustre, and statistically examine the distribution of file sizes, I/O patterns for typical data-intensive applications, as well as the utilization of CPU and memory resources in TH-1A supercomputer from the last five years. We present masFS, a novel file system for HPC that exploits available memory and SSD resources on compute nodes with little interference to applications running on the nodes. The masFS supports POSIX interface with an unique namespace and provides comprehensive file service for applications. We design and implement masFS with memory and SSD, and have deployed and evaluated it on TH-1A. Experimental results show that masFS works as general storage system and can meet the requirement of variety of I/O loads. It can achieve a speedup of 14.5x for read and 8.8x for write respectively in running benchmarks, and an average 7.6x faster I/O time in a real-world data-intensive application compared to Lustre.

We consider networks of workstations which are not only time-sharing, but also heterogeneous with a large variation in the computing power and memory capacities of different workstations. Many load sharing schemes mainly target sharing CPU resources, and have been intensively evaluated in homogeneous distributed environments. However the penalties of data accesses and movement in modern computer systems, such as page faults, have grown to the point where the overall performance of distributed systems cannot be further improved without serious considerations concerning memory resources in the design of load sharing policies. Considering both system heterogeneity and effective usage of memory resources, we design and evaluate load sharing policies in order to minimize both CPU idle times and the number of page faults in heterogeneous distributed systems. Conducting trace-driven simulations, we show that load sharing policies considering both CPU and memory resources are robust and effective in heterogeneous systems. We also show that the functionality and the nature of load sharing policies are quite independent on several memory demand distributions of workloads.

A cost effective, easy to manage, elastic and powerful resources over the internet is provided to the users by cloud environment. The resource pooling allows the use of same pool by multiple users through multi tenancy and virtualization technology. The VM Migration is the process of relocating a VM to another physical machine without shutting down the VM. The Virtual Migration is carried out for the number of reasons such as load balancing, fault tolerance and maintenance with respect to pooling and memory space management. Memory space prediction method used to predict the free memory space and allocate the memory to the new processes or the requested user. Memory allocation then becomes a selection function of selecting appropriate memory space according to the user needs a capacity. The free memory spaces are listed in the free list. It's helpful for easy allocation. During the migration phase, the contents of the VM are exposed to the network that might lead to data privacy and integrity concerns. Besides data, the space used by VM for storage and communication is also vulnerable to attackers during migration. To reduce the data loss due to attack on memory space to increase the confidentiality of the data which is migrated, here it is designed and implemented a new scheme for Virtual Machine Migration using Memory Space Prediction with compression method for storage using a cryptic methodology. The prediction based compression algorithm used to transfer data in compressed manner, it also decreases the migration time and down time and the hash algorithm used to increases the data confidentiality and security.

Many distributed in-memory data-processing systems such as Flink and Spark suffer from serious memory issues, including limited memory resources shared by many users or groups, which aggravates the competition for memory resources. If a user application is allocated with insufficient memory, significant garbage collection overheads will occur at runtime. In contrast, if the user application is provided with a memory space larger than it actually requires, memory resources will be wasted. Therefore, it is important to ensure that a user application is allocated with an appropriate memory size. In a general case, multiple applications process one specific set of data repeatedly. Often, the process logic of applications working on the same dataset is similar; for example, they can perform machine learning or graph computing tasks. This study further reveals that the process logic reflected in application programming interface and user-defined functions also affects the memory usage at runtime. Based on this observation, this paper presents a method for estimating the optimal memory size for newly submitted applications. The proposed approach was implemented on Spark. It utilizes the information of program analysis and historical applications produced when processing data to estimate a proper memory threshold for a newly submitted application based on the similarity of the data path between the new application and a historical one. The results of the experiments conducted to evaluate the method's accuracy of estimating the memory threshold and performance profit demonstrated that the proposed method is able to (1) estimate the required memory threshold with an error of 4% compared to the actual proper memory threshold, (2) guarantee the overall time overhead of estimating to be negligible compared to the execution time of a submitted job, and (3) reduce the execution time of submitted applications by up to 56% compared to when the proposed method is not applied.

Delay/Disruption Tolerant Networking is an internetworking architecture for reliable data delivery. It combined with the actual application scenarios of M/G/1 queuing model and based on the data delivery mechanism of store-and-forward and retransmission under Bundle Protocol/Licklider Transmission Protocol in DTN. The data delivery time can be divided into two parts by the last round of LTP data segments transmission process, the custody queue length model of BP data unit (i.e. bundle) in relay nodes memory space is determined by calculating the retransmission time of LTP data segments spent in different delivery parts, which can directly measure the consumption and use of relay nodes memory resources. In order to obtain a more perfect memory resources allocation strategy, the optimized length of LTP data segment and the optimized number of bundles aggregated by per LTP block are proposed as the joint optimization scheme. The simulation results show that in deep-space channel environment where bundle arrival rate and bit error rate constantly change, the joint optimization scheme proposed in this paper can always maintain the shortest average queue length, which means, occupy less memory space. Compared with the queue without optimization, the queue length of the joint optimization scheme is reduced by 76.65%, which demonstrates the superior performance of this scheme, and realizes the reasonable optimization and utilization of the memory resources of the relay nodes in DTN.

The JPEG2000 still image compression standard, while providing a remedy for the many shortcomings of its predecessor JPEG, is still slow to establish itself on the Internet. This problem is mainly due to the complexity of the COder-DECoder (CODEC) which implies its non-adoption by large firms and platforms in the field of image acquisition, processing and transmission. Indeed, the encoding and decoding process consumes a lot of CPU, memory and energy resources and takes a lot of computing time. The objective of this paper is to propose a model for decoding jpeg2000 on lightweight devices running on the Android mobile operating system. This implementation uses coroutines, which are a lightweight process model with reduced resource consumption costs compared to conventional AsyncTask threadsets. The model minimizes decoding time while minimizing CPU and memory usage, resulting in a fast and energetically economical decoded image. The results of integrating the coroutines from the main thread into the decoding process instead of the AsyncTask from the main thread produced better performance in terms of computation time, CPU and memory utilization. Indeed, the use of our model has led to a gain of around 23.41% in execution time, 9.8% in CPU utilization rate and 18.56% in memory utilization rate, compared to the model proposed in the literature which uses the threads.

FPGA behavioral synthesis has gained significant momentum recently with the growing interests in accelerating high-performance computing applications. While the latest generation of high-level synthesis (HLS) tools has made significant progress, they still lack the support for certain high-level language features such as dynamic memory allocation, despite the fact that efficiently utilization of the on-chip memory resources in FPGAs is critical to achieve the performance and power consumption target for many designs.

Modern computing platforms are increasingly complex, with multiple cores, shared caches, and NUMA architectures. Parallel applications developers have to take locality into account before they can expect good efficiency on these platforms. Thus there is a strong need for a portable tool gathering and exposing this information. The Hardware Locality project (hwloc) offers a tree representation of the hardware based on the inclusion and localities of the CPU and memory resources. It is already widely used for affinity-based task placement in high performance computing. In this article we present how hwloc is extended to describe more than computing and memory resources. Indeed, I/O device locality is becoming another important aspect of locality since high performance GPUs, network or InfiniBand interfaces possess privileged access to some of the cores and memory banks. hwloc integrates this knowledge into its topology representation and offers an interoperability API to extend existing libraries such as CUDA with locality information. We also describe how hwloc now helps process managers and batch schedulers to deal with the topology of multiple cluster nodes, together with compression for better scalability up to thousands of nodes.

Embedded applications can be structured in terms of mobile agents that are flexibly installed on available nodes. In wireless systems, such nodes typically have limited battery and memory resources; therefore it is important to place agents judiciously. In this paper we tackle the problem of placing a newcomer agent in such a system. The problem has two main components. First, enough memory space must be found or created at some node to place the agent. Second, the placement should be energy efficient. We present heuristics for tackling these two goals in a stepwise fashion, as well as a branch and bound method for achieving both goals at the same time. Our algorithms are centralized assuming a single entry point through which agents are injected into the system, with adequate knowledge of the system state and enough resources to run the proposed algorithms. The algorithms are evaluated under different simulated scenarios, and the tradeoffs across the two metrics (space, energy) are identified.

The Session Initiation Protocol (SIP) is an application-layer control protocol for creating, modifying and terminating multimedia sessions. An open issue is the control of overload that occurs when a SIP server lacks sufficient CPU and memory resources to process all messages. We prove that the problem of overload control in SIP network with a set of n servers and limited resources is in the form of NP-hard. This paper proposes a Load-Balanced Call Admission Controller (LB-CAC), based on a heuristic mathematical model to determine an optimal resource allocation in such a way that maximizes call admission rates regarding the limited resources of the SIP servers. LB-CAC determines the optimal "call admission rates" and "signaling paths" for admitted calls along optimal allocation of CPU and memory resources of the SIP servers through a new linear programming model. This happens by acquiring some critical information of SIP servers. An assessment of the numerical and experimental results demonstrates the efficiency of the proposed method.

Programmers using the C++ programming language are increasingly taught to manage memory implicitly through containers provided by the C++ standard library. However, heterogeneous programming platforms often require explicit allocation and deallocation of memory. This discrepancy in memory management strategies can be daunting and problematic for C++ developers who are not already familiar with heterogeneous programming. The C++17 standard introduces the concept of memory resources, which allow the user to control how standard library containers allocate memory; we believe that this addition to the C++17 standard is a powerful tool towards the unification of memory management for heterogeneous systems with best-practice C++ development. In this paper, we present vecmem, a library of memory resources which allows efficient and user-friendly allocation of memory on CUDA, HIP, and SYCL devices through standard C++ containers. We investigate the design and use cases of such a library, the potential performance gains over naive memory allocation, and the limitations of this memory allocation model.

Workload distribution is an important issue for user applications to obtain a good performance on software distributed shared memory (DSM) systems. The workload distribution methods that were previously proposed for DSM systems always considered only CPU power and data sharing but ignored memory availability of processors. However, the cost of memory accesses is also an important factor of determining program performance especially when processors have not sufficient memory space to cache all the data needed by program threads. Therefore, the past methods might make wrong decisions in workload distribution to degrade program performance. To resolve this problem, this paper is dedicated to adding memory resource consideration into workload distribution for software DSM systems. Our experimental result shows that one workload distribution method with memory resource consideration is indeed more effective for minimizing the execution time of DSM applications than another without memory resource consideration.