MALT: a Malloc tracker

Most application's performance is impacted by the amount of available memory. In a traditional application, which has a fixed working set size, increasing memory has a beneficial effect up until the application's working set is met. In the presence of garbage collection this relationship becomes more complex. While increasing the size of the program's heap reduces the frequency of collections, collecting a heap with memory paged to the backing store is very expensive. We first demonstrate the presence of an optimal heap size for a number of applications running on a machine with a specific configuration. We then introduce a scheme which adaptively finds this good heap size. In this scheme, we track the memory usage and number of page faults at a program's phase boundaries. Using this information, the system selects the soft heap size. By adapting itself dynamically, our scheme is independent of the underlying main memory size, code optimizations, and garbage collection algorithm. We present several experiments on real applications to show the effectiveness of our approach. Our results show that program-level heap control provides up to a factor of 7.8 overall speedup versus using the best possible fixed heap size controlled by the virtual machine on identical garbage collectors.

As the number of compute cores per chip continues to rise faster than the total amount of available memory, applications will become increasingly starved for memory storage capacity and bandwidth, making the problem of performance optimization even more critical. Also, understanding and optimizing the usage of an increasing number of hierarchical memory levels and complex cache management policies is becoming a very hard task. We propose a methodology for measuring and modeling the performance of hierarchical memories in terms of the application’s utilization of the key memory resources: capacity of a given memory level and bandwidth between two levels. This is done by actively interfering with the application’s use of these resources. The application’s sensitivity to reduced resource availability is measured by observing the effect of interference on application performance. The resulting resource-oriented model of performance both greatly simplifies application performance analysis and makes it possible to predict an application’s performance when running with various resource constraints. This is useful to predict performance for future memory-constrained architectures. This paper applies the proposed methodology to six important and well known High Performance Computing (HPC) codes to show the strength and the potential of analysis based on resource-oriented measurements.

Umpire is an open-source C++ library that provides memory and resource management functions for use with heterogeneous memory spaces. The Umpire library is being developed to simplify coordination and sharing of limited memory resources for complex integrated applications developed in the ECP/ATDM and ASC programs, and other multi-physics codes that combine many physics packages and libraries. The overarching goal of Umpire is to provide a single application-facing API for memory management on all platforms by decoupling memory operations from platform-specific memory spaces. It provides operations for: memory space discovery, memory provisioning, allocation and transfers, and introspection. Umpire development began in the Center for Applied Scientific Computing at Lawrence Livermore National Laboratory (LLNL). Umpire is funded primarily by LLNL ASC and ATMD portfolios and receives a small amount of support as part of ECP ST Project 2.3.1.06 STMP08-ISC. The source code is hosted and developed on GitHub. In this report, we explore possible integrations with other memory hierarchy management software. We examine both projects supported by ECP, as well as third-party and vendor provided tools.

With the rising demand for computational power and the increasing variety of computational scenarios, considerable interest has emerged in transforming existing CUDA programs into more general-purpose OpenCL programs, enabling them to run across diverse hardware platforms. However, manual methods, typically designed for specific applications, lack flexibility. Current automated conversion techniques also face considerable challenges, particularly in handling diverse programming interfaces, memory management, etc., and are insufficient for converting large-scale, complex CUDA projects. In this paper, we propose a novel source-to-source program transformation framework, TransCL, which automates the conversion of CUDA programs in four key aspects: source code, execution model, programming model and memory model. To achieve this, we abstract a set of conversion rules aligned with the latest CUDA standards, develop a transcoder, implement an OpenCL-compatible programming interface library, and establish a memory mapping mechanism between CUDA and OpenCL. Experiments demonstrate that TransCL provides a high level of automation in converting CUDA-based applications and is effective in handling large, complex projects such as TensorFlow. Moreover, the converted AI framework successfully conducted model training for the first time. The experiment also validates that the converted program can execute correctly across multiple platforms and demonstrate good performance.

Derivative clouds, light weight application containers provisioned in virtual machines, are becoming viable and cost-effective options for infrastructure and software-based services. Ubiquitous dynamic memory management techniques in virtualized systems are centralized at the hypervisor and are ineffective in nested derivative cloud setups. In this paper, we highlight the challenges in management of memory resources in derivative cloud systems. Hypervisor caching, an enabler of centralized disk cache management, provides flexible memory or non-volatile memory management at the hypervisor to improve the resource usage efficiency and performance of applications. Existing hypervisor caching solutions have limited effectiveness in nested setups due to their nesting agnostic design, centralized management model and lack of holistic view of memory management. We propose DoubleDecker, a decentralized disk caching framework, realized through guest OS and hypervisor cooperation, with support for efficient memory management in derivative clouds. The DoubleDecker hypervisor caching framework, an integral part of our proposed solution, provides interfaces for differentiated cache partitioning and management in nested setups and is equipped to handle both memory and SSD based caching stores. We demonstrate the flexibility of DoubleDecker to handle dynamic and changing memory provisioning requirements and its capability to simultaneously provision memory across multiple levels. Such multi-level configurations cannot be explored by centralized designs and are a key feature of DoubleDecker. Our experimentation with DoubleDecker demonstrates that application performance can be consistently improved due to the flexible policy framework for disk caching. With our setup, we report an average performance improvement of 4x and a maximum of 11x.

The paper addresses the issue of improving the performance of memory management for real time Java applications, building upon the Real-Time Specification for Java (RTSJ) from the Real-Time Java Expert Group. In a first step, a thorough analysis of the parameters influencing the performance of memory management together with ways of improvement are presented. The implementation of a memory management solution, which is compliant with the RTSJ and integrating the proposed improvements is then sketched.

Many Java programmers believe they do not have to worry about memory management because of automatic garbage collection. In fact, many Java programs run out of memory unexpectedly after performing a number of operations. A memory leak in Java is caused when an object that is no longer needed cannot be reclaimed because another object is still referring to it. Memory leaks can be difficult to solve, since the complexity of most programs prevents us from manually verifying the validity of every reference. In this paper we show a new methodology for finding the causes of memory leaks. We have identified a basic memory leak scenario which fits many important cases. In this scenario, we allow the programmer to identify a period of time in which temporary objects are expected to be created and released. Using this information we are able to identify objects that persist beyond this period and the references which are holding on to them. Scaling this methodology to real-world systems brings additional challenges. We propose a novel combination of visual syntax and reference pattern extraction to manage this additional complexity. We also describe how these techniques can be applied to a wider class of memory problems, including the exploration of large data structures. These techniques have been implemented and have been proven successful on large projects.

An important aspect of building robust systems that execute on dedicated hardware and perhaps in constrained environments is to control and manage the effects performed by program code. We present ReML, a higher-order statically-typed functional language, which allows programmers to be explicit about the effects performed by program code and in particular effects related to memory management Allowing programmers to be explicit about effects, the regions in which values reside, and the constraints under which code execute, makes programs robust to changes in the program source code and to compiler updates, including compiler optimisations. ReML is integrated with a polymorphic inference system that builds on top of region-inference, as it is implemented in the MLKit, a Standard ML compiler that uses region-based memory management as its primary memory management scheme.

Memory management is one of the most boring topics in database research. It plays a minor role in tasks like free-space management or efficient space usage. Here and there we also realize its impact on database performance when worrying about NUMA-aware memory allocation, data compacting, snapshotting, and defragmentation. But, overall, let's face it: the entire topic sounds as exciting as 'garbage collection' or 'debugging a program for memory leaks'. What if there were a technique that would promote memory management from a third class helper thingie to a first class citizen in algorithm and systems design? What if that technique turned the role of memory management in a database system (and any other data processing system) upside-down? What if that technique could be identified as a key for re-designing various core algorithms with the effect of outperforming existing state-of-the-art methods considerably? Then we would write this paper. We introduce RUMA: Rewired User-space Memory Access. It allows for physiological data management, i.e. we allow developers to freely rewire the mappings from virtual to physical memory (in user space) while at the same time exploiting the virtual memory support offered by hardware and operating system. We show that fundamental database building blocks such as array operations, partitioning, sorting, and snapshotting benefit strongly from RUMA.

We present ACRE, an Automated aspect creator, to use aspect-oriented programming (AOP) to perform memory, invariant and interferences testing for software programs written in C++. ACRE allows developers without knowledge in AOP to use aspects to test their programs without modifying the behavior of their source code. ACRE uses a domain-specific language (DSL), which statements testers insert into the source code like comments to describe the aspects to be used. The presence of DSL statements in the code does not modify the program's compilation and behavior. ACRE parses the DSL statements and automatically generates appropriate aspects that are then weaved into the source code to identify bugs due to memory leaks, incorrect algorithm implementation, or interference among threads. Thanks to the use of aspects and ACRE, testers can add or remove tests easily. Using an aspect generated by ACRE, we find a memory leak in a complex C++ software program, NOMAD, used in both industry and research. We also verify a crucial mathematical point of the algorithm behind NOMAD and collect data to find possible interference bugs, in NOMAD.

Hardware virtualization is a major component of large scale server and data center deployments due to their facilitation of server consolidation and scalability. Virtualization, however, comes at a high cost in terms of system main memory utilization. Current virtual machine (VM) memory management solutions impose a high performance penalty and are oblivious to the operating regime of the system. Therefore, there is a great need for low-impact VM memory management techniques which are aware of and reactive to current system state, to drive down the overheads of virtualization. We observe that the host machine operates under different memory pressure regimes, as the memory demand from guest VMs changes dynamically at runtime. Adapting to this runtime system state is critical to reduce the performance cost of VM memory management. In this paper, we propose a novel dynamic memory management policy called Memory Pressure Aware (MPA) ballooning. MPA ballooning dynamically allocates memory resources to each VM based on the current memory pressure regime. Moreover, MPA ballooning proactively reacts and adapts to sudden changes in memory demand from guest VMs. MPA ballooning requires neither additional hardware support, nor incurs extra minor page faults in its memory pressure estimation. We show that MPA ballooning provides an 13.2% geomean speed-up versus the current ballooning techniques across a set of application mixes running in guest VMs; often yielding performance nearly identical to that of a non-memory constrained system.

The use of a managed, type-safe languages such as Java in real-time and embedded systems can offer productivity and, in particular, safety and dependability benefits over the dominating unsafe languages at reasonable costs. A JVM that has dynamic memory-management needs to provide an implicit memory-management strategy, that is, for example, a garbage collector (GC) or stack allocation provided by the escape analysis of the JVM's compiler: Explicit management of dynamically allocated memory (i.e., by use of functions such as C's malloc() and free()) is vulnerable to programming errors such as neglected or false memory release operations causing memory leaks or dangling pointers. Such operations have the potential to break the soundness of the type system and are therefore usually not available for strongly typed languages. Type-safe languages in combination with static analyses -- which respect hardware as well as system-specific information -- can efficiently be employed to provide a runtime system including memory management (MM) that is specifically suited to an embedded application on a particular hardware device. In the context of this paper, we present novel memory-management strategy we implemented in our KESO JVM. It is a latency-aware garbage-collection algorithm called LAGC. Also, we introduce the static analyses that can assist LAGC. The application developers have to ensure that there is enough time for the GCs to run. Hardware characteristics such as soft-error proneness of the hardware or the memory layout can also be taken into consideration as demanded by the system configuration. This is achieved by integrating the GCs in the design process of the whole system just as any other user application, which is the reason why this approach is called cooperative memory management. The suggested strategies require reasonably low overhead.

In the previous chapter, we reviewed memory management in Objective-C for a non- ARC environment. But what happens when ARC is enabled? This chapter explores the changes incurred when ARC is employed. Automatic Reference Counting was briefly explained in Chapter 1, but it is really best summarized in Apple’s own words: Automatic Reference Counting (ARC) in Objective-C makes memory management the job of the compiler. By enabling ARC with the new Apple LLVM compiler, you will never need to type retain or release again, dramatically simplifying the development process, while reducing crashes and memory leaks. The compiler has a complete understanding of your objects, and releases each object the instant it is no longer used, so apps run as fast as ever, with predictable, smooth performance.1 KeywordsSource CodeObject TypeOwnership StatusStatic ArrayDynamic ArrayThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Programmers using the C++ programming language are increasingly taught to manage memory implicitly through containers provided by the C++ standard library. However, heterogeneous programming platforms often require explicit allocation and deallocation of memory. This discrepancy in memory management strategies can be daunting and problematic for C++ developers who are not already familiar with heterogeneous programming. The C++17 standard introduces the concept of memory resources, which allow the user to control how standard library containers allocate memory; we believe that this addition to the C++17 standard is a powerful tool towards the unification of memory management for heterogeneous systems with best-practice C++ development. In this paper, we present vecmem, a library of memory resources which allows efficient and user-friendly allocation of memory on CUDA, HIP, and SYCL devices through standard C++ containers. We investigate the design and use cases of such a library, the potential performance gains over naive memory allocation, and the limitations of this memory allocation model.

Memory-based learning schemes are characterized by their memorization of observed events. A memory-based learning scheme either memorizes the collected data directly or reorganizes such information and stores it distributively into a tabular memory. For the tabular type, the system requires a training process to determine the contents of the associative memory. This training process helps filter out zero-mean noise. Since the stored data are associated to pre-assigned input locations, memory management and data retrieval are easier and more efficient. Despite these merits, a drawback of tabular schemes is the difficulty in applying it to high-dimensional problems due to the curse of dimensionality. As the input dimensionality increases, the number of quantized elements in the input space increases at an exponential rate and that causes a huge demand of memory. In this paper, a dynamic tabular structure is proposed for possible relaxation of such a demand. The memorized data are organized as part of a k-d tree. Nodes in the tree, called vertices, correspond to some regularly assigned input points. Memory resource is allocated only at locations where it is needed. Being able to easily compute the vertex positions helps reduce the searching cost in data retrieval. In addition, the learning process is able to expand the tree structure into one covering the problem domain. With memory allocated based on demand, memory consumption becomes closely related to task complexity instead of input dimensionality, and is minimized. Algorithms for structure construction and training are presented in this paper. Simulation results demonstrate that the memory can be efficiently utilized. The developed scheme offers a solution for high-dimensional learning problems with a manageable size of effective input domain.