Architectural support for dynamic memory management

The memory-intensive nature of object-oriented languages such as C++ and Java has created the need for high-performance dynamic memory management. Object-oriented applications often generate higher memory intensity in the heap region. Thus, a high-performance memory manager is needed to cope with such applications. As today's VLSI technology advances, it becomes increasingly attractive to map software algorithms such as malloc(), free() and garbage collection into hardware. This paper presents a hardware design of a sweeping function (for mark-and-sweep garbage collection) that fully utilizes the advantages of combinational logic. In our scheme, the bit sweep can detect and sweep the garbage in a constant time. Bit-map marking in software can improve the cache performance and reduce number of page faults; however, it often requires several instructions to perform a single mark. In our scheme, only one hardware instruction is required per mark. Moreover, since the complexity of the sweeping phase is often higher than the marking phase, the garbage collection time may be substantially improved. The hardware complexity of the proposed scheme (bit-sweeper) is O(n), where n represents the size of the bit map.

As the size of embedded system software increase bigger and bigger, and it`s complexity is grower and grower, the usage of dynamic memory management scheme such collector also has been increased. Using the garbage collector, however, inherently lead us performance degradation. In order to resolve this kind of performance problem in the Java based embedded system. we introduce an explicit dynamic memory free method to the automated dynamic memory management environment. which can be performed by a programmer. In the worst case, the prosed scheme shows the same performance as the case of that only garbage collector is working, since the unclaimed garbage objects will eventually be collected later by the garbage collector. In the best case. our method is free from any runtime overhead because the applications can be implemented without any intervention of the garbage collector. Although the proposed method can be facilitated with all the existing garbage collection algorithms, it shows an outperform in the case of mark-and-sweep algorithm.

DMMX: Dynamic memory management extensions

Derivative clouds, light weight application containers provisioned in virtual machines, are becoming viable and cost-effective options for infrastructure and software-based services. Ubiquitous dynamic memory management techniques in virtualized systems are centralized at the hypervisor and are ineffective in nested derivative cloud setups. In this paper, we highlight the challenges in management of memory resources in derivative cloud systems. Hypervisor caching, an enabler of centralized disk cache management, provides flexible memory or non-volatile memory management at the hypervisor to improve the resource usage efficiency and performance of applications. Existing hypervisor caching solutions have limited effectiveness in nested setups due to their nesting agnostic design, centralized management model and lack of holistic view of memory management. We propose DoubleDecker, a decentralized disk caching framework, realized through guest OS and hypervisor cooperation, with support for efficient memory management in derivative clouds. The DoubleDecker hypervisor caching framework, an integral part of our proposed solution, provides interfaces for differentiated cache partitioning and management in nested setups and is equipped to handle both memory and SSD based caching stores. We demonstrate the flexibility of DoubleDecker to handle dynamic and changing memory provisioning requirements and its capability to simultaneously provision memory across multiple levels. Such multi-level configurations cannot be explored by centralized designs and are a key feature of DoubleDecker. Our experimentation with DoubleDecker demonstrates that application performance can be consistently improved due to the flexible policy framework for disk caching. With our setup, we report an average performance improvement of 4x and a maximum of 11x.

Memory allocation is the process of allocating physical or virtual memory space to computer applications and processes. There are two types of memory allocation: static memory allocation and dynamic memory allocation. Static memory allocation in C allocates memory during compile time using stack. It is mainly used when the amount of memory that is required by the program is known. Once the memory is allocated using static memory allocation then the size of the memory allocated cannot change whereas dynamic memory allocation allocates memory to a process/application during its run time. It is used when the amount of memory that a program requires is unknown. It uses a heap for managing memory and the size of the memory allocated can be altered. In this paper, a dynamic memory manager is proposed in Linux OS. The proposed dynamic memory manager is based upon the internal design and implementation of malloc() and free() library functions in C. It is capable of allocating and deallocating memory to a user program/process. It also shows the memory statistics of every object malloc()-ed by a program/process that is currently in use The proposed memory manager overcomes the problem of internal fragmentation.

The recent trend towards tile-based manycore architectures has helped to tackle the memory wall by physically distributing memories and processing nodes. However, this introduced a data-to-task locality challenge and inter-tile communication thus often imposes significant software overhead. Thus, we proposed software-defined hardware-managed SHARQ queues that enable efficient inter-tile communication by leveraging user-defined queues with arbitrarily sized elements. To ensure (remote) processing of queued elements, SHARQ introduces an optional handler task, which is scheduled by hardware on demand. Queue management, intra- and inter-tile data transfer, and handler task invocation are entirely handled by hardware. Only rare tasks, like the dynamic queue creation at run-time, are performed in software. DySHARQ, an extension of SHARQ, enables dynamic and concurrent queue memory management and queue length adjustments to be able to adapt to application and resource requirement changes. The DySHARQ hardware is able to monitor the queue memory requirements at run-time and conditionally schedules a software-defined memory management task. It further optimizes the hardware-software interaction for local queue operations. We integrated DySHARQ into the MPI library used by the NAS benchmarks. The evaluation shows a reduction in execution time by up to 43% (compared to software) for the communication intense IS kernel in a 4 $$\times$$ 4 tile design on an FPGA platform with a total of 80 LEON3 cores. The dynamic memory management reduces the memory footprint by 3.75 $$\times$$ in a 2 $$\times$$ 2 design.

A dynamic memory management system has to take care of the allocation and deallocation of memory blocks in a software system. Real-time embedded systems add some more constraints to the design and the implementation of dynamic memory management systems if compared with the PC world. An increasing number of features are added to embedded mobile devices, however, resources like dynamic memory are limited. In addition, in real-time systems, real-time deadlines must be respected and allocations and deallocations must be done in due time. In this paper we present a case study on evaluating dynamic memory management in embedded real-time systems. We have used a scenario-based approach and used a simulation environment to evaluate the performance of different dynamic memory management systems. Our contribution is to present a practical approach, the tools and the rationales to evaluate dynamic memory management in embedded real-time systems.

A software system can be called a safe-system with respect to memory, when it supports only strong-typing and it does not allow for the manual disposal of dynamic memory [2]. The first aspect guarantees that untyped, potentially dangerous operations are caught by the compiler or by run-time checks. The second issue is solved by the utilisation of an automatic memory reclamation scheme, i.e. a garbage collector.In this paper we argue that the careful choice of the programming language, along with an automatic memory reclamation scheme can optimise memory usage, while ensuring that many of the logical errors related to memory can be avoided.

Dynamic memory management has been a high cost component in many software systems. A study has shown that memory intensive C programs can consume up to 30% of the program run time in memory allocation and liberation. Especially, in C++ programs, they tend to have object creation and deletion prolifically. The C++ memory allocation rate can be as much as ten times higher than the comparable applications written in C. Despite the importance of dynamic memory management in C++, there exist few software tools to study dynamic memory in C++. This paper introduces a tracing tool, called mtrace++, to study the dynamic memory allocation behavior in C++ programs. mtrace++ is a source code level instrumented tracing tool which produces records of allocation and deallocation information. mtrace++ identifies the origins of allocated memories and life-spans of objects. With limited overheads, mtrace++ can help programmers to solve dynamic memory problems with affordable cost.

An integrated dynamic memory tracing tool for C++

Summary form only given. Modern programming languages are provided with a mechanism for automatic memory management called garbage collection. A lot of researches had been introduced in this area. On the other hand, some problems existed when applying such languages to the real-time applications. Among these problems, there was the threat of predictability and schedulability of hard real-time tasks. As a solution, different concurrent garbage collection scheduling strategies had been proposed. Unfortunately, these concurrent garbage collectors suffered from redundant usage of system memory. A latter algorithm that minimizes the worst-case system memory requirement with the schedulability of tasks not jeopardized is the deferrable server based garbage collector scheduling strategy. The deferrable server based garbage collection had been proved to surpass all other garbage collection scheduling strategies in minimizing system memory requirements under the worst-case conditions. In this paper, the performance of the deferrable server based garbage collector is explored under the actual real-time working environment. It also investigates the selection of thresholds to further reduce the memory requirement by using higher capacities of the deferrable server task. The simulation results demonstrate that the deferrable sever based garbage collector achieves better performance under actual real-time environment than other garbage collection algorithms. It also shows that using higher values for the server capacities can lead to better memory usage if a proper value of threshold is chosen.

It's been more than 50 years since John McCarthy invented garbage collection. The last 15 years or so have been the most exciting as garbage collection became mainstream and is now considered the norm for all/most modern programming languages. However, ...

Dynamic memory management in real-time systems is difficult because most traditional allocation and deallocation algorithms are unpredictable in terms of their worst-case memory utilization and execution times. This paper describes recent work on a new memory management system designed specifically to support hard-real-time allocation, automatic garbage collection, and defragmentation of the free pool. The memory management system consists of special hardware circuits placed between the CPU's level-two caches and its memory subsystem. Benefits include worst-case allocation delays of two microseconds and typical memory utilizations that exceed binary-buddy allocators by over 30%. A commercial implementation of this technology is currently under development.

Apache Spark is an increasingly popular distributed computation framework based on in-memory computations, which enables iterative or interactive applications to run faster. In Spark, memory management is the key to performance enhancement to avoid memory bloat problems. Compared with previous static memory manager, in Spark 1.6 and later versions, unified memory manager is implemented as the default memory management model, targeting to achieve optimal memory utilization by borrowing between storage and execution memory. However, the storage memory borrowed from execution memory may frequently be evicted when memory pressure arises. It is because of frequently re- computation during cache evicted and frequent garbage collection during shuffle in iterative applications. This situation will produce runtime overhead caused by garbage collection including cache eviction and cache re-computation. We propose a memory constraint strategy for unified memory manager in Spark to reduce runtime overhead caused by garbage collection by reducing the cache eviction size. We implement the strategy in Spark 1.6.1 using SparkPageRank, WordCount and GroupByTest to compare three different memory managers. Experimental results reveal that compared unified memory manager, memory constraint strategy can achieve better performance improvement with lower job runtime and garbage collection time when the dataset sizes or the iterations are increasing.

In this work, we present a dynamic memory allocation technique for a novel, horizontally partitioned memory subsystem targeting contemporary embedded processors with a memory management unit (MMU). We propose to replace the on-chip instruction cache with a scratchpad memory (SPM) and a small minicache. Serializing the address translation with the actual memory access enables the memory system to access either only the SPM or the minicache. Independent of the SPM size and based solely on profiling information, a postpass optimizer classifies the code of an application binary into apageableand acacheablecode region. The latter is placed at a fixed location in the external memory and cached by the minicache. The former, the pageable code region, is copied on demand to the SPM before execution. Both the pageable code region and the SPM are logically divided into pages the size of an MMU memory page. Using the MMU's pagefault exception mechanism, a runtime scratchpad memory manager (SPMM) tracks page accesses and copies frequently executed code pages to the SPM before they get executed. In order to minimize the number of page transfers from the external memory to the SPM, good code placement techniques become more important with increasing sizes of the MMU pages. We discuss code-grouping techniques and provide an analysis of the effect of the MMU's page size on execution time, energy consumption, and external memory accesses. We show that by using the data cache as a victim buffer for the SPM, significant energy savings are possible. We evaluate our SPM allocation strategy with fifteen applications, including H.264, MP3, MPEG-4, and PGP. The proposed memory system requires 8% less die are compared to a fully-cached configuration. On average, we achieve a 31% improvement in runtime performance and a 35% reduction in energy consumption with an MMU page size of 256 bytes.