The next generation GPS memory management

DMMX: Dynamic memory management extensions

A dynamic memory management system has to take care of the allocation and deallocation of memory blocks in a software system. Real-time embedded systems add some more constraints to the design and the implementation of dynamic memory management systems if compared with the PC world. An increasing number of features are added to embedded mobile devices, however, resources like dynamic memory are limited. In addition, in real-time systems, real-time deadlines must be respected and allocations and deallocations must be done in due time. In this paper we present a case study on evaluating dynamic memory management in embedded real-time systems. We have used a scenario-based approach and used a simulation environment to evaluate the performance of different dynamic memory management systems. Our contribution is to present a practical approach, the tools and the rationales to evaluate dynamic memory management in embedded real-time systems.

The recent trend towards tile-based manycore architectures has helped to tackle the memory wall by physically distributing memories and processing nodes. However, this introduced a data-to-task locality challenge and inter-tile communication thus often imposes significant software overhead. Thus, we proposed software-defined hardware-managed SHARQ queues that enable efficient inter-tile communication by leveraging user-defined queues with arbitrarily sized elements. To ensure (remote) processing of queued elements, SHARQ introduces an optional handler task, which is scheduled by hardware on demand. Queue management, intra- and inter-tile data transfer, and handler task invocation are entirely handled by hardware. Only rare tasks, like the dynamic queue creation at run-time, are performed in software. DySHARQ, an extension of SHARQ, enables dynamic and concurrent queue memory management and queue length adjustments to be able to adapt to application and resource requirement changes. The DySHARQ hardware is able to monitor the queue memory requirements at run-time and conditionally schedules a software-defined memory management task. It further optimizes the hardware-software interaction for local queue operations. We integrated DySHARQ into the MPI library used by the NAS benchmarks. The evaluation shows a reduction in execution time by up to 43% (compared to software) for the communication intense IS kernel in a 4 $$\times$$ 4 tile design on an FPGA platform with a total of 80 LEON3 cores. The dynamic memory management reduces the memory footprint by 3.75 $$\times$$ in a 2 $$\times$$ 2 design.

The dynamic memory manager (DMM) is a key element whose customization for a target application reports great benefits in terms of execution time, memory usage and energy consumption. Previous works presented algorithms to automatically obtain custom DMMs for a given application. Nevertheless, those approaches are based on grammatical evolution where the fitness is built as an aggregate objective function, which does not completely exploit the search space, returning the designer the DMM solution with best fitness. However, this approach may not find solutions that could fit in a concrete hardware platform due to a very low value of one of the objectives while the others remain high, which may represent a high fitness. In this work we present the first multi-objective optimization methodology applied to DMM optimization where the Pareto dominance is considered, thus providing the designer with a set of non-dominated DMM implementations on each optimization run. Our results show that the multi-objective optimization provides Pareto-optimal alternatives due to a better exploitation of the search space obtaining better hypervolume values than the aggregate objective function approach.

Dynamic memory management in real-time systems is difficult because most traditional allocation and deallocation algorithms are unpredictable in terms of their worst-case memory utilization and execution times. This paper describes recent work on a new memory management system designed specifically to support hard-real-time allocation, automatic garbage collection, and defragmentation of the free pool. The memory management system consists of special hardware circuits placed between the CPU's level-two caches and its memory subsystem. Benefits include worst-case allocation delays of two microseconds and typical memory utilizations that exceed binary-buddy allocators by over 30%. A commercial implementation of this technology is currently under development.

Memory allocation is the process of allocating physical or virtual memory space to computer applications and processes. There are two types of memory allocation: static memory allocation and dynamic memory allocation. Static memory allocation in C allocates memory during compile time using stack. It is mainly used when the amount of memory that is required by the program is known. Once the memory is allocated using static memory allocation then the size of the memory allocated cannot change whereas dynamic memory allocation allocates memory to a process/application during its run time. It is used when the amount of memory that a program requires is unknown. It uses a heap for managing memory and the size of the memory allocated can be altered. In this paper, a dynamic memory manager is proposed in Linux OS. The proposed dynamic memory manager is based upon the internal design and implementation of malloc() and free() library functions in C. It is capable of allocating and deallocating memory to a user program/process. It also shows the memory statistics of every object malloc()-ed by a program/process that is currently in use The proposed memory manager overcomes the problem of internal fragmentation.

As the size of embedded system software increase bigger and bigger, and it`s complexity is grower and grower, the usage of dynamic memory management scheme such collector also has been increased. Using the garbage collector, however, inherently lead us performance degradation. In order to resolve this kind of performance problem in the Java based embedded system. we introduce an explicit dynamic memory free method to the automated dynamic memory management environment. which can be performed by a programmer. In the worst case, the prosed scheme shows the same performance as the case of that only garbage collector is working, since the unclaimed garbage objects will eventually be collected later by the garbage collector. In the best case. our method is free from any runtime overhead because the applications can be implemented without any intervention of the garbage collector. Although the proposed method can be facilitated with all the existing garbage collection algorithms, it shows an outperform in the case of mark-and-sweep algorithm.

Increasing complexity and modularity of today's WSAN applications impose demanding challenges on the system design. This especially affects real-time operation, resource sharing and dynamic memory management. Preemptive task systems are one way to retain good reactivity within dynamic environments. Yet, since memory is often too rare for static assignment, this rapidly leads to severe compositional problems among tasks with interfering and even varying requirements. We present our novel CoMem approach for maintaining high reactivity and efficient memory usage in such systems. With respect to task priorities and the typically limited resources of sensor nodes, we facilitate compositional software design by providing independently developed tasks with runtime information for yet collaborative and self-reflective memory sharing. Thereby, we require no special hardware-support like MMUs but operate entirely software-based.

Dynamic memory management in C++ is complex, especially across the boundaries of library abstract data types. C++ libraries designed in the orthodox canonical form (OCF) alleviate some of the problems by ensuring that classes which manage any kind of heap structures faithfully copy and delete these. However, in certain common circumstances, OCF heap structures are wastefully copied multiple times. General reference counting is not an option in OCF, since a shared body violates the intended value semantics; although a copy-on-write policy can be made to work with borrowed heap structures. A simpler ownership policy, based on larceny, allows low-level memory manager objects to steal heap structures from temporary variables, in properly isolated circumstances. Various strategies for regulating theft are presented, ranging from pilfer-constructors to locks on heap data. Larceny has similarities with other transfer of ownership patterns, but is more a core implementation technique designed to improve the efficiency and effectiveness of OCF-conformant libraries.

A tunable hybrid memory allocator

Dynamic memory management has been a high cost component in many software systems. A study has shown that memory intensive C programs can consume up to 30% of the program run time in memory allocation and liberation. Especially, in C++ programs, they tend to have object creation and deletion prolifically. The C++ memory allocation rate can be as much as ten times higher than the comparable applications written in C. Despite the importance of dynamic memory management in C++, there exist few software tools to study dynamic memory in C++. This paper introduces a tracing tool, called mtrace++, to study the dynamic memory allocation behavior in C++ programs. mtrace++ is a source code level instrumented tracing tool which produces records of allocation and deallocation information. mtrace++ identifies the origins of allocated memories and life-spans of objects. With limited overheads, mtrace++ can help programmers to solve dynamic memory problems with affordable cost.

An integrated dynamic memory tracing tool for C++

The increasing computational complexity of DNNs achieved unprecedented successes in various areas such as machine vision and natural language processing (NLP), e.g., the recent advanced Transformer has billions of parameters. However, as large-scale DNNs significantly exceed GPU's physical memory limit, they cannot be trained by conventional methods such as data parallelism. Pipeline parallelism that partitions a large DNN into small subnets and trains them on different GPUs is a plausible solution. Unfortunately, the layer partitioning and memory management in existing pipeline parallel systems are fixed during training, making them easily impeded by out-of-memory errors and the GPU under-utilization. These drawbacks amplify when performing neural architecture search (NAS) such as the evolved Transformer, where different network architectures of Transformer needed to be trained repeatedly. vPipe is the first system that transparently provides dynamic layer partitioning and memory management for pipeline parallelism. vPipe has two unique contributions, including (1) an online algorithm for searching a near-optimal layer partitioning and memory management plan, and (2) a live layer migration protocol for re-balancing the layer distribution across a training pipeline. vPipe improved the training throughput of two notable baselines (Pipedream and GPipe) by 61.4-463.4 percent and 24.8-291.3 percent on various large DNNs and training settings.

Dynamic memory allocation plays a vital role in modern application programs. Modern lock-free memory allocators based on hardware atomic primitives usually provide good performance. However, threads may starve in these lock-free implementations, leading to unbounded worst-case execution time that is not allowed in real-time embedded systems. This article presents decentralized dynamic memory management, wfspan, based on non-linearizable wait-free lists. It employs a helping mechanism to ensure no starvation in the lock-free implementation. From the perspective of design tradeoff, wfspan guarantees bounded execution steps in both allocation and deallocation procedure, at the cost of increasing bounded worst-case memory footprint. The results of running benchmarks on an x86/64 and an aarch64 machine illustrate that wfspan achieves competitive performance and memory footprint compared to lock-based and lock-free practical memory allocators while showing superior to other allocators in terms of worst-case execution time.

In this work, we present a dynamic memory allocation technique for a novel, horizontally partitioned memory subsystem targeting contemporary embedded processors with a memory management unit (MMU). We propose to replace the on-chip instruction cache with a scratchpad memory (SPM) and a small minicache. Serializing the address translation with the actual memory access enables the memory system to access either only the SPM or the minicache. Independent of the SPM size and based solely on profiling information, a postpass optimizer classifies the code of an application binary into apageableand acacheablecode region. The latter is placed at a fixed location in the external memory and cached by the minicache. The former, the pageable code region, is copied on demand to the SPM before execution. Both the pageable code region and the SPM are logically divided into pages the size of an MMU memory page. Using the MMU's pagefault exception mechanism, a runtime scratchpad memory manager (SPMM) tracks page accesses and copies frequently executed code pages to the SPM before they get executed. In order to minimize the number of page transfers from the external memory to the SPM, good code placement techniques become more important with increasing sizes of the MMU pages. We discuss code-grouping techniques and provide an analysis of the effect of the MMU's page size on execution time, energy consumption, and external memory accesses. We show that by using the data cache as a victim buffer for the SPM, significant energy savings are possible. We evaluate our SPM allocation strategy with fifteen applications, including H.264, MP3, MPEG-4, and PGP. The proposed memory system requires 8% less die are compared to a fully-cached configuration. On average, we achieve a 31% improvement in runtime performance and a 35% reduction in energy consumption with an MMU page size of 256 bytes.