Allocation Of Memory Research Articles (Page 1)

Abstract Efficient data ingestion and online data augmentation remain challenges in deep learning workflows, particularly when dealing with datasets containing non-standard formats or massive multidimensional arrays that natively optimised functions cannot fully manage. This work presents a parallel framework that integrates and global shared memory through a ring buffer architecture, enabling high-throughput data loading and flexible on-the-fly augmentation. The framework decouples data production from consumption, allowing multiple CPU workers to load and preprocess batches in parallel while completely bypassing the Python GIL and memory bottlenecks. Crucially, the framework supports both CPU-side and GPU-side augmentation strategies, adapting to whether complex conditional transformations or framework-native operations are required. The proposed approach was validated on two representative tasks: (i) sign language recognition from human pose CSV sequences, and (ii) hyperspectral image classification using massive arrays. Relative to standard sequential baselines, the proposed framework achieved up to $$27\times $$ 27 × acceleration in isolated data ingestion and up to $$28\times $$ 28 × in end-to-end training. Importantly, even against natively optimised parallel TensorFlow and PyTorch pipelines, it still delivered up to $$8\times $$ 8 × faster data loading and up to $$7\times $$ 7 × faster full training in memory-intensive scenarios. Overall, the proposed framework provides a scalable, multi-GPU compatible solution for deep learning pipelines, showing robust performance across both I/O-bound and memory-constrained scenarios in TensorFlow and PyTorch while alleviating memory fragmentation and allocation constraints.

The growing availability of GPU-accelerated open-source solvers has boosted the capability of tackling complex single-phase and multiphase turbulent flows by means of direct and large-eddy simulations. GPU-accelerated solvers can leverage the heterogeneous computing architectures that are available in leading high-performance computing centers worldwide, taking advantage of the higher throughput and greater energy efficiency offered by GPUs as compared to CPUs. However, porting CPU-based numerical solvers to GPUs entails many outstanding challenges, such as parallelism exposure, inter-GPU communication, memory allocation constraints, and shared memory limitations. To overcome these challenges, GPU-friendly algorithms, performance portability strategies, and careful selection of computational paradigms and programming languages must be developed. Besides, adaptive mesh refinement and data compression may be integrated to mitigate I-O bottlenecks and enable simulations of more complex geometries on top of the existing requirements imposed by incompressible flows. When compressibility effects become significant, further considerations related to the adoption of high-performance preconditioners and multigrid solvers become crucial for tackling large, sparse linear systems and extending simulations to high-Mach flows. Finally, reduced-precision arithmetic can further enhance performance, energy efficiency, and scalability. In this work, we survey current applications of GPU-accelerated solvers in the broad area of fluid mechanics and turbulence simulations and discuss the main challenges and bottlenecks associated with code porting and optimization. We then conclude our analysis with an outlook on future perspectives for enabling efficient GPU-based exascale computing of turbulence.

Water reuse and resource recovery are priority environmental goals under increasing water scarcity and climate stress. Dissolved air flotation (DAF) is widely applied in municipal, industrial, and decentralized treatment trains because fine microbubbles (MB) enhance solids removal efficiency. Accurate, low-cost characterization of MB size and rise velocity is therefore valuable for process monitoring and optimization. This study develops and validates a smartphone-based, computer-vision pipeline for laboratory-scale DAF systems. After camera calibration and lens un-distortion, each video sequence (235 frames per run) is processed through grayscale conversion, median, Gaussian, and local-Laplacian filtering, gamma correction, and Otsu thresholding, followed by morphological refinement. Circular Hough transform then identifies MB candidates, providing their diameters and centroid locations. These detections are then linked frame-to-frame using a distance-gated nearest-neighbor tracker with dynamic memory allocation to accommodate new MBs under turbulent, bubble-clustering conditions. Rise velocity is computed from interframe centroid displacement and frame interval. The system reliably tracked up to 32 microbubbles simultaneously per video. Across four operating pressure/airflow combinations, mean MB diameters ranged 95.47–216.42 µm and mean rise velocities 9.40×10³–2.76×10⁴ µm/s. The approach is low cost, computationally lightweight, and suitable for rapid MB characterization to support DAF monitoring, optimization, and research.

The desire for comprehending user behavior is significant, since social media data is growing quickly owing to user contributions, especially in light of the recent coronavirus outbreak. In the literature works, several semantic analysis methodologies are used for examining the user behavior and emotions from the corpus data. They continue to deal with the main issues of poor accuracy, a complex system, inadequate emotion recognition, and ineffectiveness. The main objective of this paper is to build a new model for sentiment analysis and opinion mining. When compared to the conventional works, the novel concept behind this work is, it adopts some unique computational methodologies for accurate sentiment prediction. Here, a new Cycle Generative Adversarial Network (GAN) based Term Memory integrated Sentiment Analysis (CyG-TMSA) framework is developed for analyzing and classifying user emotions from corpus data. The most accurate identification of emotions is made possible by the efficient generation of augmented data using the Cycle Generative Adversarial Network (GAN) model. Moreover, a classification technique based on Term Memory Allocation (TMA) is applied to accurately predict opinions derived from the data supplementation. The best way to approximate the weight value for classifying data is to use the Elite Golden Jackle Optimization (EGJO) technique, which enhances the classifier’s decision-making ability. To verify and contrast the effectiveness and results of the suggested CyG-TAMS model, multiple corpus data sets extracted from public repositories are employed. The proposed CyG-TMSA framework demonstrated great effectiveness in sentiment analysis via CycleGAN-based data augmentation, TMA for precise classification, and optimization of parameter tuning through GJO. In experimental evaluations conducted on datasets extracted from Amazon and Twitter, the model obtained high accuracy at 98% within a very short processing time of 10.12 s, significantly outclassing other conventional methods in terms of accuracy and computational efficiency.

Skewed data streams are characterized by uneven distributions in which a small fraction of elements occur with much higher frequency than others. The detection of these high-frequency elements presents significant practical challenges, particularly under stringent memory constraints, as existing detection techniques have typically relied on predefined thresholds that require significant memory usage. However, this approach is highly inefficient since not all elements require equal storage space. To address these limitations, we introduce HeavyFinder (HF), a novel lightweight network measurement architecture designed to detect high-frequency elements in skewed data. HF employs a threshold-free update strategy that enables dynamic adaptation to variable data, thereby providing greater flexibility for tracking high-frequency elements without requiring fixed thresholds. Furthermore, an included memory-light strategy enables high accuracy for non-uniform distributions, even with limited memory allocation. Experimental results showed that HF significantly improved performance in four query tasks, producing an accuracy of 99.81% when identifying the top-k elements. The average absolute error (AAE) was also reduced to 10-4 using only 100KB of memory, which was significantly lower than that of conventional methods.

Efficiently managing cloud resources is critical for balancing performance, availability, and cost in modern cloud environments. Allocating too many resources leads to unnecessary expenses, while insufficient resources can cause performance issues, bottlenecks, and downtime. This paper aims to solve these challenges with Long-Short-Term Memory (LSTM) neural networks to forecast cloud resource usage. LSTMs excel at processing time-series data, making it ideal for predicting trends in cloud resource consumption such as CPU usage, memory allocation. The project involves collecting historical cloud usage data and refining it through preprocessing to ensure accuracy and quality. Using this data, a robust LSTM model is designed and trained to provide reliable predictions of future resource demands. With accurate forecasts, cloud service providers can optimize resource allocation and implement proactive scaling strategies. The model helps ensure resources are neither over-provisioned nor underprovisioned, reducing waste while maintaining the capacity to handle workload demands. This approach leads to significant cost savings, improved system performance, and seamless scalability. Dynamic autoscaling is another benefit of this solution, enabling real-time adjustments to resource allocation based on predicted demand. This ensures that performance remains stable during usage spikes while avoiding unnecessary operational costs. To evaluate the model’s accuracy, metrics like Mean-Absolute-Error (MAE) and Root-Mean Squared-Error (RMSE) are analysed, with results showing that the LSTM model performs better than traditional forecasting techniques. This paper highlights the transformative potential of machine learning in cloud management. By accurately predicting resource needs, it offers an innovative, cost-efficient, and scalable solution for modern cloud operations, empowering organizations to meet demand reliably while keeping costs under control.

Log-structured merge (LSM) trees typically employ Bloom Filters (BFs) to prevent unnecessary disk accesses for point queries. The size of BFs can be tuned to navigate a memory vs. performance tradeoff. State-of-the-art memory allocation strategies use a worst-case model for point lookup cost to derive a closed-form solution. However, existing approaches have three limitations: (1) the number of key-value pairs to be ingested must be known a priori , (2) the closed-form solution only works for a perfectly shaped LSM tree, and (3) the model assumes a uniform query distribution . Due to these limitations, the available memory budget for BFs is sub-optimally utilized, especially when the system is under memory pressure (i.e., less than 7 bits per key). In this paper, we design Mnemosyne, a BF reallocation framework for evolving LSM trees that does not require prior workload knowledge. We use a more general query cost model that considers the access pattern per file , and we find that no system accurately maintains access statistics per file, and that simply maintaining a counter per file significantly deviates from the ground truth for evolving LSM trees. To address this, we propose Merlin, a dynamic sliding-window-based tracking mechanism that accurately captures these statistics. The upgraded Mnemosyne^+ combines Merlin with our new cost model. In our evaluation, Mnemosyne reduces query latency by up to 20% compared to RocksDB under memory pressure, and Mnemosyne^+ further improves throughput by another 10% when workloads exhibit higher skew.

Many GaussDB customers, particularly small and medium-sized enterprises (SMEs), require high transaction throughput with occasional analytical queries. HTAP systems that deploy both OLTP and OLAP engines on a single server to manage hybrid workloads have become increasingly popular among customers for achieving high cost-efficiency and data freshness. However, co-locating these systems can lead to resource contention, particularly for memory, potentially degrading overall system performance and causing Service-Level Agreements (SLA) violations. To address this issue, we propose T 2 (Twisted Twin), an adaptive memory management approach that dynamically allocates memory between OLTP and OLAP components. This approach ensures OLTP meets SLA while optimizing the efficiency of OLAP query processing. However, this is non-trivial, as memory allocation triggers a cascade of effects, including in-memory column selection and data synchronization, both critical in HTAP systems. To overcome these challenges, we introduce a Bayesian optimization framework tailored for fluctuating workloads that adjusts memory allocation responsively. Experiments conducted on the real-world HTAP system, GaussDB-HTAP, demonstrate the effectiveness and efficiency of T 2 .

Sound over-approximation methods are effective for proving the absence of errors, but inevitably produce false alarms that can hamper programmers. In contrast, under-approximation methods focus on bug detection and are free from false alarms. In this work, we present two novel proof systems designed to locate the source of errors via backward under-approximation, namely Sufficient Incorrectness Logic (SIL) and its specialization for handling memory errors, called Separation SIL. The SIL proof system is minimal, sound and complete for Lisbon triples, enabling a detailed comparison of triple-based program logics across various dimensions, including negation, approximation, execution order, and analysis objectives. More importantly, SIL lays the foundation for our main technical contribution, by distilling the inference rules of Separation SIL, a sound and (relatively) complete proof system for automated backward reasoning in programs involving pointers and dynamic memory allocation. The completeness result for Separation SIL relies on a careful crafting of both the assertion language and the rules for atomic commands.

Enabling FEM-based absolute permeability estimation in giga-voxel porous media with a single GPU

The study presents an efficient implementation of the control volume-based finite difference method (CVFDM) integrated with a line-by-line solver for stress and strain analysis. The Navier's equation was discretized for each element, yielding fifteen displacement unknowns represented in a single equation. For this study, a three-unknown formulation per element was adopted. A line-by-line solver employing the TriDiagonal Matrix Algorithm (TDMA) was utilized to solve the equations. Dynamic memory allocation for updating displacements at previous element rows, enhancing convergence speed. Variables were solved and stored contiguously along a row in each time step, the iteration continued until the desired accuracy was achieved, eliminating the need for redundant boundary condition updates and reducing overall simulation time. A finite difference method (FDM)-based stress analysis application was developed based on the novel approach proposed in this work. Numerical simulations of three problems conducted using this application demonstrate a high level of agreement with theoretical solutions. The modified CVFDM with line-by-line solver proves to be an efficient and robust approach for stress and strain analysis, providing accurate and reliable results.

SMT-based verification of low-level code requires modeling and reasoning about memory operations. Prior work has shown that optimizing memory representations is beneficial for scaling verification—pointer analysis, for example can be used to split memory into disjoint regions leading to faster SMT solving. However, these techniques are mostly designed for C and C++ programs with explicit operations for memory allocation which are not present in all languages. For instance, on the Ethereum virtual machine, memory is simply a monolithic array of bytes which can be freely accessed by Ethereum bytecode, and there is no allocation primitive. In this paper, we present a memory splitting transformation guided by a conservative memory analysis for Ethereum bytecode generated by the Solidity compiler. The analysis consists of two phases: recovering memory allocation and memory regions, followed by a pointer analysis. The goal of the analysis is to enable memory splitting which in turn speeds up verification. We implemented both the analysis and the memory splitting transformation as part of a verification tool, CertoraProver, and show that the transformation speeds up SMT solving by up to 120x and additionally mitigates 16 timeouts when used on 229 real-world smart contract verification tasks.

Migration-aware slot-based memory request scheduler to guarantee QoS in DRAM-PCM hybrid memories

End-to-End Translation Validation is the problem of verifying the executable code generated by a compiler against the corresponding input source code for a single compilation. This becomes particularly hard in the presence of dynamically-allocated local memory where addresses of local memory may be observed by the program. In the context of validating the translation of a C procedure to executable code, a validator needs to tackle constant-length local arrays, address-taken local variables, address-taken formal parameters, variable-length local arrays, procedure-call arguments (including variadic arguments), and the alloca () operator. We provide an execution model, a definition of refinement, and an algorithm to soundly convert a refinement check into first-order logic queries that an off-the-shelf SMT solver can handle efficiently. In our experiments, we perform blackbox translation validation of C procedures (with up to 100+ SLOC), involving these local memory allocation constructs, against their corresponding assembly implementations (with up to 200+ instructions) generated by an optimizing compiler with complex loop and vectorizing transformations.

Memories are thought to be stored in neural ensembles known as engrams that are specifically reactivated during memory recall. Recent studies have found that memory engrams of two events that happened close in time tend to overlap in the hippocampus and the amygdala, and these overlaps have been shown to support memory linking. It has been hypothesized that engram overlaps arise from the mechanisms that regulate memory allocation itself, involving neural excitability, but the exact process remains unclear. Indeed, most theoretical studies focus on synaptic plasticity and little is known about the role of intrinsic plasticity, which could be mediated by neural excitability and serve as a complementary mechanism for forming memory engrams. Here, we developed a rate-based recurrent neural network that includes both synaptic plasticity and neural excitability. We obtained structural and functional overlap of memory engrams for contexts that are presented close in time, consistent with experimental and computational studies. We then investigated the role of excitability in memory allocation at the network level and unveiled competitive mechanisms driven by inhibition. This work suggests mechanisms underlying the role of intrinsic excitability in memory allocation and linking, and yields predictions regarding the formation and the overlap of memory engrams.

Memory management is fundamental to the performance of all applications. On modern server architectures, an application's memory allocator needs to balance memory utilization against the ability to use 2MB huge pages, which are crucial for achieving high performance. This paper shows that prior C++ memory allocators are fundamentally limited because optimizing this trade-off depends on the knowledge of object lifetimes, which is information allocators lack. We introduce a two-step approach to attain high memory utilization in huge pages. We first introduce a novel machine-learning approach that predicts the lifetime of freshly allocated objects using the stack trace at the time of allocation and treats stack traces as natural language. We then present a fundamentally new type of memory allocator that exploits (potentially incorrect) object lifetime predictions to achieve high memory utilization at full huge page usage. In contrast to prior memory allocators that organize their heap around size classes and free lists, our allocator organizes the heap based on predicted lifetime classes and adjusts to mispredictions on the fly. We demonstrate experimentally that this learned lifetime-aware memory allocator (LLAMA) reduces fragmentation with huge pages by up to 78%. Our approach gives rise to a new methodology for applying ML in computer systems. In addition, similar space-time bin packing problems abound in computer science and we discuss how this approach has applications beyond memory allocation to a wide range of problems.

ObjectivesThis study examined whether older adults with hearing loss (HL) experience greater difficulties in auditory sentence comprehension compared to those with typical-hearing (TH) when the linguistic burdens of syntactic complexity were systematically manipulated by varying either the sentence type (active vs. passive) or sentence length (3- vs. 4-phrases).MethodsA total of 22 individuals with HL and 24 controls participated in the study, completing sentence comprehension test (SCT), standardized memory assessments, and pure-tone audiometry tests. Generalized linear mixed effects models were employed to compare the effects of sentence type and length on SCT accuracy, while Pearson correlation coefficients were conducted to explore the relationships between SCT accuracy and other factors. Additionally, stepwise regression analyses were employed to identify memory-related predictors of sentence comprehension ability.ResultsOlder adults with HL exhibited poorer performance on passive sentences than on active sentences compared to controls, while the sentence length was controlled. Greater difficulties on passive sentences were linked to working memory capacity, emerging as the most significant predictor for the comprehension of passive sentences among participants with HL.ConclusionOur findings contribute to the understanding of the linguistic-cognitive deficits linked to age-related hearing loss by demonstrating its detrimental impact on the processing of passive sentences. Cognitively healthy adults with hearing difficulties may face challenges in comprehending syntactically more complex sentences that require higher computational demands, particularly in working memory allocation.

The memory allocator plays a key role in the performance of applications, but none of the existing profilers can pinpoint performance slowdowns caused by a memory allocator. Consequently, programmers may spend time improving application code incorrectly or unnecessarily, achieving low or no performance improvement. This paper designs the first profiler—MemPerf—to identify allocator-induced performance slowdowns without comparing against another allocator. Based on the key observation that an allocator may impact the whole life-cycle of heap objects, including the accesses (or uses) of these objects, MemPerf proposes a life-cycle based detection to identify slowdowns caused by slow memory management operations and slow accesses separately. For the prior one, MemPerf proposes a thread-aware and type-aware performance modeling to identify slow management operations. For slow memory accesses, MemPerf utilizes a top-down approach to identify all possible reasons for slow memory accesses introduced by the allocator, mainly due to cache and TLB misses, and further proposes a unified method to identify them correctly and efficiently. Based on our extensive evaluation, MemPerf reports 98% medium and large allocator-reduced slowdowns (larger than 5%) correctly without reporting any false positives. MemPerf also pinpoints multiple known and unknown design issues in widely-used allocators.

Diversity metrics are widely used to better understand biodiversity patterns and help with species conservation. Taxonomic diversity alone may lack valuable information on community ecology, such as the importance of species to ecosystem functioning and evolutionary history. By combining taxonomic, functional, and phylogenetic diversity, we can complement our knowledge, particularly under climate change, by improving conservation strategies and helping to maintain biodiversity while reducing costs in the decision‐making process. However, the main tools used for computing these metrics may demand significant processing power because of their reliance on matrices, ultimately restricting their applicability. Here, we present divraster , an R package ( www.r‐project.org ) to calculate diversity metrics directly from species rasters, with the aim of reducing memory usage and processing power, especially when working with large datasets such as large areas, high resolution, or a high number of species. The main function of divraster can calculate changes in a given community between distinct temporal scenarios, such as present and future climate scenarios, working with taxonomic, functional, and phylogenetic diversity. It includes the partition of temporal beta diversity, making it particularly useful in ecological niche models and other macroecology studies. Additionally, divraster performs spatial calculations for both alpha and beta diversity, which are helpful in studies where time is not an issue. We also conducted a performance comparison with packages that provide similar functions and demonstrated that the divraster package outperforms them in most cases regarding memory allocation and processing time.

Technology social is the fastest and most up-to-date source of information. A model that can provide mapping will help in sorting out information more precisely and quickly. Public opinion in the mass media always develops quickly to talk about an issue in just a few days or even hours, so we do not know what the opinions of the people in the mass media are on the issue. In this study, the author applied topic modeling to the results of sentiment analysis on PPKM. The source of data in this study was obtained from twitter using SNScrape. The collected data was analyzed sentiment using the Long Short-term Memory (LSTM) method, so that public opinion was obtained with positive, negative, and neutral sentiments. The classification obtained from the results of the sentiment analysis process is continued with the topic modeling process using the Latent Dirichlet Allocation (LDA) method and visualized in the form of a wordcloud to find out the relationship between one topic and another. The sentiment analysis process produces a model with an accuracy rate of 90.8% and the topic modeling process successfully presents topics that are easy to interpret so that conclusions can be known about an issue.