I/O-Efficient Hierarchical Watershed Decomposition of Grid Terrain Models

PETOOL v2.0: Parabolic Equation Toolbox with evaporation duct models and real environment data

The current living standard of industrial nations causes increasing CO2 emissions, particulate matter, and noise pollution. An essential amount of these environmental issues is induced by stop-and-go traffic within cities which is seriously characterized by short-distance freight transport trips with inner-city and suburban distances. The project Smart City Logistik (SCL) strives for a practical and short-term solution to this problem by ICT-Systems for electric vehicles (EVs). But planning, monitoring and analyzing for urban area logistics can become complicated and challenging to use. Evaluating them within acceptance tests requires a lot of experiments as well as a lot of equipment. The following approach within the SCL project, funded by the German Federal Ministry for Economic Affairs and Energy (BMWi), tries to use the ICT-system as it is and connects that system through a dynamically and procedurally generated simulation environment, based on real terrain and community data.

Dynamic culling scheme is usually implemented to handle overhead caused by rendering the massive large-scale terrain data in flight simulator. However, existing culling scheme without considering altitude is not suitable for flight simulator due to additional computational overhead. To solve this problem, in this paper, we propose hybrid approach by applying two dynamic culling schemes depending on altitude. In addition, we remove unnessary computational overhead by creating different z-map resolution when aircraft changes its altitude. The proposed scheme is implemented with open graphic library and tested with real terrain data. Through the experimental results, we can recognize the improved rendering speed about 8 to 73 percents as compared to existing scheme.

In the big data era, a huge number of services has placed a fast-growing demand on the capacity of DRAM-based main memory. However, due to the high hardware cost and serious leakage power/energy consumption, the growth rate of DRAM capacity cannot meet the increased rate of the required main memory space when the energy or hardware cost is a critical concern. To tackle this issue, hybrid main-memory devices/modules have been proposed to replace the pure DRAM main memory with a hybrid main memory module that provides a large main memory space by integrating a small-sized DRAM and a large-sized non-volatile memory (NVM) into the same memory module. Although NVMs have high-density and low-cost features, they suffer from the low read/write performance and low endurance issue, compared to DRAM. Thus, inside the hybrid main-memory module, it also includes a memory management design to use DRAM as the cache of NVMs to enhance its performance and lifetime. However, it also introduces new design challenges in both the OS and the memory module. In this work, we rethink the interactivity of OS and hybrid main-memory module, and propose a cross-layer cache design that (1) utilizes the information from the operating system to optimize the hit ratio of the DRAM cache inside the memory module, and (2) takes advantage of the bulk-size (or block-based) read/write feature of NVM to minimize the time overhead on the data movement between DRAM and NVM. At the same time, this cross-layer cache design is very lightweight and only introduces limited runtime management overheads. A series of experiments was conducted to evaluate the effectiveness of the proposed cross-layer cache design. The results show that the proposed design could improve access performance for up to 88%, compared to the investigated well-known page replacement algorithms.

In mountainous environments, terrain occlusion causes non-line-of-sight (NLoS) transmission, significantly reducing the signal propagation range. To improve emergency rescue efficiency, a mobile unmanned aerial vehicle (UAV)-based integrated localization and communication (ILAC) network should be deployed to achieve optimal performance through adaptive trajectory planning and resource allocation. However, irregular and unpredictable terrain occlusions, coupled with dynamic users, make traditional optimization ineffective and reinforcement learning (RL) inefficient. To address these challenges, this paper proposes a hybrid action space soft Actor-Critic with geographic information-based state aggregation (GeoAgg-HSAC) decision-making scheme. First, an RL state aggregation method based on graph contrastive learning is designed. Through a pre-trained graph neural network (GNN), the UAV network states experiencing the same occlusion are mapped to similar low-dimensional representations. This method reduces the state dimension and allows similar states to share policy experience, thereby improving sample efficiency and accelerating convergence. A hybrid action space SAC network is then designed, which simultaneously makes decisions for continuous UAV trajectories and discrete resource allocation. Finally, a simulation environment based on real mountain terrain and wireless data is built for the experiment. The experimental results show that the proposed scheme has significant advantages for optimizing communication and localization performance.

One of the most important metrics in measuring the performance of a database system is query response time, which is composed of I/O time and CPU time. I/O time is decided by the amount of data read/write from/to disks and how the data is located on disks. CPU time is decided by how the database system performs the query operations. So if we want to reduce the query response time we can reduce either I/O time or CPU time, or both of them. We know retrieving data from disks is much slower than retrieving data from main memory. Hence, one of the common ways to reduce I/O times is clustering data on disks so that queries will access only relevant data. This paper introduces an efficient algorithm, called AutoClust, for automatic database attribute clustering (or also called automatic database vertical partitioning) for single computers as well as cluster computers. It is based on closed item sets mined from queries and their attributes using association rule mining. The paper then presents experimental results comparing the performance of AutoClust with that of a baseline algorithm on both single computers and cluster computers using the TPC-H benchmark running on major commercial database systems. The experiments show that AutoClust has better query costs for both types of computers.

Modern computing systems are overwhelmingly designed to move data to computation. This design choice goes directly against at least three key trends in computing that cause performance, scalability and energy bottlenecks: (1) data access is a key bottleneck as many important applications are increasingly data-intensive, and memory bandwidth and energy do not scale well, (2) energy consumption is a key limiter in almost all computing platforms, especially server and mobile systems, (3) data movement, especially off-chip to on-chip, is very expensive in terms of bandwidth, energy and latency, much more so than computation. These trends are especially severely-felt in the data-intensive server and energy-constrained mobile systems of today. At the same time, conventional memory technology is facing many technology scaling challenges in terms of reliability, energy, and performance. As a result, memory system architects are open to organizing memory in different ways and making it more intelligent, at the expense of higher cost. The emergence of 3D-stacked memory plus logic, the adoption of error correcting codes inside the latest DRAM chips, proliferation of different main memory standards and chips, specialized for different purposes (e.g., graphics, low-power, high bandwidth, low latency), and the necessity of designing new solutions to serious reliability and security issues, such as the RowHammer phenomenon, are an evidence of this trend. This chapter discusses recent research that aims to practically enable computation close to data, an approach we call processing-in-memory (PIM). PIM places computation mechanisms in or near where the data is stored (i.e., inside the memory chips, in the logic layer of 3D-stacked memory, or in the memory controllers), so that data movement between the computation units and memory is reduced or eliminated. While the general idea of PIM is not new, we discuss motivating trends in applications as well as memory circuits/technology that greatly exacerbate the need for enabling it in modern computing systems. We examine at least two promising new approaches to designing PIM systems to accelerate important data-intensive applications: (1) processing using memory by exploiting analog operational properties of DRAM chips to perform massively-parallel operations in memory, with low-cost changes, (2) processing near memory by exploiting 3D-stacked memory technology design to provide high memory bandwidth and low memory latency to in-memory logic. In both approaches, we describe and tackle relevant cross-layer research, design, and adoption challenges in devices, architecture, systems, and programming models. Our focus is on the development of in-memory processing designs that can be adopted in real computing platforms at low cost. We conclude by discussing work on solving key challenges to the practical adoption of PIM.

A method of computing grid-averaged solar radiative fluxes for horizontally inhomogeneous marine boundary layer cloud fields is presented. Its underlying assumptions are as follows: i) the independent pixel approximation (IPA) is applicable and ii) for regions the size of general circulation model (GCM) grid cells, frequency distributions of cloud optical depth τ can be approximated by gamma distribution functions. Equations are furnished for albedo and transmittance that, when applied to judiciously chosen spectral bands, require about three to four times as much CPU time as plane-parallel, homogeneous (PPH) two-stream approximations, which are ubiquitous to GCMs. This is not a hindrance, as two-stream solutions command typically less than 1% of a GCM's CPU consumption. This method, referred to as the gamma IPA, requires estimates of the mean and variance of τ for each applicable grid cell. Biases associated with PPH models are assessed assuming that cloud properties in GCMs are tuned to yield a...

Two performance gaps in the memory hierarchy, between CPU cache and main memory, and main memory and mass storage, will become increasingly severe bottlenecks for computing-system performance. Although it is necessary to increase memory capacity to fill these gaps, power also increases when conventional volatile memories are used. A new nonvolatile memory for this purpose has been anticipated. Storage class memory is used to fill the second gap. Many candidates exist: ReRAM, PRAM, and 3D-cross point type with resistive change RAM. However, nonvolatile last level cache (LLC) is used to fill the first gap. Advanced STT-MRAM has achieved sub-4ns read and write accesses with perpendicular magnetic tunnel junctions (p-MTJ) [1–2]. Furthermore, mature integration processes have been developed and 8Mb STT-MRAM with sub-5ns operation has shown high reliability [3]. Moreover, because of its non-volatility, STT-MRAM can reduce operation energy by more than 81% compared to SRAM for cache [1]. This paper presents STT-MRAM-based last level cache memory (LLC) including MRAM memory core, peripherals and cache logic circuits, using novel power optimization with high-speed power gating (HS-PG),considering processor architectures and cache memory accesses. The STT-MRAM-based cache has high reliability to reduce the write-error rate with novel write-verify-write. Furthermore, a read-modify-write scheme is implemented to reduce active power without penalty. Figure 7.2.1 presents a block diagram of a 4Mb STT-MRAM based cache.

Virtual memory was developed to automate the movement of program code and data between main memory and secondary storage to give the appearance of a single. This technique greatly simplified the programmer’s job, particularly when program code and data exceeded the main memory’s size. In modern operating system, secure file system can protect confidential data from unauthorized access. However, once an authorized process has accessed data from a cryptographic file system, the data can appear as plaintext in the unprotected virtual memory backing store, even after system shutdown. This paper proposes new approach for adding multilevel security capability to virtual memory management. It uses partial swap encryption based on process’s security policy. Volatile encryption keys are chosen randomly, and remain valid only for short time periods. Performance comparison shows the total overhead of proposed method.

Summary Dynamic Local Grid Refinement (DLGR) is a well-known computational method to improve the performance of reservoir simulations by dynamically adapting the grid resolution to local physical phenomena at each location in time. This enables reservoir simulators to achieve similar accuracy with only a fraction of the number of grid cells that it would otherwise utilize. This is particularly useful for IOR/EOR processes due to the small scale and complexity of the physical and chemical processes involved. A challenge in using DLGR is the adaptation of the local grid resolution in advance of dynamic changes in physical phenomena, such as moving thermal or oil displacement fronts. Failure to intercept such changes up front by a locally high resolution grid will lead to loss of numerical accuracy. A Repeat Time Step (RTS) method that iteratively repeats time steps to implicitly evaluate the grid adaptation criteria at the next time node was previously proposed. However, the results in this paper show that using the RTS method for DLGR leads on average to considerably higher computational time spent in the linear solver. As a result, it was found that it is more efficient in most cases to tighten the tolerances for the grid adaptation criteria in order to create a buffer zone of fine grid cells in regions that require high grid resolution instead of using the RTS method. This report proposes an iDLGR-ANM method in which an Adaptive Newton's Method (ANM) is used to further reduce computational overhead spent in repeat time steps performed by DLGR. The ANM leads to reduced simulation times by only considering unconverged and adjacent grid cells in each NR iteration. Furthermore, the added benefit of ANM to the RTS method is that ANM can immediately be restricted to refined grid cells and adjacent cells from the first NR iteration in the repeat time steps rather than solving the full system of linearized equations. In order to compare the performance of the RTS method with and without the ANM, eight examples are considered involving various IOR/EOR applications and numerical schemes. Results show that iDLGR-ANM is nearly as fast in terms of CPU time spent in the linear solver as DLGR without using the RTS method. Moreover, if a post-Newton material balance smoothing technique is applied, iDLGR-ANM results in production forecasts with the same accuracy as DLGR when the RTS method is used without ANM, with differences within machine accuracy.

Extract, Transform, Load (ETL) processes form the backbone of data manage- ment and consolidation in today’s data-driven enterprises with prevalent large- scale data pipelines. One of the widely used ETL tools is Microsoft SQL Server Integration Services (SSIS), yet its optimization for performance for large-scale data loads remains a challenge. As the volumes of data grow exponentially, inefficient ETL processes create bottlenecks, increased processing time, and ex- haustion of system resources. This work discusses major SSIS optimizations that minimize ETL processing time, allowing for effective and scalable data integration. One of the key areas of optimization is data flow optimization, such as lever- aging the use of the Fast Load mode in OLE DB Destination to perform batch inserts instead of row-by-row. Similarly, Bulk Insert operations can signifi- cantly reduce data movement time. Additionally, buffer size and DefaultBuffer- MaxRows tuning allows SSIS to process data in memory more efficiently, thereby minimizing disk I/O operations. Another major area of focus is source query optimization. With the utiliza- tion of indexed views, partitioned tables, and filtering in the WHERE clause, unnecessary data extraction is avoided, restricting the load on the source sys- tem. NOLOCK hints also minimize database contention in high-concurrency environments. Parallel execution of multiple operations within SSIS can also accelerate execution, with multithreading and batch processing enabling con- current data conversion. Lookup transformations, a common performance bottleneck, can be opti- mized using cache mode, where reference data is pre-loaded instead of querying the database for each row. Furthermore, replacing row-based transformations with set-based operations significantly reduces processing overhead. For incremental data loading, change tracking or CDC (Change Data Cap- ture) enables altered record processing in place of full set loads. This saves time in processing and optimizes utilization of resources. ETL logging and error- handling mechanisms play an important role as well; selective SSIS logging and event-based error-handling mechanisms can prevent performance degradation due to overlogging. Lastly, SSIS package configurations can be tuned by having proper indexing of destination tables, turning off unnecessary constraints during loading, and applying table partitioning to maximize parallel loads of data. By utilizing these SSIS optimizations, organizations can reduce ETL pro- cessing by significant quantities, optimize data pipelines, and overall enhance enterprise-level data integration performance. These approaches make large- scale big-data scale data pipelines have very low latency, thus making SSIS a more efficient and scalable solution for enterprise-level data workflows.

The vast amount of data captured by robots in large‐scale unstructured environments brings computation and storage problems to terrain modeling methods. Considering the practical terrain geometry structures, we design a large‐scale terrain modeling framework for efficiently generating the compact and accurate terrain models from incomplete and uncertain measurements. The multiresolution elevation filtering is developed to generate terrain surfaces that preserve terrain details in the bumpy areas while achieving a sparse representation over the flat areas, which simultaneously reduces the computation time, storage space, and inference error. To further reduce the high computational complexity, we propose a computation decomposition strategy by adopting the Gaussian mixture model to partition the whole terrain surface into several subsurfaces, enabling the method to scale to much larger environments. For each subsurface, we train a Gaussian process model using the local data to adapt to the local terrain structure, concurrently resolving the data incompleteness and uncertainty. Furthermore, to provide a more informative terrain model, we efficiently derive terrain gradient through a closed‐form solution by treating the generated terrain models as continuous functions, enabling more robotics applications, such as the path planning. We perform extensive experiments to evaluate our framework, including the use of two medium‐sized terrain data sets to evaluate the overall and local performance, and two large‐scale glacier terrain and mountain terrain in the area of up to 9 km2 to validate the practical applications.

Nonvolatile memory (NVM) technologies are promising candidates as the next-generation main memory due to high scalability and low energy consumption. However, the performance bottlenecks, such as high write latency and low cell endurance, still exist in NVMs. To address these problems, frequent pattern compression schemes have been widely used, which however suffer from the lack of flexibility and adaptability. In order to overcome these shortcomings, we propose a well-adaptive NVM write scheme, called dynamic frequent pattern compression (DFPC), to significantly reduce the amount of write units and extend the lifetime. Instead of only using static frequent patterns in existing FPC schemes, which are predefined and not always efficient for all applications, the idea behind DFPC is to exploit the characteristics of data distribution in execution to obtain dynamic patterns, which often appear in the real-world applications. To further improve the compression ratio, we exploit the value locality in a cache line to extend the granularity of dynamic patterns. Hence, DFPC can encode the contents of cache lines with more kinds of frequent data patterns. Moreover, to further support efficient write and read operations in the context of multilevel/triple-level cell NVMs, we need to extend the DFPC to improve performance in terms of the access latency and energy consumption. We hence propose a latency-optimized and energy-efficient compression write scheme to encode the compressed data with low energy and latency states, i.e., enhanced DFPC, thus reducing the latency and energy consumption. We implement DFPC in GEM5 with NVMain and execute the applications from SPEC CPU2006 to evaluate our scheme. Experimental results demonstrate the efficacy and efficiency of DFPC. We have released the source codes for public use at Github https://github.com/dfpcscheme/DFPCScheme .

Graphics processing units (GPUs) have been widely applied for high-performance applications, however, the performance bottleneck is the data movement from the limited physical memory. The cost of data swapping in/out significant affects the applications' performance under memory oversubscription. We propose memory contention-aware priority assignment and virtual memory management to reduce the performance degradation caused by memory contention and memory thrashing in this study. The performance of the proposed methodology was evaluated using a series of workloads and a case study, and impressive results were obtained.