Combined partitioning and data padding for scheduling multiple loop nests

This paper addresses the scheduling problem for multi-dimensional loops applications on heterogeneous multicore processors. In the multi-dimensional loops scheduling problem, a significant issue is how to hide memory latency to reduce the schedule length. With the increasing CPU speed, the gap between the processor and memory performance is an important bottleneck for modern high-performance computer systems. To solve the bottleneck problem, a variety of techniques have been studied to hide memory latency from intermediate fast memories (caches) to various prefetching and memory management techniques. Although there are a lot of algorithms in the literature to solve the scheduling with memory management problem for multiprocessor systems, they may not deliver good quality with high performance for heterogeneous multicore processors. In this paper, we first propose a scheduling algorithm Recom_Task_Assign to reduce the write activities to main memory. Then, in conjunction with the Recom_Task_Assign algorithm, we present a new partition scheduling algorithm called heterogeneous multiprocessor partition (HMP) based on the prefetching technique for heterogeneous multicore processors, which can hide memory latencies for applications with multi-dimensional loops. This technique takes advantage of memory access pattern information and fully considers the heterogeneity of processors to achieve high processor utilization. Our HMP algorithm selects the appropriate partition size and shape according to different processors, which increases processor utilization and reduces memory latency. Experiments on DSP benchmarks show that our algorithm can efficiently reduce memory latency and enhance parallelism compared with existing methods.

Minimizing write operation for multi-dimensional DSP applications via a two-level partition technique with complete memory latency hiding

Improving data locality in programs which manipulate arrays has been the subject of a great deal of research. Much of the work on improving data locality examines individual loop nests; other work includes transformations such as loop fusion, which combines loops so that multiple loop nests can be transformed as a single loop nest. We propose a data-driven method to optimize locality across multiple loop nests. Our technique achieves loop fusion-like results even when normal loop fusion is illegal without enabling transformations. Given an array whose locality should be optimized, it also finds other calculations that can profitably be executed with the computation of that array.KeywordsTransitive ClosureArray ElementIteration SpaceLoop NestInnermost LoopThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

As applications using deep neural networks (DNNs) are increasingly deployed on mobile devices, researchers are exploring various methods to achieve low energy consumption and high performance. Recently, advances in STT-MRAM have shown promise in offering non-volatility, high performance, and low energy consumption when used to replace DRAM in main memory. Most of the memory space used in DNN applications is occupied by weight and activation data. Typically, the contents of weight data remain unchanged during inference, whereas activation data are modified to store intermediate results across the layers of the DNN. As large DNN applications consume significant energy and require high performance, STT-MRAM is an ideal candidate to fully or partially replace DRAM in main memory. However, the long write latency of STT-MRAM compared to DRAM presents a performance bottleneck. In this work, we propose a reduced retention-time MRAM-based main memory to address this issue. We divide the MRAM into multiple partitions, with each partition implemented with different retention times, tailored for DNN applications. In this approach, the DNN weights can be assigned to the long retention-time partition, while the activation data can be allocated to the short retention-time partition, optimizing for the varying characteristics of data reuse. To achieve high performance, we propose two mapping schemes: intra-segment and inter-segment circular buffers, which dynamically map DNN activation data (i.e., virtual pages of streaming data) to physical pages in a circular fashion to exploit reuse patterns of DNN data. These circular buffers are mapped to the short retention-time MRAM partition as much as possible. The intra-circular buffer mapping scheme achieves an average improvement of 14.4% in bandwidth and 12.6% in inference latency compared to DRAM for on-device DNN applications. Furthermore, the inter-circular buffer mapping scheme offers 11.1% bandwidth and 11.2% latency improvements on average using only 16 MBytes of the short retention-time partition.

Integrated Modular Avionics (IMA) architecture has emerged as the de-facto standard for hosting multiple avionic functions with different criticality levels on the same hardware platform. To further reduce size, weight, power and cost, the second generation IMA architecture aims to migrate multiple singlecore IMA partitions onto a multi-core hardware platform. In this paper, we propose a framework to safely allocate and schedule communicating, mixed-criticality IMA partitions on a cache-based multi-core platform with the added constraint that certain partition pairs should not be allocated on the same core for safety and security reasons. Simulation results demonstrate the effectiveness of our approach in allocating and scheduling partitions while respecting all constraints.

Affine loop transformations have often been used for program optimization. Usually their focus lies on single loop nests. A few recent approaches also handle global programs with multiple loop nests but they are not really scalable towards realistic applications with dozens of nests. To reduce complexity, we split affine transformations into a linear transformation step and a translation step. This translation step can be used to perform general multidimensional loop fusion. We show that loop fusion can be performed incrementally and provide a greedy algorithm, which we illustrate on a simple example. Finally, we present a heuristic for data locality and provide some experimental results.

The data intensive analytical workload becomes heavy burden for OLAP engine with increasing data volume, user population and query complexity. Large capacity random access memory, multi-level cache and multi-core hardware are main streams of computer. We propose a hardware-aware OLAP model named MOSS-DB which optimizes storage model according to data access features of dimensional tables and fact tables. A hard disk & main memory two-level storage model is employed to support directly dimensional tuple accessing join operator(DDTA-JOIN), DDTA-JOIN simplifies OLAP query processing by replacing traditional join operation with directly accessing dimensional tuple with memory address. So the star schema can be seen as virtual de-normalized table, OLAP query is also simplified to table scan, select and project operations. Query processing on sequence data structure is more suitable for multi-core parallel processing. Our proposal allows massive data DRDB(Disk Resident Database) storage technique to cooperate with MMDB(Main-Memory Database) processing technique, which breaks the main memory capacity limitation. The DDTA-JOIN operation can save cost for index, hash table, etc. For multi-core era, MOSS-DB can flexibly use parallel processing capability of CPU by dynamically dividing fact table into multiple scan partitions and gain maximum cache profit for shared dimensional data. In experiments, we measure that MOSS-DB outperforms conventional DRDB system, and it also outperforms MMDB in SSB testing.

The memory subsystem is the major performance bottleneck in terms of speed and power consumption in today's digital systems. This is especially true for application specific embedded systems where power consumption due to memory traffic, memory latency and size of the on-chip caches have a significant role in overall system performance, energy efficiency and cost. There is an urgent need of tools that help designers take informed decisions about memory subsystems for embedded applications. This paper presents a novel, fine-grained memory profiling technique that provides the designer with valuable information such as the total amount of dynamic memory requirement of an application, the most heavily accessed source level data objects, the most memory intensive portions of an application etc. Such information can aid designers to take decisions about the overall memory subsystem comprising of a number of different cache levels, scratch-pad memories and main memory. It can also be used by a compiler to perform advanced compiler controlled memory assignment techniques, and by the application programmer to optimize an application. Case studies at the end of this paper demonstrate the accuracy of our profiling technique and provide some example usage scenarios of it.

Hyperspectral remote sensing offers a powerful tool in many different application contexts. The imbalance between the high dimensionality of the data and the limited availability of training samples calls for the need to perform dimensionality reduction in practice. Among traditional dimensionality reduction techniques, feature extraction is one of the most widely used approaches due to its flexibility to transform the original spectral information into a subspace. In turn, band selection is important when the application requires preserving the original spectral information (especially the physically meaningful information) for the interpretation of the hyperspectral scene. In the case of hyperspectral image classification, both techniques need to discard most of the original features/bands in order to perform the classification using a feature set with much lower dimensionality. However, the discriminative information that allows a classifier to provide good performance is usually classdependent and the relevant information may live in weak features/bands that are usually discarded or lost through subspace transformation or band selection. As a result, in practice, it is challenging to use either feature extraction or band selection for classification purposes. Relevant lines of attack to address this problem have focused on multiple feature selection aiming at a suitable fusion of diverse features in order to provide relevant information to the classifier. In this paper, we present a new dimensionality reduction technique, called multiple criteria-based spectral partitioning, which is embedded in an ensemble learning framework to perform advanced hyperspectral image classification. Driven by the use of a multiple band priority criteria that is derived from classic band selection techniques, we obtain multiple spectral partitions from the original hyperspectral data that correspond to several band subgroups with much lower spectral dimensionality as compared with the original band set. An ensemble learning technique is then used to fuse the information from multiple features, taking advantage of the relevant information provided by each classifier. Our experimental results with two real hyperspectral images, collected by the reflective optics system imaging spectrometer (ROSIS) over the University of Pavia in Italy and the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) over the Salinas scene, reveal that our presented method, driven by multiple band priority criteria, is able to obtain better classification results compared with classic band selection techniques. This paper also discusses several possibilities for computationally efficient implementation of the proposed technique using various high-performance computing architectures.

Prefetching data ahead of use has the potential to tolerate the grow ing processor-memory performance gap by overlapping long latency memory accesses with useful computation. While sophisti cated prefetching techniques have been automated for limited domains, such as scientific codes that access dense arrays in loop nests, a similar level of success has eluded general-purpose pro grams, especially pointer-chasing codes written in languages such as C and C++. We address this problem by describing, implementing and evaluating a dynamic prefetching scheme. Our technique runs on stock hardware, is completely automatic, and works for general-purpose programs, including pointer-chasing codes written in weakly-typed languages, such as C and C++. It operates in three phases. First, the profiling phase gathers a temporal data reference profile from a running program with low-overhead. Next, the profiling is turned off and a fast analysis algorithm extracts hot data streams, which are data reference sequences that frequently repeat in the same order, from the temporal profile. Then, the system dynamically injects code at appropriate program points to detect and prefetch these hot data streams. Finally, the process enters the hibernation phase where no profiling or analysis is performed, and the program continues to execute with the added prefetch instructions. At the end of the hibernation phase, the program is de-optimized to remove the inserted checks and prefetch instructions, and control returns to the profiling phase. For long-running programs, this profile, analyze and optimize, hibernate, cycle will repeat multiple times. Our initial results from applying dynamic prefetching are promising, indicating overall execution time improvements of 5.19% for several memory-performance-limited SPECint2000 benchmarks running their largest (ref) inputs.

We have developed a superscalar RISC processor for the super technical server HITACHI SR8000. The processor includes architectural features specialized in scientific applications, in which massive amounts of data in the main memory must be processed. These features are a slide-windowed-registers scheme and a simultaneous execution of up to 16 prefetch instructions. The slide-windowed-registers scheme enables instructions of the processor to access any of 160 floating point registers (FPRs). The execution mechanism for prefetch instructions produces high efficiency of the out-of-order superscalar processor despite the long latency of the main memory. A logic simulation showed that the performance of the processor reaches over 3 floating-point operations per cycle and the memory throughput of over 12 bytes per cycle.

Introduced as a new subtask of the ImageCLEF 2010 challenge, we aim at recognizing the modality of a medical image based on its content only. Therefore, we propose to rely on a representation of images in terms of words from a visual dictionary. To this end, we introduce a very fast approach that allows the learning of implicit dictionaries which permits the construction of compact and discriminative bag of visual words. Instead of a unique computationally expensive clustering to create the dictionary, we propose a multiple random partitioning method based on Extreme Random Subspace Projection Ferns. By concatenating these multiple partitions, we can very efficiently create an implicit global quantization of the feature space and build a dictionary of visual words. Taking advantages of extreme randomization, our approach achieves very good speed performance on a real medical database, and this for a better accuracy than K-means clustering.KeywordsFeature SpaceRandom ForestVisual WordDictionary LearningModality RecognitionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The AI Engine (AIE) architecture, available in systems from mobile SoCs to server-class FPGAs, aims to efficiently execute AI/ML tasks through a two-dimensional array of compute tiles. Previous work on AIEs has explored different approaches to mapping computation across spatial arrays, but the compute kernel running on each tile has not been the focus. Additionally, the AIE-ML architecture introduces memory tiles and omits programmable logic, requiring new approaches to staging and moving data throughout the array. In this work we update analytical models developed for CPUs to produce the design of high performance kernels while introducing new model considerations such as memory structure, throughput, and latency as required by the AIE hardware. We evaluate our models by developing AIE-ML kernels for matrix multiplication in low-precision data types showing performance up to 95% of compute peak for the kernel when data resides in local memory and above 90% of compute peak when data resides in main memory.

In computer architecture, caches have primarily been viewed as a means to hide memory latency from the CPU. Cache policies have focused on anticipating the CPU's data needs, and are mostly oblivious to the main memory. In this paper, we demonstrate that the era of many-core architectures has created new main memory bottlenecks, and mandates a new approach: coordination of cache policy with main memory characteristics. Using the cache for memory optimization purposes, we propose a Virtual Write Queue which dramatically expands the memory controller's visibility of processor behavior, at low implementation overhead. Through memory-centric modification of existing policies, such as scheduled writebacks, this paper demonstrates that performance limiting effects of highly-threaded architectures can be overcome. We show that through awareness of the physical main memory layout and by focusing on writes, both read and write average latency can be shortened, memory power reduced, and overall system performance improved. Through full-system cycle-accurate simulations of SPEC cpu2006, we demonstrate that the proposed Virtual Write Queue achieves an average 10.9% system-level throughput improvement on memory-intensive workloads, along with an overall reduction of 8.7% in memory power across the whole suite.

In computer architecture, caches have primarily been viewed as a means to hide memory latency from the CPU. Cache policies have focused on anticipating the CPU's data needs, and are mostly oblivious to the main memory. In this paper, we demonstrate that the era of many-core architectures has created new main memory bottlenecks, and mandates a new approach: coordination of cache policy with main memory characteristics. Using the cache for memory optimization purposes, we propose a Virtual Write Queue which dramatically expands the memory controller's visibility of processor behavior, at low implementation overhead. Through memory-centric modification of existing policies, such as scheduled writebacks, this paper demonstrates that performance limiting effects of highly-threaded architectures can be overcome. We show that through awareness of the physical main memory layout and by focusing on writes, both read and write average latency can be shortened, memory power reduced, and overall system performance improved. Through full-system cycle-accurate simulations of SPEC cpu2006, we demonstrate that the proposed Virtual Write Queue achieves an average 10.9% system-level throughput improvement on memory-intensive workloads, along with an overall reduction of 8.7% in memory power across the whole suite.