Lock-based synchronization for GPU architectures

The popularity of general purpose Graphic Processing Unit (GPU) is largely attributed to the tremendous concurrency enabled by its underlying architecture -- single instruction multiple thread (SIMT) architecture. It keeps the context of a significant number of threads in registers to enable fast ``context switches" when the processor is stalled due to execution dependence, memory requests and etc. The SIMT architecture has a large register file evenly partitioned among all concurrent threads. Per-thread register usage determines the number of concurrent threads, which strongly affects the whole program performance. Existing register allocation techniques, extensively studied in the past several decades, are oblivious to the register contention due to the concurrent execution of many threads. They are prone to making optimization decisions that benefit single thread but degrade the whole application performance.

The most widely used parallel architectures in today's High Performance Computing systems utilize multi-core CPUs, many-core GPUs or Intel's MIC (Many Integrated Core). The effort of new algorithm and implementation development greatly influences the performance on these architectures, and the differences between their underlying ILP parallelism - namely SIMT (Single Instruction Multiple Thread) and SIMD (Single Instruction Multiple Data) - require different approaches. The aim of the work to be presented is to show how high performance can be achieved in solving multiple scalar- and block-tridiagonal system of equations. The Thomas algorithm is implemented on all three hardware platforms, and for the GPU we also implement a hybrid algorithm based on Parallel Cyclic Reduction and Thomas algorithm for solving scalar problem and a thread level, work-sharing based algorithm for block-tridiagonal problems. Performance comparisons and a discussion on efficiency are also included.

Compressive sensing 및 희소 복원 문제(sparse recovery problem)는 기존 디지털 기술의 한계를 극복할 수 있는 새로운 이론으로 많은 관심을 받고 있다. 그러나 신호 재구성에서 l1 norm 최적화 문제 해결에 많은 연산이 수행되며 따라서 병렬 처리 기법이 필요하다. 이 과정에서 무작위 행렬과 벡터 연산을 통한 변환 연산이 전체 과정 중에서 많은 부분을 차지하는데, 특히 원본 신호의 크기로 인해 이 과정에서 필요한 무작위 행렬을 메모리에 저장하기 곤란하며 계산 시 무작위 행렬의 절차적(procedural) 처리 방식이 필수적이다. 본 논문에서는 이 문제에 대한 해결책으로 단일 명령 다중 스레드(SIMT) 병렬 플랫폼 상에서 무작위 부분적 Haar 웨이블릿 변환을 절차적으로 계산할 수 있는 새로운 병렬 알고리듬을 제안한다. Many researchers expect the compressive sensing and sparse recovery problem can overcome the limitation of conventional digital techniques. However, these new approaches require to solve the l1 norm optimization problems when it comes to signal reconstruction. In the signal reconstruction process, the transform computation by multiplication of a random matrix and a vector consumes considerable computing power. To address this issue, parallel processing is applied to the optimization problems. In particular, due to huge size of original signal, it is hard to store the random matrix directly in memory, which makes one need to design a procedural approach in handling the random matrix. This paper presents a new parallel algorithm to calculate random partial Haar wavelet transform based on Single Instruction Multiple Threads (SIMT) platform.

Synthetic aperture radar processing is a complex task that involves advanced signal processing techniques and intense computational effort. While the first issue has now reached a mature stage, the question of how to produce accurately focused images in real-time, without mainframe facilities, is still under debate. The recent introduction of general-purpose graphic processing units seems to be quite promising in this view, especially for the decreased per-core cost barrier and for the affordable programming complexity. The authors explain, in this work, the main computational features of a range-Doppler Synthetic Aperture Radar (SAR) processor, trying to disclose the degree of parallelism in the operations at the light of the CUDA programming model. Given the extremely flexible structure of the Single Instruction Multiple Threads (SIMT) model, the authors show that the optimization of a SAR processing unit cannot reduce to an FFT optimization, although this is a quite extensively used kernel. Actually, it is noticeable that the most significant advantage is obtained in the range cell migration correction kernel where a complex interpolation stage is performed very efficiently exploiting the SIMT model. Performance show that, using a single Nvidia Tesla-C1060 GPU board, the obtained processing time is more than fifteen time better than our test workstation.

GPU accelerators can deliver significant improvement for DNN processing; however, their performance is limited by internal and external parameters. A well-known parameter that restricts the performance of various computing platforms in real-world setups, including GPU accelerators, is the power cap imposed usually by an external power controller. A common approach to meet the power cap constraint is using the Dynamic Voltage Frequency Scaling (DVFS) technique. However, the functionally of this technique is limited and platform-dependent. To improve the performance of DNN inference on GPU accelerators, we propose a new control knob, which is the size of input batches fed to the GPU accelerator in DNN inference applications. After evaluating the impact of this control knob on power consumption and performance of GPU accelerators and DNN inference applications, we introduce the design and implementation of a fast and lightweight runtime system, called BatchSizer. This runtime system leverages the new control knob for managing the power consumption of GPU accelerators in the presence of the power cap. Conducting several experiments using a modern GPU and several DNN models and input datasets, we show that our BatchSizer can significantly surpass the conventional DVFS technique regarding performance (up to 29%), while successfully meeting the power cap.

This paper reviews the programming landscape for parallel and network computing systems, focusing on four styles of concurrent programming models, and example languages/libraries. The four styles correspond to four scales of the targeted systems. At the smallest coprocessor scale, Single Instruction Multiple Thread (SIMT) and Compute Unified Device Architecture (CUDA) are considered. Transactional memory is discussed at the multicore or process scale. The MapReduce style is examined at the datacenter scale. At the Internet scale, Grid Service Markup Language (GSML) is reviewed, which intends to integrate resources distributed across multiple datacenters.

Owing to a new platform for high performance and general‐purpose computing, graphics processing unit (GPU) is one of the most promising candidates for faster improvement in peak processing speed, low latency and high performance. As GPUs employ multithreading to hide latency, there is a small private data cache in each single instruction multiple thread (SIMT) core. Hence, these cores communicate in many applications through the global memory. Access to this public memory takes long time and consumes large amount of power. Moreover, the memory bandwidth is limited which is quite challenging in parallel processing. The missed memory requests in last level cache that are followed by accesses to the slow off‐chip memory harm power and performance significantly. In this research, the authors introduce a light overhead mechanism to reduce off‐chip memory requests which are triggering by miss events in on‐chip caches. The authors propose a cluster‐based architecture to capture the similarity of memory requests between SIMT cores and provide data for missed requests by adjacent cores. Simulation results reveal that the proposed architecture enhances the geometric mean of instructions per cycle by 6.3% for evaluated benchmarks, whereas the maximum gain is 22%. Furthermore, the geometric mean of total energy consumption overhead is 4.8% for evaluated applications.

Locality based warp scheduling in GPGPUs

The alpha-tree, also known as the quasi-flat zone hierarchy is a widely used representation of images in Mathematical Morphology. This structure organizes the regions according to a similarity criterion into a tree, that eases the multiscale analysis of images. Many alpha-tree algorithms exist and computing this structure efficiently is still an active field of research. Indeed, the alpha-tree is commonly used in remote sensing where there is an urge for fast processing of large terabytes images. In this paper, we propose the first massively parallel alpha-tree algorithm that leverages concurrent union-find data structures to exploit the SIMT (Single Instruction Multiple Threads) programming model of GPUs. Our algorithm outperforms the State-of-the-Art parallel CPU algorithms by a factor of 10 on average on desktop computers and servers. It also opens new perspectives for using Mathematical Morphology methods on GPU pipelines.

We study a Hoare Logic to reason about GPU kernels, which are parallel programs executed on GPUs. We consider the SIMT (Single Instruction Multiple Threads) execution model, in which multiple threads execute in lockstep (that is, execute the same instruction at a time). When control branches both branches are executed sequentially but during the execution of each branch only those threads that take it are enabled; after the control converges, all threads are enabled and execute in lockstep again. In this paper we adapt Hoare Logic to the SIMT setting, by adding an extra component representing the set of enabled threads to the usual Hoare triples. It turns out that soundness and relative completeness do not hold for all programs; a difficulty arises from the fact that one thread can invalidate the loop termination condition of another thread through shared memory. We overcome this difficulty by identifying an appropriate class of programs for which soundness and relative completeness hold.KeywordsInference RuleOperational SemanticSymbolic ExecutionRelative CompletenessData RaceThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

In H.264/AVC decoder, adaptive deblocking filter is one of the most complex modules and it takes 33% of the whole decoding process time. Its optimization, not only in algorithm level but also in code level, is a very important topic. In this paper, CUDA technology is proposed for speeduping the deblocking process. First, a frame is divided into 16×16 macroblocks and/or 4×4 subblocks. Second, edges of each block are highly parallelly filtered by SIMT(single instruction multiple thread) CUDA instructions. Comparison of the time consumption by processing a 640 × 480(4:2:0) video on CUDA and CPU is carried out. The proposed CUDA approach offers almost 15× improvement on the decoding efficiency.

In this paper, we propose a novel floor planning algorithm for GPUs. Floor planning is an inherently sequential algorithm, far from the typical programs suitable for Single Instruction Multiple Thread (SIMT) style concurrency in a GPU. We propose a fundamentally different approach of exploring the floor plan solution space, where we evaluate concurrent moves on a given floor plan. We illustrate several performance optimization techniques for this algorithm in GPUs. Compared to the sequential algorithm, our techniques achieve 4-30X speedup for a range of MCNC benchmarks, while delivering comparable or better solution quality.

We report the development and implementation of electronic excited-state capabilities for semiempirical quantum chemical methods at both the Configuration Interaction Singles and Time-Dependent Hartree-Fock levels of theory, integrated within the PYSEQM 2.0 software package (https://github.com/lanl/PYSEQM). PYSEQM is a Python-based package designed for efficient and scalable quantum chemical simulations. Leveraging the PyTorch framework enables PYSEQM to benefit from automatic differentiation and GPU acceleration, leading to substantial performance gains in molecular property evaluations. In particular, our implementation enables efficient calculation of excited-state properties for large molecular systems. Benchmarking on systems with up to a thousand atoms demonstrates that excited-state computations can be completed in under a minute on modern GPUs, making this approach particularly suitable for high-throughput screening, real-time feedback in interactive simulations, and large-scale dynamical studies. Additionally, PYSEQM includes a machine learning interface that supports Hamiltonian parameter reoptimization and neural network training. These capabilities open new avenues for data-driven excited-state dynamics simulations, offering a path toward combining quantum chemical rigor with machine learning efficiency. Overall, this work facilitates access to excited-state quantum chemistry for large systems, while laying the foundation for future hybrid quantum-machine-learning approaches in photochemistry, photophysics, and materials discovery.

Employing hardware accelerators to improve the performance and energy-efficiency of DNN applications is on the rise. One challenge of using hardware accelerators, including the GPU-based ones, is that their performance is limited by internal and external factors, such as power caps. A common approach to meet the power cap constraint is using the Dynamic Voltage Frequency Scaling (DVFS) technique. However, the functionally of this technique is limited and platform-dependent. To tackle this challenge, we propose a new control knob, which is the size of input batches fed to the GPU accelerator in DNN inference applications. We first evaluate the impact of batch size on power consumption and performance of DNN inference. Then, we introduce the design and implementation of a fast and lightweight runtime system, called BatchDVFS. Dynamic batching is implemented in <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">BatchDVFS</i> to adaptively change the batch size, and hence, trade-off throughput with power consumption. It employs an approach based on binary search to find the proper batch size within a short period of time. Combining dynamic batching with the DVFS technique, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">BatchDVFS</i> can control the power consumption in wider ranges, and hence, yield higher throughput in the presence of power caps. To find near-optimal solution for long-running jobs that can afford a relatively significant profiling overhead, compared with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">BatchDVFS</i> overhead, we also design an approach, called BOBD, that employs Bayesian Optimization to wisely explore the vast state space resulted by combination of the batch size and DVFS solutions. Conducting several experiments using a modern GPU and several DNN models and input datasets, we show that our <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">BatchDVFS</i> can significantly surpass the techniques solely based on DVFS or batching, regarding throughput (up to 11.2x and 2.2x, respectively), while successfully meeting the power cap.

More than half of the Top 10 supercomputing sites worldwide use GPU accelerators and they are becoming ubiquitous in workstations and edge computing devices. GeNN is a C++ library for generating efficient spiking neural network simulation code for GPUs. However, until now, the full flexibility of GeNN could only be harnessed by writing model descriptions and simulation code in C++. Here we present PyGeNN, a Python package which exposes all of GeNN's functionality to Python with minimal overhead. This provides an alternative, arguably more user-friendly, way of using GeNN and allows modelers to use GeNN within the growing Python-based machine learning and computational neuroscience ecosystems. In addition, we demonstrate that, in both Python and C++ GeNN simulations, the overheads of recording spiking data can strongly affect runtimes and show how a new spike recording system can reduce these overheads by up to 10×. Using the new recording system, we demonstrate that by using PyGeNN on a modern GPU, we can simulate a full-scale model of a cortical column faster even than real-time neuromorphic systems. Finally, we show that long simulations of a smaller model with complex stimuli and a custom three-factor learning rule defined in PyGeNN can be simulated almost two orders of magnitude faster than real-time.