Highly efficient parallel algorithms for solving the Bates PIDE for pricing options on a GPU

This paper presents a parallel system capable of accelerating biological sequence alignment on the graphics processing unit (GPU) grid. The GPU grid in this paper is a desktop grid system that utilizes idle GPUs and CPUs in the office and home. Our parallel implementation employs a master-worker paradigm to accelerate an OpenGL-based algorithm that runs on a single GPU. We integrate this implementation into a screensaver-based grid system that detects idle resources on which the alignment code can run. We also show some experimental results comparing our implementation with three different implementations running on a single GPU, a single CPU, or multiple CPUs. As a result, we find that a single non-dedicated GPU can provide us almost the same throughput as two dedicated CPUs in our laboratory environment, where GPU-equipped machines are ordinarily used to develop GPU applications. In a dedicated environment, the GPU-accelerated code achieves five times higher throughput than the CPU-based code. Furthermore, a linear speedup of 30.7X is observed on a 32-node cluster of dedicated GPUs. We also implement a compute unified device architecture (CUDA) based algorithm to demonstrate further acceleration.

Most Computational Fluid Dynamics (CFD) applications solve the Naiver-Stokes equations using various discretization methods like Finite Difference Method (FDM), Finite Element Method (FEM), Finite Volume Method (FVM), etc. The most compute-intensive process in CFD algorithms is the pressure Poisson solver. Solving the pressure Poisson equation requires the solution of a set of linear simultaneous equations. As the problem size/complexity increases, so does the effort and time invested in solving the equations. This study presents fast and robust iterative solvers and the acceleration of the same using the Graphics Processing Unit (GPU) architecture. Serial versions of the codes are implemented in the C language and the parallelization on GPU is achieved using Compute Unified Device Architecture (CUDA). For this study, a two-dimensional heat conduction problem is considered. FDM is used for discretization. Iterative solvers Conjugate Gradient and Multi Grid methods have been implemented in serial and parallel. The performance enhancement with single GPU and CPU system over a single CPU in terms of computing time is reported. It is seen that multigrid algorithms are superior in convergence and nearly 13 times speed-up has been obtained through GPU acceleration using CUDA, for a grid size of 2048 X 2048. The tests have been done on a cluster with Intel(R) Xeon(R) CPU E5-2670 CPU and NVIDIA K10 GPU.

Multi-phase SPH modelling of violent hydrodynamics on GPUs

Nowadays, Graph Traversal (GT) is a fundamental procedure for developing a measure in computing systems and algorithms. Graph traversal algorithms usually iterate input datasets of a graph to provide a logical convergence in every iteration. Graphic Processing Units (GPUs) have been utilized broadly as accelerators of GT due to their massive parallelism and accessibility of high-transfer speed memory. Furthermore, graph-based computation is pervasive and challenging for memory systems implementation in a specific time period. This paper provides quadratic and single GPUs, allowing developers to extend graph-based GPU computation effortlessly. The paper concentrates on the creditability and validation of sparse graphs in GPUs. Since the benchmark characteristic of Breadth-First Search (BFS) plays a critical role in graphs, this paper considers a primary characteristic benchmark for BFS computation usage. This paper explores the effectiveness and applicability of BFS for analyzing graph systems by utilizing the mentioned characteristics in BFS computation. This paper also utilizes the penalization of GPU BFS during the analysis and simulation results. In addition, using a comparative simulation result, this paper concludes that improved traversal rates speed up the computation and reduce memory usage for the single and quadratic GPU structures.

Programmable graphics processing units (GPUs) nowadays offer very high performance computing power at relatively low hardware cost and power consumption. In this paper, we present the implementation of the dynamics routine of the HIRLAM weather forecast model on the NVIDIA GeForce 9800 GX2 GPU card using the Compute Unified Device Architecture (CUDA) as parallel programming model. We converted the original Fortran to C and CUDA by hand, straightforwardly, without much concern about optimization. On a single GPU, we observe speed-ups by an order of magnitude over our hosting CPU (Intel quad core, 1998 MHz). This includes the relatively very costly copying of data between GPU and CPU memories. Calculation times proper decreased by a factor of 2000. A single GPU, however, has not enough memory for practical use. Therefore, we investigated a parallel implementation on 4 GPUs. We found a parallel speed-up of 3.6, which is not very promising if memory limitations force the use of many GPUs in parallel. We discuss several options to solve this issue.

We present a spectral crystal plasticity (CP) solver for graphics processing unit (GPU) architectures that achieves a tenfold increase in efficiency over prior GPU solvers. The approach makes use of a database containing a spectral decomposition of CP simulations performed using a conventional iterative solver over a parameter space of crystal orientations and applied velocity gradients. The key improvements in efficiency come from reducing global memory transactions, exposing more instruction-level parallelism, reducing integer instructions and performing fast range reductions on trigonometric arguments. The scheme also makes more efficient use of memory than prior work, allowing for larger problems to be solved on a single GPU. We illustrate these improvements with a simulation of 390 million crystal grains on a consumer-grade GPU, which executes at a rate of 2.72 s per strain step.

A novel parallel algorithm is presented for generating random scale-free networks using the preferential attachment model. The algorithm, named cuPPA, is custom-designed for “single instruction multiple data (SIMD)” style of parallel processing supported by modern processors such as graphical processing units (GPUs). To the best of our knowledge, our algorithm is the first to exploit GPUs, and also the fastest implementation available today, to generate scale-free networks using the preferential attachment model. A detailed performance study is presented to understand the scalability and runtime characteristics of the cuPPA algorithm. Also another version of the algorithm called cuPPA-Hash tailored for multiple GPUs is presented. On a single GPU, the original cuPPA algorithm delivers the best performance, but is challenging to port to multi-GPU implementation. For multi-GPU implementation, cuPPA-Hash has been used as the parallel algorithm to achieve a perfect linear speedup up to 4 GPUs. In one of the best cases, when executed on an NVidia GeForce 1080 GPU, the original cuPPA generates a scale-free network of two billion edges in less than 3 s. On multi-GPU platforms, cuPPA-Hash generates a scale-free network of 16 billion edges in less than 7 s using a machine consisting of 4 NVidia Tesla P100 GPUs.

Hybrid MPI and CUDA paralleled finite volume unstructured CFD simulations on a multi-GPU system

Deep Learning (DL) has become a prominent machine learning technique due to the availability of efficient computational resources in the form of Graphics Processing Units (GPUs), large-scale datasets and a variety of models. The newer generation of GPUs are being designed with special emphasis on optimizing performance for DL applications. Also, the availability of easy-to-use DL frameworks—like PyTorch and TensorFlow— has enhanced productivity of domain experts to work on their custom DL applications from diverse domains. However, existing Deep Neural Network (DNN) training approaches may not fully utilize the newly emerging powerful GPUs like the NVIDIA A100—this is the primary issue that we address in this paper. Our motivating analyses show that the GPU utilization on NVIDIA A100 can be as low as 43% using traditional DNN training approaches for small-to-medium DL models and input data size. This paper proposes AccDP—a data-parallel distributed DNN training approach—to accelerate GPU-based DL applications. AccDP exploits the Message Passing Interface (MPI) communication library coupled with the NVIDIA’s Multi-Process Service (MPS) to increase the amount of work assigned to parallel GPUs resulting in higher utilization of compute resources. We evaluate our proposed design on different small-to-medium DL models and input sizes on the state-of-the-art HPC clusters. By injecting more parallelism into DNN training using our approach, the evaluation shows up to 58% improvement in training performance on a single GPU and up to 62% on 16 GPUs compared to regular DNN training. Furthermore, we conduct an in-depth characterization to determine the impact of several DNN training factors and best practices—including the batch size and the number of data loading workers— to optimally utilize GPU devices. To the best of our knowledge, this is the first work that explores the use of MPS and MPI to maximize the utilization of GPUs in distributed DNN training.

Large-scale simulations on multiple Graphics Processing Units (GPUs) for the direct simulation Monte Carlo method

GPU acceleration of conjugate gradient method obtaining Green's function for transport-property calculation

The solution of tridiagonal system of equations using graphic processing units (GPU) is assessed. The parallel-Thomas-algorithm (PTA) is developed and the solution of PTA is compared to two known parallel algorithms, i.e. cyclic-reduction (CR) and parallel-cyclic-reduction (PCR). Lid-driven cavity problem is considered to assess these parallel approaches. This problem is also simulated using the classic Thomas algorithm that runs on a central processing unit (CPU). Runtimes and physical parameters of the mentioned GPU and CPU algorithms are compared. The results show that the speedup of CR, PCR and PTA against the CPU runtime is 4.4x,5.2x and 38.5x, respectively. Furthermore, the effect of coalesced and uncoalesced memory access to GPU global memory is examined for PTA, and a 2x-speedup is achieved for the coalesced memory access. Additionally, the PTA performance in a time dependent problem, the unsteady flow over a square, is assessed and a 9x-speedup is obtained against the CPU.

While the advances in synchrotron light sources, together with the development of focusing optics and detectors, allow nanoscale ptychographic imaging of materials and biological specimens, the corresponding experiments can yield terabyte-scale volumes of data that can impose a heavy burden on the computing platform. Although graphics processing units (GPUs) provide high performance for such large-scale ptychography datasets, a single GPU is typically insufficient for analysis and reconstruction. Several works have considered leveraging multiple GPUs to accelerate the ptychographic reconstruction. However, most of these works utilize only the Message Passing Interface to handle the communications between GPUs. This approach poses inefficiency for a hardware configuration that has multiple GPUs in a single node, especially while reconstructing a single large projection, since it provides no optimizations to handle the heterogeneous GPU interconnections containing both low-speed (e.g., PCIe) and high-speed links (e.g., NVLink). In this paper, we provide an optimized intranode multi-GPU implementation that can efficiently solve large-scale ptychographic reconstruction problems. We focus on the maximum likelihood reconstruction problem using a conjugate gradient (CG) method for the solution and propose a novel hybrid parallelization model to address the performance bottlenecks in the CG solver. Accordingly, we have developed a tool, called PtyGer (Ptychographic GPU(multiple)-based reconstruction), implementing our hybrid parallelization model design. A comprehensive evaluation verifies that PtyGer can fully preserve the original algorithm’s accuracy while achieving outstanding intranode GPU scalability.

For semiempirical electronic structure methods, solving the Roothaan-Hall equations to determine the one-electron density matrix is generally the computational bottleneck. Therefore, alternatives have been proposed to directly solve for the one-electron density matrix without the need to solve for the orbitals first. In this work, we implement an efficient dense linear algebra implementation of Niklasson's density matrix purification schemes using graphics processing units (GPUs). The computational bottleneck in these methods is the matrix-matrix multiplication needed to construct the purification polynomials, which can be accelerated by using GPUs. Of particular interest in this work is the use of consumer-grade GPUs that thrive on algorithms that maximize the amount of single-precision (FP32) operations carried out. Therefore, we present a tailored mixed-precision (MP) scheme to leverage much of the FP32 performance of these GPUs without sacrificing numerical accuracy in the self-consistent field (SCF) calculations. We demonstrate that our MP implementation is faster than LAPACK (Intel oneMKL DSYGVD) and cuSOLVER DSYGVD diagonalization-based density matrix builds for molecules with more than 1000 basis functions in combination with the semiempirical GFN2-xTB method. At the same time, the numerical precision of the energies and gradients is not significantly impacted by the MP scheme compared to a full double-precision (FP64) treatment. This gives access to significant accelerations of semiempirical calculations on commodity computing hardware. Going further, we show that our asynchronous GPU implementation enables running multiple SCFs in parallel on a single GPU, which enables us to leverage our implementation for accelerating state-of-the-art conformational sampling procedures that are based on molecular dynamics and metadynamic simulations.

Purpose: To develop a Graphics Processing Unit (GPU) based Monte Carlo (MC) code that uses a dual‐GPU system to accelerate radiographic simulation and CT imaging dose calculations. Methods: We considered two clinical cases, a chest x‐ray radiography and an abdominal CT scan. In the first case, a voxelized VIP‐Man phantom with detailed 3D anatomical information was used and an x‐ray beam of 120kVp was simulated. In the second case, a voxelized abdomen phantom derived from 120 CT slices was used, and a GE LightSpeed 16‐MDCT scanner was modeled. The CPU version of the MC code was written in C++ and run on Intel Xeon X5660 2.8GHz CPU, then translated into GPU code written in CUDA C and tested on a dual Tesla m2090 GPU system. The code was featured with automatic assignment of simulation task to multiple GPUs, as well as accurate calculation of energy‐ and material‐dependent cross‐sections. Results: Double‐precision floating point format was used for accuracy. In the first case, radiograph formation was simulated and doses to the organs listed in ICRP‐60 were calculated. When running on a single GPU, the MC GPU code was found to be x13 times faster than the CPU code and x29 times faster than MCNPX. In the second case, doses to the rectum, prostate, bladder and femoral heads were calculated. A speedup of x19 was observed compared to CPU code. These speedup factors were doubled on the dual‐GPU system. The imaging dose was benchmarked against MCNPX and a maximum difference of 1% was observed when the relative error is kept below 0.1%. Conclusions: A GPU‐based MC code was developed to simulate radiography and calculate imaging dose using detailed patient and CT scanner models. Efficiency and accuracy were both guaranteed in this code. Scalability of the code was confirmed on the dual‐GPU system.