A Python/Fortran Implementation of the Lattice‐Boltzmann Kernel on Multiple GPU Using the OpenACC Framework

In the context of extremely cold or hot climates, ground coupled heat pumps offer several environmental and energy efficiency advantages over conventional methods for heating and cooling of buildings and they are increasingly used in most near-zero and net-zero energy buildings. However, the high initial cost of the ground loop portion (geothermal boreholes) has often raised the question on the economical competitiveness of the system. In the present study, a geothermal borehole is proposed in which two-phase carbon dioxide (mixture of liquid and vapor) exchanges heat with the ground to improve the thermal performance of the borehole and thus to reduce borehole length. Carbon dioxide shows several cost and environmental advantages. Moreover, it offers superior thermophysical properties and heat transfer characteristics. A numerical model has been developed to study the complex thermal behavior of a two-phase CO2-filled vertical geothermal borehole. The model can handle both two-phase and single-phase conditions along the borehole length. An explicit solution for fully coupled conservation equations of mass, momentum and energy as well as an equation of state are applied. The model accounts for the thermal interaction among the pipes and it predicts the fluid temperature, pressure and two-phase quality profiles. It is used to assess the thermal performance of the CO2-filled secondary loop geothermal borehole operating in heating mode. Results indicate that the proposed borehole offers superior performance due to the relatively high two-phase heat transfer characteristics of CO2.

Computing the solution of linear systems of equations is invariably the most time consuming task in the numerical solutions of PDEs in many fields of computational science. In this study, we focus on the numerical simulation of cardiovascular hemodynamics with rigid and deformable walls, discretized in space and time through the variational multiscale finite element method. We focus on three approaches: the problem agnostic generalized minimum residual (GMRES) and stabilized bi-conjugate gradient (BICGS) methods, and a recently proposed, problem specific, bi-partitioned (BIPN) method. We also perform a comparative analysis of several preconditioners, including diagonal, block-diagonal, incomplete factorization, multigrid, and resistance based methods. Solver performance and matrix characteristics (diagonal dominance, symmetry, sparsity, bandwidth and spectral properties) are first examined for an idealized cylindrical geometry with physiologic boundary conditions and then successively tested on several patient-specific anatomies representative of realistic cardiovascular simulation problems. Incomplete factorization preconditioners provide the best performance and results in terms of both strong and weak scalability. The BIPN method was found to outperform other methods in patient-specific models with rigid walls. In models with deformable walls, BIPN was outperformed by BICG with diagonal and Incomplete LU preconditioners.

With the ever-increasing compute demands of artificial intelligence (AI) workloads, there is extensive interest in leveraging field-programmable gate-arrays (FPGAs) to quickly deploy hardware accelerators for the latest convolutional neural networks (CNNs). Recent FPGA architectures are also evolving to better serve the needs of AI, but accelerators need extensive re-design to leverage these new features. The Stratix 10 NX chip by Intel is a new FPGA that replaces traditional DSP blocks with in-fabric AI tensor blocks that provide 15x more multipliers and up to 143 TOPS of performance, at the cost of lower precision (INT8) and significant restrictions on how many operands can be fed to the multipliers from the programmable routing. In this paper, we explore different CNN accelerator structures to leverage the tensor blocks, considering the various tensor block modes, operand bandwidth restrictions, and on-chip memory restrictions. We incorporate the most performant techniques into HPIPE, a layer-pipelined and sparse-aware CNN accelerator for FPGAs. We enhance HPIPE's software compiler to restructure the CNN computations and on-chip memory layout to take advantage of the additional multipliers offered by the new tensor block architecture, while also avoiding stalls due to data loading restrictions. We achieve cycle-by-cycle speedups in tensor mode of up to <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{8}.\mathbf{3}\mathbf{x}$</tex> for Mobilenet-v1 versus the original HPIPE design using conventional DSPs. On the FPGA, we achieve a throughput of 28,541 and 29,429 images/s on Mobilenet-v1 and Mobilenet-v2 respectively, outperforming all previous FPGA accelerators by at least 4.0x, including one on an AI-optimized Xilinx chip. We also outperform NVIDIA's V100 GPU, a machine learning targeted GPU on a similar process node with a <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{1}.\mathbf{7}\mathbf{x}$</tex> larger die size, by up to 17x with a batch size of one and 1.3x with NVIDIA's largest reported batch size of 128.

Optimal Decision Trees for Feature Based Parameter Tuning: Integer Programming Model and VNS Heuristic

Power consumption is one of the top limiting factors in high-performance computing systems and data centers, and dynamic voltage and frequency scaling (DVFS) is an important mechanism to control power. Existing works using DVFS to improve GPU energy efficiency suffer from the limitation that their policies either impact performance too much or require offline application profiling or code modification, which severely limits their applicability on large clusters. To address this issue, we propose a novel GPU DVFS policy, GEEPAFS, which improves the energy efficiency of GPUs while providing performance assurance. GEEPAFS is application-transparent as it does not require any offline profiling or code modification on user applications. To achieve this, GEEPAFS models application performance online based on our quantitative analysis of a correlation between performance and GPU memory bandwidth utilization. Based on their relationship, GEEPAFS builds a fold-line frequency-performance model for applications being executed, and it applies the model to guide the setting of GPU frequency to maximize energy efficiency while ensuring the performance loss is bounded. Through experiments on NVIDIA V100 and A100 GPUs, we show that GEEPAFS is able to improve the energy efficiency by 26.7% and 20.2% on average. While achieving this improvement, the average performance loss is only 5.8%, and the worst-case performance loss is 12.5% among all 33 tested applications.

Existing studies show that using single GPU can lead to obtaining significant performance gains. We should be able to achieve further performance speedup if we use more than one GPU. Heterogeneous processors consisting of multiple CPUs and GPUs offer immense potential and are often considered as a leading candidate for porting complex scientific applications. Unfortunately programming heterogeneous systems requires more effort than what is required for traditional multicore systems. Directive-based programming approaches are being widely adopted since they make it easy to use/port/maintain application code. OpenMP and OpenACC are two popular models used to port applications to accelerators. However, neither of the models provides support for multiple GPUs. A plausible solution is to use combination of OpenMP and OpenACC that forms a hybrid model; however, building this model has its own limitations due to lack of necessary compilers’ support. Moreover, the model also lacks support for direct device-to-device communication. To overcome these limitations, an alternate strategy is to extend OpenACC by proposing and developing extensions that follow a task-based implementation for supporting multiple GPUs. We critically analyze the applicability of the hybrid model approach and evaluate the proposed strategy using several case studies and demonstrate their effectiveness.

Optimizing makespan and resource utilization for multi-DNN training in GPU cluster

HNSC (High order Navier-Stokes simulator for Compressible flow) is an aerodynamics numerical simulation application that involves high computational cost. This paper presents our efforts porting and optimizing HNSC on the heterogeneous architecture consists of multi-core CPUs and NVIDIA GPUs. The basic GPU parallelization is done with the CUDA Fortran based on the characteristics of structured mesh and fourth-order Runge-Kutta time marching scheme. A data packing method is proposed to optimize the CPU-GPU data transfer and to improve the MPI communication efficiency. Performance evaluation is done on a server with two 12-core Intel Xeon Skylake Gold 5118 CPUs and one NVIDIA Tesla V100 GPU. The results show that the application can be significantly benefited from the GPU acceleration. The GPU version, using the V100 GPU, achieves a maximum performance speedup of 9.3 over the MPI implementation on the two Skylake CPUs.

Chapter 40 - GPU Acceleration of Iterative Digital Breast Tomosynthesis

Golang (Go) is emerging as a compelling alternative to Python for artificial intelligence applications that prioritize performance, concurrency, and scalability. While Python maintains dominance in research and model development due to its extensive ecosystem of specialized libraries, Go offers significant advantages in production environments. This article explores the growing intersection between Go and AI, examining its performance benefits, memory efficiency, and deployment simplicity. The discussion covers key libraries like Gorgonia and GoLearn, practical implementation patterns including microservices architecture and hybrid language approaches, and real-world case studies demonstrating Go's effectiveness for recommendation engines, NLP services, and edge computing. Current limitations regarding ecosystem maturity and GPU acceleration are balanced against promising future directions that position Go as an increasingly viable option for production-focused AI systems.

With machine learning being applied to increasingly varied domains, the computational needs of researchers have increased proportionately. Hobbyists, researchers and universities are turning to building their own cluster computers to meet their high performance compute needs. These clusters are typically highly efficient, low cost ARM based platforms consisting of between 4 and 8 nodes. In this paper, we present PHINEAS: Parallel Heterogeneous INdigenous Embedded ARM System, a parallel compute platform which allows for distributed computation using MPI and OpenMP and which further leverages the on-board GPU to perform general purpose compute tasks. We describe the hardware components of the cluster, the software stack installed on each node and a host of common benchmark algorithms and their results. The results show that the cluster meets the stringent latency requirements of embedded systems. We further describe how the on-board GPU’s OpenGL ES 2.0 programming model can be used to implement tasks such as image convolution and neural network inference which are common in intelligent embedded systems. Parallelisation of compute tasks across multiple GPUs is discussed as a method to combine the advantages of distributed and heterogeneous computing.

Application programming for modern heterogeneous systems which comprise multiple accelerators (multi-core CPUs and GPUs) is complex and error-prone. Popular approaches, like OpenCL and CUDA, are low-level and offer no support for the two most complicated issues: 1) programming multiple GPUs within a stand-alone computer, and 2) managing distributed systems that integrate several such computers. In particular, distributed systems require application developers to use a mix of different programming models, e.g., MPI together with OpenCL or CUDA. We propose a uniform approach based on OpenCL for programming both stand-alone and distributed systems with GPUs. The approach implementation is based on two parts: 1) the SkelCL library for high-level application programming on heterogeneous stand-alone computers with multi-core CPUs and multiple GPUs, and 2) the dOpenCL middleware for transparent execution of OpenCL programs on several stand-alone computers connected over a network.

Modern computer systems feature multiple homogeneous or heterogeneous computing units with deep memory hierarchies, and expect a high degree of thread-level parallelism from the software. Exploitation of data locality is critical to achieving scalable parallelism, but adds a significant dimension of complexity to performance optimization of parallel programs. This is especially true for programming models where locality is implicit and opaque to programmers. In this paper, we introduce the hierarchical place tree (HPT) model as a portable abstraction for task parallelism and data movement. The HPT model supports co-allocation of data and computation at multiple levels of a memory hierarchy. It can be viewed as a generalization of concepts from the Sequoia and X10 programming models, resulting in capabilities that are not supported by either. Compared to Sequoia, HPT supports three kinds of data movement in a memory hierarchy rather than just explicit data transfer between adjacent levels, as well as dynamic task scheduling rather than static task assignment. Compared to X10, HPT provides a hierarchical notion of places for both computation and data mapping. We describe our work-in-progress on implementing the HPT model in the Habanero-Java (HJ) compiler and runtime system. Preliminary results on general-purpose multicore processors and GPU accelerators indicate that the HPT model can be a promising portable abstraction for future multicore processors.

We present an efficient tensor-network-based approach for simulating large-scale quantum circuits exemplified by quantum support vector machines (QSVMs). Experimentally, leveraging the cuTensorNet library on multiple GPUs, our method effectively reduces the exponential runtime growth to near-quadratic scaling with respect to the number of qubits in practical scenarios. Traditional state-vector simulations become computationally infeasible beyond approximately 50 qubits; in contrast, our simulator successfully handles QSVMs with up to 784 qubits, executing simulations within seconds on a single high-performance GPU. Furthermore, utilizing the message passing interface for multi-GPU environments, our method demonstrates strong linear scalability, effectively decreasing computation time as dataset sizes increase. We validate our framework using the MNIST and Fashion MNIST datasets, achieving successful multiclass classification and highlighting the potential of QSVMs for high-dimensional data analysis. By integrating tensor-network techniques with advanced high-performance computing resources, this work demonstrates both the feasibility and scalability of simulating large-qubit quantum machine learning models, providing a valuable validation tool within the emerging Quantum-HPC ecosystem.

Solar desiccant cooling is widely considered as an attractive replacement for conventional vapor compression air conditioning systems because of its environmental friendliness and energy efficiency advantages. The system performance of solar desiccant cooling strongly depends on the input parameters associated with the system components, such as the solar collector, storage tank and backup heater, etc. In order to understand the implications of different design parameters on the system performance, this study has conducted a parametric analysis on the solar collector area, storage tank volume, and backup heater capacity of a solid solar desiccant cooling system for an office building in Brisbane, Australia climate. In addition, a parametric analysis on the outdoor air humidity ratio control set-point which triggers the operation of the desiccant wheel has also been investigated. The simulation results have shown that either increasing the storage tank volume or increasing solar collector area would result in both increased solar fraction (SF) and system coefficient of performance (COP), while at the same time reduce the backup heater energy consumption. However, the storage tank volume is more sensitive to the system performance than the collector area. From the economic aspect, a storage capacity of 30 m3/576 m2 has the lowest life cycle cost (LCC) of $405,954 for the solar subsystem. In addition, 100 kW backup heater capacity is preferable for the satisfaction of the design regeneration heating coil hot water inlet temperature set-point with relatively low backup heater energy consumption. Moreover, an outdoor air humidity ratio control set-point of 0.008 kgWater/kgDryAir is more reasonable, as it could both guarantee the indoor design conditions and achieve low backup heater energy consumption.