Parallel beam dynamics calculations on high performance computers

©2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Scientific applications are often irregular and characterized by large computationally-intensive parallel loops. Dynamic loop scheduling (DLS) techniques improve the performance of computationally-intensive scientific applications via load balancing of their execution on high-performance computing (HPC) systems. Identifying the most suitable choices of data distribution strategies, system sizes, and DLS techniques which improve the performance of a given application, requires intensive assessment and a large number of exploratory native experiments (using real applications on real systems), which may not always be feasible or practical due to associated time and costs. In such cases, simulative experiments are more appropriate for studying the performance of applications. This motivates the question of How realistic are the simulations of executions of scientific applications using DLS on HPC platforms? In the present work, a methodology is devised to answer this question. It involves the experimental verification and analysis of the performance of DLS in scientific applications. The proposed methodology is employed for a computer vision application executing using four DLS techniques on two different HPC plat- forms, both via native and simulative experiments. The evaluation and analysis of the native and simulative results indicate that the accuracy of the simulative experiments is strongly influenced by the approach used to extract the computational effort of the application (FLOP- or time-based), the choice of application model representation into simulation (data or task parallel), and the available HPC subsystem models in the simulator (multi-core CPUs, memory hierarchy, and network topology). The minimum and the maximum percent errors achieved between the native and the simulative experiments are 0.95% and 8.03%, respectively.

Fifth generation (5G) systems are considered as the future of telecommunications and data processing. 5G systems are envisaged to increase current network capacity by 1000-fold. In addition to increasing spectral efficiency and utilizing wider bandwidths, ultra network densification with wireless small cells is considered a significant capacity-enhancing method for meeting the ever-increasing demand of 5G data traffic requirements. The enormous amounts of data traffic generated in inter-small-cell 5G backhaul networks require efficient routing protocols to speed up the routing decisions, while ensuring high data rates, low latency, and low power consumption requirements. Therefore, a parallel routing protocol is proposed to speed up routing decisions in 5G backhaul networks, using high performance computing (HPC). We explore the efficiency of utilizing HPC to manage and speed up the parallel routing protocol using different communication network sizes. Our numerical results indicate that our HPC implementation achieves a routing speed-up of 37× for large network size (2048).

Shared libraries have become ubiquitous and are used to achieve great resource efficiencies on many platforms. The same properties that enable efficiencies on time-shared computers and convenience on small clusters prove to be great obstacles to scalability on large clusters and High Performance Computing platforms. In addition, Light Weight operating systems such as Catamount have historically not supported the use of shared libraries specifically because they hinder scalability. In this report we will outline the methods of supporting shared libraries on High Performance Computing platforms using Light Weight kernels that we investigated. The considerations necessary to evaluate utility in this area are many and sometimes conflicting. While our initial path forward has been determined based on this evaluation we consider this effort ongoing and remain prepared to re-evaluate any technology that might provide a scalable solution. This report is an evaluation of a range of possible methods of supporting dynamically linked executables on capability class1 High Performance Computing platforms. Efforts are ongoing and extensive testing at scale is necessary to evaluate performance. While performance is a critical driving factor, supporting whatever method is used in a production environment is an equally important and challenging task.

The development of big remote sensing data related technologies and applications poses a big challenge that massive computing capability is needed to support big data processing. In order to solve this challenge, this paper proposes an architecture of heterogeneous platform of high performance computing, which employs the computer hardware resources to improve the efficiency of big remote sensing data processing by optimizing scheduling strategies and designing high-performance algorithms. Furthermore, the proposed platform can dynamically incorporated with a workflow engine regarding big remote sensing data processing. These algorithms are modular to meet the flexible combination of different processes.

Beam halo is an important issue for accelerator driven transmutation technologies and other applications of CW proton accelerators. These projects include, for example, the accelerator transmutation of waste, accelerator-based conversion of plutonium, accelerator production of tritium, and the development of a next-generation spallation neutron source. To keep radioactivation within acceptable limits these accelerators must operate with a very low beam loss (less than a nanoampere per meter). Beam loss is associated with the presence of a low density halo far from the beam core. Understanding the physics of halo production and determining methods to control beam halo are important to these projects. In recent years significant advances have been made, both analytically and computationally. In the following we describe recent developments in beam halo theory and simulation, including results from multi-million particle simulations.

The rapid adoption of machine learning (ML) has increased the computational and storage demands on high-performance computing (HPC) and cloud datacenters. Unfortunately, deploying ML models on HPC and cloud platforms is challenging because it is often difficult to manage, financially costly, and poses substantial environmental concerns. This dissertation attempts to answer a simple but ambitious question: can we deploy ML models to meet the Quality of Service (QoS) requirements while remaining cost-effective and environmentally sustainable? The dissertation demonstrates that achieving this balance is indeed possible, albeit through employing non-intuitive strategies and user engagements. We show that a combination of fast and slow hardware can outperform the common strategy of exclusively employing the latest and fastest hardware when co-optimizing for both performance and cost. Realizing this potential, however, requires a strategy to create an effective hardware mix, a process rigorously developed within this dissertation. This dissertation further reveals that the carbon footprint of ML models can be managed by opportunistically trading off answer quality transiently and re-configuring/partitioning the GPU hardware resources, depending on the carbon intensity of available energy sources in the datacenter's region. Finally, this dissertation discovers that ML deployments can be made more environmentally sustainable by strategically engaging users and tuning response brevity, especially in the context of large language models (LLMs). The scientific advancements made in this dissertation aim to relieve the community from the cost constraints, hardware inefficiencies, and potential negative environmental impacts of large-scale deployment of ML models on HPC and cloud platforms. We are hopeful that its open-source contributions will promote further innovation and broader engagement toward sustainable ML deployment.--Author's abstract

To reproduce eScience, several challenges need to be solved: scientific workflows need to be automated; the involved software versions need to be provided in an unambiguous way; input data needs to be easily accessible; High-Performance Computing (HPC) clusters are often involved and to achieve bit-to-bit reproducibility, it might be even necessary to execute the code on a particular cluster to avoid differences caused by different HPC platforms (and unless this is a scientist's local cluster, it needs to be accessed across (administrative) borders). Preferably, to allow even inexperienced users to (re-)produce results, all should be user-friendly. While some easy-to-use web-based scientific portals support already to access HPC resources, this typically only refers to computing and data resources that are local. By the example of two community-specific portals in the fields of biodiversity and climate research, we present a solution for accessing remote HPC (and cloud) compute and data resources from scientific portals across borders, involving rigorous container-based packaging of the software version and setup automation, thus enhancing reproducibility.

The state-of-the-art for auditing and reproducing scientific applications on high-performance computing (HPC) systems is through a data provenance subsystem. While recent advances in data provenance lie in reducing the performance overhead and improving the user's query flexibility, the fidelity of data provenance is often overlooked: there is no such way to ensure that the provenance data itself has not been fabricated or falsified. This paper advocates leveraging blockchains to deliver immutable and autonomous data provenance services such that scientific discoveries are trustworthy. The challenges for adopting blockchains to HPC include designing a new blockchain architecture compatible with the HPC platforms and, more importantly, a set of new consensus protocols for scientific applications atop blockchains. To this end, we have designed the proof-of-scalable-traceability (POST) protocol and implemented it in a blockchain prototype, namely SciChain, the very first practical blockchain system for provenance services on HPC. We evaluated SciChain by comparing it with multiple state-of-the-art systems; experimental results showed that SciChain guaranteed trustworthy data provenance while incurring orders of magnitude lower overhead than existing solutions.

Interactive Parallelization Tool (IPT) is a semi-automatic tool that can be used by domain experts and students for transforming certain classes of existing applications into multiple parallel variants. An end-user of IPT provides existing application and high-level specifications for parallelization as input. On the basis of the specifications provided by the end-user, IPT carries out the code changes in the given existing application to generate parallel variants that can be run on different High Performance Computing (HPC) platforms. The parallel programming paradigms that are currently supported by IPT are MPI, OpenMP, and CUDA. The supported base languages are C and C++. Though IPT is still under active development, it has been recently made available on a web-enabled platform, named DiaGrid Hub, with the support from the XSEDE Extended Collaborative Support Service (ECSS). While the main goal of IPT is to make parallel programming easy for its end-users, the main goal of DiaGrid Hub is to enable the research community with instant access to HPC and High Throughput Computing platforms through a user-friendly web-interface. By deploying IPT on DiaGrid Hub, our goal is to enable the end-users to generate parallel versions of their existing applications without having to install IPT locally. They can also immediately compile and run the generated applications on Purdue and XSEDE resources that are available through DiaGrid Hub. Hence, the collaborative project that is reported in this paper lowers the entry-barriers to parallel programming and the usage of the national CyberInfrastructure (CI). In this paper, we present our ongoing work on deploying IPT over DiaGrid and testing the usability of IPT through a web-interface.

The detection of duplicate and related content is a critical data curation task in the context of digital research collections. This task can be challenging, if not impossible, to do manually in large, unstructured, and noisy collections. While there are many automated solutions for deduplicating data that contain large numbers of identical copies, it can be particularly difficult to find a solution for identifying redundancy within image-heavy collections that have evolved over a long span of time or have been created collaboratively by large groups. These types of collections, especially in academic research settings, in which the datasets are used for a wide range of publication, teaching, and research activities, can be characterized by (1) large numbers of heterogeneous file formats, (2) repetitive photographic documentation of the same subjects in a variety of conditions (3) multiple copies or subsets of images with slight modifications (e.g., cropping or color-balancing) and (4) complex file structures and naming conventions that may not be consistent throughout. In this chapter, we present a scalable and automated approach for detecting duplicate, similar, and related images, along with subimages, in digital data collections. Our approach can assist in efficiently managing redundancy in any large image collection on High Performance Computing (HPC) resources. While we illustrate the approach with a large archaeological collection, it is domain-neutral and is widely applicable to image-heavy collections within any HPC platform that has general-purpose processors.

Reliability is set to become a major concern on emergent large-scale architectures. While there are many parallel languages, and indeed many parallel functional languages, very few address reliability. The notable exception is the widely emulated Erlang distributed actor model that provides explicit supervision and recovery of actors with isolated state. We investigate scalable transparent fault tolerant functional computation with automatic supervision and recovery of tasks. We do so by developing HdpH-RS , a variant of the Haskell distributed parallel Haskell (HdpH) DSL with Reliable Scheduling. Extending the distributed work stealing protocol of HdpH for task supervision and recovery is challenging. To eliminate elusive concurrency bugs, we validate the HdpH-RS work stealing protocol using the SPIN model checker. HdpH-RS differs from the actor model in that its principal entities are tasks, i.e. independent stateless computations, rather than isolated stateful actors. Thanks to statelessness, fault recovery can be performed automatically and entirely hidden in the HdpH-RS runtime system. Statelessness is also key for proving a crucial property of the semantics of HdpH-RS: fault recovery does not change the result of the program, akin to deterministic parallelism. HdpH-RS provides a simple distributed fork/join-style programming model, with minimal exposure of fault tolerance at the language level, and a library of higher level abstractions such as algorithmic skeletons. In fact, the HdpH-RS DSL is exactly the same as the HdpH DSL, hence users can opt in or out of fault tolerant execution without any refactoring. Computations in HdpH-RS are always as reliable as the root node, no matter how many nodes and cores are actually used. We benchmark HdpH-RS on conventional clusters and an High Performance Computing platform: all benchmarks survive Chaos Monkey random fault injection; the system scales well e.g. up to 1,400 cores on the High Performance Computing; reliability and recovery overheads are consistently low even at scale.

The United States Navy is actively researching techniques for creating high-fidelity, real-time simulations of antiship cruise missiles (ASCM) in order to develop improved defensive countermeasures for Navy ships. One active area of investigation is the combined use of OpenMP and MPI to reach real-time constraints on stand-alone cluster computers with high-speed interconnect fabrics. The separate nodes of this high performance computing (HPC) platform calculate the successive responses of a single cruise missile to successive reflections of the RF transmitter radar returns from the target ship in a pipeline fashion using MPI. Numerically intensive portions of the calculation of the missile-ship system behavior for an individual RF pulse can be calculated in parallel on the individual nodes of the HPC platform using OpenMP. The speed at which these portions can be calculated determines the length of the pipeline and thus the total number of computing nodes required. This approach incurs some approximations into the simulation that are proportional to the length of the pipeline because there is a feedback from the ship-radar response back to the missile guidance. While this use of OpenMP has proven effective, it is limited by the number of cores available at each node. This code, however, presents opportunities for parallelism well beyond the available computational resources at each node. Additionally, the ratio of computation to data transfer for this portion of the simulation is very high. These two factors have led us to investigate executing the most compute-intensive portion, the calculation of the RF responses of the individual ship scatterers, on Graphics Processing Units (GPUs).

Sparse matrix-sparse vector (SpMSpV) multiplication is one of the fundamental and important operations in many high-performance scientific and engineering applications. The inherent irregularity and poor data locality lead to two main challenges to scaling SpMSpV over high-performance computing (HPC) systems: (i) a large amount of redundant data limits the utilization of bandwidth and parallel resources; (ii) the irregular access pattern limits the exploitation of computing resources. This paper proposes a fine-grained parallel SpMSpV (fgSpMSpV) framework on Sunway TaihuLight supercomputer to alleviate the challenges for large-scale real-world applications. First,fgSpMSpVadopts an MPI\( + \)OpenMP\( +X \)parallelization model to exploit the multi-stage and hybrid parallelism of heterogeneous HPC architectures and accelerate both pre-/post-processing and main SpMSpV computation. Second,fgSpMSpVutilizes an adaptive parallel execution to reduce the pre-processing, adapt to the parallelism and memory hierarchy of the Sunway system, while still tame redundant and random memory accesses in SpMSpV, including a set of techniques like the fine-grained partitioner, re-collection method, and Compressed Sparse Column Vector (CSCV) matrix format. Third,fgSpMSpVuses several optimization techniques to further utilize the computing resources.fgSpMSpVon the Sunway TaihuLight gains a noticeable performance improvement from the key optimization techniques with various sparsity of the input. Additionally,fgSpMSpVis implemented on an NVIDIA Tesal P100 GPU and applied to the breath-first-search (BFS) application.fgSpMSpVon a P100 GPU obtains the speedup of up to\( 134.38\times \)over the state-of-the-art SpMSpV algorithms, and the BFS application usingfgSpMSpVachieves the speedup of up to\( 21.68\times \)over the state-of-the-arts.

Summary Full Waveform Inversion (FWI) is a mathematically and computationally challenging inverse problem of finding a quantitative rock-property description of the subsurface to match observed seismogram data. The inversion employs a forward model, which relates the subsurface to the observed seismograms. FWI and other seismic applications require High Performance Computing (HPC) to simulate the dynamics of such complex models. Not a long ago, companies, research institutes, and universities used to acquire clusters of computers to maintain on-premise. Recently, cloud computing has become an alternative posing a challenge to the end-users, who have to decide whether they should execute their applications: on their local clusters or burst them to a remote cloud provider. In this paper, we present a decision support method to choose the correct environment considering trade-offs, such as resource costs, performance, and availability on such heterogeneous execution platforms. We evaluated the system using our FWI application and preliminary results indicate that users of HPC applications can benefit from such a cloud advisory system to reduce costs, turnaround times, and even boost local platforms.