Discovering the SUPER in computing - dagster-slurm for reproducible research on HPC

Traditional high performance computing (HPC) centers that operate a single large supercomputer cluster have not required sophisticated mechanisms to manage and enforce network policies. Recently, HPC centers have expanded to support a wide range of computational infrastructure, such as OpenStack-based private clouds and Ceph object stores, each with its own unique characteristics and network security requirements. Network security policies are becoming more complex and harder to manage. To address the challenge, this paper explores ways to define and manage the new network policies required by emerging HPC systems. As the first step, we identify the new types of policies that are required and the technical capabilities needed to support them. We present example policies and discuss ways to implement those policies using emerging programmable networks and intent-based networks. We describe our initial work toward automatically converting human readable network policies into network configurations and programmable network controllers that implement those policies using business rule management systems.

Modern High Performance Computing (HPC) applications process very large amounts of data. A critical research challenge lies in transporting input data to the HPC center from a number of distributed sources, e.g., scientific experiments and web repositories, etc., and offloading the result data to geographically distributed, intermittently available end-users, often over under-provisioned connections. Such end-user data services are typically performed using point-to-point transfers that are designed for well-endowed sites and are unable to reconcile the center's resource usage and users' delivery deadlines, unable to adapt to changing dynamics in the end-to-end data path and are not fault-tolerant. To overcome these inefficiencies, decentralized HPC data services are emerging as viable alternatives. In this paper, we develop and enhance such distributed data services by designing CATCH, a Cloud-based Adaptive data Transfer service for HPC. CATCH leverages a bevy of cloud storage resources to orchestrate a decentralized data transport with fail-over capabilities. Our results demonstrate that CATCH is a feasible approach, and can help improve the data transfer times at the HPC center by as much as 81.1\% for typical HPC workloads.

Users of High Performance Computing (HPC) centers such as Livermore Computing have historically required expert knowledge of nearly every aspect of the computer center operation in order to take full advantage of the immense resources available. This knowledge takes months or more to develop and can make entering the world of HPC intimidating for new users.

As the science and engineering demand for high performance computing (HPC) grows beyond leading edge research institutions and communities to encompass routine activities of many disciplines, computing center infrastructures expand in their size, density and power demands. Long-practiced HPC center enhancements are increasingly demanding cooling methods that extend well beyond the capabilities of implementations that have become the staple of facility designs, even beyond those of the now-dominant architecture - commodity-processor-based, air-cooled rack clusters. As compute capacity aggressively increases in HPC centers, the power and cooling requirements, and thus cost of operation of the facilities, continues to rise correspondingly. This growth increasingly taxes the abilities of the hosting organizations to accommodate these demands. To address the challenge of meeting such pervasive demands, this paper examines energy efficiency in existing data centers from a two-pronged approach: employing direct water cooling and optimizing the facility infrastructure with as little capital investment to the building as possible.

Continuous integration (CI) has become a ubiquitous practice in modern software development, with major code hosting services offering free automation on popular platforms. CI offers major benefits, as it enables detecting bugs in code prior to committing changes. While high-performance computing (HPC) research relies heavily on software, HPC machines are not considered “common” platforms. This presents several challenges that hinder the adoption of CI in HPC environments, making it difficult to maintain bug-free HPC projects, and resulting in adverse effects on the research community. In this article, we explore the challenges that impede HPC CI, such as hardware diversity, security, isolation, administrative policies, and nonstandard authentication, environments, and job submission mechanisms. We propose several solutions that could enhance the quality of HPC software and the experience of developers. Implementing these solutions would require significant changes at HPC centers, but if these changes are made, it would ultimately enable faster and better science.

Salmonella enterica and Escherichia coli are major food-borne human pathogens, and their genomes are routinely sequenced for clinical surveillance. Computational pipelines designed for analyzing pathogen genomes should both utilize the most current information from annotation databases and increase the coverage of these databases over time. We report the development of the GEA pipeline to analyze large batches of E. coli and S. enterica genomes. The GEA pipeline takes as input paired Illumina raw reads files which are then assembled followed by annotation. Alternatively, assemblies can be provided as input and directly annotated. The pipeline provides predictive genome annotations for E. coli and S. enterica with a focus on the Center for Genomic Epidemiology tools. Annotation results are provided as a tab delimited text file. The GEA pipeline is designed for large-scale E. coli and S. enterica genome assembly and characterization using the Center for Genomic Epidemiology command-line tools and high-performance computing. Large scale annotation is demonstrated by an analysis of more than 14,000 Salmonella genome assemblies. Testing the GEA pipeline on E. coli raw reads demonstrates reproducibility across multiple compute environments and computational usage is optimized on high performance computers.

Cloud archiving and data mining of High-Resolution Rapid Refresh forecast model output

Monitoring and assessing the energy efficiency of supercomputers and data centers is crucial in order to limit and reduce their energy consumption. Applications from the domain of High Performance Computing (HPC), such as MPI applications, account for a significant fraction of the overall energy consumed by HPC centers. Simulation is a popular approach for studying the behavior of these applications in a variety of scenarios, and it is therefore advantageous to be able to study their energy consumption in a cost-efficient, controllable, and also reproducible simulation environment. Alas, simulators supporting HPC applications commonly lack the capability of predicting the energy consumption, particularly when target platforms consist of multi-core nodes. In this work, we aim to accurately predict the energy consumption of MPI applications via simulation. Firstly, we introduce the models required for meaningful simulations: The computation model, the communication model, and the energy model of the target platform. Secondly, we demonstrate that by carefully calibrating these models on a single node, the predicted energy consumption of HPC applications at a larger scale is very close (within a few percents) to real experiments. We further show how to integrate such models into the SimGrid simulation toolkit. In order to obtain good execution time predictions on multi-core architectures, we also establish that it is vital to correctly account for memory effects in simulation. The proposed simulator is validated through an extensive set of experiments with wellknown HPC benchmarks. Lastly, we show the simulator can be used to study applications at scale, which allows researchers to save both time and resources compared to real experiments.

Performance models of high performance computing (HPC) applications are important for several reasons. First, they provide insight to designers of HPC systems on the role of subsystems such as the processor or the network in determining application performance. Second, they allow HPC centers more accurately to target procurements to resource requirements. Third, they can be used to identify application performance bottlenecks and to provide insights about scalability issues. The suitability of a performance model, however, for a particular performance investigation is a function of both the accuracy and the cost of the model. A semi-empirical model previously published by the authors for an astrophysics application was shown to be inaccurate when predicting communication cost for large numbers of processors. It is hypothesized that this deficiency is due to the inability of the model adequately to capture communication contention (threshold effects) as well as other unmodeled components such as noise and I/O contention. In this paper we present a new approach to capture these unknown features to improve the predictive capabilities of the model. This approach uses a systematic model error-correction procedure that uses evolutionary algorithms to find an error correction term to augment the eXisting model. Four variations of this procedure were investigated and all were shown to produce better results than the original model. Successful cross-platform application of this approach showed that it adequately captures machine dependent characteristics. This approach was then successfully demonstrated for a second application, further showing its versatility.

With the advent of myriad different technologies in the modern computing era, the future of high performance computing (HPC) data centers comprising heterogeneous compute servers and networks involves a significantly greater degree of cyber-threats than we are observing currently. As a result, conducting cybersecurity operations at the HPC data centers encompasses a highly cognitive workload environment, where information is rapidly generated, is exceedingly dynamic and is vastly unpredictable. An important challenge faced by cybersecurity specialists is managing the extreme levels of stress and utilizing emerging technologies for monitoring, assessing and defending multiple cyber-attacks in real-time. In this paper, we present an augmented reality based framework for storing, analyzing, visualizing and augmenting data at the HPC centers. This framework is proposed with the goal of improving the data presentation, forecasting, analysis and decision making, thus reducing the time to insight and learning the HPC security operations to combat increasing cyber physical threats.

The Naval Oceanographic Office (NAVOCEANO) Department of Defense (DoD) Major Shared Resource Center (MSRC) operates as a DoD shared high performance computing (HPC) center, serving over 2000 users nationwide. Scientific visualization is an essential element of HPC that provides the methodology to explore, define, and present the results of computations. Accurate and understandable representation of data impacts a project in several ways: it helps scientists understand the physical phenomena they are studying, it helps communicate this work to their colleagues, and it helps explain the work and its significance to the public. Our scientific visualization team, in conjunction with researchers at our facility and other DoD facilities, have accomplished numerous visualizations of large data sets. In this paper we present examples of the methods, techniques, and results of our efforts to visualize observational data and the output of numeric ocean models.

Earthquake rupture forecasting is a critical component of seismic hazard analysis and requires identifying physically plausible rupture paths across complex fault networks. At its core, this task could be formulated as a large-scale combinatorial optimization problem, which involves selecting an optimal subset of fault segments from a set of candidates. Such problems pose significant challenges for traditional algorithms, as the number of admissible multi-fault ruptures grows combinatorially. Current operational workflows rely on locally applied plausibility filters. While computationally efficient, such greedy local heuristics risk excluding globally competitive rupture scenarios, particularly in regions with dense fault connectivity and competing rupture pathways. This work investigates whether quantum annealing hardware can serve as a sampling accelerator for exploring the ensemble of physically plausible ruptures beyond what is currently accessible to classical approaches. Designing a problem formulation that remains scientifically meaningful while respecting the constraints of current quantum hardware (size, noise, etc.) is nontrivial, and naïve encodings often collapse under these limitations. We encode rupture plausibility modeling as low energy solutions to a quadratic unconstrained binary optimization (QUBO) problem defined on a fault-network middle graph. Binary variables represent activated fault segments, while local interaction terms encode Coulomb stress transfer (as a proxy for pairwise rupture compatibility), continuity preferences, and branching behavior. Here, we adopt a classical-quantum hybrid workflow: the quantum annealer is used to sample globally competitive rupture candidates, while classical post-processing implements physical constraints to filter out non-physical ruptures. The workflow is demonstrated on a subset of the fault network used in the 2023 USGS National Seismic Hazard Model including over 200 fault segments, using both simulated thermal and quantum annealing on D-Wave hardware. Results show that our hardware-aware formulation and conditioning enable robust sampling on comparatively large fault-network instances. Views on quantum computing are polarized: some overstate its power, while others dismiss it as impractical. Our results help bridge this by establishing a concrete, testable pathway for integrating quantum annealing into rupture-modeling workflows on existing purpose-built quantum devices.—---This work is supported by the ICSC National Research Centre for High Performance Computing, Big Data and Quantum Computing (CN00000013, CUP D53C22001300005) within the European Union-NextGenerationEU program (National Recovery and Resilience Plan (PNRR) - Mission 4 Component 2 Investment 1.4.)

High-Performance Computing (HPC) has become a powerful tool to solve highly complex projects that could not otherwise be addressed. However, developing parallel programs to take advantage of its benefits, making a proper and effective use of the available computing resources, can be a difficult task for beginners or scientists of specific branches of science which have very little to do with computer programming. For this purpose, a transcompiler for automatic parallelization of sequential codes was developed to train beginners in parallel programming and achieve better performances in HPC centers. Current efforts are focused on expanding the transcompiler with an additional module for optimizing sequential and parallel code fragments with the objective of speeding up their execution times. To accomplish this, several techniques for writing efficient code have been evaluated with a view to integrate them into the transcompiler in the future. This manuscript discusses the importance of these techniques to decrease running times of HPC programs. In addition, a set of tests have been developed for analysing and measuring the improvements achieved when applying each of these techniques, all of them with a significant impact on the execution times.

Cyber Equations, Inc. is reporting on the results of its DOE SBIR Phase I project for the development of a Scalable HPC Insider Threat Monitoring System. This final report to the Department of Energy for Task DE-FOA-0001618 covers the period from 02/21/2017 to 02/20/2018 and summarizes the larger research accomplishments, which can be divided in theoretical and laboratory experiments. One of the many challenges for the U.S. Department of Energy (DOE) is the support of the high-performance computing (HPC) centers while providing quality information services, strategic vision, leadership, technical expertise, and capabilities necessary to enable the DOE to deliver effective, efficient, economical, and secure HPC services. In order to realize a secure DOE laboratory enterprise, DOE facilities operations must counter a wide range of cyber insider threats, from unsophisticated adversaries up to advanced, nation state sponsored actors. The DOE laboratories also presents an enormous insider attack surface of an ever-growing number of computers, networks, and devices all running a myriad of software. In addition, the DOE laboratories are connected to a variety of external organizations which multiplies the possibility of insider threat. During this SBIR project Cyber Equations developed a strategic combination of supervised and unsupervised machine learning techniques to control scale and classify whether entities, such as hosts, actors, or documents, are showing suspicious insider behavior. We conducted experiments using real cyber data to show that changes in observable behavior (e.g., changes in outbound traffic patterns) are indicative of attacks executing (e.g., insiders exfiltrating data, bots calling home, or an attack spreading). The features and methods we use, have been used in the anomaly detection research and development community for a significant amount of time and robust implementations are openly available. K-Means clustering, Decision Trees, and Support Vector Machine based classifiers are readily available through the R language and open source H20 machine learning framework. The Cyber Equations approach is novel in its use of these classifiers combined with Hadoop based parallel processing technology. The five stages of processing used by CyberInsider improve scalability and reduce the high number of false positives normally associated with network anomaly detection. In summary, the Cyber Equations has successfully accomplished all of its Phase I tasks and demonstrated the feasibility of the Insider Threat Monitoring System.

The EUDAT project is a pan-European data initiative that started in October 2011. The project brings together a unique consortium of 25 partners - including research communities, national data and high performance computing (HPC) centres, technology providers, and funding agencies - from 13 countries. EUDAT aims to build a sustainable cross-disciplinary and cross-national data infrastructure that provides a set of shared services for accessing and preserving research data. The design and deployment of these services is being coordinated by multi-disciplinary task forces comprising representatives from research communities and data centres. This short paper presents the achievements of the project during its first year and describes the services that have been chosen to meet the requirements of the initial research communities involved in the project.