S06---Computing protection in open HPC environments

The efficient implementation of a diffraction tomography method for medical imaging is addressed within the framework of High Performance Computing (HPC) environment. A non-linear optimization method for the solution of the inverse scattering problem is implemented on a shared memory model computer. Linear speed-up and significant reduction in the total execution time is achieved when the program is executed in parallel, enabling the feasibility of the method for realistic medical imaging applications.

Purpose The present paper aims to address the description of a numerical optimization procedure, based on mesh morphing, and its application for the improvement of the aerodynamic performance of an industrial glider which suffers of a large separation occurring in the wing–fuselage junction region at high incidence angles. Design/methodology/approach Shape variations were applied to the baseline configuration through a mesh morphing technique founded on the mathematical framework of radial basis functions (RBF). The aerodynamic solutions were obtained coupling an RANS code with the mesh morphing tool RBF Morph™. Two shape modifiers were set up to generate a parametric numerical model. An optimization procedure, based on a design of experiment sampling, was set up implementing the fully automated workflow within a high performance computing (HPC) environment. The optimal candidates maximizing the aerodynamic efficiency were identified by means of a cubic RBF response surface approach. Findings The separation was significantly reduced, modifying the local geometry of fuselage and fairing and maintaining the wing aerofoil unchanged. A relevant aerodynamic efficiency improvement was finally gained. Practical implications The developed procedure proved to be a very powerful and efficient tool in facing aerodynamic design problems. However, it might be computationally very expensive if a large number of design variables are adopted and, in those cases, the method can be suitably used only within the HPC environment. Originality/value Such an optimization study is part of an explorative set of analyses that focused on better addressing the numerical strategies to be used in the development of the EU FP7 Project RBF4AERO.

SummaryThe wide range of RNA-seq applications and their high-computational needs require the development of pipelines orchestrating the entire workflow and optimizing usage of available computational resources. We present aRNApipe, a project-oriented pipeline for processing of RNA-seq data in high-performance cluster environments. aRNApipe is highly modular and can be easily migrated to any high-performance computing (HPC) environment. The current applications included in aRNApipe combine the essential RNA-seq primary analyses, including quality control metrics, transcript alignment, count generation, transcript fusion identification, alternative splicing and sequence variant calling. aRNApipe is project-oriented and dynamic so users can easily update analyses to include or exclude samples or enable additional processing modules. Workflow parameters are easily set using a single configuration file that provides centralized tracking of all analytical processes. Finally, aRNApipe incorporates interactive web reports for sample tracking and a tool for managing the genome assemblies available to perform an analysis.Availability and documentationhttps://github.com/HudsonAlpha/aRNAPipe; DOI: 10.5281/zenodo.202950Supplementary informationSupplementary data are available at Bioinformatics online.

The lightweight nature, application portability, and deployment flexibility of containers is driving their widespread adoption in cloud solutions. Data analysis and deep learning (DL)/machine learning (ML) applications have especially benefited from containerization. As such data analysis is adopted in high performance computing (HPC), the need for container support in HPC has become paramount. However, containers face crucial performance and I/O challenges in HPC. One obstacle is that while there have been HPC containers, such solutions have not been thoroughly investigated, especially from the aspect of their impact on the crucial HPC I/O throughput. To this end, this paper provides a first-of-its-kind empirical analysis of state-of-the-art representative container solutions (Docker, Podman, Singularity, and Charliecloud) in HPC environments. We also explore how containers interact with an HPC parallel file system like Lustre. We present the design of an analysis framework that is deployed on all nodes in an HPC environment, and captures CPU, memory, network, and file I/O statistics from the nodes and the storage system. We are able to garner key insights from our analysis, e.g., Charliecloud outperforms other container solutions in terms of container start-up time, while Singularity and Charliecloud are equivalent in I/O throughput. But this comes at a cost, as Charliecloud invokes the most metadata and I/O operations on the underlying Lustre file system. By identifying such trade-offs and optimization opportunities, we can enhance HPC containers performance and the ML/DL applications that increasingly rely on them.

98P - Standardized and compliant bio-informatics pipeline for neoepitope analysis in oncology clinical trials

Abstract. Most Earth system models (ESMs) are running under different high-performance computing (HPC) environments. This has several advantages, from allowing different groups to work with the same tool in parallel to leveraging the burden of ensemble climate simulations, but it also offers alternative solutions in the case of shutdown (expected or not) of any of the environments. However, for obvious scientific reasons, it is critical to ensure that ESMs provide identical results under changes in computing environment. While strict bit-for-bit reproducibility is not always guaranteed with ESMs, it is desirable that results obtained under one computing environment are at least statistically indistinguishable from those obtained under another environment, which we term a “replicability” condition following the metrology nomenclature. Here, we develop a protocol to assess the replicability of the EC-Earth ESM. Using two versions of EC-Earth, we present one case of non-replicability and one case of replicability. The non-replicable case occurs with the older version of the model and likely finds its origin in the treatment of river runoff along Antarctic coasts. By contrast, the more recent version of the model provides replicable results. The methodology presented here has been adopted as a standard test by the EC-Earth consortium (27 institutions in Europe) to evaluate the replicability of any new model version across platforms, including for CMIP6 experiments. To a larger extent, it can be used to assess whether other ESMs can safely be ported from one HPC environment to another for studying climate-related questions. Our results and experience with this work suggest that the default assumption should be that ESMs are not replicable under changes in the HPC environment, until proven otherwise.

As the national focus on cyber security increases, there is an evolving need for a capability to provide for high-speed sensing of events, correlation of events, and decision-making based on the adverse events seen across multiple independent large-scale network environments. The purpose of this Shared Vision project, Cyber Security Indications and Warning System, was to combine both Sandia's and LMC's expertise to discover new solutions to the challenge of protecting our nation's infrastructure assets. The objectives and scope of the proposal was limited to algorithm and High Performance Computing (HPC) model assessment in the unclassified environment within funding and schedule constraints. The interest is the identification, scalability assessment, and applicability of current utilized cyber security algorithms as applied in an HPC environment.

Modern High Performance Computing (HPC) applications, such as Earth science simulations, produce large amounts of data due to the surging of computing power, while big data applications have become more compute-intensive due to increasingly sophisticated analysis algorithms. The needs of both HPC and big data technologies for advanced HPC and big data applications create a demand for integrated system support. In this study, we introduce Scientific Data Processing (SciDP) to support both HPC and big data applications via integrated scientific data processing. SciDP can directly process scientific data stored on a Parallel File System (PFS), which is typically deployed in an HPC environment, in a big data programming environment running atop Hadoop Distributed File System (HDFS). SciDP seamlessly integrates PFS, HDFS, and the widely-used R data analysis system to support highly efficient processing of scientific data. It utilizes the merits of both PFS and HDFS for fast data transfer, overlaps computing with data accessing, and integrates R into the data transfer process. Experimental results show that SciDP accelerates analysis and visualization of a production NASA Center for Climate Simulation (NCCS) climate and weather application by 6x to 8x when compared to existing solutions.

Scientific workflows are models composed of activities, data and dependencies whose objective is to represent a computer simulation. Workflows are managed by Scientific Workflow Management System (SWfMS). Such workflows commonly demand for many computational resources once their executions may involve a number of different programs processing a huge volume of data. Thus, the use of High Performance Computing (HPC) environments allied to parallelization techniques provides the support for the execution of such experiments. Some resources provided by clouds can be used to build HPC environments. Although clouds offer advantages such as elasticity and availability, failures are a reality rather than a possibility. Thus, SWfMS must be fault-tolerant. There are several types of fault tolerance techniques used in SWfMS such as checkpoint-restart and replication, but which fault tolerance technique best fits with a specific workflow? This work aims at analyzing several fault tolerance techniques in SWfMSs and recommending the suitable one for the user’s workflow using machine learning techniques and provenance data, thus improving resiliency.

The Pasadena Workshop on System Software and Tools for High Performance Computing Environments was held at the Jet Propulsion Laboratory from April 14 through April 16, 1992. The workshop was sponsored by a number of Federal agencies committed to the advancement of high performance computing (HPC) both as a means to advance their respective missions and as a national resource to enhance American productivity and competitiveness. Over a hundred experts in related fields from industry, academia, and government were invited to participate in this effort to assess the current status of software technology in support of HPC systems. The overall objectives of the workshop were to understand the requirements and current limitations of HPC software technology and to contribute to a basis for establishing new directions in research and development for software technology in HPC environments. This report includes reports written by the participants of the workshop`s seven working groups. Materials presented at the workshop are reproduced in appendices. Additional chapters summarize the findings and analyze their implications for future directions in HPC software technology development.

With the advent of myriad different technologies in the modern computing era, the future of high performance computing (HPC) data centers comprising heterogeneous compute servers and networks involves a significantly greater degree of cyber-threats than we are observing currently. As a result, conducting cybersecurity operations at the HPC data centers encompasses a highly cognitive workload environment, where information is rapidly generated, is exceedingly dynamic and is vastly unpredictable. An important challenge faced by cybersecurity specialists is managing the extreme levels of stress and utilizing emerging technologies for monitoring, assessing and defending multiple cyber-attacks in real-time. In this paper, we present an augmented reality based framework for storing, analyzing, visualizing and augmenting data at the HPC centers. This framework is proposed with the goal of improving the data presentation, forecasting, analysis and decision making, thus reducing the time to insight and learning the HPC security operations to combat increasing cyber physical threats.

Containers in High Performance Computing (HPC) provide a flexible and efficient alternative to traditional modular software. They allow researchers and engineers to package applications and their dependencies in a single file, ensuring portability, reproducibility and consistent performance across different computing environments. With the emergence of AI, industrial users and SMEs in HPC, containers have become even more convenient to use, as they can also meet the security requirements for isolating user workloads. As the HPC community embraces container technology, it is ushering in a new era of more efficient, repeatable and scalable research computing. However, adoption of containers in HPC has been relatively slow, largely due to misconceptions about how they work and concerns about potential performance degradation, energy inefficiency and security risks. This article looks at the security considerations when using containers and uses practical examples to show that, with the right configuration and tools, containers can be as secure as any other system application.

Arc4nix: A cross-platform geospatial analytical library for cluster and cloud computing

Continuous integration (CI) has become a ubiquitous practice in modern software development, with major code hosting services offering free automation on popular platforms. CI offers major benefits, as it enables detecting bugs in code prior to committing changes. While high-performance computing (HPC) research relies heavily on software, HPC machines are not considered “common” platforms. This presents several challenges that hinder the adoption of CI in HPC environments, making it difficult to maintain bug-free HPC projects, and resulting in adverse effects on the research community. In this article, we explore the challenges that impede HPC CI, such as hardware diversity, security, isolation, administrative policies, and nonstandard authentication, environments, and job submission mechanisms. We propose several solutions that could enhance the quality of HPC software and the experience of developers. Implementing these solutions would require significant changes at HPC centers, but if these changes are made, it would ultimately enable faster and better science.

The design of High Performance Computing (HPC) relies to a large extent on simulations to optimize components of such complex systems. A key hardware component of the interconnection network in HPC environments is the Network Interface Card (NIC). In spite of the popularity of simulation-based approaches in the computer architecture domain, few authors have focused on simulators design methodologies. In this paper, we describe the stages of implementing a simulation model to solve a real problem—modeling NIC buffer. We present a general methodology for helping users to build Hardware Description Language (HDL)/SystemC models targeted to fulfil features such as performance evaluation of compute nodes. The developed VHDL model allows reproducibility and can be used as a tool in the area of HPC education.