DAPT: A package enabling distributed automated parameter testing

Abstract. Permafrost thaw has been observed at several locations across the Arctic tundra in recent decades; however, the pan-Arctic extent and spatiotemporal dynamics of thaw remains poorly explained. Thaw-induced differential ground subsidence and dramatic microtopographic transitions, such as transformation of low-centered ice-wedge polygons (IWPs) into high-centered IWPs can be characterized using very high spatial resolution (VHSR) commercial satellite imagery. Arctic researchers demand for an accurate estimate of the distribution of IWPs and their status across the tundra domain. The entire Arctic has been imaged in 0.5 m resolution by commercial satellite sensors; however, mapping efforts are yet limited to small scales and confined to manual or semi-automated methods. Knowledge discovery through artificial intelligence (AI), big imagery, and high performance computing (HPC) resources is just starting to be realized in Arctic science. Large-scale deployment of VHSR imagery resources requires sophisticated computational approaches to automated image interpretation coupled with efficient use of HPC resources. We are in the process of developing an automated Mapping Application for Permafrost Land Environment (MAPLE) by combining big imagery, AI, and HPC resources. The MAPLE uses deep learning (DL) convolutional neural nets (CNNs) algorithms on HPCs to automatically map IWPs from VHSR commercial satellite imagery across large geographic domains. We trained and tasked a DLCNN semantic object instance segmentation algorithm to automatically classify IWPs from VHSR satellite imagery. Overall, our findings demonstrate the robust performances of IWP mapping algorithm in diverse tundra landscapes and lay a firm foundation for its operational-level application in repeated documentation of circumpolar permafrost disturbances.

While High-Performance Computing (HPC) resources are powerful for tackling complex, computationally intensive analysis and modeling problems, access to these resources varies across disciplines. Domain scientists in a variety of fields such as social and environmental sciences often lack in-depth technical skills (e.g., familiarity with terminal, knowledge of job schedulers) to effectively utilize HPC resources, hindering desired research. In this context, CyberGIS-Compute is a middleware toolkit designed to democratize HPC access with the main goal of enabling domain scientists in diverse fields to solve computationally intensive problems. A key challenge facing model developers on CyberGIS-Compute is to create a containerized software environment for their models. Domain experts unfamiliar with HPC are generally unfamiliar with containerization technologies (e.g., Docker, Singularity) and thus unable to create/test containers to execute their models. But if they have access to science gateways, they would want to use these familiar software environments on HPC resources. This paper describes a novel approach to integrating the Cern Virtual Machine File System (CVMFS) into CyberGIS-Compute to provide consistent software environments across science gateways and HPC resources.

Biomedical data are quickly growing in volume and in variety, providing clinicians an opportunity for better clinical decision support. Here, we demonstrate a robust platform that uses software automation and high performance computing (HPC) resources to achieve real-time analytics of clinical data, specifically magnetic resonance imaging (MRI) data. We used the Agave application programming interface to facilitate communication, data transfer, and job control between an MRI scanner and an off-site HPC resource. In this use case, Agave executed the graphical pipeline tool GRAphical Pipeline Environment (GRAPE) to perform automated, real-time, quantitative analysis of MRI scans. Same-session image processing will open the door for adaptive scanning and real-time quality control, potentially accelerating the discovery of pathologies and minimizing patient callbacks. We envision this platform can be adapted to other medical instruments, HPC resources, and analytics tools.

Managing very large computing systems with up to 100.000 nodes has become a very complex issue. Existing tools reach their limits especially for High Performance Computing (HPC) resources because they are slightly different from other compute resources. First we will introduce the specific HPC obstacles and what we suppose to be challenges for future resources to support the system management. After that we propose the framework designed in scope of the TIMaCS Project (http://www.timacs.de). Assuming that we once have a corresponding solution implemented we will show how this solution can change administration far beyond the current situation. This is separated into a more technical part describing how the administration can be simplified or where we can add new capabilities in resources provisioning and a business part where we outline the need for business policy based management and scheduling, and show a possible approach investigating these relationships. In the end we will show what might be possible far beyond the scope of the project.

We present our development of load balancing algorithms to efficiently distribute and parallelize the running of large-scale complex agent-based modeling (ABM) simulators on High-Performance Computing (HPC) resources. Our algorithm is based on partitioning the co-location network that emerges from an ABM's underlying synthetic population. Variations of this algorithm are experimentally applied to investigate algorithmic choices on two factors that affect run-time performance. We report these experiments' results on the CityCOVID ABM, built to model the spread of COVID-19 in the Chicago metropolitan region.

The meteorological high-performance computing resource is the support platform for the weather forecast and climate prediction numerical model operation. The scientific and objective method to evaluate the application of meteorological high-performance computing resources can not only provide reference for the optimization of active resources, but also provide a quantitative basis for future resource construction and planning. In this paper, the concept of the utility value B and index compliance rate E of the meteorological high performance computing system are presented. The evaluation process, evaluation index and calculation method of the high performance computing resource application benefits are introduced.

The NERSC Web Toolkit (NEWT) brings High Performance Computing (HPC) to the web through easy to write web applications. Our work seeks to make HPC resources more accessible and useful to scientists who are more comfortable with the web than they are with command line interfaces. The effort required to get a fully functioning web application is decreasing, thanks to Web 2.0 standards and protocols such as AJAX, HTML5, JSON and REST. We believe HPC can speak the same language as the web, by leveraging these technologies to interface with existing grid technologies. NEWT presents computational and data resources through simple transactions against URIs. In this paper we describe our approach to building web applications for science using a RESTful web service. We present the NEWT web service and describe how it can be used to access HPC resources in a web browser environment using AJAX and JSON. We discuss our REST API for NEWT, and address specific challenges in integrating a heterogeneous collection of backend resources under a single web service. We provide examples of client side applications that leverage NEWT to access resources directly in the web browser. The goal of this effort is to create a model whereby HPC becomes easily accessible through the web, allowing users to interact with their scientific computing, data and applications entirely through such web interfaces.

In this article we examine a teacher’s and multilingual students’ use of multiple resources and their potential for students’ meaning-making of sound and sound transmission. Students were 14–15 years old, Swedish grade 8, speaking Swedish as a second language. We examine how different strategies and multiple resources interact in creating thematic patterns in a multilingual science classroom. Data comprise 64 hours of video- and audio recordings, digital photos, field-notes, textbooks, worksheets and student notebooks. As analytical tools we use thematic development strategies, control and social interaction strategies as well as strategies of bridging multiple resources. In co-constructing the content using various resources, thematic patterns were developed through a continuous shift between everyday and scientific language due to the teacher’s awareness of the unit’s abstract and technical content. Findings also reveal that a strategy of control performed by the teacher marked the importance of using ‘physics words’. Strategies of social interaction accentuated by earlier experiences as well as personal and humorous connotations expressed in everyday language supported meaning-making. In addition, a number of multiple resources, such as models, gestures, bodily action, drawings, reading and writing were used.

The National Energy Research Scientific Computing Center (NERSC) is the leading scientific computing facility supporting research within the Department of Energy's Office of Science. NERSC provides high-performance computing (HPC) resources to approximately 4,000 researchers working on about 400 projects. In addition to hosting large-scale computing facilities, NERSC provides the support and expertise scientists need to effectively and efficiently use HPC systems. In February 2010, NERSC, DOE's Office of Advanced Scientific Computing Research (ASCR) and DOE's Office of Basic Energy Sciences (BES) held a workshop to characterize HPC requirements for BES research through 2013. The workshop was part of NERSC's legacy of anticipating users future needs and deploying the necessary resources to meet these demands. Workshop participants reached a consensus on several key findings, in addition to achieving the workshop's goal of collecting and characterizing computing requirements. The key requirements for scientists conducting research in BES are: (1) Larger allocations of computational resources; (2) Continued support for standard application software packages; (3) Adequate job turnaround time and throughput; and (4) Guidance and support for using future computer architectures. This report expands upon these key points and presents others. Several 'case studies' are included as significant representative samples of the needs of science teams within BES. Research teams scientific goals, computational methods of solution, current and 2013 computing requirements, and special software and support needs are summarized in these case studies. Also included are researchers strategies for computing in the highly parallel, 'multi-core' environment that is expected to dominate HPC architectures over the next few years. NERSC has strategic plans and initiatives already underway that address key workshop findings. This report includes a brief summary of those relevant to issues raised by researchers at the workshop.

Vocabulary knowledge plays a vital role in a successful language acquisition. It is essential and a fundamental element in language learning since it is considered the basis of any ESL learners. Students face tremendous difficulties in learning English vocabulary and show an aversion towards learning vocabulary. The present work investigated the use of multiple resources in learning vocabulary. A survey was carried out upon thirty Year 6 pupils of a primary school in Selangor. The instrument used in this research was a questionnaire to collect quantitative data. The findings showed that learners used a variety of resources in acquiring vocabulary in the four major skills. The finding also revealed that learners are more open to other resources outside the classroom. The usage of new technology enhances their vocabulary learning and they do not rely solely on teachers.

<div class="">In this work, we demonstrate the importance of High Performance Computing (HPC) and Urgent Computing (UC) in rapid disaster response to man-made catastrophes. Using the specific case study of a potential dam breach at the Kyiv cistern reservoir in Ukraine, we show how these technologies can be used to assess the potential hazards and impacts of such an event.</div> <div class=""> </div> <div class="">The Copernicus Emergency Management Service Risk & Recovery Mapping service was activated to derive hazard assessment mapping for the protection of citizens and infrastructure in the event of a potential dam breach. The Kiev Hydroelectric Power Plant, located upstream of Kiev, has been the target of a previous rocket attack, and the threat of another successful attack cannot be ruled out. <span class="">The potential consequences of a dam breach are severe and include flooding and the erosion and transport of radioactive sediment.</span><span class=""> </span></div> <div class=""> </div> <div class="">To determine the impact of a potential dam breach and predict its effects on citizens and infrastructure, we used the Dambreak-HySEA model, which is part of the HySEA suite. This model is specifically designed to accurately reproduce the evolution of wet/dry fronts in geophysical flows such as river flooding, flooding in rural and urban areas, and dam failures. Developed by the research group EDANYA of the University of Malaga, the model was implemented using Graphics Processor Units (GPUs) with CUDA, resulting in a significant speed-up compared to a traditional CPU implementation.</div> <div class=""> </div> <div class="">The use of HPC resources, specifically those provided by the Spanish Network for Supercomputing (RES) program Urgent computing for citizen problems, was crucial in obtaining the results within the 10-day delivery time frame with the simulation of four scenarios required by the activation: partial and complete breakage of the Kyiv Dam and the same scenarios for the Irpin Dam. These simulations were computationally complex, requiring the solution of nearly 700 million computational cells per scenario.</div> <div class=""> </div> <div class="">Overall, this study highlights the importance of utilizing HPC and UC in disaster response and risk management. By utilizing advanced numerical models and computational resources, we can more accurately predict the potential hazards and impacts of man-made catastrophes such as dam breaches, and take necessary measures to protect citizens and infrastructure.</div>

During the last years, High Performance Computing (HPC) resources have undergone a dramatic transformation, with an explosion on the available parallelism and the use of special purpose processors. There are international initiatives focusing on redesigning hardware and software in order to achieve the Exaflop capability. With this aim, the HPC4E project is applying the new exascale HPC techniques to energy industry simulations, customizing them if necessary, and going beyond the state-of-the-art in the required HPC exascale simulations for different energy sources that are the present and the future of energy: wind energy production and design, efficient combustion systems for biomass-derived fuels (biogas), and exploration geophysics for hydrocarbon reservoirs. HPC4E joins efforts of several institutions settled in Brazil and Europe.

This paper demonstrates a heterogeneous windows cluster system to obtain high performance computing (HPC) resources for medical image reconstruction applications. The communication in the cluster system is powered by MPICH open source library. Our works are to provide a simple processes management (PM) system with the functions of load balance in the windows version 1.25 of MPICH library and some PM extensions in the windows version 1.03 of MPICH2 library. These extensions ensure our medical image reconstruction applications can smoothly and robustly be running on a heterogeneous windows network environment, which is considered as a windows MPI cluster system with a low rate of cost/performance. In the study cases, the OSEM algorithm and Katsevich algorithm are implemented on our windows cluster system. The experiment results in a broad variety of image volumes are presented to verify system's feasibility and applicability to increase speedup of HPC powered CT image reconstruction.

Ecological sciences are using imagery from a variety of sources to monitor and survey populations and ecosystems. Very High Resolution (VHR) satellite imagery provide an effective dataset for large scale surveys. Convolutional Neural Networks have successfully been employed to analyze such imagery and detect large animals. As the datasets increase in volume, O(TB), and number of images, O(1k), utilizing High Performance Computing (HPC) resources becomes necessary. In this paper, we investigate a task-parallel data-driven workflows design to support imagery analysis pipelines with heterogeneous tasks on HPC. We analyze the capabilities of each design when processing a dataset of 3,000 VHR satellite images for a total of 4~TB. We experimentally model the execution time of the tasks of the image processing pipeline. We perform experiments to characterize the resource utilization, total time to completion, and overheads of each design. Based on the model, overhead and utilization analysis, we show which design approach to is best suited in scientific pipelines with similar characteristics.

It is well known that the quantitative potential of emission computed tomography (ECT) relies on the ability to compensate for resolution, attenuation and scatter effects. Reconstruction algorithms which are able to take these effects into account are highly demanding in terms of computing resources. The reported work aimed to investigate the use of a parallel high-performance computing platform for ECT reconstruction taking into account an accurate model of the acquisition of single-photon emission tomographic (SPET) data. An iterative algorithm with an accurate model of the variable system response was ported on the MIMD (Multiple Instruction Multiple Data) parallel architecture of a 64-node Cray T3D massively parallel computer. The system was organized to make it easily accessible even from low-cost PC-based workstations through standard TCP/IP networking. A complete brain study of 30 (64x64) slices could be reconstructed from a set of 90 (64x64) projections with ten iterations of the conjugate gradients algorithm in 9 s, corresponding to an actual speed-up factor of 135. This work demonstrated the possibility of exploiting remote high-performance computing and networking resources from hospital sites by means of low-cost workstations using standard communication protocols without particular problems for routine use. The achievable speed-up factors allow the assessment of the clinical benefit of advanced reconstruction techniques which require a heavy computational burden for the compensation effects such as variable spatial resolution, scatter and attenuation. The possibility of using the same software on the same hardware platform with data acquired in different laboratories with various kinds of SPET instrumentation is appealing for software quality control and for the evaluation of the clinical impact of the reconstruction methods.