Toward self-describing and workflow integrated Earth system models: A coupled atmosphere-ocean modeling system application

Soil carbon (C) is a critical component of Earth system models (ESMs), and its diverse representations are a major source of the large spread across models in the terrestrial C sink from the third to fifth assessment reports of the Intergovernmental Panel on Climate Change (IPCC). Improving soil C projections is of a high priority for Earth system modeling in the future IPCC and other assessments. To achieve this goal, we suggest that (1) model structures should reflect real‐world processes, (2) parameters should be calibrated to match model outputs with observations, and (3) external forcing variables should accurately prescribe the environmental conditions that soils experience. First, most soil C cycle models simulate C input from litter production and C release through decomposition. The latter process has traditionally been represented by first‐order decay functions, regulated primarily by temperature, moisture, litter quality, and soil texture. While this formulation well captures macroscopic soil organic C (SOC) dynamics, better understanding is needed of their underlying mechanisms as related to microbial processes, depth‐dependent environmental controls, and other processes that strongly affect soil C dynamics. Second, incomplete use of observations in model parameterization is a major cause of bias in soil C projections from ESMs. Optimal parameter calibration with both pool‐ and flux‐based data sets through data assimilation is among the highest priorities for near‐term research to reduce biases among ESMs. Third, external variables are represented inconsistently among ESMs, leading to differences in modeled soil C dynamics. We recommend the implementation of traceability analyses to identify how external variables and model parameterizations influence SOC dynamics in different ESMs. Overall, projections of the terrestrial C sink can be substantially improved when reliable data sets are available to select the most representative model structure, constrain parameters, and prescribe forcing fields.

Data intensive research continues to drive innovation and discovery across Earth system science (ESS). ESS datasets maintained in science discipline specific repositories, including climate model projections, historical reanalysis products, and observational datasets, provide rich resources to support these initiatives. While significant progress has been made through the hosting of datasets by commercial cloud providers, many of these data resources– sometimes stored in non-standard formats–are primarily maintained in unconnected, domain-focused data systems designed to support the legacy “download, clean and analyze model”. This is a time consuming process with bandwidth and storage requirements that may be prohibitive, particularly for institutions with limited resources. This combination of the download, clean, and analyze model, and the use of non-standard formats, combine to create a barrier to realizing the full research potential of ESS data assets. This presentation will highlight the National Science Foundation National Center for Atmospheric Research (NSF NCAR) efforts to develop and deploy its Geoscience Data Exchange, Research Data Commons (GDEX, https://gdex.ucar.edu). GDEX is designed to overcome the challenges described above by: 1) curating standards based (FAIR), Analysis and AI optimized (AR/AI) versions of global and regional atmospheric reanalysis outputs, earth systems simulation outputs, and observations produced at NSF NCAR and partner organizations, 2) providing direct access to these datasets through its integration with on-premise computational resources, and 3) providing performant distributed access through its integration the Open Science Data Federation’s (OSDF, https://osg-htc.org/services/osdf). The OSDF supports streaming data access and integration with a variety of data and compute services through its system of geographically distributed data caches, including commercial cloud hosted open datasets. GDEX’s integration with OSDF supports a wider variety of cross-domain research use cases by enabling efficient access to the spectrum of datasets hosted through OSDF’s origin access points. Finally, GDEX is integrated with colocated data analytics services to support rapid development and iteration of data science (e.g. AL/ML) workflows, and facilitate open sharing of those workflows. To promote user adoption of these services, an example set of reference data analysis workflows have been seeded in public collaboration software repositories and documented in JupyterBook style web pages. GDEX users are encouraged to submit their own workflow examples through this resource, amplifying the impact of their science by allowing others to more easily build upon their work.

Substantial amount of research has been done recently to address the shimming problem in scientific workflows, in which a special kind of adaptors, called shims, are inserted between workflow tasks to resolve the data type incompatibility issue. Recently, scientific workflows are increasingly used for big data analysis and processing, which poses additional challenges, such as volume, velocity and variety of data to the shimming problem. One issue is to scale the registration and configuration procedure to a large number of workflow tasks. Another issue is the ease of integrating a large number of remote Web services and other heterogeneous task components that can consume and produce data in various formats and models into a uniform and interoperable workflow. Existing approaches fall short in usability and scalability in addressing these issues. In this paper we 1) propose a new simplified single-component based task model based on extensive experiences and lessons learned from our original multiple-component based task model. The new model separates registration from configuration and eases the process of registering external functional components (such as Web services) into p-workflows, 2) propose a shim generation algorithm that elegantly solves the shimming problem raised by Web service based scientific workflows, and 3) we integrate MongoDB, a NoSQL document-oriented database system for storing and managing large-scale unstructured documents. A new version of the DATAVIEW system has been developed to support the proposed techniques and a case study has been conducted to show the feasibility and usability of our proposed techniques.

Data-centric scientific workflows are often modeled as dataflow process networks. The simplicity of the dataflow framework facilitates workflow design, analysis, and optimization. However, modeling tasks using dataflow constructs often leads to overly complicated workflows that are hard to comprehend, reuse, and maintain. We describe a generic framework, based on scientific workflow templates and frames, for embedding control-flow intensive subtasks within dataflow process networks. This approach can seamlessly handle complex control-flow without sacrificing the benefits of dataflow. We illustrate our approach with a real-world scientific workflow from the astrophysics domain, requiring remote execution and file transfer in a semi-reliable environment. For such workflows, we also describe a 3-layered architecture based on frames and templates where the top-layer consists of an overall dataflow process network, the second layer consists of a tranducer template for modeling the desired control-flow behavior, and the bottom layer consists of frames inside the template that are specialized by embedding the desired component implementation. Our approach can enable scientific workflows that are more robust (faulttolerance strategies can be defined by control-flow driven transducer templates) and at the same time more reusable, since the embedding of frames and templates yields more structured and modular workflow designs.

Scientific workflow technology has become a de facto standard in e-Science. It provides a powerful unifying platform that allows scientists to build arbitrarily complicated applications by combining predefined components. With a wide range of components, with the functions ranging from acquiring data inputs from sensors, querying databases, performing data-mining, performing execution, through to visualising the results, this paradigm allows scientists to automate, manage and execute various steps in scientific research. Although powerful, scientific workflow systems still require some programming knowledge, which might not be optimal for many scientists. Recently, scientific workflows have been integrated into the Web, creating workflow-based science gateways. These gateways allow scientists to launch large-scale computational experiments without concern for the underlying complexity of the infrastructure. This approach delivers scientific workflows to more scientists and is becoming increasingly popular. To date, much workflow-based science gateway development effort has been invested in creating user-friendly and secure interfaces to setup parameters, execute workflows and collect results. However, interaction between users and executing workflows has not received a lot of attention. Scientific workflows have typically been executed in batch mode. This model does not work with human-in-the-loop workflows, which require continuous interaction between users and executing workflows. A fundamental contribution of this thesis is to show that gateways can support complex human- workflow interactions, facilitating monitoring and steering of the workflow execution as it progresses. This is important because scientific workflows increasingly require user decisions and interactions at various steps; human decision-making is an important component of real-world computational science. In order to achieve this goal, the thesis presents the design, implementation and demonstration of WorkWays, an interactive workflow-based science gateway for supporting development and execution of human-in-the-loop scientific workflows. WorkWays consists of two distinct frameworks with well defined APIs and bindings: a framework that enables human- workflow interaction and a framework for workflow development and execution.

Goelzer et al (2012) paint a portentous picture of what is likely to happen to the global sea-level over the next 1000 years. This worrying assessment is based on our current best understanding of how the world's giant ice sheets of Greenland and Antarctica, as well as a quarter of a million smaller glacial ice masses, and the ocean collectively respond to ongoing climate change. Theirs is a state of the science study that integrates these key contributors of sea-level change based on the latest models and current understanding, and an integrated Earth systems modelling approach termed LOVECLIM. As they point out in their study, only a handful of global climate models to date—i.e. models that are used to make predictions of future climate change—incorporate dynamically (fully) coupled ice-sheet models.

Abstract. Emissions are a central component of atmospheric chemistry models. The Harmonized Emissions Component (HEMCO) is a software component for computing emissions from a user-selected ensemble of emission inventories and algorithms. It allows users to re-grid, combine, overwrite, subset, and scale emissions from different inventories through a configuration file and with no change to the model source code. The configuration file also maps emissions to model species with appropriate units. HEMCO can operate in offline stand-alone mode, but more importantly it provides an online facility for models to compute emissions at runtime. HEMCO complies with the Earth System Modeling Framework (ESMF) for portability across models. We present a new version here, HEMCO 3.0, that features an improved three-layer architecture to facilitate implementation into any atmospheric model and improved capability for calculating emissions at any model resolution including multiscale and unstructured grids. The three-layer architecture of HEMCO 3.0 includes (1) the Data Input Layer that reads the configuration file and accesses the HEMCO library of emission inventories and other environmental data, (2) the HEMCO Core that computes emissions on the user-selected HEMCO grid, and (3) the Model Interface Layer that re-grids (if needed) and serves the data to the atmospheric model and also serves model data to the HEMCO Core for computing emissions dependent on model state (such as from dust or vegetation). The HEMCO Core is common to the implementation in all models, while the Data Input Layer and the Model Interface Layer are adaptable to the model environment. Default versions of the Data Input Layer and Model Interface Layer enable straightforward implementation of HEMCO in any simple model architecture, and options are available to disable features such as re-gridding that may be done by independent couplers in more complex architectures. The HEMCO library of emission inventories and algorithms is continuously enriched through user contributions so that new inventories can be immediately shared across models. HEMCO can also serve as a general data broker for models to process input data not only for emissions but for any gridded environmental datasets. We describe existing implementations of HEMCO 3.0 in (1) the GEOS-Chem “Classic” chemical transport model with shared-memory infrastructure, (2) the high-performance GEOS-Chem (GCHP) model with distributed-memory architecture, (3) the NASA GEOS Earth System Model (GEOS ESM), (4) the Weather Research and Forecasting model with GEOS-Chem (WRF-GC), (5) the Community Earth System Model Version 2 (CESM2), and (6) the NOAA Global Ensemble Forecast System – Aerosols (GEFS-Aerosols), as well as the planned implementation in the NOAA Unified Forecast System (UFS). Implementation of HEMCO in CESM2 contributes to the Multi-Scale Infrastructure for Chemistry and Aerosols (MUSICA) by providing a common emissions infrastructure to support different simulations of atmospheric chemistry across scales.

The advances in Earth System Models (ESM), jointly with the availability of more powerful computing infrastructures and novel solutions for Big Data and Machine Learning (ML) is allowing to push research in the climate change field forward. In such context, workflows are fundamental tools to automate the complex processes of model simulations, data preparation and analyses. Such tools are becoming more important as the complexity and heterogeneity in the software and computing infrastructures, as well as the data volumes to be handled, grow. However, integrating into a single workflow simulation and data centric processes can be very challenging due to their different requirements.This work presents an end-to-end workflow including the steps from the numerical ESM simulation run to the analysis of extreme weather events (e.g., heat waves and tropical cyclones) developed in the context of the eFlows4HPC EuroHPC project. It represents a real case study which requires components from High Performance Computing (HPC), Big Data and ML to carry out the workflow. In particular, the contribution demonstrates how the eFlows4HPC software stack can simplify the development, deployment, orchestration and execution of complex end-to-end workflows for climate science, as well as improve their portability over different computing infrastructures.

The project's long-term goal is to assert and maintain an international scientific leadership position in the development of Earth system models that address the grand challenge of actionable predictions of Earth system variability and change, with an emphasis on the most critical scientific questions facing the nation and DOE. The grand challenge will be addressed by overcoming three major limitations: Pushing the high-resolution frontier of Earth system modeling through a combination of strategic model development and computational advances targeting exascale computers and beyond. Bridging the gap in scales and processes by developing a new class of models that couple the natural, managed, and man-made systems across scales to address energy sector vulnerability to variability and change. The project will achieve its goal through four intersecting project elements: A series of prediction and simulation experiments addressing scientific questions and mission needs; A well-documented and tested, continuously advancing, evolving, and improving system of model codes that comprise the E3SM Earth system model; The ability to use effectively leading (and “bleeding”) edge computational facilities soon after their deployment at DOE national laboratories; and An infrastructure to support code development, hypothesis testing, simulation execution, and analysis of results. To achieve the Grand Challenge of actionable Earth system predictions to address the most critical scientific questions facing our nation and DOE, the E3SM project defined its strategy in a fifteen-year roadmap with four intersecting project elements: A series of prediction and simulation experiments addressing scientific questions and mission needs; A well-documented and tested, continuously advancing, evolving, and improving system of model codes that comprise the E3SM Earth system model; The ability to use effectively leading (and “bleeding”) edge computational facilities soon after their deployment at DOE national laboratories; and An infrastructure to support code development, hypothesis testing, simulation execution, and analysis of results. Figure 1 depicts the E3SM Project Roadmap, showing the relationships among the first three major project elements: the simulations, the modeling system to perform those simulations, and the machines on which they will be executed. Unlike the other three elements that have distinct but overlapping phases, the fourth element, the infrastructure, will evolve continuously based on the requirements imposed by project needs.

<strong class="journal-contentHeaderColor">Abstract.</strong> We develop a new data assimilative (DA) approach by combining two parallel frameworks: a parallel DA framework (PDAF) and a flexible model coupling framework (ESMF). The new DA system is built on the ESMF at the top level that drives the PDAF and any combination of earth system modeling (ESM) components, to allow maximum flexibility and easy implementation of data assimilation for fully coupled ESM applications. We demonstrate the new DA system using a 3D unstructured-grid ocean model as ESM in this paper. The new system is validated using a simple benchmark and applied to a realistic case of Kuroshio simulation around Taiwan. The new system is demonstrated to significantly improve the model skill for temperature, velocity and surface elevation before, during and after typhoon events. The flexibility and ease of implementation make the new system widely applicable for other coupled ESMs.

&amp;lt;div&amp;gt; &amp;lt;div&amp;gt; &amp;lt;div&amp;gt; &amp;lt;p&amp;gt;Earth system models (ESMs) are state-of-the-art models which integrate previously separate models of the ocean, atmosphere and vegetation in one comprehensive modelling system enabling the investigation of interactive feedbacks between different components of the Earth system. Global isoprene and monoterpene emissions from terrestrial vegetation, which represents the most important source of VOCs in the Earth system, are needed for a suitable representation in global and regional chemical transport models given their impacts on the atmosphere. Consequently, to accurately determine the budget of isoprene and monoterpenes in the atmosphere, adequate emissions from the terrestrial vegetation are a requirement for input into regional and global scale chemistry-transport or chemistry-climate models. Due to the feedbacks of vegetation activity involving interactions with the weather and climate, a coupled modelling system between vegetation and atmospheric chemistry is a recommended tool to address the fate of biogenic volatile organic compounds (bVOCs). In this work, we present further development in linking LPJ-GUESS, a global dynamic vegetation model, to the atmospheric chemistry-enabled atmosphere-ocean general circulation model EMAC. We evaluate terrestrial bVOC emission estimates from the submodel ONEMIS in EMAC with (1) prescribed climatological vegetation boundary conditions at the land-atmosphere interface; and (2) dynamic vegetation states calculated in LPJ-GUESS (replacing the &amp;amp;#8220;offline&amp;amp;#8221; vegetation inputs). LPJ-GUESS-driven global emission estimates for isoprene and monoterpenes were found to be 599 Tg yr&amp;lt;sup&amp;gt;&amp;amp;#8722;1&amp;lt;/sup&amp;gt; and 111 Tg yr&amp;lt;sup&amp;gt;&amp;amp;#8722;1&amp;lt;/sup&amp;gt;, respectively. Additionally, we evaluated the sensitivity of the new coupled system in doubling CO&amp;lt;sub&amp;gt;2&amp;lt;/sub&amp;gt; scenarios. Higher temperatures resulted in an increase in bVOC emissions (+47% and +69% for isoprene and monoterpenes, respectively), whereas CO&amp;lt;sub&amp;gt;2&amp;lt;/sub&amp;gt;-fertilisation resulted in an increase of 18% in isoprene emissions and 37% in monoterpene emissions. We provide evidence that the new coupled model yields suitable estimates for global bVOC emissions that are responsive to vegetation dynamics, thus enabling further research in land-biosphere-atmosphere interactions.&amp;lt;/p&amp;gt; &amp;lt;/div&amp;gt; &amp;lt;/div&amp;gt; &amp;lt;/div&amp;gt;

Service-oriented workflow provides a mechanism to automate the coordination of a sequence of tasks within a process, whereas each task is presented as a Web service and integrated into the workflow. It has become increasingly important to have an appropriate workflow platform to manage the scientific workflow lifecycle. In order to address this need, we propose to use Pipeline Pilot to manage the lifecycle of grid-based scientific workflow using SOA approach. Pipeline Pilot is a commercial software package dedicated to scientific research. It is reliable, lightweight and comprehensive. We then describe a case study of how we employ Pipeline Pilot to manage a workflow lifecycle for calculating material dielectric properties in grid environment. We also discuss lessons we learned in managing lifecycle of a service-oriented scientific workflow within e-Science.

: To realize its potential, a U.S. Earth system modeling and prediction capability must encompass a network of agencies and organizations that contribute model components, infrastructure, and scientific and technical expertise. The model component contributions must be integrated using coupling software, optimized as a whole, and the predictive skill of the resulting models assessed using standard metrics. We propose to provide these integrative functions for the Earth System Prediction Capability (ESPC), using as a reference application a version of the Community Earth System Model (CESM) running an optimized version of the HYbrid Coordinate Ocean Model (HYCOM). We will establish a Coupling Testbed as a resource available to ESPC modelers, with Testbed leads who have developed infrastructure for major modeling systems and can engage and motivate interactions with modelers at NASA, NOAA, and other centers. The Testbed will be used to explore and prototype design and performance challenges related to integration, migrate optimizations to ESPC codes, and advance convergence of coupling interfaces to support knowledge transfer and controlled experimentation. We will work to broaden existing multi-agency computational working groups to address ESPC performance concerns and new application requirements. We will advance and update a coupled CESM-HYCOM system prototyped under other funding with a version of HYCOM optimized for accelerator-based architectures. This HYCOM version will be obtained from a linked proposal, entitled Accelerated Prediction of the Polar Ice and Global Ocean. Performance of the ocean component is of particular interest because we choose to emphasize science applications that are of greatest interest to the Navy, and this is best done at ocean eddy resolving scales. The coupled system will be suitable for data assimilative prediction problems with timescales from one week to up to one year.

We describe the ocean general circulation model Icosahedral Nonhydrostatic Weather and Climate Model (ICON‐O) of the Max Planck Institute for Meteorology, which forms the ocean‐sea ice component of the Earth system model ICON‐ESM. ICON‐O relies on innovative structure‐preserving finite volume numerics. We demonstrate the fundamental ability of ICON‐O to simulate key features of global ocean dynamics at both uniform and non‐uniform resolution. Two experiments are analyzed and compared with observations, one with a nearly uniform and eddy‐rich resolution of ∼10 km and another with a telescoping configuration whose resolution varies smoothly from globally ∼80 to ∼10 km in a focal region in the North Atlantic. Our results show first, that ICON‐O on the nearly uniform grid simulates an ocean circulation that compares well with observations and second, that ICON‐O in its telescope configuration is capable of reproducing the dynamics in the focal region over decadal time scales at a fraction of the computational cost of the uniform‐grid simulation. The telescopic technique offers an alternative to the established regionalization approaches. It can be used either to resolve local circulation more accurately or to represent local scales that cannot be simulated globally while remaining within a global modeling framework.

European Earth system science increasingly depends on seamless access to authoritative and interoperable spatial data across administrative and national borders. While many European research infrastructures focus on scientific data production, long-term sustainability and reproducibility also require the systematic inclusion of legally mandated public-sector geodata, which can complement initiatives such as the European Open Science Cloud (EOSC). This contribution presents how Germany’s Spatial Data Infrastructure (GDI-DE), through its national discovery and access portal Geoportal.de and its European coordination activities, supports this integration. Geoportal.de is Germany’s central discovery and access portal for geospatial data and services provided by federal, state and local authorities. It implements a federated, legally grounded model in which hundreds of data providers publish INSPIRE-compliant metadata and services through standardized interfaces. This allows scientists to discover, evaluate and access authoritative reference data, environmental monitoring data and thematic geodata that are often not available through purely research-driven infrastructures, thereby providing a stable backbone for FAIR-aligned access to public-sector Earth system data in Germany.Beyond national service provision, the German Coordination Office for Spatial Data Infrastructure actively contributes to European technical coordination through the MIG-T, the permanent technical subgroup of the INSPIRE MIG. Within this framework, the coordination office of the GDI-DE coordinated the German stakeholder involvement in the recent INSPIRE consolidation process, which aimed at simplifying and modernizing the INSPIRE framework while ensuring continuity for operational infrastructures. This national coordination complemented parallel activities across Europe, helping to align national and European perspectives on the future of INSPIRE.We argue that the combination of legally mandated national SDIs and European-level technical coordination is a key enabler for Earth system science, providing long-term, quality-controlled and interoperable access to public-sector geodata while allowing research infrastructures to focus on scientific value generation.