Providentia: an evaluation software package for the atmospheric modelling community

<p indent="0mm"> National Field Scientific Observation and Research Station of Farmland Ecosystem in Changwu, Shaanxi Province (Changwu Station) is located in a region that represents the typical farmland of the tableland-gully region of the Loess Plateau. The installation of meteorological automatic observation systems at Changwu Station for long-term positioning observation of meteorological elements is of great significance for studying the farmland ecosystem′s water, soil, atmospheric, and biological processes under the background of climate change in this region. This dataset processes, controls and evaluates the original meteorological data from Changwu Station during 2005—2019, and discloses the information on 18 meteorological observation indexes (i.e. atmospheric temperature, relative humidity, precipitation, atmospheric pressure, total radiation, reflected radiation, ultraviolet radiation, net radiation, photosynthetic active radiation, sunshine hours and soil temperature at depths of <sc>0 cm,</sc> <sc>5 cm,</sc> <sc>10 cm,</sc> <sc>15 cm,</sc> <sc>20 cm,</sc> <sc>40 cm,</sc> <sc>60 cm</sc> and <sc>100 cm)</sc> , comprising 180 monthly entries and 15 annual entries. The aim is to provide essential data support for agro-ecological research, weather disaster early warning and climatic potential productivity simulation in the tableland-gully region of the Loess Plateau.

Evergreen broad-leaved forests are one of the main types of Chinese forests. The meteorological data of them are of great value for the study on the function and ecological process of evergreen broad-leaved forest ecosystem, and can serve as a major data source for weather forecast and disaster warning. Zhejiang Tiantong National Forest Ecosystem Observation and Research Station located in the subtropical broad-leaved evergreen forest in Tiantong Forest Park in eastern China. Since April 2003, we had collected long-term meteorological data by setting up the automatic meteorological observation system. This dataset contains the monthly data (177 entries) and annual data (15 entries) from 2013 to 2017, including 18 meteorological observation indexes, such as atmospheric temperature, atmospheric humidity, precipitation, atmospheric pressure, total radiation, net radiation, reflected radiation, UV radiation, photosynthetically active radiation, sunshine hours and soil temperature in different depths (0 cm, 5 cm, 10 cm, 15 cm, 20 cm, 40 cm, 60 cm and 100 cm). It is expected to provide data support for studying the ecological process, evaluating ecosystem service function and increasing disaster warning accuracy of local forests in Tiantong zone.

Advancements in phenotyping technology have enabled plant science researchers to gather large volumes of information from their experiments, especially those that evaluate multiple genotypes. To fully leverage these complex and often heterogeneous data sets (i.e. those that differ in format and structure), scientists must invest considerable time in data processing, and data management has emerged as a considerable barrier for downstream application. Here, we propose a pipeline to enhance data collection, processing, and management from plant science studies comprising of two newly developed open-source programs. The first, called AgTC, is a series of programming functions that generates comma-separated values file templates to collect data in a standard format using either a lab-based computer or a mobile device. The second series of functions, AgETL, executes steps for an Extract-Transform-Load (ETL) data integration process where data are extracted from heterogeneously formatted files, transformed to meet standard criteria, and loaded into a database. There, data are stored and can be accessed for data analysis-related processes, including dynamic data visualization through web-based tools. Both AgTC and AgETL are flexible for application across plant science experiments without programming knowledge on the part of the domain scientist, and their functions are executed on Jupyter Notebook, a browser-based interactive development environment. Additionally, all parameters are easily customized from central configuration files written in the human-readable YAML format. Using three experiments from research laboratories in university and non-government organization (NGO) settings as test cases, we demonstrate the utility of AgTC and AgETL to streamline critical steps from data collection to analysis in the plant sciences.

This presentation provides an overview of the open source training materials produced by the EUMETSAT Atmospheric Composition training team. The training materials covered in this presentation include: (1) LTPy - the Learning Tool for Python on Atmospheric Composition Data, (2) FANGS - Fire Applications with Next-Generation Satellites, (3) Dust Aerosol Detection, Monitoring and Forecasting, and (4) a self-paced training course on Identifying and Quantifying Dust using Satellite Data.&amp;#160; The first three sets of training materials were developed using Jupyter notebooks, which allow for a high-level of interactive learning, as it makes code, instructions and visualisations available in the same location. Executable notebooks are available on a dedicated Jupyterhub-based course platform which has the required programming environment and data already preinstalled. In addition, an accompanying Jupyter Book is also available for two of the training modules. The final self-paced training course consists of a series of mini-modules on the Moodle platform. The Learning tool for Python on Atmospheric Composition Data is a Python-based training course on Atmospheric Composition Data. The training course covers notebooks on data access, handling and processing, visualisation, case studies and exercises. LTPy features data from six different satellites, including the Copernicus satellites Sentinel-3 and Sentinel-5 as well as the polar-orbiting meteorological satellite series, Metop, and five different model-based product types from the two Copernicus services on Atmosphere Monitoring (CAMS) and Emergency Management (CEMS). The course facilitates the uptake and use of atmospheric composition data and showcases possible application areas.&amp;#160; FANGS - Fire Applications with Next-Generation Satellites features Python-based training material and application cases on fire detection and monitoring of the fire life-cycle. The training material makes use of proxy and simulated data, including data from precursor instruments of the Meteosat Third Generation (MTG) and EUMETSAT Polar System - Second Generation (EPS-SG) satellite missions. The training material consists of modular Jupyter notebook case studies on the 2020 wildfires in California, USA and the Mediterranean wildfires in 2021. In total, 24 notebooks were developed comprising five narrative notebooks and 19 workflow notebooks.&amp;#160; The training course on &amp;#8216;Dust Aerosol Detection, Monitoring and Forecasting&amp;#8217; provides a hands-on introduction to satellite-, ground- and model-based data used for dust monitoring and forecasting. This Python-based course is organised in three main chapters: (i) observations (satellite- and ground-based), (ii) forecast models and a (iii) practical case study. It features twelve different datasets derived from satellites, ground-based measurement networks and forecast models. The course material is developed in the form of well-described and modular Jupyter notebooks. In total, the course consists of 17 notebooks; 12 data workflows and five practical exercise notebooks. This presentation finally introduces a self-paced training course on identifying and quantifying dust using satellite data. This course is targeted at two audiences, namely, forecasters and researchers. At the end of the self-paced course, learners would have gained the skills to either (1) visualise dust events using Level 1 and Level 2 satellite data or (2) plot and interpret a time series of dust aerosol optical depth (AOD).&amp;#160;

A systematic comparison of the surface wind speed records extracted from the North American Regional Reanalysis (NARR) database and the network of automated surface observation stations of the Mexican National Weather Service has been conducted. The work presented here is based on a multi-step procedure that initiates with a point-to-point comparison of surface observations and NARR grid point, followed by the reconstruction of missing data from the quality-controlled observational data set using an iterative technique based on principal component analysis (PCA). Subsequently, the empirical orthogonal function (EOF) sets for NARR and the reconstructed observational data set were compared globally using a group comparison criterion that is also based on PCA. Different observational periods were evaluated, leading to the highest similarity between the two sets for the case in the year 2010. A wind speed map for Mexico was created by joining the quality-controlled, filtered and reconstructed observational data with the NARR map using a weighting technique based on distance weighting, local similarity of the EOFs and data recovery.

Maize is one of the major grain crops in China. Maize high production and stable yield are of great importance to ensure national food security. The dataset of physiological and ecological observations during the whole growth period of maize with different sowing dates in successive years can provide important data support for the study of the effects of climate change on maize ecosystem and its adaptation. Located in the northern part of the North China Plain, Hebei Gucheng Agricultural Meteorology National Observation and Research Station (Gucheng Station for short) is the only national platform for long-term experiments and observations of farmland ecosystem featuring by agrometeorology. This dataset is the experimental results of maize agroecosystem adaptation to climate change from 2018 to 2021 in Gucheng Station. It mainly contains the data of interannual and annual variation data of growth period, biomass, leaf area index, photosynthetic physiology, spectral characteristics, soil moisture, grain filling rate and yield of the same variety of maize at different sowing dates. This dataset is of great significance for the revision of agrometeorological business service indexes, the improvement and regional application of agrometeorological simulation model, and the development of agrometeorological applicable technology for the study of maize ecosystem adaptation to climate change.

Abstract. This contribution describes the development of a plugin for the geographic information system QGIS to interface the openModeller software package. The aim is to use openModeller to generate species’ potential distribution models for various archaeological applications (site catchment analysis, for example). Since the usage of openModeller’s command-line interface and configuration files can be a bit inconvenient, an extension of the QGIS user interface to handle these tasks, in combination with the management of the geographic data, was required. The implementation was realized in Python using PyQGIS and PyQT. The plugin, in combination with QGIS, handles the tasks of managing geographical data, data conversion, generation of configuration files required by openModeller and compilation of a project folder. The plugin proved to be very helpful with the task of compiling project datasets and configuration files for multiple instances of species occurrence datasets and the overall handling of openModeller. In addition, the plugin is easily extensible to take potential new requirements into account in the future.

We present a comprehensive study integrating satellite observations of ozone pollution, in situ measurements and chemistry transport model simulations for quantifying the role of anthropogenic emission reductions during the COVID-19 lockdown in spring 2020 over Europe. Satellite observations are derived from the IASI+GOME2 multispectral synergism, which provides particularly enhanced sensitivity to near-surface ozone pollution. These observations are first analysed in terms of differences between the average on 1–15 April 2020, when the strictest lockdown restrictions took place, and the same period in 2019. They show clear enhancements of near-surface ozone in Central Europe and Northern Italy, and some other hotspots, which are typically characterized by VOC-limited chemical regimes. An overall reduction of ozone is observed elsewhere, where ozone chemistry is limited by the abundance of NOx. The spatial distribution of positive and negative ozone concentration anomalies observed from space is in relatively good quantitative agreement with surface in situ measurements over the continent (a correlation coefficient of 0.55, a root-mean-squared difference of 11 ppb and the same standard deviation and range of variability). An average bias of ∼8 ppb between the two observational datasets is remarked, which can partly be explained by the fact the satellite approach retrieves partial columns of ozone with a peak sensitivity above the surface (near 2 km of altitude). For assessing the impact of the reduction of anthropogenic emissions during the lockdown, we adjust the satellite and in situ surface observations for withdrawing the influence of meteorological conditions in 2020 and 2019. This adjustment is derived from the chemistry transport model simulations using the meteorological fields of each year and identical emission inventories. This observational estimate of the influence of lockdown emission reduction is consistent for both datasets. They both show lockdown-associated ozone enhancements in hotspots over Central Europe and Northern Italy, with a reduced amplitude with respect to the total changes observed between the two years, and an overall reduction elsewhere over Europe and the ocean. Satellite observations additionally highlight the ozone anomalies in the regions remote from in situ sensors, an enhancement over the Mediterranean likely associated with maritime traffic emissions and a marked large-scale reduction of ozone elsewhere over ocean (particularly over the North Sea), in consistency with previous assessments done with ozonesondes measurements in the free troposphere. These observational assessments are compared with model-only estimations, using the CHIMERE chemistry transport model. For analysing the uncertainty of the model estimates, we perform two sets of simulations with different setups, differing in the emission inventories, their modifications to account for changes in anthropogenic activities during the lockdown and the meteorological fields. Whereas a general qualitative consistency of positive and negative ozone anomalies is remarked between all model and observational estimates, significant changes are seen in their amplitudes. Models underestimate the range of variability of the ozone changes by at least a factor 2 with respect to the two observational data sets, both for enhancements and decreases of ozone, while the large-scale ozone decrease is not simulated. With one of the setups, the model simulates ozone enhancements a factor 3 to 6 smaller than with the other configuration. This is partly linked to the emission inventories of ozone precursors (at least a 30 % difference), but mainly to differences in vertical mixing of atmospheric constituents depending on the choice of the meteorological model.

We present a comprehensive study integrating satellite observations of ozone pollution, in situ measurements and chemistry transport model simulations for quantifying the role of anthropogenic emission reductions during the COVID-19 lockdown in spring 2020 over Europe. Satellite observations are derived from the IASI+GOME2 multispectral synergism, which provides particularly enhanced sensitivity to near-surface ozone pollution. These observations are first analysed in terms of differences between the average on 1–15 April 2020, when the strictest lockdown restrictions took place, and the same period in 2019. They show clear enhancements of near-surface ozone in Central Europe and Northern Italy, and some other hotspots, which are typically characterized by VOC-limited chemical regimes. An overall reduction of ozone is observed elsewhere, where ozone chemistry is limited by the abundance of NOx. The spatial distribution of positive and negative ozone concentration anomalies observed from space is in relatively good quantitative agreement with surface in situ measurements over the continent (a correlation coefficient of 0.55, a root-mean-squared difference of 11 ppb and the same standard deviation and range of variability). An average bias of ∼8 ppb between the two observational datasets is remarked, which can partly be explained by the fact the satellite approach retrieves partial columns of ozone with a peak sensitivity above the surface (near 2 km of altitude). For assessing the impact of the reduction of anthropogenic emissions during the lockdown, we adjust the satellite and in situ surface observations for withdrawing the influence of meteorological conditions in 2020 and 2019. This adjustment is derived from the chemistry transport model simulations using the meteorological fields of each year and identical emission inventories. This observational estimate of the influence of lockdown emission reduction is consistent for both datasets. They both show lockdown-associated ozone enhancements in hotspots over Central Europe and Northern Italy, with a reduced amplitude with respect to the total changes observed between the two years, and an overall reduction elsewhere over Europe and the ocean. Satellite observations additionally highlight the ozone anomalies in the regions remote from in situ sensors, an enhancement over the Mediterranean likely associated with maritime traffic emissions and a marked large-scale reduction of ozone elsewhere over ocean (particularly over the North Sea), in consistency with previous assessments done with ozonesondes measurements in the free troposphere. These observational assessments are compared with model-only estimations, using the CHIMERE chemistry transport model. For analysing the uncertainty of the model estimates, we perform two sets of simulations with different setups, differing in the emission inventories, their modifications to account for changes in anthropogenic activities during the lockdown and the meteorological fields. Whereas a general qualitative consistency of positive and negative ozone anomalies is remarked between all model and observational estimates, significant changes are seen in their amplitudes. Models underestimate the range of variability of the ozone changes by at least a factor 2 with respect to the two observational data sets, both for enhancements and decreases of ozone, while the large-scale ozone decrease is not simulated. With one of the setups, the model simulates ozone enhancements a factor 3 to 6 smaller than with the other configuration. This is partly linked to the emission inventories of ozone precursors (at least a 30 % difference), but mainly to differences in vertical mixing of atmospheric constituents depending on the choice of the meteorological model.

We present a comprehensive study integrating satellite observations of ozone pollution, in situ measurements and chemistry transport model simulations for quantifying the role of anthropogenic emission reductions during the COVID-19 lockdown in spring 2020 over Europe. Satellite observations are derived from the IASI+GOME2 multispectral synergism, which provides particularly enhanced sensitivity to near-surface ozone pollution. These observations are first analysed in terms of differences between the average on 1–15 April 2020, when the strictest lockdown restrictions took place, and the same period in 2019. They show clear enhancements of near-surface ozone in Central Europe and Northern Italy, and some other hotspots, which are typically characterized by VOC-limited chemical regimes. An overall reduction of ozone is observed elsewhere, where ozone chemistry is limited by the abundance of NOx. The spatial distribution of positive and negative ozone concentration anomalies observed from space is in relatively good quantitative agreement with surface in situ measurements over the continent (a correlation coefficient of 0.55, a root-mean-squared difference of 11 ppb and the same standard deviation and range of variability). An average bias of ∼8 ppb between the two observational datasets is remarked, which can partly be explained by the fact the satellite approach retrieves partial columns of ozone with a peak sensitivity above the surface (near 2 km of altitude). For assessing the impact of the reduction of anthropogenic emissions during the lockdown, we adjust the satellite and in situ surface observations for withdrawing the influence of meteorological conditions in 2020 and 2019. This adjustment is derived from the chemistry transport model simulations using the meteorological fields of each year and identical emission inventories. This observational estimate of the influence of lockdown emission reduction is consistent for both datasets. They both show lockdown-associated ozone enhancements in hotspots over Central Europe and Northern Italy, with a reduced amplitude with respect to the total changes observed between the two years, and an overall reduction elsewhere over Europe and the ocean. Satellite observations additionally highlight the ozone anomalies in the regions remote from in situ sensors, an enhancement over the Mediterranean likely associated with maritime traffic emissions and a marked large-scale reduction of ozone elsewhere over ocean (particularly over the North Sea), in consistency with previous assessments done with ozonesondes measurements in the free troposphere. These observational assessments are compared with model-only estimations, using the CHIMERE chemistry transport model. For analysing the uncertainty of the model estimates, we perform two sets of simulations with different setups, differing in the emission inventories, their modifications to account for changes in anthropogenic activities during the lockdown and the meteorological fields. Whereas a general qualitative consistency of positive and negative ozone anomalies is remarked between all model and observational estimates, significant changes are seen in their amplitudes. Models underestimate the range of variability of the ozone changes by at least a factor 2 with respect to the two observational data sets, both for enhancements and decreases of ozone, while the large-scale ozone decrease is not simulated. With one of the setups, the model simulates ozone enhancements a factor 3 to 6 smaller than with the other configuration. This is partly linked to the emission inventories of ozone precursors (at least a 30 % difference), but mainly to differences in vertical mixing of atmospheric constituents depending on the choice of the meteorological model.

We develop a deep learning model (DL) for Indian Summer Monsoon (ISM) short-range precipitation forecasting using a ConvLSTM network. The model is built using daily precipitation records from both ground-based observations and remote sensing. Precipitation datasets from the Tropical Rainfall Measuring Mission and the India Meteorological Department are used for training, testing, forecasting, and comparison. For lead days 1 and 2, the correlation coefficient (CC), which was determined using predicted data from the previous five years and corresponding observational records (from both in-situ and remote sensing products), yielded values of 0.67 and 0.42, respectively. Interestingly, the CCs are even higher over the Western Ghats and Monsoon trough region. The model performance evaluated based on skill scores, Normalized Root Mean Squared Error (NRMSE), Mean absolute percentage error (MAPE) and ROC curves show a reasonable skill in short-range precipitation forecasting. Incorporating multivariable-based DL has the potential to match or even better the forecasts made by the state-of-the-art numerical weather prediction models.

JupyterLab is a web-based interactive development platform that is widely used in the Earth science community. Using Jupyter Notebooks, it is possible to perform data analysis tasks, annotate and visualize results in a way that is easy to reproduce, present and share with others. JupyterLab allows the use of &amp;#8220;extensions&amp;#8221;, which add functionality to the platform. One of these is Jupyter-AI [1], which allows the use of Large Language Models (LLMs), such as ChatGPT, Claude Sonnet and Ollama, within the JupyterLab environment, through&amp;#160; a chat interface or directly within notebooks. By integrating LLMs into JupyterLab, it is possible to leverage their code generation capabilities to assist a user to translate their analysis tasks from an idea to actual executable code in an efficient manner. One drawback of using these LLMs in tasks involving spatio&amp;#8211;temporal data is that the models typically do not have access to the data necessary for the analysis task and will often resort to generating fictional data or using placeholders in the code that they create. This requires the user to adapt the provided code to their data, which removes some of the utility provided by the LLM.In this context we make use of FrevaGPT, an approach for using LLMs in climate data analysis that allows for quick, complex and reproducible analyses of data sets, such as decadal climate model forecasts. Leveraging LLM&amp;#8217;s capability to write code and using few-shot prompting (in-context learning) allows the LLM to utilize Freva [2,3] (Free Evaluation Framework), a data search and analysis platform, which provides a standardised interface to spatio-temporal datasets hosted on an HPC cluster [4].&amp;#160;&amp;#160;FrevaGPT integrates seamlessly into Jupyter-AI and, by making use of the Freva library, combines the code-generating capabilities of LLMs with contextual understanding of how to access relevant datasets on the HPC cluster. This in addition with FrevaGPT&amp;#8217;s ability to execute generated code in an isolated environment on an HPC node, annotating and explaining any intermediate results, as well as automatically correcting errors encountered along the way, could serve as a starting ramp for researchers to efficiently produce new analysis products based on spatio-temporal climate data.&amp;#160;This PICO will include examples of using FrevaGPT within JupyterLab to analyse spatio-temporal datasets from the climate of the past, as well seasonal to decadal climate predictions.&amp;#160;&amp;#160;

HadUK‐Grid is a new dataset of gridded climate observations for the UK produced by the Met Office Hadley Centre for Climate Science and Services. The dataset interpolates in situ observations to a regular grid using methods developed in a previous equivalent dataset that had been made available to users since 2002 through the UK Climate Projections project (UKCIP02, UKCP09). The new dataset differs from the existing one in a number of key respects: higher spatial resolution, longer time series for some variables, improved consistency with regard to the pre‐processing of station observations, the use of publicly‐accessible ancillary data sources, a revised calculation sequence for some variables and improved version control. This makes for a dataset that is more internally consistent, more traceable and more reproducible. The result is a dataset of key UK climate variables of up to 1 km resolution from 1862 for monthly rainfall, 1884 for monthly temperature, 1891 for daily rainfall, 1929 for monthly sunshine and a wider set of variables with start dates from the 1960s to support the need for national climate monitoring and climate research.

Application of Google earth engine python API and NAIP imagery for land use and land cover classification: A case study in Florida, USA

Persistent Scatterer Interferometric SAR (PS-InSAR) is a widely used time-series technique for estimating surface deformation from multi-temporal SAR data. The rapidly increasing volume and resolution of SAR acquisitions from modern satellite missions pose significant challenges for scalability and extensibility. Meanwhile, novel algorithms developed by the InSAR community call for the improvement of maintainability of existing PS-InSAR software. We present DePSI, an open-source Python software package for PS-InSAR analysis designed to efficiently handle large InSAR datasets while adhering to modern Python software engineering standards. DePSI is based on the established DePSI algorithm originally implemented in MATLAB (van Leijen, 2014), and extends it with a scalable, modular, and community-oriented architecture. To address the challenges of large-scale InSAR processing, DePSI is built on Xarray and Dask, enabling efficient manipulation of multi-dimensional datasets and seamless scalability from local laptops to High-Performance Computing (HPC) environments. This design allows DePSI to process large SAR stacks while maintaining memory efficiency and parallel performance. DePSI adopts a functional programming–oriented design, facilitating the integration of new PSI algorithms alongside existing conventional methods. Comprehensive user and developer documentation, including example Jupyter notebooks, is provided to lower the barrier for adoption and extension. Modern software quality practices—such as unit testing, continuous integration, and version control—are fully implemented, ensuring robustness and long-term maintainability and fostering community-driven development. DePSI aims to provide a scalable, extensible, and high-quality open-source platform for next-generation PS-InSAR research and applications. In our contribution we will present the software design and use, together with example use cases.