A systematic method of parameterisation estimation using data assimilation

In this study, we present a probabilistic urban inundation modeling framework that combines high-resolution process-based modeling with observed information via ensemble data assimilation (DA). We investigate the impact of multivariate flood observations on improving urban inundation prediction accuracy through synthetic experiments. The framework leverages diverse flood observations from both the urban surface and sewer system, integrating them into the modeling process using non-Gaussian sequential DA methods, such as particle filtering. The modeling framework employs the H12 model for integrated 1D sewer network and 2D surface inundation analyses, with synthetic experiments conducted in an urban catchment in Osaka, Japan. Prior to implementing DA, a sensitivity analysis is performed to assess the effects of uncertainties in inundation modeling. Major uncertainty components, such as input forcings and storm drain box efficiency, are perturbed to evaluate their influence. The synthetic DA experiments analyze the influence of various types of flood observations, including urban surface inundation depths and sewer water levels, on the posterior distributions of the model ensemble. Additionally, the impact of observation location and density on DA performance is evaluated. This study demonstrates the potential of utilizing diverse flood observations in data assimilation to enhance the accuracy and reliability of urban flood predictions.

The Kuramoto–Sivashinsky equation plays an important role as a low‐dimensional prototype for complicated fluid dynamics systems having been studied due to its chaotic pattern forming behavior. Up to now, efforts to carry out data assimilation with this 1‐D model were restricted to variational adjoint methods domain and only Chorin and Krause (Proc. Natl. Acad. Sci. 2004; 101(42):15013–15017) tested it using a sequential Bayesian filter approach. In this work we compare three sequential data assimilation methods namely the Kalman filter approach, the sequential Monte Carlo particle filter approach and the maximum likelihood ensemble filter methods. This comparison is to the best of our knowledge novel. We compare in detail their relative performance for both linear and nonlinear observation operators. The results of these sequential data assimilation tests are discussed and conclusions are drawn as to the suitability of these data assimilation methods in the presence of linear and nonlinear observation operators. Copyright © 2009 John Wiley & Sons, Ltd.

In the past few years sequential data assimilation (SDA) methods have emerged as the best possible method at hand to properly treat all sources of error in hydrological modeling. However, very few studies have actually implemented SDA methods using realistic input error models for precipitation. In this study we use particle filtering as a SDA method to propagate input errors through a conceptual hydrologic model and quantify the state, parameter and streamflow uncertainties. Recent progress in satellite‐based precipitation observation techniques offers an attractive option for considering spatiotemporal variation of precipitation. Therefore, we use the PERSIANN‐CCS precipitation product to propagate input errors through our hydrologic model. Some uncertainty scenarios are set up to incorporate and investigate the impact of the individual uncertainty sources from precipitation, parameters and also combined error sources on the hydrologic response. Also probabilistic measures are used to quantify the quality of ensemble prediction.

Challenges for data assimilation – from convective‐scale to climate

The Arctic is the most rapidly warming region on Earth, and it is facing considerable environmental change. These changes are beginning to alter the way humanity uses and exploits the Arctic region, with commercial activities such as tourism, fishing, mineral and oil extraction, and shipping on the increase. Increased human activity in the Arctic has implications for safety and environmental conservation, and can present difficulties for local communities who rely on subsistence hunting and fishing, and community resupply. Environmental prediction for the Arctic is therefore becoming increasingly important, but our ability to accurately predict the Arctic atmosphere–sea-ice–ocean system is relatively immature, with short-term forecasts significantly less accurate than those produced for the mid-latitudes. There are numerous challenges with respect to forecasting systems in the Arctic: there is a relatively sparse in-situ observing network; some satellite-based observations are hard to use because of the snow and ice-covered surfaces; it is difficult and expensive to make detailed research-quality observations; and the dominant physical processes are different to those at mid-latitudes and over the tropics. To address these issues, the Year of Polar Prediction (YOPP1) was developed as one of the key elements of the WMO's Polar Prediction Project2. On Wednesday 21 November 2018, the Royal Meteorological Society hosted a meeting at the Society of Chemical Industry (SCI) in London to discuss recent advances and challenges associated with the understanding and prediction of key Arctic processes using numerical models and observations – including new insights provided by several field campaigns and modelling experiments carried out under the YOPP umbrella. The first talk was entitled: What are the challenges and priorities for improved prediction and climate monitoring of the Arctic? (Irina Sandu, ECMWF). Irina explained that in order to improve Arctic predictions and reanalyses, which constitute a great tool for Arctic climate monitoring, work is needed in three areas: (i) enhanced coupled modelling, (ii) data assimilation methods and (iii) the effective use of observations in the numerical weather prediction systems. Arctic regions pose specific challenges for each of these three areas because model errors are large, model uncertainty is not necessarily well represented, and in-situ observations are sparse. Moreover, while the Arctic is very rich in terms of remote sensing observations from polar orbiting satellites, some of these observations are difficult to use in data assimilation because of ambiguous signal properties, particularly over snow and sea-ice and in cloudy situations. Irina introduced the challenges and priorities in each of these three key areas and described recent and ongoing efforts to evaluate and improve predictions in the Arctic, and beyond, made in the framework of the YOPP modelling activities and the EU-APPLICATE3 project. She highlighted the fact that, compared to previous similar initiatives, YOPP puts an additional emphasis on numerical experimentation, in a concerted effort to exploit observations for model improvement and drive developments in data assimilation and the design of observing systems. The next talk presented was: What are the limitations of Arctic sea ice remote sensing products, and what opportunities can they provide for improving predictive skill of Arctic forecasts? (Ed Blockley, Met Office). Ed explained that, in response to declining sea-ice cover, human activity in the Arctic is increasing, and that accurate forecasts of Arctic sea ice – on a variety of different timescales – are therefore becoming increasingly important for the safety of human activities in the region. He then provided a brief overview of how sea ice has been traditionally initialised in short-range forecasting and seasonal prediction systems, and explained that most operational systems initialise sea ice purely using passive microwave sea ice concentration products. He gave an overview of the passive microwave remote sensing products typically used for initialisation of sea ice concentration, as well as the (relatively new) satellite-derived sea ice thickness products currently available. Information was provided about how these observations are derived, what the uncertainties and limitations are, and what new avenues of research are currently underway to improve measurements of Arctic sea ice from space. Ed also illustrated the potential impact of assimilating satellite sea ice thickness products into our operational prediction systems. Using the Met Office GloSea seasonal prediction system, he showed how sea ice thickness values inferred from the CryoSat-2 satellite altimetry could be used within the initialisation process to considerably improve forecasts of Arctic summer sea ice on seasonal timescales. Next up was: Chasing the source of the AMOC (Atlantic Meridional Overturning Circulation) atmosphere–ocean coupling in the Iceland and Greenland Seas (Ian Renfrew, University of East Anglia) Ian described a coordinated meteorological and oceanographic field campaign over the Iceland and southern Greenland Seas that took place in February and March 2018. The aim of the campaign was to characterise the atmospheric forcing and the ocean response of coupled atmosphere–ocean processes – in particular, cold-air outbreaks in the vicinity of the marginal-ice-zone, and their triggering of oceanic heat loss and the generation of dense waterbody body masses. He described how the team observed the spatial structure and variability of surface flux fields in the region, and the weather systems that dictate these fluxes, through the first meteorological field campaign in the Iceland Sea. This was done as part of a coupled atmosphere–ocean field campaign in winter 2018 involving a rare wintertime research cruise, airborne observations and a host of ocean and atmosphere observing systems. In-situ observations of air–sea interaction processes were made from several platforms. Ian presented some highlights from the field campaign and introduced some early findings from the post-campaign analysis – most notably a comprehensive set of observations of a marine cold air outbreak which were obtained via radiosonde releases and the research aircraft. Mixing it up: What connects Arctic clouds and sea ice? was a talk presented by Ian Brooks (University of Leeds). Ian explained that the single largest source of uncertainty in climate models arises from the representation of clouds; this is particularly true in the Arctic where, for much of the year, low-level boundary-layer clouds dominate the surface energy budget. The time-evolution and properties of these clouds are intimately linked their interactions with the surface and boundary layer structure. The properties of both clouds and boundary layer (and indeed the surface) are essentially sub-grid-scale in climate and weather forecast models, and thus many important features are poorly represented by current parameterisations. Ian emphasised that observations show that the summertime boundary-layer over sea ice is often decoupled below cloud base, inhibiting turbulent mixing – and hence the transport of aerosols and water vapour – between cloud and the surface. This decoupling is not reproduced by models, and the controls on this turbulent structure are poorly understood. He provided a review of the state of knowledge, alongside some of the recent research on Arctic boundary layer processes and their effects on clouds and the surface energy budget. A unique feature of polar boundary layers is the occurrence of a humidity inversion, where absolute humidity increases across the top of the boundary layer. This allows boundary-layer clouds to extend into the temperature inversion and results in a weakening of turbulence generated by cloud-top radiative cooling. Finally, he showed how recent field measurements have improved the parameterisation of surface turbulent fluxes over marginal sea ice, which will ultimately affect the model representation of the evolution of sea ice. The view from above Arctic snow at 89-325GHz: What can surface emissivity on these channels tell us about snowpack stratigraphy? was the next talk by Chawn Harlow ( Met Office). Chawn introduced Measurements of Arctic Cloud, Snow and Sea Ice in the Marginal Ice ZonE (MACSSIMIZE) – an airborne campaign carried out by the Met Office in March 2018 that focussed on evaluation of snow microwave emission models in the 89–325GHz frequency range. He described how the campaign included a ground-based component to collect snow pit measurements co-located with the airborne radiometric measurements, and that these were now being used to improve snow physical models for use in future Met Office numerical weather prediction systems. An explanation of the main motivation for MACSSIMIZE followed, both from the point of view of snow remote sensing and for improving atmospheric data assimilation, along with a description of the emissivity retrievals obtained during the campaign at 89, 118, 157, 183.3, 243 and 325GHz. Chawn then described how the MACSSIMIZE observations will be used to evaluate the snow thermodynamic and emissivity models used at the Met Office in the short term, and how they will eventually be used to improve assimilation of microwave atmospheric sounding data over snow-covered surfaces in the polar regions. Finally, Richard Essery (University of Edinburgh) presented: Why tundra snow is upside down in models, and why it matters?. Richard described how Earth System and Numerical Weather Prediction models are beginning to use more sophisticated representations of terrestrial snow, drawing on snow physics models that were first developed for forecasting avalanche risk. He explained that, having been developed for deep mid-latitude mountain snow, these models neglect important physical processes occurring in shallow tundra snow subjected to high winds and low temperatures. Understanding these processes is important for representing the thermal properties of Arctic snow and exploiting information from microwave remote sensing over snow-covered surfaces. Richard provided an overview of the ground-based component of the YOPP-endorsed MACSSIMIZE campaign, carried out at Trail Valley Creek in northwestern Canada. He presented results obtained from the snow pit measurements within MACSSIMIZE and drew attention to the existence of a granular depth hoar layer located below the wind-slab, which is a common feature of Arctic tundra snow. The resulting vertical density profile – with low-density depth hoar below a high-density wind-slab layer – is not reproduced by current snow models. Field instruments that can replace visual and subjective measurements of snow structure are now becoming available and can provide data for improving models. With speakers focussing on operational forecasting, Earth observation, and on observation and modelling of key Arctic processes such as clouds, sea ice and snow, this meeting illustrated the broad range of the UK's expertise and interest in Arctic prediction, and highlighted the significant contribution that scientists in the UK are making to YOPP. This RMetS meeting was convened by Ed Blockley and Chawn Harlow (Met Office) and was chaired by Ed Blockley (Met Office), with Jonathan Day (ECMWF) as rapporteur. The event was sponsored by the Centre for Polar Observation and Modelling (CPOM).

This Special Issue presents efficient formulations and implementations of sequential and variational data assimilation methods. The methods address three important issues in the context of operational data assimilation: efficient implementation of localization methods, sampling methods for approaching posterior ensembles under non-linear model errors, and adjoint-free formulations of four dimensional variational methods.

Data Assimilation for Marine Ecosystem Models

Quantifying continental-scale river discharge is essential to understanding the terrestrial water cycle but is susceptible to errors caused by a lack of observations and the limitations of hydrodynamic modeling. Data assimilation (DA) methods are increasingly used to estimate river discharge in combination with emerging river-related remote sensing products (e.g., water surface elevation [WSE], water surface slope, river width, and flood extent). However, directly comparing simulated WSE to satellite altimetry data remains challenging (e.g., because of large biases between simulations and observations or uncertainties in parameters), and large errors can be introduced when satellite observations are assimilated into hydrodynamic models. In this study we performed direct, anomaly, and normalized value assimilation experiments to investigate the capacity of DA to improve river discharge within the current limitations of hydrodynamic modeling. We performed hydrological DA using a physically-based empirical localization method applied to the Amazon Basin. We used satellite altimetry data from ENVISAT, Jason 1, and Jason 2. Direct DA was the baseline assimilation method and was subject to errors due to biases in the simulated WSE. To overcome these errors, we used anomaly DA as an alternative to direct DA. We found that the modeled and observed WSE distributions differed considerably (e.g., differences in amplitude, seasonal flow variation, and a skewed distribution due to limitations of the hydrodynamic models). Therefore, normalized value DA was performed to improve discharge estimation. River discharge estimates were improved at 24 %, 38 %, and 62 % of stream gauges in the direct, anomaly, and normalized value assimilations relative to simulations without DA. Normalized value assimilation performed best for estimating river discharge given the current limitations of hydrodynamic models. Most gauges within the river reaches covered by satellite observations accurately estimated river discharge, with Nash-Sutcliffe efficiency (NSE) > 0.6. The amplitudes of WSE variation were improved in the normalized DA experiment. Furthermore, in the Amazon Basin, normalized assimilation (median NSE = 0.50) improved river discharge estimation compared to open-loop simulation with the global hydrodynamic model (median NSE = 0.42). River discharge estimation using direct DA methods was improved by 7 % with calibration of river bathymetry based on NSE. The direct DA approach outperformed the other DA approaches when runoff was considerably biased, but anomaly DA performed best when the river bathymetry was erroneous. The uncertainties in hydrodynamic modeling (e.g., river bottom elevation, river width, simplified floodplain dynamics, and the rectangular cross-section assumption) should be improved to fully realize the advantages of river discharge DA through the assimilation of satellite altimetry. This study contributes to the development of a global river discharge reanalysis product that is consistent spatially and temporally.

Ensuring a reliable water supply in the face of changing conditions and growing demand is a critical global challenge. Water distribution networks (WDNs) are essential infrastructure, but traditional modeling methods based on historical data often struggle to adapt in real-time and integrate new information. In order to lower model errors in WDN models, this study explores the use of a Data Assimilation (DA) method that makes use of the Ensemble Kalman Filter. The study explores the effectiveness of the DA method, and a novel Greedy Algorithm (GA) for optimizing the location of sensors. The study shows that the DA method improves the WDN model and the GA is able to successfully determine optimal sensor locations. It was observed that increasing the number of sensors in the WDN increased the effectiveness of the DA method. This study highlights the potential of data assimilation to improve WDN modeling. Water utilities can gain from more precise predictions, greater system performance, and efficient decision-making for the management and maintenance of water distribution networks by allowing models to adapt and improve dynamically

Reconstructing spatiotemporal systems from sparse observations remains a long-standing challenge in several domains, including geoscience, air pollution and fluid dynamics. While various data assimilation (DA) and machine learning (ML) methods have shown potential, they still face significant challenges (see [1]): The computational burden of conventional DA algorithms (including error covariance specification), particularly for multivariate, high-dimensional systems. Sparse and movable sensor placement, which makes conventional ML models (typically requiring fixed and regularly distributed input data) cumbersome. The ill-defined nature of the sparse reconstruction problem, which poses significant risks of overfitting. We will present our recent works aimed at addressing these challenges. More specifically, we developed latent DA algorithms [2] to reduce the computational burden of variational DA methods. These algorithms demonstrate great potential in efficiently assimilating sparse observations within a reduced-order latent space constructed by neural networks, thanks to the TorchDA library [3]. The latter enables GPU implementation of mainstream data assimilation methods and supports non-explicit state-observation transformation functions, provided they can be learned by a neural network.We have also employed advanced deep learning techniques, including Voronoi-tessellation CNNs [4] and Vision Transformer-based autoencoders [5], to learn mappings from sparse observations to the complete physical space. These approaches effectively address challenges such as movable sensor placements and varying sensor numbers. Their integration with DA algorithms has also been evaluated.Finally, our recent work explores [6] the utility of generative AI techniques, particularly denoising diffusion models, for field reconstruction from sparse observations. Generative AI methods offer two main advantages: first, they produce a sample from a probability distribution rather than predicting the mean as a fixed output, which can help mitigate overfitting caused by the illy-defined problem. Second, they inherently function as ensemble predictors by generating several samples, facilitating uncertainty quantification, which is essential in data assimilation. The numerical results tested on cases ranging from fluid dynamics benchmarks to semi-operational air pollution simulations will also be discussed.[1] Cheng, S., Quilodrán-Casas, C., Ouala, S., Farchi, A., Liu, C., Tandeo, P., Fablet, R., Lucor, D., Iooss, B., Brajard, J., Xiao, D., Janjic, T., Ding, W., Guo, Y., Carrassi, A., Bocquet, M. and Arcucci, R, 2023. Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review. IEEE/CAA Journal of Automatica Sinica[2] Cheng, S., Chen, J., Anastasiou, C., Angeli, P., Matar, O.K., Guo, Y.K., Pain, C.C. and Arcucci, R., 2023. Generalised latent assimilation in heterogeneous reduced spaces with machine learning surrogate models. Journal of Scientific Computing[3] Cheng, S., Min, J., Liu, C. and Arcucci, R., 2025. TorchDA: A Python package for performing data assimilation with deep learning forward and transformation functions. Computer Physics Communications[4] Cheng, S., Liu, C., Guo, Y. and Arcucci, R., 2024. Efficient deep data assimilation with sparse observations and time-varying sensors. Journal of Computational Physics[5] Fan, H., Cheng, S., de Nazelle, A.J. and Arcucci, R., 2024. ViTAE-SL: a vision transformer-based autoencoder and spatial interpolation learner for field reconstruction. Computer Physics Communications [6] Zhuang, Y., Cheng, S. and Duraisamy, K., 2025. Spatially-aware diffusion models with cross-attention for global field reconstruction with sparse observations. Computer Methods in Applied Mechanics and Engineering

<div><span>Snow as a major part of the cryosphere is an important component of Earth’s hydrological cycle and energy balance. Understanding the microstructural, macrophysical, thermal and optical properties of the snowpack is essential for integration into numerical models and there is a great need for accurate snow data at different spatial and temporal resolutions to address the challenges of changing snow conditions.</span></div><div><span>Physical snow properties are currently determined by traditional ground-based measurements as well as remote sensing, over a range of temporal and spatial scales, following considerable developments in instrument technology over recent years. </span></div><div><span>Data assimilation (DA)</span><span> methods are widely used</span> <span>to combine data from different observations</span><span> with numerical model using uncertainties </span><span>of observed and modeled variables  to produce an optimal estimate. DA provides a reliable improvement of the initial states of the numerical model and a benefit for hydrological and snow model forecasts. </span></div><div> </div><div><span>European efforts to harmonize approaches for validation, and methodologies of snow measurement practices, instrumentation, algorithms and data assimilation techniques were coordinated by the European Cooperation in Science and Technology (COST) Action ES1404 “HarmoSnow”, entitled, “A European network for a harmonized monitoring of snow for the benefit of climate change scenarios, hydrology and numerical weather prediction” (2014-2018) .</span></div><div><span>One of the key objectives of the action was “Advance the application of snow DA in numerical weather prediction (NWP) and hydrological models, and show its benefit for weather and hydrological forecasting as well as other applications.” </span></div><div><span>One key result from COST HarmoSnow is a better knowledge about the diversity of usage of snow observations in DA, forcing, monitoring, validation, or verification within NWP, hydrology, snow and climate models. The main parts of this knowledge are retrieved from a COST HarmoSnow survey exploring the common practices on the use of snow observations in different modeling environments. We will show results from the survey and their implications towards standardized and improved usage of snow observations in various data assimilation applications.</span></div>

Sequential data assimilation (DA) was performed on three-dimensional flow fields of a circular jet measured by tomography particle image velocimetry (tomo-PIV). The work focused on an in-depth analysis of the flow enhancement and the pressure determination from volumetric flow measurement data. The jet was issued from a circular nozzle with an inner diameter of D= 20 mm. A split-screen configuration including two high-speed cameras was used to capture the particle images from four different views for a tomography reconstruction of the voxels in the tomo-PIV measurement. Planar PIV was also performed to obtain the benchmark two-dimensional velocity fields for validation. The adjoint-based sequential DA scheme was used with the measurement uncertainty implanted using a threshold function to recover the flow fields with high fidelity and fewer measurement errors. The pressure was determined by either the direct mode, with implementation directly in the DA solver, or by the separate mode, which included solving the Poisson equation on the DA-recovered flow fields. Sequential DA recovered high signal-to-noise flow fields that had piecewise-smooth temporal variations due to the intermittent constraints of the observations, while only the temporal sequence of the fields at the observational instances was selected as the DA output. Errors were significantly reduced, and DA improved the divergence condition of the three-dimensional flow fields. DA also enhanced the dynamical features of the vortical structures, and the pressure determined by both modes successfully captured the downstream convection signatures of the vortex rings.

Since the publication of the first edition of this highly regarded textbook, the value of data assimilation has become widely recognized across the Earth sciences and beyond. Data assimilation methods are now being applied to many areas of prediction and forecasting, including extreme weather events, wildfires, infectious disease epidemics, and economic modeling. This second edition provides a broad introduction to applications across the Earth systems and coupled Earth–human systems, with an expanded range of topics covering the latest developments of variational, ensemble, and hybrid data assimilation methods. New toy models and intermediate-complexity atmospheric general circulation models provide hands-on engagement with key concepts in numerical weather prediction, data assimilation, and predictability. The inclusion of computational projects, exercises, lecture notes, teaching slides, and sample exams makes this textbook an indispensable and practical resource for advanced undergraduate and graduate students, researchers, and practitioners who work in weather forecasting and climate prediction.

Data assimilation and inverse methods play a key role in integrating remote-sensing and in-situ Earth and Geospace observations into models of the Earth and Geospace system and subsystems, enabling weather prediction and climate projection of high societal relevance. The problem presents considerable methodological and computational challenges because of the high-dimensionality and non-linearity of the underling dynamics of the Earth and Geospace system and the large volumes of heterogeneous observations from various observing platforms, including Low-Earth-Orbit and geosynchronous-Earth-orbit satellites as well as ground-based and airborne platforms. To make the problem tractable, most data assimilation methods resort to scalable methods derived from well-established Gaussian process-based formulation of the sequential Bayesian estimation problem. One such approach is to use approximate Kalman filters that rely on low-rank approximations of the covariance matrices. In ensemble data assimilation, the covariance is expressed in the subspace spanned by the model ensemble that evolves dynamically. Though the low-rank ensemble-based approximation allows the covariance to track non-linear dynamics with a moderate computational cost, the impact of the observations that lie outside of the subspace represented by the ensemble cannot be effectively incorporated into the assimilation analysis. One heuristic remedy widely used in numerical weather prediction is to adopt a hybrid method that uses the full-rank stationary covariance and the ensemble-based covariance. As a scalable approach to modeling the full-rank nonstationary covariance, the use of a multiresolution-based covariance is discussed.

A new sequential data assimilation method is discussed. It is based on forecasting the error statistics using Monte Carlo methods, a better alternative than solving the traditional and computationally extremely demanding approximate error covariance equation used in the extended Kalman filter. The unbounded error growth found in the extended Kalman filter, which is caused by an overly simplified closure in the error covariance equation, is completely eliminated. Open boundaries can be handled as long as the ocean model is well posed. Well‐known numerical instabilities associated with the error covariance equation are avoided because storage and evolution of the error covariance matrix itself are not needed. The results are also better than what is provided by the extended Kalman filter since there is no closure problem and the quality of the forecast error statistics therefore improves. The method should be feasible also for more sophisticated primitive equation models. The computational load for reasonable accuracy is only a fraction of what is required for the extended Kalman filter and is given by the storage of, say, 100 model states for an ensemble size of 100 and thus CPU requirements of the order of the cost of 100 model integrations. The proposed method can therefore be used with realistic nonlinear ocean models on large domains on existing computers, and it is also well suited for parallel computers and clusters of workstations where each processor integrates a few members of the ensemble.