Ensemble Data Assimilation with Model Error Inclusion for Estimating Fault-Slip Occurrence in Large-Scale Laboratory Experiments

We present a data assimilation system to estimate surface fluxes of CO 2 and other trace gases from observations of their atmospheric abundances. The system is based on ensemble data assimilation methods under development for Numerical Weather Prediction (NWP) and is the first of its kind to be used for CO 2 flux estimation. The system was developed to overcome computational limitations encountered when a large number of observations are used to estimate a large number of unknown surface fluxes. The ensemble data assimilation approach is attractive because it returns an approximation of the covariance, does not need an adjoint model or other linearization of the observation operator, and offers the possibility to optimize fluxes of chemically active trace gases (e.g., CH 4 , CO) in the same framework. We assess the performance of this new system in a pseudodata experiment that resembles the real problem we will apply this system to. The sensitivity of the method to the choice of several parameters such as the assimilation window size and the number of ensemble members is investigated. We conclude that the system is able to provide satisfactory flux estimates for the relatively large scales resolved by our current observing network and that the loss of information in the approximated covariances is an acceptable price to pay for the efficient computation of a large number of surface fluxes. The full potential of this data assimilation system will be used for near–real time operational estimates of North American CO 2 fluxes. This will take advantage of the large amounts of atmospheric data that will be collected by NOAA‐CMDL in conjunction with the implementation of the North American Carbon Program (NACP).

The ensemble Kalman filter (EnKF) is a data assimilation scheme based on the traditional Kalman filter update equation. An ensemble of forecasts are used to estimate the background-error covariances needed to compute the Kalman gain. It is known that if the same observations and the same gain are used to update each member of the ensemble, the ensemble will systematically underestimate analysis-error covariances. This will cause a degradation of subsequent analyses and may lead to filter divergence. For large ensembles, it is known that this problem can be alleviated by treating the observations as random variables, adding random perturbations to them with the correct statistics. Two important consequences of sampling error in the estimate of analysis-error covariances in the EnKF are discussed here. The first results from the analysis-error covariance being a nonlinear function of the backgrounderror covariance in the Kalman filter. Due to this nonlinearity, analysis-error covariance estimates may be negatively biased, even if the ensemble background-error covariance estimates are unbiased. This problem must be dealt with in any Kalman filter‐based ensemble data assimilation scheme. A second consequence of sampling error is particular to schemes like the EnKF that use perturbed observations. While this procedure gives asymptotically correct analysis-error covariance estimates for large ensembles, the addition of perturbed observations adds an additional source of sampling error related to the estimation of the observation-error covariances. In addition to reducing the accuracy of the analysis-error covariance estimate, this extra source of sampling error increases the probability that the analysis-error covariance will be underestimated. Because of this, ensemble data assimilation methods that use perturbed observations are expected to be less accurate than those which do not. Several ensemble filter formulations have recently been proposed that do not require perturbed observations. This study examines a particularly simple implementation called the ensemble square root filter, or EnSRF. The EnSRF uses the traditional Kalman gain for updating the ensemble mean but uses a ‘‘reduced’’ Kalman gain to update deviations from the ensemble mean. There is no additional computational cost incurred by the EnSRF relative to the EnKF when the observations have independent errors and are processed one at a time. Using a hierarchy of perfect model assimilation experiments, it is demonstrated that the elimination of the sampling error associated with the perturbed observations makes the EnSRF more accurate than the EnKF for the same ensemble size.

Methods to mitigate loss of variance due to sampling errors in ensemble data assimilation with non-local model parameters

Parametric uncertainty in convection parameterization is one major source of model errors that cause model climate drift. Convection parameter tuning has been widely studied in atmospheric models to help mitigate the problem. However, in a fully coupled general circulation model (CGCM), convection parameters which impact the ocean as well as the climate simulation may have different optimal values. This study explores the possibility of estimating convection parameters with an ensemble coupled data assimilation method in a CGCM. Impacts of the convection parameter estimation on climate analysis and forecast are analyzed. In a twin experiment framework, five convection parameters in the GFDL coupled model CM2.1 are estimated individually and simultaneously under both perfect and imperfect model regimes. Results show that the ensemble data assimilation method can help reduce the bias in convection parameters. With estimated convection parameters, the analyses and forecasts for both the atmosphere and the ocean are generally improved. It is also found that information in low latitudes is relatively more important for estimating convection parameters. This study further suggests that when important parameters in appropriate physical parameterizations are identified, incorporating their estimation into traditional ensemble data assimilation procedure could improve the final analysis and climate prediction.

Ensemble data assimilation methods for improving river water quality forecasting accuracy

Ensemble data assimilation methods, particularly iterative forms of ensemble smoother, are very useful assisted history matching techniques. One of the main challenges in the application of these methods is the excessive variance loss due to sampling errors occasioned by the limited size ensembles used in practical cases. The standard procedure to mitigate this problem is called distance-based localization. However, in this case, the data assimilation performance becomes highly dependent on the choice of the localization region. Moreover, there are several relevant examples of non-local model parameters and data types, in which cases, distance-based localization does not apply. This paper proposes a distance-free localization procedure that combines a least-squares support vector (LS-SVR) proxy with a non-isotropic tapering function proposed in the literature. We tested the proposed method in two versions of the PUNQ-S3 problem. In the first version, we tested the method to localize scalar (non-local) parameters. The results show relevant improvements, especially in terms of preserving the posterior variance of the ensemble compared to methods investigated in Lacerda et al. (J. Pet. Sci. Eng. 172,690–706, 2019). The second version considered the problem of localization of grid parameters. In this case, the proposed method outperformed a distance-based localization approach proposed in the literature.

<p><span>The Madden-Julian oscillation (MJO) is the dominant component of tropical intraseasonal variability with wide reaching impacts even on extratropical weather and climate patterns. However, predicting the MJO is challenging. One reason are suboptimal state estimates obtained with standard data assimilation (DA) approaches. Those are typically based on filtering methods with Gaussian approximations and do not consider physical properties that are specifically important for the MJO.</span></p><p><span>In our recent paper (Gleiter et al. 2022), a constrained ensemble DA method is applied to study the impact of different physical constraints on the state estimation and prediction of the MJO with the Skeleton model. The utilized quadratic programming ensemble (QPEns) algorithm extends the standard stochastic ensemble Kalman filter (EnKF) with specifiable constraints on the updates of all ensemble members. This allows to recover physically more consistent states and to respect possible associated non-Gaussian statistics. Our results demonstrate an overall improvement in the filtering and forecast skill when the model's total energy is conserved in the initial condition. The degree of benefit is found to be dependent on the observational setup and the strength of the model's nonlinear dynamics. It is also shown that even in cases where the statistical error in some waves remains comparable to the stochastic EnKF during the DA stage, their prediction is remarkably improved when using the initial state resulting from the QPEns.</span></p><p><strong>Gleiter, T., T. Janjic, N. Chen, </strong>2022, <em>Ensemble Kalman Filter based Data Assimilation for Tropical Waves in the MJO Skeleton Model, </em>QJR Meteorol Soc., https://doi.org/10.1002/qj.4245</p>

The Madden–Julian oscillation (MJO) is the dominant component of tropical intraseasonal variability, with wide‐reaching impacts even on extratropical weather and climate patterns. However, predicting the MJO is challenging. One reason is the suboptimal state estimates obtained with standard data assimilation (DA) approaches. These are typically based on filtering methods with Gaussian approximations and do not take into account physical properties that are important specifically for the MJO. In this article, a constrained ensemble DA method is applied to study the impact of different physical constraints on the state estimation and prediction of the MJO. The quadratic programming ensemble (QPEns) algorithm utilized extends the standard stochastic ensemble Kalman filter (EnKF) with specifiable constraints on the updates of all ensemble members. This allows us to recover physically more consistent states and to respect possible associated non‐Gaussian statistics. The study is based on identical twin experiments with an adopted nonlinear model for tropical intraseasonal variability. This so‐called skeleton model succeeds in reproducing the main large‐scale features of the MJO and closely related tropical waves, while keeping adequate simplicity for fast experiments on intraseasonal time‐scales. Conservation laws and other crucial physical properties from the model are examined as constraints in the QPEns. Our results demonstrate an overall improvement in the filtering and forecast skill when the model's total energy is conserved in the initial conditions. The degree of benefit is found to be dependent on the observational setup and the strength of the model's nonlinear dynamics. It is also shown that, even in cases where the statistical error in some waves remains comparable with the stochastic EnKF during the DA stage, their prediction is improved remarkably when using the initial state resulting from the QPEns.

A particle filter (PF) is an ensemble data assimilation method that does not assume Gaussian error distributions. Recent studies proposed local PFs (LPFs), which use localization as in the ensemble Kalman filter, to apply the PF for high-dimensional dynamics efficiently. Among others, Penny and Miyoshi developed an LPF in the form of the ensemble transform matrix of the Local Ensemble Transform Kalman Filter (LETKF). The LETKF has been widely accepted for various geophysical systems including numerical weather prediction (NWP) models. Therefore, implementing consistently with an existing LETKF code is useful. This study developed a software platform for the LPF and its Gaussian mixture extension (LPFGM) by making slight modifications to the LETKF code with a simplified global climate model known as Simplified Parameterizations, Primitive Equation Dynamics (SPEEDY). A series of idealized twin experiments were accomplished under the ideal model assumption. With large inflation by the relaxation to prior spread, the LPF showed stable filter performance with dense observations but became unstable with sparse observations. The LPFGM showed more accurate and stable performances than the LPF with both dense and sparse observations. In addition to the relaxation parameter, regulating the resampling frequency and the amplitude of Gaussian kernels was important for the LPFGM. With a spatially inhomogeneous observing network, the LPFGM was superior to the LETKF in sparsely observed regions where the background ensemble spread and non-Gaussianity are larger. The SPEEDY-based LETKF, LPF, and LPFGM systems were available as open-source software on Github (https://github.com/skotsuki/speedy-lpf) and can be adapted to various models relatively easily like the LETKF.

<p>This study investigates the effect of erroneous parameter values for state and parameter estimation using data assimilation. The numerical model chosen for this study solves the van der Pol equation, a second-order differential equation that can be used to simulate oscillatory processes, such as earthquakes. In the model, discrepancies in the parameter values can have a significant influence on the forecasted states of the model, which is even more significant if its behaviour is highly nonlinear. When observations of the state variables are assimilated to update the parameters along with the state variables, this improves the quality of the state forecasts. The results suggest that corrections in the model parameter not only recover the actual parameter values but also reduce state-variable errors after a certain time period. However, data assimilation that updates the state variables but not the parameter can lead to erroneous estimates as well as forecasts of the oscillation. Since the study is performed on a simplified nonlinear model framework, the consequences of these results for data assimilation in more realistic models remains to be investigated.</p>

Assimilating multi-site measurements for semi-distributed hydrological model updating

The uncertainty associated with many observed and modeled quantities of interest in Earth system prediction can be represented by mixed probability distributions that are neither discrete nor continuous. For instance, a forecast probability of precipitation can have a finite probability of zero precipitation, consistent with a discrete distribution. However, nonzero values are not discrete and are represented by a continuous distribution; the same is true for rainfall rate. Other examples include snow depth, sea ice concentration, the amount of a tracer, or the source rate of a tracer. Some Earth system model parameters may also have discrete or mixed distributions. Most ensemble data assimilation methods do not explicitly consider the possibility of mixed distributions. The quantile-conserving ensemble filter framework is extended to explicitly deal with discrete or mixed distributions. An example is given using bounded normal rank histogram probability distributions applied to observing system simulation experiments in a low-order tracer advection model. Analyses of tracer concentration and tracer source are shown to be improved when using the extended methods. A key feature of the resulting ensembles is that there can be ensemble members with duplicate values. An extension of the rank histogram diagnostic method to deal with potential duplicates shows that the ensemble distributions from the extended assimilation methods are more consistent with the truth. Significance Statement Data assimilation is a statistical method that is used to combine information from computer forecasts with measurements of the Earth system. The result is a better estimate of what is occurring in the physical system. As an example, data assimilation is used for making weather predictions. Some Earth system quantities, like precipitation, have special values that can occur very frequently. For instance, zero rainfall is quite common, while any other specific amount of rainfall, say, 0.42 in., is unusual. New data assimilation tools that work well for quantities like this are introduced and should lead to better estimates and predictions of the Earth system.

A particle filter (PF) is an ensemble data assimilation method that does not assume Gaussian error distributions. Recent studies proposed local PFs (LPFs), which use localization as in the ensemble Kalman filter, to apply the PF for high-dimensional dynamics efficiently. Among others, Penny and Miyoshi developed an LPF in the form of the ensemble transform matrix of the Local Ensemble Transform Kalman Filter (LETKF). The LETKF has been widely accepted for various geophysical systems including numerical weather prediction (NWP) models. Therefore, implementing consistently with an existing LETKF code is useful. This study developed a software platform for the LPF and its Gaussian mixture extension (LPFGM) by making slight modifications to the LETKF code with a simplified global climate model known as Simplified Parameterizations, Primitive Equation Dynamics (SPEEDY). A series of idealized twin experiments were accomplished under the ideal model assumption. With large inflation by the relaxation to prior spread, the LPF showed stable filter performance with dense observations but became unstable with sparse observations. The LPFGM showed more accurate and stable performances than the LPF with both dense and sparse observations. In addition to the relaxation parameter, regulating the resampling frequency and the amplitude of Gaussian kernels was important for the LPFGM. With a spatially inhomogeneous observing network, the LPFGM was superior to the LETKF in sparsely observed regions where the background ensemble spread and non-Gaussianity are larger. The SPEEDY-based LETKF, LPF, and LPFGM systems were available as open-source software on Github (https://github.com/skotsuki/speedy-lpf) and can be adapted to various models relatively easily like the LETKF.

Abstract. Climate reanalyses are essential for studying climate variability, understanding climate processes, and initializing climate predictions. We present CoRea1860+ (Wang and Counillon, 2025, https://doi.org/10.11582/2025.00009), a 30-member coupled reanalysis spanning from 1860 to the present, produced using the Norwegian Climate Prediction Model (NorCPM) and assimilating sea surface temperature (SST) observations. NorCPM combines the Norwegian Earth System Model with the ensemble Kalman filter data assimilation method. SST, available throughout the entire period, serves as the primary source of instrumental oceanic measurements prior to the 1950s. CoRea1860+ belongs to the category of sparse-input reanalyses, designed to minimize artefacts arising from changes in the observation network over time. By exclusively assimilating oceanic data, this reanalysis offers valuable insights into the ocean's role in driving climate system variability, including its influence on the atmosphere and sea ice. This study first describes the numerical model, the SST dataset, and the assimilation implementation used to produce CoRea1860+. It then provides a comprehensive evaluation of the reanalysis across four key aspects, namely reliability, ocean variability, sea ice variability, and atmospheric variability, benchmarked against more than 10 independent reanalyses and observational datasets. Overall, CoRea1860+ demonstrates strong reliability, particularly in observation-rich periods, and provides a reasonable representation of climate variability. It successfully captures key features such as multi-decadal variability and long-term trends in ocean heat content, the Atlantic meridional overturning circulation, and sea ice variability in both hemispheres. Furthermore, to some extent, CoRea1860+ agrees with the reference atmospheric datasets for surface air temperature, precipitation, sea level pressure, and 500 hPa geopotential height, especially in the tropics where air–sea interactions are most pronounced.

Since the publication of the first edition of this highly regarded textbook, the value of data assimilation has become widely recognized across the Earth sciences and beyond. Data assimilation methods are now being applied to many areas of prediction and forecasting, including extreme weather events, wildfires, infectious disease epidemics, and economic modeling. This second edition provides a broad introduction to applications across the Earth systems and coupled Earth–human systems, with an expanded range of topics covering the latest developments of variational, ensemble, and hybrid data assimilation methods. New toy models and intermediate-complexity atmospheric general circulation models provide hands-on engagement with key concepts in numerical weather prediction, data assimilation, and predictability. The inclusion of computational projects, exercises, lecture notes, teaching slides, and sample exams makes this textbook an indispensable and practical resource for advanced undergraduate and graduate students, researchers, and practitioners who work in weather forecasting and climate prediction.