Comment on gmd-2021-375

<strong class="journal-contentHeaderColor">Abstract.</strong> Atmospheric inversion of carbon dioxide (CO<span class="inline-formula">₂</span>) measurements to better understand carbon sources and sinks has made great progress over the last 2 decades. However, most of the studies, including a four-dimensional variational ensemble Kalman filter and Bayesian synthesis approaches, directly obtain only fluxes, while CO<span class="inline-formula">₂</span> concentration is derived with the forward model as part of a post-analysis. Kang et al. (2012) used the local ensemble transform Kalman filter (LETKF), which updates the CO<span class="inline-formula">₂</span>, surface carbon flux (SCF), and meteorology fields simultaneously. Following this track, a system with a short assimilation window and a long observation window was developed (Liu et al., 2019). However, this data assimilation system faces the challenge of maintaining carbon mass conservation. To overcome this shortcoming, here we apply a constrained ensemble Kalman filter (CEnKF) approach to ensure the conservation of global CO<span class="inline-formula">₂</span> mass. After a standard LETKF procedure, an additional assimilation is used to adjust CO<span class="inline-formula">₂</span> at each model grid point and to ensure the consistency between the analysis and the first guess of the global CO<span class="inline-formula">₂</span> mass. Compared to an observing system simulation experiment without mass conservation, the CEnKF significantly reduces the annual global SCF bias from <span class="inline-formula">∼</span> 0.2 to less than 0.06 Gt and slightly improves the seasonal and annual performance over tropical and southern extratropical regions. We show that this system can accurately track the spatial distribution of annual mean SCF. And the system reduces the seasonal flux root mean square error from a priori to analysis by 48 %–90 %, depending on the continental region. Moreover, the 2015–2016 El Niño impact is well captured with anomalies mainly in the tropics.

Abstract. Atmospheric inversion of carbon dioxide (CO2) measurements to better understand carbon sources and sinks has made great progress over the last 2 decades. However, most of the studies, including a four-dimensional variational ensemble Kalman filter and Bayesian synthesis approaches, directly obtain only fluxes, while CO2 concentration is derived with the forward model as part of a post-analysis. Kang et al. (2012) used the local ensemble transform Kalman filter (LETKF), which updates the CO2, surface carbon flux (SCF), and meteorology fields simultaneously. Following this track, a system with a short assimilation window and a long observation window was developed (Liu et al., 2019). However, this data assimilation system faces the challenge of maintaining carbon mass conservation. To overcome this shortcoming, here we apply a constrained ensemble Kalman filter (CEnKF) approach to ensure the conservation of global CO2 mass. After a standard LETKF procedure, an additional assimilation is used to adjust CO2 at each model grid point and to ensure the consistency between the analysis and the first guess of the global CO2 mass. Compared to an observing system simulation experiment without mass conservation, the CEnKF significantly reduces the annual global SCF bias from ∼ 0.2 to less than 0.06 Gt and slightly improves the seasonal and annual performance over tropical and southern extratropical regions. We show that this system can accurately track the spatial distribution of annual mean SCF. And the system reduces the seasonal flux root mean square error from a priori to analysis by 48 %–90 %, depending on the continental region. Moreover, the 2015–2016 El Niño impact is well captured with anomalies mainly in the tropics.

[1] In ensemble Kalman filter, space localization is used to reduce the impact of long-distance sampling errors in the ensemble estimation of the forecast error covariance. When two variables are not physically correlated, their error covariance is still estimated by the ensemble and, therefore, it is dominated by sampling errors. We introduce a “variable localization” method, zeroing out such covariances between unrelated variables to the problem of assimilating carbon dioxide concentrations into a dynamical model using the local ensemble transform Kalman filter (LETKF) in an observing system simulation experiments (OSSE) framework. A system where meteorological and carbon variables are simultaneously assimilated is used to estimate surface carbon fluxes that are not directly observed. A range of covariance structures are explored for the LETKF, with emphasis on configurations allowing nonzero error covariance between carbon variables and the wind field, which affects transport of atmospheric CO2, but not between CO2 and the other meteorological variables. Such variable localization scheme zeroes out the background error covariance among prognostic variables that are not physically related, thus reducing sampling errors. Results from the identical twin experiments show that the performance in the estimation of surface carbon fluxes obtained using variable localization is much better than that using a standard full covariance approach. The relative improvement increases when the surface fluxes change with time and model error becomes significant.

This paper develops a parallel implementation of the ensemble Kalman filter (EnKF) based on shrinkage covariancematrix estimation. The EnKF is a sequential Monte Carlo method for parameter and state estimation of highly nonlinearmodels. In the context of ensemble based methods, background error correlations are estimated via anensemble of model realizations. In practice, the model dimension (number of model components) is several timeslarger than the ensemble size owing to the computational effort involved in one single model propagation. Thisconstraint yields to spurious correlations (correlation between distant model components in space) in the backgrounderror estimations. Commonly, localization methods are utilized in order to reduce the impact of spuriouscorrelations. For instance, correlations of model components are reduced based on some function of their physicaldistances. Some EnKF implementations are formulated in such manner that, localization methods are implicitlyinvolved in the assimilation of observations. In this context, one of the best ensemble Kalman filter implementationsis the local ensemble transform Kalman filter (LETKF). In the LETKF, each model component is surrounded by alocal box of size r and then, the information contained within the local domain (i.e., observed components) is used toperform the assimilation. However, for sparse observational networks, r can become sufficiently large with respectto the ensemble size and therefore, the local analysis corrections can be impacted by spurious correlations. Wepropose an ensemble Kalman filter implementation in which the background error covariance matrix is estimated viathe Rao-Blackwell Ledoit and Wolf (RBLW) estimator. This estimator has proven to be better conditioned than thetrue (background) error covariance matrix. In addition, the explicit computation of the RBLW estimator is notrequired in our formulation. This implies enormous savings in terms of memory considering the typical order ofbackground error covariance matrices O(10e8) Furthermore, the proposed EnKF implementation can be efficientlyperformed in parallel: each model component is surrounded by a local box, a local background error covariancematrix is estimated based on the RBLW estimator and then, the ensemble Kalman filter equations are locally used inorder to compute the local analysis corrections. After the local assimilation, each model component is mapped backonto the full model domain from where the global analysis solution is obtained. In operational data assimilation, EnKF implementations are attractive owing to their potential for parallelization. Experiments are performed making use of the Atmospheric General Circulation Model SPEEDY. The proposed method is coded in FORTRAN and MPI is used for the local assimilation of observations. The predicted variables are the zonal and meridional wind components, the temperature and the specific humidity. For each model variable, 8 layers with the T-63 resolution (192 x 96 grid points) are considered which provide a total number of 589,824 model components. The ensemble size is 94. The number of processors utilized are 96, 256, 512, 768, 1024 and 1536. The results reveal that, the use of the RBLW estimator can mitigate the impact of spurious correlations under sparse observational networks and even more, for different number of processors, the computational effort of the proposed EnKF implementation is comparable to that of the LETKF where no covariance estimation is performed. In terms of accuracy, the proposed implementation performs better than the LETKF formulation in the Root Mean Square Error sense.

A particle filter (PF) is an ensemble data assimilation method that does not assume Gaussian error distributions. Recent studies proposed local PFs (LPFs), which use localization as in the ensemble Kalman filter, to apply the PF for high-dimensional dynamics efficiently. Among others, Penny and Miyoshi developed an LPF in the form of the ensemble transform matrix of the Local Ensemble Transform Kalman Filter (LETKF). The LETKF has been widely accepted for various geophysical systems including numerical weather prediction (NWP) models. Therefore, implementing consistently with an existing LETKF code is useful. This study developed a software platform for the LPF and its Gaussian mixture extension (LPFGM) by making slight modifications to the LETKF code with a simplified global climate model known as Simplified Parameterizations, Primitive Equation Dynamics (SPEEDY). A series of idealized twin experiments were accomplished under the ideal model assumption. With large inflation by the relaxation to prior spread, the LPF showed stable filter performance with dense observations but became unstable with sparse observations. The LPFGM showed more accurate and stable performances than the LPF with both dense and sparse observations. In addition to the relaxation parameter, regulating the resampling frequency and the amplitude of Gaussian kernels was important for the LPFGM. With a spatially inhomogeneous observing network, the LPFGM was superior to the LETKF in sparsely observed regions where the background ensemble spread and non-Gaussianity are larger. The SPEEDY-based LETKF, LPF, and LPFGM systems were available as open-source software on Github (https://github.com/skotsuki/speedy-lpf) and can be adapted to various models relatively easily like the LETKF.

A particle filter (PF) is an ensemble data assimilation method that does not assume Gaussian error distributions. Recent studies proposed local PFs (LPFs), which use localization as in the ensemble Kalman filter, to apply the PF for high-dimensional dynamics efficiently. Among others, Penny and Miyoshi developed an LPF in the form of the ensemble transform matrix of the Local Ensemble Transform Kalman Filter (LETKF). The LETKF has been widely accepted for various geophysical systems including numerical weather prediction (NWP) models. Therefore, implementing consistently with an existing LETKF code is useful. This study developed a software platform for the LPF and its Gaussian mixture extension (LPFGM) by making slight modifications to the LETKF code with a simplified global climate model known as Simplified Parameterizations, Primitive Equation Dynamics (SPEEDY). A series of idealized twin experiments were accomplished under the ideal model assumption. With large inflation by the relaxation to prior spread, the LPF showed stable filter performance with dense observations but became unstable with sparse observations. The LPFGM showed more accurate and stable performances than the LPF with both dense and sparse observations. In addition to the relaxation parameter, regulating the resampling frequency and the amplitude of Gaussian kernels was important for the LPFGM. With a spatially inhomogeneous observing network, the LPFGM was superior to the LETKF in sparsely observed regions where the background ensemble spread and non-Gaussianity are larger. The SPEEDY-based LETKF, LPF, and LPFGM systems were available as open-source software on Github (https://github.com/skotsuki/speedy-lpf) and can be adapted to various models relatively easily like the LETKF.

Groundwater contamination assessment can be useful in taking proper actions during environmental emergency. A three-dimensional deterministic model was taken into consideration to simulate the advective-diffusive transport of non-conservative contaminant in groundwater. Multiple stochastic data assimilation techniques, such as, ensemble Kalman filter (EnKF), local ensemble transform Kalman filter (LETKF), and the global form of the LETKF, denoted as GETKF were applied to the model. The results show that data assimilation improved contaminant concentration prediction. The EnKF method reduced the root-mean-square-error (RMSE) of the contaminant prediction from 12.5 mg/L to 1.31 mg/L, whereas the LETKF reduced that to 0.46 mg/L and GETKF reduced that to 0.38 mg/L. Approximately 89.48%, 96.30% and 96.82% improvement were made by EnKF, LETKF, and GETKF, respectively. The sensitivity analysis suggest that these data assimilation techniques are very sensitive to the observation noise, process noise, and ensemble size.

Summary When the ensemble Kalman filter (EnKF) is used for history matching, the resulting updates to reservoir properties sometimes exceed physical bounds, especially when the problem is highly nonlinear. Problems of this type are often encountered during history matching compositional models using the EnKF. In this paper, we illustrate the problem using an example in which the updated molar density of CO2 in some regions is observed to take negative values while molar densities of the remaining components are increased. Standard truncation schemes avoid negative values of molar densities but do not address the problem of increased molar densities of other components. The results can include a spurious increase in reservoir pressure with a subsequent inability to maintain injection. In this paper, we present a method for constrained EnKF (CEnKF), which takes into account the physical constraints on the plausible values of state variables during data assimilation. In the proposed method, inequality constraints are converted to a small number of equality constraints, which are used as virtual observations for calibrating the model parameters within plausible ranges. The CEnKF method is tested on a 2D compositional model and on a highly heterogeneous three-phase-flow reservoir model. The effect of the constraints on mass conservation is illustrated using a 1D Buckley-Leverett flow example. Results show that the CEnKF technique is able to enforce the nonnegativity constraints on molar densities and the bound constraints on saturations (all phase saturations must be between 0 and 1) and achieve a better estimation of reservoir properties than is obtained using only truncation with the EnKF.

We proposed (and accomplished) the development of an Ensemble Kalman Filter (EnKF) approach for the estimation of surface carbon fluxes as if they were parameters, augmenting the model with them. Our system is quite different from previous approaches, such as carbon flux inversions, 4D-­Var, and EnKF with approximate background error covariance (Peters et al., 2008). We showed (using observing system simulation experiments, OSSEs) that these differences lead to a more accurate estimation of the evolving surface carbon fluxes at model grid-scale resolution. The main properties of the LETKF-­C are: a) The carbon cycle LETKF is coupled with the simultaneous assimilation of the standard atmospheric variables, so that the ensemble wind transport of the CO2 provides an estimation of the carbon transport uncertainty. b) The use of an assimilation window (6hr) much shorter than the months-long windows used in other methods. This avoids the inevitable “blurring” of the signal that takes place in long windows due to turbulent mixing since the CO2 does not have time to mix before the next window. In this development we introduced new, advanced techniques that have since been adopted by the EnKF community (Kang, 2009, Kang et al., 2011, Kang et al. 2012). These advances include “variable localization” that reduces sampling errors in the estimation of the forecast error covariance, more advanced adaptive multiplicative and additive inflations, and vertical localization based on the time scale of the processes. The main result has been obtained using the LETKF-­C with all these advances, and assimilating simulated atmospheric CO2 observations from different observing systems (surface flask observations of CO2 but no surface carbon fluxes observations, total column CO2 from GoSAT/OCO-­2, and upper troposphere AIRS retrievals). After a spin-­up of about one month, the LETKF-­C succeeded in reconstructing the true evolving surface fluxes of carbon at a model grid resolution. When applied to the CAM3.5 model, the LETKF gave very promising results as well, although only one month is available.

Abstract. Prediction of spatiotemporal chaotic systems is important in various fields, such as numerical weather prediction (NWP). While data assimilation methods have been applied in NWP, machine learning techniques, such as reservoir computing (RC), have recently been recognized as promising tools to predict spatiotemporal chaotic systems. However, the sensitivity of the skill of the machine-learning-based prediction to the imperfectness of observations is unclear. In this study, we evaluate the skill of RC with noisy and sparsely distributed observations. We intensively compare the performances of RC and local ensemble transform Kalman filter (LETKF) by applying them to the prediction of the Lorenz 96 system. In order to increase the scalability to larger systems, we applied a parallelized RC framework. Although RC can successfully predict the Lorenz 96 system if the system is perfectly observed, we find that RC is vulnerable to observation sparsity compared with LETKF. To overcome this limitation of RC, we propose to combine LETKF and RC. In our proposed method, the system is predicted by RC that learned the analysis time series estimated by LETKF. Our proposed method can successfully predict the Lorenz 96 system using noisy and sparsely distributed observations. Most importantly, our method can predict better than LETKF when the process-based model is imperfect.

A data assimilation system based on the ensemble square-root Kalman filter (EnSRF) is extended to include the additional capability of assimilating polarimetric radar variables. It is used to assess the impact of assimilating additional polarimetric observations on convective storm analysis in the Observing System Simulation Experiment (OSSE) framework. The polarimetric variables considered include differential reflectivity ZDR, reflectivity difference Zdp, and specific differential phase KDP. To simulate the observational data more realistically, a new error model is introduced for characterizing the errors of the nonpolarimetric and polarimetric radar variables. The error model includes both correlated and uncorrelated error components for reflectivities at horizontal and vertical polarizations (ZH and ZV, respectively). It is shown that the storm analysis is improved when polarimetric variables are assimilated in addition to ZH or in addition to both ZH and radial velocity Vr. Positive impact is largest when ZDR, Zdp, and KDP are assimilated all together. Improvement is generally larger in vertical velocity, water vapor, and rainwater mixing ratios. The rainwater field benefits the most while the impacts on horizontal wind components and snow mixing ratio are smaller. Improvement is found at all model levels even though the polarimetric data, after the application of thresholds, are mostly limited to the lower levels. Among ZDR, Zdp, and KDP, ZDR is found to produce the largest positive impact on the analysis. It is suggested that ZDR provides more independent information than the other variables. The impact of polarimetric data is also expected to be larger when they are used to retrieve drop size distribution parameters. The polarimetric radar data thresholding prior to assimilation is found to be necessary to minimize the impact of noise. This study is believed to be the first to directly assimilate (simulated) polarimetric data into a numerical model.

Abstract. A local particle filter (LPF) is introduced that outperforms traditional ensemble Kalman filters in highly nonlinear/non-Gaussian scenarios, both in accuracy and computational cost. The standard sampling importance resampling (SIR) particle filter is augmented with an observation-space localization approach, for which an independent analysis is computed locally at each grid point. The deterministic resampling approach of Kitagawa is adapted for application locally and combined with interpolation of the analysis weights to smooth the transition between neighboring points. Gaussian noise is applied with magnitude equal to the local analysis spread to prevent particle degeneracy while maintaining the estimate of the growing dynamical instabilities. The approach is validated against the local ensemble transform Kalman filter (LETKF) using the 40-variable Lorenz-96 (L96) model. The results show that (1) the accuracy of LPF surpasses LETKF as the forecast length increases (thus increasing the degree of nonlinearity), (2) the cost of LPF is significantly lower than LETKF as the ensemble size increases, and (3) LPF prevents filter divergence experienced by LETKF in cases with non-Gaussian observation error distributions.

Numerical weather prediction is becoming increasingly reliant on the assimilation of cloud‐affected satellite observations. Their assimilation implicitly linearizes nonlinear observation operators in ensemble Kalman filters. The linearization causes the posterior to deviate from its linear approximation, which is often used for analysis verification. We investigate the linearization error for visible and infrared radiances in the ensemble adjustment Kalman filter (EAKF) using observing‐system simulation experiments (OSSEs). We found that increments can be detrimental for small first‐guess departures, but they are beneficial on average. The increments were typically about half of their linear approximation. Similarly, the ensemble spread reduction was smaller than its linear approximation, and sometimes negative (spread increase), but overall the mean nonlinear variance adjustment was consistent with the mean nonlinear squared error reduction. Lastly, the linear approximation overestimated the analysis mean absolute error (MAE) reduction by 37% for visible reflectance and 71% for infrared brightness temperature. Thus, the linear approximation of the posterior of observed variables, such as satellite radiances, should not be used for verification.

This study develops an ensemble Kalman filter (EnKF)-based regional ocean data assimilation system, in which the local ensemble transform Kalman filter (LETKF) is implemented with the Stony Brook Parallel Ocean Model (sbPOM) version 1.0 to assimilate satellite and in-situ observations at a daily frequency. A series of sensitivity experiments are performed with various settings of the incremental analysis update (IAU) and covariance inflation methods, for which the relaxation-to-prior perturbations and spread (RTPP and RTPS, respectively) and multiplicative inflation (MULT) are considered. We evaluate the geostrophic balance and the analysis accuracy compared with the control experiment in which the IAU and covariance inflation are not applied. The results show that the IAU improves the geostrophic balance, degrades the accuracy, and reduces the ensemble spread, and that the RTPP and RTPS have the opposite effect. The experiment using the combination of the IAU and RTPP results in significant improvement for both balance and accuracy when the RTPP parameter is 0.8–0.9. The combination of the IAU and RTPS improves the balance when the RTPS parameter is ≤ 0.8 and increases the accuracy for the parameter values between 1.0 and 1.1, but the balance and accuracy are not improved significantly at the same time. The experiments with MULT do not demonstrate sufficient skill in maintaining the balance and reproducing the surface flow field regardless of whether the IAU is applied or not. Therefore, the combination of the IAU and RTPP with the parameter of 0.8–0.9 is found to be the best setting for the EnKF-based ocean data assimilation system.

ABSTRACTWe introduce a new formulation of the ensemble forecast sensitivity developed by Liu and Kalnay with a small correction from Li et al. The new formulation, like the original one, is tested on the simple Lorenz 40-variable model. We find that, except for short-range forecasts, the use of localization in the analysis, necessary in ensemble Kalman filter (EnKF) when the number of ensemble members is much smaller than the model's degrees of freedom, has a negative impact on the accuracy of the sensitivity. This is because the impact of an observation during the analysis (i.e. the analysis increment associated with the observation) is transported by the flow during the integration, and this is ignored when the ensemble sensitivity uses a fixed localization. To address this problem, we introduce two approaches that could be adapted to evolve the localization during the estimation of forecast sensitivity to the observations. The first one estimates the non-linear evolution of the initial localization but is computationally expensive. The second one moves the localization with a constant estimation of the group velocity. Both methods succeed in improving the ensemble estimations for longer forecasts.Overall, the adjoint and ensemble forecast impact estimations give similarly accurate results for short-range forecasts, except that the new formulation gives an estimation of the fraction of observations that improve the forecast closer to that obtained by data denial (Observing System Experiments). For longer-range forecasts, they both deteriorate for different reasons. The adjoint sensitivity becomes noisy due to the forecast non-linearities not captured in the linear tangent model and the adjoint. The ensemble sensitivity becomes less accurate due to the use of a fixed localization, a problem that could be ameliorated with an evolving adaptive localization. Advantages of the new formulation include it being simpler than the original formulation and computationally more efficient and that it can be applied to other EnKF methods in addition to the local ensemble transform Kalman filter.