Efficient Ensemble Covariance Localization in Variational Data Assimilation

At the National Centers for Environmental Prediction, the global ensemble forecasts from the ensemble Kalman filter scheme in the Global Forecast System are applied in a regional three-dimensional (3D) and a four dimensional (4D) ensemble–variational (EnVar) data assimilation system. The application is a one-way variational method using hybrid static and ensemble error covariances. To enhance impact, three new features have been added to the existing EnVar system in the Gridpoint Statistical Interpolation (GSI). First, the constant coefficients that assign relative weight between the ensemble and static background error are now allowed to vary in the vertical. Second, a new formulation is introduced for the ensemble contribution to the analysis surface pressure. Finally, in order to make use of the information in the ensemble mean that is disregarded in the existing EnVar in GSI, the trajectory correction, a novel approach, is introduced. Relative to the application of a 3D variational data assimilation algorithm, a clear positive impact on 1–3-day forecasts is realized when applying 3DEnVar analyses in the North American Mesoscale Forecast System (NAM). The 3DEnVar DA system was operationally implemented in the NAM Data Assimilation System in August 2014. Application of a 4DEnVar algorithm is shown to further improve forecast accuracy relative to the 3DEnVar. The approach described in this paper effectively combines contributions from both the regional and the global forecast systems to produce the initial conditions for the regional NAM system.

Data assimilation is a method that combines observations (ie, real world data) of a state of a system with model output for that system in order to improve the estimate of the state of the system and thereby the model output. The model is usually represented by a discretized partial differential equation. The data assimilation problem can be formulated as a large scale Bayesian inverse problem. Based on this interpretation we will derive the most important variational and sequential data assimilation approaches, in particular three‐dimensional and four‐dimensional variational data assimilation (3D‐Var and 4D‐Var) and the Kalman filter. We will then consider more advanced methods which are extensions of the Kalman filter and variational data assimilation and pay particular attention to their advantages and disadvantages. The data assimilation problem usually results in a very large optimization problem and/or a very large linear system to solve (due to inclusion of time and space dimensions). Therefore, the second part of this article aims to review advances and challenges, in particular from the numerical linear algebra perspective, within the various data assimilation approaches.

The effect on weather forecast performance of incorporating ensemble covariances into the initial covariance model of the four-dimensional variational data assimilation (4D-Var) Naval Research Laboratory Atmospheric Variational Data Assimilation System-Accelerated Representer (NAVDAS-AR) is investigated. This NAVDAS-AR-hybrid scheme linearly combines the static NAVDAS-AR initial background error covariance with a covariance derived from an 80-member flow-dependent ensemble. The ensemble members are generated using the ensemble transform technique with a (three-dimensional variational data assimilation) 3D-Var-based estimate of analysis error variance. The ensemble covariances are localized using an efficient algorithm enabled via a separable formulation of the localization matrix. The authors describe the development and testing of this scheme, which allows for assimilation experiments using differing linear combinations of the static and flow-dependent background error covariances. The tests are performed for two months of summer and two months of winter using operational model resolution and the operational observational dataset, which is dominated by satellite observations. Results show that the hybrid mode data assimilation scheme significantly reduces the forecast error across a wide range of variables and regions. The improvements were particularly pronounced for tropical winds. The verification against radiosondes showed a greater than 0.5% reduction in vector wind RMS differences in areas of statistical significance. The verification against self-analysis showed a greater than 1% reduction from verifying against analyses between 2- and 5-day lead time at all eight vertical levels examined in areas of statistical significance. Using the Navy's summary of verification results, the Navy Operational Global Atmospheric Prediction System (NOGAPS) scorecard, the improvements resulted in a score (+1) that justifies a major system upgrade.

Mechanisms controlling the heat budget of the North Sea are investigated based on a combination of satellite sea surface temperature measurements and numerical model simulations. Lateral heat fluxes across the shelf edge and into the Baltic Sea as well as vertical ocean‐atmosphere heat exchange are considered. A 3‐D variational (3DVAR) data assimilation (DA) scheme is applied, which contains assumed model error correlations that depend on the mixed layer depth derived from a coupled circulation/ocean wave model. The analysis balances pressure gradients introduced by temperature modifications. Significant hydrodynamic model response to DA was found, which should be considered in the heat budget estimations. The observed change of the current velocity field decreases the lateral advective volume/heat exchanges between the North Sea and the Atlantic, yielding an increased heat flux from the Atlantic into the North Sea and more heat flux from the sea to the atmosphere. The largest DA impact on volume/heat transport is in the Norwegian Channel, where the dominant process is Eulerian transport, followed by tidal pumping and wind pumping. Further analysis reveals an acceleration of the along‐shelf current at the northern edge of the North Sea, a decrease in the horizontal pressure gradient from the Atlantic to the North Sea, and a reduction of the Eulerian transport of volume/heat outward the North Sea. Furthermore, the coupling between the circulation model and the wave model has significant impacts on lateral heat advection in the DA run, which is due to the wave impact on the mixed layer depth.

Data assimilation is predominantly used for state estimation, combining observational data with model predictions to produce an updated model state that most accurately approximates the true system state whilst keeping the model parameters fixed. This updated model state is then used to initiate the next model forecast. Even with perfect initial data, inaccurate model parameters will lead to the growth of prediction errors. To generate reliable forecasts, we need good estimates of both the current system state and the model parameters. This article presents research into data assimilation methods for morphodynamic model state and parameter estimation. First, we focus on state estimation and describe implementation of a three‐dimensional variational (3D‐Var) data assimilation scheme in a simple 2D morphodynamic model of Morecambe Bay, UK. The assimilation of observations of bathymetry derived from synthetic aperture radar (SAR) satellite imagery and a ship‐borne survey is shown to significantly improve the predictive capability of the model over a 2‐year run. Here, the model parameters are set by manual calibration; this is laborious and is found to produce different parameter values depending on the type and coverage of the validation dataset. The second part of this article considers the problem of model parameter estimation in more detail. We explain how, by employing the technique of state augmentation, it is possible to use data assimilation to estimate uncertain model parameters concurrently with the model state. This approach removes inefficiencies associated with manual calibration and enables more effective use of observational data. We outline the development of a novel hybrid sequential 3D‐Var data assimilation algorithm for joint state‐parameter estimation and demonstrate its efficacy using an idealised 1D sediment transport model. The results of this study are extremely positive and suggest that there is great potential for the use of data assimilation‐based state‐parameter estimation in coastal morphodynamic modelling. Copyright © 2012 Royal Meteorological Society

Many ensemble-based data assimilation (DA) methods use observation space localization to mitigate the sampling errors due to the insufficient ensemble members. Observation space localization is simpler and more timesaving than model space localization in implementation, but more difficult to directly assimilate satellite radiance observations, a kind of non-local observations. The vertical locations of radiance observations are undetermined and the transmission of observational information is thereby obstructed. To determine the vertical coordinates of radiance observations, a weighted average hypsometry is proposed. Using this hypsometry, AMSU-A radiance observations are directly assimilated with an ensemble four-dimensional variational (En4DVar) DA system. It consists of a four-dimensional ensemble-variational (4DEnVar) system providing ensemble covariance and a 4DVar system. Observing system simulation experiments show that the hypsometry alleviates the degradations in the late period of medium-range forecast in the Northern Extratropics that occur in the traditional peak-based hypsometry. It obviously improves the analysis qualities and forecast skills of the En4DVar system and its two components, especially in the Southern Extratropics, when incorporating AMSU-A radiance observations. The improvement in the En4DVar-initialized forecast is comparable to that in the 4DVar-initialized forecast in the Southern Extratropics and Tropics. It indicates that a proper hypsometry enables efficient extraction of useful information from AMSU-A radiance observations by 4DEnVar with observation space localization. Therefore, the 4DEnVar provides high-quality ensemble covariances for En4DVar.

The use of Four-Dimensional variational (4D-Var) data assimilation technology in the context of sea dynamics problems, with a sensitivity analysis of model results to observation errors, is presented. The technology is applied to a numerical model of ocean circulation developed at the Marchuk Institute of Numerical Mathematics, Russian Academy of Sciences (INM RAS), with the use of the splitting method and complemented by 4D-Var data assimilation with covariance matrices of background and observation errors. The variational data assimilation involves iterative procedures to solve inverse problems so as to correct sea surface heat fluxes for the model under consideration. An algorithm is formulated to study the sensitivity of the model outputs, considered as output functions after assimilation, to the observation errors. The algorithm reveals the regions where the output function gradient is the largest for the average sea surface temperature (SST) in a selected area, obtained by assimilation. In the numerical experiments, a 4D variational problem of SST assimilation for the Baltic Sea area is solved.

Kriging-enhanced ensemble variational data assimilation for scalar-source identification in turbulent environments

Previous studies and different methods of estimating short-range forecast errors are summarized. Zonally and temporally averaged statistics based on differences of one- and two-day forecasts valid at the same time are presented and an attempt is made to explain many of the features by reference to dynamical concepts. Vertical correlation length-scale tends to increase with horizontal correlation scale but to be very short in the Tropics; horizontal scale is longest in the Tropics and in the stratosphere. The variations in vertical correlation are much more pronounced for largely balanced variables such as rotational wind and temperature than they are for divergent wind or humidity. The extratropics are dominated by an equivalent barotropic mode with the level of the maximum wind amplitude (and the zero crossing of the temperature correlation) being determined by the tropopause. Surface pressure is negatively correlated with low-level temperature as expected (except over the Antarctic plateau where it is positively correlated); it is also negatively correlated with temperatures near/above the tropopause in the extratropics. The covariance model used in The Met. Office Global Three-Dimensional Variational (3D-Var) Data Assimilation system represents the variation of vertical covariances with latitude reasonably well, but the longer horizontal scales in the stratosphere are not currently reproduced. The implied covariances used operationally have been modified so that the correlation length-scales, both horizontal and vertical, are somewhat shorter than those direct from the forecast differences. Recent changes to the representation are briefly described, with an indication of their impact on the forecasts. The impacts are significant relative to other changes tested, and the covariance model has played a major role in the successful implementation and subsequent improvement of our 3D-Var system.

A four-dimensional variational data assimilation (4DVar) system of the LASG/IAP Climate Ocean Model, version 1.0 (LICOM1.0), named LICOM-3DVM, has been developed using the three-dimensional variational data assimilation of mapped observation (3DVM), a 4DVar method newly proposed in the past two years. Two experiments with 12-year model integrations were designed to validate it. One is the assimilation run, called ASSM, which incorporated the analyzed weekly sea surface temperature (SST) fields from Reynolds and Smith (OISST) between 1990 and 2001 once a week by the LICOM-3DVM. The other is the control run without any assimilation, named CTL. ASSM shows that the simulated temperatures of the upper ocean (above 50 meters), especially the SST of equatorial Pacific, coincide with the Tropic Atmosphere Ocean (TAO) mooring data, the World Ocean Atlas 2001 (WOA01) data and the Met Office Hadley Centre’s sea ice and sea surface temperature (HadISST) data. It decreased the cold bias existing in CTL in the eastern Pacific and produced a Nino index that agrees with observation well. The validation results suggest that the LICOM-3DVM is able to effectively adjust the model results of the ocean temperature, although it’s hard to correct the subsurface results and it even makes them worse in some areas due to the incorporation of only surface data. Future development of the LICOM-3DVM is to include subsurface in situ observations and satellite observations to further improve model simulations.

Extending an oceanographic variational scheme to allow for affordable hybrid and four-dimensional data assimilation

Chapter 21 - Non-Gaussian Variational Data Assimilation

In variational assimilation, the most probable state of a dynamical system under Gaussian assumptions for the prior and likelihood can be found by solving a least‐squares minimization problem. In recent years, we have seen the popularity of hybrid variational data assimilation methods for Numerical Weather Prediction. In these methods, the prior error covariance matrix is a weighted sum of a climatological part and a flow‐dependent ensemble part, the latter being rank deficient. The nonlinear least squares problem of variational data assimilation is solved using iterative numerical methods, and the condition number of the Hessian is a good proxy for the convergence behavior of such methods. In this article, we study the conditioning of the least squares problem in a hybrid four‐dimensional variational data assimilation (Hybrid 4D‐Var) scheme by establishing bounds on the condition number of the Hessian. In particular, we consider the effect of the ensemble component of the prior covariance on the conditioning of the system. Numerical experiments show that the bounds obtained can be useful in predicting the behavior of the true condition number and the convergence speed of an iterative algorithm

This paper is focused on variational data assimilation as an approach to mathematical modeling. Realization of the approach requires a sequence of connected inverse problems with different sets of observational data to be solved.Two variational data assimilation schemes, “implicit” and “explicit”, are considered in the article. Their equivalence is shown and the numerical results are given on a basis of non-linear Robertson system. To avoid the “inverse problem crime” different schemes were used to produce synthetic measurement and to solve the data assimilation problem.

<p>While fog can severely affect human activities (air, land and marine transportation), its forecast with current numerical weather prediction (NWP) models stays challenging, especially due to the lack of observations in the atmospheric boundary layer and the misrepresentation of non linear small scale processes. To improve knowledge on fog formation, evolution and dissipation, several instruments have been deployed during the SOFOG3D (SOuth west FOGs 3D experiment for processes study) experimental campaign to provide an unprecedent database of detailed 3D observations. In that context, a network of 8 ground-based microwave radiometers (MWR) provided continuous temperature profiling as well as integrated water vapor and liquid water path measurements during a 6 month period. Additionally, Martinet <em>et al</em> (2020) highlighted large temperature errors in the AROME-France (Application of Research to Operations at Mesoscale) NWP model background profiles during fog forecasts, leading to temperature differences up to 6 K when compared to tower measurements. Nevertheless, this study also demonstrate that the assimilation of MWR observations with a one dimensional variational data assimilation scheme could leads to improved initial conditions. To go further in that direction, MWR temperature profile observations from the SOFOG3D experiment have been added in the AROME-France operational data assimilation system, which uses a three dimensional variational algorithm (3D-Var) and climatological and homogeneous background error covariances (<strong>B</strong> matrix), to quantify the benefit on operational analyses and forecasts of several fog events. Then, the recently developped ensemble variational (EnVar) data assimilation system has been used to conduct new assimilation experiments. The main advantage of such method is to prescribe a fully flow dependent <strong>B</strong> matrix which is spatially and temporally coherent with the forecasted meteorological conditions. In consequences, it leads to more realistic increments. The results obtained with the different assimilation experiments will be presented. Firstly, a statistical analysis of the impact on the AROME-France analyses and short-range forecasts against conventional observations will be discussed. Secondly, specific SOFOG3D observations will be used to investigate the benefit on dedicated fog case studies.</p>