Hybrid Background Error Covariances for a Limited-Area Deterministic Weather Prediction System

Dual-resolution (DR) hybrid variational-ensemble analysis capability was implemented within the community Weather Research and Forecasting (WRF) Model data assimilation (DA) system, which is designed for limited-area applications. The DR hybrid system combines a high-resolution (HR) background, flow-dependent background error covariances (BECs) derived from a low-resolution ensemble, and observations to produce a deterministic HR analysis. As DR systems do not require HR ensembles, they are computationally cheaper than single-resolution (SR) hybrid configurations, where the background and ensemble have equal resolutions. Single-observation tests were performed to document some characteristics of limited-area DR hybrid analyses. Additionally, the DR hybrid system was evaluated within a continuously cycling framework, where new DR hybrid analyses were produced every 6 h over ~3.5 weeks. In the DR configuration presented here, the deterministic backgrounds and analyses had 15-km horizontal grid spacing, but the 32-member WRF Model–based ensembles providing flow-dependent BECs for the hybrid had 45-km horizontal grid spacing. The DR hybrid analyses initialized 72-h WRF Model forecasts that were compared to forecasts initialized by an SR hybrid system where both the ensemble and background had 15-km horizontal grid spacing. The SR and DR hybrid systems were coupled to an ensemble adjustment Kalman filter that updated ensembles each DA cycle. On average, forecasts initialized from 15-km DR and SR hybrid analyses were not statistically significantly different, although tropical cyclone track forecast errors favored the SR-initialized forecasts. Although additional studies over longer time periods and at finer grid spacing are needed to further understand sensitivity to ensemble perturbation resolution, these results suggest users should carefully consider whether SR hybrid systems are worth the extra cost.

[1] In ensemble Kalman filter, space localization is used to reduce the impact of long-distance sampling errors in the ensemble estimation of the forecast error covariance. When two variables are not physically correlated, their error covariance is still estimated by the ensemble and, therefore, it is dominated by sampling errors. We introduce a “variable localization” method, zeroing out such covariances between unrelated variables to the problem of assimilating carbon dioxide concentrations into a dynamical model using the local ensemble transform Kalman filter (LETKF) in an observing system simulation experiments (OSSE) framework. A system where meteorological and carbon variables are simultaneously assimilated is used to estimate surface carbon fluxes that are not directly observed. A range of covariance structures are explored for the LETKF, with emphasis on configurations allowing nonzero error covariance between carbon variables and the wind field, which affects transport of atmospheric CO2, but not between CO2 and the other meteorological variables. Such variable localization scheme zeroes out the background error covariance among prognostic variables that are not physically related, thus reducing sampling errors. Results from the identical twin experiments show that the performance in the estimation of surface carbon fluxes obtained using variable localization is much better than that using a standard full covariance approach. The relative improvement increases when the surface fluxes change with time and model error becomes significant.

In this study several approaches for obtaining more accurate background-error covariances for atmospheric data assimilation are evaluated. Experiments are conducted by replacing the covariances in the operational three-dimensional variational analysis system at the Canadian Meteorological Centre. In the current system, these covariances are computed using the so-called NMC method that is known to suffer from several deficiencies. The approaches evaluated in this study attempt to more realistically sample the probability distribution of background error by simulating (using a Monte Carlo approach) the error generated at each stage of the forecast-analysis process. The ensemble Kalman filter and a simpler approach applied to an existing forecast-analysis system are both used to generate these error samples. In addition, error samples are generated directly from the covariances of the operational system to allow the effects of sampling error to be quantified. Several strategies for estimating the full covariance matrix from a relatively small number of error samples are then employed. Approaches include the use of a spatially localized ensemble representation of the correlations that allows the usual assumptions of homogeneity and isotropy to be relaxed. In addition, the use of a weighted average between such a covariance matrix and a covariance matrix with homogeneous and isotropic correlations is evaluated. Several diagnostic results from the estimated background-error covariances are presented in addition to verification statistics computed from two-week forecast-analysis experiments. Modest forecast improvements are obtained by using the new background-error covariance estimates, mostly in the southern hemisphere. However, additional results suggest that further improvements may be gained by increasing the number of error samples and a preliminary quantitative estimate of the expected gain is computed. © Crown copyright, 2005. Royal Meteorological Society

The effect on weather forecast performance of incorporating ensemble covariances into the initial covariance model of the four-dimensional variational data assimilation (4D-Var) Naval Research Laboratory Atmospheric Variational Data Assimilation System-Accelerated Representer (NAVDAS-AR) is investigated. This NAVDAS-AR-hybrid scheme linearly combines the static NAVDAS-AR initial background error covariance with a covariance derived from an 80-member flow-dependent ensemble. The ensemble members are generated using the ensemble transform technique with a (three-dimensional variational data assimilation) 3D-Var-based estimate of analysis error variance. The ensemble covariances are localized using an efficient algorithm enabled via a separable formulation of the localization matrix. The authors describe the development and testing of this scheme, which allows for assimilation experiments using differing linear combinations of the static and flow-dependent background error covariances. The tests are performed for two months of summer and two months of winter using operational model resolution and the operational observational dataset, which is dominated by satellite observations. Results show that the hybrid mode data assimilation scheme significantly reduces the forecast error across a wide range of variables and regions. The improvements were particularly pronounced for tropical winds. The verification against radiosondes showed a greater than 0.5% reduction in vector wind RMS differences in areas of statistical significance. The verification against self-analysis showed a greater than 1% reduction from verifying against analyses between 2- and 5-day lead time at all eight vertical levels examined in areas of statistical significance. Using the Navy's summary of verification results, the Navy Operational Global Atmospheric Prediction System (NOGAPS) scorecard, the improvements resulted in a score (+1) that justifies a major system upgrade.

: The final goal of this project is to provide the US Navy with an increased capability of using Doppler radar observations in the detection and prediction of hazardous weather events that usually have a strong randomness in nature and affect the Navy operations, especially over oceans and in remote areas. By developing a high-resolution data assimilation capability that can effectively assimilate Doppler radar observations along with other conventional and remotely-sensed data, the US Navy will have the ability to analyze and forecast the battlespace atmospheric conditions with sufficient detail and accuracy for supporting the Navy mission in threat detection, weapons deployment, and weather safe operations. The objective of the study is to develop an advanced ensemble-based radar data assimilation system for the US Navy and to address some critical scientific and technique issues associated with ensemble radar data assimilation. The radar data system that will be developed will use flow-dependent background error covariance (instead of the static background error covariance) to account for the complexity and rapid change in the dynamical and microphysical structures inside and outside storms. The system will assimilate all the observed variables from different types of sensors, including Doppler radars, satellites, UASs, and conventional meteorological observations, simultaneously to allow full interactions among the assimilated variables during the data assimilation to keep the balances among the dynamics, thermodynamics and microphysics in the model initial fields. The system will be able to use the observations from many types of radars on different platforms (WSR-88D, DoD meteorological radars and tactical radars both on-land and shipboard, etc.) with an appropriate quality control. Multi-scale data assimilation capability will also be one of the major features of the new radar data assimilation system that allows observational data at different scales to be as

The atmospheric planetary boundary layer (PBL) plays a key role in controlling surface heating by solar radiation to influence atmospheric instability, convective initiation, and the location and intensity of precipitation. In this study, the PBL-type-dependent background error covariances (BECs) are first constructed separately for the convective, near-neutral, stable, and all-type boundary layers. The convective boundary layer exhibits the longest vertical length scales and the largest standard deviations of both temperature and relative humidity error among the four BECs. The PBL-type-dependent BECs are then employed in the assimilation of GNSS radio occultation (RO) refractivity observations from FY-3D/FY-3E, MetOp-B/MetOp-C, COSMIC-2, SPIRE, and GeoOptics, together with brightness temperature observations at channels 9 (6.25 μm) and 10 (7.1 μm) from the Advanced Geostationary Radiation Imager (AGRI) onboard FY-4A, aiming at improving quantitative precipitation forecasts (QPFs) of squall-line cases over eastern China. Compared with the data assimilation experiment using the traditional all-type BEC, the PBL-type-dependent data assimilation experiment produced a moister lower atmosphere and a higher PBL, and a slightly shallower 500-hPa trough and a farther northward cold front, resulting in a more northward precipitation forecast consistent with observations. Higher QPF threat scores, higher probabilities of detection, and lower false alarm rates confirm the benefit of using PBL-type-dependent BECs in the assimilation of GNSS RO and FY-4A AGRI data. Additional ensemble forecasts further demonstrate the robustness of these improvements, with consistent increases in threat scores across multiple cases and forecast lead times, indicating reduced model spin-up and more balanced initial conditions with PBL-type-dependent BECs.

This chapter deals with the estimation and specification of realistic background error covariances, which is a key issue in data assimilation, since these covariances are used to filter and propagate observations. The underlying equations of error evolution are summarized, and associated simulation techniques are also presented, based on either ensemble techniques or the so-called NMC method. Another important source of information on background error covariances corresponds to innovation-based estimates, which may be combined with ensemble data assimilation to estimate model error covariances, for instance. Moreover, because the covariance matrix is huge, diagnostics of its main components and salient features have to be employed. The possibility of modelling this matrix using a sequence of sparse operators is then reviewed, in addition to filtering methods that account for the finite ensemble size and associated sampling noise effects.

An ensemble method combined with a four‐dimensional variational data assimilation system is used to provide monthly estimates of the background error covariance matrix for global stratospheric and upper tropospheric ozone for the year 2008. The system is composed of the Mocage chemical transport model and the Valentina assimilation system. The ensemble was obtained from parallel analyses of perturbed data from the Microwave Limb Sounder (MLS) instrument. The monthly estimates of background error covariances have then been introduced in the assimilation suite. To assess the separate contribution of each of its components, a number of analyses were realized, using only some estimated components of the background error covariances and a basic model for the others. The evaluation is realized by comparing the analyses with independent ozone profiles (from ozonesondes) and total ozone columns (from the Ozone Monitoring Instrument). It demonstrates that using the estimated statistics compared to basic models for the background error covariance matrix globally slightly improves the analysis quality; however, using the estimated statistics more largely improves the analysis quality for special situations encountered in April and October. In these situations, the most important parameter for the analysis quality is the use of estimated correlations. Copyright © 2011 Royal Meteorological Society

Diurnally varying background error covariances estimated in RMAPS-ST and their impacts on operational implementations

Challenges for data assimilation – from convective‐scale to climate

Background error covariance (BEC) is crucial in data assimilation. This paper addresses the multivariate BEC associated with black carbon, organic carbon, nitrates, sulfates, and other con- stituents of aerosol species. These aerosol species are modeled and predicted using the Model for Simulating Aerosol Interactions and Chemistry scheme (MOSAIC) in the Weather Research and Forecasting/Chemistry (WRF/Chem) model at a resolution of 4 km in Southern California. The BEC is estimated from the differences between the 36-hour and 12-hour forecasts using the NMC method. The results indicated that the maximum background error standard deviation is associated with nitrate and is larger than that of black carbon, organic carbon, and sulfate. The horizontal and vertical scale of the correlation of nitrate is much smaller than that of other species. A significant cross-correlation is found between the species of black carbon and organic carbon. The cross-correlations between nitrate and other variables are relatively smaller and exhibit a relatively smaller length scale. Single observation data assimilation experiments are performed to illustrate the effect of the BEC on analysis increments.

The ensemble-based tangent linear model (ETLM), which evolves the static background error covariance (BEC) in time in the data assimilation window, is beneficial for improving the analysis and avoids the need for developing the tangent linear forecast model and its adjoint as in four-dimensional variational data assimilation (4D-Var). This study proposes to apply the filtered ETLM (FETLM), which is composed of ensemble perturbations at limited time slots and localized with a quasi-Gaussian filter in the way that avoids calculation of the inverse matrix. Tests with the Lorenz model showed that FETLM evolved the static BEC as in 4D-Var and improved the analysis. In a case study with the FETLM implemented in the hourly updated regional forecast system, the Rapid Refresh Forecast System, the imbalance of the analysis was mitigated due to the flow-dependent BEC. Moreover, the cross-variable covariance in the FETLM made it possible to assimilate radar reflectivity effectively even without hydrometeors as control variables. Significance Statement In atmospheric variational data assimilation, the ensemble-based tangent linear model (ETLM), which evolves the static background error covariance (BEC) in time, is beneficial for improving the analysis. However, it generally requires significant computation with high-frequency ensemble forecasts. This study proposes a filtered ETLM (FETLM) to avoid the huge computation. Tests with an hourly updated regional forecast system showed that FETLM mitigated the imbalance of the analysis and made it possible to assimilate radar reflectivity more effectively due to the flow-dependent BEC.

As a contribution to the Year of Polar Prediction (YOPP), Environment and Climate Change Canada (ECCC) developed the Canadian Arctic Prediction System (CAPS), a high-resolution (3-km horizontal grid-spacing) deterministic Numerical Weather Prediction (NWP) system that ran in real-time from February 2018 to November 2021. During YOPP, ECCC was also running two other operational systems that cover the Arctic: the 10-km Regional Deterministic Prediction System (RDPS) and the 25-km Global Deterministic Prediction System (GDPS). The performance of these three systems over the Arctic was monitored and routinely compared during 2018, both subjectively and with objective verification scores. This work provides a description of CAPS and compares the surface variable objective verification for the Canadian deterministic NWP systems operational during YOPP, focusing on the Arctic winter and summer Special Observing Periods (Feb-March and July-Aug-Sept, 2018). CAPS outperforms RDPS and GDPS in predicting near-surface temperature, dew-point temperature, wind and precipitation, in both seasons and domains. All three systems exhibit a diurnal cycle in the near-surface temperature biases, with maxima at night and minima in day-time. In order to mitigate representativeness issues associated with complex topography, model tile temperatures are adjusted to the station elevation by applying a standard atmosphere lapse-rate: especially for the coarse-resolution models, the lapse-rate adjustment reduces the temperature cold biases characterising mountain terrains. Verification of winter precipitation is performed by adjusting solid precipitation measurement errors from the undercatch in windy conditions: the Canadian models’ systematic positive bias, which was artificially inflated by the undercatch, is reduced by the adjustment, to attain neutral bias. These YOPP dedicated intense verification activities have identified some strengths, weaknesses and systematic behaviours of the Canadian deterministic prediction systems at high latitudes: these results can serve as a benchmark, for comparison and further development. Moreover, this YOPP verification exercise has revealed some issues related to the verification of surface variables and has led to the development of better verification practices for the polar regions (and beyond).

ABSTRACTAssimilation of satellite observations in numerical weather models has helped in improving the prediction of tropical cyclones (TCs) in recent times. Utilizing observations from microwave sensors such as MeghaTropiques SAPHIR (Sounder for Probing Vertical Profiles of Humidity) which samples the tropical atmosphere more frequently can further improve the quality of analysis obtained. Further improved estimates of the atmospheric system can be obtained through the specification of better estimates of background error covariances (BECs or ) in a variational system. An investigation on the impact of multivariate analysis of humidity variable on the simulation of three TCs over the Bay of Bengal region is presented here. Assimilation of SAPHIR radiances does improve the cyclone simulation in the Weather Research and Forecasting (WRF) model. The study indicates that utilizing the multivariate BEC does impact the simulation of TC features. The results of the three cyclones considered here indicate that inclusion of multivariate BEC leads to intensification of the TCs in terms on minimum sea level pressure as well as maximum wind speed. The rainfall simulated by the model for the three cyclones is also positively impacted by the use of multivariate BEC in WRF three-dimensional variational system.

This paper develops a parallel implementation of the ensemble Kalman filter (EnKF) based on shrinkage covariancematrix estimation. The EnKF is a sequential Monte Carlo method for parameter and state estimation of highly nonlinearmodels. In the context of ensemble based methods, background error correlations are estimated via anensemble of model realizations. In practice, the model dimension (number of model components) is several timeslarger than the ensemble size owing to the computational effort involved in one single model propagation. Thisconstraint yields to spurious correlations (correlation between distant model components in space) in the backgrounderror estimations. Commonly, localization methods are utilized in order to reduce the impact of spuriouscorrelations. For instance, correlations of model components are reduced based on some function of their physicaldistances. Some EnKF implementations are formulated in such manner that, localization methods are implicitlyinvolved in the assimilation of observations. In this context, one of the best ensemble Kalman filter implementationsis the local ensemble transform Kalman filter (LETKF). In the LETKF, each model component is surrounded by alocal box of size r and then, the information contained within the local domain (i.e., observed components) is used toperform the assimilation. However, for sparse observational networks, r can become sufficiently large with respectto the ensemble size and therefore, the local analysis corrections can be impacted by spurious correlations. Wepropose an ensemble Kalman filter implementation in which the background error covariance matrix is estimated viathe Rao-Blackwell Ledoit and Wolf (RBLW) estimator. This estimator has proven to be better conditioned than thetrue (background) error covariance matrix. In addition, the explicit computation of the RBLW estimator is notrequired in our formulation. This implies enormous savings in terms of memory considering the typical order ofbackground error covariance matrices O(10e8) Furthermore, the proposed EnKF implementation can be efficientlyperformed in parallel: each model component is surrounded by a local box, a local background error covariancematrix is estimated based on the RBLW estimator and then, the ensemble Kalman filter equations are locally used inorder to compute the local analysis corrections. After the local assimilation, each model component is mapped backonto the full model domain from where the global analysis solution is obtained. In operational data assimilation, EnKF implementations are attractive owing to their potential for parallelization. Experiments are performed making use of the Atmospheric General Circulation Model SPEEDY. The proposed method is coded in FORTRAN and MPI is used for the local assimilation of observations. The predicted variables are the zonal and meridional wind components, the temperature and the specific humidity. For each model variable, 8 layers with the T-63 resolution (192 x 96 grid points) are considered which provide a total number of 589,824 model components. The ensemble size is 94. The number of processors utilized are 96, 256, 512, 768, 1024 and 1536. The results reveal that, the use of the RBLW estimator can mitigate the impact of spurious correlations under sparse observational networks and even more, for different number of processors, the computational effort of the proposed EnKF implementation is comparable to that of the LETKF where no covariance estimation is performed. In terms of accuracy, the proposed implementation performs better than the LETKF formulation in the Root Mean Square Error sense.