Emulating radiative transfer in a numerical weather prediction model

A new parametrization for the surface energy balance of urban areas is presented. It is shown that this new method can represent some of the important urban phenomena, such as an urban heat island and the occurrence of a near-neutral nocturnal boundary layer with associated positive turbulent heat fluxes, unlike the traditional method for representing urban areas within operational numerical weather prediction (NWP) models. The basis of the new parametrization is simple and can be applied easily within an operational NWP model. Also, it has no additional computational expense compared to the traditional scheme and is hence applicable for operational forecasting requirements. The results show that the errors for London within the Met Office operational mesoscale model have been significantly reduced since the new scheme was introduced. The bias and root-mean-square (rms) errors have been approximately halved, with the rms error now similar to the model as a whole. The results also show that a seasonal cycle still exists in the model errors, but it is suggested that this may be caused by anthropogenic heat sources that are neglected in the urban scheme.

Downscaling in Weather Forecasting The first downscaling methods had been invented already in the late 1940s (Klein 1948) and became operational in the early days of numerical weather prediction at the end of the 1950s. Back then, operational numerical weather prediction models were by far too coarse to predict local weather, and furthermore they did not forecast all variables of interest but only a few such as pressure and temperature. At that time a considerable network of observed weather time series was available already. Klein et al. (1959) employed this data to infer statistical relationships between the observed large-scale circulation – for those variables that were simulated by the models – and the observed local-scale weather variables of interest. The statistical model was then applied to downscale the actual numerical forecast of the large-scale circulation to a forecast of the local weather. The key assumption of this approach is that the large-scale predictor has been perfectly forecasted by the numerical model, hence the approach itself has been coined perfect prognosis (PP). After some years, a considerable database of past forecasts had been archived. Analyses of this data revealed that numerical forecasts even of the largescale weather were of course not perfect but showed systematic deviations compared to observations. Yet this database also became key to mitigate this problem: Glahn and Lowry (1972) developed a new approach that – during calibration – did not take the predictors from observations but from the archived numerical forecasts. For a new weather prediction, the inferred statistical link is then applied to the new numerical forecast. As this approach is basically a post-processing of numerical model data, it has been coined model output statistics (MOS). The key advantage of MOS is that it contains by construction a bias correction of the numerical model. Current weather prediction systems employ complex MOS approaches with several predictors that are continually recalibrated to provide the highest predictive skill. In parallel, numerical approaches were developed to improve the resolution and accuracy of forecasts over a target region. The first limited-area model was developed at the US National Meteorological Center (Howcroft 1966, Gerrity and McPherson 1969) and became operational in 1971. This model covered the US, Canada and the Arctic Ocean at a horizontal resolution of 190.5km at 60◦N and was driven at the lateral boundaries with input from a Northern Hemisphere numerical weather prediction model.

The ocean and the atmosphere are coupled by the fluxes of momentum, heat, and water, but in situ measurements of these fluxes are sparse and uneven. Global fields of these fluxes can be derived from satellite data or obtained from operational numerical weather prediction (NWP) models. The NWP models provide quality control and dynamic interpolation of sparse and uneven data. However, dynamic interpolation is only as good as the model physics and parameterization scheme, and over most of the ocean, there may not be sufficient data for any useful interpolation. Operational NWP models change frequently; the results are inconsistent of climatic studies. Reanalysis to produce long period of consistent data have only been started recently. Adequate observations at temporal and spatial scales significant to global change research can only be achieved from the vantage point of space. Spaceborne sensors provide repeated global observations of some parameters from which these fluxes can be derived. The status and shortfalls of spacebased estimation of the fluxes are summarized and assessed in this report. Satellite data can be assimilated into numerical model to simulate surface fluxes, but the progress in developing assimilation techniques for unconventional satellite data has been slow.

Artificial intelligence and numerical weather prediction models: A technical survey

Vibrational Raman lidar measurements of the water vapour mixing ratio (WVMR) were conducted during the WaLiNeAs (Water Vapor Lidar Network Assimilation) field campaigns in the western Mediterranean during autumn and winter 2022–2023 and in southwestern France (Toulouse) between June and September 2023. These campaigns, which spanned different seasons and geographical locations, provided an opportunity to sample various meteorological phenomena, including a dry winter, rainstorms, long-range aerosol transport, and an intense heat wave. Consequently, the water vapour content recorded in the lower troposphere showed significant variability during WaLiNeAs, ranging from less than 1 g kg-1 to more than 17 g kg-1. For operational purposes, a vertical resolution of 100 m and a temporal resolution between 15 and 60 min have been chosen. These resolutions are aligned with the spatio-temporal resolution of the ERA5 dataset from ECMWF's Integrated Forecasting System (IFS) global numerical weather prediction models. The processing of the lidar data has resulted in a scientific publication explaining the methods used to invert the lidar data and recover various atmospheric parameters. Lidar measurements address a critical gap left by operational instruments, which struggle to capture the diurnal cycle of water vapour from the planetary boundary layer to the lower free troposphere. The primary aim of this study is to compare ERA5 data with lidar-derived WVMR profiles. The results reveal altitude-dependent differences in Pearson correlation coefficient (COR), mean bias (MB), and root mean square deviation (RMSD), particularly during periods of high-water vapour content (> 10 g kg⁻¹). Over all periods the MB ranges from 0.1 to 3 g kg⁻¹, and the RMSD varies between 0.6 and 3.7 g kg⁻¹. COR ranges from 0.16 to 0.94, with lower values observed in the free troposphere during warmer periods. These variations underline the differences in the performance of the reanalysis model over different periods and altitudes when compared to lidar profiles. We show that the reanalysis constantly underestimated the WVMR at all altitudes. This study highlights the importance of scrutinising WVMR and the challenges faced by models during high water vapour meteorological events. The results provide valuable insights into the performance of operational numerical weather prediction models and highlight the need to refine their representation of WVMR vertical profiles in the lower troposphere by incorporating ground-based lidar measurements.We give special thanks to the ANR grant #ANR-20-CE04-0001 for its contribution to the WaLiNeAs programme, to Meteo-France for its help with the measurements in Toulouse, and to the CNRS INSU national LEFE programme for its financial contribution to this project.

Forecasting the weather is an increasingly data-intensive exercise. Numerical weather prediction (NWP) models are becoming more complex, with higher resolutions, and there are increasing numbers of different models in operation. While the forecasting skill of NWP models continues to improve, the number and complexity of these models poses a new challenge for the operational meteorologist: how should the information from all available models, each with their own unique biases and limitations, be combined in order to provide stakeholders with well-calibrated probabilistic forecasts to use in decision making? In this paper, we use a road surface temperature example to demonstrate a three-stage framework that uses machine learning to bridge the gap between sets of separate forecasts from NWP models and the ‘ideal’ forecast for decision support: probabilities of future weather outcomes. First, we use quantile regression forests to learn the error profile of each numerical model, and use these to apply empirically derived probability distributions to forecasts. Second, we combine these probabilistic forecasts using quantile averaging. Third, we interpolate between the aggregate quantiles in order to generate a full predictive distribution, which we demonstrate has properties suitable for decision support. Our results suggest that this approach provides an effective and operationally viable framework for the cohesive post-processing of weather forecasts across multiple models and lead times to produce a well-calibrated probabilistic output.This article is part of the theme issue ‘Machine learning for weather and climate modelling’.

In this study we examine the determination accuracy of both the mean radiant temperature (Tmrt) and the Universal Thermal Climate Index (UTCI) within the scope of numerical weather prediction (NWP), and global (GCM) and regional (RCM) climate model simulations. First, Tmrt is determined and the so-called UTCI-Fiala model is then used for the calculation of UTCI. Taking into account the uncertainties of NWP model (among others the HIgh Resolution Limited Area Model HIRLAM) output (temperature, downwelling short-wave and long-wave radiation) stated in the literature, we simulate and discuss the uncertainties of Tmrt and UTCI at three stations in different climatic regions of Europe. The results show that highest negative (positive) differences to reference cases (under assumed clear-sky conditions) of up to -21°C (9°C) for Tmrt and up to -6°C (3.5°C) for UTCI occur in summer (winter) due to cloudiness. In a second step, the uncertainties of RCM simulations are analyzed: three RCMs, namely ALADIN (Aire Limitée Adaptation dynamique Développement InterNational), RegCM (REGional Climate Model) and REMO (REgional MOdel) are nested into GCMs and used for the prediction of temperature and radiation fluxes in order to estimate Tmrt and UTCI. The inter-comparison of RCM output for the three selected locations shows that biases between 0.0 and ±17.7°C (between 0.0 and ±13.3°C) for Tmrt (UTCI), and RMSE between ±0.5 and ±17.8°C (between ±0.8 and ±13.4°C) for Tmrt (UTCI) may be expected. In general the study shows that uncertainties of UTCI, due to uncertainties arising from calculations of radiation fluxes (based on NWP models) required for the prediction of Tmrt, are well below ±2°C for clear-sky cases. However, significant higher uncertainties in UTCI of up to ±6°C are found, especially when prediction of cloudiness is wrong.

This summary, covering the three-month period from January to March 2007, continues the series reporting on the performances of numerical weather prediction (NWP) models used operationally in the Australian Bureau of Meteorology.

Accurate wind speed forecasting is essential for daily activities and socio‐economic production. This study employs an explainable machine learning (ML) model to correct wind speed forecasts from a numerical weather prediction (NWP) model over weather stations across lead times of 1–24 hr in Zhejiang Province, China. Eleven meteorological variables derived from the NWP model are used as predictors. The results demonstrate that ML model substantially improves the original NWP forecasts, reducing the root mean square error of approximately 20%–40% at most stations and about 10% at certain coastal stations. The corrected forecasts also consistently achieve higher Kling‐Gupta Efficiency scores and Forecast Accuracy across all lead times. To enhance model interpretability, we apply Split Count and SHapley Additive exPlanation analyses. The results indicate that the friction velocity (UST) is the most influential predictor and reveal a threshold beyond which the ML model's correction performance improves markedly. The ML model performance slightly declines in coastal regions, likely due to the inconsistent representation of UST between the land and water surface in the NWP model. Monthly analysis further shows superior model performance in November and December, when UST values are higher. This study underscores the value of explainable ML models in improving NWP wind forecasts and provides insights for correcting meteorological variables in NWP models.

Abstract. The increasing resolution of numerical weather prediction models makes inter-column three-dimensional (3D) radiative transport more and more important. However, 3D radiative-transfer solvers are still computationally expensive, largely preventing their use in operational weather forecasting. To address this issue, Jakub and Mayer (2015) developed the TenStream solver. It extends the well-established two-stream method to three dimensions by using 10 instead of 2 streams to describe the transport of radiative energy through Earth's atmosphere. Building upon this method, this paper presents the dynamic TenStream solver, which provides a further acceleration of the original TenStream model. Compared to traditional solvers, this speedup is achieved by utilizing two main concepts. First, radiation is not calculated from scratch every time the model is called. Instead, a time-stepping scheme is introduced to update the radiation field, based on the result from the previous radiation time step. Secondly, the model is based on incomplete solves, with just the first few steps of an iterative scheme towards convergence performed every time it is called. Essentially, the model thereby just uses the ingoing fluxes of a grid box to update its outgoing fluxes. Combined, these two approaches move radiative transfer much closer to the way advection is handled in the dynamical core of a numerical weather prediction (NWP) model, as both use previously calculated results to update their variables and thereby just require access to the neighboring values of an individual grid box, facilitating model parallelization. To demonstrate the feasibility of this new solver, we apply it to a precomputed shallow-cumulus-cloud time series and test its performance in terms of both speed and accuracy. In terms of speed, the dynamic TenStream solver is shown to be about 3 times slower than a traditional 1D δ-Eddington approximation but noticeably faster than currently available 3D solvers. To evaluate the accuracy of the dynamic TenStream solver, we compare its results as well as calculations carried out using a 1D δ-Eddington approximation and the original TenStream solver, to benchmark calculations performed with the 3D Monte Carlo solver MYSTIC. We demonstrate that at the grid box level, dynamic TenStream is able to calculate heating rates and net irradiances at domain boundaries that are very close to those obtained by the original TenStream solver, thus offering a much better representation of the MYSTIC benchmark than the 1D δ-Eddington results. By calling the dynamic TenStream solver less frequently than the δ-Eddington approximation, we furthermore show that our new solver produces significantly better results than a 1D δ-Eddington approximation carried out with a similar computational demand. At these lower calling frequencies, however, the incomplete solves in the dynamic TenStream solver also lead to a buildup of bias with time, which becomes larger the lower the calling frequency is.

Abstract. Broadband short-wave (SW) surface direct and diffuse irradiances are not typically within the set of output variables produced by numerical weather prediction (NWP) models. However, they are frequently requested for solar energy applications. In order to compute them, a detailed representation of the aerosol optical properties is important. Nonetheless, NWP models typically oversimplify aerosol representation or even neglect their effect. In this work, a flexible method to account for the SW aerosol optical properties in the computation of broadband SW surface direct and diffuse irradiances is presented. It only requires aerosol optical depth at 0.55 μm and knowledge of the type of predominant aerosol. Other parameters needed to consider spectral aerosol extinction, namely, Angström exponent, aerosol single-scattering albedo and aerosol asymmetry factor, are parameterized. The parameterization has been tested using the Rapid Radiative Transfer Model for climate and weather models (RRTMG) SW scheme of the Weather Research and Forecasting (WRF) NWP model for data over the continental US. In principle, it can be adapted to any other SW radiative transfer band model. It has been verified against a control experiment and using data from five radiometric stations in the contiguous US. The control experiment consisted of a clear-sky evaluation of the RRTMG solar radiation estimates obtained in WRF when RRTMG is driven with ground-observed aerosol optical properties. Overall, the verification has shown satisfactory results for both broadband SW surface direct and diffuse irradiances. The parameterization has proven effective in significantly reducing the prediction error and constraining the seasonal bias in clear-sky conditions to within the typical observational error expected in well maintained radiometers.

A semi-Lagrangian numerical weather prediction (NWP) model developed for both real-time prediction and for research simulations has been evaluated. The model is second-order in time, high-order (≥ 3) in space, and employs a non-staggered grid in both the horizontal and vertical. A version of the model which is third-order in space was compared with two Eulerian models: the Australian Bureau of Meteorology's current operational regional model which is quasi-second-order in rime and space, and also a new version of this model with a third-order upwind scheme that was developed for use as the Australian Bureau of Meteorology's next operational limited-area NWP model. In a three month trial of twice-daily 48-hour forecasts it was found that both the third-order models were significantly more skillful than the current operational model, as measured by the standard performance statistics such as the S1 skill score (Teweles and Wobus, 1954), and root-mean-square (RMS) errors. The specific implications of this greater accuracy were examined in case studies of severe weather events. The semi-Lagrangian model also bas been adapted to global form and run on a daily basis out to 5 days using archived operational data over a period of almost 6 months. Finally, the semi-Lagrangian model code was parallelized on a workstation cluster and also on a scalable parallel computer, and it was found that the model was well-suited to parallelization on both computer platforms.

A unique analysis is applied to the normal modes of the U.S. Navy's global operational numerical weather prediction model to investigate time-varying responses in the tropics and extratropics to tropical Pacific sea surface temperature anomalies. With this new analysis, the modes are partitioned according to their latitudinal variances. This allows the response energy to be separated into tropical and extratropical contributions. The partitioned responses are derived by grouping those modes whose fractional variance within a prescribed latitudinal band δμ exceeds some threshold value β. Since the parameters δμ and β may be chosen arbitrarily, this technique greatly increases the flexibility of the normal-mode diagnostic approach. The partitioned responses reveal distinct differences between the evolution and vertical scales of the dominant modes in the tropics and extratropics. In the tropics, the structure is dominated by the external mode and a medium-depth internal mode. The internal mode is d...

Solar irradiance was forecasted by the operational numerical weather prediction model in Korea Institute of Energy Research. The quantitative evaluation was made against the in situ measurements at 37 ground observing stations. The relative mean bias errors and root mean square errors for daily ahead forecasts of daily total solar irradiation at 18 UTC from January to May, 2014 are averaged as being 0.34% and 23.6%,respectively.

Machine learning techniques are increasingly used in weather forecasting, to either improve results from numerical weather prediction (NWP) models or emulate some aspects of the forward integration part of NWP models using a purely data-driven approach. So far most of the techniques presented have focused on producing deterministic forecasts, as many standard machine learning techniques lack the ability to express uncertainty.   Whilst methods that attempt to fully emulate NWP models are both very complex and computationally expensive, in this talk I will show how much simpler and computationally cheaper techniques can be used to create a probabilistic forecast from a single global high-resolution forecast by post-processing. Specifically, I will focus on the relatively novel technique of a Bayesian Neural Network and show how it can predict the distribution of the forecast error relative to its own analysis. By adding these error distributions to the original forecast, we can create a probabilistic forecast. Time permitting, I will discuss how more advanced techniques can be applied to take these distributions and generate spatially consistent ensemble members.  This methodology is particularly useful for NWP models at very high resolutions where running an ensemble is too computationally expensive and for machine learning approaches where no uncertainty information is available. In this talk, I will show how this methodology can be successfully applied to both ECMWF’s high-resolution forecast and a purely data-driven weather forecast model being run at ECMWF, for both the surface variable of 2m temperature and the atmospheric variable of Z500. These probabilistic forecasts have been verified using standard metrics and, in the case of the high-resolution numerical forecast, have as good as or better CRPS scores than the ECMWF ensemble forecast for the lead times tested. Moreover, these probabilistic forecasts are reliable with spread-skill ratios close to one. Hence, this novel machine learning post-processing technique has the potential to produce probabilistic forecasts that are valuable and useful to forecasters.