Ensemble solar forecasting using data-driven models with probabilistic post-processing through GAMLSS

[Departement_IRSTEA]Eaux [TR1_IRSTEA]ARCEAU

Improving ensemble forecasts is a complex process which involves proper scores such as the continuous ranked probability score (CRPS). A homogeneous Gaussian (hoG) model is introduced in order to better understand the characteristics of the CRPS. An analytical formula is derived for the expected CRPS of an ensemble in the hoG model. The score is a function of the variance of the error of the ensemble mean, the mean error of the ensemble mean and the ensemble variance. The hoG model also provides a score decomposition into reliability and resolution components. We examine whether the hoG model provides a useful approximation of the CRPS when applied to operational ECMWF medium‐range ensemble forecasts. The hoG approximation describes the spatial variations of the CRPS well while moderately overestimating the mean score. Seasonal averages over large domains are within 10% of the actual CRPS. Furthermore, the ability to approximate score changes is evaluated by (a) comparing raw ensemble forecasts with postprocessed ensemble forecasts, and (b) by examining score changes associated with a recent upgrade of the IFS. Overall, the hoG approximation predicts the actual CRPS changes well. One of the main anticipated applications of the hoG approximation are new diagnostics in verification software used by NWP developers routinely. The purpose of the diagnostics is to help developers explain impacts of forecast system changes on the CRPS in terms of the changes in mean error, changes in error variance and changes in ensemble variance. The diagnostics require little additional computational resources compared to the alternative of verifying postprocessed versions of the ensemble forecasts. Therefore, it will be feasible to apply the diagnostics easily to all variables that are examined as part of the model development process.

This study used a quantile regression model and a non-homogeneous regression model to calibrate probabilistic forecasts of wind speed. These techniques were applied to the forecasts of wind speed over Pyeongchang area using 51-member European Centre for Medium-Range Weather Forecast (ECMWF). Reliability analysis was carried out by using rank histogram to identify the statistical consistency of ensemble forecasts and corresponding observations. The performances were evaluated by rank histogram, mean absolute error, root mean square error and continuous ranked probability score. The results showed that the forecasts of quantile regression and non-homogeneous regression models performed better than the raw ensemble forecasts. However, the differences of prediction skills between quantile regression and nonhomogeneous regression models were insignificant.

The National Center for Environmental Predictions (NCEP) has produced an ensemble meteorological reforecast product by using a fixed version of Global Forecast System (GFS) ensemble prediction system since 1 January 1979. The 15‐member ensemble product, with a global coverage at a 2.5° × 2.5° spatial resolution and a 14‐day lead time, has been used successfully by the River Forecast Centers of the National Weather Service (NWS) to produce basin scale precipitation and temperature ensemble forecasts in the US for several years now. This study evaluates the predictive skill of post‐processed ensemble forecasts based on GFS precipitation reforecast in China's Huai river basin. The evaluation is carried out in 15 sub‐areas of the Huai river basin and covers the 1/1/1981–31/12/2003 period. The Ensemble Pre‐Processing system version 3 (EPP3), developed at NWS, is used to develop joint probability distributions between forecasted ensemble mean precipitation and corresponding observations and to generate individual ensemble members that preserve space–time correlation of the observed precipitation data. Several statistical verification measures are used to quantify the goodness of fit between post‐processed (i.e. EPP3 processed) ensemble mean and observation and to assess the ensemble spread. Results indicate that the post‐processed forecasts have meaningful predictive skill for the first few days for ensemble daily precipitation forecasts. Predictive skill of ensemble forecasts of cumulative precipitation for lead times up to 14 days are significant. The forecast skill is highly dependent on seasonality, with relatively lower skills seen for wet summer season, when convective storm patterns dominate, as compared with other seasons. The predictive skill of the post‐processed ensemble precipitation is much better than the raw forecasts and the climatological ensemble forecasts. The results from this study suggest that the NCEP's GFS reforecasts can be a valuable resource for places other than the US. Copyright © 2012 John Wiley & Sons, Ltd.

Bias correction of ensemble precipitation forecasts in the improvement of summer streamflow prediction skill

An analogue of the linear continuous ranked probability score is introduced that applies to probabilistic forecasts of circular quantities.This scoring rule is proper and thereby discourages hedging.The circular continuous ranked probability score reduces to angular distance when the forecast is deterministic, just as the linear continuous ranked probability score generalizes the absolute error.Furthermore, the continuous ranked probability score provides a direct way of comparing deterministic forecasts, discrete forecast ensembles, and post-processed forecast ensembles that can take the form of probability density functions.The circular continuous ranked probability score is used in this study to assess predictions of 10 m wind direction for 361 cases of mesoscale, short-range ensemble forecasts over the North American Pacific Northwest.Reference probability forecasts based on the ensemble mean and its forecast error history over the period outperform probability forecasts constructed directly from the ensemble sample statistics.These results suggest that short-term forecast uncertainty is not yet well predicted at mesoscale resolutions near the surface, despite the inclusion of multi-scheme physics diversity and surface boundary parameter perturbations in the mesoscale ensemble design.

Although inflow forecasts are useful information that can be used for improving reservoir operation efficiency, uncertainty is still a challenge for getting sound operation results. Ensemble precipitation forecasts can take uncertainties under consideration, so they have been a research topic for improving reservoir operations. In this paper, a rainfall–runoff model, combined of a multiple linear regression model (for non-flood seasons) and the Xinanjiang model (for flood seasons), for inflow forecasts and a stochastic dynamic programming model for reservoir operations are developed in order to effectively use the ensemble precipitation forecast. To explore the best way for using the ensemble precipitation forecast, two post-processing techniques, i.e., ensemble forecast averaging (EFA) and interval value (IV), are tested. The ensemble precipitation forecast from the European Centre for Medium-Range Weather Forecasts (ECMWF) was chosen due to its high accuracy, and the Huanren reservoir, located in China, was used to test the newly developed models. The results show that, compared with the traditional rule curve, hydropower generation increases by 4.86% and 4.55%, respectively, when EFA and IV are used, which indicates that the use of ensemble forecasts facilitates considerable improvements in operating performance.

Uncertainty propagation within a water level ensemble prediction system

<p>This talk focusses on progress in ensemble forecasting methodology (Part I) and ensemble verification methodology (Part II).</p><p>Operational ECMWF ensemble forecasts are global predictions from days to months ahead. At all forecast ranges, model uncertainties are represented stochastically with the Stochastically Perturbed Parametrization Tendency scheme (SPPT). Recently, considerable progress has been made in developing the Stochastically Perturbed Parametrization scheme (SPP). The SPP scheme offers improved physical consistency by naturally preserving the local conservation properties for energy and moisture of the unperturbed version of the corresponding parametrization. In contrast, the SPPT scheme lacks such local conservation properties, mainly because the scheme does not perturb fluxes at the surface and at the top of the atmosphere consistently with the tendency perturbations in the column.</p><p>NWP research and development relies on scoring rules to judge whether or not a change to the forecast systems results in better ensemble forecasts. A new tool will be presented that can improve the understanding of score differences between sets of forecasts for a widely used proper score, the Continuous Ranked Probability Score (CRPS). An analytical expression has been derived for the CRPS when a homogeneous Gaussian (hoG) forecast-observation distribution is considered. This leads to an approximation of the CRPS when actual verification data are considered, which deviate from a homogeneous Gaussian distribution. The hoG approximation of the CRPS permits a useful decomposition of score differences. The methodology will be illustrated with verification data for medium-range weather forecasts.</p>

By the end of 2022, the renewable energy share of the global electricity capacity reached 40.3% and the new installations were dominated by solar energy, showing a global increase of 21.7%. Due to the high volatility of photovoltaic energy sources, their successful integration into the electrical grid requires accurate short-term power forecasts. These forecasts are obtained from the predictions of solar irradiance, where the most advanced method is the probabilistic approach based on ensemble forecasts.  However, ensemble forecasts are often underdispersive and subject to systematic bias. Hence, they require some form of statistical post-processing, where parametric models provide full predictive distributions of the weather variables at hand.We propose a general two-step machine learning-based approach to calibrating ensemble weather forecasts, where, in the first step, improved point forecasts are generated, which then together with various ensemble statistics serve as input features of the neural network estimating the parameters of the predictive distribution [1]. In a case study based on global horizontal irradiance forecasts of the operational ensemble prediction system of the Hungarian Meteorological Service, the predictive performance of this novel method is compared with the forecast skill of the raw ensemble and the state-of-the-art ensemble model output statistics approaches [2]. We show that at least up to 48h, statistical post-processing substantially improves the predictive performance of the raw ensemble for all forecast horizons considered; the maximal gain e.g. in terms of the mean continuous ranked probability score is above 20%. Furthermore, the proposed two-step machine learning-based approach outperforms in skill its competitors.

Providing reservoirs with accurate forecasts is crucial for effective real-time flood control. This research focuses on the key role of forecasts in real-time flood management for reservoirs. A new approach was developed in this study, integrating a forecast-driven methodology to handle uncertainty in reservoir flood control operations. This involves a novel hybrid of two post-processing techniques: the Cloud model and error-based copula functions, together termed as the stochastic errors-based Cloud (SE-Cloud). Additionally, a multi-objective robust optimization model (MRO) was proposed, encompassing risk, resilience, and vulnerability, to address flood control challenges using ensemble forecasts. For comparative purposes, a two-objective stochastic optimization model (TSO) was also created, aiming to reduce both the highest expected reservoir level and peak discharge. The proposed methodology was applied to the Lishimen reservoir in the Shifeng River subbasin, China, aiming to comprehensively verify the relationships among deterministic forecasts, ensemble forecasts, and flood control performance. The main findings of this study are: (1) The SE-Cloud model was proved to be more efficient in predicting peak flow events and in representing uncertainties in forecasts, with an improvement in hypervolume values ranging from 13.14% to 39.65% over the Cloud model. (2) The MRO strategy resulted in a higher inflow release compared to the TSO, leading to a 0.05m reduction in the anticipated highest water level and a 4.29% increase in peak discharge. (3) With the resilience value downstream remaining constant, it was suggested that increasing upstream vulnerability by using the MRO strategy would not lead to a decrease in resilience. The findings highlight the potential of AI-based ensemble forecasts in augmenting flood control robustness.

Bayesian model averaging (BMA) probability quantitative precipitation forecast (PQPF) models were established by calibrating their parameters using 1–7-day ensemble forecasts of 24-h accumulated precipitation, and observations from 43 meteorological stations in the Huaihe Basin. Forecasts were provided by four single-center (model) ensemble prediction systems (EPSs) and their multicenter (model) grand ensemble systems, which consider exchangeable members (EGE) in The Observing System Research and Predictability Experiment (THORPEX) Interactive Grand Global Ensemble (TIGGE). The four single-center EPSs were from the China Meteorological Administration (CMA), the European Centre for Medium-Range Weather Forecasts (ECMWF), the National Centers for Environment Prediction (NCEP), and the Met Office (UKMO). Comparisons between the raw ensemble, logistic regression, and BMA for PQPFs suggested that the BMA predictive models performed better than the raw ensemble forecasts and logistic regression. The verification and comparison of five BMA EPSs for PQPFs in the study area showed that the UKMO and ECMWF were a little superior to the NCEP and CMA in general for lead times of 1–7 days for the single-center EPSs. The BMA model for EGE outperformed those for single-center EPSs for all 1–7-day ensemble forecasts, and mostly improved the quality of PQPF. Based on the percentile forecasts from the BMA predictive PDFs for EGE, a heavy-precipitation warning scheme is proposed for the test area.

In this study, we investigated the prospect of calibrating probabilistic forecasts of surface air temperature (SAT) over South Korea by using Bayesian model averaging (BMA). We used 63 months of simulation results from four regional climate models (RCMs) with two boundary conditions (NCEP-DOE and ERA-interim) over the CORDEX East Asia. Rank histograms and residual quantile-quantile (R-Q-Q) plots showed that the simulation skills of the RCMs differ according to season and geographic location, but the RCMs show a systematic cold bias irrespective of season and geographic location. As a result, the BMA weights are clearly dependent on geographic location, season, and correlations among the models. The one-month equal weighted ensemble (EWE) outputs for the 59 stations over South Korea were calibrated using the BMA method for 48 monthly time periods based on BMA weights obtained from the previous 15 months of training data. The predictive density function was calibrated using BMA and the individual forecasts were weighted according to their performance. The raw ensemble forecasts were assessed using the flatness of the rank histogram and the R-Q-Q plot. The results showed that BMA improves the calibration of the EWE and the other weighted ensemble forecasts irrespective of season, simulation skill of the RCM, and geographic location. In addition, deterministic-style BMA forecasts usually perform better than the deterministic forecast of the single best member.

Accurate and reliable seasonal climate forecasts are frequently sought by climate‐sensitive sectors to support decision‐making under climate variability and change. Temperature trend is discernible globally over the past decades, but seasonal forecasts produced by a global climate model (GCM) generally underestimate such trend. Current statistical methods used for calibrating seasonal climate forecasts mostly do not explicitly account for climate trends. Consequently, the calibrated forecasts also fail to capture the observed trend. Solving this problem can enhance user confidence in seasonal climate forecasts. In this study, we extend the capability of the Bayesian joint probability (BJP) modelling approach for statistical calibration of seasonal climate forecasts. A trend component is introduced into the BJP algorithm for embedding the observed trend into calibrated ensemble forecasts. We apply the new model (named BJP‐t) to three test stations in Australia. Seasonal forecasts of daily maximum temperatures from the SEAS5 model, operated by the European Centre for Medium‐Range Weather Forecasts (ECMWF), are calibrated and evaluated. The BJP‐t calibrated ensemble forecasts can reproduce the observed trend, when the raw ensemble forecasts and the BJP calibrated ensemble forecasts both fail to do so. The BJP‐t calibration leads to more skilful, more reliable and sharper forecasts than the BJP calibration.

A regression model was developed for use with ensemble forecasts. Ensemble members are assumed to represent a set of equally likely solutions, one of which will best fit the observation. If standard linear regression assumptions apply to the best member, then a regression relationship can be derived between the full ensemble and the observation without explicitly identifying the best member for each case. The ensemble regression equation is equivalent to linear regression between the ensemble mean and the observation, but is applied to each member of the ensemble. The “best member” error variance is defined in terms of the correlation between the ensemble mean and the observations, their respective variances, and the ensemble spread. A probability density function representing the ensemble prediction is obtained from the normalized sum of the best-member error distribution applied to the regression forecast from each ensemble member. Ensemble regression was applied to National Centers for Environmental Prediction (NCEP) Climate Forecast System (CFS) forecasts of seasonal mean Niño-3.4 SSTs on historical forecasts for the years 1981–2005. The skill of the ensemble regression was about the same as that of the linear regression on the ensemble mean when measured by the continuous ranked probability score (CRPS), and both methods produced reliable probabilities. The CFS spread appears slightly too high for its skill, and the CRPS of the CFS predictions can be slightly improved by reducing its ensemble spread to about 0.8 of its original value prior to regression calibration.