A Novel Reservoir Ensemble Forecasting Method Based on Constrained Multi-Model Weight Optimization

[Departement_IRSTEA]Eaux [TR1_IRSTEA]ARCEAU

Accurate runoff forecasting is of great significance for water resource allocation flood control and disaster reduction. However, due to the inherent strong randomness of runoff sequences, this task faces significant challenges. To address this challenge, this study proposes a new SMGformer runoff forecast model. The model integrates Seasonal and Trend decomposition using Loess (STL), Informer’s Encoder layer, Bidirectional Gated Recurrent Unit (BiGRU), and Multi-head self-attention (MHSA). Firstly, in response to the nonlinear and non-stationary characteristics of the runoff sequence, the STL decomposition is used to extract the runoff sequence’s trend, period, and residual terms, and a multi-feature set based on ‘sequence-sequence’ is constructed as the input of the model, providing a foundation for subsequent models to capture the evolution of runoff. The key features of the input set are then captured using the Informer’s Encoder layer. Next, the BiGRU layer is used to learn the temporal information of these features. To further optimize the output of the BiGRU layer, the MHSA mechanism is introduced to emphasize the impact of important information. Finally, accurate runoff forecasting is achieved by transforming the output of the MHSA layer through the Fully connected layer. To verify the effectiveness of the proposed model, monthly runoff data from two hydrological stations in China are selected, and eight models are constructed to compare the performance of the proposed model. The results show that compared with the Informer model, the 1th step MAE of the SMGformer model decreases by 42.2% and 36.6%, respectively; RMSE decreases by 37.9% and 43.6% respectively; NSE increases from 0.936 to 0.975 and from 0.487 to 0.837, respectively. In addition, the KGE of the SMGformer model at the 3th step are 0.960 and 0.805, both of which can maintain above 0.8. Therefore, the model can accurately capture key information in the monthly runoff sequence and extend the effective forecast period of the model.

Applying Bayesian Model Averaging to mechanistic models: An example and comparison of methods

The main approach to probabilistic weather forecasting has been the use of ensemble forecasting. In ensemble forecasting, the probability information is generally derived by using several numerical model runs, with perturbation of the initial conditions, physical schemes or dynamic core of the numerical weather prediction (NWP) models. However, ensemble forecasting usually tends to be under‐dispersive. Statistical post‐processing has, therefore, become an essential component of any ensemble prediction system aiming to improve the quality of numerical weather forecasts as they seek to generate calibrated and sharp predictive distributions of future weather quantities. Different versions of the ensemble model output statistics (EMOS) and the Bayesian model averaging (BMA) post‐processing methods are used in the present paper to calibrate 24, 48 and 72 hr forecasts of 24 hr accumulative precipitation. The ensemble employs the weather and research forecasting (WRF) model with eight different configurations which were run over Iran for six months (September 2015–February 2016). The results reveal that the BMA and EMOS‐censored, shifted gamma (CSG) techniques are substantially successful at improving the raw WRF ensemble forecasts; however, each approach improves different aspects of the forecast quality. The BMA method is more accurate, skilful and reliable than the EMOS‐CSG method, but has poorer discrimination. Moreover, it has better resolution in predicting the probability of high‐precipitation events than the EMOS‐CSG method.

A multiple crop model ensemble for improving broad-scale yield prediction using Bayesian model averaging

In this study, we investigated the prospect of calibrating probabilistic forecasts of surface air temperature (SAT) over South Korea by using Bayesian model averaging (BMA). We used 63 months of simulation results from four regional climate models (RCMs) with two boundary conditions (NCEP-DOE and ERA-interim) over the CORDEX East Asia. Rank histograms and residual quantile-quantile (R-Q-Q) plots showed that the simulation skills of the RCMs differ according to season and geographic location, but the RCMs show a systematic cold bias irrespective of season and geographic location. As a result, the BMA weights are clearly dependent on geographic location, season, and correlations among the models. The one-month equal weighted ensemble (EWE) outputs for the 59 stations over South Korea were calibrated using the BMA method for 48 monthly time periods based on BMA weights obtained from the previous 15 months of training data. The predictive density function was calibrated using BMA and the individual forecasts were weighted according to their performance. The raw ensemble forecasts were assessed using the flatness of the rank histogram and the R-Q-Q plot. The results showed that BMA improves the calibration of the EWE and the other weighted ensemble forecasts irrespective of season, simulation skill of the RCM, and geographic location. In addition, deterministic-style BMA forecasts usually perform better than the deterministic forecast of the single best member.

Bayesian model averaging (BMA) is a statistical way of post-processing forecasts ensembles to create predictive probability density functions (pdfs) for weather quantities. BMA has been proposed as way of correcting underdispersion in ensemble forecasts. The output of ensemble BMA is a weighted average of pdfs centered on the individual bias-corrected ensemble forecast. The BMA weights are posterior probabilities of the models generating the forecast, and show the relative contribution of the component models to the prediction over the training period. This paper focuses on short range forecasting of daily maximum temperature. Forecasting temperature, we have approximated the conditional pdf by normal distribution centered at a linear function of the forecast. Since we are not able to get an ensemble of forecasts for Ethiopia, we have used the previous eight years from 2003 to 2010 data as ensemble and 2011 as observation. The BMA approach yields predictive distribution for temperature thatwere much better calibrated than those based on naive averaging of the forecasts.

Precipitation is an important and difficult climate variable to predict. Skillful sub-seasonal precipitation forecast can provide useful information for agriculture and water resources management communities. Nevertheless, sub-seasonal forecasts have been given less attention compared with forecasts of shorter/longer time horizons. Recently, the S2S database has made sub-seasonal to seasonal forecasts/reforecasts from 11 operational centers available to researchers. In this work, reforecasts of the European Centre for Medium-Range Weather Forecasts (ECMWF) spanning over a 20-year period were evaluated. Raw and post-processed precipitation forecasts were put against observed precipitation series of a number of synoptic stations with different precipitation regimes for all months of the year. By comparison, precipitation forecasts were more skillful in wet months. Raw forecasts in December and February were better than that in other months, while the weakest results were detected in August. There was no significant relationship between precipitation regime and prediction skill. Furthermore, quantile mapping (QM), Bayesian model averaging (BMA), and QM_BMA combination were adopted for post-processing. BMA outperformed QM and QM_BMA through improving the correlation coefficient between observations and average of ensemble forecasts; however, BMA performed weaker in other evaluation criteria, in particular in humid regions. It is concluded that the application of post-processing techniques greatly improved the results of ensemble precipitation forecasts. However, in a number of stations/months, the forecast results were not acceptable even after post-processing.

We investigate the ability of hydrological multimodel ensemble predictions to enhance the skill of streamflow forecasts at short‐ to medium‐range timescales. To generate the multimodel ensembles, we implement a new statistical postprocessor, namely, quantile regression‐Bayesian model averaging (QR‐BMA). Quantile regression‐Bayesian model averaging uses quantile regression to bias correct the ensemble streamflow forecasts from the individual models and Bayesian model averaging to optimally combine their probability density functions. Additionally, we use an information‐theoretic measure, namely, conditional mutual information, to quantify the skill enhancements from the multimodel forecasts. We generate ensemble streamflow forecasts at lead times from 1 to 7 days using three hydrological models: (i) Antecedent Precipitation Index‐Continuous, (ii) Hydrology Laboratory‐Research Distributed Hydrologic Model, and (iii) Weather Research and Forecasting Hydrological modeling system. As forcing to the hydrological models, we use weather ensemble forecasts from the National Centers for Environmental Prediction 11‐member Global Ensemble Forecast System Reforecast version 2. The forecasting experiments are performed for four nested basins of the North Branch Susquehanna River, USA. We find that after bias correcting the streamflow forecasts from each model, their skill performance becomes comparable. We find that the multimodel ensemble forecasts have higher skill than the best single‐model forecasts. Furthermore, the skill enhancements obtained by the multimodel ensemble forecasts are found to be dominated by model diversity, rather than by increased ensemble size alone. This result, obtained using conditional mutual information, indicates that each hydrological model contributes additional information to enhance forecast skill. Overall, our results highlight benefits of hydrological multimodel forecasting for improving streamflow predictions.

Evaluation of crop model prediction and uncertainty using Bayesian parameter estimation and Bayesian model averaging

Probabilistic forecasts of wind vectors are becoming critical as interest grows in wind as a clean and renewable source of energy, in addition to a wide range of other uses, from aviation to recreational boating. Unlike other common forecasting problems, which deal with univariate quantities, statistical approaches to wind vector forecasting must be based on bivariate distributions. The prevailing paradigm in weather forecasting is to issue deterministic forecasts based on numerical weather prediction models. Uncertainty can then be assessed through ensemble forecasts, where multiple estimates of the current state of the atmosphere are used to generate a collection of deterministic predictions. Ensemble forecasts are often uncalibrated, however, and Bayesian model averaging (BMA) is a statistical way of postprocessing these forecast ensembles to create calibrated predictive probability density functions (PDFs). It represents the predictive PDF as a weighted average of PDFs centered on the individual bias-corrected forecasts, where the weights reflect the forecasts’ relative contributions to predictive skill over a training period. In this paper the authors extend the BMA methodology to use bivariate distributions, enabling them to provide probabilistic forecasts of wind vectors. The BMA method is applied to 48-h-ahead forecasts of wind vectors over the North American Pacific Northwest in 2003 using the University of Washington mesoscale ensemble and is shown to provide better-calibrated probabilistic forecasts than the raw ensemble, which are also sharper than probabilistic forecasts derived from climatology.

The development of high resolution mapping models of forest attributes based on employing machine or deep learning techniques has increasingly accelerated in the last couple of years. The consequence of this is the widespread availability of multiple sources of information, which can either lead to a potential confusion, or to a possibility to get an "extended” insight into the state of our forests by interpreting these sources jointly. This contribution aims at addressing the latter, by relying on the Bayesian model averaging (BMA) approach.BMA is a method that can be used in building a consensus from an ensemble of different model predictions. It can be seen as weighted mean of different predictions with weights reflecting the predictive performances of different models, or as a finite mixture model which estimates the probability that each observation from the independent validation dataset has been generated by one of the models belonging to the ensemble. BMA can thus be used to diagnose and understand the difference in the predictions and to possibly interpret them.The predictions in our case are the forest canopy height estimations for the metropolitan France coming from 5 different AI models [1-5], while the independent validation dataset comes from the French National Forest Inventory (NFI) disposing with some 6000 plots per year, distributed across the territory of interest. For every plot we have several measurements/estimations of the forest canopy height out of which the following two are considered in this study: h_m – the maximum total height (from the tree's base level to the terminal bud of the tree's main stem) measured within the plot, and h_dom – the average height of the seven largest dominant trees per hectare.In this contribution we present for every considered plot the dominant model with respect to both references i.e. the model having the highest probability to be the one generating measurements/estimations at NFI plot (h_m and h_dom). We present as well as the respective inter-model and the intra-model variance estimations, allowing us to propose a series of hypotheses concerning the established differences between predictions of individual models in function of their specificities.[1] Schwartz, M., et al.: FORMS: Forest Multiple Source height, wood volume, and biomass maps in France at 10 to 30 m resolution based on Sentinel-1, Sentinel-2, and Global Ecosystem Dynamics Investigation (GEDI) data with a deep learning approach, Earth Syst. Sci. Data, 15, 4927–4945, 2023, https://doi.org/10.5194/essd-15-4927-2023[2] Lang, N., et al.: A high-resolution canopy height model of the Earth, Nat Ecol Evol 7, 1778–1789, 2023. https://doi.org/10.1038/s41559-023-02206-6[3] Morin, D. et al.: Improving Heterogeneous Forest Height Maps by Integrating GEDI-Based Forest Height Information in a Multi-Sensor Mapping Process, Remote Sens., 14, 2079. 2022, https://doi.org/10.3390/rs14092079[4] Potapov, P., et al.: Mapping global forest canopy height through integration of GEDI and Landsat data, Remote Sensing of Environment, 253, 2021, https://doi.org/10.1016/j.rse.2020.112165.[5] Liu, S. et al.: The overlooked contribution of trees outside forests to tree cover and woody biomass across Europe, Sci. Adv. 9, eadh4097, 2023, 10.1126/sciadv.adh4097.

Methods of ensemble postprocessing in which continuous probability density functions are constructed from ensemble forecasts by centering functions around each of the ensemble members have come to be called Bayesian model averaging (BMA) or “dressing” methods. Here idealized ensemble forecasting experiments are used to show that these methods are liable to produce systematically unreliable probability forecasts of climatologically extreme weather. It is argued that the failure of these methods is linked to an assumption that the distribution of truth given the forecast can be sampled by adding stochastic perturbations to state estimates, even when these state estimates have a realistic climate. It is shown that this assumption is incorrect, and it is argued that such dressing techniques better describe the likelihood distribution of historical ensemble-mean forecasts given the truth for certain values of the truth. This paradigm shift leads to an approach that incorporates prior climatological information into BMA ensemble postprocessing through Bayes’s theorem. This new approach is shown to cure BMA’s ill treatment of extreme weather by providing a posterior BMA distribution whose probabilistic forecasts are reliable for both extreme and nonextreme weather forecasts.

Ensembles used for probabilistic weather forecasting often exhibit a spread-error correlation, but they tend to be underdispersive. This paper proposes a statistical method for postprocessing ensembles based on Bayesian model averaging (BMA), which is a standard method for combining predictive distributions from different sources. The BMA predictive probability density function (PDF) of any quantity of interest is a weighted average of PDFs centered on the individual bias-corrected forecasts, where the weights are equal to posterior probabilities of the models generating the forecasts and reflect the models' relative contributions to predictive skill over the training period. The BMA weights can be used to assess the usefulness of ensemble members, and this can be used as a basis for selecting ensemble members; this can be useful given the cost of running large ensembles. The BMA PDF can be represented as an unweighted ensemble of any desired size, by simulating from the BMA predictive distribution. The BMA predictive variance can be decomposed into two components, one corresponding to the between-forecast variability, and the second to the within-forecast variability. Predictive PDFs or intervals based solely on the ensemble spread incorporate the first component but not the second. Thus BMA provides a theoretical explanation of the tendency of ensembles to exhibit a spread-error correlation but yet be underdispersive. The method was applied to 48-h forecasts of surface temperature in the Pacific Northwest in January–June 2000 using the University of Washington fifth-generation Pennsylvania State University–NCAR Mesoscale Model (MM5) ensemble. The predictive PDFs were much better calibrated than the raw ensemble, and the BMA forecasts were sharp in that 90% BMA prediction intervals were 66% shorter on average than those produced by sample climatology. As a by-product, BMA yields a deterministic point forecast, and this had root-mean-square errors 7% lower than the best of the ensemble members and 8% lower than the ensemble mean. Similar results were obtained for forecasts of sea level pressure. Simulation experiments show that BMA performs reasonably well when the underlying ensemble is calibrated, or even overdispersed.

Prediction of various weather quantities is mostly based on deterministic numerical weather forecasting models. Multiple runs of these models with different initial conditions result ensembles of forecasts which are applied for estimating the distribution of future weather quantities. However, the ensembles are usually under-dispersive and uncalibrated, so post-processing is required. In the present work Bayesian Model Averaging (BMA) is applied for calibrating ensembles of wind speed forecasts produced by the operational Limited Area Model Ensemble Prediction System of the Hungarian Meteorological Service (HMS). We describe two possible BMA models for wind speed data of the HMS and show that BMA post-processing significantly improves the calibration and precision of forecasts.