Uncertainty Analysis of Hydrologic Forecasts Based on Copulas

Quantification of the inherent uncertainty in hydrologic forecasting is essential for flood control and water resources management. The existing approaches, such as Bayesian model averaging (BMA), hydrologic uncertainty processor (HUP), copula-BMA (CBMA), aim at developing reliable probabilistic forecasts to characterize the uncertainty induced by model structures. In the probability forecast framework, these approaches either assume the probability density function (PDF) to follow a certain distribution, or are unable to reduce bias effectively for complex hydrological forecasts. To overcome these limitations, a copula Bayesian processor associated with BMA (CBP-BMA) method is proposed with ensemble lumped hydrological models. Comparing with the BMA and CBMA methods, the CBP-BMA method relaxes any assumption on the distribution of conditional PDFs. Several evaluation criteria, such as containing ratio, average bandwidth and average deviation amplitude of probabilistic application, are utilized to evaluate the model performance. The case study results demonstrate that the CBP-BMA method can improve hydrological forecasting precision with higher cover ratios more than 90%, which are increased by 4.4% and 3.2%, 2.2% and 1.7% over those of BMA and CBMA during the calibration and validation periods, respectively. The proposed CBP-BMA method provides an alternative approach for uncertainty estimation of hydrological multi-model forecasts.

This paper briefly discusses the nature, the causes and the role of predictive uncertainty in flood forecasting and proposes a novel approach to its estimation. Following the definition of predictive uncertainty, its importance in the decision process is highlighted in relation to the different sources of errors (model, parameter, observations, boundary conditions) that affect flood forecasting. Moreover, the paper briefly analyses the importance of using a full predictive uncertainty, obtained by marginalising the parameter uncertainty, instead of the predictive uncertainty conditional to a single parameter set. Finally, a new Model Conditional Processor (MCP) for the assessment of predictive uncertainty is then proposed as an alternative to the Hydrologic Uncertainty Processor (HUP) introduced by Krzysztofowicz as well as to the Bayesian Model Averaging (BMA) approach due to Raftery et al. The new MCP approach, which aims at assessing, and possibly reducing, predictive uncertainty, allows combination of the observations with one or several models’ forecasts in a multi‐Normal space, by transforming observations and model forecasts in a multivariate Normal space by means of the Normal Quantile Transform. The results of the new approach are shown for the case of the River Po in Italy, and compared with the results obtainable both with the HUP and the BMA.

Hydrologic uncertainty processor for probabilistic river stage forecasting: precipitation-dependent model

Purpose Low-dose radiation risks are generally extrapolated from groups exposed at high levels of dose. Measurement error substantially alters dose-response shape and hence extrapolated risk. Much attention has been paid to methods of dealing with shared errors, common in many datasets, in particular using Bayesian model averaging (BMA) methods. Materials and methods We test two types of BMA model using simulated data. The first we term the quasi-two-dimensional Monte Carlo with BMA (quasi-2DMC + BMA) method, similar to the BMA method proposed by Hoeting et al. but distinct from the 2DMC + BMA method of Kwon et al. The second we term the marginal-quasi-2DMC + BMA method, which uses a more complicated marginal calculation, and may be closer to the 2DMC + BMA method of Kwon et al. Results Assuming a true linear model of dose response, the coverage probabilities for the linear coefficient are 90–95%, for the quasi-2DMC + BMA method, but lower than this, 52–60%, for the marginal-quasi-2DMC + BMA method. Assuming a true linear-quadratic model the coverage probabilities of both linear/quadratic excess relative risk (ERR) coefficients for quasi-2DMC + BMA are too low, and when shared Berkson error is large (50%) the probabilities do not exceed 5%. By comparison, the coverage probabilities for both linear and quadratic coefficients for the marginal-quasi-2DMC + BMA method are generally too high, about 100%. For the linear model quasi-2DMC + BMA yields good estimates of the ERR coefficient, but for marginal-quasi-2DMC + BMA these are upwardly biased. For the linear-quadratic model both quasi-2DMC + BMA and marginal-quasi-2DMC + BMA methods yield substantially biased estimates. Conclusions The performance of both quasi-2DMC + BMA and marginal-quasi-2DMC + BMA methods is bad, with bias and poor coverage.

Characterization of aquifer heterogeneity is inherently difficult because of the insufficiency of data, the inflexibility of parameterization methods, and non-uniqueness of parameterization methods. Groundwater predictions are greatly affected by multiple interpretations of aquifer properties and the uncertainties of model parameters. This study introduces a Bayesian model averaging (BMA) method along with multiple generalized parameterization (GP) methods to identify hydraulic conductivity and along with multiple simulation models to predict groundwater head and quantify the prediction uncertainty. Two major issues about BMA are discussed. The first problem is with using Occam’s window in usual BMA applications. Occam’s window only accepts models in a very narrow range, tending to single out the best method and discard other good methods. A variance window is proposed to replace Occam’s window to cope with this problem. The second problem is with using the Kashyap information criterion (KIC) in the approximation of posterior model probabilities, which tends to prefer highly uncertain model by considering the Fisher information matrix. The Bayesian information criterion (BIC) is recommended because it is able to avoid controversial results and it is computationally efficient. Numerical examples are designed to test the Bayesian model averaging method on hydraulic conductivity identification and groundwater head prediction. The proposed methodologies are then applied to the hydraulic conductivity identification of the Alamitos Gap area, and the hydraulic conductivity estimation and groundwater head prediction of the “1,500-foot” sand in East Baton Rouge Parish, Louisiana. The results show that the GP method provides great flexibility in parameterization with small conditional variance. The use of the variance window is necessary to avoid a dominant model when many models perform equally well. Compared to KIC, BIC is able to give an unbiased posterior model probability. It is also concluded that the uncertainty increases by including multiple models under the BMA framework, but risks are reduced by avoiding overconfidence in the solution from one model.

A multiple crop model ensemble for improving broad-scale yield prediction using Bayesian model averaging

Assessing the performance of climate models in surface air temperature (SAT) simulation and projection have received increasing attention during the recent decades. This paper assesses the performance of the Coupled Model Intercomparison Project phase 5 (CMIP5) in simulating intra-annual, annual and decadal temperature over Northern Eurasia from 1901 to 2005. We evaluate the skill of different multi-model ensemble techniques and use the best technique to project the future SAT changes under different emission scenarios. The results show that most of the general circulation models (GCMs) overestimate the annual mean SAT in Northern Eurasia and the difference between the observation and the simulations primarily comes from the winter season. Most of the GCMs can approximately capture the decadal SAT trend; however, the accuracy of annual SAT simulation is relatively low. The correlation coefficient R between each GCM simulation and the annual observation is in the range of 0.20 to 0.56. The Taylor diagram shows that the ensemble results generated by the simple model averaging (SMA), reliability ensemble averaging (REA) and Bayesian model averaging (BMA) methods are superior to any single GCM output; and the decadal SAT change generated by SMA, REA and BMA are almost identical during 1901–2005. Heuristically, the uncertainty of BMA simulation is the smallest among the three multi-model ensemble simulations. The future SAT projection generated by the BMA shows that the SAT in Northern Eurasia will increase in the 21st century by around 1.03 °C/100 yr, 3.11 °C/100 yr and 7.14 °C/100 yr under the RCP 2.6, RCP 4.5 and RCP 8.5 scenarios, respectively; and the warming accelerates with the increasing latitude. In addition, the spring season contributes most to the decadal warming occurring under the RCP 2.6 and RCP 4.5 scenarios, while the winter season contributes most to the decadal warming occurring under the RCP 8.5 scenario. Generally, the uncertainty of the SAT projections increases with time in the 21st century.

Application of Bayesian model averaging in modeling long-term wind speed distributions

Runoff is simulated using the Soil and Water Assessment Tool (SWAT) in the Shiyang River Basin and considering the precipitation and temperature obtained from five climate models in the Coupled Model Intercomparison Project Phase 5 (CMIP5), including MRI-CGCM3, CanESM2, CNRM-CM5, GFDL-CM3, and FGOALS-g2. Precipitation and temperature were downscaled as the input for SWAT, and the uncertainty in runoff simulation under climate models was evaluated based on the Bayesian information criterion (BIC) and expectation-maximization (EM) algorithms of Bayesian model average (BMA) method, also runoff variation in the future was predicted. Results showed that, for the downscaled precipitation, the accuracy index R2 are mainly concentrated in 0.42–0.58. For the downscaled temperature, R2 is greater than 0.5. The predicted runoff using BMA is better than that using the single climate model in most sub-basins. MRI-CGCM3 and GFDL-CM3 have larger contribution than other three models using BMA based on either BIC or EM algorithms. Compared to the period of 1990–1999, temperature of each climate model is obviously increasing during 2018–2100. While for precipitation, CNRM-CM5 and FGOALS-g2 showed an increasing trend in all sub-basins. But GFDL-CM3, CanESM2, and MRI-CGCM3 vary across sub-basins. The trend in runoff is consistent with precipitation. To sum up, precipitation, temperature, and runoff under most climate models will increase in the future. The results will provide a reference for the utilization and management of water resources in the future.

Integrated dynamic framework for predicting urban flooding and providing early warning

Possible risks in reservoir flood control and regulation cannot be objectively assessed by deterministic flood forecasts, resulting in the probability of reservoir failure. We demonstrated a risk analysis of reservoir flood routing calculation accounting for inflow forecast uncertainty in a sub-basin of Huaihe River, China. The Xinanjiang model was used to provide deterministic flood forecasts, and was combined with the Hydrologic Uncertainty Processor (HUP) to quantify reservoir inflow uncertainty in the probability density function (PDF) form. Furthermore, the PDFs of reservoir water level (RWL) and the risk rate of RWL exceeding a defined safety control level could be obtained. Results suggested that the median forecast (50th percentiles) of HUP showed better agreement with observed inflows than the Xinanjiang model did in terms of the performance measures of flood process, peak, and volume. In addition, most observations (77.2%) were bracketed by the uncertainty band of 90% confidence interval, with some small exceptions of high flows. Results proved that this framework of risk analysis could provide not only the deterministic forecasts of inflow and RWL, but also the fundamental uncertainty information (e.g., 90% confidence band) for the reservoir flood routing calculation.

To study the impact of training sample heterogeneity on the performance of Bayesian model averaging (BMA), two BMA experiments were performed on probabilistic quantitative precipitation forecasts (PQPFs) in the northern China region in July and August of 2010 generated from an 11-member short-range ensemble forecasting system. One experiment, as in many conventional BMA studies, used an overall training sample that consisted of all available cases in the training period, while the second experiment used stratified sampling BMA by first dividing all available training cases into subsamples according to their ensemble spread, and then performing BMA on each subsample. The results showed that ensemble spread is a good criterion to divide ensemble precipitation cases into subsamples, and that the subsamples have different statistical properties. Pooling the subsamples together forms a heterogeneous overall sample. Conventional BMA is incapable of interpreting heterogeneous samples, and produces unreliable PQPF. It underestimates the forecast probability at high-threshold PQPF and local rainfall maxima in BMA percentile forecasts. BMA with stratified sampling according to ensemble spread overcomes the problem reasonably well, producing sharper predictive probability density functions and BMA percentile forecasts, and more reliable PQPF than the conventional BMA approach. The continuous ranked probability scores, Brier skill scores, and reliability diagrams of the two BMA experiments were examined for all available forecast days, along with a logistic regression experiment. Stratified sampling BMA outperformed the raw ensemble and conventional BMA in all verifications, and also showed better skill than logistic regression in low-threshold forecasts.

Ensembles used for probabilistic weather forecasting often exhibit a spread-error correlation, but they tend to be underdispersive. This paper proposes a statistical method for postprocessing ensembles based on Bayesian model averaging (BMA), which is a standard method for combining predictive distributions from different sources. The BMA predictive probability density function (PDF) of any quantity of interest is a weighted average of PDFs centered on the individual bias-corrected forecasts, where the weights are equal to posterior probabilities of the models generating the forecasts and reflect the models' relative contributions to predictive skill over the training period. The BMA weights can be used to assess the usefulness of ensemble members, and this can be used as a basis for selecting ensemble members; this can be useful given the cost of running large ensembles. The BMA PDF can be represented as an unweighted ensemble of any desired size, by simulating from the BMA predictive distribution. The BMA predictive variance can be decomposed into two components, one corresponding to the between-forecast variability, and the second to the within-forecast variability. Predictive PDFs or intervals based solely on the ensemble spread incorporate the first component but not the second. Thus BMA provides a theoretical explanation of the tendency of ensembles to exhibit a spread-error correlation but yet be underdispersive. The method was applied to 48-h forecasts of surface temperature in the Pacific Northwest in January–June 2000 using the University of Washington fifth-generation Pennsylvania State University–NCAR Mesoscale Model (MM5) ensemble. The predictive PDFs were much better calibrated than the raw ensemble, and the BMA forecasts were sharp in that 90% BMA prediction intervals were 66% shorter on average than those produced by sample climatology. As a by-product, BMA yields a deterministic point forecast, and this had root-mean-square errors 7% lower than the best of the ensemble members and 8% lower than the ensemble mean. Similar results were obtained for forecasts of sea level pressure. Simulation experiments show that BMA performs reasonably well when the underlying ensemble is calibrated, or even overdispersed.

<p>In recent years, the number of large-scale hydrological forecasting systems has been steadily growing. This may lead to regions having numerous models spatially overlapping each other. Some of these regions have what we will refer to as a regional, more specialized, model for the area that performs generally better than their large-scale counterpart, considering the coarser spatial resolution and sometimes lack of calibration of the latter. Our work explored the possibility of using simple methods to retrieve hydrological information from a large-scale model, namely the National Surface and River Prediction System (NSRPS) that will eventually cover the Canadian territory, in order to improve the forecasts from a local system, namely the Système de Prévision Hydrologique (SPH) that covers most of the province of Quebec. Outputs from the two forecasting systems were thus combined using methods including the simple mean, a weighted average in which the weights are optimized either using the Kling-Gupta Efficiency (KGE) or the Continuous Ranked Probability Score (CRPS) as cost functions, or weights calculated from the residual errors of the forecasts. Bayesian Model Averaging (BMA) was also explored to combine the ensemble forecasts from both systems. The results show that it is possible to improve the local hydrological forecasts by using simple weighted combinations with forecasts from the large-scale system. Performance was assessed using many well-known criteria such as the Nash-Sutcliffe Efficiency (NSE), KGE and CRPS. Results were averaged over the 61 available gauging stations and analyzed at lead times ranging from 3 to 120 hours. We observed improvements in all criteria for lead times over 60 hours as well as no loss in performance at any lead times. Finally, the methods were also used in a leave-one-out setup to simulate performance on ungauged basins. The performance gain for ungauged basins is similar to that of the gauged basins, hinting at the fact that these simple methods could also improve forecasts in more remote territories where no measurements are available.</p>

Bayesian model averaging (BMA) is a popular approach to combine hydrologic forecasts from individual models and characterize the uncertainty induced by model structure. In the original form of BMA, the conditional probability density function (PDF) of each model is assumed to be a particular probability distribution (e.g., Gaussian, gamma, etc.). If the predictions of any hydrologic model do not follow certain distribution, a data transformation procedure is required prior to model averaging. Moreover, it is strongly recommended to apply BMA on unbiased forecasts, whereas it is sometimes difficult to effectively remove bias from the predictions of complex hydrologic models. To overcome these limitations, we develop an approach to integrate a group of multivariate functions, the so‐called copula functions, into BMA. Here we introduce a copula‐embedded BMA (Cop‐BMA) method that relaxes any assumption on the shape of conditional PDFs. Copula functions have a flexible structure and do not restrict the shape of posterior distributions. Furthermore, copulas are effective tools in removing bias from hydrologic forecasts. To compare the performance of BMA with Cop‐BMA, they are applied to hydrologic forecasts from different rainfall‐runoff and land‐surface models. We consider the streamflow observation and simulations for 10 river basins provided by the Model Parameter Estimation Experiment (MOPEX) project. Results demonstrate that the predictive distributions are more accurate and reliable, less biased, and more confident with small uncertainty after Cop‐BMA application. It is also shown that the postprocessed forecasts have better correlation with observation after Cop‐BMA application.