Automatic Calibration of Hydrological Model by Shuffled Complex Evolution Metropolis Algorithm

Markov Chain Monte Carlo (MCMC) methods have become increasingly popular for estimating the posterior probability distribution of parameters in hydrologic models. However, MCMC methods require the a priori definition of a proposal or sampling distribution, which determines the explorative capabilities and efficiency of the sampler and therefore the statistical properties of the Markov Chain and its rate of convergence. In this paper we present an MCMC sampler entitled the Shuffled Complex Evolution Metropolis algorithm (SCEM‐UA), which is well suited to infer the posterior distribution of hydrologic model parameters. The SCEM‐UA algorithm is a modified version of the original SCE‐UA global optimization algorithm developed by Duan et al. [1992]. The SCEM‐UA algorithm operates by merging the strengths of the Metropolis algorithm, controlled random search, competitive evolution, and complex shuffling in order to continuously update the proposal distribution and evolve the sampler to the posterior target distribution. Three case studies demonstrate that the adaptive capability of the SCEM‐UA algorithm significantly reduces the number of model simulations needed to infer the posterior distribution of the parameters when compared with the traditional Metropolis‐Hastings samplers.

Quantitative analysis of polycyclic aromatic hydrocarbons in soil by infrared spectroscopy combined with hybrid variable selection strategy and partial least squares

Multiple batch extraction test to estimate contaminant release parameters using a Bayesian approach

The resolution of digital elevation model (DEM) used for watershed delineation and data extracted from the delineated watershed relevant to hydrography plays an important role in the calibration of hydrological models. The effect of DEM resolution on the drainage length, watershed area and average slope are studied in many earlier works. DEM resolution can have impact on hydrologic modeling parameters, water quality simulation results. The impacts of DEM resolution on results of the watershed response and water quality modeling are studied using HSPF (Hydrologic Simulation Program Fortran) model applied to a watershed in Kentucky in this study. Three major streams in this watershed are impaired due to pathogen pollution according to the current state of Kentucky water quality standard set for recreation. The effects of DEM resolution on water quality modeling and calibration for fecal coliform bacteria are studied using a HSPF model. Results from this study indicate that the DEM resolution affects hydrography parameters of the delineated watershed, influence the calibration process, and the non-point and point source loadings to the streams in the watershed.

Hydrological models generally contain parameters that cannot be measured directly, but can only be meaningfully inferred by calibration against a historical record of input–output data. While considerable progress has been made in the development and application of automatic procedures for model calibration, such methods have received criticism for their lack of rigor in treating uncertainty in the parameter estimates. In this paper, we apply the recently developed Shuffled Complex Evolution Metropolis algorithm (SCEM-UA) to stochastic calibration of the parameters in the Sacramento Soil Moisture Accounting model (SAC-SMA) model using historical data from the Leaf River in Mississippi. The SCEM-UA algorithm is a Markov Chain Monte Carlo sampler that provides an estimate of the most likely parameter set and underlying posterior distribution within a single optimization run. In particular, we explore the relationship between the length and variability of the streamflow data and the Bayesian uncertainty associated with the SAC-SMA model parameters and compare SCEM-UA derived parameter values with those obtained using deterministic SCE-UA calibrations. Most significantly, for the Leaf River catchments under study our results demonstrate that most of the 13 SAC-SMA parameters are well identified by calibration to daily streamflow data suggesting that this data contains more information than has previously been reported in the literature.

(Un)certainty of overall binding constants of Al with dissolved organic matter determined by the Scatchard approach

Colloids have the potential to enhance the mobility of strongly sorbing radionuclide contaminants in groundwater at underground nuclear test sites. This study presents an experimental and numerical investigation of colloid-facilitated plutonium transport in fractured porous media to identify plutonium reactive transport processes. The transport parameters for dispersion, diffusion, sorption, and filtration are estimated with inverse modeling by minimizing the least-squares objective function of multicomponent concentration data from multiple transport experiments with the shuffled complex evolution metropolis algorithm. Capitalizing on an unplanned experimental artifact that led to colloid formation, we adopt a stepwise strategy to first interpret the data from each experiment separately and then to incorporate multiple experiments simultaneously to identify a suite of plutonium-colloid transport processes. Nonequilibrium or kinetic attachment and detachment of plutonium-colloid in fractures were clearly demonstrated and captured in the inverted modeling parameters along with estimates of the source plutonium fraction that formed plutonium-colloids. The results from this study provide valuable insights for understanding the transport mechanisms and environmental impacts of plutonium in groundwater aquifers.

Abstract. In hydrological modeling, model structures are developed in an iterative cycle as more and different types of measurements become available and our understanding of the hillslope or watershed improves. However, with increasing complexity of the model, it becomes more and more difficult to detect which parts of the model are deficient, or which processes should also be incorporated into the model during the next development step. In this study, we first compare two methods (the Shuffled Complex Evolution Metropolis algorithm (SCEM-UA) and the Simultaneous parameter Optimization and Data Assimilation algorithm (SODA)) to calibrate a purposely deficient 3-D hillslope-scale model to error-free, artificially generated measurements. We use a multi-objective approach based on distributed pressure head at the soil–bedrock interface and hillslope-scale discharge and water balance. For these idealized circumstances, SODA's usefulness as a diagnostic methodology is demonstrated by its ability to identify the timing and location of processes that are missing in the model. We show that SODA's state updates provide information that could readily be incorporated into an improved model structure, and that this type of information cannot be gained from parameter estimation methods such as SCEM-UA. We then expand on the SODA result by performing yet another calibration, in which we investigate whether SODA's state updating patterns are still capable of providing insight into model structure deficiencies when there are fewer measurements, which are moreover subject to measurement noise. We conclude that SODA can help guide the discussion between experimentalists and modelers by providing accurate and detailed information on how to improve spatially distributed hydrologic models.

It is imperative that the magnetometer is properly calibrated for sensor errors caused by the bias, scaling factors and non-orthogonality. It is difficult for the three-axis magnetometer to be used directly to measure the magnetic field magnitude. In this paper, we present a method based on Multiple Genetic Algorithm that can be used to estimate calibration model parameters. The Genetic Algorithm put the parameters of the calibration model as the evolutionary population; According to the fitness, the poor individual is eliminated step by step and the optimal individual is obtained after crossing and mutation; The optimal parameter estimation corresponding to optimal individuals is achieved. In the practical application of Genetic Algorithm to estimate parameters of calibration model, premature situation is fond. To avoid this situation, Multiple Genetic Algorithm is used to solve this problem. It is effective for Multiple Genetic Algorithm to estimate parameters of calibration model overcoming the premature. The simulation and experiment show that the method based on Multiple Genetic Algorithm can converge steadily and can achieve high estimation precision.

Hydrological models are necessary tools for simulating the water cycle and for understanding changes in water resources. To achieve realistic model simulation results, real-world observations are used to determine model parameters within a “calibration” procedure. Optimization techniques are usually applied in the model calibration step, which assures a maximum similarity between model outputs and observations. Practical experiences of hydrological model calibration have shown that single-objective approaches might not be adequate to tune different aspects of model simulations. These limitations can be as a result of (i) using observations that do not sufficiently represent the dynamics of the water cycle, and/or (ii) due to restricted efficiency of the applied calibration techniques. To address (i), we assess how adding daily Total Water Storage (dTWS) changes derived from the Gravity Recovery And Climate Experiment (GRACE) as an extra observations, besides the traditionally used runoff data, improves calibration of a simple 4-parameter conceptual hydrological model (GR4J, in French: modèle du Génie Rural à 4 paramètres Journalier) within the Danube River Basin. As selecting a proper calibration approach (in ii) is a challenging task and might have significant influence on the quality of model simulations, for the first time, four evolutionary optimization techniques, including the Non-dominated Sorting Genetic Algorithm II (NSGA-II), the Multi-objective Particle Swarm Optimization (MPSO), the Pareto Envelope-Based Selection Algorithm II (PESA-II), and the Strength Pareto Evolutionary Algorithm II (SPEA-II) along with the Combined objective function and Genetic Algorithm (CGA) are tested to calibrate the model in (i). A number of quality measures are applied to assess cardinality, accuracy, and diversity of solutions, which include the Number of Pareto Solutions (NPS), Generation Distance (GD), Spacing (SP), and Maximum Spread (MS). Our results indicate that according to MS and SP, NSGA-II performs better than other techniques for calibrating GR4J using GRACE dTWS and in situ runoff data. Considering GD as a measure of efficiency, MPSO is found to be the best technique. CGA is found to be an efficient method, while considering the statistics of the GR4J’s 4 calibrated parameters to rank the optimization techniques. The Nash-Sutcliffe model efficiency coefficient is also used to assess the predictive power of the calibrated hydrological models, for which our results indicate satisfactory performance of the assessed calibration experiments.

With the popularity of complex hydrologic models, the time taken to run these models is increasing substantially. Comparing and evaluating the efficacy of different optimization algorithms for calibrating computationally intensive hydrologic models is becoming a nontrivial issue. In this study, five global optimization algorithms (genetic algorithms, shuffled complex evolution, particle swarm optimization, differential evolution, and artificial immune system) were tested for automatic parameter calibration of a complex hydrologic model, Soil and Water Assessment Tool (SWAT), in four watersheds. The results show that genetic algorithms (GA) outperform the other four algorithms given model evaluation numbers larger than 2000, while particle swarm optimization (PSO) can obtain better parameter solutions than other algorithms given fewer number of model runs (less than 2000). Given limited computational time, the PSO algorithm is preferred, while GA should be chosen given plenty of computational resources. When applying GA and PSO for parameter optimization of SWAT, small population size should be chosen. Copyright © 2008 John Wiley & Sons, Ltd.

The International Roughness Index (IRI) is the one of the most important roughness indexes to quantify road surface roughness. In this paper, we propose a new hybrid approach between adaptive network based fuzzy inference system (ANFIS) and various meta-heuristic optimizations such as the genetic algorithm (GA), particle swarm optimization (PSO), and the firefly algorithm (FA) to develop several hybrid models namely GA based ANGIS (GANFIS), PSO based ANFIS (PSOANFIS), FA based ANFIS (FAANFIS), respectively, for the prediction of the IRI. A benchmark model named artificial neural networks (ANN) was also used to compare with those hybrid models. To do this, a total of 2811 samples in the case study of the north of Vietnam (Northwest region, Northeast region, and the Red River Delta Area) within the scope of management of the DRM-I Department were used to validate the models in terms of various criteria like coefficient of determination (R) and the root mean square error (RMSE). Experimental results affirmed the potentiality and effectiveness of the proposed prediction models whereas the PSOANFIS (RMSE = 0.145 and R = 0.888) is better than the other models named GANFIS (RMSE = 0.155 and R = 0.872), FAANFIS (RMSE = 0.170 and R = 0.849), and ANN (RMSE = 0.186 and R = 0.804). The results of this study are helpful for accurate prediction of the IRI for evaluation of quality of road surface roughness.

An autoregressive moving average (ARMA) is a time series model that is applied in everyday life for pattern recognition and forecasting. The ARMA model contains a noise which is assumed to have a specific distribution. The noise is often considered to have a Gaussian distribution. However in applications, the noise is sometimes found that does not have a Gaussian distribution. The first objective is to develop the ARMA model in which noise has a Laplacian distribution. The second objective is to estimate the parameters of the ARMA model. The ARMA model parameters include ARMA model orders, ARMA model coefficients, and noise variance. The parameter estimation of the ARMA model is carried out in the Bayesian framework. In the Bayesian framework, the ARMA model parameters are treated as a variable that has a prior distribution. The prior distribution for the ARMA model parameters is combined with the likelihood function for the data to get the posterior distribution for the parameter. The posterior distribution for parameters has a complex form so that the Bayes estimator cannot be determined analytically. The reversible jump Markov chain Monte Carlo (MCMC) algorithm was adopted to determine the Bayes estimator. The first result, the ARMA model can be developed by assuming Laplacian distribution noise. The second result, the performance of the algorithm was tested using simulation studies. The simulation shows that the reversible jump MCMC algorithm can estimate the parameters of the ARMA model correctly.

As the role of reservoir flow simulation increasingly impacts existing operations and field development decisions, it follows that rigor, fitness and consistency should be imposed on the calibration of reservoir flow models to dynamic data through history matching. Although a wealth of history matching techniques exist in the petroleum literature that propose novel algorithms or share case studies, seldom does the content guide the modeler in fit-for-purpose reservoir model calibration for an operating asset. To evaluate the applicability of these diverse techniques against standards required for reservoir management, an internal study was performed to benchmark four assisted history matching (AHM) techniques commonly promoted in the oil and gas industry. The techniques were vetted against a comprehensive suite of modeling requirements for multiple asset classes, integrating a variety of historical dynamic data types through the calibration of reservoir properties that control flow behavior from the field to inter-well scale. The methods benchmarked were: (1) Design of Experiments (DoE)-based, (2) Ensemble Kalman Filter and Ensemble Smoother, (3) Genetic Algorithm and (4) Generalized Travel Time Inversion. This manuscript focuses solely on the DoE-based technique.In order to consistently benchmark the techniques, a set of standards was defined against which each was evaluated to determine its suitability for widespread history matching applications. The standards involve: the capacity to parameterize (and therefore calibrate) a diversity of reservoir flow model attributes, the capacity to integrate different types of dynamic data, the level of independence from the flow simulator and the capability to provide probabilistic outcomes for predictive uncertainty assessment. Of the four techniques, the DoE-based approach uniquely satisfied all requirements. Its history matching workflow has the flexibility to incorporate any form of reservoir model parameter and to assimilate a history matching error metric for any individual or group of historical data types; therefore, benchmarking established DoE-based techniques as unambiguously the most compliant with generic asset modeling requirements. The approach was also identified as the most straightforward, both theoretically and in practical computation, and therefore applicable to the broadest range of practitioners. Perhaps most importantly, the approach demonstrated the capacity for accurate quantification of uncertainty (or non-uniqueness) in reservoir quality resulting from an exhaustive, although approximate, exploration of model parameter space and the associated history matching error metric(s).This manuscript compiles the results and insights gained from benchmarking of the DoE-based techniques through the proposal of a comprehensive assisted history matching workflow. The workflow is designed for generality while providing best practices that guide the modeler in fit-for-purpose application. Limitations of the workflow are also recognized. Key components include: selection and screening of calibration parameters based on statistical significance, development of surrogate models to characterize the relationship between parameters and the simulated historical data being integrated into the reservoir model, use of the surrogate models for exhaustive yet efficient exploration of parameter space to identify (non-unique) history matched models, and (deterministic or probabilistic) discrete reservoir model selection for use in forecast-based decision making. Each step of the AHM workflow is presented from a conceptual and applied perspective, and field applications are provided to demonstrate key concepts. Although the applications presented include two deepwater Gulf of Mexico assets, the workflow and insights provided are developed from benchmarking across a diverse suite of asset types.

Generalized likelihood uncertainty estimation (GLUE) using adaptive Markov Chain Monte Carlo sampling