Towards Conditional Parameter Estimation for Automatic Model Structure Identification: Using Mixed-Integer Calibration for Model Development

<p>Recent investigations have shown it is possible to simultaneously calibrate model structures and model parameters to identify appropriate models for a given task (Spieler et al., 2020). However, this is computationally challenging, as different model structures may use a different number of parameters. While some parameters may be shared between model structures, others might be relevant for only a few structures, which theoretically requires the calibration of conditionally active parameters. Additionally, shared model parameters might cause different effects in different model structures, causing their optimal values to differ across structures. In this study, we tested how two current “of the shelf” mixed-integer optimization algorithms perform when having to handle these peculiarities during the automatic model structure identification (AMSI) process recently introduced by Spieler et al. (2020).</p><p>To validate the current performance of the AMSI approach, we conduct a benchmark experiment with a model space consisting of 6912 different model structures.  First, all model structures are independently calibrated and validated for three hydro-climatically differing catchments using the CMA-ES algorithm and KGE as the objective function. This is referred to as standard calibration procedure. We identify the best performing model structure(s) based on validation performance and analyze the range of performance as well as the number of structures performing in a similar range. Secondly, we run AMSI on all three catchments to automatically identify the most feasible model structure based on the KGE performance. Two different mixed-integer optimization algorithms are used – namely DDS and CMA-ES. Afterwards, we compare the results to the best performing models of the standard calibration of all 6912 model structures.</p><p>Within this experimental setup, we analyze if the best performing model structure(s) AMSI identifies are identical to the best performing structures of the standard calibration and if there are differences in performance when using different optimization algorithms for AMSI. We also validate if AMSI can identify the best performing model structures for a catchment at a fraction of the computational cost than the standard calibration procedure requires by using “off the shelf” mixed-integer optimization algorithms.</p><p> </p><p> </p><p> </p><p>Spieler, D., Mai, J., Craig, J. R., Tolson, B. A., & Schütze, N. (2020). Automatic Model Structure Identification for Conceptual Hydrologic Models. Water Resources Research, 56(9). https://doi.org/10.1029/2019WR027009</p>

<p>Recent studies have introduced methods to simultaneously calibrate model structure choices and parameter values to identify an appropriate (conceptual) model structure for a given catchment. This can be done through mixed-integer optimization to identify the graph structure that links dominant flow processes (Spieler et al., 2020) or, likewise, by continuous optimization of weights when blending multiple flux equations to describe flow processes within a model (Chlumsky et al., 2021). Here, we use the combination of the mixed-integer optimization algorithm DDS and the modular modelling framework RAVEN and refer to it as Automatic Model Structure Identification (AMSI) framework.</p><p>This study validates the AMSI framework by comparing the performance of the identified AMSI model structures to two different benchmark ensembles. The first ensemble consists of the best model structures from the brute force calibration of all possible structures included in the AMSI model space (7488+). The second ensemble consists of 35+ MARRMoT structures representing a structurally more divers set of models than currently implemented in the AMSI framework. These structures stem from the MARRMoT Toolbox introduced by Knoben et al. (2019) providing established conceptual model structures based on hydrologic literature.</p><p>We analyze if the model structure(s) AMSI identifies are identical to the best performing structures of the brute force calibration and comparable in their performance to the MARRMoT ensemble. We can conclude that model structures identified with the AMSI framework can compete with the structurally more divers MARRMoT ensemble. In fact, we were surprised to see how well we do with a simple two storage structure over the 12 tested MOPEX catchments (Duan et al.,2006). We aim to discuss several emerging questions, such as the selection of a robust model structure, Equifinality in model structures, and the role of structural complexity.</p><p> </p><p>Spieler et al. (2020). https://doi.org/10.1029/2019WR027009</p><p>Chlumsky et al. (2021). https://doi.org/10.1029/2020WR029229</p><p>Knoben et al. (2019). https://doi.org/10.5194/gmd-12-2463-2019</p><p>Duan et al. (2006). https://doi.org/10.1016/j.jhydrol.2005.07.031</p>

The problems of identifying the most appropriate model structure for a given problem and quantifying the uncertainty in model structure remain outstanding research challenges for the discipline of hydrology. Progress on these problems requires understanding of the nature of differences between models. This paper presents a methodology to diagnose differences in hydrological model structures: the Framework for Understanding Structural Errors (FUSE). FUSE was used to construct 79 unique model structures by combining components of 4 existing hydrological models. These new models were used to simulate streamflow in two of the basins used in the Model Parameter Estimation Experiment (MOPEX): the Guadalupe River (Texas) and the French Broad River (North Carolina). Results show that the new models produced simulations of streamflow that were at least as good as the simulations produced by the models that participated in the MOPEX experiment. Our initial application of the FUSE method for the Guadalupe River exposed relationships between model structure and model performance, suggesting that the choice of model structure is just as important as the choice of model parameters. However, further work is needed to evaluate model simulations using multiple criteria to diagnose the relative importance of model structural differences in various climate regimes and to assess the amount of independent information in each of the models. This work will be crucial to both identifying the most appropriate model structure for a given problem and quantifying the uncertainty in model structure. To facilitate research on these problems, the FORTRAN‐90 source code for FUSE is available upon request from the lead author.

Climate warming is rapidly modifying glacier and snowmelt processes, with impacts that propagate across spatial and temporal scales, from high-elevation cryospheric dynamics to basin-scale streamflow. Capturing these cross-scale interactions requires hydrological models that explicitly represent cryospheric processes, such as glacier dynamics and snow redistribution, while also accounting for uncertainty arising from data scarcity and model structure. To address these challenges, we present a one-way coupling that integrates the Global Glacier Evolution Model (GloGEM) with Raven, a flexible hydrological modeling framework designed to emulate multiple model structures for systematic uncertainty assessment. Glacier simulations are first improved by incorporating snowline observations as an additional calibration constraint to the geodetic mass balance, reducing parameter equifinality and enhancing glacier runoff estimates. The resulting glacier runoff is then provided to the hydrological model, leading to a more robust representation of snowmelt processes and their contribution to streamflow. By evaluating multiple hydrological model structures within Raven, we further quantify uncertainties in simulated melt contributions arising from structural model choices. The framework is applied to multiple data-scarce Indus headwaters as representative high-mountain catchments, simulating melt contributions and streamflow across scales. By explicitly coupling cryosphere and hydrological models, this framework aims to improve projections of future water availability in data-scarce regions and provides a transferable approach for integrated cryosphere-hydrology modeling in complex mountain regions.

Various methods are available for assessing uncertainties in climate impact studies. Among such methods, model weighting by expert elicitation is a practical way to provide a weighted ensemble of models for specific real-world impacts. The aim is to decrease the influence of improbable models in the results and easing the decision-making process. In this study both climate and hydrological models are analyzed and the result of a research experiment is presented using model weighting with the participation of 6 climate model experts and 6 hydrological model experts. For the experiment, seven climate models are a-priori selected from a larger Euro-CORDEX ensemble of climate models and three different hydrological models are chosen for each of the three European river basins. The model weighting is based on qualitative evaluation by the experts for each of the selected models based on a training material that describes the overall model structure and literature about climate models and the performance of hydrological models for the present period. The expert elicitation process follows a three-stage approach, with two individual elicitations of probabilities and a final group consensus, where the experts are separated into two different community groups: a climate and a hydrological modeller group. The dialogue reveals that under the conditions of the study, most climate modellers prefer the equal weighting of ensemble members, whereas hydrological impact modellers in general are more open for assigning weights to different models in a multi model ensemble, based on model performance and model structure. Climate experts are more open to exclude models, if obviously flawed, than to put weights on selected models in a relatively small ensemble. The study shows that expert elicitation can be an efficient way to assign weights to different hydrological models, and thereby reduce the uncertainty in climate impact. However, for the climate model ensemble, comprising seven models, the elicitation in the format of this study could only reestablish a uniform weight between climate models.

This paper is concerned with analysis of two anticancer therapy models focused on sensitivity of therapy outcome with respect to model structure and parameters. Realistic periodic therapies are considered, combining cytotoxic and antiangiogenic agents, defined on a fixed time horizon. Tumor size at the end of therapy and average tumor size calculated over therapy horizon are chosen to represent therapy outcome. Sensitivity analysis has been performed numerically, concentrating on model parameters and structure at one hand, and on treatment protocol parameters, on the other. The results show that sensitivity of the therapy outcome highly depends on the model structure, helping to discern a good model. Moreover, it is possible to use this analysis to find a good protocol in case of heterogeneous tumors.

Hydrological modelling of ungauged catchments, which lack observed streamflow data, is an important practical goal in hydrology. A major challenge is to identify a model structure that reflects the hydrological processes relevant to the catchment of interest. Paraphrasing a well-known adage, “all models are wrong, but some model-mechanisms (process representations) might be useful.”We extend a method previously introduced for mechanism identification in gauged basins, by formulating the Bayesian inference equations in the space of (regionalized) flow indices principal components and by accounting for posterior parameter uncertainty. We use a flexible hydrological model to generate candidate mechanisms and model structures. Then, we use statistical hypothesis testing to identify the "dominant" (more a posteriori probable) hydrological mechanism. We assume that the error in the regionalization of flow indices principal components dominates the error of the hydrological model structure.The method is illustrated in 92 catchments from northern Spain. We treat 16 out of the 92 catchments as ungauged. We use 624 model-structures from FUSE (flexible hydrological model framework). The case study includes real data and synthetic experiments.The findings show that routing is among the most identifiable processes, whereas percolation and unsaturated zone processes are the least identifiable. The probability of making an identification (correct or wrong), remains stable at ~25%, both in the real and in the synthetic experiments. In the synthetic experiments, where the “true” mechanism is known, we can evaluate the reliability, i.e., the probability of identifying the true mechanism when the method makes an identification. Reliability varies between 60%-95% depending on the magnitude of the combined regionalization and hydrological error. The study contributes perspectives on hydrological mechanism identification under data-scarce conditions.Prieto et al. (2022) An Exploration of Bayesian Identification of Dominant Hydrological Mechanisms in Ungauged Catchments, WRR, 58(3), doi:10.1029/2021WR030705.

Hydrological models as common simulation tools for water resources management play a key role in improving our understanding of hydrological processes on the catchment and global scales. The reliability of hydrological simulations depends on the model structure, the quality of input data, and the calibration of model parameters. A large number of model parameters and interactions among each hydrological variable increase the complexity of the model calibration. Multi-objective model calibration is beneficial to reduce hydrological modeling uncertainty by different calibration criteria, which can lead to more realistic simulations. The FLEXG model is a conceptual glacio-hydrological model within the flexible modeling framework. The FLEXG model considers the effects of topography on the spatial distribution of temperature, precipitation, and runoff generation to have a better understanding of the impacts of landscape on hydrological processes. The FLEXG can simulate various glacio-hydrological variables such as runoff from different sources (e.g. snow and glacier melt), glacier mass balance, snow cover area, and snow water equivalent. This study aims to evaluate the influences of several calibration strategies on the FLEXG model's performance in simulating simulated runoff, snow cover area, glacier mass balance, and snow water equivalent in a glacierized catchment in Sweden. To this end, the FLEXG model was calibrated based on remotely sensed snow cover area data and compared to the traditional calibration strategy (calibrating merely against gauged streamflow data). The FLEXG model was also calibrated based on both gauged streamflow data and satellite snow cover area data as a multi-objective calibration. Glacio-hydrological simulations from the FLEXG model using different calibration strategies were evaluated with multiple metrics and at different temporal scales. Results showed that calibrating the FLEXG using only one variable (runoff or snow cover area) can provide acceptable results for only one variable, while the multi-objective calibration strategy can have acceptable simulation for both runoff and snow cover area. In addition, calibrating the FLEXG model using only snow cover area may underestimate runoff simulation.

The assessment of climate change impacts on water resources involves several methodological decisions, including choices of global climate models (GCMs), emission scenarios, downscaling techniques, and hydrologic modeling approaches. Among these, hydrologic model structure selection and parameter calibration are particularly relevant and usually have a strong subjective component. The goal of this research is to improve understanding of the role of these decisions on the assessment of the effects of climate change on hydrologic processes. The study is conducted in three basins located in the Colorado headwaters region, using four different hydrologic model structures [PRMS, VIC, Noah LSM, and Noah LSM with multiparameterization options (Noah-MP)]. To better understand the role of parameter estimation, model performance and projected hydrologic changes (i.e., changes in the hydrology obtained from hydrologic models due to climate change) are compared before and after calibration with the University of Arizona shuffled complex evolution (SCE-UA) algorithm. Hydrologic changes are examined via a climate change scenario where the Community Climate System Model (CCSM) change signal is used to perturb the boundary conditions of the Weather Research and Forecasting (WRF) Model configured at 4-km resolution. Substantial intermodel differences (i.e., discrepancies between hydrologic models) in the portrayal of climate change impacts on water resources are demonstrated. Specifically, intermodel differences are larger than the mean signal from the CCSM–WRF climate scenario examined, even after the calibration process. Importantly, traditional single-objective calibration techniques aimed to reduce errors in runoff simulations do not necessarily improve intermodel agreement (i.e., same outputs from different hydrologic models) in projected changes of some hydrological processes such as evapotranspiration or snowpack.

A progressive perceptual understanding approach was used to identify a model structure able to represent the non‐linear behaviour of the hydrological cycle in a small intermittent Mediterranean stream. The initial lumped model structure consisting of a series of four connected water tanks (LU3) progressed to a model with five tanks (LU4), and finally to a semidistributed model structure (SD4) in which spatial variability of the evapotranspiration according to the vegetation cover and to the local aspect was considered. In the final model structure, which gave the best fit (Nash–Sutcliffe efficiency index = 0·78), an additional tank representing the riparian zone was included (SD4‐R). Results showed that the abrupt changes of the riparian water table during summer and the formation of a perched water table during the transition from dry to wet conditions were the main mechanisms leading to the non‐linear hydrological behaviour. The transpiration process from the saturated zone and the spatial variability of evapotranspiration resulted in key factors successfully representing the annual water balance. The spatial and temporal validations carried out for each of the four model structures considered in this study supported the hypothesis adopted during the calibration process. Copyright © 2008 John Wiley & Sons, Ltd.

<p>Hydrological models are conventionally evaluated in terms of their response surface or likelihood surface constructed with the model parameter space. To evaluate models as hypotheses, we developed the method of <em>Flux Mapping</em> to construct a hypothesis space based on model process representation. Here we defined the hypothesis space based on dominant runoff generating mechanisms, and acceptable model runs are defined as total simulated flow with similar (and minimal) model error simulated by distinct combinations of runoff components. We demonstrate that the hypothesis space in each modeling case is the result of interplay between the factors of model structure, parameter sampling, choice of error metric, and data information content. The aim of this study is to disentangle the role of each factor in this interplay. We used two model structures (SACRAMENTO and SIMHYD), two parameter sampling approaches (small samples based on guided-search and large samples based on Latin Hypercube Sampling), three widely used error metrics (NSE, KGE, and WIA — Willmott’s Index of Agreement), and hydrological data from a range of Australian catchments. First, we characterized how the three error metrics behave under different error regimes independent of any modeling. We then conducted a series of controlled experiments, i.e. a type of one-factor-at-a-time sensitivity analysis, to unpack the role of each factor in runoff simulation. We show that KGE is a more reliable error metric compared to NSE and WIA for model evaluation. We also argue that robust error metrics and sufficient parameter sampling are necessary conditions for evaluating models as hypotheses under uncertainty. We particularly argue that sampling sufficiency, regardless of the sampling strategy, should be further evaluated based on its interaction with other modeling factors determining the model response. We conclude that the interplay of these modeling factors is complex and unique to each modeling case, and hence generalizing model-based inferences should be done with caution particularly in characterizing hydrological processes in large-sample hydrology.</p>

Hydrologic models use relatively simple mathematical equations to conceptualize and aggregate the complex, spatially distributed, and highly interrelated water, energy, and vegetation processes in a watershed. A consequence of process aggregation is that the model parameters often do not represent directly measurable entities and must therefore be estimated using measurements of the system inputs and outputs. During this process, known as model calibration, the parameters are adjusted so that the behavior of the model approximates, as closely and consistently as possible, the observed response of the hydrologic system over some historical period of time. In practice, however, because of errors in the model structure and the input (forcing) and output data, this has proven to be difficult, leading to considerable uncertainty in the model predictions. This paper surveys the limitations of current model calibration methodologies, which treat the uncertainty in the input‐output relationship as being primarily attributable to uncertainty in the parameters and presents a simultaneous optimization and data assimilation (SODA) method, which improves the treatment of uncertainty in hydrologic modeling. The usefulness and applicability of SODA is demonstrated by means of a pilot study using data from the Leaf River watershed in Mississippi and a simple hydrologic model with typical conceptual components.

We study an optimal inventory control and allocation for a seller who uses sequential Internet auctions to sell replenishment products. This problem is faced not only by retailers/distributors in the retail industry but also by manufacturers who use sequential Internet auctions to sell products that can be replenished. We consider the total expected discounted profit criterion in a finite horizon and both the discounted and average profit criteria over an infinite horizon, and show the optimality of the so-called base-stock-allocate-all (BSAA) policies. Then, we simplify the problem of computing the optimal policies and profits for either the discounted or average profit criterion in an infinite horizon to an optimization problem with only one integer decision variable. We show these results for the list price case, and also for the case where the reserve price is a decision variable. Finally, numerical results are given, where the effects of several model parameters on the optimal policies and profits are numerically analyzed.

Abstract. Low flows are often poorly reproduced by commonly used hydrological models, which are traditionally designed to meet peak flow situations. Hence, there is a need to improve hydrological models for low flow prediction. This study assessed the impact of model structure on low flow simulations and recession behaviour using the Framework for Understanding Structural Errors (FUSE). FUSE identifies the set of subjective decisions made when building a hydrological model and provides multiple options for each modeling decision. Altogether 79 models were created and applied to simulate stream flows in the snow dominated headwater catchment Narsjø in Norway (119 km2). All models were calibrated using an automatic optimisation method. The results showed that simulations of summer low flows were poorer than simulations of winter low flows, reflecting the importance of different hydrological processes. The model structure influencing winter low flow simulations is the lower layer architecture, whereas various model structures were identified to influence model performance during summer.

The understanding of the behavior of the particles inside the rotating drum is fundamental for the improvement of its efficiency. By allowing the individual monitoring of each particle, the Lagrangean numerical approach is an excellent tool to aid studies in particulate systems. Thus, the purpose of this work was to determine the combinations of the numerical parameters of the Lagrangean approach that best represent the dynamics of the particles inside a rotating drum for different types of materials under different operating conditions. Experimental methodologies for the determination of the coefficients of restitution, static friction and rolling friction were tested and validated for particle-particle and particle-wall interactions. The experimental values of these coefficients were applied in the model to verify if they would be representative in a bulk effect in the rotatory drum. A calibration process was also performed to determine the best combination of parameters and to compare it with the results of the experimental coefficients. The experimental apparatus used was suitable for the measurement of each of the coefficients. Since the value of the coefficient of restitution depends on the thickness of the material in which the particles are colliding, while in order to determine the coefficient of static friction correctly it is necessary to use the glued spheres to avoid rolling and to ensure correct measurement of the parameter. The values found in the calibration and the values of the measured parameters showed good agreement when compared with the experimental data and the Eulerian approach reported in the literature, for materials of different physical properties under different operating conditions. These were also able to successfully describe the transitional phenomena of different flow regimes. The values found in the calibration process were: 0.80 and 0.50 for the coefficient of static friction, 0.03 and 0.06 for the coefficient of rolling friction, particle-particle and particle-wall respectively, while for the measured experimentally were 0.80 and 0.49 for particle-particle and particle-wall static friction and 0.011 and 0.054 for particle-particle and particle-wall friction respectively. The results were close, showing that the calibration process was able to preserve the physical meanings of the parameters.