Model structure exploration for index flood regionalization

Multi-model studies are widespread in large-sample hydrology. However, significant challenges remain in identifying interpretable connections between high-performing model structures and catchment characteristics, and thus in developing a coherent strategy for developing tailored multi-model ensembles. Here, we assess the importance of selecting model structures that are consistent with the expected hydrological variability across the study domain. We compare results of two modular modelling frameworks across 998 catchments in Great Britain. The RRMT framework includes model structures historically evolved in the UK, while the FUSE framework employs model structures from diverse global origins. While both groups of model structures contain high-performing members, the historically evolved group members separate between catchments in line with our expectation of hydrologic differences. We find that four hydrologic signatures organize these distinctions. Our results emphasize (1) the importance of model structure selection based on explicit perceptual models, and (2) the need to look beyond statistical performance alone.

<p>Recent investigations have shown it is possible to simultaneously calibrate model structures and model parameters to identify appropriate models for a given task (Spieler et al., 2020). However, this is computationally challenging, as different model structures may use a different number of parameters. While some parameters may be shared between model structures, others might be relevant for only a few structures, which theoretically requires the calibration of conditionally active parameters. Additionally, shared model parameters might cause different effects in different model structures, causing their optimal values to differ across structures. In this study, we tested how two current “of the shelf” mixed-integer optimization algorithms perform when having to handle these peculiarities during the automatic model structure identification (AMSI) process recently introduced by Spieler et al. (2020).</p><p>To validate the current performance of the AMSI approach, we conduct a benchmark experiment with a model space consisting of 6912 different model structures.  First, all model structures are independently calibrated and validated for three hydro-climatically differing catchments using the CMA-ES algorithm and KGE as the objective function. This is referred to as standard calibration procedure. We identify the best performing model structure(s) based on validation performance and analyze the range of performance as well as the number of structures performing in a similar range. Secondly, we run AMSI on all three catchments to automatically identify the most feasible model structure based on the KGE performance. Two different mixed-integer optimization algorithms are used – namely DDS and CMA-ES. Afterwards, we compare the results to the best performing models of the standard calibration of all 6912 model structures.</p><p>Within this experimental setup, we analyze if the best performing model structure(s) AMSI identifies are identical to the best performing structures of the standard calibration and if there are differences in performance when using different optimization algorithms for AMSI. We also validate if AMSI can identify the best performing model structures for a catchment at a fraction of the computational cost than the standard calibration procedure requires by using “off the shelf” mixed-integer optimization algorithms.</p><p> </p><p> </p><p> </p><p>Spieler, D., Mai, J., Craig, J. R., Tolson, B. A., & Schütze, N. (2020). Automatic Model Structure Identification for Conceptual Hydrologic Models. Water Resources Research, 56(9). https://doi.org/10.1029/2019WR027009</p>

<p>Recent studies have introduced methods to simultaneously calibrate model structure choices and parameter values to identify an appropriate (conceptual) model structure for a given catchment. This can be done through mixed-integer optimization to identify the graph structure that links dominant flow processes (Spieler et al., 2020) or, likewise, by continuous optimization of weights when blending multiple flux equations to describe flow processes within a model (Chlumsky et al., 2021). Here, we use the combination of the mixed-integer optimization algorithm DDS and the modular modelling framework RAVEN and refer to it as Automatic Model Structure Identification (AMSI) framework.</p><p>This study validates the AMSI framework by comparing the performance of the identified AMSI model structures to two different benchmark ensembles. The first ensemble consists of the best model structures from the brute force calibration of all possible structures included in the AMSI model space (7488+). The second ensemble consists of 35+ MARRMoT structures representing a structurally more divers set of models than currently implemented in the AMSI framework. These structures stem from the MARRMoT Toolbox introduced by Knoben et al. (2019) providing established conceptual model structures based on hydrologic literature.</p><p>We analyze if the model structure(s) AMSI identifies are identical to the best performing structures of the brute force calibration and comparable in their performance to the MARRMoT ensemble. We can conclude that model structures identified with the AMSI framework can compete with the structurally more divers MARRMoT ensemble. In fact, we were surprised to see how well we do with a simple two storage structure over the 12 tested MOPEX catchments (Duan et al.,2006). We aim to discuss several emerging questions, such as the selection of a robust model structure, Equifinality in model structures, and the role of structural complexity.</p><p> </p><p>Spieler et al. (2020). https://doi.org/10.1029/2019WR027009</p><p>Chlumsky et al. (2021). https://doi.org/10.1029/2020WR029229</p><p>Knoben et al. (2019). https://doi.org/10.5194/gmd-12-2463-2019</p><p>Duan et al. (2006). https://doi.org/10.1016/j.jhydrol.2005.07.031</p>

The problems of identifying the most appropriate model structure for a given problem and quantifying the uncertainty in model structure remain outstanding research challenges for the discipline of hydrology. Progress on these problems requires understanding of the nature of differences between models. This paper presents a methodology to diagnose differences in hydrological model structures: the Framework for Understanding Structural Errors (FUSE). FUSE was used to construct 79 unique model structures by combining components of 4 existing hydrological models. These new models were used to simulate streamflow in two of the basins used in the Model Parameter Estimation Experiment (MOPEX): the Guadalupe River (Texas) and the French Broad River (North Carolina). Results show that the new models produced simulations of streamflow that were at least as good as the simulations produced by the models that participated in the MOPEX experiment. Our initial application of the FUSE method for the Guadalupe River exposed relationships between model structure and model performance, suggesting that the choice of model structure is just as important as the choice of model parameters. However, further work is needed to evaluate model simulations using multiple criteria to diagnose the relative importance of model structural differences in various climate regimes and to assess the amount of independent information in each of the models. This work will be crucial to both identifying the most appropriate model structure for a given problem and quantifying the uncertainty in model structure. To facilitate research on these problems, the FORTRAN‐90 source code for FUSE is available upon request from the lead author.

In the paper some quantitative criteria for evaluating alternative linear model structures are presented and applied to two metabolic systems. Linear time-invariant structured models and input/output descriptions are proposed for the two metabolic processes considered, bilirubin and riphamicin kinetics, and maximum likelihood parameter estimates are obtained from input/output test data. The considered criteria include model plausibility and quantitative measures, e.g. the F-test, Akaike information criterion and the inverse of the information matrix, which are based on the results of model identification from input/output data. The combined use of the Akaike criterion and of the inverse of the information matrix proves to be, in conjunction with model plausibility, a very reliable tool in the metabolic model structure determination.

Simulations of human behavior in water resources systems are challenged by uncertainty in model structure and parameters. The increasing availability of observations describing these systems provides the opportunity to infer a set of plausible model structures using data‐driven approaches. This study develops a three‐phase approach to the inference of model structures and parameterizations from data: problem definition, model generation, and model evaluation, illustrated on a case study of land use decisions in the Tulare Basin, California. We encode the generalized decision problem as an arbitrary mapping from a high‐dimensional data space to the action of interest and use multiobjective genetic programming to search over a family of functions that perform this mapping for both regression and classification tasks. To facilitate the discovery of models that are both realistic and interpretable, the algorithm selects model structures based on multiobjective optimization of (1) their performance on a training set and (2) complexity, measured by the number of variables, constants, and operations composing the model. After training, optimal model structures are further evaluated according to their ability to generalize to held‐out test data and clustered based on their performance, complexity, and generalization properties. Finally, we diagnose the causes of good and bad generalization by performing sensitivity analysis across model inputs and within model clusters. This study serves as a template to inform and automate the problem‐dependent task of constructing robust data‐driven model structures to describe human behavior in water resources systems.

This study presents a goal programming approach to determine the best self-drive tourism route in Terengganu. The goal programming was formulated based on the model of Traveling Salesman Problem (TSP). This study involved seven cities in Terengganu which can be connected by road network. The data of the tourist attractions for each city was used in this study. The model of self-drive tourism route was constructed so as to connect all these seven cities by minimizing the travel distance, minimizing accommodation costs and maximizing the total number of tourist attractions. To achieve the objectives of this study, we develop the goal programming model that considers three model structures with each of them consists three goal constraints which are subjected to four hard constraints. This model has been solved using LINGO12.0 software. The results obtained from each model structures will be compared in order to get the most optimum route for self-drive tourism. The result showed that the first model structure was suitable to be used because this model structure gave the optimum result compared to other model structures. It produced the minimum total traveling cost of RM967.70 compared to the model structure two of RM977.10 and model structure three of RM1233.10.

BackgroundIdentifying a proper model structure, using methods that address both structural and parameter uncertainty, is a crucial problem within the systems approach to biology. And yet, it has a marginal presence in the recent literature. While many existing approaches integrate methods for simulation and parameter estimation of a single model to address parameter uncertainty, only few of them address structural uncertainty at the same time. The methods for handling structure uncertainty often oversimplify the problem by allowing the human modeler to explicitly enumerate a relatively small number of alternative model structures. On the other hand, process-based modeling methods provide flexible modular formalisms for specifying large classes of plausible model structures, but their scope is limited to deterministic models. Here, we aim at extending the scope of process-based modeling methods to inductively learn stochastic models from knowledge and data.ResultsWe combine the flexibility of process-based modeling in terms of addressing structural uncertainty with the benefits of stochastic modeling. The proposed method combines search trough the space of plausible model structures, the parsimony principle and parameter estimation to identify a model with optimal structure and parameters. We illustrate the utility of the proposed method on four stochastic modeling tasks in two domains: gene regulatory networks and epidemiology. Within the first domain, using synthetically generated data, the method successfully recovers the structure and parameters of known regulatory networks from simulations. In the epidemiology domain, the method successfully reconstructs previously established models of epidemic outbreaks from real, sparse and noisy measurement data.ConclusionsThe method represents a unified approach to modeling dynamical systems that allows for flexible formalization of the space of candidate model structures, deterministic and stochastic interpretation of model dynamics, and automated induction of model structure and parameters from data. The method is able to reconstruct models of dynamical systems from synthetic and real data.

Abstract. In this second part of the two-part paper, the data driven modeling (DDM) experiment, presented and explained in the first part, is implemented. Inputs for the five case studies (half-hourly actual evapotranspiration, daily peat soil moisture, daily till soil moisture, and two daily rainfall-runoff datasets) are identified, either based on previous studies or using the mutual information content. Twelve groups (realizations) were randomly generated from each dataset by randomly sampling without replacement from the original dataset. Neural networks (ANNs), genetic programming (GP), evolutionary polynomial regression (EPR), Support vector machines (SVM), M5 model trees (M5), K-nearest neighbors (K-nn), and multiple linear regression (MLR) techniques are implemented and applied to each of the 12 realizations of each case study. The predictive accuracy and uncertainties of the various techniques are assessed using multiple average overall error measures, scatter plots, frequency distribution of model residuals, and the deterioration rate of prediction performance during the testing phase. Gamma test is used as a guide to assist in selecting the appropriate modeling technique. Unlike two nonlinear soil moisture case studies, the results of the experiment conducted in this research study show that ANNs were a sub-optimal choice for the actual evapotranspiration and the two rainfall-runoff case studies. GP is the most successful technique due to its ability to adapt the model complexity to the modeled data. EPR performance could be close to GP with datasets that are more linear than nonlinear. SVM is sensitive to the kernel choice and if appropriately selected, the performance of SVM can improve. M5 performs very well with linear and semi linear data, which cover wide range of hydrological situations. In highly nonlinear case studies, ANNs, K-nn, and GP could be more successful than other modeling techniques. K-nn is also successful in linear situations, and it should not be ignored as a potential modeling technique for hydrological applications.

The biological sequences that occur in DNA, RNA and proteins can be considered as strings formed over the well defined chemical alphabets. Such gene sequences form structure based on the complementary pair and the structures can be interpreted as languages. Matrix insertion-deletion system has been introduced a few years back that modelled several bio-molecular structures occur at intramolecular and intermolecular level. In this paper, we identify some structures that are frequently noticed during RNA folding process such as double bulge loop, extended internal loop, triple stem and loop and we give the corresponding formal language representation to such structures. Further, we model the structures using Matrix insertion-deletion systems. This work is pioneering to give the language representation and modelling the structures of RNA folding process using formal grammar.

Hydrological simulations in the Indian subcontinent are impacted by substantial uncertainty contributed by the selection of model structure and parameterization. Nevertheless, most studies in the Indian subcontinent have relied on a single model structure to simulate streamflow, without evaluating how such a choice might impact simulations in ungauged basins. In this study, we comprehensively evaluate the 277 gauged basins spanning across the diverse hydro-climatic regions of the Indian subcontinent using the Framework for Understanding Structural Errors (FUSE). FUSE allows testing different model structures within a controlled experimental framework, enabling the systematic evaluation of the impact of different model structures on streamflow simulations. For each gauged basin, we calibrate 78 FUSE structures and evaluate their performance with respect to the basic benchmarking models using the HydroBM python package and simulations from Noah-Multiparameterization Land Surface Model (Noah-MP LSM).For regionalization, we generate ensembles of 500 parameter sets for each selected decision structure using Latin Hypercube Sampling and propagated these through FUSE to characterize predictive uncertainty in ungauged basin simulations. These simulations are subsequently used to train surrogate emulators of the performance surface response, enabling efficient transfer of parameters from gauge to ungauged basins.Results indicate strong variability in predictive skill, uncertainty, and regionalization performance across model structures, highlighting that structures identified as optimal in calibrated basins do not necessarily generalize under parameter transfer. These findings underscore the need to consider structural benchmarking, uncertainty reliability, and regionalization performance when developing hydrological modelling frameworks for data-scarce regions such as the Indian subcontinent.

Adaptive control in an artificial pancreas for people with type 1 diabetes

Probabilistic logic programming offers a powerful way to describe and evaluate structured statistical models. To investigate the practicality of probabilistic logic programming for structure learning in bioinformatics, we undertook a simplified bacterial gene-finding benchmark in PRISM, a probabilistic dialect of Prolog. We evaluate Hidden Markov Model structures for bacterial protein-coding gene potential, including a simple null model structure, three structures based on existing bacterial gene finders and two novel model structures. We test standard versions as well as ADPH length modeling and three-state versions of the five model structures. The models are all represented as probabilistic logic programs and evaluated using the PRISM machine learning system in terms of statistical information criteria and gene-finding prediction accuracy, in two bacterial genomes. Neither of our implementations of the two currently most used model structures are best performing in terms of statistical information criteria or prediction performances, suggesting that better-fitting models might be achievable. The source code of all PRISM models, data and additional scripts are freely available for download at: http://github.com/somork/codonhmm. Supplementary data are available at Bioinformatics online.

In relevance logic it has become commonplace to associate with each logic both an algebraic counterpart and a relational counterpart. The former comes from the Lindenbaum construction; the latter, called a model structure, is designed for semantical purposes. Knowing that they are related through the logic, we may enquire after the algebraic relationship between the algebra and the model structure. This paper offers a complete solution for the relevance logic R⌝. Namely, R⌝-algebras and R⌝-model structures can be obtained from each other, and represented in terms of each other, by application of power constructions.

A progressive perceptual understanding approach was used to identify a model structure able to represent the non‐linear behaviour of the hydrological cycle in a small intermittent Mediterranean stream. The initial lumped model structure consisting of a series of four connected water tanks (LU3) progressed to a model with five tanks (LU4), and finally to a semidistributed model structure (SD4) in which spatial variability of the evapotranspiration according to the vegetation cover and to the local aspect was considered. In the final model structure, which gave the best fit (Nash–Sutcliffe efficiency index = 0·78), an additional tank representing the riparian zone was included (SD4‐R). Results showed that the abrupt changes of the riparian water table during summer and the formation of a perched water table during the transition from dry to wet conditions were the main mechanisms leading to the non‐linear hydrological behaviour. The transpiration process from the saturated zone and the spatial variability of evapotranspiration resulted in key factors successfully representing the annual water balance. The spatial and temporal validations carried out for each of the four model structures considered in this study supported the hypothesis adopted during the calibration process. Copyright © 2008 John Wiley & Sons, Ltd.