A hybrid approach for efficient and robust parameter estimation in biochemical pathways

Identifiability analysis is a precondition for reliable parameter estimation. Building on previous work on structural identifiability, this paper focuses on the practical identifiability and optimal experimental design (OED) of the EBPR anaerobic submodel. The nonnegative determinant of the Fisher information matrix (FIM) found in this study clearly demonstrates that the parametersYPO4,KA,qPHA, andXPAOin the submodel are practically identifiable usingSAandSPO4as the measured variables and fixingKPPas the default value. Furthermore, fixingKPPto study the practical identifiability of the other parameters and to estimate their values is shown to be valid. Subsequently, a modeling-based procedure for the OED for parameter estimation was proposed and applied successfully to anaerobic phosphorus release experiments. According to the FIMD-criterion, the optimal experimental condition was determined to be an initialSAconcentration of 300 mg/L. Under the optimal experimental condition, errors in the values ofYPO4,KA,qPHA, andXPAOare all below 20%, and the estimated values were 0.35 ± 0.02 mg P/mg COD, 3.88 ± 0.41 mg COD/L, 3.35 ± 0.27 mg P/(mgCOD⁎d-1), and 1500 ± 72 mg COD/L, respectively. Compared to the results from the nonoptimal experimental condition, the practical identifiability and the estimation precision of the four parameters were improved.

This paper gives a comprehensive review of the application of meta-heuristics to optimization problems in systems biology, mainly focussing on the parameter estimation problem (also called the inverse problem or model calibration). It is intended for either the system biologist who wishes to learn more about the various optimization techniques available and/or the meta-heuristic optimizer who is interested in applying such techniques to problems in systems biology. First, the parameter estimation problems emerging from different areas of systems biology are described from the point of view of machine learning. Brief descriptions of various meta-heuristics developed for these problems follow, along with outlines of their advantages and disadvantages. Several important issues in applying meta-heuristics to the systems biology modelling problem are addressed, including the reliability and identifiability of model parameters, optimal design of experiments, and so on. Finally, we highlight some possible future research directions in this field.

BackgroundMathematical models play a key role in systems biology: they summarize the currently available knowledge in a way that allows to make experimentally verifiable predictions. Model calibration consists of finding the parameters that give the best fit to a set of experimental data, which entails minimizing a cost function that measures the goodness of this fit. Most mathematical models in systems biology present three characteristics which make this problem very difficult to solve: they are highly non-linear, they have a large number of parameters to be estimated, and the information content of the available experimental data is frequently scarce. Hence, there is a need for global optimization methods capable of solving this problem efficiently.ResultsA new approach for parameter estimation of large scale models, called Cooperative Enhanced Scatter Search (CeSS), is presented. Its key feature is the cooperation between different programs (“threads”) that run in parallel in different processors. Each thread implements a state of the art metaheuristic, the enhanced Scatter Search algorithm (eSS). Cooperation, meaning information sharing between threads, modifies the systemic properties of the algorithm and allows to speed up performance. Two parameter estimation problems involving models related with the central carbon metabolism of E. coli which include different regulatory levels (metabolic and transcriptional) are used as case studies. The performance and capabilities of the method are also evaluated using benchmark problems of large-scale global optimization, with excellent results.ConclusionsThe cooperative CeSS strategy is a general purpose technique that can be applied to any model calibration problem. Its capability has been demonstrated by calibrating two large-scale models of different characteristics, improving the performance of previously existing methods in both cases. The cooperative metaheuristic presented here can be easily extended to incorporate other global and local search solvers and specific structural information for particular classes of problems.

The problem of optimal experimental design (OED) for parameter estimation of nonlinear dynamic systems is considered. It is shown how this problem can be formulated as a dynamic optimization (optimal control) problem where the performance index is usually a scalar function of the Fisher information matrix. Numerical solutions can be obtained using direct methods, which transform the original problem into a nonlinear programming (NLP) problem via parametrizations. However, because of the frequent nonsmoothness of the cost functions, the use of gradient-based methods to solve this NLP might lead to local solutions. Stochastic methods of global optimization are suggested as robust alternatives. A case study considering the OED for parameter estimation in a fed-batch bioreactor is used to illustrate the performance and advantages of two selected stochastic algorithms.

BackgroundThe development of large-scale kinetic models is one of the current key issues in computational systems biology and bioinformatics. Here we consider the problem of parameter estimation in nonlinear dynamic models. Global optimization methods can be used to solve this type of problems but the associated computational cost is very large. Moreover, many of these methods need the tuning of a number of adjustable search parameters, requiring a number of initial exploratory runs and therefore further increasing the computation times.Here we present a novel parallel method, self-adaptive cooperative enhanced scatter search (saCeSS), to accelerate the solution of this class of problems. The method is based on the scatter search optimization metaheuristic and incorporates several key new mechanisms: (i) asynchronous cooperation between parallel processes, (ii) coarse and fine-grained parallelism, and (iii) self-tuning strategies.ResultsThe performance and robustness of saCeSS is illustrated by solving a set of challenging parameter estimation problems, including medium and large-scale kinetic models of the bacterium E. coli, bakerés yeast S. cerevisiae, the vinegar fly D. melanogaster, Chinese Hamster Ovary cells, and a generic signal transduction network.The results consistently show that saCeSS is a robust and efficient method, allowing very significant reduction of computation times with respect to several previous state of the art methods (from days to minutes, in several cases) even when only a small number of processors is used.ConclusionsThe new parallel cooperative method presented here allows the solution of medium and large scale parameter estimation problems in reasonable computation times and with small hardware requirements. Further, the method includes self-tuning mechanisms which facilitate its use by non-experts. We believe that this new method can play a key role in the development of large-scale and even whole-cell dynamic models.

Numerical strategies for optimal experimental design for parameter identification of non-linear dynamic (Bio-)chemical processes

Deep learning for parameter estimation of supermassive binary black holes with simulated LISA data

BackgroundMathematical modeling is a powerful tool to analyze, and ultimately design biochemical networks. However, the estimation of the parameters that appear in biochemical models is a significant challenge. Parameter estimation typically involves expensive function evaluations and noisy data, making it difficult to quickly obtain optimal solutions. Further, biochemical models often have many local extrema which further complicates parameter estimation. Toward these challenges, we developed Dynamic Optimization with Particle Swarms (DOPS), a novel hybrid meta-heuristic that combined multi-swarm particle swarm optimization with dynamically dimensioned search (DDS). DOPS uses a multi-swarm particle swarm optimization technique to generate candidate solution vectors, the best of which is then greedily updated using dynamically dimensioned search.ResultsWe tested DOPS using classic optimization test functions, biochemical benchmark problems and real-world biochemical models. We performed mathcal {T} = 25 trials with mathcal {N} = 4000 function evaluations per trial, and compared the performance of DOPS with other commonly used meta-heuristics such as differential evolution (DE), simulated annealing (SA) and dynamically dimensioned search (DDS). On average, DOPS outperformed other common meta-heuristics on the optimization test functions, benchmark problems and a real-world model of the human coagulation cascade.ConclusionsDOPS is a promising meta-heuristic approach for the estimation of biochemical model parameters in relatively few function evaluations. DOPS source code is available for download under a MIT license at http://www.varnerlab.org.

PFIM 4.0, an extended R program for design evaluation and optimization in nonlinear mixed-effect models

Unlike steady-state optimization and linear model predictive control, dynamic optimization and nonlinear model predictive control is underway to industry application. This paper discusses the implementation issues of two-level control strategy integrated dynamic real-time optimization and nonlinear model predictive control of a heat-integrated distillation column system. To realize this scheme, four nonlinear programming problems: economic operation optimization, data reconciliation and state estimation, parameter estimation and nonlinear model predictive control, are considered based on system rigorous dynamic model. The economic operation optimization is to minimize the cost of the plant. The unmeasured states are estimated by measurements of temperatures, pressures and flow rates. Time-variant parameters such as tray efficiencies and feed composition are updated on-line. The sequential quadratic programming and orthogonal collocation are used to solve the dynamic nonlinear optimization problems.

Experiment Design for an Optimized Identification of Kinetic Parameters of a Fluidized Bed Combustion

When non-linear models are fitted to experimental data, parameter estimates can be poorly constrained albeit being identifiable in principle. This means that along certain paths in parameter space, the log-likelihood does not exceed a given statistical threshold but remains bounded. This situation, denoted as practical non-identifiability, can be detected by Monte Carlo sampling or by systematic scanning using the profile likelihood method. In contrast, any method based on a Taylor expansion of the log-likelihood around the optimum, e.g., parameter uncertainty estimation by the Fisher Information Matrix, reveals no information about the boundedness at all. In this work, we present a geometric approach, approximating the original log-likelihood by geodesic coordinates of the model manifold. The Christoffel Symbols in the geodesic equation are fixed to those obtained from second order model sensitivities at the optimum. Based on three exemplary non-linear models we show that the information about the log-likelihood bounds and flat parameter directions can already be contained in this local information. Whereas the unbounded case represented by the Fisher Information Matrix is embedded in the geometric framework as vanishing Christoffel Symbols, non-vanishing constant Christoffel Symbols prove to define prototype non-linear models featuring boundedness and flat parameter directions of the log-likelihood. Finally, we investigate if those models could allow to approximate and replace computationally expensive objective functions originating from non-linear models by a surrogate objective function in parameter estimation problems.

The Fisher information matrix provides a way to measure the amount of information given observed data based on parameters of interest. There are many relative applications in statistical modeling, system identification, and parameter estimation. This article presents an enhanced resampling-based method with independent perturbation to estimate the Fisher information matrix. We show its accuracy via variance reduction that is reduced by a factor of n, where n is the sample size, with a numerical example under the signal-plus-noise setting also illustrating same.

Modeling and Analysis Challenges in Systems Biology

BackgroundMathematical models provide abstract representations of the information gained from experimental observations on the structure and function of a particular biological system. Conferring a predictive character on a given mathematical formulation often relies on determining a number of non-measurable parameters that largely condition the model's response. These parameters can be identified by fitting the model to experimental data. However, this fit can only be accomplished when identifiability can be guaranteed.ResultsWe propose a novel iterative identification procedure for detecting and dealing with the lack of identifiability. The procedure involves the following steps: 1) performing a structural identifiability analysis to detect identifiable parameters; 2) globally ranking the parameters to assist in the selection of the most relevant parameters; 3) calibrating the model using global optimization methods; 4) conducting a practical identifiability analysis consisting of two (a priori and a posteriori) phases aimed at evaluating the quality of given experimental designs and of the parameter estimates, respectively and 5) optimal experimental design so as to compute the scheme of experiments that maximizes the quality and quantity of information for fitting the model.ConclusionsThe presented procedure was used to iteratively identify a mathematical model that describes the NF-κB regulatory module involving several unknown parameters. We demonstrated the lack of identifiability of the model under typical experimental conditions and computed optimal dynamic experiments that largely improved identifiability properties.