Parameter Estimation Using Metaheuristics in Systems Biology: A Comprehensive Review

Many key problems in science and engineering can be formulated and solved using global optimization techniques. In the particular case of computational biology, the development of dynamic (kinetic) models is one of the current key issues. In this context, the problem of parameter estimation (model calibration) remains as a very challenging task. The complexity of the underlying models requires the use of efficient solvers to achieve adequate results in reasonable computation times. Metaheuristics have been the focus of great consideration as an efficient way of solving hard global optimization problems. Even so, in most realistic applications, metaheuristics require a very large computation time to obtain an acceptable result. Therefore, several parallel schemes have been proposed, most of them focused on traditional parallel programming interfaces and infrastructures. However, with the emergence of cloud computing, new programming models have been proposed to deal with large-scale data processing on clouds. In this paper we explore the applicability of these new models for global optimization problems using as a case study a set of challenging parameter estimation problems in systems biology. We have developed, using Spark, an island-based parallel version of Differential Evolution. Differential Evolution is a simple population-based metaheuristic that, at the same time, is very popular for being very efficient in real function global optimization. Several experiments were conducted both on a cluster and on the Microsoft Azure public cloud to evaluate the speedup and efficiency of the proposal, concluding that the Spark implementation achieves not only competitive speedup against the serial implementation, but also good scalability when the number of nodes grows. The results can be useful for those interested in using parallel metaheuristics for global optimization problems benefiting from the potential of new cloud programming models.

The use of metamodeling techniques in the design and analysis of computer experiments has progressed remarkably in the past two decades, but how far have we really come? This is the question that we investigate in this paper, namely, the extent to which the use of metamodeling techniques in multidisciplinary design optimization have evolved in the two decades since the seminal paper on Design and Analysis of Computer Experiments by Sacks et al. As part of this review, we examine the motivation for advancements in metamodeling techniques from both a historical perspective and the research itself. Based on current thrusts in the field, we emphasize multi-level/multi-fidelity approximations and ensembles of metamodels, as well as the availability of metamodels within commercial software and for design space exploration and visualization in this review. Our closing remarks offer insight into future research directions – nearly the same ones that have motivated us in the past.

A hybrid approach for efficient and robust parameter estimation in biochemical pathways

MotivationParameter estimation is a cornerstone of data-driven modeling in systems biology. Yet, constructing such problems in a reproducible and accessible manner remains challenging. The PEtab format has established itself as a powerful community standard to encode parameter estimation problems, promoting interoperability and reusability. However, its reliance on multiple interlinked files—often edited manually—can introduce inconsistencies, and new users often struggle to navigate them. Here, we present PEtab-GUI, an open-source Python application designed to streamline the creation, editing, and validation of PEtab problems through an intuitive graphical user interface. PEtab-GUI integrates all PEtab components, including SBML models and tabular files, into a single environment with live error checking and customizable defaults. Interactive visualization and simulation capabilities enable users to inspect the relationship between the model and the data. PEtab-GUI lowers the barrier to entry for specifying standardized parameter estimation problems, making dynamic modeling more accessible, especially in educational and interdisciplinary settings.Availability and implementationPEtab-GUI is implemented in Python, open-source under a 3-Clause BSD license. The code, designed to be modular and extensible, is hosted on https://github.com/PEtab-dev/PEtab-GUI, available as a Zenodo repository at https://doi.org/10.5281/zenodo.15355752, and can be installed from PyPI.

Reproducibility and reusability of the results of data-based modeling studies are essential. Yet, there has been—so far—no broadly supported format for the specification of parameter estimation problems in systems biology. Here, we introduce PEtab, a format which facilitates the specification of parameter estimation problems using Systems Biology Markup Language (SBML) models and a set of tab-separated value files describing the observation model and experimental data as well as parameters to be estimated. We already implemented PEtab support into eight well-established model simulation and parameter estimation toolboxes with hundreds of users in total. We provide a Python library for validation and modification of a PEtab problem and currently 20 example parameter estimation problems based on recent studies.

Modeling and Analysis Challenges in Systems Biology

BackgroundThe development of large-scale kinetic models is one of the current key issues in computational systems biology and bioinformatics. Here we consider the problem of parameter estimation in nonlinear dynamic models. Global optimization methods can be used to solve this type of problems but the associated computational cost is very large. Moreover, many of these methods need the tuning of a number of adjustable search parameters, requiring a number of initial exploratory runs and therefore further increasing the computation times.Here we present a novel parallel method, self-adaptive cooperative enhanced scatter search (saCeSS), to accelerate the solution of this class of problems. The method is based on the scatter search optimization metaheuristic and incorporates several key new mechanisms: (i) asynchronous cooperation between parallel processes, (ii) coarse and fine-grained parallelism, and (iii) self-tuning strategies.ResultsThe performance and robustness of saCeSS is illustrated by solving a set of challenging parameter estimation problems, including medium and large-scale kinetic models of the bacterium E. coli, bakerés yeast S. cerevisiae, the vinegar fly D. melanogaster, Chinese Hamster Ovary cells, and a generic signal transduction network.The results consistently show that saCeSS is a robust and efficient method, allowing very significant reduction of computation times with respect to several previous state of the art methods (from days to minutes, in several cases) even when only a small number of processors is used.ConclusionsThe new parallel cooperative method presented here allows the solution of medium and large scale parameter estimation problems in reasonable computation times and with small hardware requirements. Further, the method includes self-tuning mechanisms which facilitate its use by non-experts. We believe that this new method can play a key role in the development of large-scale and even whole-cell dynamic models.

A bootstrapping-based statistical procedure for multivariate calibration of sensor arrays

An important and challenging problem in systems biology is the inference of gene regulatory networks from short non-stationary time series of transcriptional profiles. A popular approach that has been widely applied to this end is based on dynamic Bayesian networks (DBNs), although traditional homogeneous DBNs fail to model the non-stationarity and time-varying nature of the gene regulatory processes. Various authors have therefore recently proposed combining DBNs with multiple changepoint processes to obtain time varying dynamic Bayesian networks (TV-DBNs). However, TV-DBNs are not without problems. Gene expression time series are typically short, which leaves the model over-flexible, leading to over-fitting or inflated inference uncertainty. In the present paper, we introduce a Bayesian regularization scheme that addresses this difficulty. Our approach is based on the rationale that changes in gene regulatory processes appear gradually during an organism's life cycle or in response to a changing environment, and we have integrated this notion in the prior distribution of the TV-DBN parameters. We have extensively tested our regularized TV-DBN model on synthetic data, in which we have simulated short non-homogeneous time series produced from a system subject to gradual change. We have then applied our method to real-world gene expression time series, measured during the life cycle of Drosophila melanogaster, under artificially generated constant light condition in Arabidopsis thaliana, and from a synthetically designed strain of Saccharomyces cerevisiae exposed to a changing environment.

BackgroundMathematical models play a key role in systems biology: they summarize the currently available knowledge in a way that allows to make experimentally verifiable predictions. Model calibration consists of finding the parameters that give the best fit to a set of experimental data, which entails minimizing a cost function that measures the goodness of this fit. Most mathematical models in systems biology present three characteristics which make this problem very difficult to solve: they are highly non-linear, they have a large number of parameters to be estimated, and the information content of the available experimental data is frequently scarce. Hence, there is a need for global optimization methods capable of solving this problem efficiently.ResultsA new approach for parameter estimation of large scale models, called Cooperative Enhanced Scatter Search (CeSS), is presented. Its key feature is the cooperation between different programs (“threads”) that run in parallel in different processors. Each thread implements a state of the art metaheuristic, the enhanced Scatter Search algorithm (eSS). Cooperation, meaning information sharing between threads, modifies the systemic properties of the algorithm and allows to speed up performance. Two parameter estimation problems involving models related with the central carbon metabolism of E. coli which include different regulatory levels (metabolic and transcriptional) are used as case studies. The performance and capabilities of the method are also evaluated using benchmark problems of large-scale global optimization, with excellent results.ConclusionsThe cooperative CeSS strategy is a general purpose technique that can be applied to any model calibration problem. Its capability has been demonstrated by calibrating two large-scale models of different characteristics, improving the performance of previously existing methods in both cases. The cooperative metaheuristic presented here can be easily extended to incorporate other global and local search solvers and specific structural information for particular classes of problems.

A First Course in Systems Biology is intended to serve as the textbook for a semester-long course in systems biology directed toward advanced undergraduate or graduate students. The book meets this goal admirably. Eberhard O. Voit writes in a style that is both interesting to read and highly informational. A First Course in Systems Biology manages to combine the fields of mathematics, engineering, and biology in a textbook that could be read and assimilated (with varying degrees of difficulty) by students of all three disciplines. The text is broken into three main sections: mathematical modeling, “molecular inventories” of the requisite biological systems, and case studies. Despite the extensive mathematical component, A First Course in Systems Biology is not set up like a traditional mathematics textbook. Instead of long proofs and stark equations, concepts are first introduced intuitively before the equations are formalized, and readers are walked through a concrete example. A number of plots and descriptive images accompany the math, ensuring that even a novice number-cruncher can grasp the basics of Boolean network models, stability analysis, and parameter estimation. The middle third of the text covers the most pertinent biological details required for studying gene regulatory systems, protein interaction networks, metabolic pathways, and cell signaling cascades from a systems biology perspective. The book closes with a series of case studies that focus on the proper selection and application of biological models. For example, in Chapter 12, Voit demonstrates the type of experimental data and computational simulation that would be required for modeling several aspects of heart function, from a physiological model of blood flow through the atria and ventricles to one that examines oscillations of calcium signaling. Throughout, Voit provides clear and concise explanations of the most important and basic modeling and biological concepts without resorting to oversimplification, though explanations can sometimes be vague. Further, some topics, particularly that of epigenetics, are disappointingly terse. However, the brevity is always compensated for with extensive references for the curious reader. Indeed, despite the approachability of the text, this book requires great concentration and interaction on the part of the reader. Frequently, Voit urges that “the reader should confirm” the mathematical conclusions stated in the text. Extensive and open-ended exercises are given at the end of each chapter. These problems are not for the faint of heart and often require additional research, computer coding, or essay writing to complete. While A First Course in Systems Biology does not teach students precisely how to code mathematical models or provide protocols for biological experiments, the book does grant students an entry into the proper mindset of systems biologists. The continual emphasis on determining how to choose an appropriate model for a given problem and understanding what type of information disparate models could provide prepares readers well for generating systems biology questions and models of their own.

Fine-Grained Parallel and Distributed Spatial Stochastic Simulation of Biological Reactions

In this article I address the interactions between biological knowledge and ideas about the kinds of entity that are suited to appropriation. I start by arguing that commodification and reductionism are closely linked, and that patenting suits entities that are discrete and isolable, such as those that are the focus of molecular biology. I then turn to the new field of systems biology, which recognizes that traditional reductionist approaches to biology are no longer adequate and attempts to provide a more integrative understanding of biological systems. In doing this, systems biology has to deal with emergent phenomena. But patenting does not suit the dynamic and interactive complexity that is the object of study in systems biology. If systems biology rejects reductionism where does that leave commodification? I examine attempts to commodify predictive computational models in systems biology. I then turn to systems biology's sister discipline, synthetic biology, which deals with emergence by reducing the complexity of biological systems. By factoring out messy contingencies, synthetic biology is, in theory, well suited to commodification. Drawing on both these examples I explore how ideas about appropriation, including open source, are influencing the nature and course of research in biology.

We consider the problem of A-optimal design of experiment under a Bayesian probabilistic model with both categorical and continuous response variables. The utility function of the local design problem is derived by applying Bayesian experimental design framework. We also develop an efficient optimization algorithm to obtain the local optimal design by combining the particle swarm optimization and the blocked coordinate descent methods. In addition, we discuss two different ways of constructing the global optimal design based on the algorithm for local optimal design. Simulation studies are presented to illustrate the efficiency of our approach.

A selection of papers prepared by participants of 7th Young Scientists School SBB’15 is presented in three Supplementary Issues to the journals of BMC series, namely, BMC Genomics, BMC Genetics and BMC Microbiology. SBB’15 took place June 22nd-25th in Novosibirsk, Russia (http://conf.nsc.ru/sbb2015). The series of Young Scientists Schools in Systems Biology and Bioinformatics (SBB) started several years ago, in 2008, as educational workshop associated with the large international conference series on bioinformatics known as BGRS\SB (Bioinformatics of Genome Regulation and Structure\Systems Biology) (http://conf.bionet.nsc.ru/bgrssb2016). After its inauguration year of 1998, BGRS/SBB held biannually at the Institute of Cytology and Genetics SB RAS in Novosibirsk. In a recent decade, it became a prime meeting venue for biologists, computer scientists, physicists, mathematicians and biochemists working in an interdisciplinary field of systems biology and computer genomics. Being the largest system biology meeting series in Russia, BGRS\SB-14 attracted participants from 27 countries. In the past BioMedCentral had published series of special issues based on best materials presented at the conference in BMC Genomics (http://www.biomedcentral.com/bmcgenomics/supplements/15/S12), BMC Evolutionary Biology (http://www.biomedcentral.com/bmcevolbiol/supplements/15/S1), BMC Genetics (http://www.biomedcentral.com/bmcgenet/supplements/16/S1) and BMC Systems biology (http://www.biomedcentral.com/bmcsystbiol/supplements/9/S2). In addition, the protein structure related studies presented by BGRS/SBB participants were collated as special issues in Journal of Bioinformatics and Computational Biology (http://www.worldscientific.com/toc/jbcb/4113/01; http://www.worldscientific.com/toc/jbcb/11/01). [1] In addition, the SBB School series has its own history to be proud of. The first generation of its participants already matured enough to substantially advance in the field. A majority of the participants graduated with PhDs, and now are contributing to the school as the lecturers and to a number of world universities as professors. For some students, the conference became instrumental in obtaining their post-doc or first independent position in academic world. In many cases, the informal atmosphere of the school helped to spark a collaboration that later led to publications in leading journals or the dramatic changes in their science career. The Program Committee of the conference and SBB Schools series includes data analysis, bioinformatics and computational biology professionals from different countries. For the last decade, the SBB conference was co-chaired by Academician Prof. Nikolay A. Kolchanov of the Institute of Cytology and Genetics SB RAS, Russia and Prof. Dr. Ralf Hofestadt of Bielefeld University, Germany. System biology and bioinformatics are rapidly developing and wide fields of science knowledge. For Young Scientists Schools, every year is different as it comprises the topics for the trainings that are the hottest at the time of the event. In 2015, the SBB School concentrated on the modern genomics and an analysis of high-throughput sequencing data. The materials presented in the current special issue provide examples of seamless integration of experimental studies performed in the lab and the computer-assisted inquiries into in the complex patterns of the organization and the functioning of biologic systems at the molecular, cell, tissue, organ, and body levels. The scientific topics discussed in 2015 were presented at the following sections: Next generation sequencing (NGS) and data analysis Evolutionary bioinformatics Systems biology and gene network modeling In SBB’15 Supplements to BMC journals Genomics, we collected the best studies presented at the conference. Due to the general breadth of the field of Systems Biology and Bioinformatic, the manuscripts were accepted into three separate tracks, thus forming three Supplementary Issues, BMC Genomics, BMC Genetics and BMC Microbiology. Below we will describe contents of these issues.