Optimal control of probabilistic Boolean control networks: A scalable infinite horizon approach

A genetic regulatory network (GRN) is a dynamic system to describe interactions among a large number of different substances in a living biological cell. There have been numerous attempts to model the dynamical behaviour of genetic regulatory networks. Boolean models are a class of discrete models for modelling genetic regulatory networks, which are conditioned on the premise that genes interact with each other through Boolean logic. As mentioned previously, Boolean models include Boolean Networks, Instantaneously Random Probabilistic Boolean Networks and Context-Sensitive Boolean Networks. There are a large number of issues surrounding Boolean models, including Boolean Networks and Probabilistic Boolean Networks. The model inference is the procedure of determining whether the modelled networks are consistent with the given data sample, and choosing one model, from among many, that makes more sense for the data. This paper has reviewed the research into modelling gene regulatory networks, especially upon Boolean models.

An integer programming approach to optimal control problems in context-sensitive probabilistic Boolean networks

State-flipped control design for the stabilization of probabilistic Boolean control networks.

Intervention in a gene regulatory network is used to help it avoid undesirable states, such as those associated with a disease. Several types of intervention have been studied in the framework of a probabilistic Boolean network (PBN), which is essentially a finite collection of Boolean networks in which at any discrete time point the gene state vector transitions according to the rules of one of the constituent networks. For an instantaneously random PBN, the governing Boolean network is randomly chosen at each time point. For a context-sensitive PBN, the governing Boolean network remains fixed for an interval of time until a binary random variable determines a switch. The theory of automatic control has been previously applied to find optimal strategies for manipulating external (control) variables that affect the transition probabilities of an instantaneously random PBN to desirably affect its dynamic evolution over a finite time horizon. This paper extends the methods of external control to context-sensitive PBNs. This paper treats intervention via external control variables in context-sensitive PBNs by extending the results for instantaneously random PBNs in several directions. First, and most importantly, whereas an instantaneously random PBN yields a Markov chain whose state space is composed of gene vectors, each state of the Markov chain corresponding to a context-sensitive PBN is composed of a pair, the current gene vector occupied by the network and the current constituent Boolean network. Second, the analysis is applied to PBNs with perturbation, meaning that random gene perturbation is permitted at each instant with some probability. Third, the (mathematical) influence of genes within the network is used to choose the particular gene with which to intervene. Lastly, PBNs are designed from data using a recently proposed inference procedure that takes steady-state considerations into account. The results are applied to a context-sensitive PBN derived from gene-expression data collected in a study of metastatic melanoma, the intent being to devise a control strategy that reduces the WNT5A gene's action in affecting biological regulation, since the available data suggest that disruption of this influence could reduce the chance of a melanoma metastasizing.

Driven by the desire to understand genomic functions through the interactions among genes and gene products, the research in gene regulatory networks has become a heated area in genomic signal processing. Among the most studied mathematical models are Boolean networks and probabilistic Boolean networks, which are rule-based dynamic systems. This tutorial provides an introduction to the essential concepts of these two Boolean models, and presents the up-to-date analysis and simulation methods developed for them. In the Analysis section, we will show that Boolean models are Markov chains, based on which we present a Markovian steady-state analysis on attractors, and also reveal the relationship between probabilistic Boolean networks and dynamic Bayesian networks (another popular genetic network model), again via Markov analysis; we dedicate the last subsection to structural analysis, which opens a door to other topics such as network control. The Simulation section will start from the basic tasks of creating state transition diagrams and finding attractors, proceed to the simulation of network dynamics and obtaining the steady-state distributions, and finally come to an algorithm of generating artificial Boolean networks with prescribed attractors. The contents are arranged in a roughly logical order, such that the Markov chain analysis lays the basis for the most part of Analysis section, and also prepares the readers to the topics in Simulation section.

Control of gene regulatory networks is one of the fundamental topics in systems biology. In the last decade, control theory of Boolean networks (BNs), which is well known as a model of gene regulatory networks, has been widely studied. In this review paper, our previously proposed methods on optimal control of probabilistic Boolean networks (PBNs) are introduced. First, the outline of PBNs is explained. Next, an optimal control method using polynomial optimization is explained. The finite-time optimal control problem is reduced to a polynomial optimization problem. Furthermore, another finite-time optimal control problem, which can be reduced to an integer programming problem, is also explained.

Intervention in a gene regulatory network is used to avoid undesirable states, such as those associated with a disease. Several types of intervention have been studied in the framework of a probabilistic Boolean network (PBN), which is a collection of Boolean networks in which the gene state vector transitions according to the rules of one of the constituent networks and where network choice is governed by a selection distribution. The theory of automatic control has been applied to find optimal strategies for manipulating external control variables that affect the transition probabilities to desirably affect dynamic evolution over a finite time horizon. In this paper we treat a case in which we lack the governing probability structure for Boolean network selection, so we simply have a family of Boolean networks, but where these networks possess a common attractor structure. This corresponds to the situation in which network construction is treated as an ill-posed inverse problem in which there are many Boolean networks created from the data under the constraint that they all possess attractor structures matching the data states, which are assumed to arise from sampling the steady state of the real biological network. Given a family of Boolean networks possessing a common attractor structure composed of singleton attractors, a control algorithm is derived by minimizing a composite finite-horizon cost function that is a weighted average over all the individual networks, the idea being that we desire a control policy that on average suits the networks because these are viewed as equivalent relative to the data. The weighting for each network at any time point is taken to be proportional to the instantaneous estimated probability of that network being the underlying network governing the state transition. The results are applied to a family of Boolean networks derived from gene-expression data collected in a study of metastatic melanoma, the intent being to devise a control strategy that reduces the WNT5A gene's action in affecting biological regulation

Reinforcement learning-based optimal false data injection attack against control signals in probabilistic Boolean control networks

In this paper, we propose a novel random forest (RF) Q-learning hybrid with experience replay for feedback stabilization of probabilistic Boolean control networks (PBCNs). In particular, by resorting to a model-free reinforcement learning (RL) framework, we present a random forest Q-learning (QLRF) algorithm to design optimal state feedback controllers, thereby stabilizing PBCNs to a given equilibrium point. In reference to better the process of learning the Q-table by replacing it with a function approximator, we substitute the existent neural network (NN) architecture by a RF. We provide insights on the overall computational complexity between the two ways of solving the same problem, proving RF better than its NN counterparts for such applications. The simulations performed on some of the standard examples in the literature demonstrates the effectiveness of the proposed idea.

Data mining approach for improving the optimal control of HVAC systems: An event-driven strategy

BackgroundA salient purpose for studying gene regulatory networks is to derive intervention strategies, the goals being to identify potential drug targets and design gene-based therapeutic intervention. Optimal stochastic control based on the transition probability matrix of the underlying Markov chain has been studied extensively for probabilistic Boolean networks. Optimization is based on minimization of a cost function and a key goal of control is to reduce the steady-state probability mass of undesirable network states. Owing to computational complexity, it is difficult to apply optimal control for large networks.ResultsIn this paper, we propose three new greedy stationary control policies by directly investigating the effects on the network long-run behavior. Similar to the recently proposed mean-first-passage-time (MFPT) control policy, these policies do not depend on minimization of a cost function and avoid the computational burden of dynamic programming. They can be used to design stationary control policies that avoid the need for a user-defined cost function because they are based directly on long-run network behavior; they can be used as an alternative to dynamic programming algorithms when the latter are computationally prohibitive; and they can be used to predict the best control gene with reduced computational complexity, even when one is employing dynamic programming to derive the final control policy. We compare the performance of these three greedy control policies and the MFPT policy using randomly generated probabilistic Boolean networks and give a preliminary example for intervening in a mammalian cell cycle network.ConclusionThe newly proposed control policies have better performance in general than the MFPT policy and, as indicated by the results on the mammalian cell cycle network, they can potentially serve as future gene therapeutic intervention strategies.

This paper investigates the finite-time observability of probabilistic Boolean multiplex control networks (PBMCNs). Firstly, the finite-time observability of the PBMCNs is converted into the set reachability issue according to the parallel interconnection technique (a minor modification of the weighted pair graph method in the literature). Secondly, the necessary and sufficient condition for the finite-time observability of PBMCNs is presented based on the set reachability. Finally, the main conclusions are substantiated by providing illustrative examples.

The construction and control of genetic regulatory networks using gene expression data is an important research topic in bioinformatics. Probabilistic Boolean Networks (PBNs) have been served as an effective tool for this purpose. However, PBNs are difficult to be used in practice when the number of genes is large because of the huge computational cost. In this paper, we propose a simplified multivariate Markov model for approximating a PBN. The new model can preserve the strength of PBNs and at the same time reduce the complexity of the network and therefore the computational cost. We then present an optimal control model with hard constraints for the purpose of control/intervention of a genetic regulatory network. Numerical experimental examples based on the yeast data are then given to demonstrate the effectiveness of our proposed model and control policy.

Different cross-sectional and clinical research studies investigated that chronic HBV infected individuals’ co-epidemic with COVID-19 infection will have more complicated liver infection than HBV infected individuals in the absence of COVID-19 infection. The main objective of this study is to investigate the optimal impacts of four time dependent control strategies on the HBV and COVID-19 co-epidemic transmission using compartmental modeling approach. The qualitative analyses of the model investigated the model solutions non-negativity and boundedness, calculated all the models effective reproduction numbers by applying the next generation operator approach, computed all the models disease-free equilibrium point (s) and endemic equilibrium point (s) and proved their local stability, shown the phenomenon of backward bifurcation by applying the Center Manifold criteria. By applied the Pontryagin’s Maximum principle, the study re-formulated and analyzed the co-epidemic model optimal control problem by incorporating four time dependent controlling variables. The study also carried out numerical simulations to verify the model qualitative results and to investigate the optimal impacts of the proposed optimal control strategies. The main finding of the study reveals that implementation of protections, COVID-19 vaccine, and treatment strategies simultaneously is the most effective optimal control strategy to tackle the HBV and COVID-19 co-epidemic spreading in the community.

Boolean networks are well-known as one of the models of biological networks such as gene regulatory networks. In this paper, we propose a solving method of the optimal control problem of context-sensitive probabilistic Boolean networks (CS-PBNs), which are one of the extended models of Boolean networks. In the existing solving methods, it is necessary to compute state transition diagrams with 2n nodes for a given CS-PBN with n states. So the existing methods cannot be applied to large-scale networks. To avoid the computation of state transition diagrams, an integer programming-based approach is proposed. In the proposed method, a CS-PBN is transformed into a linear system with binary variables, and the optimal control problem is reduced to an integer linear programming problem, which can be computed relatively easier than the existing methods using state transition diagrams.