On approximate stochastic control in genetic regulatory networks

Optimal control simulations have shown that both musculoskeletal dynamics and physiological noise are important determinants of movement. However, due to the limited efficiency of available computational tools, deterministic simulations of movement focus on accurately modelling the musculoskeletal system while neglecting physiological noise, and stochastic simulations account for noise while simplifying the dynamics. We took advantage of recent approaches where stochastic optimal control problems are approximated using deterministic optimal control problems, which can be solved efficiently using direct collocation. We were thus able to extend predictions of stochastic optimal control as a theory of motor coordination to include muscle coordination and movement patterns emerging from non-linear musculoskeletal dynamics. In stochastic optimal control simulations of human standing balance, we demonstrated that the inclusion of muscle dynamics can predict muscle co-contraction as minimal effort strategy that complements sensorimotor feedback control in the presence of sensory noise. In simulations of reaching, we demonstrated that nonlinear multi-segment musculoskeletal dynamics enables complex perturbed and unperturbed reach trajectories under a variety of task conditions to be predicted. In both behaviors, we demonstrated how interactions between task constraint, sensory noise, and the intrinsic properties of muscle influence optimal muscle coordination patterns, including muscle co-contraction, and the resulting movement trajectories. Our approach enables a true minimum effort solution to be identified as task constraints, such as movement accuracy, can be explicitly imposed, rather than being approximated using penalty terms in the cost function. Our approximate stochastic optimal control framework predicts complex features, not captured by previous simulation approaches, providing a generalizable and valuable tool to study how musculoskeletal dynamics and physiological noise may alter neural control of movement in both healthy and pathological movements.

Recent advances of experimental techniques in biology have led to the production of enormous amounts of data on the dynamics of genetic regulatory networks. In this paper, we present an approach for the identification of PieceWise-Affine (PWA) models of genetic regulatory networks from experimental data, focusing on the reconstruction of switching thresholds associated with regulatory interactions. In particular, our method takes into account geometric constraints specific to models of genetic regulatory networks. We show the feasibility of our approach by the reconstruction of switching thresholds in a PWA model of the carbon starvation response in the bacterium Escherichia coli.

A stochastic minimax optimal time-delay state feedback control strategy for uncertain quasi-integrable Hamiltonian systems is proposed. First, a stochastic optimal state feedback control problem of uncertain quasi-integrable Hamiltonian system with time delay in feedback control subjected to Gaussian white noise is formulated. Then, the time-delayed state feedback control forces are approximated by the control forces without time delay and the original problem is converted into a stochastic optimal state feedback control problem of an uncertain system without time delay. After that, by following a procedure based on the stochastic averaging method and stochastic differential game, the worst-case disturbances and the optimal controls are obtained from solving a Hamiltonian–Jacobi–Isaacs (HJI) equation. As an example, the stochastic minimax optimal state feedback control of a Duffing oscillator with parametric disturbances and time-delayed feedback control is worked out in detail to illustrate the procedure and effectiveness of the proposed control strategy.

Innovative procedures for the stochastic optimal time-delay control and stabilization are proposed for a quasi-integrable Hamiltonian system subject to Gaussian white noises. First, the problem of stochastic optimal control with time delay is formulated. Then, the problem is converted into a stochastic optimal control without time delay, and the converted control problems are then solved by applying the stochastic averaging method and the stochastic dynamical programming principle. The optimal time-delay stabilization of quasi-integrable Hamiltonian systems is formulated as an ergodic control with a cost function determined by minimizing the largest Lyapunov exponent of the controlled system. As an example, a two-degree-of-freedom quasi-integrable Hamiltonian system with time delay in feedback control forces is investigated in detail to illustrate the procedures and their effectiveness.KeywordsTime-delay feedback controlStochastic averaging methodStochastic optimal controlFeedback stabilization

The stochastic optimal nonlinear control of coupled adjacent building structures is studied based on the stochastic dynamical programming principle and the stochastic averaging method. The coupled structures with control devices under random seismic excitation are first condensed to form a reduced-order structural model for the control analysis. The stochastic averaging method is applied to the reduced model to yield stochastic differential equations for structural modal energies as controlled diffusion processes. Then a dynamical programming equation for the energy processes is established based on the stochastic dynamical programming principle, and solved to determine the optimal nonlinear control law. The seismic response mitigation of the coupled structures is achieved through the structural energy control and the dimension of the optimal control problem is reduced. The seismic excitation spectrum is taken into account according to the stochastic dynamical programming principle. Finally, the nonlinear controlled structural response is predicted by using the stochastic averaging method and compared with the uncontrolled structural response to evaluate the control efficacy. Numerical results are given to demonstrate the response mitigation capabilities of the proposed stochastic optimal control method for coupled adjacent building structures.

A minimax stochastic optimal semi-active control strategy for stochastically excited quasi-integrable Hamiltonian systems with parametric uncertainty by using magneto-rheological (MR) dampers is proposed. Firstly, the control problem is formulated as an n-degree-of-freedom (DOF) controlled, uncertain quasi-integrable Hamiltonian system and the control forces produced by MR dampers are split into the passive part and the semi-active part. Then the passive part is incorporated into the uncontrolled system. After that, the stochastic optimal semi-active control problem is solved by applying the minimax stochastic optimal control strategy based on the stochastic averaging method and stochastic differential game. The worst-case disturbances and the optimal controls are obtained by the minimax dynamical programming equation with the constraints of disturbance bounds and MR damper dynamics. Finally, the system response and the control performance are evaluated by using Monte Carlo simulation. An example of a two-DOF system with coupling damping and parametric uncertainty under Gaussian white noise excitations is worked out in detail to illustrate the procedure and effectiveness of the proposed control strategy, which is also compared with the clipped linear-quadratic-Gaussian control strategy to show the advantages.

In this paper, we use the solutions of forward-backward stochastic differential equations with random jumps to get the optimal controls for forward-backward stochastic linear quadratic optimal control problem and the Nash equilibrium point for nonzero sum differential games problem.

This article examines an economic growth model that expresses the interaction between production, technology stock, and research and development (R&D) investments. The goal of this study is to maximize production. Considering the presence of Gaussian white noises, this model is reformulated as a stochastic optimal control problem, where the R&D investment rate is defined as the control input. We aim to explore the efficiency of Kalman filtering approaches for solving this problem. Here, the extended Kalman filter (EKF) and unscented Kalman filter (UKF) are applied for state estimation. The state equation linearization is made in the EKF, while the unscented transform is taken in the UKF for generating a set of sigma points. These approaches aim to estimate the state dynamics from different perspectives. With these state estimates, two different computational algorithms are proposed, the EKF for state-control (EKF4SC) and UKF for state-control (UKF4SC) algorithms. The optimal control policy is designed to minimize the cost function. For illustration, the model's parameters are considered in the simulation experiment. The simulation results showed that the UKF has higher accuracy in the state estimation with the smallest mean squares of error compared with the EKF. Moreover, the optimal control policy based on the state estimate generated from the UKF could optimize the cost function of the problem. Hence, the results show the efficiency of the algorithms proposed. In conclusion, the application of the Kalman filtering algorithms to the economic growth model is presented. The significance of this study is to provide a stochastic optimal control model for the economic growth problem and to suggest an efficient computational approach for solving this stochastic optimal control problem.

OF THE DISSERTATION Optimal Control in Gene Mutation by Juanyi Yu Doctor of Philosophy in Electrical Engineering Washington University in St. Louis, 2011 Research Advisor: Professor Tzyh-Jong Tarn, Professor Jr-Shin Li Gene mutations are the radical causes of many diseases, including inheritance diseases and cancers. Current medical treatments usually focus on changing the concentrations of related chemicals or mRNAs at the cellular level to stop protein productions or cell duplications, which can only control the diseases under certain circumstances but cannot cure them. Little research work has been done at the molecular level, the fundamental of inheritance, to search possible ways to cure those severe diseases. In this dissertation, we propose a molecular level control system view of the gene mutations in DNA replication from the finite field concept. By treating DNA sequences as state variables, chemical mutagens and radiation as control inputs, one cell cycle as a step increment, and the measurements of the resulting DNA sequence as outputs, we derive system equations for both deterministic and stochastic discrete-time, finite-state systems of different scales. Defining the cost function as a summation of the costs of applying mutagens and the off-trajectory penalty, we solve the deterministic and stochastic optimal control problems by dynamic programming algorithm. In

Systems of ordinary differential equations (ODE) of special form are considered in this paper. These systems appear in various models of genetic regulatory networks and telecommunication networks. The model of a genetic network with a chaotic attractor of dimension four is constructed. Geometrical considerations are used to study the properties of the systems. Computation of Lyapunov exponents is used to prove the existence of chaotic attractors. Problems of control and management of such regulatory systems are challenging issues in the theory of genetic networks.

Stochastic bifurcation characteristics and optimal control of giant magnetostrictive film (GMF)-shape memory alloy (SMA) composite plate subjected to in-plane stochastic excitation were studied in this paper. Nonlinear difference item was introduced to interpret the hysteretic phenomena of both GMF and SMA, and then the nonlinear dynamic model of GMF-SMA composite plate subjected to in-plane stochastic excitation was developed. The stochastic stability of the system was analyzed, and the condition of stochastic Hopf bifurcation was discussed. The reliability function of the system was solved from backward Kolmogorov equation, and then the probability density of the first-passage time was obtained. Finally, the stochastic optimal control strategy was obtained in stochastic dynamic programming method. Numerical simulation shows that the stability of the trivial solution varies with bifurcation parameters, and stochastic Hopf bifurcation appears in the process; the reliability of the system is improved by stochastic optimal control, and the first-passage time is delayed. GMF-SMA composite plate combines the advantages of both GMF and SMA, and can reduce vibration through passive control and active control effectively. The results of this paper are helpful to application of GMF-SMA composite plate in engineering fields.

A computational approach is taken to solve the optimal nonlinear stochastic control problem. The approach is to systematically solve the stochastic dynamic programming equations forward in time, using a nested stochastic approximation technique. Although computationally intensive, this provides a straightforward numerical solution for this class of problems and provides an alternative to the usual dimensionality problem associated with solving the dynamic programming equations backward in time. It is shown that the cost degrades monotonically as the complexity of the algorithm is reduced. This provides a strategy for suboptimal control with clear performance/computation tradeoffs. A numerical study focusing on a generic optimal stochastic adaptive control example is included to demonstrate the feasibility of the method.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

The perhaps most important signaling network in living cells is constituted by the interactions of proteins with the genome – the regulatory genetic network of the cell. From a system-level point of view, the various interactions and control loops, which form a genetic network, represent the basis upon which the vast complexity and flexibility of life processes emerges. Here we provide a review over some efforts towards gaining a quantitative understanding of regulatory genetic networks by means of large scale computational models. After a brief description of the biological principles of gene regulation, we summarize recent advances in massive gene-expression measurements by DNA-microarrays, which form the to date most powerful data basis for models of genetic networks. One class of models such as reaction-diffusion networks and nonlinear dynamical descriptions are biased towards using explicit molecular biological knowledge. A second class, centered around machine learning approaches like neural networks and Bayesian networks, adopts a more data-driven approach and thereby makes massive use of the novel gene expression measurement techniques.

Stochastic decomposition applied to large-scale hydro valleys management

Integrating product and process design with optimal control: A case study of solvent recycling