Robust Controllability of Boolean Control Networks via Dynamic Programming.

In this paper, the application of the dynamic programming approach to constrained stochastic control problems with expected value constraints is demonstrated. Specifically, two such problems are analyzed using this approach. The problems analyzed are the problem of minimizing a discounted cost infinite horizon expectation objective subject to an identically structured constraint, and the problem of minimizing a discounted cost infinite horizon minimax objective subject to a discounted expectation constraint. Using the dynamic programming approach, optimality equations, which are the chief contribution of this paper, are obtained for these problems. In particular, the dynamic programming operators for problems with expectation constraints differ significantly from those of standard dynamic programming and problems with worst-case constraints. For the discounted cost infinite horizon cases, existence and uniqueness of solutions to the dynamic programming equations are explicitly shown by using the Banach fixed point theorem to show that the corresponding dynamic programming operators are contractions. The theory developed is illustrated by numerically solving the constrained stochastic control dynamic programming equations derived for simple example problems. The example problems are based on a two-state Markov model that represents an error prone system that is to be maintained.

Optimal Consumption Plans—A Dynamic Programming Approach

This paper presents a novel method to find the profit-maximizing Nash equilibriums in allocating generation amounts with consideration of ramp-rates under competitive market environment. In order to find the Nash equilibriums it is necessary to search all the feasible combinations of generators' outputs which satisfy various constraints. The procedure to eliminate the dominated strategies can be formulated using Bellman's optimality principle of dynamic programming problem and hence the backward or forward search algorithm of dynamic programming can be easily applied. Therefore, the Nash equilibriums are found using dynamic programming method and we found that there exist several Nash equilibriums in the generation allocation problem. Individual generators participate in a game to maximize its profit through competitions and play aa game with bidding strategies of its generation quantities in a spot market. The ramp-rate physically or technically limits generators to increase or decrease outputs in its range and restricts the number of bidding strategies of each generator. We suggest the dynamic programming to find the Nash equilibriums while removing the dominated strategies in each stage (or each time). In the case studies, we analyzed the generation allocation game for a 12-hour multi-stage and compared it with the results of dynamic economic dispatch. Both of the two cases were considered generator's ramp-rate effects.

This research is about the development of a dynamic programming model for solving fuzzy linear programming problems. Initially, fuzzy dynamic linear programming model FDLP is developed. This research revises the established dynamic programming model for solving linear programming problems in a crisp environment. The mentioned approach is upgraded to address the problem in an uncertain environment. Dynamic programming model can either be passing forward or backward. In the proposed approach backward dynamic programming approach is adopted to address the problem. It is then followed by implementing the proposed method on the education system of Pakistan. The education system of Pakistan comprises of the Primary, Middle, Secondary, and Tertiary education stages. The problem is to maximize the efficiency of the education system while achieving the targets with minimum usage of the constrained resources. Likewise the model tries to maximize the enrollment in the Primary, Middle, Secondary and Tertiary educational categories, subject to the total available resources in a fuzzy uncertain environment. The solution proposes that the enrollment can be increased by an amount 9997130, by increasing the enrollment in the Middle and Tertiary educational categories. Thus the proposed method contributes to increase the objective function value by 30%. Moreover, the proposed solutions violate none of the constraints. In other words, the problem of resources allocation in education system is efficiently managed to increase efficiency while remaining in the available constrained resources. The motivation behind using the dynamic programming methodology is that it always possesses a numerical solution, unlike the other approaches having no solution at certain times. The proposed fuzzy model takes into account uncertainty in the linear programming modeling process and is more robust, flexible and practicable.

Controllability and observability of Boolean control networks subject to stuck-at fault

In this article, the controllability of asynchronous Boolean control networks (ABCNs) with time-invariant delay is studied. Using semi-tensor product, Boolean control networks under Harvey’s asynchronous update with time delay are converted into linear representation, and a general formula of control-depending network transition matrices is achieved. Firstly, a necessary and sufficient condition is proposed to verify that only control-depending fixed points of ABCNs can be deterministically controlled. Secondly, respectively for three kinds of controls, the controllability of the non-deterministic logical dynamics is investigated, and formulas are given to show: a) the reachable sets at time s under a group of specified controls; b) the reachable sets at time s under arbitrary controls and c) the probability values from a given initial state to different target states. Moreover, we also discuss how to find the particular control sequence to reach a specified target state with the maximum probability. Examples are shown to illustrate the feasibility of the proposed scheme.

This research focuses on designing a min–max robust control based on a neural dynamic programming approach using a class of continuous differential neural networks (DNNs). The proposed controller solves the robust optimization of a proposed cost function that depends on the trajectories of a system with an uncertain mathematical model satisfying a class of non-linear perturbed systems. The dynamic programming min–max formulation enables robust control concerning bounded modelling uncertainties and disturbances. The Hamilton–Jacobi–Bellman (HJB) equation’s value function, approximated by a DNN, permits to estimate the closed-loop formulation of the controller. The controller design is based on an estimated state trajectory with the worst possible uncertainties/perturbations that provide the degree of robustness using the proposed controller. The class of learning laws for the time-varying weights in the DNN is produced by studying the HJB partial differential equation. The controller uses the solution of the obtained learning laws and a time-varying Riccati equation. A recurrent algorithm based on the Kiefer–Wolfowitz method leads to adjusting the initial conditions for the weights to satisfy the final condition of the given cost function. The robust control suggested in this work is evaluated using a numerical example confirming the optimizing solution based on the DNN approximate for Bellman’s value function.

In this article, the controllability of asynchronous Boolean multiplex control networks (ABMCNs) is studied. First, the model of Boolean multiplex control networks under Harvey' asynchronous update is presented. By means of semi-tensor product approach, the logical dynamics is converted into linear representation, and a generalized formula of control-depending network transition matrices is achieved. Second, a necessary and sufficient condition is proposed to verify that only control-depending fixed points of ABMCNs can be controlled with probability one. Third, using two types of controls, the controllability of system is studied and formulae are given to show: (a) when an initial state is given, the reachable set at time s under a group of specified controls; (b) the reachable set at time s under arbitrary controls; (c) the specific probability values from a given initial state to destination states. Based on the above formulae, an algorithm to calculate overall reachable states from a specified initial state is presented. Moreover, we also discuss an approach to find the particular control sequence which steers the system between two states with maximum probability. Examples are shown to illustrate the feasibility of the proposed scheme.

A EROSPACE engineering applications greatly stimulated the development of optimal control theory during the 1950s and 1960s, where the objective was to drive the system states in such a way that some defined cost was minimized. This turned out to have very useful applications in the design of regulators (where some steady state is to be maintained) and in tracking control strategies (where some predetermined state trajectory is to be followed). Among such applications was the problem of optimal flight trajectories for aircraft and space vehicles. Linear optimal control theory in particular has been very well documented and widely applied, where the plant that is controlled is assumed linear and the feedback controller is constrained to be linear with respect to its input. However, the availability of powerful low-cost microprocessors has spurred great advantages in the theory and applications of nonlinear control. The competitive era of rapid technological change, particularly in aerospace exploration, now demands stringent accuracy and cost requirements in nonlinear control systems. This has motivated the rapid development of nonlinear control theory for application to challenging, complex, dynamical real-world problems, particularly those that bear major practical significance in aerospace, marine, and defense industries. Infinite-time horizon nonlinear optimal control (ITHNOC) presents a viable option for synthesizing stabilizing controllers for nonlinear systems by making a state-input tradeoff, where the objective is to minimize the cost given by a performance index. The original theory of nonlinear optimal control dates from the 1960s. Various theoretical and practical aspects of the problem have been addressed in the literature over the decades since. In particular, the continuous-time nonlinear deterministic optimal control problem associated with autonomous (time-invariant) nonlinear regulator systems that are affine (linear) in the controls has been studied by many authors. The long-established theory of optimal control offers quite mature and well-documented techniques for solving this control-affine nonlinear optimization problem, based on dynamic programming or calculus of variations, but their application is generally a very tedious task. Bellman’s dynamic programming approach reduces to solving a nonlinear first-order partial differential equation (PDE), expressed by the Hamilton–Jacobi–Bellman (HJB) equation. The solution to the HJB equation gives the optimal performance/cost value (or storage) function and determines an optimal control in feedback form under some smoothness assumptions. Alternatively, in the classical calculus of variations, optimal control problems can be characterized locally in terms of the Hamiltonian dynamics arising from Pontryagin’s minimum principle. These are the characteristic equations of the HJB PDE, which result in a nonlinear, constrained two-point boundary value problem (TPBVP) that, in general, can only be solved by successive approximation of the optimal control input using iterative numerical techniques for each set of initial conditions. Numerically, even though the nonlinear TPBVP is somewhat easier to solve than the HJBPDE, control signals can only be determined offline and are thus best suited for feedforward control of plants for which the state trajectories are known a priori. Therefore, contrary to the dynamic programming approach, the resultant control law is not generally in feedback form. Open-loop control, however, is sensitive to random disturbances and requires that the initial state be on the optimal trajectory. In contrast, nonlinear optimal feedback has inherent robustness properties (inherent in the sense that it is obtained by ignoring uncertainty and disturbances). The potential difficulty with the HJB approach is that no efficient algorithm is available to solve the PDE when it is nonlinear and the problem dimension is high, making it impossible to derive exact expressions for optimal controls for most nontrivial problems of interest. The optimal can only be computed in special cases, such as linear dynamics and quadratic cost, or very low-dimensional systems. In particular, if the plant is linear time invariant (LTI) and the (infinite-time) performance index is quadratic, then the corresponding HJB equation for this infamous linear-quadratic regulator (LQR) problem reduces to an algebraic Riccati equation (ARE). Contrary to the well-developed and widely applied theory and computational tools for theRiccati equation (for example, see [1]), theHJB equation is difficult, if not impossible, to solve for most practical applications. The exact solution for the optimal control policies is very complex

For Meyer type optimal control problems the principle of optimality states that every subarc of an optimal solution that extends to the final time furnishes again a solution to the same optimal control problem, only with different initial conditions. Through a simple counter example it is shown in this paper that this principle is not satisfied, in general, for solutions obtained from a standard collocation approach. Introduction The principle of optimality [1] is one of the most intuitive and yet most fundamental theorems in the theory of optimal control. Its basic statement is that for Meyer type optimal control problems every subarc of an optimal solution that extends to the final time furnishes again a solution to the same optimal control problem, only with different initial conditions. This theorem has significant implications. For example, it is the starting point for the derivation of the Hamilton-Jacobi-Bellman equations [2] and it is the underlying principle in Bellman's dynamic programming approach [1], [3]. The present paper shows that the principle of optimality is not satisfied, in general, for solutions obtained from a standard collocation approach[4]. For a simple example whose collocation solution can be generated analytically it is shown that further improvements are possible along an optimal collocation solution if the design parameters along a number of leading nodes are frozen and the remainder of the trajectory is re-optimized. I. Continuous Optimal Control Problem In this section of the paper we introduce a simple class of optimal control problems without state and control constraints. In Section 1.2 we present the associated necessary conditions for optimality [2], [5]. For ease of notation we assume that the right-hand side of the 'Supervising Engineer, Member AIAA, t Senior Engineer, Member AIAA, *17 Research Drive, Hampton, VA 23666. ^Graduate Student, Member AIAA, Department, 304 Town Engineering, Ames, IA 50011 equations of motion does not depend explicitly on time t. This does not represent a loss of generality, as it is always possible to transform explicit time dependence away through the introduction of an additional, timelike state variable. 1. Problem Formulation Let us consider the following class of optimal control problems: min «e(Pwc[*o,*/]), (1) subject to the conditions ±(t) = /(*(*),«(*)), tf.(z(*oMo) = 0, (2)

On the ensemble controllability of Boolean control networks using STP method

The paper deals with problems on the stability and stabilization of Boolean networks with respect to a given part of variables characterizing the Boolean network. Using the semi-tensor product of matrices and the matrix expression of logic, the dynamics of a Boolean (control) network can be converted to discrete time linear (bilinear) dynamics called the algebraic form of the Boolean (control) network. The two main results are as follows: (i) From the algebraic form of a Boolean (control) network, two partial stability definitions are introduced. One is partial stability with respect to another variable set, and the other is partial stability with respect to arbitrary initial states. (ii) Based on the algebraic form, necessary and sufficient conditions for the stability of global partial states with respect to the variable set and arbitrary initial states are obtained. Examples are provided to illustrate the effectiveness of our results.

Robust controllability assessment and optimal actuator placement in dynamic networks

Controllability and observability of singular Boolean control networks are proposed in this paper from its algebraic form via semi-tensor product. First, the input-state incidence matrix of singular Boolean control networks is presented based on which the generalized input-state incidence matrix is also defined. Second, a necessary and sufficient condition for the controllability of singular Boolean control networks is given, as well as an algorithm to design controls which drive a state to a reachable state. At last, similar to Boolean control networks, a sufficient condition of observability of singular Boolean control networks is obtained.

SummaryThis paper considers optimal consensus control problem for unknown nonlinear multiagent systems (MASs) subjected to control constraints by utilizing event‐triggered adaptive dynamic programming (ETADP) technique. To deal with the control constraints, we introduce nonquadratic energy consumption functions into performance indices and formulate the Hamilton‐Jacobi‐Bellman (HJB) equations. Then, based on the Bellman's optimality principle, constrained optimal consensus control policies are designed from the HJB equations. In order to implement the ETADP algorithm, the critic networks and action networks are developed to approximate the value functions and consensus control policies respectively based on the measurable system data. Under the event‐triggered control framework, the weights of the critic networks and action networks are only updated at the triggering instants which are decided by the designed adaptive triggered conditions. The Lyapunov method is used to prove that the local neighbor consensus errors and the weight estimation errors of the critic networks and action networks are ultimately bounded. Finally, a numerical example is provided to show the effectiveness of the proposed ETADP method.