Open-Loop Decentralized LQ Control Problem With Multiplicative Noise

Duality between estimation and optimal control is a problem of rich historical significance. The first duality principle appears in the seminal paper of Kalman-Bucy, where the problem of minimum variance estimation is shown to be dual to a linear quadratic (LQ) optimal control problem. Duality offers a constructive proof technique to derive the Kalman filter equation from the optimal control solution.This paper generalizes the classical duality result of Kalman-Bucy to the nonlinear filter: The state evolves as a continuous-time Markov process and the observation is a nonlinear function of state corrupted by an additive Gaussian noise. A dual process is introduced as a backward stochastic differential equation (BSDE). The process is used to transform the problem of minimum variance estimation into an optimal control problem. Its solution is obtained from an application of the maximum principle, and subsequently used to derive the equation of the nonlinear filter. The classical duality result of Kalman-Bucy is shown to be a special case.

This paper focuses on addressing the linear quadratic (LQ) optimal control problem on an infinite time horizon for stochastic systems controlled by impulses. No constraint regarding the sign of the quadratic functional is applied. That is why our first concern is to find conditions which guarantee that the considered optimal control problem is well posed. Then, when the optimal control problem is well posed, it is natural to look for conditions which guarantee the attainability of the optimal control problem that is being evaluated. The main tool involved in the solution of the problems stated before is a backward jump matrix linear differential equation (BJMLDE) with a Riccati-type jumping operator. This is formulated using the matrix coefficients of the controlled system and the weight matrices of the performance criterion. We show that the well posedness of the optimal control problem under investigation is guaranteed by the existence of the maximal and bounded solution of the associated BJMLDE with a Riccati-type jumping operator. Further, we show that when the associated BJMLDE with a Riccati-type jumping operator has a maximal solution which satisfies a suitable sign condition, then the optimal control problem is attainable if and only if it has an optimal control in a state feedback form, or if and only if the maximal solution of the BJMLDE with a Riccati-type jumping operator is a stabilizing solution. In order to make the paper more self-contained, we present a set of conditions that correspond to the existence of the maximal solution of the BJMLDE satisfying the desired sign condition.

In this note, we consider the finite-horizon quadratic optimal control problem of discrete-time Markovian jump linear systems driven by a wide sense white noise sequence. We assume that the output variable and the jump parameters are available to the controller. It is desired to design a dynamic Markovian jump controller such that the closed-loop system minimizes the quadratic functional cost of the system over a finite horizon period of time. As in the case with no jumps, we show that an optimal controller can be obtained from two coupled Riccati difference equations, one associated to the optimal control problem when the state variable is available, and the other one associated to the optimal filtering problem. This is a principle of separation for the finite horizon quadratic optimal control problem for discrete-time Markovian jump linear systems. When there is only one mode of operation our results coincide with the traditional separation principle for the linear quadratic Gaussian control of discrete-time linear systems.

Based on continuous Hopfield neural network (CHNN), a new alternative is developed for solving linear quadratic (LQ) optimal control problem of discrete-time systems. In this method, the LQ performance index is transformed into the energy function of CHNN, and the control sequence into the output vector of the neurons of CHNN. As a result, solving LQ dynamic optimization problem is equivalent to operating associated CHNN from its initial state to the terminal state. The stable output vector of CHNN represents the optimal control sequence. Because CHNN works in parallel and is of real-time characteristic, the present method is easier to satisfy the requirement of real-time control and will be promising in application.

In this paper, we consider optimal control problems of stochastic systems with delay. The control domains are not necessarily convex and the diffusion coefficients explicitly depend on control variables. The delay term of the state is regarded as a control process and then the Ekeland's variational principle is applied to obtain a general maximum principle for the optimal control problems. As applications, we use the theoretical results to a delayed stochastic linear quadratic (LQ) optimal control problem with a non-convex control domain.

In this article, we consider a linear-quadratic optimal control problem (LQ problem) for a controlled linear stochastic differential equation driven by a multidimensional Browinan motion and a Poisson random martingale measure in the general case, where the coefficients are allowed to be predictable processes or random matrices. By the duality technique, the dual characterization of the optimal control is derived by the optimality system (so-called stochastic Hamilton system), which turns out to be a linear fully coupled forward-backward stochastic differential equation with jumps. Using a decoupling technique, the connection between the stochastic Hamilton system and the associated Riccati equation is established. As a result, the state feedback representation is obtained for the optimal control. As the coefficients for the LQ problem are random, here, the associated Riccati equation is a highly nonlinear backward stochastic differential equation (BSDE) with jumps, where the generator depends on the unknown variables K, L, and H in a quadratic way (see (5.9) herein). For the case where the generator is bounded and is linearly dependent on the unknown martingale terms L and H, the existence and uniqueness of the solution for the associated Riccati equation are established by Bellman's principle of quasi-linearization.

ABSTRACTA finite horizon linear quadratic (LQ) optimal control problem is studied for a class of discrete-time linear fractional systems (LFSs) affected by multiplicative, independent random perturbations. Based on the dynamic programming technique, two methods are proposed for solving this problem. The first one seems to be new and uses a linear, expanded-state model of the LFS. The LQ optimal control problem reduces to a similar one for stochastic linear systems and the solution is obtained by solving Riccati equations. The second method appeals to the principle of optimality and provides an algorithm for the computation of the optimal control and cost by using directly the fractional system. As expected, in both cases, the optimal control is a linear function in the state and can be computed by a computer program. A numerical example and comparative simulations of the optimal trajectory prove the effectiveness of the two methods. Some other simulations are obtained for different values of the fractional order.

This article considers the decentralized control for networked control systems (NCSs) with asymmetric information. In this NCSs model, the controller 2 (C2) shares its observations and part of its historical control inputs with the controller 1 (C1), whereas C2 cannot obtain the information of C1 due to network constraints. Under the linear control strategies assumption, we present the optimal estimators for C1 and C2 respectively based on asymmetric observations. Since the information for C1 and C2 are asymmetric, the estimation error covariance (EEC) is coupled with the controller which means that the classical separation principle fails. By applying the Pontryagin's maximum principle, we obtain a solution to the forward and backward stochastic difference equations. Based on this solution, we derive the optimal controllers to minimize a quadratic cost function. Combining the linear optimal controllers with the EEC, the controller C1 is decoupled from the EEC. It should be emphasized that the control gain is dependent on the estimation gain. What is more, the estimation gain satisfies the forward Riccati equation and the control gain satisfies the backward Riccati equation which makes the problem more challenging. We propose iterative solutions to the Riccati equations and give a suboptimal solution to the optimal decentralized control problem.

The decentralized linear–quadratic–Gaussian (LQG) control problem for networked control systems (NCSs) with asymmetric information is investigated, where controller 1 shares its historical information with controller 2, and not vice versa. The asymmetry of the information structure leads to the coupling between controller 2 and estimator 1, and hence the classical separation principle fails. Through the assumption of linear control strategy, the coupling between controller 2 and estimator 1 (CCE) is decoupled, but the estimation gain is still coupled with the control gain. It is noted that the control gain conforms to the backward Riccati equation while estimation gain abides by the forward equation, which is computationally challenging. Applying the stochastic maximum principle, the solvability of the decentralized LQG control problem is reduced to that of corresponding forward and backward stochastic difference equations (FBSDEs). Further, necessary and sufficient conditions for the solvability of optimal control problem are presented by two Riccati equations, one of which is nonsymmetric. Moreover, a novel iterative forward method is proposed to calculate the coupled backward control gain and forward estimation gain.

It is nontrivial to control a dynamic system that is switched consistently with a completely unknown switched modes. This problem is further complicated if the system is subject to stochastic disturbance. This paper studies the linear quadratic optimal control problem of linear continuous systems with stochastic disturbance and unknown switched process. By integrating one-step adaptive estimator with optimal control theory, a linear quadratic optimal stabilization controller based on sampled feedback is developed for systems that are continuous in nature yet switched consistently with unknown modes. It is shown that with the proposed control scheme, both parameter estimate error and system stabilization error are ensured to be bounded, and the existence of the upper bound is explicitly confirmed. The results compliment and extend the existing works on digital feedback control of switched linear continuous systems with unknown switched processes and stochastic disturbances. Copyright © 2015 John Wiley & Sons, Ltd.

Error estimates of fully discrete mixed finite element methods for semilinear quadratic parabolic optimal control problem

This paper is an overview of fundamental linear–quadratic optimal control techniques used for linear dynamic systems. The presentation is suitable for undergraduate and graduate students and practicing engineers. The paper can be used by class instructors as supplemental material for undergraduate and graduate control system courses. The paper shows how to find the solution to a dynamic optimization problem: optimize an integral quadratic performance criterion along trajectories of a linear dynamic system over an infinite time period (steady-state linear–quadratic optimal control problem). The solution is obtained by solving a static optimization problem. All derivations done in the paper require only elementary knowledge of linear algebra and state space linear system analysis. The results are presented also for the observer-driven linear–quadratic steady-state optimal controller, output feedback-based linear–quadratic optimal controller, and the Kalman filter-driven linear–quadratic stochastic optimal controller. Having full understanding of derivations of the linear–quadratic optimal controller, observer-driven linear–quadratic optimal controller, optimal linear–quadratic output feedback controller, and optimal linear–quadratic stochastic controller, students and engineers will feel confident to use these controllers in numerous engineering and scientific applications. Several optimal linear–quadratic control case studies involving models of real physical systems, with the corresponding Simulink block diagrams and MATLAB codes, are included in the paper.

A new algorithmic setting is proposed for the discrete-time finite-horizon linear quadratic (LQ) optimal control problem with constrained or unconstrained final state, no matter whether the problem is cheap, singular, or regular. The proposed solution, based on matrix pseudoinversion, is completed and made practically implementable by a nesting procedure for welding optimal subarcs that enables arbitrary enlargement of the control time interval.

A stochastic control problem is formulated and we get the explicit form of the optimal control for initial-data-parameterized linear quadratic stochastic optimal control problems with random jumps. The optimal control can be proved to be unique. A stochastic Riccati equation is rigorous derived from the stochastic Hamilton system, which provides an optimal feedback control. This completes the the interrelationship between the stochastic Riccati equation and stochastic Hamilton system as two different but equivalent tools for the stochastic linear quadratic problem.

This paper is concerned with backward stochastic differential equations with Markov switching driven by Brownian motion and Poisson random measure. The motivation is a constrained stochastic Riccati equation derived from a stochastic linear quadratic optimal control problem with both Poisson and Markovian jumps. The existence and uniqueness of an adapted solution under global Lipschitz condition on the generator is obtained. The continuous dependence of the solution on parameters is proved. Two comparison theorems are also derived by a generalized Girsanov transformation theorem.