Existence, uniqueness, and stability of solutions to singular linear quadratic optimal control problems

This paper focuses on addressing the linear quadratic (LQ) optimal control problem on an infinite time horizon for stochastic systems controlled by impulses. No constraint regarding the sign of the quadratic functional is applied. That is why our first concern is to find conditions which guarantee that the considered optimal control problem is well posed. Then, when the optimal control problem is well posed, it is natural to look for conditions which guarantee the attainability of the optimal control problem that is being evaluated. The main tool involved in the solution of the problems stated before is a backward jump matrix linear differential equation (BJMLDE) with a Riccati-type jumping operator. This is formulated using the matrix coefficients of the controlled system and the weight matrices of the performance criterion. We show that the well posedness of the optimal control problem under investigation is guaranteed by the existence of the maximal and bounded solution of the associated BJMLDE with a Riccati-type jumping operator. Further, we show that when the associated BJMLDE with a Riccati-type jumping operator has a maximal solution which satisfies a suitable sign condition, then the optimal control problem is attainable if and only if it has an optimal control in a state feedback form, or if and only if the maximal solution of the BJMLDE with a Riccati-type jumping operator is a stabilizing solution. In order to make the paper more self-contained, we present a set of conditions that correspond to the existence of the maximal solution of the BJMLDE satisfying the desired sign condition.

In this article, we consider a linear-quadratic optimal control problem (LQ problem) for a controlled linear stochastic differential equation driven by a multidimensional Browinan motion and a Poisson random martingale measure in the general case, where the coefficients are allowed to be predictable processes or random matrices. By the duality technique, the dual characterization of the optimal control is derived by the optimality system (so-called stochastic Hamilton system), which turns out to be a linear fully coupled forward-backward stochastic differential equation with jumps. Using a decoupling technique, the connection between the stochastic Hamilton system and the associated Riccati equation is established. As a result, the state feedback representation is obtained for the optimal control. As the coefficients for the LQ problem are random, here, the associated Riccati equation is a highly nonlinear backward stochastic differential equation (BSDE) with jumps, where the generator depends on the unknown variables K, L, and H in a quadratic way (see (5.9) herein). For the case where the generator is bounded and is linearly dependent on the unknown martingale terms L and H, the existence and uniqueness of the solution for the associated Riccati equation are established by Bellman's principle of quasi-linearization.

In this paper, an iterative computational method is proposed to handle constrained nonlinear quadratic optimal control problems. with terminal state constraints, saturation state and saturation control variables constraints. The method is based on replacing the original constrained nonlinear optimal control problem by a sequence of constrained linear time-varying quadratic optimal control problems. To this end, an iterative technique is used to replace the original constrained nonlinear dynamic system by a sequence of constrained linear time-varying systems. Then each of constrained linear time-varying quadratic optimal control problems is approximated by a quadratic programming problem by parameterizing each of the state variable by a finite length Legendre polynomials with unknown parameters. To show the effectiveness of the proposed method, simulation results of a constrained nonlinear optimal control problem are presented.

On the stochastic linear quadratic control problem with piecewise constant admissible controls

This communication presents a spectral method for solving time-varying linear quadratic optimal control problems. Legendre–Gauss–Lobatto nodes are used to construct the mth-degree polynomial approximation of the state and control variables. The derivative x·(t) of the state vector x(t) is approximaed by the analytic derivative of the corresponding interpolating polynomial. The performance index approximation is based on Gauss–Lobatto integration. The optimal control problem is then transformed into a linear programming problem. The proposed technique is easy to implement, efficient and yields accurate results. Numerical examples are included and a comparison is made with an existing result.

We propose a new dual projected pseudo quasi-Newton method for discrete-time linear quadratic optimal control problems of a large-scale multi-destination network. Our method has successfully overcome the difficulties caused by large dimensions and inequality constraints by having a constant Hessian matrix and an efficient successive projection method, respectively. Application of our method to the congestion control of multi-destination networks is presented. Above all, it takes less than one second to obtain the solution of a large-scale discrete-time linear quadratic optimal control problem with the problem size of 2800 variables, 700 equality constraints and 2400 inequality constraints.

On the regularity of solutions of an operator Riccati equation arising in linear quadratic optimal control problems for hereditary differential systems

On the generalized algebraic Riccati equations

An iterative numerical solution method is proposed for the general linear quadratic optimal control problem with phase constraints. The method is based on the extension principle. Realization features of the method are described for Matlab environment.

This paper extends the generating function approach to the state constrained linear quadratic optimal control problem by utilizing penalties. The original constrained problem is approximated by an unconstrained problem with a novel penalty function by which the latter problem can be solved via generating function method based on Taylor series approximation. In view of this, the optimal solution can be given as the state feedback control with explicit boundary conditions and pre-computed coefficients. This structure is useful in the repetitive on-line computation for optimal solutions satisfying a number of different boundary conditions. Examples demonstrate the effectiveness of the developed method.

This paper presents a simple and efficient method based on artificial neural network to solve distributed optimal control of Poisson’s equation with Dirichlet boundary condition. The trial solutions are used to approximate the state and control variables. These trial solutions are considered by using a single layer neural network. By replacing the trial solutions in objective function and Poisson’s equation, then using the weighted residual method, distributed optimal control of Poisson’s equation is converted to a linear quadratic optimal control problem. The weights of the trial solutions are computed by solving the new problem. In order to solve the linear quadratic optimal control problem, the Pontryagin maximum principle is used. Finally we apply the proposed method on several examples that in computational experiments, the high efficiency of the presented method is illustrated.

An assumption on the non-negative solutions of quadratic equations has been used to study linear quadratic optimal control problems with positive controllers. This note investigates this assumption in more detail.

In the paper we present a new method for solving optimal control problems of a class of hybrid systems. We describe the new effective algorithm based on memetic algorithm (MA) for optimal control of switched systems. We concentrate on a class of problems in which a pre-specified sequence of active subsystems is known. Our aim is to find both the optimal switching instants and the optimal continuous inputs. The new approach, which we propose, decomposes the cost functional of the basic optimal control problem in Bolza form in two terms. The first term depends explicitly on a value of state variables at the final time. The second term depends on state and control trajectories. In oreder to solve those two tasks we used MA as the multi-objective optimization algorithm. In this paper we considered a fundamental bi-criteria case with two mentioned before functions: the value of state variables at the final time and the state and control trajectories. In order to find an approximation of Pareto frontier, we proposed new effective method based on genetic algorithm (GA) and local search (LS). Problems properties were taken into consideration in the design of our new approach of solving it. They were used to construct new algorithm inspired by the LS NSGA-II, which performed rather well in multi-criteria scheduling problem. Since simple genetic algorithms are efficient heuristics in searching for optimal solutions, but lack the accuracy of some more computational complex algorithms, a hybrid algorithm was constructed. It uses fast non-dominated sorting, in order to evaluate child population and allocate solutions to corresponding Pareto frontiers. I addition a local search method was used, in order to find more differentiated and better solutions. Clustering solutions from Pareto frontiers also improved diversity of solutions in child population. This approach can be used as a start point for searching for algorithm for solving optimal control problems of switched systems without pre-specified sequence of active subsystems. The performance of the algorithm is illustrated by the examples: (a) a hybrid optimal control problem with nonlinear dynamics and (b) a switched linear quadratic optimal control problem with long switching time intervals.

A modified genetic algorithm for optimal control problems

A non-iterative solution for a class of discrete-time finite-horizon linear quadratic optimal control problems is obtained through the characterization of a pair of structural invariants of the singular Hamiltonian system associated to the H2 optimal control problem stated for the generic, discrete-time quadruple (A, B, C, D). On the assumption that the final state weighting function in the performance index is represented by a quadratic surface, it is shown that the optimal cost is a function of the initial state with the same structure. Optimal control laws and state trajectories are analytically expressed as functions of the initial state as well. The results hold under rather extensive conditions: those that guarantee the existence and uniqueness of the stabilizing solution of the corresponding discrete algebraic Riccati equation.