Estimation and Stochastic Control for Linear Infinite-Dimensional Systems

Stochastic optimal control of unknown linear networked control system in the presence of random delays and packet losses

The problem of optimal control of time-varying linear singular systems with quadratic performance index has been studied using the Runge–Kutta–Butcher algorithm. The results obtained using the Runge–Kutta (RK) method based on the arithmetic mean (RKAM) and the RK–Butcher algorithms are compared with the exact solutions of the time-varying optimal control of linear singular systems. It is observed that the result obtained using the RK–Butcher algorithm is closer to the true solution of the problem. Stability regions for the RKAM algorithm, the single-term Walsh series method and the RK–Butcher algorithms are presented. Error graphs for the simulated results and exact solutions are presented in graphical form to highlight the efficiency of the RK–Butcher algorithm. This algorithm can easily be implemented using a digital computer. An additional advantage of this method is that the solution can be obtained for any length of time for this type of optimal control of time-varying linear singular systems.

This article presents the central finite-dimensional H ∞ controller for linear time-varying systems with unknown parameters, that is suboptimal for a given threshold γ with respect to a modified Bolza–Meyer quadratic criterion including the attenuation control term with the opposite sign. In contrast to the previously obtained results, this article reduces the original H ∞ controller problem to the corresponding H 2 controller problem, using the technique proposed in Doyle et al. [Doyle, J.C., Glover, K., Khargonekar, P.P., and Francis, B.A. (1989), ‘State-space Solutions to Standard H 2 and H Infinity Control Problems’, IEEE Transactions Automatic Control, 34, 831–847]. This article yields the central suboptimal H ∞ controller for linear systems with unknown parameters in a closed finite-dimensional form, based on the corresponding H 2 controller obtained in Basin and Calderon-Alvarez [Basin, M.V., and Calderon-Alvarez, D. (2008), ‘Optimal LQG Controller for Linear Systems with Unknown Parameters’, Journal of The Franklin Institute, 345, 293–302]. Numerical simulations are conducted to verify performance of the designed central suboptimal controller for uncertain linear systems with unknown parameters against the conventional central suboptimal H ∞ controller for linear systems with exactly known parameter values.

This paper studies data-driven learning-based methods for the finite-horizon optimal control of linear time-varying discrete-time systems. First, a novel finite-horizon Policy Iteration (PI) method for linear time-varying discrete-time systems is presented. Its connections with existing infinite-horizon PI methods are discussed. Then, both data-driven off-policy PI and Value Iteration (VI) algorithms are derived to find approximate optimal controllers when the system dynamics is completely unknown. Under mild conditions, the proposed data-driven off-policy algorithms converge to the optimal solution. Finally, the effectiveness and feasibility of the developed methods are validated by a practical example of spacecraft attitude control.

1 A Survey on Computational Optimal Control.- Issues in the Direct Transcription of Optimal Control Problems to Sparse Nonlinear Programs.- Optimization in Control of Robots.- Large-scale SQP Methods and their Application in Trajectory Optimization.- Solving Optimal Control and Pursuit-Evasion Game Problems of High Complexity.- 2 Theoretical Aspects of Optimal Control and Nonlinear Programming.- Continuation Methods In Boundary Value Problems.- Second Order Optimality Conditions for Singular Extremals.- Synthesis of Adaptive Optimal Controls for Linear Dynamic Systems.- Control Applications of Reduced SQP Methods.- Time Optimal Control of Mechanical Systems.- 3 Algorithms for Optimal Control Calculations.- Second Order Algorithm for Time Optimal Control of a Linear System.- An SQP-type Solution Method for Constrained Discrete-Time Optimal Control Problems.- Numerical Methods for Solving Differential Games, Prospective Applications to Technical Problems.- Construction of the Optimal Feedback Controller for Constrained Optimal Control Problems with Unknown Disturbances.- Repetitive Optimization for Predictive Control of Dynamic Systems under Uncertainty.- Optimal Control of Multistage Systems Described by High-Index Differential-Algebraic Equations.- A New Class of a High Order Interior Point Method for the Solution of Convex Semiinfinite Optimization Problems.- A Structured Interior Point SQP Method for Nonlinear Optimal Control Problems.- 4 Software for Optimal Control Calculations.- Automated Approach for Optimizing Dynamic Systems.- ANDECS: A Computation Environment for Control Applications of Optimization.- Application of Automatic Differentiation to Optimal Control Problems.- OCCAL: A mixed symbolic-numeric Optimal Control CALculator.- 5 Applications of Optimal Control.- A Robotic Satellite with Simplified Design.- Nonlinear Control under Constraints of a Biological System.- An Object-Oriented Approach to Optimally Describe and Specify a SCADA System Applied to a Power Network.- Near-Optimal Flight Trajectories Generated by Neural Networks.- Performance of a Feedback Method with Respect to Changes in the Air-Density during the Ascent of a Two-Stage-To-Orbit Vehicle.- Linear Optimal Control for Reentry Flight.- Steady-State Modelling of Turbine Engine with Controllers.- Shortest Paths for Satellite Mounted Robot Manipulators.- Optimal Control of the Industrial Robot Manutec r3.

We consider the nonstandard infinite horizon quadratic cost minimization problem for a stable abstract linear control system, and show that it can be reduced to a J-inner coprime factorization problem (or equivalently, to a canonical spectral factorization problem) in the (often finite-dimensional) control space. More precisely, we show that the optimal solution of the quadratic cost minimization problem is of static state feedback type if and only if a certain spectral factorization problem has a solution. If both the system and the spectral factor are regular together with their adjoints, then the feedback operator can be expressed in terms of the Riccati operator, and the Riccati operator is a self-adjoint solution of an algebraic Riccati equation. This Riccati equation is similar to the classical algebraic Riccati equation, but one of its coefficients differs from the expected one. We apply our main theorems to prove the first available versions of the positive real and bounded real lemmas for abstract linear systems. Similar results are true for unstable systems.

In the field of control theory, there has been an increase in interest in studies on the equivalence of robust and optimum control of linear control systems. To provide appropriate constraints for increased stability and control performance, we study in this work the link between linear systems under robust and optimum control as well as the equivalence between the two. We will present in this research significant changes to the way constraints are designed, which may also improve system control and stability. We will show that with theorems and examples.

Optimal control of multivariable systems with pure time delays

In this paper, the problem about optimal iterative learning control of general nonlinear discrete-time systems has been studied. Based on contraction mapping principle, if some mild assumptions are given, we can obtain some sufficient conditions for the existence of optimal iterative learning control of general nonlinear discrete-time systems. These conditions are generalizations of the existence conditions of optimal iterative learning control of linear discrete-time systems.

Optimal control of fractional order systems is a long established domain of fractional calculus. Nevertheless, it relies on equations expressed in terms of pseudo-state variables which raise fundamental questions. So in order remedy these problems, the authors propose in this paper a new and original approach to fractional optimal control based on a frequency distributed representation of fractional differential equations called the infinite state approach, associated with an original formulation of fractional energy, which is intended to really control the internal system state. In the first step, the fractional calculus of variations is revisited to express appropriate Euler Lagrange equations. Then, the quadratic optimal control of fractional linear systems is formulated. Thanks to a frequency discretization technique, the previous theoretical equations are converted into an equivalent large dimension integer order system which permits the implementation of a feasible optimal solution. A numerical example illustrates the validity of this new approach.

ABSTRACTA finite horizon linear quadratic (LQ) optimal control problem is studied for a class of discrete-time linear fractional systems (LFSs) affected by multiplicative, independent random perturbations. Based on the dynamic programming technique, two methods are proposed for solving this problem. The first one seems to be new and uses a linear, expanded-state model of the LFS. The LQ optimal control problem reduces to a similar one for stochastic linear systems and the solution is obtained by solving Riccati equations. The second method appeals to the principle of optimality and provides an algorithm for the computation of the optimal control and cost by using directly the fractional system. As expected, in both cases, the optimal control is a linear function in the state and can be computed by a computer program. A numerical example and comparative simulations of the optimal trajectory prove the effectiveness of the two methods. Some other simulations are obtained for different values of the fractional order.

Controllability of linear impulsive stochastic systems in Hilbert spaces

In this two-part paper we study stabilization and optimal control of linear time-invariant systems with stochastic multiplicative uncertainties. We consider structured multiplicative perturbations, which, unlike in robust control theory, consist of static, zero-mean stochastic processes, and we assess the stability and performance of such systems using mean-square measures. While Part 2 of this paper tackles and solves optimal control problems under the mean-square criterion, Part 1 is devoted to the stabilizability problem. We develop fundamental conditions of mean-square stabilizability which ensure that an open-loop unstable system can be stabilized by output feedback in the mean-square sense. For single-input single-output systems, a general, explicit stabilizability condition is obtained. This condition, both necessary and sufficient, provides a fundamental limit imposed by the system's unstable poles, nonminimum phase zeros and time delay. For multi-input multi-output systems, we provide a complete, computationally efficient solution for minimum phase systems possibly containing time delays, in the form of a generalized eigenvalue problem readily solvable by means of linear matrix inequality optimization. Limiting cases and nonminimum phase plants are analyzed in depth for conceptual insights, revealing, among other things, how the directions of unstable poles and nonminimum phase zeros may affect mean-square stabilizability in MIMO systems. Other than their independent interest, stochastic multiplicative uncertainties have found utilities in modeling networked control systems pertaining to, e.g., packet drops, network delays, and fading. Our results herein lend solutions applicable to networked control problems addressing these issues.

Stabilizability of Bimodal Piecewise Linear Systems With Continuous Vector Field

This paper investigates the application of associative reinforcement learning techniques to the optimal control of linear discrete-time dynamic systems. Associative reinforcement learning involves the trial and error interaction with a dynamic system to determine the control actions that optimally achieve some desired performance index. The methodology can be applied either online or off-line and in a model based or model free manner. Associative reinforcement learning techniques are applied to the optimal regulator (LQR) control of discrete-time linear systems. Adaptive critic designs are implemented and the convergence speed compared for the different approaches. These methods can determine the optimal state and state/action value function and the optimal policy without requiring system models. (4 pages)