Optimal stochastic control for linear dynamic systems with quadratic payoff

Linear quadratic regulator (LQR) and proportional-integral-derivative (PID) control methods, which are generally used for control of linear dynamical systems, are used in this paper to control the nonlinear dynamical system. LQR is one of the optimal control techniques, which takes into account the states of the dynamical system and control input to make the optimal control decisions. The nonlinear system states are fed to LQR which is designed using a linear state-space model. This is simple as well as robust. The inverted pendulum, a highly nonlinear unstable system, is used as a benchmark for implementing the control methods. Here the control objective is to control the system such that the cart reaches a desired position and the inverted pendulum stabilizes in the upright position. In this paper, the modeling and simulation for optimal control design of nonlinear inverted pendulum-cart dynamic system using PID controller and LQR have been presented for both cases of without and with disturbance input. The Matlab-Simulink models have been developed for simulation and performance analysis of the control schemes. The simulation results justify the comparative advantage of LQR control method.

The methodology of estimation and control theory is considered in terms of response, stability, estimation, and control of linear dynamic systems. Within the context of discrete-time systems, multi-input, multi-output, nth-order linear systems are discussed, and general results for optimal estimation, optimal control, and other topics are presented. The application of these results to modeling human behavior is considered with special emphasis on man-machine system models.

Necessary and sufficient conditions for impulsive controllability of linear dynamical systems are obtained, which provide a novel approach to problems that are basically defined by continuous dynamical systems, but on which only discrete‐time actions are exercised. As an application, impulsive maneuvering of a spacecraft is discussed.

The linear dynamic systems theory appears to be the most developed part of the control theory. Comparative simplicity of this theory due to the superposition principle dominating there, makes it widely accessible when developing control systems for different kind of processes. Monographs [1, 2, 3] are devoted to numerous analysis and synthesis problems concerning these systems. In the following an approach is developed to linear dynamic plants control algorithms aimed at the estimation of the plant reverse operator for the case when the order of the equation describing the process is unknown.

A linear dynamic system is considered. Orthogonal series procedures are developed to recover the state and its derivative. The uniform mean square error convergence is investigated. An application to identification and open-loop control is discussed.

Stochastic optimal control of unknown linear networked control system in the presence of random delays and packet losses

In this article we present a tutorial introduction to Generalized Proportional-Integral-Derivative Control (GPID). We particularly emphasize the links between flatness and GPID control, in the context of linear dynamic systems. We present several illustrative design examples, some of them of physical flavor. The performance of the controlled systems is evaluated by means of digital computer simulations.

This paper addresses to the constrained optimization problem for continuous-time dynamical systems. We extend and modify the existing results in constrained optimization to multi-input open-loop unstable systems, and the generalized nonquadratic cost is introduced. The paper elaborates a new procedure that exploits the beneficial advantages of an innovative design framework. Necessary and sufficient conditions for optimality are embedded and a computationally tractable technique is developed for synthesizing the nonlinear control laws. The key theorem, two propositions and three lemmas are formulated and proved. The procedure is illustrated through an example. This example illustrates the application of the explored optimization methodology to design a bounded smooth controller for the AFTI/F-16 fighter. This control law ensures the optimality in an entire flight envelope and belongs to a specified admissible set that satisfies the bounds imposed on deflections of the control surfaces.

In the present paper finite-dimensional dynamical control systems described by linear higherorder ordinary differential state equations with multiple point delays in control are considered. Using algebraic methods, necessary and sufficient condition for relative controllability in a given time interval for linear dynamical system with multiple point delays in control is formulated and proved. This condition is generalization to relative controllability case some previous results concerning controllability of linear dynamical systems without multiple point delays in the control. Simple numerical example, which illustrates theoretical result is also given. Finally, some remarks and comments on the existing results for controllability of dynamical systems with delays in control are also presented.

The basic idea behind infinite horizon state feedback k-cost-cumulant (kCC) control is to seek a constant state feedback law which minimizes the value of a finite linear combination of the first k cost cumulants of a traditional integral quadratic cost associated with a linear stochastic system, when there is no specification of a large terminal time. The paper begins with the development of matrix algebraic equations for the cost cumulants. These equations permit the incorporation of classes of linear feedback controllers for linear dynamical systems defined on infinite horizon. The infinite horizon control problem with the optimization goal of a finite linear combination of the first k cost cumulants is then stated. Because the control optimization problem at hand involves matrix equality constraints, the optimal feedback solution is investigated by utilizing a Lagrange multipliers technique. The efficacy and applicability of this control paradigm, based upon the first three cost cumulants, are demonstrated on the second earthquake generation benchmark for response control of cable-stayed bridges (Caicedo, J.M., et al., 2003). The simulation results indicate that the cost cumulants in the infinite horizon case offer significant improvements in robust stability margin while keeping comparable levels of structural performance when comparing to those of the baseline LQG in the benchmark.

Metacognitive Control of Linear Dynamic Systems with Self-confidence Adaptation

This paper investigates the application of associative reinforcement learning techniques to the optimal control of linear discrete-time dynamic systems. Associative reinforcement learning involves the trial and error interaction with a dynamic system to determine the control actions that optimally achieve some desired performance index. The methodology can be applied either online or off-line and in a model based or model free manner. Associative reinforcement learning techniques are applied to the optimal regulator (LQR) control of discrete-time linear systems. Adaptive critic designs are implemented and the convergence speed compared for the different approaches. These methods can determine the optimal state and state/action value function and the optimal policy without requiring system models. (4 pages)

1 A Survey on Computational Optimal Control.- Issues in the Direct Transcription of Optimal Control Problems to Sparse Nonlinear Programs.- Optimization in Control of Robots.- Large-scale SQP Methods and their Application in Trajectory Optimization.- Solving Optimal Control and Pursuit-Evasion Game Problems of High Complexity.- 2 Theoretical Aspects of Optimal Control and Nonlinear Programming.- Continuation Methods In Boundary Value Problems.- Second Order Optimality Conditions for Singular Extremals.- Synthesis of Adaptive Optimal Controls for Linear Dynamic Systems.- Control Applications of Reduced SQP Methods.- Time Optimal Control of Mechanical Systems.- 3 Algorithms for Optimal Control Calculations.- Second Order Algorithm for Time Optimal Control of a Linear System.- An SQP-type Solution Method for Constrained Discrete-Time Optimal Control Problems.- Numerical Methods for Solving Differential Games, Prospective Applications to Technical Problems.- Construction of the Optimal Feedback Controller for Constrained Optimal Control Problems with Unknown Disturbances.- Repetitive Optimization for Predictive Control of Dynamic Systems under Uncertainty.- Optimal Control of Multistage Systems Described by High-Index Differential-Algebraic Equations.- A New Class of a High Order Interior Point Method for the Solution of Convex Semiinfinite Optimization Problems.- A Structured Interior Point SQP Method for Nonlinear Optimal Control Problems.- 4 Software for Optimal Control Calculations.- Automated Approach for Optimizing Dynamic Systems.- ANDECS: A Computation Environment for Control Applications of Optimization.- Application of Automatic Differentiation to Optimal Control Problems.- OCCAL: A mixed symbolic-numeric Optimal Control CALculator.- 5 Applications of Optimal Control.- A Robotic Satellite with Simplified Design.- Nonlinear Control under Constraints of a Biological System.- An Object-Oriented Approach to Optimally Describe and Specify a SCADA System Applied to a Power Network.- Near-Optimal Flight Trajectories Generated by Neural Networks.- Performance of a Feedback Method with Respect to Changes in the Air-Density during the Ascent of a Two-Stage-To-Orbit Vehicle.- Linear Optimal Control for Reentry Flight.- Steady-State Modelling of Turbine Engine with Controllers.- Shortest Paths for Satellite Mounted Robot Manipulators.- Optimal Control of the Industrial Robot Manutec r3.

Complete controllability of linear dynamic systems

The problem of optimal control of time-varying linear singular systems with quadratic performance index has been studied using the Runge–Kutta–Butcher algorithm. The results obtained using the Runge–Kutta (RK) method based on the arithmetic mean (RKAM) and the RK–Butcher algorithms are compared with the exact solutions of the time-varying optimal control of linear singular systems. It is observed that the result obtained using the RK–Butcher algorithm is closer to the true solution of the problem. Stability regions for the RKAM algorithm, the single-term Walsh series method and the RK–Butcher algorithms are presented. Error graphs for the simulated results and exact solutions are presented in graphical form to highlight the efficiency of the RK–Butcher algorithm. This algorithm can easily be implemented using a digital computer. An additional advantage of this method is that the solution can be obtained for any length of time for this type of optimal control of time-varying linear singular systems.