Relaxed Optimal Control With Self-Learning Horizon for Discrete-Time Stochastic Dynamics.

In this study, an optimal adaptive control approach is established to solve the robust output tracking problem of a class of continuous time uncertain linear systems based on the policy iteration (PI) in actor-critic algorithm. First, by augmenting the integral variables of the tracking error into state variables, the robust tracking problem is transformed into a robust control problem of an augmented uncertain linear system. It is proven that the robust control law of the augmented system enables the output of the considered system to track a polynomial time signal asymptotically. Second, an optimal control method in the corresponding auxiliary nominal system is established, and based on the Bellman optimality principle, PI algorithms are proposed to solve online tracking controllers for the matched and the mismatched uncertain systems. Finally, for testing the availability of the proposed approach and theoretical results, two numerical experiments are provided.

Optimal [formula omitted] tracking control of nonlinear systems with zero-equilibrium-free via novel adaptive critic designs

In this paper, we study the optimal control problem for Takagi–Sugeno (T-S) fuzzy systems with disturbances. Due to the presence of disturbances in the T-S fuzzy systems, a fuzzy optimal state feedback control approach is presented by employing a two-player zero-sum game. Since the analytical optimal control solutions and the worst-case disturbance policies of T-S fuzzy systems can be boiled down to solving the algebraic Riccati equations (AREs), which are difficult to be obtained directly, a model-free policy iteration (PI) learning algorithm is proposed to obtain their approximation solutions. It is proved that the developed fuzzy optimal state feedback controller can ensure the fuzzy systems to be asymptotically stable and satisfy the disturbance attenuation condition simultaneously. Also, the designed PI learning algorithm can converge to their optimal solutions. Finally, we apply the developed fuzzy optimal state feedback control method to the truck-trailer system and the simulation results demonstrated the effectiveness of the developed scheme.

In this chapter, by using neural network (NN)-based online learning optimal control approach, a decentralized control strategy is developed to stabilize a class of continuous-time large-scale interconnected nonlinear systems . It is proven that the decentralized control strategy of the overall system can be established by adding appropriate feedback gains to the optimal control laws of the isolated subsystems . Then, an online policy iteration (PI) algorithm is developed to solve the Hamilton–Jacobi–Bellman equations related to the optimal control problem. By constructing a set of critic NNs, the cost functions can be obtained by NN approximation, followed by the control laws. Furthermore, as a generalization, an NN-based decentralized control law is developed to stabilize the large-scale interconnected nonlinear systems using an online model-free integral PI algorithm. The model-free PI approach can solve the decentralized control problem for large-scale interconnected nonlinear systems with unknown dynamics. Finally, two simulation examples are provided to illustrate the effectiveness of the present decentralized control scheme.

This thesis is concerned with numerical methods for Mixed-Integer Optimal Control Problems with Combinatorial Constraints. We establish an approximation theorem relating a Mixed-Integer Optimal Control Problem with Combinatorial Constraints to a continuous relaxed convexified Optimal Control Problems with Vanishing Constraints that provides the basis for numerical computations. We develop a a Vanishing- Constraint respecting rounding algorithm to exploit this correspondence computationally. Direct Discretization of the Optimal Control Problem with Vanishing Constraints yield a subclass of Mathematical Programs with Equilibrium Constraints. Mathematical Programs with Equilibrium Constraint constitute a class of challenging problems due to their inherent non-convexity and non-smoothness. We develop an active-set algorithm for Mathematical Programs with Equilibrium Constraints and prove global convergence to Bouligand stationary points of this algorithm under suitable technical conditions. For efficient computation of Newton-type steps of Optimal Control Problems, we establish the Generalized Lanczos Method for trust region problems in a Hilbert space context. To ensure real-time feasibility in Online Optimal Control Applications with tracking-type Lagrangian objective, we develop a Gaus-Newton preconditioner for the iterative solution method of the trust region problem. We implement the proposed methods and demonstrate their applicability and efficacy on several benchmark problems.

This article presents a nearly optimal solution to the cooperative formation control problem for large-scale multiagent system (MAS). First, multigroup technique is widely used for the decomposition of the large-scale problem, but there is no consensus between different subgroups. Inspired by the hierarchical structure applied in the MAS, a hierarchical leader-following formation control structure with multigroup technique is constructed, where two layers and three types of agents are designed. Second, adaptive dynamic programming technique is conformed to the optimal formation control problem by the establishment of performance index function. Based on the traditional generalized policy iteration (PI) algorithm, the multistep generalized policy iteration (MsGPI) is developed with the modification of policy evaluation. The novel algorithm not only inherits the advantages of high convergence speed and low computational complexity in the generalized PI algorithm but also further accelerates the convergence speed and reduces run time. Besides, the stability analysis, convergence analysis, and optimality analysis are given for the proposed multistep PI algorithm. Afterward, a neural network-based actor-critic structure is built for approximating the iterative control policies and value functions. Finally, a large-scale formation control problem is provided to demonstrate the performance of our developed hierarchical leader-following formation control structure and MsGPI algorithm.

This paper studies data-driven learning-based methods for the finite-horizon optimal control of linear time-varying discrete-time systems. First, a novel finite-horizon Policy Iteration (PI) method for linear time-varying discrete-time systems is presented. Its connections with existing infinite-horizon PI methods are discussed. Then, both data-driven off-policy PI and Value Iteration (VI) algorithms are derived to find approximate optimal controllers when the system dynamics is completely unknown. Under mild conditions, the proposed data-driven off-policy algorithms converge to the optimal solution. Finally, the effectiveness and feasibility of the developed methods are validated by a practical example of spacecraft attitude control.

The paper presents a reinforcement learning approach for optimal tracking control of switched systems with application to a grid-tied hybrid generation system. To enhance interaction with the irregular environment, reference trajectory is learned via controller from states to optimal control. The main issue is to solve the optimal tracking control problem for a hybrid generation system consisting of multiple switched subsystems, and reinforcement learning can seek the globally optimal solution well without knowing accurate system dynamics. The investigated learning algorithm is used to generate an optimum map based on the learned ultimate value without knowledge of system parameters and obtains the optimal control law via deriving of algebraic Riccati equation (ARE) with unnecessary knowing of command generator dynamics. The optimal control solution can converge the online learning algorithm well based on policy iteration as verification in the simulation.

In this paper, a hybrid algorithm for robot's path planning is proposed. This algorithm consists of the optimal control algorithm and PSO meta-heuristic algorithm. In this method, a two-point boundary value equation is derived through optimal control method and the initial solution is needed to solve it. To solve these equations, the PSO provides preliminary solutions and optimizing the solution of equations (robot path) with optimizing initial solution concerning cost function results in global optimization. So the formulation of proposed method is presented and it is used for path planning of a 4-DOF surgical robotic system. The results show that in this algorithm, the cost function does not depend on initial guess in comparison with optimal control and PSO methods.

<div class="section abstract"><div class="htmlview paragraph">Autonomous driving related technologies have become a hot topic in academia and industry. Planning control is one of the core technologies of autonomous driving, which is conducive to vehicles safe and efficient driving. This paper proposes a novel optimal speed control algorithm, which considers the power system's energy consumption, the speed limit on the road, and the safe distance of the vehicle in front. An optimal speed control model of “From battery to wheel” energy consumption is established by constructing a performance index function based on the best-fitting formula of motor power, motor speed and torque. Based on the optimal control principle, the fourth-order ordinary differential equation of the speed control model is established, based on the indirect adjoining approach, the speed control model under the restriction of the road speed limit and safe distance of the preceding vehicle is derived and the analytical expression is obtained. Under the restriction of the road speed limit and safe distance of the preceding vehicle, the model predictive control method is introduced, the urban working conditions in the NEDC cycle are selected for simulation, and the optimal control input is solved in real-time, compared with the intelligent driver driving strategy, this algorithm is proved to be superior in the aspect of vehicle speed optimization control. Finally, the influence mechanism of the road speed limit constraint on the vehicle speed control is analyzed. This paper realizes the design of the optimal speed control algorithm for energy consumption under various constraints. The results can be used to guide the energy-saving driving of vehicles, and provide theoretical guidance for ecological driving technology in the context of intelligent transportation.</div></div>

We wish to study a class of optimal controls for problems governed by forward-backward doubly stochastic differential equations (FBDSDEs). Firstly, we prove existence of optimal relaxed controls, which are measure-valued processes for nonlinear FBDSDEs, by using some tightness properties and weak convergence techniques on the space of Skorokhod{\mathbb{D}}equipped with theS-topology of Jakubowski. Moreover, when the Roxin-type convexity condition is fulfilled, we prove that the optimal relaxed control is in fact strict. Secondly, we prove the existence of a strong optimal controls for a linear forward-backward doubly SDEs. Furthermore, we establish necessary as well as sufficient optimality conditions for a control problem of this kind of systems. This is the first theorem of existence of optimal controls that covers the forward-backward doubly systems.

Fuzzy optimal tracking control for nonlinear underactuated unmanned surface vehicles

Optimal control strategies are very useful in manipulation problems of robots. However, most optimal controls are highly sensitive to plant-parameter variations and disturbances because optimal controls try to pick up exact information from the mathematical models in order to realize the best performance. We recognize that difficulties are involved in exact modeling of plants and prevention of disturbances. Therefore, we need a optimal control method with low sensitivity. The ideas of closed-loop control system design using LQG/TR techniques (linear quadratic Gaussian control with transfer recovery) are discussed in this paper. We combine optimal controls and LQG/TR control into two-degrees-of-freedom control systems to reduce the sensitivities of optimal control strategies. In digital simulations, we obtained excellent results of a two-link manipulator in an optimal time control problem using our method. Some modeling errors and considerable disturbances are allowed in the method. Thus, we can realize good performances in practical manipulation problems using our method.

1 A Survey on Computational Optimal Control.- Issues in the Direct Transcription of Optimal Control Problems to Sparse Nonlinear Programs.- Optimization in Control of Robots.- Large-scale SQP Methods and their Application in Trajectory Optimization.- Solving Optimal Control and Pursuit-Evasion Game Problems of High Complexity.- 2 Theoretical Aspects of Optimal Control and Nonlinear Programming.- Continuation Methods In Boundary Value Problems.- Second Order Optimality Conditions for Singular Extremals.- Synthesis of Adaptive Optimal Controls for Linear Dynamic Systems.- Control Applications of Reduced SQP Methods.- Time Optimal Control of Mechanical Systems.- 3 Algorithms for Optimal Control Calculations.- Second Order Algorithm for Time Optimal Control of a Linear System.- An SQP-type Solution Method for Constrained Discrete-Time Optimal Control Problems.- Numerical Methods for Solving Differential Games, Prospective Applications to Technical Problems.- Construction of the Optimal Feedback Controller for Constrained Optimal Control Problems with Unknown Disturbances.- Repetitive Optimization for Predictive Control of Dynamic Systems under Uncertainty.- Optimal Control of Multistage Systems Described by High-Index Differential-Algebraic Equations.- A New Class of a High Order Interior Point Method for the Solution of Convex Semiinfinite Optimization Problems.- A Structured Interior Point SQP Method for Nonlinear Optimal Control Problems.- 4 Software for Optimal Control Calculations.- Automated Approach for Optimizing Dynamic Systems.- ANDECS: A Computation Environment for Control Applications of Optimization.- Application of Automatic Differentiation to Optimal Control Problems.- OCCAL: A mixed symbolic-numeric Optimal Control CALculator.- 5 Applications of Optimal Control.- A Robotic Satellite with Simplified Design.- Nonlinear Control under Constraints of a Biological System.- An Object-Oriented Approach to Optimally Describe and Specify a SCADA System Applied to a Power Network.- Near-Optimal Flight Trajectories Generated by Neural Networks.- Performance of a Feedback Method with Respect to Changes in the Air-Density during the Ascent of a Two-Stage-To-Orbit Vehicle.- Linear Optimal Control for Reentry Flight.- Steady-State Modelling of Turbine Engine with Controllers.- Shortest Paths for Satellite Mounted Robot Manipulators.- Optimal Control of the Industrial Robot Manutec r3.

Existence theory of optimal relaxed control problem for a class of stochastic hereditary evolution equations driven by Lévy noise has been studied. We formulate the problem in the martingale sense of Stroock and Varadhan to establish existence of optimal controls. The construction of the solution is based on the classical Faedo–Galerkin approximation, the compactness method and the Jakubowski version of the Skorokhod theorem for nonmetric spaces, and certain compactness properties of the class of Young measures on Suslin metrizable control sets. As application of the abstract theory, Oldroyd and Jeffreys fluids have been studied and existence of optimal relaxed control is established. Existence and uniqueness of a strong solution and uniqueness in law for the two-dimensional Oldroyd and Jeffreys fluids are also shown.