Some second-order necessary conditions in optimal control

An extension of the classical Pontryagin maximum principle for Mayer problems without terminal constraints, subject to affine control systems $$\dot x = X_0 (x) + \sum\nolimits_{j = 1}^m u _j X_j (x)$$ , is proved. In connection with a suitable dilation on the state space ℝ n , we introduce a class of “homogeneous tangent vectors” which provide a nonlinear, high-order approximation of the attainable set in the case where the usual linear approximation proves to be inadequate. By studying control variations which generate homogeneous tangent vectors, we derive new necessary conditions for optimality that are particularly effective for basically nonlinear optimal control problems where other high-order tests provide no conclusive information.

Systematic application of Pontryagin's maximum principle to a general class of nonlinear mixed boundary value control problems yields necessary conditions for optimal control. Also, the features of certain pathological situations including the abnormal case and singular control are clearly revealed in this treatment.

The classical Pontryagin maximum principle is a first-order necessary condition (FONC for short) for optimal controls of ordinary differential equations (ODEs for short). When the FONC degenerates, people introduce second-order necessary conditions (SONCs for short) for optimal controls. SONCs are well studied for optimal controls of systems described by ODEs and partial differential equations (PDEs for short). Some results for SONC of optimal controls for systems governed by stochastic differential equations (SDEs for short) are also obtained. However, nothing is known about the SONC for optimal controls of systems described by stochastic (infinite dimensional) evolution equations (SEEs for short). This paper aims to give a solution to this difficult problem, under some mild conditions.

In this note, we study the second order optimality conditions of optimal control problems on Riemannian manifolds. On one hand, we give the second order necessary conditions for optimal controls when the initial state is constrained to a closed subset of a Riemannian manifold. On the other hand, we obtain the second order sensitivity relations between the necessary conditions and the value function of an optimal control problem: the first and second order dual variables are related to the superjet of the value function. Both results show that the second order optimality conditions of optimal control problems on a Riemannian manifold are closely related to the curvature tensor of the manifold.

The concept of the optimal feedback controllability of a nonlinear plant with respect to an accompanying optimal control problem is introduced. An illustrative example in the context of the second-order necessary conditions of optimal control is presented. An extension to the wider class of not optimal feedback controllable plants with respect to a chosen performance index is elaborated by a suitable type of transformation. >

In recent years, the analysis and synthesis of control systems in the descriptor form have been established. This general description of dynamical systems by differential/algebraic equations is important for many applications in mechanics and mechatronics, in electrical and electronic engineering, and in chemical engineering as well. The present contribution deals with the design of optimal control for such dynamic systems. This means that the calculus of variations and Pontryagin's maximum principle have to be generalized for descriptor systems. An important role in the applicability of usual design algorithms plays the properness of the systems. The distinction between proper and nonproper systems is necessary in determining descriptor systems which are exclusively governed by the control inputs or additionally by their higher-order time-derivatives. In case of nonproper systems, quite different optimal control problems appear. For both cases, necessary conditions for optimal control are presented.

In this paper, necessary conditions of optimality, in the form of a maximum principle, are obtained for singular stochastic control problems. This maximum principle is derived for a state process satisfying a general stochastic differential equation where the coefficient associated to the control process can be dependent on the state. We consider the class of so called robust nonlinear impulsive systems, those discontinuous solutions can be considered also as point-wise limits of ordinary solutions. The special conditions of robustness permit to derive the backward equations for adjoint variables in concise form of differential equation with measure and thereby to derive the optimality condition in the form of strong (point-wise) maximum principle.

Multiple shooting is an indirect solution method for complicated optimal control problems. It yields precise numerical solutions. The rate of convergence is usually quadratic due to the underlying Newton type method. However, it was recently noticed that if the general necessary conditions of optimal control are applied to state constrained problem formulations from aerospace engineering the method decelerates often to a slow linear rate of convergence. We identify the culprit, namely the appearance of a quadratic control component in the optimal control problem and certain boundary and/or interior point conditions. Finally we provide a reformulation of these boundary and/or interior point conditions in order to avoid this situation.

In this article, the singular extremal control problem with time delay is studied from the viewpoint of the calculus of variation and matrix theory. Some sufficient conditions for the non-negativity of the second variation are obtained for an optimal control problem with a delayed state vector. The hypothesis of Hass is first assumed, and the total singular situation of Jacobson and Bell is also considered. A second-order necessary condition for optimality is also presented.

For Meyer type optimal control problems the principle of optimality states that every subarc of an optimal solution that extends to the final time furnishes again a solution to the same optimal control problem, only with different initial conditions. Through a simple counter example it is shown in this paper that this principle is not satisfied, in general, for solutions obtained from a standard collocation approach. Introduction The principle of optimality [1] is one of the most intuitive and yet most fundamental theorems in the theory of optimal control. Its basic statement is that for Meyer type optimal control problems every subarc of an optimal solution that extends to the final time furnishes again a solution to the same optimal control problem, only with different initial conditions. This theorem has significant implications. For example, it is the starting point for the derivation of the Hamilton-Jacobi-Bellman equations [2] and it is the underlying principle in Bellman's dynamic programming approach [1], [3]. The present paper shows that the principle of optimality is not satisfied, in general, for solutions obtained from a standard collocation approach[4]. For a simple example whose collocation solution can be generated analytically it is shown that further improvements are possible along an optimal collocation solution if the design parameters along a number of leading nodes are frozen and the remainder of the trajectory is re-optimized. I. Continuous Optimal Control Problem In this section of the paper we introduce a simple class of optimal control problems without state and control constraints. In Section 1.2 we present the associated necessary conditions for optimality [2], [5]. For ease of notation we assume that the right-hand side of the 'Supervising Engineer, Member AIAA, t Senior Engineer, Member AIAA, *17 Research Drive, Hampton, VA 23666. ^Graduate Student, Member AIAA, Department, 304 Town Engineering, Ames, IA 50011 equations of motion does not depend explicitly on time t. This does not represent a loss of generality, as it is always possible to transform explicit time dependence away through the introduction of an additional, timelike state variable. 1. Problem Formulation Let us consider the following class of optimal control problems: min «e(Pwc[*o,*/]), (1) subject to the conditions ±(t) = /(*(*),«(*)), tf.(z(*oMo) = 0, (2)

This paper analyzes a mathematical model for the growth of bone marrow cells under cell-cycle-specific cancer chemotherapy originally proposed by Fister and Panetta [8]. The model is formulated as an optimal control problem with control representing the drug dosage ( respectively its effect ) and objective of Bolza type depending on the control linearly, a so-called L(1)- objective. We apply the Maximum Principle, followed by high-order necessary conditions for optimality of singular arcs and give sufficient conditions for optimality based on the method of characteristics. Singular controls are eliminated as candidates for optimality, and easily verifiable conditions for strong local optimality of bang-bang controls are formulated in the form of transversality conditions at switching surfaces. Numerical simulations are given.

This paper is mainly concerned with a class of optimal control problems of systems governed by the nonlinear dynamic systems on time scales. Introducing the reasonable weak solution of nonlinear dynamic systems, the existence of the weak solution for the nonlinear dynamic systems on time scales and its properties are presented. Discussing L1-strong-weak lower semicontinuity of integral functional, we give sufficient conditions for the existence of optimal controls. Using integration by parts formula and Hamiltonian function on time scales, the necessary conditions of optimality are derived respectively. Some examples on continuous optimal control problems, discrete optimal control problems, mathematical programming and variational problems are also presented for demonstration.

The optimal control of a spacecraft as it transitions between specified states using continuous thrust in a fixed amount of time is studied using a recently developed technique based on Hamilton-Jacobi theory. Starting from the 1st order necessary conditions for optimality, we derive a Hamiltonian system for the state and adjoints with split boundary conditions. Then, recognizing the two point boundary value problem as a canonical transformation, we employ generating functions to find the optimal feedback control as well as the optimal trajectory. Though we formulate the optimal control problem in the context of the necessary conditions for optimality, our closed-loop solution also formally satisfies the sufficient conditions for optimality via the fundamental connection between the optimal cost function and generating functions. A solution procedure for these generating functions is posed and numerically tested on a non-linear optimal rendezvous problem in the vicinity of a circular orbit. Generating functions are developed as series expansions, and the optimal trajectories obtained from them are compared favorably with those of a numerical solution to the two point boundary value problem using a forward shooting method.

The derivation of necessary conditions of optimality for optimal control problems with state/control constraints is a challenging task, not always clearly understood. In this paper, we present a didactic approach based on the method of exponential penalty functions allowing, in our setting, to significantly simplify the proofs. This work may be viewed as a proposal of a course designed for students familiar with the basics of optimal control theory and elements of functional analysis. To make the analysis as simple as possible, we concentrate on the case of optimal control problem with fixed initial point and free end state.

1 A Survey on Computational Optimal Control.- Issues in the Direct Transcription of Optimal Control Problems to Sparse Nonlinear Programs.- Optimization in Control of Robots.- Large-scale SQP Methods and their Application in Trajectory Optimization.- Solving Optimal Control and Pursuit-Evasion Game Problems of High Complexity.- 2 Theoretical Aspects of Optimal Control and Nonlinear Programming.- Continuation Methods In Boundary Value Problems.- Second Order Optimality Conditions for Singular Extremals.- Synthesis of Adaptive Optimal Controls for Linear Dynamic Systems.- Control Applications of Reduced SQP Methods.- Time Optimal Control of Mechanical Systems.- 3 Algorithms for Optimal Control Calculations.- Second Order Algorithm for Time Optimal Control of a Linear System.- An SQP-type Solution Method for Constrained Discrete-Time Optimal Control Problems.- Numerical Methods for Solving Differential Games, Prospective Applications to Technical Problems.- Construction of the Optimal Feedback Controller for Constrained Optimal Control Problems with Unknown Disturbances.- Repetitive Optimization for Predictive Control of Dynamic Systems under Uncertainty.- Optimal Control of Multistage Systems Described by High-Index Differential-Algebraic Equations.- A New Class of a High Order Interior Point Method for the Solution of Convex Semiinfinite Optimization Problems.- A Structured Interior Point SQP Method for Nonlinear Optimal Control Problems.- 4 Software for Optimal Control Calculations.- Automated Approach for Optimizing Dynamic Systems.- ANDECS: A Computation Environment for Control Applications of Optimization.- Application of Automatic Differentiation to Optimal Control Problems.- OCCAL: A mixed symbolic-numeric Optimal Control CALculator.- 5 Applications of Optimal Control.- A Robotic Satellite with Simplified Design.- Nonlinear Control under Constraints of a Biological System.- An Object-Oriented Approach to Optimally Describe and Specify a SCADA System Applied to a Power Network.- Near-Optimal Flight Trajectories Generated by Neural Networks.- Performance of a Feedback Method with Respect to Changes in the Air-Density during the Ascent of a Two-Stage-To-Orbit Vehicle.- Linear Optimal Control for Reentry Flight.- Steady-State Modelling of Turbine Engine with Controllers.- Shortest Paths for Satellite Mounted Robot Manipulators.- Optimal Control of the Industrial Robot Manutec r3.