Sufficiency for Purely Essentially Bounded Optimal Controls

This paper considers a general nonlinear econometric model framework that contains a large class of estimators defined as solutions to optimization problems. For this framework we derive several asymptotically equivalent forms of a test statistic for the local (in a way made precise in the paper) multivariate nonlinear inequality constraints testH:h(β) ≥ 0 versusK: β ∈RK. We extend these results to consider local hypotheses tests of the formH:h1(β) ≥ 0 andh2(β) = 0 versusK: β ∈RK. For each test we derive the asymptotic distribution for any size test as a weighted sum of χ2-distributions. We contrast local as opposed to global inequality constraints testing and give conditions on the model and constraints when each is possible. This paper also extends the well-known duality results in testing multivariate equality constraints to the case of nonlinear multivariate inequality constraints and combinations of nonlinear inequality and equality constraints.

In this study, a finite‐time online optimal controller was designed for a nonlinear wheeled mobile robotic system (WMRS) with inequality constraints, based on reinforcement learning (RL) neural networks. In addition, an extended cost function, obtained by introducing a penalty function to the original long‐time cost function, was proposed to deal with the optimal control problem of the system with inequality constraints. A novel Hamilton‐Jacobi‐Bellman (HJB) equation containing the constraint conditions was defined to determine the optimal control input. Furthermore, two neural networks (NNs), a critic and an actor NN, were established to approximate the extended cost function and the optimal control input, respectively. The adaptation laws of the critic and actor NN were obtained with the gradient descent method. The semi‐global practical finite‐time stability (SGPFS) was proved using Lyapunov's stability theory. The tracking error converges to a small region near zero within the constraints in a finite period. Finally, the effectiveness of the proposed optimal controller was verified by a simulation based on a practical wheeled mobile robot model.

This paper investigates the single-evader and multi-pursuer (SEMP) reach-avoid differential graphical (RADG) games of nonlinear heterogeneous players subject to saturated input and limited communication channels. The evader’s objective is to reach a designated target area while avoiding the pursuers, whose aim is to intercept the evader. First, we reformulate the SEMP RADG game as an optimal control problem within a weighted communication topology graph by incorporating the interception and control errors. Then, optimal control strategies that account for input saturation are derived by solving the coupled Hamilton-Jacobi (HJ) equations. These strategies are shown to constitute the Nash Equilibrium (NE) of the game. In addition, four types of pursuers, namely, isolated, passive, invisible, and regular pursuers, are defined based on the communication topology, and the conditions for achieving an Interactive Nash Equilibrium (INE), which is proposed for SEMP RADG games, are analyzed. Moreover, a single-network approximate dynamic programming (ADP) algorithm using concurrent learning (CL) is proposed to provide the near-optimal solutions to the coupled HJ equations. Asymptotic capture conditions are established through an examination of equilibrium points, and extensions to general pursuit-evasion (PE) games and half-space targets are further discussed. Our results are validated through numerical simulations.

The optimal control of a spacecraft as it transitions between specified states using continuous thrust in a fixed amount of time is studied using a recently developed technique based on Hamilton-Jacobi theory. Starting from the 1st order necessary conditions for optimality, we derive a Hamiltonian system for the state and adjoints with split boundary conditions. Then, recognizing the two point boundary value problem as a canonical transformation, we employ generating functions to find the optimal feedback control as well as the optimal trajectory. Though we formulate the optimal control problem in the context of the necessary conditions for optimality, our closed-loop solution also formally satisfies the sufficient conditions for optimality via the fundamental connection between the optimal cost function and generating functions. A solution procedure for these generating functions is posed and numerically tested on a non-linear optimal rendezvous problem in the vicinity of a circular orbit. Generating functions are developed as series expansions, and the optimal trajectories obtained from them are compared favorably with those of a numerical solution to the two point boundary value problem using a forward shooting method.

Adaptive nearly optimal control for a class of continuous-time nonaffine nonlinear systems with inequality constraints

A chebyshev polynomial method for optimal control with state constraints

Algorithm for overcoming the curse of dimensionality for state-dependent Hamilton-Jacobi equations

An optimal control method for distributed parameter systems is developed and applied to active vibration control of a plate. Piezoelectric material is used as an actuator. The structural dynamics are modeled using the space—time assumed mode expansion technique (a Rayleigh—Ritz method applied to both time and space variables). In particular, the generalized coordinates (both mechanical and electrical) and all generalized forces are described in terms of expansion functions. Using Hamilton's law of varying action along with the space—time assumed mode expansion results in algebraic equations of motion. These are then used as the constraint equations in the optimal controller design. Using space—time expansion, the usual variational optimal control problem is transformed into an equivalent algebraic problem. Optimal solutions are then obtained in a closed form and the solution is global within the time period considered. The solution procedure does not lead to a Riccati equation as is the case in the conventional optimal control solutions. The direct optimal control problem of a vibrating plate is illustrated through simulations where a parametric study is undertaken to check the controller performance. This study shows that the developed optimal control method is simple and the structure—control interaction is an inherent property of the method.

In general, optimal control problems rely on numerically rather than analytically solving methods, due to their nonlinearities. The direct method, one of the numerically solving methods, is mainly to transform the optimal control problem into a nonlinear optimization problem with finite dimensions, via discretizing the objective functional and the forced dynamical equations directly. However, in the procedure of the direct method, the classical discretizations of the forced equations will reduce or affect the accuracy of the resulting optimization problem as well as the discrete optimal control. In view of this fact, more accurate and efficient numerical algorithms should be employed to approximate the forced dynamical equations. As verified, the discrete variational difference schemes for forced Birkhoffian systems exhibit excellent numerical behaviors in terms of high accuracy, long-time stability and precise energy prediction. Thus, the forced dynamical equations in optimal control problems, after being represented as forced Birkhoffian equations, can be discretized according to the discrete variational difference schemes for forced Birkhoffian systems. Compared with the method of employing traditional difference schemes to discretize the forced dynamical equations, this way yields faithful nonlinear optimization problems and consequently gives accurate and efficient discrete optimal control. Subsequently, in the paper we are to apply the proposed method of numerically solving optimal control problems to the rendezvous and docking problem of spacecrafts. First, we make a reasonable simplification, i.e., the rendezvous and docking process of two spacecrafts is reduced to the problem of optimally transferring the chaser spacecraft with a continuously acting force from one circular orbit around the Earth to another one. During this transfer, the goal is to minimize the control effort. Second, the dynamical equations of the chaser spacecraft are represented as the form of the forced Birkhoffian equation. Then in this case, the discrete variational difference scheme for forced Birkhoffian system can be employed to discretize the chaser spacecraft's equations of motion. With further discretizing the control effort and the boundary conditions, the resulting nonlinear optimization problem is obtained. Finally, the optimization problem is solved directly by the nonlinear programming method and then the discrete optimal control is achieved. The obtained optimal control is efficient enough to realize the rendezvous and docking process, even though it is only an approximation of the continuous one. Simulation results fully verify the efficiency of the proposed method for numerically solving optimal control problems, if the fact that the time step is chosen to be very large to limit the dimension of the optimization problem is noted.

An approach to minimize the control costs and ensuring a stable deviation control is the Riccati controller and we use it to control constrained dynamical systems (differential algebraic equations of Index 3) with pulsed disturbances. To describe their discrete dynamics, a constrained variational integrator [1] is used. Using a discrete version of the Lagrange‐d'Alembert principle yields a forced constrained discrete Euler‐Lagrange equation in a position‐momentum form that depends on the current and future time steps [2]. The desired optimal trajectory (qopt, popt) and according control input uopt is determined solving the discrete mechanics and optimal control (DMOC) algorithm [3] based on the variational integrator. Then, during time stepping of the perturbed system, the discrete Riccati equation yields the optimal deviation control input uR. Adding uopt and uR to the discrete Euler‐Lagrange equation causes a structure preserving trajectory as both DMOC and Riccati equations are based on the same variational integrator. Furthermore, coordinate transformations are implemented (minimal, redundant and nullspace) enabling the choice of different coordinates in the feedback loop and in the optimal control problem. (© 2017 Wiley‐VCH Verlag GmbH & Co. KGaA, Weinheim)

This paper considers the pit shell optimization problem in open-pit mine planning within an optimal control framework. From a control perspective, we formulate the optimization problem by incorporating not only the actual workflows of open-pit mining activities but also various real-world constraints of operational and geological origin. Spatial discretization is applied to transform the continuous optimal control problem with partial differential equation dynamics into an ordinary differential equation. The resulting continuous optimal control problem is subsequently discretized into a nonlinear programming problem using the control parametrization method together with the constraint transcription technique. The nonlinear programming problem is then solved using a standard gradient-based method. Numerical simulations are conducted to evaluate the validity of the model and transformation, as well as the effectiveness of the gradient-based solution technique. Additionally, the limitations of the study and potential directions for future research are discussed.

In this paper, the distributed optimal containment control problem for multiple nonholonomic mobile robots (NHMRs) differential game is studied via reinforcement learning. An approximation-based optimal control strategy is developed to ensure the optimal performance index and avoid the potential collision among agents. Firstly, the collision avoidance problem considered in this paper is addressed by exploiting a consensus-like interconnection on a directed graph and an error transformation function. Then, on the basis of the optimal backstepping technique, a single critic neural network is adopted to obtain the solution of the coupled Hamilton-Jacobi (HJ) equation, in which an improved learning mechanism is constructed to relax the requirement on initial control conditions. In addition, based on the Lyapunov stability theory, it is proved that all signals in the closed-loop optimal control are uniformly ultimately bounded. Finally, the proposed control protocol is applied to NHMRs system, which verifies that the solution of the coupled HJ equation solves the containment problem of differential game.

An approach to dealing with the linear-quadratic optimal control problems for the continuous-time descriptor systems is proposed. A Riccati equation with a new form is established through the use of the necessary conditions for the optimal control. Unlike the existing approaches, transformation to the Riccati equation is made directly to find its solution. The sufficient conditions for the existence of the unique solution of the Riccati equation are found. The solution of the Riccati equation is then used in the construction of the optimal control strategies. It is believed that the full-order Riccati equation derived here may have some relation to the dynamic programming theory for the descriptor system. >

We prove the maximum principle for the problem outlined in the title with the aid (or more precisely in the class) of so-called sliding variations. INTRODUCTION The by-now classical maximum principle for the simplest nonlinear optimal control problem was formulated by Pontryagin and proved by Boltyanskii over 30 years ago (see [1, 2, 3] and also [8]). Since then, the maximum principle has been derived for various optimal control problems in an enormous number of studies. This is well-known and we do not give a full list of references here; we only mention [4-7, 9-24, 27-30]. In our view, the deepest and most thorough studies are those of Milyutin and Dubovitskii [9-14]. These authors advance very far in their development of the maximum . principle for problems with ordinary differential equations - first to problems with phase constraints [9] and then to the general problem with phase and arbitrary (in general, nonregular) mixed constraints [10-14]. These authors completely resolve the issue of deriving the necessary first-order conditions for these problems. Despite these achievements, we are still witnessing the publication of studies that derive the maximum principle for various problems which are particular cases of the general Dubovitskii-Milyutin problem. The authors of some of these studies possibly do not realize that they are rederiving well-established results.** This is partly due to the fact that, on the one hand, optimal control results are primarily interesting for applied engineers, while the references [9-14] are far from being accessible to every engineer, because they use a fairly complex mathematical apparatus which is not familiar to engineers; on the other hand, optimal control theory as such so far has failed to attract a sufficiently wide followign of mathematicians. "Optimizers" still feel that if a problem originates in engineering practice, then its solution (and preferably the process of its analysis) should remain in the domain of concepts which are familiar in the engineering environment (compare this situation with the totally different state of things in the theory of partial differential equations or in probability theory). This, in our view, accounts for repeated attempts to simplify the proof of the maximum principle. However, even if some studies achieve a certain simplification of the proof, this is the result of extreme simplification of the problem, and not an achievement of the particular method of analysis. The positive impact of these attempts is that they highlight the complexity of the problem. They have shown that the problem cannot be solved by elementary techniques, without invoking new methods (specifically, methods of functional analysis). We are thus forced to admit that the complexity of the work of Dubovitskii-Milyutin is not contrived: it is inherent to the complexity of the underlying problem. Yet there is a fairly wide class of problems for which the maximum principle can be derived relatively simply. These are the problems with pure phase constraints and so-called "regular" mixed constraints. They cover most of the problems considered in the literature. The maximum principle for these problems was derived by Milyutin and Dubovitskii back in the 1960s, immediately after their derivation of the maximum principle for problems with phase constraints [9]. However, it has never been published because, in the authors' opinion, the addition of regular mixed constraints to phase constraints did not constitute a significant advance.

An adaptive sparse grid local discontinuous Galerkin method for Hamilton-Jacobi equations in high dimensions