Optimal Open-Loop Control of Discrete Deterministic Systems by Application of the Perch School Metaheuristic Optimization Algorithm

This paper is devoted to applications of modern methods of variational analysis to constrained optimization and control problems generally formulated in infinite-dimensional spaces. The main focus is on the study of problems with nonsmooth structures, which require the usage of advanced tools of generalized differentiation. In this way we derive new necessary optimality conditions in optimization problems with functional and operator constraints and then apply them to optimal control problems governed by discrete-time inclusions in infinite dimensions. The principal difference between finite-dimensional and infinite-dimensional frameworks of optimization and control consists of the “lack of compactness” in infinite dimensions, which leads to imposing certain “normal compactness” properties and developing their comprehensive calculus, together with appropriate calculus rules of generalized differentiation. On the other hand, one of the most important achievements of the paper consists of relaxing the latter assumptions for certain classes of optimization and control problems. In particular, we fully avoid the requirements of this type imposed on target endpoint sets in infinite-dimensional optimal control for discrete-time inclusions.

For Meyer type optimal control problems the principle of optimality states that every subarc of an optimal solution that extends to the final time furnishes again a solution to the same optimal control problem, only with different initial conditions. Through a simple counter example it is shown in this paper that this principle is not satisfied, in general, for solutions obtained from a standard collocation approach. Introduction The principle of optimality [1] is one of the most intuitive and yet most fundamental theorems in the theory of optimal control. Its basic statement is that for Meyer type optimal control problems every subarc of an optimal solution that extends to the final time furnishes again a solution to the same optimal control problem, only with different initial conditions. This theorem has significant implications. For example, it is the starting point for the derivation of the Hamilton-Jacobi-Bellman equations [2] and it is the underlying principle in Bellman's dynamic programming approach [1], [3]. The present paper shows that the principle of optimality is not satisfied, in general, for solutions obtained from a standard collocation approach[4]. For a simple example whose collocation solution can be generated analytically it is shown that further improvements are possible along an optimal collocation solution if the design parameters along a number of leading nodes are frozen and the remainder of the trajectory is re-optimized. I. Continuous Optimal Control Problem In this section of the paper we introduce a simple class of optimal control problems without state and control constraints. In Section 1.2 we present the associated necessary conditions for optimality [2], [5]. For ease of notation we assume that the right-hand side of the 'Supervising Engineer, Member AIAA, t Senior Engineer, Member AIAA, *17 Research Drive, Hampton, VA 23666. ^Graduate Student, Member AIAA, Department, 304 Town Engineering, Ames, IA 50011 equations of motion does not depend explicitly on time t. This does not represent a loss of generality, as it is always possible to transform explicit time dependence away through the introduction of an additional, timelike state variable. 1. Problem Formulation Let us consider the following class of optimal control problems: min «e(Pwc[*o,*/]), (1) subject to the conditions ±(t) = /(*(*),«(*)), tf.(z(*oMo) = 0, (2)

This paper studies a class of renewable resource allocation problems for the processing of dynamically arriving tasks with deterministic deadlines. This class of problems has many applications, however, it conforms to neither the standard resource allocation model nor the standard optimal control model. A new problem formulation has to be developed and analyzed to provide a satisfactory solution. In this paper, a new formulation is presented. By augmenting the state variabls, the problem is converted into a Markovian decision problem. It can then be treated, at least in principle, by using the stochastic dynamic programming (SDP) method. However, since the system dynamics involves the evolution of sets (of tasks and resources), the implementation of the dynamic programming equation is by no means straightforward. For a problem with infinite planning horizon, the optimal strategy is shown to be stationary under mild conditions. An SDP algorithm based on a successive approximation technique is developed to obtain the optimal stationary strategy. The implementation of the algorithm employs a special coding scheme to handle set variables, and utilizes a dominance property for computational efficiency. Effects of key system parameters on optimal decisions are investigated and analyzed through numerical examples. As the computational complexity of the algorithm is of exponential increase, practical applications of the algorithm is limited to problems of moderate size. Two heuristic rules are therefore investigated and compared to the optimal policy.

The stochastic optimal control solution to the Linear-Quadratic-Gaussian problem with partial information is considered where the system parameters operating on the control are assumed to be unknown Gaussian random processes. This class of problems has special properties that are exploited. First, the state and the random parameters are conditionally Gaussian with respect to the measurement history. This allows the optimal stochastic control problem to be reformulated as a stochastic optimal control problem of full information where the new state space is the state and parameter estimates and the error variance of the state and parameters. Even in this somewhat simple form, there is no closed form solution. Therefore, it is assumed that there is only a first order dependence on the random fluctuation of the parameter about its mean. In this way an asymptotic series solution is obtained in terms of the new state space. For this problem a closed form solution for the controller is obtained up to second order. The controller is derived for the scalar case and then extended to the case of vector state with single input and multiple outputs. The characteristics of the new solution is investigated and a scalar numerial example is given to assess the performance of this new controller.

Beyond Lack of Compactness and Lack of Stability of a Coupled Parabolic-Hyperbolic Fluid-Structure System.- A Continuous Adjoint Approach to Shape Optimization for Navier Stokes Flow.- Recent Advances in the Analysis of State-constrained Elliptic Optimal Control Problems.- Fast and Strongly Localized Observation for a perturbed Plate Equation.- Representations, Composition, and Decomposition of C 1,1-hypersurfaces.- On Some Nonlinear Optimal Control Problems with Vector-valued Affine Control Constraints.- Weak Solutions to a Model for Crystal Growth from the Melt in Changing Magnetic Fields.- Lavrentiev Prox-regularization Methods for Optimal Control Problems with Pointwise State Constraints.- Nonlinear Feedback Solutions for a Class of Quantum Control Problems.- Optimal Feedback Synthesis for Bolza Control Problem Arising in Linearized Fluid Structure Interaction.- Single-step One-shot Aerodynamic Shape Optimization.- Shape Differentiability of Drag Functional for Compressible Navier-Stokes Equations.- Null-controllability for a Coupled Heat-Finite-dimensional Beam System.- Feedback Modal Control of Partial Differential Equations.- Optimization Problems for Thin Elastic Structures.- A New Non-linear Semidefinite Programming Algorithm with an Application to Multidisciplinary Free Material Optimization.- How to Check Numerically the Sufficient Optimality Conditions for Infinite-dimensional Optimization Problems.- Hidden Boundary Shape Derivative for the Solution to Maxwell Equations and Non Cylindrical Wave Equations.

A nonlinear stochastic control problem related to ow control is considered. It is assumed that the state of a link is described by a controlled hidden Markov process with a finite state set, while the loss ow is described by a counting process with intensity depending on a current transmission rate and an unobserved link state. The control is the transmission rate, and it has to be chosen as a nonanticipating process depending on the observation of the loss process. The aim of the control is to achieve the maximum of some utility function that takes into account losses of the transmitted information. Originally, the problem belongs to the class of stochastic control problems with incomplete information; however, optimal filtering equations that provide estimation of the current link state based on observations of the loss process allow one to reduce the problem to a standard stochastic control problem with full observations. Then a necessary optimality condition is derived in the form of a stochastic maximum principle, which allows us to obtain explicit analytic expressions for the optimal control in some particular cases. Optimal and suboptimal controls are investigated and compared with the ow control schemes used in TCP/IP (Transmission Control Protocols/Internet Protocols) networks. In particular, the optimal control demonstrates a much smoother behavior than the TCP/IP congestion control currently used.

Discrete control problems with phase constraints

Optimality Conditions and Duality Models for a Class of Nonsmooth Constrained Fractional Optimal Control Problems

1 A Survey on Computational Optimal Control.- Issues in the Direct Transcription of Optimal Control Problems to Sparse Nonlinear Programs.- Optimization in Control of Robots.- Large-scale SQP Methods and their Application in Trajectory Optimization.- Solving Optimal Control and Pursuit-Evasion Game Problems of High Complexity.- 2 Theoretical Aspects of Optimal Control and Nonlinear Programming.- Continuation Methods In Boundary Value Problems.- Second Order Optimality Conditions for Singular Extremals.- Synthesis of Adaptive Optimal Controls for Linear Dynamic Systems.- Control Applications of Reduced SQP Methods.- Time Optimal Control of Mechanical Systems.- 3 Algorithms for Optimal Control Calculations.- Second Order Algorithm for Time Optimal Control of a Linear System.- An SQP-type Solution Method for Constrained Discrete-Time Optimal Control Problems.- Numerical Methods for Solving Differential Games, Prospective Applications to Technical Problems.- Construction of the Optimal Feedback Controller for Constrained Optimal Control Problems with Unknown Disturbances.- Repetitive Optimization for Predictive Control of Dynamic Systems under Uncertainty.- Optimal Control of Multistage Systems Described by High-Index Differential-Algebraic Equations.- A New Class of a High Order Interior Point Method for the Solution of Convex Semiinfinite Optimization Problems.- A Structured Interior Point SQP Method for Nonlinear Optimal Control Problems.- 4 Software for Optimal Control Calculations.- Automated Approach for Optimizing Dynamic Systems.- ANDECS: A Computation Environment for Control Applications of Optimization.- Application of Automatic Differentiation to Optimal Control Problems.- OCCAL: A mixed symbolic-numeric Optimal Control CALculator.- 5 Applications of Optimal Control.- A Robotic Satellite with Simplified Design.- Nonlinear Control under Constraints of a Biological System.- An Object-Oriented Approach to Optimally Describe and Specify a SCADA System Applied to a Power Network.- Near-Optimal Flight Trajectories Generated by Neural Networks.- Performance of a Feedback Method with Respect to Changes in the Air-Density during the Ascent of a Two-Stage-To-Orbit Vehicle.- Linear Optimal Control for Reentry Flight.- Steady-State Modelling of Turbine Engine with Controllers.- Shortest Paths for Satellite Mounted Robot Manipulators.- Optimal Control of the Industrial Robot Manutec r3.

An optimal control problem with a prescribed performance index for parabolic systems with time delays is investigated. A necessary condition for optimality is formulated and proved in the form of a maximum principle. Under additional conditions, the maximum principle gives sufficient conditions for optimality. It is also shown that the optimal control is unique. As an illustration of the theoretical consideration, an analytic solution is obtained for a time-delayed diffusion system.

An important class of applied problems is that of optimal control of some objects’ state. It is required to select control actions in such a way as to achieve desired effect. We deal with distributed systems, since their state is described by a partial differential equation. In this paper we study an iterative process for solving the problem of optimal control for an elliptic type system. Similar problem arises during the control of thermal processes. The quality of system state control is estimated by a given quality functional defined on the solution of the Dirichlet problem for an elliptic equation. One of the most important classes of thermal process control problems is temperature control, which means maintaining given temperature in the computational domain due to certain thermal effects. Here, a distributed internal heat source acts as a control. In the paper, we study statement correctness of the optimal control problem with a regularized functional. More precisely, we examine control problem for a system described by an elliptic type equation and formulate its optimality condition in the form of a system of equations for initial and conjugate states. An iterative method is proposed for solving the optimal control problem of an elliptic type system. Convergence of the iterative process is studied, and the rate of convergence is estimated.

In this paper, we consider a class of optimal switching control problems with multiple time-delays and a cost on changing control and subject to terminal state constraints. A computational method involving three stages is developed to solve this class of optimal control problems. First, by parameterizing the control function with piecewise-constant functions, the optimal switching control problem is approximated by a sequence of finite-dimensional optimization problems, where the original switching times, the control heights and the control switching times are decision variables. Second, by introducing new variables, the total variation of the control variables is transformed into an equivalently smooth function. Third, we convert the constrained optimization problem into one only with box constraints by an exact penalty function method. The gradients of the cost functional are then derived, which can be combined with any gradient-based optimization method to determine the optimal solution. Finally, a numerical example is given to illustrate the effectiveness of the proposed algorithm.

Path integral methods have recently been shown to be applicable to a very general class of optimal control problems. Here we examine the path integral formalism from a decision-theoretic point of view, since an optimal controller can always be regarded as an instance of a perfectly rational decision-maker that chooses its actions so as to maximize its expected utility. The problem with perfect rationality is, however, that finding optimal actions is often very difficult due to prohibitive computational resource costs that are not taken into account. In contrast, a bounded rational decision-maker has only limited resources and therefore needs to strike some compromise between the desired utility and the required resource costs. In particular, we suggest an information-theoretic measure of resource costs that can be derived axiomatically. As a consequence we obtain a variational principle for choice probabilities that trades off maximizing a given utility criterion and avoiding resource costs that arise due to deviating from initially given default choice probabilities. The resulting bounded rational policies are in general probabilistic. We show that the solutions found by the path integral formalism are such bounded rational policies. Furthermore, we show that the same formalism generalizes to discrete control problems, leading to linearly solvable bounded rational control policies in the case of Markov systems. Importantly, Bellman's optimality principle is not presupposed by this variational principle, but it can be derived as a limit case. This suggests that the information-theoretic formalization of bounded rationality might serve as a general principle in control design that unifies a number of recently reported approximate optimal control methods both in the continuous and discrete domain.

In general, optimal control problems rely on numerically rather than analytically solving methods, due to their nonlinearities. The direct method, one of the numerically solving methods, is mainly to transform the optimal control problem into a nonlinear optimization problem with finite dimensions, via discretizing the objective functional and the forced dynamical equations directly. However, in the procedure of the direct method, the classical discretizations of the forced equations will reduce or affect the accuracy of the resulting optimization problem as well as the discrete optimal control. In view of this fact, more accurate and efficient numerical algorithms should be employed to approximate the forced dynamical equations. As verified, the discrete variational difference schemes for forced Birkhoffian systems exhibit excellent numerical behaviors in terms of high accuracy, long-time stability and precise energy prediction. Thus, the forced dynamical equations in optimal control problems, after being represented as forced Birkhoffian equations, can be discretized according to the discrete variational difference schemes for forced Birkhoffian systems. Compared with the method of employing traditional difference schemes to discretize the forced dynamical equations, this way yields faithful nonlinear optimization problems and consequently gives accurate and efficient discrete optimal control. Subsequently, in the paper we are to apply the proposed method of numerically solving optimal control problems to the rendezvous and docking problem of spacecrafts. First, we make a reasonable simplification, i.e., the rendezvous and docking process of two spacecrafts is reduced to the problem of optimally transferring the chaser spacecraft with a continuously acting force from one circular orbit around the Earth to another one. During this transfer, the goal is to minimize the control effort. Second, the dynamical equations of the chaser spacecraft are represented as the form of the forced Birkhoffian equation. Then in this case, the discrete variational difference scheme for forced Birkhoffian system can be employed to discretize the chaser spacecraft's equations of motion. With further discretizing the control effort and the boundary conditions, the resulting nonlinear optimization problem is obtained. Finally, the optimization problem is solved directly by the nonlinear programming method and then the discrete optimal control is achieved. The obtained optimal control is efficient enough to realize the rendezvous and docking process, even though it is only an approximation of the continuous one. Simulation results fully verify the efficiency of the proposed method for numerically solving optimal control problems, if the fact that the time step is chosen to be very large to limit the dimension of the optimization problem is noted.

Q-learning solution for optimal consensus control of discrete-time multiagent systems using reinforcement learning