Data-Driven Optimal Control of Linear Time-Invariant Systems

SummaryA method is developed to numerically solve chance constrained optimal control problems. The chance constraints are reformulated as nonlinear constraints that retain the probability properties of the original constraint. The reformulation transforms the chance constrained optimal control problem into a deterministic optimal control problem that can be solved numerically. The new method developed in this paper approximates the chance constraints using Markov Chain Monte Carlo sampling and kernel density estimators whose kernels have integral functions that bound the indicator function. The nonlinear constraints resulting from the application of kernel density estimators are designed with bounds that do not violate the bounds of the original chance constraint. The method is tested on a nontrivial chance constrained modification of a soft lunar landing optimal control problem and the results are compared with results obtained using a conservative deterministic formulation of the optimal control problem. Additionally, the method is tested on a complex chance constrained unmanned aerial vehicle problem. The results show that this new method can be used to reliably solve chance constrained optimal control problems.

This paper investigates a reliability-based trajectory optimization method for the design of soft landing trajectory on an irregular shape asteroid with highly uncertain gravitational field. First, the gravitational field of the irregular asteroid is described by the finite particle model. Second, to avoid the singularity and reduce the sensitivity, the original finite particle model is modified to an “N-body/two-body” switching dynamic model. The trajectory optimization problem in the switching dynamic model is summarized as an optimal control problem and is then transformed into a two-point boundary value problem by Pontryagin’s maximum principle. By solving the two-point boundary value problem with a homotopic continuation procedure, the nominal optimal trajectory is obtained. Third, the uncertainty caused by the nonuniform mass distribution of the asteroid is considered. With high uncertainty, the deterministic optimal control problem becomes a parameter optimization problem with reliability constraints. This problem is then solved by a sequential optimization and reliability assessment, and the parallel computation technique is adopted. Finally, two soft landing trajectories are optimized using the proposed method to demonstrate the effectiveness of the technique.

In this paper, we present a bilevel optimal motion planning (BOMP) model for autonomous parking. The BOMP model treats motion planning as an optimal control problem, in which the upper level is designed for vehicle nonlinear dynamics, and the lower level is for geometry collision-free constraints. The significant feature of the BOMP model is that the lower level is a linear programming problem that serves as a constraint for the upper-level problem. That is, an optimal control problem contains an embedded optimization problem as constraints. Traditional optimal control methods cannot solve the BOMP problem directly. Therefore, the modified approximate Karush–Kuhn–Tucker theory is applied to generate a general nonlinear optimal control problem. Then the pseudospectral optimal control method solves the converted problem. Particularly, the lower level is the J_2-function that acts as a distance function between convex polyhedron objects. Polyhedrons can approximate objects in higher precision than spheres or ellipsoids. As a result, a fast high-precision BOMP algorithm for autonomous parking concerning dynamical feasibility and collision-free property is proposed. Simulation results and experiment on Turtlebot3 validate the BOMP model, and demonstrate that the computation speed increases almost two orders of magnitude compared with the area criterion based collision avoidance method.

In this paper, we investigate the problem of maximizing the expected exponential utility for an insurer. In the problem setting, the insurer can invest his/her wealth into the market and he/she can also purchase the proportional reinsurance. To control the risk exposure, we impose a value-at-risk constraint on the portfolio, which results in a constrained stochastic optimal control problem. It is difficult to solve a constrained stochastic optimal control problem by using traditional dynamic programming or Martingale approach. However, for the frequently used exponential utility function, we show that the problem can be simplified significantly using a decomposition approach. The problem is reduced to a deterministic constrained optimal control problem, and then to a finite dimensional optimization problem. To show the effectiveness of the approach proposed, we consider both complete and incomplete markets; the latter arises when the number of risky assets are fewer than the dimension of uncertainty. We also conduct numerical experiments to demonstrate the effect of the risk constraint on the optimal strategy.

This article presents a unified approach to simultaneous solution of optimization and optimal control problems in batch distillation, operating under different modes of operation: variable, constant, or optimal reflux. The simplified, computationally efficient short‐cut method and a novel algorithm to solve the optimal control problems in batch distillation is the basis of this unified approach. The short‐cut method identifies the feasible region of operation essential for optimization and optimal control problems, and provides analytical partial derivatives of the model parameters crucial to the solution.The new algorithm for the solution of optimal combination of the maximum principle and NLP optimization techniques. It circumvents the problems associated with the maximum principle approach (iterative solution of a two‐point boundary value problem, undounded control variables, and inability to handle the simultaneous optimization and optimal control problem), and the coupled ODE discretization‐NLP optimization scheme for nonlinear models (higher system nonlinearities, multiplicity of solutions, sensitivity of convergence to initial guesses). This algorithm reduces the dimensionality of the problem, and the nature of the algorithm allows a common platform to optimal solutions of different operating conditions. This article also shows that different categories of the optimal control problems in batch distillation essentially involve the solution of the maximum distillate problem.

1 A Survey on Computational Optimal Control.- Issues in the Direct Transcription of Optimal Control Problems to Sparse Nonlinear Programs.- Optimization in Control of Robots.- Large-scale SQP Methods and their Application in Trajectory Optimization.- Solving Optimal Control and Pursuit-Evasion Game Problems of High Complexity.- 2 Theoretical Aspects of Optimal Control and Nonlinear Programming.- Continuation Methods In Boundary Value Problems.- Second Order Optimality Conditions for Singular Extremals.- Synthesis of Adaptive Optimal Controls for Linear Dynamic Systems.- Control Applications of Reduced SQP Methods.- Time Optimal Control of Mechanical Systems.- 3 Algorithms for Optimal Control Calculations.- Second Order Algorithm for Time Optimal Control of a Linear System.- An SQP-type Solution Method for Constrained Discrete-Time Optimal Control Problems.- Numerical Methods for Solving Differential Games, Prospective Applications to Technical Problems.- Construction of the Optimal Feedback Controller for Constrained Optimal Control Problems with Unknown Disturbances.- Repetitive Optimization for Predictive Control of Dynamic Systems under Uncertainty.- Optimal Control of Multistage Systems Described by High-Index Differential-Algebraic Equations.- A New Class of a High Order Interior Point Method for the Solution of Convex Semiinfinite Optimization Problems.- A Structured Interior Point SQP Method for Nonlinear Optimal Control Problems.- 4 Software for Optimal Control Calculations.- Automated Approach for Optimizing Dynamic Systems.- ANDECS: A Computation Environment for Control Applications of Optimization.- Application of Automatic Differentiation to Optimal Control Problems.- OCCAL: A mixed symbolic-numeric Optimal Control CALculator.- 5 Applications of Optimal Control.- A Robotic Satellite with Simplified Design.- Nonlinear Control under Constraints of a Biological System.- An Object-Oriented Approach to Optimally Describe and Specify a SCADA System Applied to a Power Network.- Near-Optimal Flight Trajectories Generated by Neural Networks.- Performance of a Feedback Method with Respect to Changes in the Air-Density during the Ascent of a Two-Stage-To-Orbit Vehicle.- Linear Optimal Control for Reentry Flight.- Steady-State Modelling of Turbine Engine with Controllers.- Shortest Paths for Satellite Mounted Robot Manipulators.- Optimal Control of the Industrial Robot Manutec r3.

This paper builds up two equivalence theorems for different kinds of optimal control problems of internally controlled Schrodinger equations. The first one is an equivalence of the minimal norm and the minimal time control problems where the target sets are the origin of the state space. (The minimal time control problems are also called the first type of optimal time control problems.) The second one is an equivalence of three optimal control problems: optimal target control problems, optimal norm control problems, and the second type of optimal time control problems.

In general, optimal control problems rely on numerically rather than analytically solving methods, due to their nonlinearities. The direct method, one of the numerically solving methods, is mainly to transform the optimal control problem into a nonlinear optimization problem with finite dimensions, via discretizing the objective functional and the forced dynamical equations directly. However, in the procedure of the direct method, the classical discretizations of the forced equations will reduce or affect the accuracy of the resulting optimization problem as well as the discrete optimal control. In view of this fact, more accurate and efficient numerical algorithms should be employed to approximate the forced dynamical equations. As verified, the discrete variational difference schemes for forced Birkhoffian systems exhibit excellent numerical behaviors in terms of high accuracy, long-time stability and precise energy prediction. Thus, the forced dynamical equations in optimal control problems, after being represented as forced Birkhoffian equations, can be discretized according to the discrete variational difference schemes for forced Birkhoffian systems. Compared with the method of employing traditional difference schemes to discretize the forced dynamical equations, this way yields faithful nonlinear optimization problems and consequently gives accurate and efficient discrete optimal control. Subsequently, in the paper we are to apply the proposed method of numerically solving optimal control problems to the rendezvous and docking problem of spacecrafts. First, we make a reasonable simplification, i.e., the rendezvous and docking process of two spacecrafts is reduced to the problem of optimally transferring the chaser spacecraft with a continuously acting force from one circular orbit around the Earth to another one. During this transfer, the goal is to minimize the control effort. Second, the dynamical equations of the chaser spacecraft are represented as the form of the forced Birkhoffian equation. Then in this case, the discrete variational difference scheme for forced Birkhoffian system can be employed to discretize the chaser spacecraft's equations of motion. With further discretizing the control effort and the boundary conditions, the resulting nonlinear optimization problem is obtained. Finally, the optimization problem is solved directly by the nonlinear programming method and then the discrete optimal control is achieved. The obtained optimal control is efficient enough to realize the rendezvous and docking process, even though it is only an approximation of the continuous one. Simulation results fully verify the efficiency of the proposed method for numerically solving optimal control problems, if the fact that the time step is chosen to be very large to limit the dimension of the optimization problem is noted.

In this paper, the task of achieving the soft landing of a lunar module such that the fuel consumption and the flight time are minimized is formulated as an optimal control problem. The motion of the lunar module is described in a three dimensional coordinate system. We obtain the form of the optimal closed loop control law, where a feedback gain matrix is involved. It is then shown that this feedback gain matrix satisfies a Riccati-like matrix differential equation. The optimal control problem is first solved as an open loop optimal control problem by using a time scaling transform and the control parameterization method. Then, by virtue of the relationship between the optimal open loop control and the optimal closed loop control along the optimal trajectory, we present a practical method to calculate an approximate optimal feedback gain matrix, without having to solve an optimal control problem involving the complex Riccati-like matrix differential equation coupled with the original system dynamics. Simulation results show that the proposed approach is highly effective.

In Chapters 10 to 15, we have shown the convergence of properly designed numerical approximations for a wide range of stochastic and deterministic optimal control problems. The approach to proving the convergence has been based on demonstrating the convergence of a sequence of controlled Markov chains to a controlled process (diffusion, jump diffusion, etc.) appropriate to the given stochastic or deterministic optimal control problem.KeywordsMarkov ChainNumerical SchemeViscosity SolutionBellman EquationFinite Difference ApproximationThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Decomposition method for solving two optimal control problems and one optimization problem in batch fermentation is proposed. The problems are formulated based on a nonstructured mathematical model with slowly varying parameters and a finite cost criterion of maximum end production. Dependence of the model parameters on one physical or chemical parameter, which could easily be used as a control input is introduced analytically in the model equations and three model descriptions are obtained by nonlinear difference equations. Sensitivity functions of state trajectories towards slowly varying coefficients are introduced to account for model uncertainties. Based on them extended optimal control and optimization problems are formulated.

In this paper, we present robust adaptive controller design for SISO linear systems with zero relative degree under noisy output measurements. We formulate the robust adaptive control problem as a nonlinear H/sup /spl infin// optimal control problem under imperfect state measurements, and then solve it using game theory. By using the a priori knowledge of the parameter vector, we apply a soft projection algorithm, which guarantees the robustness property of the closed-loop system without any persistency of excitation assumption on the reference signal. Due to our formulation in state space, we allow the true system to be uncontrollable, as long as the uncontrollable part is stable in the sense of Lyapunov, and the uncontrollable modes on the jw-axis are uncontrollable from the exogenous disturbance input. This assumption allows the adaptive controller to asymptotically cancel out, at the output, the effect of exogenous sinusoidal disturbance, inputs with unknown magnitude, phase, and frequency. These strong robustness properties are illustrated by a numerical example.

Optimal control simulations have shown that both musculoskeletal dynamics and physiological noise are important determinants of movement. However, due to the limited efficiency of available computational tools, deterministic simulations of movement focus on accurately modelling the musculoskeletal system while neglecting physiological noise, and stochastic simulations account for noise while simplifying the dynamics. We took advantage of recent approaches where stochastic optimal control problems are approximated using deterministic optimal control problems, which can be solved efficiently using direct collocation. We were thus able to extend predictions of stochastic optimal control as a theory of motor coordination to include muscle coordination and movement patterns emerging from non-linear musculoskeletal dynamics. In stochastic optimal control simulations of human standing balance, we demonstrated that the inclusion of muscle dynamics can predict muscle co-contraction as minimal effort strategy that complements sensorimotor feedback control in the presence of sensory noise. In simulations of reaching, we demonstrated that nonlinear multi-segment musculoskeletal dynamics enables complex perturbed and unperturbed reach trajectories under a variety of task conditions to be predicted. In both behaviors, we demonstrated how interactions between task constraint, sensory noise, and the intrinsic properties of muscle influence optimal muscle coordination patterns, including muscle co-contraction, and the resulting movement trajectories. Our approach enables a true minimum effort solution to be identified as task constraints, such as movement accuracy, can be explicitly imposed, rather than being approximated using penalty terms in the cost function. Our approximate stochastic optimal control framework predicts complex features, not captured by previous simulation approaches, providing a generalizable and valuable tool to study how musculoskeletal dynamics and physiological noise may alter neural control of movement in both healthy and pathological movements.

The transfer-function form of the stationary algebraic Riccati equation is investigated. A generalized spectral factorization technique that copes with unstable systems is introduced. This factorization is used to provide an efficient way of solving the Riccati equation and to establish the exact equivalence between the time domain and the transfer-function approaches to the linear stationary filtering and the deterministic optimal control problems. The proposed method is easily extended to cope, in the filtering problem, with coloured signals and the superiority of its computational method for systems having a small number of inputs and outputs is demonstrated. Finally, an application of the transfer-function approach in determining the class of all systems that share the same optimal solution is introduced.

Solutions to optimal control problems obtained via direct methods are usually of low resolution. As the desired resolution becomes higher, the number of parameters in the resulting parameter optimization problem increases nonlinearly. A concatenated approach to trajectory optimization previously introduced by the authors to obtain high resolution optimal solutions with low storage requirements is applicable only to a specific class of problems, such as minimum time and minimum energy optimal control problems. The hodograph ballooning approach introduced in this paper generalizes the concatenated approach to the most general optimal control problem. Introduction The development of powerful nonlinear programming software [1, 2] has made direct approaches to trajectory optimization the method of choice for most practical applications [3-5]. The discretization of an optimal control problem to a finite dimensional parameter optimization problem avoids many of the troublesome pathologies that are typical for the optimization over infinite dimensional function spaces [6-8]. Furthermore, direct approaches usually do not. require guessing the switching structure, and the dynamical system is devoid of the nonintuitive costate variables. However, to achieve good resolution it is usually necessary to refine some type of discretization mesh, and, ultimately, to use a large number of design parameters. For typical gradient search methods, this increase in the dimension of the optimization problem has nearly always a negative effect on the convergence rate and the robustness. Additionally, CPU time and RAM requirements for each iteration increase with the second power of the number of parameters (third power for non-sparse solvers) so that direct optimization approaches can become quite expensive. In [9], a method was introduced that achieves excellent resolution without solving high-dimensional nonlinear programming problems. This was achieved by lo'Senior Engineer, Member AIAA. t Supervising Engineer, Member AIAA. 117 Research Drive, Hampton, VA 23666. Copyright ©1996 by the American Institute of Aeronautics and Astronautics, Inc. All rights reserved cally optimizing the trajectory on an iterative sequence of short, partly overlapping subarcs. However, this approach is not applicable to the most general problem formulation. Loosely sneaking, it was required in [9] that the cost state does not appear explicitly in the right-hand side of the equations of motion. The present paper introduces a method through which the approach introduced in [9] becomes applicable to the most general problem formulation. Application of this method, however, requires a priori knowledge of the optimal cost. In practice it is observed that the optimal cost can be obtained with very high precision with reasonably low order discretizations. Once a good guess for the optimal cost is obtained presented in this paper can be applied to refine the trajectory, or, if the optimal cost value is not known a priori with sufficient precision, it can be applied iteratively with the prescribed cost value adjusted in an outer loop. Problem Formulation Let us consider the following simple optimal control problem stated in Mayer form: ^ 4 ( x ( t f ) , t f ) (1 )