STOCHASTIC MODEL PREDICTIVE CONTROL AND PORTFOLIO OPTIMIZATION

We consider a finite-horizon continuous-time optimal control problem with nonlinear dynamics, an integral cost, control constraints and a time-varying parameter which represents perturbations or uncertainty. After time-discretization of the problem we employ a Model Predictive Control (MPC) algorithm with a shrinking horizon, which uses a "prediction"/forecast for the uncertain parameter and (possibly inexact) measurements of the state vector, and generates a piecewise constant control signal by solving auxiliary open-loop control problems. In our main result we derive an estimate of the difference between the MPC-generated control and the optimal feedback control, both obtained for the same value of the perturbation parameter, in terms of the step-size of the discretization and the measurement error. We also give an estimate for a norm of the difference between the MPC-generated control and the optimal open-loop control in the problem with the "true" value of the uncertain parameter, depending on the prediction error.

This paper presents a functional model predictive control (MPC) approach based on an adaptive dynamic programming (ADP) algorithm with the abilities of handling control constraints and disturbances for the optimal control of nonlinear discrete-time systems. In the proposed ADP-based nonlinear MPC (NMPC) structure, a neural-network-based identification is established first to reconstruct the unknown system dynamics. Then, the actor-critic scheme is adopted with a critic network to estimate the index performance function and an action network to approximate the optimal control input. Meanwhile, as the MPC strategy can effectively determine the current control by solving a finite horizon open-loop optimal control problem, in the proposed algorithm, the infinite horizon is decomposed into a series of finite horizons to obtain the optimal control. In each finite horizon, the finite ADP algorithm solves the optimal control problem subject to the terminal constraint, the control constraint, and the disturbance. The uniform ultimate boundedness of the closed-loop system is verified by the Lyapunov approach. Finally, the ADP-based NMPC is conducted on two different cases and the simulation results demonstrate the quick response and strong robustness of the proposed method.

This thesis aims at developing efficient mathematical methods for solving hierarchical dynamic optimization problems. The main motivation is to model processes in nature, for which there is evidence to assume that they run optimally. We describe models of such processes by optimal control problems (called optimal control models (OCMs)). However, an OCM typically includes unknown parameters that cannot be derived entirely on a theoretical basis, which is in particular the case for the cost function. Therefore, we develop parameter estimation techniques to estimate the unknowns in an OCM from observation data of the process. Mathematically, this leads to a hierarchical dynamic optimization problem with a parameter estimation problem on the upper level and an optimal control problem on the lower level. We focus on multi-stage equality and inequality constrained optimal control problems based on nonlinear ordinary differential equations. The main goal of this thesis is to derive numerically efficient mathematical methods for solving hierarchical dynamic optimization problems, and to use these methods to estimate parameters in high-dimensional OCMs from real-world measurement data. We develop parameter-dependent OCMs for the gait of cerebral palsy patients and able-bodied subjects. The unknown parameters in the OCMs are then estimated from real-world motion capture data provided by the Heidelberg MotionLab of the Orthopedic University Clinic Heidelberg by using the mathematical methods developed within this work. The main novelties and contributions of this thesis to the field of hierarchical dynamic optimization are summarized herein. - We establish a novel mathematical method, a so-called direct all-at-once approach, for solving hierarchical dynamic optimization problems based on the direct multiple shooting method and first-order optimality conditions. - Furthermore, we propose an efficient numerical algorithm for large-scale hierarchical dynamic optimization problems, which fully exploits the structures inherited from both the hierarchical setting and the discretization. - Pontryagin's maximum principle is used to analyze solution properties of hierarchical dynamic optimization problems like second-order optimality conditions of the lower-level problem. - In addition, we propose and discuss alternative methods for hierarchical dynamic optimization that are based on derivative-free optimization and a bundle approach. These methods keep the hierarchical problem setting and do not reformulate the lower-level problem using first-order optimality conditions. - We establish a novel lifting method for regularizing mathematical programs with complementarity constraints, which is discussed and numerically investigated by means of a well-known collection of benchmark problems. - Proofs of regularity and convergence results for sequential quadratic programming methods applied to lifted mathematical programs with complementarity constraints are provided. - Efficient state-of-the-art implementations of all mathematical methods derived in this thesis, as well as a benchmark collection of hierarchical dynamic optimization problems are presented. - High-dimensional optimal control gait models for cerebral palsy patients and able-bodied subjects are developed. The mathematical methods derived in this thesis are used to estimate the unknown model parameters from real-world motion capture data provided by the Heidelberg MotionLab of the Orthopedic University Clinic Heidelberg. The theoretical and practical results presented in this thesis can be considered an initial motivating step towards answering open questions in current medical research in fields like treatment planning, classification of gaits, or the evaluation of surgeries by means of hierarchical dynamic optimization.

1 A Survey on Computational Optimal Control.- Issues in the Direct Transcription of Optimal Control Problems to Sparse Nonlinear Programs.- Optimization in Control of Robots.- Large-scale SQP Methods and their Application in Trajectory Optimization.- Solving Optimal Control and Pursuit-Evasion Game Problems of High Complexity.- 2 Theoretical Aspects of Optimal Control and Nonlinear Programming.- Continuation Methods In Boundary Value Problems.- Second Order Optimality Conditions for Singular Extremals.- Synthesis of Adaptive Optimal Controls for Linear Dynamic Systems.- Control Applications of Reduced SQP Methods.- Time Optimal Control of Mechanical Systems.- 3 Algorithms for Optimal Control Calculations.- Second Order Algorithm for Time Optimal Control of a Linear System.- An SQP-type Solution Method for Constrained Discrete-Time Optimal Control Problems.- Numerical Methods for Solving Differential Games, Prospective Applications to Technical Problems.- Construction of the Optimal Feedback Controller for Constrained Optimal Control Problems with Unknown Disturbances.- Repetitive Optimization for Predictive Control of Dynamic Systems under Uncertainty.- Optimal Control of Multistage Systems Described by High-Index Differential-Algebraic Equations.- A New Class of a High Order Interior Point Method for the Solution of Convex Semiinfinite Optimization Problems.- A Structured Interior Point SQP Method for Nonlinear Optimal Control Problems.- 4 Software for Optimal Control Calculations.- Automated Approach for Optimizing Dynamic Systems.- ANDECS: A Computation Environment for Control Applications of Optimization.- Application of Automatic Differentiation to Optimal Control Problems.- OCCAL: A mixed symbolic-numeric Optimal Control CALculator.- 5 Applications of Optimal Control.- A Robotic Satellite with Simplified Design.- Nonlinear Control under Constraints of a Biological System.- An Object-Oriented Approach to Optimally Describe and Specify a SCADA System Applied to a Power Network.- Near-Optimal Flight Trajectories Generated by Neural Networks.- Performance of a Feedback Method with Respect to Changes in the Air-Density during the Ascent of a Two-Stage-To-Orbit Vehicle.- Linear Optimal Control for Reentry Flight.- Steady-State Modelling of Turbine Engine with Controllers.- Shortest Paths for Satellite Mounted Robot Manipulators.- Optimal Control of the Industrial Robot Manutec r3.

Receding horizon control (RHC) or model predictive control (MPC) solves online a finite horizon open-loop optimal control problem repeatedly in an infinite horizon context and provides a suboptimal control solution. It has been widely used in industry. For continuous-time (CT) systems, two categories of RHC have been investigated in literature, namely instantaneous RHC and sampled-data RHC. This paper focuses on the sampled-data RHC for continuous linear time-invariant (LTI) systems. The usual way to ensure stability of RHC scheme is to impose some constraints on the terminal cost and/or the terminal state and/or the horizon length. A new RHC algorithm is proposed, which we call updated terminal cost receding horizon control (UTC-RHC). Compared with the standard RHC which uses the same terminal cost all the time, our algorithm updates the terminal cost at each sampling instant. The advantage of UTC-RHC is that the uniform exponential stability of the closed-loop system can be guaranteed without imposing the usual constraints on the terminal state or the terminal cost or the horizon size. Moreover, the UTC-RHC control gain approaches the optimal value associated with the infinite horizon optimal control problems.

Unlike steady-state optimization and linear model predictive control, dynamic optimization and nonlinear model predictive control is underway to industry application. This paper discusses the implementation issues of two-level control strategy integrated dynamic real-time optimization and nonlinear model predictive control of a heat-integrated distillation column system. To realize this scheme, four nonlinear programming problems: economic operation optimization, data reconciliation and state estimation, parameter estimation and nonlinear model predictive control, are considered based on system rigorous dynamic model. The economic operation optimization is to minimize the cost of the plant. The unmeasured states are estimated by measurements of temperatures, pressures and flow rates. Time-variant parameters such as tray efficiencies and feed composition are updated on-line. The sequential quadratic programming and orthogonal collocation are used to solve the dynamic nonlinear optimization problems.

This paper presents a survey of the most recent literature developments in the field of computational guidance and control for space applications. In recent years, the space sector has changed significantly, shifting towards solutions requiring increased performance and autonomy. This change has also been accelerated by the technical and economic needs of the new space approach. However, traditional guidance and control techniques cannot always meet the increasing requirements of autonomy and performance. For this reason, computational guidance and control has become of increasing interest to the aerospace sector. In contrast to traditional guidance and control, computational guidance and control generates commands in real time by using on-board numerical computation. It represents, therefore, a fundamental step on the way to system autonomy and autonomous operations, with the potential to increase system capabilities and reduce operational costs. On-board computation introduces, however, a series of challenges, including the need to develop algorithms which are computationally efficient, reliable and robust. The literature has classified computational guidance and control techniques into two major categories, namely model-based and data-based. The model-based category encompasses all techniques in which the guidance and control laws are explicitly related to the system dynamic and kinematic models. On the contrary, data-based techniques infer guidance and control laws from a set of data, derived from the models, but do not explicitly use such models in the algorithm itself. Model-based guidance and control includes optimal control, model predictive control and convex optimisation. For the specific case of space attitude re-orientation manoeuvres, additional techniques are path planning and artificial potential functions. The general concept behind the Model Predictive Control (MPC) framework is to solve an optimal control problem repeatedly, for example whenever new measurements are available. The MPC implements only the first element of the returned solution, and the process is then repeated. By doing so, the system is better able to cope autonomously with model uncertainties, failures, or errors. For spacecraft with limited on-board capability, having to repeatedly solve an optimisation problem on-board could represent a challenge. In this case, explicit implementations of the MPC have been proposed, in which the guidance and control laws are precomputed offline and implemented on-board using look-up tables or simple functions. If the optimal control problem of the MPC is convex, or if it can be convexified, convex optimisation can be used directly. If the problem cannot be convexified, the solution can be found by sequentially solving a convex approximation of the original non-convex problem. Convex optimisation can be used either repeatedly in the context of an MPC framework, or as a single function call to generate guidance and control laws, e.g. at the beginning of an attitude re-orientation manoeuvre. Convex optimisation has received increased attention in recent years, also due to its suitability to on-board implementation. In fact, if the optimal control problem is convex, it can be solved reliably and efficiently on-board. Convex optimisation has been successfully applied to the space sector with the notable examples of the SpaceX rocket booster landing and of the Mars pinpoint landing demonstration aboard the Masten Xombie sounding rocket. Model-based guidance techniques, not based on optimisation, have been increasingly explored in recent years, as guidance strategies employed in other engineering fields, such as robotics, are being applied to spacecraft. Among these strategies, of particular importance are path planning and artificial potential field methods, due to their applicability to spacecraft autonomous attitude manoeuvres. Path planning algorithms have been applied for decades to ground vehicles and robotic manipulators. These algorithms consist of techniques to find the best route between two points of a graph representing the search space of a problem. For spacecraft re-orientation applications, the literature has proposed different attitude representation conventions and various cost functions associated to the graph search algorithm. Each approach comes with advantages and disadvantages, which will be analysed in the paper. Along with path planning, Artificial Potential Function (APF) techniques have been developed in the field of robotics, and are now also considered for space applications, to track or maintain a specific reference under a set of constraints. The main idea behind these methods is to synthetise the controller using a potential function. This function is specifically built to grant asymptotic stability to the system, according to Lyapunov’s second stability theorem. This formulation grants flexibility to accommodate system constraints, in a computationally light-weight and efficient manner. Data-based guidance and control laws are mainly based on Artificial Intelligence (AI) technologies, and generate guidance commands after training an algorithm with a large set of data. The interest of the space sector towards AI has accelerated in the last years due to the greater maturity of the technology. Some of the most powerful examples lie in the area of machine learning, like reinforcement learning, supervised learning and deep learning. These algorithms are used to extract patterns from large amounts of data derived from complex nonlinear dynamical and kinematical models. Examples of such data can be numerical solutions of an optimal control problem. The patterns extracted from the data allow the system to identify, in a short computational time, the critical parameters needed to relate the state and control variables. Creating a sufficiently large dataset for training the algorithm requires a considerable computational effort, though, and for certain methods optimality cannot be guaranteed due to the lack of theoretical support derived from the optimality conditions. Advancements made in the last years span several research directions and open a wide variety of opportunities and challenges, which need to be examined to guide further exploration. In this paper, both model-based and data-based techniques and applications are presented. The considered techniques include model predictive control, convex optimisation and machine learning. In addition, for the implementation of autonomous attitude manoeuvres, path planning methods and artificial potential function methods are analysed. These techniques have been applied, in the literature, to a range of different applications. The applications discussed in this paper include, in addition to attitude manoeuvres, powered descent landing guidance, low-thrust orbital transfers, proximity operations and station keeping.

In this paper, a learning-based predictive control (LPC) scheme is proposed for adaptive optimal control of discrete-time nonlinear systems under stochastic disturbances. The proposed LPC scheme is different from conventional model predictive control (MPC), which uses open-loop optimization or simplified closed-loop optimal control techniques in each horizon. In LPC, the control task in each horizon is formulated as a closed-loop nonlinear optimal control problem and a finite-horizon iterative reinforcement learning (RL) algorithm is developed to obtain the closed-loop optimal/suboptimal solutions. Therefore, in LPC, RL and adaptive dynamic programming (ADP) are used as a new class of closed-loop learning-based optimization techniques for nonlinear predictive control with stochastic disturbances. Moreover, LPC also decomposes the infinite-horizon optimal control problem in previous RL and ADP methods into a series of finite horizon problems, so that the computational costs are reduced and the learning efficiency can be improved. Convergence of the finite-horizon iterative RL algorithm in each prediction horizon and the Lyapunov stability of the closed-loop control system are proved. Moreover, by using successive policy updates between adjoint time horizons, LPC also has lower computational costs than conventional MPC which has independent optimization procedures between two different prediction horizons. Simulation results illustrate that compared with conventional nonlinear MPC as well as ADP, the proposed LPC scheme can obtain a better performance both in terms of policy optimality and computational efficiency.

In general, optimal control problems rely on numerically rather than analytically solving methods, due to their nonlinearities. The direct method, one of the numerically solving methods, is mainly to transform the optimal control problem into a nonlinear optimization problem with finite dimensions, via discretizing the objective functional and the forced dynamical equations directly. However, in the procedure of the direct method, the classical discretizations of the forced equations will reduce or affect the accuracy of the resulting optimization problem as well as the discrete optimal control. In view of this fact, more accurate and efficient numerical algorithms should be employed to approximate the forced dynamical equations. As verified, the discrete variational difference schemes for forced Birkhoffian systems exhibit excellent numerical behaviors in terms of high accuracy, long-time stability and precise energy prediction. Thus, the forced dynamical equations in optimal control problems, after being represented as forced Birkhoffian equations, can be discretized according to the discrete variational difference schemes for forced Birkhoffian systems. Compared with the method of employing traditional difference schemes to discretize the forced dynamical equations, this way yields faithful nonlinear optimization problems and consequently gives accurate and efficient discrete optimal control. Subsequently, in the paper we are to apply the proposed method of numerically solving optimal control problems to the rendezvous and docking problem of spacecrafts. First, we make a reasonable simplification, i.e., the rendezvous and docking process of two spacecrafts is reduced to the problem of optimally transferring the chaser spacecraft with a continuously acting force from one circular orbit around the Earth to another one. During this transfer, the goal is to minimize the control effort. Second, the dynamical equations of the chaser spacecraft are represented as the form of the forced Birkhoffian equation. Then in this case, the discrete variational difference scheme for forced Birkhoffian system can be employed to discretize the chaser spacecraft's equations of motion. With further discretizing the control effort and the boundary conditions, the resulting nonlinear optimization problem is obtained. Finally, the optimization problem is solved directly by the nonlinear programming method and then the discrete optimal control is achieved. The obtained optimal control is efficient enough to realize the rendezvous and docking process, even though it is only an approximation of the continuous one. Simulation results fully verify the efficiency of the proposed method for numerically solving optimal control problems, if the fact that the time step is chosen to be very large to limit the dimension of the optimization problem is noted.

— The objective of this paper is to investigate the Model Predictive Control (MPC) strategy, analyze and compare the control effects with Proportional-Integral-Derivative (PID) control strategy in maintaining a level of spherical tank system using VFD. An advanced control method, MPC has been widely used and well received in a wide variety of applications in process control, it utilizes an explicit process model to predict the future response of a process and solve an optimal control problem with a finite horizon at each sampling instant. In this paper, an algorithm is developed for controlling the speed of the pump using VFD to maintain the level of the spherical tank system. Next, the system is modeled and linearized for analysis and design. Model is implemented in a simulation environment based on LabVIEW. Both MPC and PID control methods are tried to design the controller for the water level system in a spherical tank and compared the results in terms of peak time, settling time, overshoot, and steady-state error under various operational conditions including time delays. The results showed the advantage of MPC for dealing with the system dynamic over PID and could be designed for more complex and fast system dynamics even in presence of constraints and Reduction of power consumption by the pump due to usage of VFD for controlling level by varying the speed of the pump based on the level of the tank. Keywords: Non Linear - Spherical Tank System – Level Control – Variable Frequency Drive – PID Controller – MPC Controller – Modeling - LabVIEW. Cite this Article R. Dhanasekar, S. Vijayachitra, M. Pravin Savaridass. Controlling of Level Process in Nonlinear System Using Variable Frequency Drive. Current Trends in Information Technology. 2019; 9(1): 24–33p.

ABSTRACTThis article considers a dynamic optimization problem arising in a one-dimensional magnetohydrodynamic flow, which can be modelled by coupled partial differential equations, where the external control input (external induction of magnetic field) takes a multiplicative effect on the system states (momentum and magnetic components). The aim is to drive the flow velocity to within close proximity of a desired target of flow velocity at the pre-indicated terminal time. First, the Galerkin method is utilized to reduce the original dynamic optimization problem to a lower finite-dimensional dynamic optimization problem governed by a set of ordinary differential equations. Then the control parameterization method is employed to parameterize the finite-dimensional optimization problem and thus obtain an approximate optimal parameter selection problem, which can be solved using gradient-based optimization techniques, such as sequential quadratic programming. The exact gradients of the cost functional with respect to the decision parameters, which are the key advantages of this approach, are computed using the analytical equations. Finally, numerical examples are illustrated to verify the effectiveness of the proposed method. The novel scheme of this article is to develop an effective computational optimal control method for realizing the optimal tracking control of flow velocity in a one-dimensional magnetohydrodynamic flow.

This article addresses the optimal tracking control problem with prescribed performance for uncertain nonlinear systems subject to input constraint and unknown disturbances. First, a fixed‐time monotonic convergence function is introduced to restrain tracking error, and a nonlinear mapping technique is employed to transform the constrained error into an unconstrained variable, then the fixed‐time output tracking issue is boiled down to the boundedness problem of the transformed variable. With the aid of a nonquadratic cost function, the input constraint is encoded into the optimization problem. To solve the unknown disturbances, an auxiliary system and an auxiliary disturbance policy are constructed, and the optimal control problem is formulated as a two‐player zero‐sum game. Moreover, a Hamilton–Jacobi–Isaacs (HJI) equation associated with this nonquadratic zero‐sum game is established to give the optimal control and the worst‐case disturbance policy solution. Subsequently, to avoid using knowledge of the system dynamics, three neural network approximators, namely, actor, critic, and disturbance, which are tuned online and simultaneously for approximating the solution of HJI, are constructed based on the integral reinforcement learning algorithm. Theoretical analysis shows that the reconstructed error system states and the weight estimation errors are semi‐globally uniformly ultimately bounded. Finally, the simulation study further tests the availability of the proposed control strategy.

Reservoir computing refers to a computational framework based on recurrent neural networks that can process time-series data. In an echo state network (ESN), which is a type of reservoir computing framework, the reservoir consists of a recursive network of artificial neurons with nonlinear activation functions. A model predictive control (MPC) technique can determine the control signals by solving the optimization problem of a system using the finite-time domain of each control period. However, real-time optimization cannot be achieved unless the optimal control problem can be solved within the next control period. To overcome this limitation, we propose a new control method based on MPC that explicitly incorporates the predicted disturbance of a time-varying trajectory using ESN to achieve the active vibration control of hybrid electric vehicle (HEV) powertrains. Once the ESN has been trained, the associated MPC explicitly satisfies the constraints over a moving horizon without further training. Instead of completing the real-time optimization within the control period, ESN predicts the future disturbance and applies it to the MPC in the future control period. Based on the predicted future disturbance, the system calculates the optimal control signals required for the future. Thus, real-time control can be realized because the optimal signals are determined before the subsequent control period occurs. The proposed method can be implemented in MPC even if the control period is too short to optimize as long as the disturbance can be reasonably measured and predicted. In this study, the simulation approach was demonstrated using the engine start condition in an HEV powertrain. The importance of this study is that the limitation of MPC relevant to real-time optimization can be relaxed by applying our proposed method.

This paper builds up two equivalence theorems for different kinds of optimal control problems of internally controlled Schrodinger equations. The first one is an equivalence of the minimal norm and the minimal time control problems where the target sets are the origin of the state space. (The minimal time control problems are also called the first type of optimal time control problems.) The second one is an equivalence of three optimal control problems: optimal target control problems, optimal norm control problems, and the second type of optimal time control problems.

This article presents a unified approach to simultaneous solution of optimization and optimal control problems in batch distillation, operating under different modes of operation: variable, constant, or optimal reflux. The simplified, computationally efficient short‐cut method and a novel algorithm to solve the optimal control problems in batch distillation is the basis of this unified approach. The short‐cut method identifies the feasible region of operation essential for optimization and optimal control problems, and provides analytical partial derivatives of the model parameters crucial to the solution.The new algorithm for the solution of optimal combination of the maximum principle and NLP optimization techniques. It circumvents the problems associated with the maximum principle approach (iterative solution of a two‐point boundary value problem, undounded control variables, and inability to handle the simultaneous optimization and optimal control problem), and the coupled ODE discretization‐NLP optimization scheme for nonlinear models (higher system nonlinearities, multiplicity of solutions, sensitivity of convergence to initial guesses). This algorithm reduces the dimensionality of the problem, and the nature of the algorithm allows a common platform to optimal solutions of different operating conditions. This article also shows that different categories of the optimal control problems in batch distillation essentially involve the solution of the maximum distillate problem.