Koopman Operator Approach to Optimal Control Selection Under Uncertainty

We provide a data-driven framework for optimal control of a continuous-time control-affine stochastic dynamical system. The proposed framework relies on the linear operator theory involving Perron-Frobenius (P-F) and Koopman operators. Our first result involving the P-F operator provides a convex formulation to the optimal control problem in the dual space of densities. This convex formulation of the stochastic optimal control problem leads to an infinite-dimensional convex program. The finite-dimensional approximation of the convex program is obtained using a data-driven approximation of the linear operators. We provide rigorous error bounds and convergence rate for the data-driven approximation of the linear operators and comment on the convergence rate of the optimal control w.r.t. data size. Our second results demonstrate using the Koopman operator, dual to the P-F operator, for the stochastic optimal control design. We show that the Hamilton Jacobi Bellman (HJB) equation can be expressed using the Koopman operator. We provide an iterative procedure along the lines of a popular policy iteration algorithm based on the data-driven approximation of the Koopman operator for solving the HJB equation. The two formulations, namely the convex formulation involving the P-F operator and the Koopman-based formulation using the HJB equation, can be viewed as dual to each other where the duality follows due to the dual nature of the P-F and Koopman operators. Finally, we present examples to demonstrate the efficacy of the developed framework and verify the convergence rates for the operator and optimal control numerically.

The amount of digital video content available to users is rapidly increasing. Developments in computer, digital network, and storage technologies all contribute to broaden the offer of digital video. Only users’ attention and time remain scarce resources. Users face the problem of choosing the right content to watch among hundreds of potentially interesting offers. Video and audio have a dynamic nature: they cannot be properly perceived without considering their temporal dimension. This property makes it difficult to get a good idea of what a video item is about without watching it. Video previews aim at solving this issue by providing compact representations of video items that can help users making choices in massive content collections. This thesis is concerned with solving the problem of automatic creation of video previews. To allow fast and convenient content selection, a video preview should take into consideration more than thirty requirements that we have collected by analyzing related literature on video summarization and film production. The list has been completed with additional requirements elicited by interviewing end-users, experts and practitioners in the field of video editing and multimedia. This list represents our collection of user needs with respect to video previews. The requirements, presented from the point of view of the end-users, can be divided into seven categories: duration, continuity, priority, uniqueness, exclusion, structural, and temporal order. Duration requirements deal with the durations of the preview and its subparts. Continuity requirements request video previews to be as continuous as possible. Priority requirements indicate which content should be included in the preview to convey as much information as possible in the shortest time. Uniqueness requirements aim at maximizing the efficiency of the preview by minimizing redundancy. Exclusion requirements indicate which content should not be included in the preview. Structural requirements are concerned with the structural properties of video, while temporal order requirements set the order of the sequences included in the preview. Based on these requirements, we have introduced a formal model of video summarization specialized for the generation of video previews. The basic idea is to translate the requirements into score functions. Each score function is defined to have a non-positive value if a requirement is not met, and to increase depending on the degree of fulfillment of the requirement. A global objective function is then defined that combines all the score functions and the problem of generating a preview is translated into the problem of finding the parts of the initial content that maximize the objective function. Our solution approach is based on two main steps: preparation and selection. In the preparation step, the raw audiovisual data is analyzed and segmented into basic elements that are suitable for being included in a preview. The segmentation of the raw data is based on a shot-cut detection algorithm. In the selection step various content analysis algorithms are used to perform scene segmentation, advertisements detection and to extract numerical descriptors of the content that, introduced in the objective function, allow to estimate the quality of a video preview. The core part of the selection step is the optimization step that consists in searching the set of segments that maximizes the objective function in the space of all possible previews. Instead of solving the optimization problem exactly, an approximate solution is found by means of a local search algorithm using simulated annealing. We have performed a numerical evaluation of the quality of the solutions generated by our algorithm with respect to previews generated randomly or by selecting segments uniformly in time. The results on thirty content items have shown that the local search approach outperforms the other methods. However, based on this evaluation, we cannot conclude that the degree of fulfillment of the requirements achieved by our method satisfies the end-user needs completely. To validate our approach and assess end-user satisfaction, we conducted a user evaluation study in which we compared six aspects of previews generated using our algorithm to human-made previews and to previews generated by subsampling. The results have shown that previews generated using our optimization-based approach are not as good as manually made previews, but have higher quality than previews created using subsample. The differences between the previews are statistically significant.

In this chapter we first introduce Markov operators and study their general properties. Then we define Frobenius-Perron operators, a special class of Markov operators that will be mainly studied in the book. We also present a decomposition theorem for the Frobenius-Perron operator and its spectral analysis. We employ the Frobenius-Perron operator and its dual operator, the Koopman operator, to study ergodicity, mixing, and exactness of chaotic transformations.

This paper investigates L∞-estimates for the general optimal control problems governed by two-dimensional nonlinear elliptic equations with pointwise control constraints using mixed finite element methods. The state and the co-state are approximated by the lowest order Raviart-Thomas mixed finite element spaces and the control is approximated by piecewise constant functions. The authors derive L∞-estimates for the mixed finite element approximation of nonlinear optimal control problems. Finally, the numerical examples are given.

Solutions to optimal control problems obtained via direct methods are usually of low resolution. As the desired resolution becomes higher, the number of parameters in the resulting parameter optimization problem increases nonlinearly. A concatenated approach to trajectory optimization previously introduced by the authors to obtain high resolution optimal solutions with low storage requirements is applicable only to a specific class of problems, such as minimum time and minimum energy optimal control problems. The hodograph ballooning approach introduced in this paper generalizes the concatenated approach to the most general optimal control problem. Introduction The development of powerful nonlinear programming software [1, 2] has made direct approaches to trajectory optimization the method of choice for most practical applications [3-5]. The discretization of an optimal control problem to a finite dimensional parameter optimization problem avoids many of the troublesome pathologies that are typical for the optimization over infinite dimensional function spaces [6-8]. Furthermore, direct approaches usually do not. require guessing the switching structure, and the dynamical system is devoid of the nonintuitive costate variables. However, to achieve good resolution it is usually necessary to refine some type of discretization mesh, and, ultimately, to use a large number of design parameters. For typical gradient search methods, this increase in the dimension of the optimization problem has nearly always a negative effect on the convergence rate and the robustness. Additionally, CPU time and RAM requirements for each iteration increase with the second power of the number of parameters (third power for non-sparse solvers) so that direct optimization approaches can become quite expensive. In [9], a method was introduced that achieves excellent resolution without solving high-dimensional nonlinear programming problems. This was achieved by lo'Senior Engineer, Member AIAA. t Supervising Engineer, Member AIAA. 117 Research Drive, Hampton, VA 23666. Copyright ©1996 by the American Institute of Aeronautics and Astronautics, Inc. All rights reserved cally optimizing the trajectory on an iterative sequence of short, partly overlapping subarcs. However, this approach is not applicable to the most general problem formulation. Loosely sneaking, it was required in [9] that the cost state does not appear explicitly in the right-hand side of the equations of motion. The present paper introduces a method through which the approach introduced in [9] becomes applicable to the most general problem formulation. Application of this method, however, requires a priori knowledge of the optimal cost. In practice it is observed that the optimal cost can be obtained with very high precision with reasonably low order discretizations. Once a good guess for the optimal cost is obtained presented in this paper can be applied to refine the trajectory, or, if the optimal cost value is not known a priori with sufficient precision, it can be applied iteratively with the prescribed cost value adjusted in an outer loop. Problem Formulation Let us consider the following simple optimal control problem stated in Mayer form: ^ 4 ( x ( t f ) , t f ) (1 )

This paper presents an optimal feedback reentry guidance for the hypersonic vehicle. The proposed method implements a closed-loop feedback approach via fast computation of a series of open loop optimal control problems. At each sampling period, the optimal control problem is solved by the improved Gauss pseudospectral method (IGPM), which is an efficient method to solve the general nonlinear optimal control problem. The IGPM approximates the state and control by the polynomials at the Legendre-Gauss (LG). The costate can be obtained and the optimality of the solution is also checked easily. The proposed method can deal with the state and control constraints explicitly. The various performance can be realized by considering different cost function. Simulation results demonstrate that the hypersonic vehicle tracks the reference trajectory well in the presence of the uncertainty of aerodynamic force.

In this paper, we present a bilevel optimal motion planning (BOMP) model for autonomous parking. The BOMP model treats motion planning as an optimal control problem, in which the upper level is designed for vehicle nonlinear dynamics, and the lower level is for geometry collision-free constraints. The significant feature of the BOMP model is that the lower level is a linear programming problem that serves as a constraint for the upper-level problem. That is, an optimal control problem contains an embedded optimization problem as constraints. Traditional optimal control methods cannot solve the BOMP problem directly. Therefore, the modified approximate Karush–Kuhn–Tucker theory is applied to generate a general nonlinear optimal control problem. Then the pseudospectral optimal control method solves the converted problem. Particularly, the lower level is the J_2-function that acts as a distance function between convex polyhedron objects. Polyhedrons can approximate objects in higher precision than spheres or ellipsoids. As a result, a fast high-precision BOMP algorithm for autonomous parking concerning dynamical feasibility and collision-free property is proposed. Simulation results and experiment on Turtlebot3 validate the BOMP model, and demonstrate that the computation speed increases almost two orders of magnitude compared with the area criterion based collision avoidance method.

In this article, we consider a linear-quadratic optimal control problem (LQ problem) for a controlled linear stochastic differential equation driven by a multidimensional Browinan motion and a Poisson random martingale measure in the general case, where the coefficients are allowed to be predictable processes or random matrices. By the duality technique, the dual characterization of the optimal control is derived by the optimality system (so-called stochastic Hamilton system), which turns out to be a linear fully coupled forward-backward stochastic differential equation with jumps. Using a decoupling technique, the connection between the stochastic Hamilton system and the associated Riccati equation is established. As a result, the state feedback representation is obtained for the optimal control. As the coefficients for the LQ problem are random, here, the associated Riccati equation is a highly nonlinear backward stochastic differential equation (BSDE) with jumps, where the generator depends on the unknown variables K, L, and H in a quadratic way (see (5.9) herein). For the case where the generator is bounded and is linearly dependent on the unknown martingale terms L and H, the existence and uniqueness of the solution for the associated Riccati equation are established by Bellman's principle of quasi-linearization.

In this paper, a methodology for propagation of uncertainty in stochastic nonlinear dynamical systems is investigated. The process noise is approximated using Karhunen-Loeve (KL) expansion. Perron-Frobenius (PF) operator is used to predict the evolution of uncertainty. A multivariate Kolmogorov-Smirnov test is used to verify the proposed framework. The method is applied to predict uncertainty evolution in a Duffing oscillator and a Vanderpol's oscillator. It is observed that the solution of the approximated stochastic dynamics converges to the true solution in distribution. Finally, the proposed methodology is combined with Bayesian inference to estimate states of a nonlinear dynamical system, and its performance is compared with particle filter. The proposed estimator was found to be computationally superior than the particle filter.

Transportation problems are a crucial area in logistics and operations research that focuses on effectively transporting products and resources from origin to destination. The complex nature of real-world situations requires the utilization of advanced mathematical models and optimization approaches to formulate effective solutions. Quantifying the accurate supply, demand, and transportation costs of transportation problems is highly challenging due to the market’s unpredictable economic and environmental conditions. Several methodologies have been proposed to address the issue of transportation problems with uncertain parameters, utilizing fuzzy theory and its derivatives. Complex Pythagorean fuzzy numbers (CPyFNs) are highly suitable and effective for representing uncertain and ambiguous information in real-world scenarios. This article introduces an innovative analysis of transportation problems in the complex Pythagorean fuzzy information framework. Initially,we establish an improved score function value to evaluate the ordering of CPyFNs more precisely and propose several ranking rules. The following section of the study presents mathematical formulations and optimization models for transportation problems, where the parameters are expressed as CPyFNs. The solution approach is formulated using Vogel’s approximation method (VAM) and modified distribution method (MODI) to solve transportation problems in a complex Pythagorean fuzzy setting. An illustrative numerical example is provided to showcase the practicality and effectiveness of the proposed method in real-life scenarios. The significance of the research and the extent of future investigation are also emphasized.

In recent years data-driven analysis of dynamical systems has attracted a lot of attention and transfer operator techniques, namely, Perron-Frobenius and Koopman operators are being used almost ubiquitously. Since data is always obtained in discrete-time, in this paper, we propose a purely data-driven approach for the design of a stabilizing feedback control law for a general class of discrete-time control-affine non-linear systems. In particular, we use the Koopman operator to lift a control-affine system to a higher-dimensional space, where the control system's evolution is bilinear. We analyze the controllability of the lifted bilinear system and relate it to the controllability of the underlying non-linear system. We then leverage the concept of Control Lyapunov Function (CLF) to design a state feedback law that stabilizes the origin. Furthermore, we demonstrate the efficacy of the proposed method to stabilize the origin of the Van der Pol oscillator and the chaotic Henon map from the time-series data.

Pseudospectral approximation techniques have been shown to provide effective and flexible methods for solving optimal control problems in a variety of applications. In this paper, we provide the conditions for the convergence of the pseudospectral method for general nonlinear optimal control problems. Further, we show that this proof is directly extendible to the multidimensional pseudospectral method for optimal ensemble control of a class of parameterized dynamical systems. Examples from quantum control and neuroscience are included to demonstrate the method.

Existence, uniqueness, and stability of solutions to singular linear quadratic optimal control problems

Fractional optimal control problems via a wide class of fractional operators with a general analytic kernel are introduced. Necessary optimality conditions of Pontryagin type for the considered problem are obtained after proving a Gronwall type inequality as well as results on continuity and differentiability of perturbed trajectories. Moreover, a Mangasarian type sufficient global optimality condition for the general analytic kernel fractional optimal control problem is proved. An illustrative example is discussed.

An iterative numerical solution method is proposed for the general linear quadratic optimal control problem with phase constraints. The method is based on the extension principle. Realization features of the method are described for Matlab environment.