Modeling uncertain passenger arrivals in the elevator dispatching problem with destination control

In this paper, the stochastic optimal control problems, which frequently occur in economic and finance are investigated. First, using Bellman’s dynamic programming method the stochastic optimal control problems are converted to Hamilton-Jacobi-Bellman (HJB) equation. Then, obtained HJB equation is solved through the method of separation of variables by guessing a solution via its terminal condition. Also, the non-linear optimal feedback control law is constructed. Finally, the solution procedure is illustrated for solving some examples that two of them are financial models. In fact, to highlight the applications of stochastic optimal control problems in financial mathematics, some financial models are presented.

We introduce a novel generic methodology to solve continuous finite-horizon stochastic optimal control problems (SOCPs). We treat controlled stochastic differential equations (SDEs) within the Wiener chaos framework by utilizing Malliavin calculus and developing innovative ideas to preserve the feedback character of optimal Markov decision rules. This allows a direct reformulation of SOCPs into deterministic ones. Hence, it facilitates using Bock’s direct multiple shooting method for solving SOCPs and pioneers the extension of sophisticated methods for deterministic control to the broad context of SDEs. Numerical examples validate this new framework with huge computational advantages compared to standard ideas in SOC.

A global maximum principle for stochastic optimal control problems with delay and applications

A nonlinear stochastic control problem related to ow control is considered. It is assumed that the state of a link is described by a controlled hidden Markov process with a finite state set, while the loss ow is described by a counting process with intensity depending on a current transmission rate and an unobserved link state. The control is the transmission rate, and it has to be chosen as a nonanticipating process depending on the observation of the loss process. The aim of the control is to achieve the maximum of some utility function that takes into account losses of the transmitted information. Originally, the problem belongs to the class of stochastic control problems with incomplete information; however, optimal filtering equations that provide estimation of the current link state based on observations of the loss process allow one to reduce the problem to a standard stochastic control problem with full observations. Then a necessary optimality condition is derived in the form of a stochastic maximum principle, which allows us to obtain explicit analytic expressions for the optimal control in some particular cases. Optimal and suboptimal controls are investigated and compared with the ow control schemes used in TCP/IP (Transmission Control Protocols/Internet Protocols) networks. In particular, the optimal control demonstrates a much smoother behavior than the TCP/IP congestion control currently used.

Most of the existing literature concerning lifetime extension of battery energy storage (BES) devices mainly focus on designing controllers in BES devices themselves. In this paper, the problem of BES lifetime extension in an islanded microgrid is considered from a different perspective, in the sense that such target is achieved by adjusting output power of controllable power generation devices. To obtain accurate power models of photovoltaic panels and loads, a new hybrid modeling method utilizing both neural networks and stochastic differential equations is proposed. We formulate the problem of extending the service life of BES devices as a stochastic optimal control problem. Meanwhile, the situation of over-control in micro-turbines is effectively avoided. We solve such stochastic optimization problem using the dynamic programming solver provided by the open source C++ program BOCOPHJB. Finally, the obtained optimal controller is verified with simulations.

In this research article, we focus on a stochastic optimal control problem with two types of terminal constraints. These specific conditions provide real-valued and stochastic Lagrange multipliers. Our model evolves according to a Markov regime-switching jump diffusion model with memory. In this context, the memory is represented by a Stochastic Differential Delay Equation. We present two theorems for each constraint within the general formulation of stochastic optimal control theory in a Lagrangian environment. We approach to this task from a theoretical perspective and provide mild technical assumptions, which make our theorems applicable for a broad class of stochastic control problems as well as for a wide range of disciplines such as engineering, biology, operations research, medicine, computer science and economics. In this work, we apply Stochastic Maximum Principle to demonstrate an optimal dividend policy corresponding to a time-delayed wealth process of a company. Moreover, we determine the real-valued Lagrange multiplier of this control problem explicitly.

A stochastic minimax optimal time-delay state feedback control strategy for uncertain quasi-integrable Hamiltonian systems is proposed. First, a stochastic optimal state feedback control problem of uncertain quasi-integrable Hamiltonian system with time delay in feedback control subjected to Gaussian white noise is formulated. Then, the time-delayed state feedback control forces are approximated by the control forces without time delay and the original problem is converted into a stochastic optimal state feedback control problem of an uncertain system without time delay. After that, by following a procedure based on the stochastic averaging method and stochastic differential game, the worst-case disturbances and the optimal controls are obtained from solving a Hamiltonian–Jacobi–Isaacs (HJI) equation. As an example, the stochastic minimax optimal state feedback control of a Duffing oscillator with parametric disturbances and time-delayed feedback control is worked out in detail to illustrate the procedure and effectiveness of the proposed control strategy.

This article examines an economic growth model that expresses the interaction between production, technology stock, and research and development (R&D) investments. The goal of this study is to maximize production. Considering the presence of Gaussian white noises, this model is reformulated as a stochastic optimal control problem, where the R&D investment rate is defined as the control input. We aim to explore the efficiency of Kalman filtering approaches for solving this problem. Here, the extended Kalman filter (EKF) and unscented Kalman filter (UKF) are applied for state estimation. The state equation linearization is made in the EKF, while the unscented transform is taken in the UKF for generating a set of sigma points. These approaches aim to estimate the state dynamics from different perspectives. With these state estimates, two different computational algorithms are proposed, the EKF for state-control (EKF4SC) and UKF for state-control (UKF4SC) algorithms. The optimal control policy is designed to minimize the cost function. For illustration, the model's parameters are considered in the simulation experiment. The simulation results showed that the UKF has higher accuracy in the state estimation with the smallest mean squares of error compared with the EKF. Moreover, the optimal control policy based on the state estimate generated from the UKF could optimize the cost function of the problem. Hence, the results show the efficiency of the algorithms proposed. In conclusion, the application of the Kalman filtering algorithms to the economic growth model is presented. The significance of this study is to provide a stochastic optimal control model for the economic growth problem and to suggest an efficient computational approach for solving this stochastic optimal control problem.

A Class of Linear Quadratic Gaussian Hybrid Optimal Control Problems with Realization–Independent Riccati Equations

This article addresses the problem of quantifying the uncertainty in planning aircraft ground movement operations using towbarless robotic tractors taking into account the inherent uncertainties of the problem, specifically, the uncertainties in the weight of the aircraft and in the rolling resistance of the wheels of the main landing gear. The tractor‐aircraft system is represented as a tractor‐trailer system with random parameters characterized by means of probability density functions. The quantification of the uncertainties is conducted within an optimal control framework using the formulation of a stochastic optimal control problem, which is solved through a stochastic collocation method based on generalized polynomial chaos. Specifically, the stochastic optimal control problem is converted into a set of deterministic optimal control problems, in which a reduced number of sample values of the random parameters are employed to solve particular instances of the problem. Using these sample values, it is possible to express the obtained optimal solutions as orthogonal polynomial expansions in terms of the uncertain parameters, which allows both statistical and global sensitivity analysis of the stochastic optimal solutions to be carried out in an efficient way. The objectives of this article are to understand the effects of the uncertainties in the model parameters of the tractor‐aircraft system on the solutions of the optimal control problem and to identify which uncertain parameters have more influence on the variability of these solutions. The ultimate goal is to determine the best approach for executing aircraft ground movement operations in the presence of such uncertainties.

ABSTRACTThis article proposes a parallel algorithm of stochastic model predictive control (SMPC) for linear discrete‐time systems. The proposed algorithm uses multiple threads to solve respective optimization problems at different time steps in parallel, without inter‐thread communication during each optimization. Each thread sequentially solves a stochastic optimal control problem (SOCP) for optimizing closed‐loop performance against future disturbances and a partial minimization problem for optimizing a subset of decision variables of the SOCP. The partial minimization problem computes the control input according to the last measured state to satisfy chance constraints and can be solved much faster than the original SOCP. Consequently, the proposed algorithm can reduce the implementable minimum sampling period compared to the original SMPC. A numerical example of controlling room temperature showed that the proposed algorithm achieves control performance almost identical to that of closed‐loop SMPC, which optimizes closed‐loop performance. However, the implementable minimum sampling period of the proposed algorithm is much shorter than that of the closed‐loop SMPC, even with parallelization overhead.

In this paper, a numerical algorithm is presented to solve stochastic optimal control problems via the Markov chain approximation method. This process is based on state and time spaces discretization followed by a backward iteration technique. First, the original controlled process by an appropriate controlled Markov chain is approximated. Then, the cost functional is appropriate for the approximated Markov chain. Also, the finite difference approximations are used to the construction of locally consistent approximated Markov chain. Furthermore, the coefficients of the resulting discrete equation can be considered as the desired transition probabilities and interpolation interval. Finally, the performance of the presented algorithm on a test case with a well-known explicit solution, namely the Merton’s portfolio selection model, is demonstrated.

Development of adaptive dual predictive control schemes based on Wiener–Hammerstein models

In this paper, we present an Uzawa-based heuristic that is adapted to certain type of stochastic optimal control problems. More precisely, we consider dynamical systems that can be divided into small-scale subsystems linked through a static almost sure coupling constraint at each time step. This type of problem is common in production/portfolio management where subsystems are, for instance, power units, and one has to supply a stochastic power demand at each time step. We outline the framework of our approach and present promising numerical results on a simplified power management problem.

This work presents the approach to solving the functional equation of non-local type that appears in the study of a stochastic optimal control problem. This experience demonstrates the challenges and possible solutions in the analysis of optimal control problems with Poisson uncertainty.