Singular ergodic control for multidimensional Gaussian–Poisson processes

Dynamic optimization and its relation to classical and quantum constrained systems

Consider the Hamilton–Jacobi–Bellman equation $\max _m \{ A_m u(x) - f_m (x)\} = 0$ a.e. in $R^n $, where $A_m (m = 1,2, \cdots )$ are the infinitesimal generators of diffusion processes with constant coefficients and with discount $c_m \geqq \alpha > 0$. It is known that the solution can be represented as the optimal cost functional in which one can switch from one stochastic system to another without penalty. In this paper it is shown that if, for some k, $A_k f_m (x) - A_m f_k (x) \geqq c > 0$ for all $m \ne k$, $| x | > R$, then $A_k u(x) - f_k (x) = 0$ if $| x | > R_1 $ for some $R_1 $ sufficiently large; that means that the optimal strategy when $| x | > R_1 $ is to stay with the diffusion and cost associated with $A_k $, $f_k $.

In this paper we investigate the existence and uniqueness of weak solutions of the nonautonomous Hamilton–Jacobi–Bellman equation on the domain $$(0,\infty ) \times \Omega $$ . The Hamiltonian is assumed to be merely measurable in time variable and the open set $$\Omega $$ may be unbounded with nonsmooth boundary. The set $$\overline{\Omega }$$ is called here a state constraint. When state constraints arise, then classical analysis of Hamilton–Jacobi–Bellman equation lacks appropriate notion of solution because continuous solutions could not exist. In this work we propose a notion of weak solution for which, under a suitable controllability assumption, existence and uniqueness theorems are valid in the class of lower semicontinuous functions vanishing at infinity.

7 - Numerical optimization in allocation, storage, and recovery of thermal energy and resources

We study a class of optimal control problems with state constraints, where the state equation is a differential equation with delays. This class includes some problems arising in economics, in particular, the so-called models with time to build; see [P. K. Asea and P. J. Zak, J. Econom. Dynam. Control, 23 (1999), pp. 1155–1175; M. Bambi, J. Econom. Dynam. Control, 32 (2008), pp. 1015–1040; F. E. Kydland and E. C. Prescott, Econometrica, 50 (1982), pp. 1345–1370]. We embed the problem in a suitable Hilbert space H and consider the associated Hamilton–Jacobi–Bellman (HJB) equation. This kind of infinite dimensional HJB equation has not been previously studied and is difficult due to the presence of state constraints and the lack of smoothing properties of the state equation. Our main result on the regularity of solutions to such an HJB equation seems to be entirely new. More precisely, we prove that the value function is continuous in a sufficiently big open set of H, that it solves in the viscosity sense the associated HJB equation, and that it has continuous classical derivative in the direction of the “present.” This regularity result is the starting point to define a feedback map in the classical sense, which gives rise to a candidate optimal feedback strategy.

In this article, we study the optimal control of stochastic differential equations with random impulses. We optimize the performance index and add the influence of random impulses to the performance index with a random compensation function. Using the idea of stochastic analysis and dynamic programming principle, a new Hamilton–Jacobi–Bellman (HJB) equation is obtained, and the existence and uniqueness of its viscosity solution are proved.

Design of nonlinear optimal control for chaotic synchronization of coupled stochastic neural networks via Hamilton–Jacobi–Bellman equation

Optimal control of Markovian jump systems via a neural network-based ADP iterative algorithm

Viscosity Solutions of Hamilton-Jacobi Equations in Infinite Dimensions. VII. The HJB Equation Is Not Always Satisfied

Hamilton–Jacobi–Bellman equation (HJBE) and backward stochastic differential equation (BSDE) are the two faces of stochastic control. We explore their equivalence focusing on a system of self-exciting and affine stochastic differential equations (SDEs) arising in streamflow dynamics. Our SDE is a finite-dimensional Markovian embedding of an infinite-dimensional jump-driven process called the superposition of continuous-state branching processes (a supCBI process). We formulate new ergodic control problems to evaluate the worst-case streamflow discharge in the long run and derive their HJBEs and ergodic BSDEs. The constant ambiguity aversion classically used in assessing model ambiguity must be modified in our case so that the optimality equations become well-posed. With a suitable modification of the ambiguity-aversion coefficient depending on the distributed reversion speed, we demonstrate that the solutions to the optimality equations are equivalent to each other in the sense that they lead to the same result. Finally, we apply the proposed framework to the computation of realistic cases with an existing record of discharge through a numerical Markovian embedding.

We investigate the optimal strategy over a finite time horizon for a portfolio of stock and bond and a derivative in an multiplicative Markovian market model with transaction costs (friction). The optimization problem is solved by a Hamilton–Jacobi–Bellman equation, which by the verification theorem has well-behaved solutions if certain conditions on a potential are satisfied. In the case at hand, these conditions simply imply arbitrage-free ("Black–Scholes") pricing of the derivative. While pricing is hence not changed by friction allow a portfolio to fluctuate around a delta hedge. In the limit of weak friction, we determine the optimal control to essentially be of two parts: a strong control, which tries to bring the stock-and-derivative portfolio towards a Black–Scholes delta hedge; and a weak control, which moves the portfolio by adding or subtracting a Black–Scholes hedge. For simplicity we assume growth-optimal investment criteria and quadratic friction.

Second order Hamilton–Jacobi–Bellman equations with an unbounded operator

This paper is concerned with the optimal portfolio problem for a company that can invest in two risky assets, where a novel Lévy-process-driven model is constructed to describe the dynamics of the wealth process by using a constant elasticity of variance model and a jump-diffusion process. A delicately designed value function is proposed under the mean–variance criterion to reflect the optimal portfolio for the stochastic volatility model. By using the verification theorem, the desired optimal portfolio strategy is proposed by the solution to certain Hamilton–Jacobi–Bellman equations. Furthermore, the corresponding expressions are achieved by using the stochastic analysis theory. Finally, a numerical simulation example is provided to verify the effectiveness of the proposed optimal portfolio strategy.

This chapter is devoted upon optimality, a topic in which the central result is the Pontryagin’s minimum principle. This important result is briefly approached from the classical calculus of variation. We show that the classical calculus of variation has some major limitations to modern control problems and motivate Pontryagin’s minimum principle. The Hamilton–Jacobi–Bellman method is discussed as an alternative approach to gain first-order necessary conditions for optimality. It is shown that both approaches correspond to each other under restrictive assumptions. The original Pontryagin’s minimum principle for continuous optimal control problems is not suitable for hybrid optimal control problems. However, a quite natural reformulation of the hybrid optimal control problem admits the classical theory for deduction of first-order necessary conditions in the sense of Pontryagin. The charm of this methodology is its comprehensible derivation.

We consider the optimal control problem of minimizing the terminal cost functional for a dynamical system whose motion is described by a differential equation with Caputo fractional derivative. The relationship between the necessary optimality condition in the form of Pontryagin's maximum principle and the Hamilton–Jacobi–Bellman equation with so-called fractional coinvariant derivatives is studied. It is proved that the costate variable in the Pontryagin maximum principle coincides, up to sign, with the fractional coinvariant gradient of the optimal result functional calculated along the optimal motion.