3
For $i = 1, \cdots ,d$, let $B_{s_i }^i $ be a one-dimensional Brownian motion on the interval $[0,a_i ]$ with absorption at the endpoints. At each instant in time, one must decide to run some subset of these d Brownian motions while holding the others fixed at their current state. The resulting process evolves in the rectangle $D = [0,a_1 ] \times \cdots \times [0,a_d ]$. If, at some instant, one decides to freeze all of the Brownian motions, then a reward is received in accordance with this final position. Two types of reward functions are considered. First, it is assumed that the reward is zero everywhere in D, except along the d edges that correspond to the coordinate axes. Along these edges, it is given by $C^3 $ strictly concave functions $\gamma _i (x_i )$, which are zero at the endpoints 0 and $a_i $ of their domains. The optimal control for this problem has a simple description. Let \[\Gamma _i \left( {x_i } \right) = - \int_0^{x_i } {u\gamma ''_i } (u)du\] and put \[M_i = \left\{ {x \in D:\Gamma _i \left( {x_i } \right) = \mathop {\max }\limits_j \Gamma _j \left( {x_i } \right)} \right\}.\] It is proved that the optimal control is: On $M_i $ run any Brownian motion except the ith and stop the first time an edge is reached. The second class of reward functions are assumed to be zero everywhere except on the facets of D that meet at the origin. On the ith such facet (i.e., where $x_i = 0$), the reward function is the product of $\gamma _j (x_j )$ for $j \ne i$. Put \[N_i = \left\{ {x \in D:\Gamma _i \left( {x_i } \right) = \mathop {\min }\limits_j \Gamma _j \left( {x_j } \right)} \right\}.\] The optimal control is: On $N_i $ run the ith Brownian motion and stop when a facet of D is reached.
11Markov processes conditioned on their location at large exponential times
Markov processes conditioned on their location at large exponential times
12Pitman transforms and Brownian motion in the interval viewed as an affine alcove
Pitman transforms and Brownian motion in the interval viewed as an affine alcove
On singularity as a function of time of a conditional distribution of an exit time
We establish the singularity with respect to Lebesgue measure as a function of time of the conditional probability distribution that the sum of two one-dimensional Brownian motions will exit from the unit interval before time t, given the trajectory of the second Brownian motion up to the same time. On the way of doing so we show that if one solves the one-dimensional heat equation with zero condition on a trajectory of a one-dimensional Brownian motion, which is the lateral boundary, then for each moment of time with probability one the normal derivative of the solution is zero, provided that the diffusion of the Brownian motion is sufficiently large.
Read more23On the Fourier structure of the zero set of fractional Brownian motion
On the Fourier structure of the zero set of fractional Brownian motion
231One-dimensional Brownian motion and the three-dimensional Bessel process
A simple path transformation is described which connects one-dimensional Brownian motion with the radial part of three-dimensional Brownian motion. This provides simple proofs of various path decompositions for these processes described by David Williams.
Read more1Capacitary Upper Estimates for Symmetric Dirichlet Forms
Upper estimates for capacities are studied for symmetric Dirichlet forms which do not necessarily admit intrinsic metric. Firstly, a general method of obtaining a sharp estimate from upper estimates for cut-off functions are provided in the local and non-local cases. Secondly, a capacitary upper estimate is established for the skew product of two symmetric Dirichlet forms for which suitable capacitary estimates are given. Several examples of capacitary upper inequalities in the local and non-local cases are also given. Especially, an upper estimate for the capacity is proved for the symmetric Dirichlet form associated to the Markov process subordinated (in the Bochner sense) to the skew product of two one-dimensional Brownian motions with respect to the local time of the first Brownian motion at the origin. This estimate is used in the recent work by the present author to establish the sharp criteria for recurrence and transience of the above-mentioned process.
Read more3Recurrence of multi-dimensional diffusion processes in Brownian environments
We consider limiting behavior of multi-dimensional diffusion processes in two types of Brownian environments. One is given values at different $d$ points of a one-dimensional Brownian motion, which is supposed to be a multi-parameter environment, and other is given by $d$ independent one-dimensional Brownian motions. We show recurrence of multi-dimensional diffusion processes in both Brownian environments above for any dimension and almost all environments. Their limiting behavior is quite different from that of ordinary multi-dimensional Brownian motion. We also consider cases of reflected Brownian environments.
Read more20Principal Values of the Integral Functionals of Brownian Motion: Existence, Continuity and an Extension of Itô’s Formula
Let B be a one-dimensional Brownian motion and \(f: {\mathbb{R}}{\longrightarrow}{\mathbb{R}}\) be a Borel function that is locally integrable on \(\mathbb{R}\backslash \{0\}\). We present necessary and sufficient conditions (in terms of the function f) for the existence of the limit $$\lim_{\varepsilon\downarrow 0}\int_0^t f(B_s)I(\vert B_s\vert > \varepsilon)ds$$ in probability and almost surely. This limit (if it exists) can be called the principal value of the integral \(\int_0^tf(B_s)ds\).
Read more10Optimal tracking control of switched systems applied in grid-connected hybrid generation using reinforcement learning
The paper presents a reinforcement learning approach for optimal tracking control of switched systems with application to a grid-tied hybrid generation system. To enhance interaction with the irregular environment, reference trajectory is learned via controller from states to optimal control. The main issue is to solve the optimal tracking control problem for a hybrid generation system consisting of multiple switched subsystems, and reinforcement learning can seek the globally optimal solution well without knowing accurate system dynamics. The investigated learning algorithm is used to generate an optimum map based on the learned ultimate value without knowledge of system parameters and obtains the optimal control law via deriving of algebraic Riccati equation (ARE) with unnecessary knowing of command generator dynamics. The optimal control solution can converge the online learning algorithm well based on policy iteration as verification in the simulation.
Read moreOptimal Control for Switched Uncertain Systems
Many practical systems operate by switching between different subsystems or modes. They are called switched systems. The optimal control problems of switched systems arise naturally when the control systems under consideration have multiple operating modes. A powertrain system [1] can also be viewed as a switched system which needs switching between different gears to achieve an objective such as fast and smooth acceleration response to the driver’s commands, low fuel consumption and low levels of pollutant emissions. For switched systems, the aim of optimal control is to seek both the optimal switching law and the optimal continuous input to optimize a certain performance criterion. Many successful algorithms have already been developed to seek the optimal control of switched systems. It is worth mentioning that Xu and Antsaklis [2] considered the optimal control of continuous-time switched systems. A two-stage optimization strategy was proposed in [2]. Stage (a) is a conventional optimal control problem under a given switching law and Stage (b) is a constrained nonlinear optimization problem that finds the local optimal switching instants. A general continuous-time switching problem was investigated in [3] based on the maximum principle and an embedding method. Furthermore, Teo et al. proposed a control parameterization technique [4] and the time scaling transform method [5] to find the approximate optimal control inputs and switching instants, which have been used extensively.
Read more36General Linear Quadratic Optimal Stochastic Control Problem Driven by a Brownian Motion and a Poisson Random Martingale Measure with Random Coefficients
In this article, we consider a linear-quadratic optimal control problem (LQ problem) for a controlled linear stochastic differential equation driven by a multidimensional Browinan motion and a Poisson random martingale measure in the general case, where the coefficients are allowed to be predictable processes or random matrices. By the duality technique, the dual characterization of the optimal control is derived by the optimality system (so-called stochastic Hamilton system), which turns out to be a linear fully coupled forward-backward stochastic differential equation with jumps. Using a decoupling technique, the connection between the stochastic Hamilton system and the associated Riccati equation is established. As a result, the state feedback representation is obtained for the optimal control. As the coefficients for the LQ problem are random, here, the associated Riccati equation is a highly nonlinear backward stochastic differential equation (BSDE) with jumps, where the generator depends on the unknown variables K, L, and H in a quadratic way (see (5.9) herein). For the case where the generator is bounded and is linearly dependent on the unknown martingale terms L and H, the existence and uniqueness of the solution for the associated Riccati equation are established by Bellman's principle of quasi-linearization.
Read more
2Maximizing the Mean Exit Time of a Brownian Motion from an Interval
Let X(t) be a controlled one-dimensional standard Brownian motion starting from x∈(−d,d). The problem of optimally controlling X(t) until |X(t)|=d for the first time is solved explicitly in a particular case. The maximal value that the instantaneous reward given for survival in (−d,d) can take is determined.
Read more31On the Potential Theory of One-dimensional Subordinate Brownian Motions with Continuous Components
Suppose that S is a subordinator with a nonzero drift and W is an independent 1-dimensional Brownian motion. We study the subordinate Brownian motion X defined by X t = W(S t ). We give sharp bounds for the Green function of the process X killed upon exiting a bounded open interval and prove a boundary Harnack principle. In the case when S is a stable subordinator with a positive drift, we prove sharp bounds for the Green function of X in (0, ∞ ), and sharp bounds for the Poisson kernel of X in a bounded open interval.
Read moreData-based control, optimization, modeling and applications
We are honored to organize this special issue of Neural Computation and Applications, based on contributions to the Ninth International Symposium on Neural Networks (ISNN 2012) held on July 11–14, 2012, in Shenyang, China. Evaluated by the contributions and the recommendation of ISNN 2012 organizers, 22 papers were selected for the further review process with their extended form. Each paper has been reviewed thoroughly by three independent experts in related research area. We are very grateful for the hard work of the reviewers, which help to facilitate the outcome of this special issue with greatly improved quality of the contributions. Finally, 20 papers are accepted, to be introduced as follows. In brief, the contributions of this special issue can be roughly classified into three major categories: data-based control, data-based optimization, and data-based modeling. In each category, not only the theoretical foundation is analyzed, but also some prospective applications are demonstrated. In this introduction, we like to borrow some ideas from the contributions, such as hierarchical, and cluster as the subsidiary category idea. The first category consisting of 6 papers is devoted to the data-based control and applications area, by which an optimal or near-optimal control can be derived from the input and output data. The first 4 papers focus on the hot research direction, adaptive dynamic programming and reinforcement learning (ADPRL), and the remaining 2 papers relate to iterative learning control and optimal control. Derong Liu et al. develop an online algorithm based on policy iteration for continuous-time optimal control with infinite horizon cost for nonlinear systems. In the proposed method, a discount value function is employed, which is considered to be a more general case for optimal control problems. Meanwhile, without knowledge of the internal system dynamics, the algorithm can converge uniformly online to the optimal solution of the modified Hamilton– Jacobi–Bellman equation. By means of two neural networks, the algorithm is able to find suitable approximations of both the optimal control and the optimal cost. Dual iterative adaptive dynamic programming for a class of discrete-time nonlinear systems with time-delays is investigated by Qinglai Wei et al. Dual iterative ADP algorithm is introduced to obtain the optimal solutions of the optimal performance index function and control, where in each iteration, the performance index function and the system states are both updated. Convergence analysis is presented to prove the performance index function to reach D. Zhao (&) State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China e-mail: Dongbin.zhao@ia.ac.cn
Read more27Best Tests for Zero Drift Based on First Passage Times in Brownian Motion
We give best tests and corresponding power functions for testing the hypothesis of no drift versus the alternative of positive linear drift in a one-dimensional Brownian motion process. The tests are based on a simple random sample from a first passage time distribution associated with the Brownian motion process. In reliability applications the first passage times would be the lifetimes of objects which fail when some noisy physical parameter reaches a critical value. In this case the life distribution is determined by the assumption that the physical parameter behaves like a Brownian motion and then the problem is equivalent to testing the hypothesis of infinite mean life versus the alternative of finite mean life. These tests are also applicable (and remain best) in the case of a single realization of Brownian motion for which only certain crossing time data is available.
Read more
Copyright 2026 Cactus Communications. All rights reserved.
