Finding an NARE whose minimal nonnegative solution represents first-passage increments in two-dimensional Markov modulated Brownian motion

In this paper, we consider the nonsymmetric algebraic Riccati equation whose four coefficient matrices form an M-matrix K. When K is a regular M-matrix, the Riccati equation is known to have a minimal nonnegative solution. We present two new numerical methods which can be applied directly in the case where K is a regular M-matrix. Furthermore, we find that the alternately linearized implicit iteration method is also feasible. In addition, we study the monotone convergence property of the proposed methods. Numerical experiments show that the above three numerical iteration methods are feasible and effective for solving the nonsymmetric algebraic Riccati equation.

On algebraic Riccati equations associated with regular singular M-matrices

In order to determine the stationary distribution for discrete time quasi-birth-death Markov chains, it is necessary to find the minimal nonnegative solution of a quadratic matrix equation. The Newton-Shamanskii method is applied to solve this equation, and the sequence of matrices produced is monotonically increasing and converges to its minimal nonnegative solution. Numerical results illustrate the effectiveness of this procedure.

This review article is concerned to provide a global context to several works on the fitting of continuous time nonhomogeneous Markov chains with finite state space and also to point out some selected aspects of two techniques previously introduced—estimation and calibration—relevant for applications and used to fit a continuous time Markov chain model to data by the adequate selection of parameters. The denomination estimation suits the procedure better when statistical techniques—e.g., maximum likelihood estimators—are employed, while calibration covers the case where, for instance, some optimisation technique finds a best approximation parameter to ensure good model fitting. For completeness, we provide a short summary of well-known important notions and results formulated for nonhomogeneous Markov chains that, in general, can be transferred to the homogeneous case. Then, as an illustration for the homogeneous case, we present a selected Billingsley’s result on parameter estimation for irreducible chains with finite state space. In the nonhomogeneous case, we quote two recent results, one of the calibration type and the other with more of a statistical flavour. We provide an ample set of bibliographic references so that the reader wanting to pursue her/his studies will be able to do so more easily and productively.

We report a theoretical study of the phase diffusion in a gain-switched single-mode semiconductor laser. We use stochastic rate equations for the electric field to analyze the phase statistics of the gain-switched laser. Their use avoids the instabilities obtained with rate equations for the photon number and optical phase when the photon number is small. However, we show that a new problem appears when the field equations are integrated: the variance of the optical phase becomes divergent. This divergence cannot be observed with the numerical integration of the commonly used equations for the photon number and optical phase because of the previous instabilities. The divergence of the phase variance means that this quantity does not reach a fixed value as the integration time step is decreased. We find that the phase variance increases as the integration time step decreases, with no sign of saturation behavior even for tiny steps. We explain the divergence by making the analogy of our problem with two-dimensional Brownian motion. The fact that the divergence appears is not surprising because in 1940 Paul L\`evy demonstrated that the variance of the polar angle in two-dimensional Brownian motion is a divergent quantity. Our results show that stochastic rate equations for the photon number and phase are not appropriate for describing the phase statistics when the photon number is small. Simulation of the stochastic rate equations for the electric field are consistent with L\`evy's results but gives unphysical results since an infinite value is obtained for a quantity that can be measured.

The theory of LΩEGF and the scaling theory on multidimensional Fokker-Planck equations with gradient potential are applied to derive the probability transition rate for Brownian motion in a symmetric two-dimensional bistable potential. Based on the rate method in adiabatic elimination, the phenomenon of stochastic resonance is studied in this system. Compared with the known results, the analytic expression for signal-to-noise ratio derived in this study is independent of the intensity of friction,i.e. not only applicable to an overdamped system, but also applicable to a lightly damped system. Numerical simulations fruther verify the significance of the analytically approximate result.

We have suggested that sine-wave based acuity gratings (such as Gabor functions) should be replaced by a new set of test functions - functions which act as the eigenfunctions of an appropriate operator (Stewart & Pinkham, in press). The Hermite functions emerge as an important special case, and have recently been applied to problems in image analysis (Martens, 1990). The Hermite functions are generated as the eigenfunctions of a differential operator which produces a simple and appealing mathematical model of space-variant acuity. The operator’s mathematical properties are in better agreement with the spatial inhomogeneity of the visual field than is the Fourier kernel. Surprisingly, these functions are their own Fourier transformations, and so are easily related to the Fourier transforms which are used in image analysis. If we are correct, human factors work on visual acuity can be placed on a coherent mathematical foundation, and new approaches to evaluating the quality of visual images will emerge. Just as important, a space-variant model of human visual acuity includes as a special case, the space-invariant models of image analysis used in engineering studies of image processing. The Hermite decomposition of the stimulus is directly related to the eigenfunctions of our space-variant acuity operator. The theory of Markov chains can be tied directly to the components of such a Hermite decomposition. The Hermite breakdown of two-dimensional and one-dimensional fractional Brownian motion produces components having corresponding spectra. Consequently, the information inherent in two- dimensional fractional Brownian motion is preserved under the Hermite decomposition, and finds a natural expression as a Markov process. Markov models are widely used for automatic speech recognition. Similar methods can be used to capture the inherent structure of visual images. Some natural scenes can be represented by fractals. Many images can be treated as

Tandem queues are good mathematical models for description of information transmission in various communication systems and networks. These queues play also an important role for the validation of different decomposition algorithms designed for investigating more general queueing networks. So, their investigation is interesting for theory and applications. In this paper, we consider tandem queue suitable for modeling the systems and networks where information flows are correlated and bursty what is typical for many modern telecommunication networks. Possible correlation of customers inter-arrival times and batch arrivals are taken into account via of consideration of the Batch Markovian Arrival Process (BMAP) as input stream to the system. The system consists of two stations. The service time at the station 1 is assumed to be generally distributed. There is no buffer at this station, and customers who meet the busy server repeat attempts to enter the system in random time intervals. The service process at the station 2 is assumed to be described by the continuous time Markov chain with a finite state space. This assumption holds good, e.g., if the station 2 has a finite buffer, consists of a finite number of identical or heterogeneous servers where the service time distribution is assumed to be of PH (PHase) type. Markov chain embedded at service completion epochs at the station 1 and the process of system states at arbitrary time are under study. Ergodicity condition and algorithms for computing the steady state probabilities are presented.

A batch Markov arrival process (BMAP) X* = (N, J) is a 2-dimensional Markov process with two components, one is the counting process N and the other one is the phase process J. It is proved that the phase process is a time-homogeneous Markov chain with a finite state-space, or for short, Markov chain. In this paper, a new and inverse problem is proposed firstly: given a Markov chain J, can we deploy a process N such that the 2-dimensional process X* = (N, J) is a BMAP? The process X* = (N, J) is said to be an adjoining BMAP for the Markov chain J. For a given Markov chain the adjoining processes exist and they are not unique. Two kinds of adjoining BMAPs have been constructed. One is the BMAPs with fixed constant batches, the other one is the BMAPs with independent and identically distributed (i.i.d) random batches. The method we used in this paper is not the usual matrix-analytic method of studying BMAP, it is a path-analytic method. We constructed directly sample paths of adjoining BMAPs. The expressions of characteristic (D k , k = 0, 1, 2 · · ·) and transition probabilities of the adjoining BMAP are obtained by the density matrix Q of the given Markov chain J. Moreover, we obtained two frontal Theorems. We present these expressions in the first time.

In this paper, we consider continuous‐time Markov chains with a finite state space under nonlinear expectations. We define so‐called Q‐operators as an extension of Q‐matrices or rate matrices to a nonlinear setup, where the nonlinearity is due to model uncertainty. The main result gives a full characterization of convex Q‐operators in terms of a positive maximum principle, a dual representation by means of Q‐matrices, time‐homogeneous Markov chains under convex expectations, and a class of nonlinear ordinary differential equations. This extends a classical characterization of generators of Markov chains to the case of model uncertainty in the generator. We further derive an explicit primal and dual representation of convex semigroups arising from Markov chains under convex expectations via the Fenchel–Legendre transformation of the generator. We illustrate the results with several numerical examples, where we compute price bounds for European contingent claims under model uncertainty in terms of the rate matrix.

Robust μ-stability analysis of Markovian switching uncertain stochastic genetic regulatory networks with unbounded time-varying delays

In this work, we establish a framework for balanced realization of linear systems subject to regime switching modulated by a continuous-time Markov chain with a finite state space. First, a definition of balanced realization is given. Then a ρ-balanced realization is developed to approximate the system of balancing equations, which is a system of time-varying algebraic equations. When the state space of the Markov chain is large, the computational effort becomes a real concern. To resolve this problem, we introduce a two-time-scale formulation and use decomposition/aggregation and averaging techniques to reduce the computational complexity. Based on the two-time-scale formulation, further approximation procedures are developed. Numerical examples are also presented for demonstration.

In this paper, we present several novel results on controllability and stabilizability properties of the Kolmogorov forward equation of a continuous time Markov chain (CTMC) evolving on a finite state space, using the transition rates as the control parameters. First, we characterize all the stationary distributions that are stabilizable using time-independent control parameters. We then present a result on small-time local and global controllability of the system from and to strictly positive equilibrium distributions when the underlying graph is strongly connected. Additionally, we show that any target distribution can be reached asymptotically using time-varying control parameters. For bidirected graphs, we construct rational and polynomial density feedback laws that stabilize strictly positive stationary distributions while satisfying the additional constraint that that the feedback law takes zero value at equilibrium. This last result enables the construction of decentralized density feedback controllers, using tools from linear systems theory and sum-of-squares based polynomial optimization, that stabilize a swarm of agents modeled as a CTMC to a target state distribution with no state-switching at equilibrium. We validate the effectiveness of the constructed feedback laws with stochastic simulations of the CTMC for finite numbers of agents and numerical solutions of the corresponding mean-field models.

In this chapter we consider a Markov chain {ξ k , k ∈ Z +} having a finite state space but random transition probabilities. Of interest here is the asymptotic behavior of the chain and its transition probabilities as n → ∞ under the assumption that the transition probabilities of the Markov chain are close to those of a homogeneous Markov chain having nonrandom transition probabilities.

It is well known that nonsymmetric algebraic Riccati equation can be efficiently solved by using the structure-preserving doubling algorithm (SDA) with the shift-and-shrink transformation and the corresponding doubling algorithm is called the SDA-ss. In this paper, we propose an improved SDA-ss algorithm. Preliminary numerical experiments show that the algorithm is efficient to derive the minimal nonnegative solution of nonsymmetric algebraic Riccati equation with M-matrix.