Markov chains under nonlinear expectation

Pairwise sequence alignment forms the basis of numerous other applications in bioinformatics. The quality of an alignment is gauged by statistical significance rather than by alignment score alone. Therefore, accurate estimation of statistical significance of a pairwise alignment is an important problem in sequence comparison. Recently, it was shown that pairwise statistical significance does better in practice than database statistical significance, and also provides quicker individual pairwise estimates of statistical significance without having to perform time-consuming database search. Under an evolutionary model, a substitution matrix can be derived using a rate matrix and a fixed distance. Although the commonly used substitution matrices like BLOSUM62, etc. were not originally derived from a rate matrix under an evolutionary model, the corresponding rate matrices can be back calculated. Many researchers have derived different rate matrices using different methods and data. In this paper, we show that pairwise statistical significance using rate matrices with sequence-pair-specific distance performs significantly better compared to using a fixed distance. Pairwise statistical significance using sequence-pair-specific distanced substitution matrices also outperforms database statistical significance reported by BLAST.

Solution of nonlinear partial differential equations from base equations

Mathematically, stochastic hybrid systems can be described by Markov processes with discrete and continuous behaviours. This chapter provides in an incremental way the necessary mathematical background for understanding such complex systems. The chapter starts with discrete and continuous-time Markov chains. In this book, a Markov chain is understood as a stochastic process with the Markov property (or memoryless property, i.e. its future evolution depends only on the current state) defined on a discrete (finite or countable) state space. The Markov property is illustrated in the Chapman-Kolmogorov equation satisfied by the transition probabilities. Usually, in the continuous-time Markov case, the evolution of these transition probabilities is described by forward/backward Kolmogorov equations. These equations are expressed in terms of the stochastic matrix (called also infinitesimal generator) associated to the Markov chain. This is the matrix of transition rates and the practical use of Markov chains resides in our ability of handling it.The second part of this chapter is dedicated to Markov processes defined on continuous state spaces that can be equipped with additional structures as sigma-algebras, metrics, topologies, norms or others like specific algebraic structures (lattice, vector space). Characterisations of such processes are usually given using functional analysis operators like: the operator semigroup/resolvent, the infinitesimal generator or other operators that can be associated to a Markov process. At an abstract level, these operators describe the evolution the evolution of transition probabilities for a Markov process. Chapman-Kolmogorov equation, Kolmogorov equations and other properties of these operators (e.g. Dynkin formula, martingale problem) represent the natural tools for studying such Markov processes.KeywordsMarkov ChainTopological SpaceMarkov ProcessMarkov PropertyPolish SpaceThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

A batch Markov arrival process (BMAP) X* = (N, J) is a 2-dimensional Markov process with two components, one is the counting process N and the other one is the phase process J. It is proved that the phase process is a time-homogeneous Markov chain with a finite state-space, or for short, Markov chain. In this paper, a new and inverse problem is proposed firstly: given a Markov chain J, can we deploy a process N such that the 2-dimensional process X* = (N, J) is a BMAP? The process X* = (N, J) is said to be an adjoining BMAP for the Markov chain J. For a given Markov chain the adjoining processes exist and they are not unique. Two kinds of adjoining BMAPs have been constructed. One is the BMAPs with fixed constant batches, the other one is the BMAPs with independent and identically distributed (i.i.d) random batches. The method we used in this paper is not the usual matrix-analytic method of studying BMAP, it is a path-analytic method. We constructed directly sample paths of adjoining BMAPs. The expressions of characteristic (D k , k = 0, 1, 2 · · ·) and transition probabilities of the adjoining BMAP are obtained by the density matrix Q of the given Markov chain J. Moreover, we obtained two frontal Theorems. We present these expressions in the first time.

This review article is concerned to provide a global context to several works on the fitting of continuous time nonhomogeneous Markov chains with finite state space and also to point out some selected aspects of two techniques previously introduced—estimation and calibration—relevant for applications and used to fit a continuous time Markov chain model to data by the adequate selection of parameters. The denomination estimation suits the procedure better when statistical techniques—e.g., maximum likelihood estimators—are employed, while calibration covers the case where, for instance, some optimisation technique finds a best approximation parameter to ensure good model fitting. For completeness, we provide a short summary of well-known important notions and results formulated for nonhomogeneous Markov chains that, in general, can be transferred to the homogeneous case. Then, as an illustration for the homogeneous case, we present a selected Billingsley’s result on parameter estimation for irreducible chains with finite state space. In the nonhomogeneous case, we quote two recent results, one of the calibration type and the other with more of a statistical flavour. We provide an ample set of bibliographic references so that the reader wanting to pursue her/his studies will be able to do so more easily and productively.

A Markov process in discrete time with a finite state space is controlled by choosing the transition probabilities from a given convex family of distributions depending on the present state. The immediate cost is prescribed for each choice and it is required to minimise the average expected cost over an infinite future. The paper considers a special case of this general problem and provides the foundation for a general solution. The main result is that an optimal policy exists if each state of the system can be reached with positive probability from any other state by choosing a suitable policy.

We show that Algorithm H* for the determination of the rate matrix of a block-GI/M/1 Markov chain is related by duality to Algorithm H for the determination of the fundamental matrix of a block-M/G/1 Markov chain. Duality is used to generate some efficient algorithms for finding the rate matrix in a quasi-birth-and-death process.

In this chapter we consider a Markov chain {ξ k , k ∈ Z +} having a finite state space but random transition probabilities. Of interest here is the asymptotic behavior of the chain and its transition probabilities as n → ∞ under the assumption that the transition probabilities of the Markov chain are close to those of a homogeneous Markov chain having nonrandom transition probabilities.

The purpose of this paper is to determine the assets of Small and Medium-sized Enterprises (SMEs), reasonably value this type of enterprise, and effectively use available data to measure their financing risks. Based on the previous research, the nonlinear expectation and the stochastic differential equation are adopted to combine the enterprise’s financing risk with the psychological factors of the consumer market to construct different risk measurement models according to the current technology-based enterprise’s pricing and financing problems. Besides, the nonparametric estimation method is applied to the nonlinear expectation, which improves the application of the model in finance. The effectiveness of the model in technology-based SMEs pricing and financing risk measurement evaluation is verified using the instance data of technology-based SMEs. Results demonstrate that the traditional pricing models for technology-based SMEs need to ensure that every investor is rational. Although the price of the enterprise cannot be effectively predicted, the nonlinear expectation and stochastic differential equation can reflect the changes in the value of the enterprises and the impacts of investors’ psychological factors on the financing effect of the enterprises. Different risk measurement models constructed can effectively price and measure technology-based SMEs. The non-parametric estimation method can effectively improve the accuracy of the model prediction. The results can provide some research ideas and practical value for the market valuation and financing risk assessment of technology-based SMEs.

Practical stability of nonlinear stochastic hybrid parabolic systems of Itô-type: Vector Lyapunov functions approach

Robust μ-stability analysis of Markovian switching uncertain stochastic genetic regulatory networks with unbounded time-varying delays

Tandem queues are good mathematical models for description of information transmission in various communication systems and networks. These queues play also an important role for the validation of different decomposition algorithms designed for investigating more general queueing networks. So, their investigation is interesting for theory and applications. In this paper, we consider tandem queue suitable for modeling the systems and networks where information flows are correlated and bursty what is typical for many modern telecommunication networks. Possible correlation of customers inter-arrival times and batch arrivals are taken into account via of consideration of the Batch Markovian Arrival Process (BMAP) as input stream to the system. The system consists of two stations. The service time at the station 1 is assumed to be generally distributed. There is no buffer at this station, and customers who meet the busy server repeat attempts to enter the system in random time intervals. The service process at the station 2 is assumed to be described by the continuous time Markov chain with a finite state space. This assumption holds good, e.g., if the station 2 has a finite buffer, consists of a finite number of identical or heterogeneous servers where the service time distribution is assumed to be of PH (PHase) type. Markov chain embedded at service completion epochs at the station 1 and the process of system states at arbitrary time are under study. Ergodicity condition and algorithms for computing the steady state probabilities are presented.

An Introduction to Markov Chain Monte Carlo

This paper deals with the asymptotic optimality of a stochastic dynamic system driven by a singularly perturbed Markov chain with finite state space. The states of the Markov chain belong to several groups such that transitions among the states within each group occur much more frequently than transitions among the states in different groups. Aggregating the states of the Markov chain leads to a limit control problem, which is obtained by replacing the states in each group by the corresponding average distribution. The limit control problem is simpler to solve as compared with the original one. A nearly-optimal solution for the original problem is constructed by using the optimal solution to the limit problem. To demonstrate, the suggested approach of asymptotic optimal control is applied to examples of manufacturing systems of production planning.

This article aims to derive the Laplace-Stieltjes transform matrix for the total increment of a one-level process during the first passage of another level process to level zero in so-called the two-dimensional Markov modulated Brownian motion. The process comprises an irreducible continuous-time Markov process with a finite state space, alongside two level processes modulated by the Markov process. These paired level processes can be viewed as a two-dimensional Brownian motion, with Brownian parameters varying based on the Markov process. Due to the infeasibility of explicit computation, we formulate a nonsymmetric algebraic Riccati equation with a minimal nonnegative solution that represents the transform matrix through a matrix exponential function. To our knowledge, this achievement is innovative within the context of the two- dimensional Markov modulated Brownian motion.