Perturbed Markov Chains with Damping Component

Let $P(x,t,A)$ be the probability of transition from state $x \in \Omega $ to a measurable set A in time t of a homogeneous Markov process in the space $\Omega $. Let $P_t (A)$ be the probability that the system is in the measurable set A at time t if at $t = 0$ the distribution $P_0 $ is valid. The stationary distribution P is called ergodic if for each initial distribution $P_0 $ the variation $V(P_t - P) \to 0$ for $t \to \infty $.The following ergodic theorem is proved:Theorem 1.A Markov process homogeneous in time has a unique stationary probability distribution which is ergodic if for any$\varepsilon > \infty $there exists a measurable setC, a probability distributionRin$\Omega $, and values$t_1 > 0,k > 0,K > 0$such that 1) $kR(A) \leqq P(x,t_1 ,A)$for all points$x \in C$and measurable sets$A \subseteq C$; for any initial distribution$P_0 $there exists a$t_0 $such that for any$t \geqq t_0 $ 2) $P_t (C) \geqq 1 - \varepsilon $; 3) $P_t (A) \leqq KR(A) + \varepsilon $for any measurable set$A \subseteq...

This paper presents a robust reliability analysis method for systems of multimodular redundant (MMR) controllers using the method of partitioning and parallel processing of a Markov chain (PPMC). A Markov chain is formulated to represent the N distinct states of the MMR controllers. Such a Markov chain has N2 directed edges, and each edge corresponds to a transition probability between a pair of start and end states. Because N can be easily increased substantially, the system reliability analysis may require large computational resources, such as the central processing unit usage and memory occupation. By the PPMC, a Markov chain's transition probability matrix can be partitioned and reordered, such that the system reliability can be evaluated through only the diagonal submatrices of the transition probability matrix. In addition, calculations regarding the submatrices are independent of each other and thus can be conducted in parallel to assure the efficiency. The simulation results show that, compared with the sequential method applied to an intact Markov chain, the proposed PPMC can improve the performance and produce allowable accuracy for the reliability analysis on large-scale systems of MMR controllers.

In order to forecast and evaluate a system more reasonably, this paper establishes a mathematical model, based on the theory that the finite irreducible aperiodic homogeneous Markov chain has one and only stationary distribution. At first, calculates the proportions of every sort of members in a system, as the initial distribution. After one unit of time, ciphers the system's state distribution again. According to the two different state distributions, the transition probability matrix can be got through cyphering. Then we can compute the final state distribution of the system when it becomes stable, using the properties of homogeneous Markov chain. So the system can be predicted. In addition we illustrate the application of this model through an example. After verification, the model is more objective and appropriate than the traditional methods.

Condition assessment in the Swiss bridge management system (KUBA-MS) is performed on the element level. Five condition states are defined based on visual appearance. In order to forecast the condition states of any given element at any given time a relationship must be established between the element age and its condition state. This relationship, which describes the condition evolution, can be obtained empirically from statistical analysis of pairs of consecutive condition assessments (inspections). Markov chains are used in KUBA-MS to represent condition evolution and the transition probabilities are determined using regression analysis of pairs of inspections. Unfortunately there are almost no inspection data for the worst and second worst condition states. The forecasts made using Markov chains are therefore not always reliable. In this paper an alternative approach is suggested, which takes into consideration the physical phenomena underlying element deterioration. This alternative approach is applied to chloride-induced corrosion of steel reinforcement, by far the most common deterioration mechanism in Switzerland. The chloride-induced corrosion is modeled mathematically and numerical simulations of the condition evolution for different values of model parameters are performed. The simulation results have been mapped to condition states as defined in KUBA-MS and Markov transition matrices have been calibrated to fit simulation results.

Cross-product ratios (αs), which are structurally analogous to odds ratios, are statistically sound and demographically meaningful measures. Assuming constant cross-product ratios in the elements of a matrix of multistate transition probabilities provides a new basis both for calculating probabilities from minimal data and for modeling populations with changing demographic rates. Constant-α estimation parallels log linear modeling, in which the αs are the fixed interactions, and the main effects are calculated from relevant data. Procedures are presented showing how an N state model's matrix of transition probabilities can be found from the constant αs and (1) the state composition of adjacent populations, (2) (N - 1) known probabilities, (3) (N - 1) known transfer rates, or (4) (2N - 1) known numbers of transfers. The scope and flexibility of constant-α models makes them applicable to a broad range of demographic subjects, including marital/union status, political affiliation, residential status, and labor force status. Here, an application is provided to the important but understudied topic of poverty status. Census data, separately for men and women, provide age-specific numbers of persons in three poverty statuses for the years 2009 and 2014. Using an estimated transition matrix that furnishes a set of cross-product ratios, the constant-α approach allows the calculation of male and female poverty status life tables for the 2009-2014 period. The results describe the time spent in each poverty state and the transitions between states over the entire life course.

Purpose Pavement deterioration prediction models play a crucial role in determining maintenance strategies and future funding needs. While deterioration prediction models have been studied extensively in the past, applications of these models to local street networks have been limited. This study aims to address this gap by sharing the results of network level deterioration prediction models developed at a local level. Design/methodology/approach Network level pavement deterioration prediction models are developed using Markov chains for the local street network in Syracuse, New York, based on pavement condition rating data collected over a 15-year time period. Transition probability matrices are generated by calculating the percentage of street sections that transition from one state to another within one duty cycle. Bootstrap sampling with replacement is used to numerically generate 95% confidence intervals around the transition probability values. Findings The overall local street network is divided into three cohorts based on street type (i.e. avenues, streets and roads) and two cohorts based on pavement type. All cohorts demonstrated very similar deterioration trends, indicating the existence of a fast-paced deterioration mechanism for the local street network of Syracuse. Originality/value This study contributes to the body of knowledge in deterioration modeling of local street networks, especially in the absence of key predictor variables. Furthermore, this study introduces the use of bootstrap sampling with replacement method in generating confidence intervals for transition probability values.

This paper considers a class of Markov chains on a bivariate state space , whose transition probabilities have a particular ‘block-partitioned' structure. Examples of such chains include those studied by Neuts [8] who took E to be finite; they also include chains studied in queueing theory, such as (Nn , Sn ) where Nn is the number of customers in a GI/G/1 queue immediately before, and Sn the remaining service time immediately after, the nth arrival. We show that the stationary distribution Πfor these chains has an ‘operator-geometric' nature, with , where the operator S is the minimal solution of a non-linear operator equation. Necessary and sufficient conditions for Πto exist are also found. In the case of the GI/G/1 queueing chain above these are exactly the usual stability conditions. G1/G/1 QUEUE; PHASE-TYPE; INVARIANT MEASURE; FOSTER'S CONDITIONS

Random walks are a fundamental model in applied mathematics and are a common example of a Markov chain. The limiting stationary distribution of the Markov chain represents the fraction of the time spent in each state during the stochastic process. A standard way to compute this distribution for a random walk on a finite set of states is to compute the Perron vector of the associated transition matrix. There are algebraic analogues of this Perron vector in terms of transition probability tensors of higher-order Markov chains. These vectors are nonnegative, have dimension equal to the dimension of the state space, and sum to one and are derived by making an algebraic substitution in the equation for the joint-stationary distribution of a higher-order Markov chains. Here, we present the spacey random walk, a non-Markovian stochastic process whose stationary distribution is given by the tensor eigenvector. The process itself is a vertex-reinforced random walk, and its discrete dynamics are related to a continuous dynamical system. We analyze the convergence properties of these dynamics and discuss numerical methods for computing the stationary distribution. Finally, we provide several applications of the spacey random walk model in population genetics, ranking, and clustering data, and we use the process to analyze taxi trajectory data in New York. This example shows definite non-Markovian structure.

Article Free Access Share on Exact sampling and approximate counting techniques Author: Mark Huber School of Operations Research and Industrial Engineering, Cornell University, Ithaca, NY School of Operations Research and Industrial Engineering, Cornell University, Ithaca, NYView Profile Authors Info & Claims STOC '98: Proceedings of the thirtieth annual ACM symposium on Theory of computingMay 1998 Pages 31–40https://doi.org/10.1145/276698.276709Published:23 May 1998Publication History 25citation666DownloadsMetricsTotal Citations25Total Downloads666Last 12 Months66Last 6 weeks15 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

This work analyzes distributed linear averaging within a connected network of sensors that each track the stationary distribution of an ergodic Markov chain with a slowly switching regime. Our approach is based on a two-time-scale stochastic approximation. A hyperparameter modeled as a Markov chain on a slower time-scale modulates the regime of each observed Markov chain. The average of all currently observed stationary distributions constitutes the average-consensus estimate to be reached by all sensors. Assuming the Markov chains do not share a common stationary distribution conditioned on their regime, then under the proposed linear averaging algorithm, the exchange graph conditions required for the sequence of sensor state values to converge weakly to the average-consensus are obtained. Estimation of a weighted average of all observed stationary distributions, not only the current ones, is proved feasible over a long-run time horizon, provided an additional communication condition holds. The sensor st...

Human mobility describes physical patterns of movement of people within a spatial system. Many of these patterns, including daily commuting, are cyclic and quantifiable. These patterns capture physical phenomena tied to processes studied in urban planning, epidemiology, and other social, behavioral, and economic sciences. This paper advances human mobility research by proposing a statistical method for identifying locations that individual move to and through at a rate proportionally higher than other locations, using commuting data for the country of New Zealand as a case study. We term these locations mobility loci and they capture a global property of communities in which people commute. The method makes use of a directed-graph representation where vertices correspond to locations, and traffic between locations correspond to edge weights. Following a normalization, the graph can be regarded as a Markov chain whose stationary distribution can be calculated. The proposed permutation procedure is then applied to determine which stationary distribution values are larger than what would be expected, given the structure of the directed graph and traffic between locations. The results of this method are evaluated, including a comparison to what is already known about commuting patterns in the area as well as a comparison with similar features.

Markov chains with doubly stochastic transition matrices and application to a sequence of non-selective quantum measurements

The problems of robust finite-time stochastic stability and stabilization of uncertain networked control systems (NCSs) in the presence of random delays are considered in this paper. First, network-induced random delays are modeled as a Markov chain, and the resulting closed-loop system is transformed into a Markovian jump linear system (MJLS). Since the accurate access to the transition probabilities (TPs) is hard or even impossible, some of the elements in the transition probability matrix (TPM) are considered as unknown parameters. Based on this model, the sufficient conditions for the robust finite-time stochastic stability and stabilization of the uncertain NCS are proposed. Moreover, a new linear matrix inequality (LMI) approach is employed to calculate the static output-feedback controllers. Additionally, the results of the proposed scheme are verified by two examples and simulations, which include a practical example.

A general model for the description of a discrete-time Markov chain in random environment is introduced. We study the stationary distribution of this model and its relationship with the Palm (embedded) distributions at certain environmental transition epochs. We also study the Events See Time Averages (ESTA) property for this model and we show that it is related to certain partial balance equations and a product-form stationary distribution. Next, we study certain forms of interaction between the environmental process and the process of interest that lead to tractable stationary distributions. More specifically, the stationary distributions of a large class of Markov chains in random environment can be computed efficiently using α-potentials of smaller Markov chains. This decomposability result is also the basis for developing approximations for the stationary distributions of intractable models that evolve in a slowly changing random environment. Moreover, explicit error bounds for these approximations are given. The model can be used for the performance evaluation of discrete time queueing systems in random environment.

We present in this paper a mathematical method based on stochastic comparisons of Markov chains in order to compute in a mobile network performance measures that are important Q parameters for users: the dropping handover of voice and the blocking probability of a new voice call. The key idea of this methodology is that given a complex system represented by a multi-dimensional Markov chain which is too large to be solved, we propose to reduce the state space, and so to define a new Markov chain which is a simplified version of the original one. This reduced Markov chain is defined as an aggregated one, which represents a stochastic bound for performance measures written as increasing reward function on the stationary distribution. The main steps of the construction of the aggregated Markov chain applied into mobile networks are presented in this paper. As the number of mobile users with different kinds of applications increases, the associated model is more complex and it can be represented by multi-dimensional continuous time Markov chain with a very large size. Thus, we define an aggregated Markov chain represented by a multi-dimensional birth and death process which is very easy to solve. We have proved that there is a weak ordering between the original Markov chain and the aggregated one using increasing set formalism. We have computed upper bounds of dropping handover and blocking probability for different values of input parameters. Numerical results prove that upper bounds give good results and so the stochastic methodology is an interesting mathematical tool for the performance evaluation of complex systems. Keywords: Mobile Networks, Quality of Service, Stochastic ordering, stochastic comparisons, Continuous Time Markov Chains.