Censoring, Factorizations, and Spectral Analysis for Transition Matrices with Block-Repeating Entries

For a finite Markov chain, by rotating the transition matrix by 180°, or relabelling the states, one can define a new Markov chain. This Markov chain in fact is the imbedded Markov chain of an inverse process. Duality properties about this Markov chain sometimes are not difficult to obtain. Similarly, one can discuss an infinite Markov chain with states 0,±1,±2,…, However, many applications involve transition matrices with various boundary modifications where rotation cannot directly apply. After introducing some duality properties for a boundary-free or finite Markov chain, we will mainly focus on some interesting application problems and show how to use the duality from rotation to these problems.

The goal of this research is to implement network analysis using Markov chains in Python.Networks exist almost everywhere in life.There are networks of computers, people, articles, posts, etc. Network analysis is used to understand the structure, function, and performance of the network.Markov chains method is used to predict the future state based on the present state and not on the previous states.The basic steps of network analysis using Markov chains are explained: defining network (states, transition matrix, and distribution vector), performing matrix multiplication (computing stationary distribution vector and computing stationary transition vector), performing random walk (computing stationary distribution vector), comparing results, and plotting charts.The developed program was tested on an experimental data.The program has successfully performed the basic steps of network analysis using Markov chains and provided the required results.

In this paper, a transition matrix is developed in order to estimate duration of concurrent iterations among coupled design activities. Since, this type of iteration cannot be modeled by the basic transition matrix in a Markov Chain(MC). To do this, A Design Structure Matrix (DSM) is utilized and developed. DSM is a tool for illustrating different types of relationship among design activities. The proposed model enhanced Design Structure Matrix as a random variable to be used in a transition matrix of the MC. The DSM enables MC to include simultaneous transitions by extending the dimension of states in a transition matrix including concurrent transitions. In addition, it contributes designers to estimate required time for completing design activities through calculating iterations. At the end, the proposed model is applied in a case study with concurrent information transfer among coupled design activities. Demonstrates the way to estimate the numbers of iterations

Minimising the largest mean first passage time of a Markov chain: The influence of directed graphs

A common task for network protocols is the broadcasting of information from one node to the rest of the nodes in the network. This task is often required during the execution of parallel algorithms in a network of processors, or other situations where the nodes of a mesh network generate packets to be broadcast at random time instances. We consider processors communicating over a mesh network with the objective of broadcasting information among each other. One instance of the problem involves a number of nodes all with the same message to be broadcasted. For that problem, a lower-bound on the time to complete the broadcast, and an algorithm which achieves this bound are presented. In another instance, every node in the mesh has packets to be broadcast arriving independently, according to a Poisson random process. The stability region for performing such broadcasts is characterized, and broadcast algorithms which operate efficiently within that region are presented. These algorithms involve interacting queues whose analysis is known to be very difficult. Toward that end we develop an approximation which models an n-dimensional infinite Markov chain as a single-dimensional infinite Markov chain together with an n-dimensional finite Markov chain. This approximate model can be analyzed and the results compare favorably with simulation.

This paper presents an efficient equilibrium solution algorithm for infinite multi-dimensional Markov chains of the quasi-birth-and-death (QBD) type. The algorithm is not based on an iterative approach, so that the exact solution can be computed in a known, finite number of steps. The key step on which the algorithm is based, is the identification of a linear dependence among variables. This dependence is expressed in terms of a matrix whose size is finite. The equilibrium solution of the Markov chain is obtained operating on this finite matrix. An extremely attractive feature of the newly proposed algorithm is that it allows the computation of approximate solutions with any desired degree of accuracy. The solution algorithm, in fact, computes a succession of approximate solutions with growing accuracy, until the exact solution is achieved in a finite number of steps. Results for a case study show that the proposed algorithm is very efficient and quite accurate, even when providing approximate solutions.

Continuous Time Markov chains (CTMCs) are widely used to model and analyze networked systems. A common analysis approach is to solve the system of balance equations governing the state transitions of a CTMC to obtain its steadystate probability distribution, and use the state probabilities to derive or compute various performance measures. In many systems, the state space of the underlying CTMC is infinite and multi-dimensional with state-dependent transitions; exact analysis of such models is challenging. For example, the exact probability distribution of the number of jobs in the Discriminatory Processor Sharing (DPS) system, first proposed by Kleinrock in 1967 [4], is still an open challenge. Likewise, obtaining the exact state probabilities of quasi-birth-and-death (QBD) processes with leveldependent transitions is known to be challenging [1]; QBDs are infinite state space multi-dimensional Markov chains in which states are organized into levels and transitions are skip-free between the levels.

Abstract. The general random walk on the nonnegative integers with absorbing boundaries at 0 and n has the transition probabilities p0j = 0j , pnj = nj, pi;i 1 = pi, pi;i+1 = qi, and pii = ri, where pi + ri + qi = 1. The fundamental matrix B of this Markov chain is the inverse of matrix (I Q) where Q results from P by deleting the rows and columns 0 and n. Entry bij represents the expected number of occurrences of the transient state j prior to absorption if the random walk starts at state i. The absorption time as well as the absorption probabilities are easily derived once the fundamental matrix is known. Here, it is shown that the fundamental matrix can be determined in elementary manner via the adjugate of matrix (I Q).

This work presents further development of the octet formalism established by the authors for the classical Kirchhoff anisotropic plate theory. The structure of the fundamental elastic plate matrix is fully explored and the explicit expression is provided. The matrices N2 and N3 are proved to be positive semi‐definite. Thus, H and L are positive definite. Further studies are concerned with a rotated coordinate system. The transform relation between the eigenvectors in the original and the rotated coordinate system is given. The fundamental elastic plate matrix associated with the eigenrelation referring to the dual coordinate systems, NΘ, is studied. The major properties that hold in the Stroh sextic formalism for generalized plane strain problems are also valid in the octet formalism for thin plate bending problems. In particular, we generalize a property in Stroh's formalism for any non‐semisimple matrix NΘ. We show a new property in the octet formalism. The non‐semisimple cases of NΘ are discussed. Finally, we make it transparent that the mixed/hybrid formalism of others is precisely one of sixteen permuted forms of the octet formalism.

Probabilistic model checkers like PRISM check the satisfiability of probabilistic CTL (pCTL) formulas against discrete-time Markov chains. We prove soundness and completeness of their underlying algorithm in Isabelle/HOL. We define Markov chains given by a transition matrix and formalize the corresponding probability measure on sets of paths. The formalization of pCTL formulas includes unbounded cumulated rewards.KeywordsMarkov ChainModel CheckProbability SpaceProof AssistantLinear Equation SystemThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Simple SummaryUnderstanding and predicting vector population and related disease dynamics, is crucial for gaining insight into the abundance and dynamics of arthropod disease vectors, and for the design of effective vector control strategies. Several mathematical and standard epidemiological models have been proposed in studying vector transmitted infectious disease dynamics. However, most models are of deterministic nature and are not able to estimate other relevant metrics such as the probability of vector population emergence as well as the probability and expected time to reach certain population and/or infection state. Here we are focusing on stochastic modeling of mosquito abundance data using weather driven Markov chains (MCs) and are particularly interested in estimating transition probabilities (TPs) between different population levels. A MC model is based on the assumption that the future state of the variable is only dependent on the present state and is suitable in cases of short and noisy data characterized by a complex and random behavior. The aim is to introduce and generalize a formulation of conditional Markov chain models (CMSs) for predicting probability transition estimates of arthropod vector populations. In this context, first we present the basic principles and assumptions behind Markov chain modeling approach, with an intuitive interpretation of the integration of conditional Markov chains (CMCs) and then demonstrate the usefulness of the approach in predicting the abundance of Culex sp. We conclude that the conditional Markov chain technique is recommended as viable for modeling populations that explicit random dynamics and predict their future evolution. Although, the Markov models generated in this work provide an accurate abstraction of the vector disease progress observed within the dataset used for their generation, we envision the current approach as an entry point into the medical entomology literature and methods for predicting arthropod vector diseases dynamics.Understanding and predicting mosquito population dynamics is crucial for gaining insight into the abundance of arthropod disease vectors and for the design of effective vector control strategies. In this work, a climate-conditioned Markov chain (CMC) model was developed and applied for the first time to predict the dynamics of vectors of important medical diseases. Temporal changes in mosquito population profiles were generated to simulate the probabilities of a high population impact. The simulated transition probabilities of the mosquito populations achieved from the trained model are very near to the observed data transitions that have been used to parameterize and validate the model. Thus, the CMC model satisfactorily describes the temporal evolution of the mosquito population process. In general, our numerical results, when temperature is considered as the driver of change, indicate that it is more likely for the population system to move into a state of high population level when the former is a state of a lower population level than the opposite. Field data on frequencies of successive mosquito population levels, which were not used for the data inferred MC modeling, were assembled to obtain an empirical intensity transition matrix and the frequencies observed. Our findings match to a certain degree the empirical results in which the probabilities follow analogous patterns while no significant differences were observed between the transition matrices of the CMC model and the validation data (ChiSq = 14.58013, df = 24, p = 0.9324451). The proposed modeling approach is a valuable eco-epidemiological study. Moreover, compared to traditional Markov chains, the benefit of the current CMC model is that it takes into account the stochastic conditional properties of ecological-related climate variables. The current modeling approach could save costs and time in establishing vector eradication programs and mosquito surveillance programs.

SummaryReversible Markov chains are the basis of many applications. However, computing transition probabilities by a finite sampling of a Markov chain can lead to truncation errors. Even if the original Markov chain is reversible, the approximated Markov chain might be non‐reversible and will lose important properties, like the real‐valued spectrum. In this paper, we show how to find the closest reversible Markov chain to a given transition matrix. It turns out that this matrix can be computed by solving a convex minimization problem. Copyright © 2015 John Wiley & Sons, Ltd.

In this paper, we present some algebraic properties of a particular class of probability transition matrices, namely, Hamiltonian transition matrices. Each matrix P in this class corresponds to a Hamiltonian cycle in a given graph G on n nodes and to an irreducible, periodic, Markov chain. We show that a number of important matrices traditionally associated with Markov chains, namely, the stationary, fundamental, deviation and the hitting time matrix all have elegant expansions in the first n−1 powers of P , whose coefficients can be explicitly derived. We also consider the resolvent-like matrices associated with any given Hamiltonian cycle and its reverse cycle and prove an identity about the product of these matrices.

In previous work, the embedding problem is examined within the entire set of discrete-time Markov chains. However, for several phenomena, the states of a Markov model are ordered categories and the transition matrix is state-wise monotone. The present paper investigates the embedding problem for the specific subset of state-wise monotone Markov chains. We prove necessary conditions on the transition matrix of a discrete-time Markov chain with ordered states to be embeddable in a state-wise monotone Markov chain regarding time-intervals with length 0.5: A transition matrix with a square root within the set of state-wise monotone matrices has a trace at least equal to 1.

We consider a singularly perturbed (finite state) Markov chain and provide a complete characterization of the fundamental matrix. In particular, we obtain a formula for the regular part simpler than a previous formula obtained by Schweitzer, and the singular part is obtained via a reduction process similar to Delebecque's reduction for the stationary distribution. In contrast to previous approaches, one works with aggregate Markov chains of much smaller dimension than the original chain, an essential feature for practical computation. An application to mean first-passage times is also presented.