Implementation of Network Analysis using Markov Chains in Python

In this paper, we use the Markov chain censoring technique to study infinite state Markov chains whose transition matrices possess block-repeating entries. We demonstrate that a number of important probabilistic measures are invariant under censoring. Informally speaking, these measures involve first passage times or expected numbers of visits to certain levels where other levels are taboo;they are closely related to the so-called fundamental matrix of the Markov chain which is also studied here. Factorization theorems for the characteristic equation of the blocks of the transition matrix are obtained. Necessary and sufficient conditions are derived for such a Markov chain to be positive recurrent, null recurrent, or transient based either on spectral analysis, or on a property of the fundamental matrix. Explicit expressions are obtained for key probabilistic measures, including the stationary probability vector and the fundamental matrix, which could be potentially used to develop various recursivealgorithms for computing these measures.

This paper proposes a new algorithm for computing the stationary distribution vector in continuous-time upper block-Hessenberg Markov chains. To this end, we consider the last-block-column-linearly-augmented (LBCL-augmented) truncation of the (infinitesimal) generator of the upper block-Hessenberg Markov chain. The LBCL-augmented truncation is a linearly-augmented truncation such that the augmentation distribution has its probability mass only on the last block column. We first derive an upper bound for the total variation distance between the respective stationary distribution vectors of the original generator and its LBCL-augmented truncation. Based on the upper bound, we then establish a series of linear fractional programming (LFP) problems to obtain augmentation distribution vectors such that the bound converges to zero. Using the optimal solutions of the LFP problems, we construct a matrix-infinite-product (MIP) form of the original (i.e., not approximate) stationary distribution vector and develop a sequential update algorithm for computing the MIP form. Finally, we demonstrate the applicability of our algorithm to BMAP/M/$\infty$ queues and M/M/$s$ retrial queues.

The Physical or Mathematical behaviour of this model may be represented by describing all the different states it may occupy and by indicating how it moves among these states. In this study, the stationary distribution of Markov chains was solved using iterative methods that begin with an initial estimate of the solution vector and then modified it in a way that brings it closer and closer to the real solution with each step or iteration. These methods also involved matrix operations like multiplication with one or more vectors, which preserves the transition matrices while speeding up the process. We computed the solutions using Jacobi iterative method and Gauss-Seidel iterative method in order to shed more light on the solutions of stationary distribution in Markov chain. This was done with the aid of several already-existing laws, theorems, and formulas of Markov chain and the application of normalization principle and matrix operations such as lower, upper, and diagonal matrices. The stationary distribution vector’s 𝜋𝑖,𝑖=1,2,…,4 are obtained for the illustrative example one as 𝜋(3) = (0.078125,0.109375,0.21875,0.59375) as well as the four eigenvalues of the matrix as 𝜆1=1.0, 𝜆2=−0.7718, 𝜆3,4=−0.1141±0.5576𝑖 using Jacobi iterative technique, and for illustrative example two using Gauss-Siedel method as 𝝅(𝟑) = (0.090909, 0.181818, 0.363636, 0.363636). The research shown that Gauss Siedel method converged faster than Jacobi method.

Convergence theory of some classes of iterative aggregation/disaggregation methods for computing stationary probability vectors of stochastic matrices

BackgroundCombinatorial drug therapy for complex diseases, such as HSV infection and cancers, has a more significant efficacy than single-drug treatment. However, one key challenge is how to effectively and efficiently determine the optimal concentrations of combinatorial drugs because the number of drug combinations increases exponentially with the types of drugs.ResultsIn this study, a searching method based on Markov chain is presented to optimize the combinatorial drug concentrations. In this method, the searching process of the optimal drug concentrations is converted into a Markov chain process with state variables representing all possible combinations of discretized drug concentrations. The transition probability matrix is updated by comparing the drug responses of the adjacent states in the network of the Markov chain and the drug concentration optimization is turned to seek the state with maximum value in the stationary distribution vector. Its performance is compared with five stochastic optimization algorithms as benchmark methods by simulation and biological experiments. Both simulation results and experimental data demonstrate that the Markov chain-based approach is more reliable and efficient in seeking global optimum than the benchmark algorithms. Furthermore, the Markov chain-based approach allows parallel implementation of all drug testing experiments, and largely reduces the times in the biological experiments.ConclusionThis article provides a versatile method for combinatorial drug screening, which is of great significance for clinical drug combination therapy.

Suppose that A is an irreducible stochastic matrix of order n, and denote its eigenvalues by 1, λ2, . . . , λn. The Kemeny constant, K(A) for the Markov chain associated with A is defined as K(A) = ∑n j=2 1 1−λj , and can be interpreted as the mean first passage from an unknown initial state to an unknown destination state in the Markov chain. Let w denote the stationary distribution vector for A, and suppose that w1 ≤ w2 ≤ · · · ≤ wn. In this paper, we show that K(A) ≥ ∑n j=1(j − 1)wj , and we characterise the matrices yielding equality in that bound. The results are established using techniques from matrix theory and the theory of directed graphs.

In this paper, a transition matrix is developed in order to estimate duration of concurrent iterations among coupled design activities. Since, this type of iteration cannot be modeled by the basic transition matrix in a Markov Chain(MC). To do this, A Design Structure Matrix (DSM) is utilized and developed. DSM is a tool for illustrating different types of relationship among design activities. The proposed model enhanced Design Structure Matrix as a random variable to be used in a transition matrix of the MC. The DSM enables MC to include simultaneous transitions by extending the dimension of states in a transition matrix including concurrent transitions. In addition, it contributes designers to estimate required time for completing design activities through calculating iterations. At the end, the proposed model is applied in a case study with concurrent information transfer among coupled design activities. Demonstrates the way to estimate the numbers of iterations

In this work, we characterize the fixed points of the doubly stochastic operators defined on discrete spaces ℓ p ( I ) , where I is a nonempty set and p ∈ [ 1 , + ∞ ) . We use our results to characterize all Markov chains with infinite state space and doubly stochastic transition probability matrix that have at least one stationary distribution vector.

Analysis of Network Security Access Space Agency Based Firewall Log In LAPAN Center. Supervisor LIPUR SUGIYANTA, Ph.D and Drs. BACHREN ZAINI, M.Pd. Increased Denial Of Service attacks, and other types of computer network interference, making security an important issue to be considered by all those who take advantage of the presence of the virtual world today. LAPAN Center system along with the information contained within is no exception contained attacks on access computer network security. Such attacks can be prevented at an early stage by analyzing at each access that will go on network security in LAPAN. This research at Analysis of Network Access to know and learn access of network security in firewall log LAPAN Center. This research was conducted with several stages of observation and interviews to employees of LAPAN Center infrastructure. After the stage of observation and interviews, the researchers conducted further data collection and analysis of firewall logs. The study states that the analysis of security access network based firewall log in LAPAN Center in the form of six messages that occur when a network will pass through the firewall. It can be concluded that by doing some analysis phases of network security access based firewall log in LAPAN Center produces a record of every incident inside the firewall logs. The study states that the analysis of security access network based firewall log in LAPAN Center in the form of six messages that occur when a network will pass through the firewall. It can be concluded that by doing some analysis phases of network security access based firewall log in LAPAN Center produces a record of every incident inside the firewall logs. The study states that the analysis of security access network based firewall log in LAPAN Center in the form of six messages that occur when a network will pass through the firewall. It can be concluded that by doing some analysis phases of network security access based firewall log in LAPAN Center produces a record of every incident inside the firewall logs

Minimising the largest mean first passage time of a Markov chain: The influence of directed graphs

Markov chains with periodic graphs arise frequently in a wide range of modelling experiments. Application areas range from flexible manufacturing systems in which pallets are treated in a cyclic manner to computer communication networks. The primary goal of this paper is to show how advantage may be taken of this property to reduce the amount of computer memory and computation time needed to compute stationary probability vectors of periodic Markov chains. After reviewing some basic properties of Markov chains whose associated graph is periodic, we introduce a ‘reduced scheme’ in which only a subset of the probability vector need be computed. We consider the effect of the application of both direct and iterative methods to the original Markov chain (permuted to normal periodic form) as well as to the reduced system. We show how the periodicity of the Markov chain may be efficiently computed as the chain itself is being generated. Finally, some numerical experiments to illustrate the theory, are presented.

Improved bounds for a condition number for Markov chains

For a finite Markov chain, by rotating the transition matrix by 180°, or relabelling the states, one can define a new Markov chain. This Markov chain in fact is the imbedded Markov chain of an inverse process. Duality properties about this Markov chain sometimes are not difficult to obtain. Similarly, one can discuss an infinite Markov chain with states 0,±1,±2,…, However, many applications involve transition matrices with various boundary modifications where rotation cannot directly apply. After introducing some duality properties for a boundary-free or finite Markov chain, we will mainly focus on some interesting application problems and show how to use the duality from rotation to these problems.

We consider large Markov chains which posses specific decomposable structure, the so called Nearly Completely Decomposable Chains (NCD chains). A new theoretical approach for approximate computations of NCD chains has been recently introduced in [11,12]. The method of forest expansions gives raise to aggregation algorithms, which approximate effectively the characteristics of Markov chain. The algorithms are based on grouping the states of a Markov chain in such a way that the probability of changing the state inside the group is of greater order of magnitude than interactions between groups. In [5] the algorithm approximating stationary distribution (described by transposed system L T x = b, where L is derived from transition matrix) was presented; in this paper we illustrate that combinatorial aggregation in the case of non-transposed system (defining e.g. mean hitting time) is not less effective. This novel approach allows us to treat both types of problems in the unified manner. To our knowledge for the first time an aggregation scheme was used to calculate Markov chain characteristics other than stationary distribution.

Simple SummaryUnderstanding and predicting vector population and related disease dynamics, is crucial for gaining insight into the abundance and dynamics of arthropod disease vectors, and for the design of effective vector control strategies. Several mathematical and standard epidemiological models have been proposed in studying vector transmitted infectious disease dynamics. However, most models are of deterministic nature and are not able to estimate other relevant metrics such as the probability of vector population emergence as well as the probability and expected time to reach certain population and/or infection state. Here we are focusing on stochastic modeling of mosquito abundance data using weather driven Markov chains (MCs) and are particularly interested in estimating transition probabilities (TPs) between different population levels. A MC model is based on the assumption that the future state of the variable is only dependent on the present state and is suitable in cases of short and noisy data characterized by a complex and random behavior. The aim is to introduce and generalize a formulation of conditional Markov chain models (CMSs) for predicting probability transition estimates of arthropod vector populations. In this context, first we present the basic principles and assumptions behind Markov chain modeling approach, with an intuitive interpretation of the integration of conditional Markov chains (CMCs) and then demonstrate the usefulness of the approach in predicting the abundance of Culex sp. We conclude that the conditional Markov chain technique is recommended as viable for modeling populations that explicit random dynamics and predict their future evolution. Although, the Markov models generated in this work provide an accurate abstraction of the vector disease progress observed within the dataset used for their generation, we envision the current approach as an entry point into the medical entomology literature and methods for predicting arthropod vector diseases dynamics.Understanding and predicting mosquito population dynamics is crucial for gaining insight into the abundance of arthropod disease vectors and for the design of effective vector control strategies. In this work, a climate-conditioned Markov chain (CMC) model was developed and applied for the first time to predict the dynamics of vectors of important medical diseases. Temporal changes in mosquito population profiles were generated to simulate the probabilities of a high population impact. The simulated transition probabilities of the mosquito populations achieved from the trained model are very near to the observed data transitions that have been used to parameterize and validate the model. Thus, the CMC model satisfactorily describes the temporal evolution of the mosquito population process. In general, our numerical results, when temperature is considered as the driver of change, indicate that it is more likely for the population system to move into a state of high population level when the former is a state of a lower population level than the opposite. Field data on frequencies of successive mosquito population levels, which were not used for the data inferred MC modeling, were assembled to obtain an empirical intensity transition matrix and the frequencies observed. Our findings match to a certain degree the empirical results in which the probabilities follow analogous patterns while no significant differences were observed between the transition matrices of the CMC model and the validation data (ChiSq = 14.58013, df = 24, p = 0.9324451). The proposed modeling approach is a valuable eco-epidemiological study. Moreover, compared to traditional Markov chains, the benefit of the current CMC model is that it takes into account the stochastic conditional properties of ecological-related climate variables. The current modeling approach could save costs and time in establishing vector eradication programs and mosquito surveillance programs.