Approximating Markov Chains for Bootstrapping and Simulation

The goal of this paper is to develop a general method to establish conditional ergodicity of infinite-dimensional Markov chains. Given a Markov chain in a product space, we aim to understand the ergodic properties of its conditional distributions given one of the components. Such questions play a fundamental role in the ergodic theory of nonlinear filters. In the setting of Harris chains, conditional ergodicity has been established under general nondegeneracy assumptions. Unfortunately, Markov chains in infinite-dimensional state spaces are rarely amenable to the classical theory of Harris chains due to the singularity of their transition probabilities, while topological and functional methods that have been developed in the ergodic theory of infinite-dimensional Markov chains are not well suited to the investigation of conditional distributions. We must therefore develop new measure-theoretic tools in the ergodic theory of Markov chains that enable the investigation of conditional ergodicity for infinite dimensional or weak-* ergodic processes. To this end, we first develop local counterparts of zero-two laws that arise in the theory of Harris chains. These results give rise to ergodic theorems for Markov chains that admit asymptotic couplings or that are locally mixing in the sense of H. F\"{o}llmer, and to a non-Markovian ergodic theorem for stationary absolutely regular sequences. We proceed to show that local ergodicity is inherited by conditioning on a nondegenerate observation process. This is used to prove stability and unique ergodicity of the nonlinear filter. Finally, we show that our abstract results can be applied to infinite-dimensional Markov processes that arise in several settings, including dissipative stochastic partial differential equations, stochastic spin systems and stochastic differential delay equations.

In many applications, successive observations of a process, say X 1, X 2,…, have an inherent time component associated with them. For example, the X i could be the state of the weather at a particular location on the i th day, counting from some fixed day. In a simplistic model, the state of the weather could be “dry” or “wet,” quantified as, say, 0 and 1. It is hard to believe that in such an example, the sequence X 1, X 2,… could be mutually independent. The question then arises how to model the dependence among the X i ′s. Probabilists have numerous dependency models. A particular model that has earned a very special status is called the Markov chain. In a Markov chain model, we assume that the future, given the entire past and the present state of a process, depends only on the present state. In the weather example, suppose we want to assign a probability that tomorrow, say March 10, will be dry, and suppose that we have available to us the precipitation history for each day from January 1 to March 9. In a Markov chain model, our probability that March 10 will be dry will depend only on the state of the weather on March 9, even though the entire past precipitation history was available to us. As simple as it sounds, Markov chains are enormously useful in applications, perhaps more than any other specific dependency model. They also are independently relevant to statistical computing in very important ways. The topic has an incredibly rich and well-developed theory, with links to many other topics in probability theory. Familiarity with basic Markov chain terminology and theory is often considered essential for anyone interested in studying statistics and probability.We present an introduction to basic Markov chain theory in this chapter.KeywordsMarkov ChainStationary DistributionMarkov Chain ModelTransition Probability MatrixStationary Markov ChainThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The most complex models with the high quality of the results obtained are, as a rule, more expensive in terms of developer qualifications and computational resources. During process design, a detailed description of the object is often not required, and the accuracy of the results obtained should not be higher than the accuracy of the measuring instruments used. Thus, the optimal combination of simplicity and quality of the mathematical description of technological processes is an urgent task of mathematical modeling. To identify a cell model developed based on the theory of Markov chains, data obtained by solving discrete models of the Boltzmann equation are used. A method to identify cell models of a fluidized bed reactor has been developed using data obtained based on solving discrete models of the Boltzmann equation. The adequacy of the identified model of a fluidized bed reactor has been verified. An approach to develop computational support for a cell model based on the theory of Markov chains is presented. The analysis of the results obtained has shown an adequate description of the processes in fluidized bed reactors in terms of cell models. The models are developed based on the theory of Markov chains and identified based on the results obtained within the framework of discrete models of the Boltzmann equation. The proposed method to identify and verify cell models provides the possibility to obtain simultaneously acceptable indicators of model simplicity and the accuracy of calculation of the design and operating parameters of fluidized bed reactors.

New heat resisting alloy

Product quality is very important because many faults have a real impact on users directly. It is significative that Markov chain theory is applied in product quality prediction. A new method of state transition probability matrix in Markov chain is used to provide fault diagnosis and quality prediction in this paper. The aim of this study is to deliver a quality prediction method for mechanical and electrical product by using the data collection and analysis techniques. Firstly, a new approach of fault diagnosis was presented by using the theory of state transition probability matrix. Then, Markov chain theory is used to build the mathematical patterns in the quality prediction with the state transition probability matrix. Finally, we illustrate the veracity rationality and science of the approved method using a numerical example of ball valves.

One of the central considerations in the theory of Markov chains is their convergence to an equilibrium. Coefficients of ergodicity provide an efficient method for such an analysis. Besides giving sufficient and sometimes necessary conditions for convergence, they additionally measure its rate. In this paper we explore coefficients of ergodicity for the case of imprecise Markov chains. The latter provide a convenient way of modelling dynamical systems where parameters are not determined precisely. In such cases a tool for measuring the rate of convergence is even more important than in the case of precisely determined Markov chains, since most of the existing methods of estimating the limit distributions are iterative. We define a new coefficient of ergodicity that provides necessary and sufficient conditions for convergence of the most commonly used class of imprecise Markov chains. This so-called weak coefficient of ergodicity is defined through an endowment of the structure of a metric space to the class of imprecise probabilities. Therefore we first make a detailed analysis of the metric properties of imprecise probabilities.

SummaryConsider a system of n players in which each initially starts on a different team. At each time step, we select an individual winner and an individual loser randomly and the loser joins the winner's team. The resulting Markov chain and stochastic matrix clearly have one absorbing state, in which all players are on the same team, but the combinatorics along the way are surprisingly elegant. The expected number of time steps until each team is eliminated is a ratio of binomial coefficients. When a team is eliminated, the probabilities that the players are configured in various partitions of n into t teams are given by multinomial coefficients. The expected value of the time to absorbtion is (n - 1)2 steps. The results depend on elementary combinatorics, linear algebra, and the theory of Markov chains.

The aim of our research is to contribute to the modeling of relative humidity with Markov chains and its use in crop protection forecasting in the Tokaj-Hegyalja wine region. The two most common wine grape varieties in this region are Vitis vinifera cv. Furmint and V. vinifera cv. Hárslevelű. For noble rot on these varieties, humidity in the optimum range is an essential environmental condition. Using 10-min and 60-min resolution humidity data sets from the Hungarian Meteorological Service (HungaroMet) for the period 2002−2020, we estimated the probability that a random humidity measurement will fall into a given result (five humidity categories) in the future. The mathematical model used for this research is the theory of Markov chains, a new approach to statistical analysis of relative humidity data. It is pointed out that the Markov process of relative humidity variation and the results of a 60-min resolution data series across 19 years of study can be used to model humidity.

Purpose. To improve the operational reliability and safety of mining facilities by updating the existing methods and developing new ones for determining the residual life; to analyse the causes of the most typical defects in mining buildings and structures and, based on the results, to estimate the residual life using probabilistic methods based on Markov chains. Research methods. General scientific research methods were used: statistical, probability theory, information theory, Markov chain theory, formalisation, analysis (including retrospective) - during solution development at the level of individual structural elements; basic principles of system organisation and system analysis - during the construction of a diagnostic model. Results. A new method of determining the residual life and the categories of technical state of buildings and structures based on a Markov chain is proposed. Recommendations for systematising the interconnection of subsystems, their technical state and relative reliability of the facility as a whole are formulated. Scientific novelty. Scientific novelty. The developed methodology, based on Markov chains, allows not only to thoroughly analyze the history of defects and their interrelationships, but also to take into account the probability of new defects in structural elements based on previous data, which in turn makes it possible to manage the technical condition of building structures. Practical significance. The proposed methodology for analysing and assessing the residual life based on Markov chains reflects in-depth perceptions of the complex processes occurring in buildings and structures of the mining industry. The Markov chain-based methodology both increases the efficiency of inspection and diagnostics of facilities and simplifies this process, reducing labour costs and workload. As a result, it improves the speed and accuracy of maintenance and repair decisions, thereby reducing the risk of accidents

We have suggested that sine-wave based acuity gratings (such as Gabor functions) should be replaced by a new set of test functions - functions which act as the eigenfunctions of an appropriate operator (Stewart & Pinkham, in press). The Hermite functions emerge as an important special case, and have recently been applied to problems in image analysis (Martens, 1990). The Hermite functions are generated as the eigenfunctions of a differential operator which produces a simple and appealing mathematical model of space-variant acuity. The operator’s mathematical properties are in better agreement with the spatial inhomogeneity of the visual field than is the Fourier kernel. Surprisingly, these functions are their own Fourier transformations, and so are easily related to the Fourier transforms which are used in image analysis. If we are correct, human factors work on visual acuity can be placed on a coherent mathematical foundation, and new approaches to evaluating the quality of visual images will emerge. Just as important, a space-variant model of human visual acuity includes as a special case, the space-invariant models of image analysis used in engineering studies of image processing. The Hermite decomposition of the stimulus is directly related to the eigenfunctions of our space-variant acuity operator. The theory of Markov chains can be tied directly to the components of such a Hermite decomposition. The Hermite breakdown of two-dimensional and one-dimensional fractional Brownian motion produces components having corresponding spectra. Consequently, the information inherent in two- dimensional fractional Brownian motion is preserved under the Hermite decomposition, and finds a natural expression as a Markov process. Markov models are widely used for automatic speech recognition. Similar methods can be used to capture the inherent structure of visual images. Some natural scenes can be represented by fractals. Many images can be treated as

A very important class of models widely used nowadays to describe and predict, at least in stochastic terms, the behavior of many-particle systems (where the word “particle” is not meant in the purely mechanical sense: particles can be cells of a living tissue, or cars in a traffic flow, or even members of an animal or human population) is the Kinetic Theory for Active Particles, i.e., a scheme of possible generalizations and re-interpretations of the Boltzmann equation. Now, though in the literature on the subject this point is systematically disregarded, this scheme is based on Markov Chains, which are special stochastic processes with important properties they share with many natural processes. This circumstance is here carefully discussed not only to suggest the different ways in which Markov Chains can intervene in equations describing the stochastic behavior of any many-particle system, but also, as a preliminary methodological step, to point out the way in which the notion of a Markov Chain can be suitably generalized to this aim. As a final result of the discussion, we find how to develop new very plausible and likely ways to take into account possible effects of the external world on a non-isolated many-particle system, with particular attention paid to socio-economic problems.

Article Free Access Share on Exact sampling and approximate counting techniques Author: Mark Huber School of Operations Research and Industrial Engineering, Cornell University, Ithaca, NY School of Operations Research and Industrial Engineering, Cornell University, Ithaca, NYView Profile Authors Info & Claims STOC '98: Proceedings of the thirtieth annual ACM symposium on Theory of computingMay 1998 Pages 31–40https://doi.org/10.1145/276698.276709Published:23 May 1998Publication History 25citation666DownloadsMetricsTotal Citations25Total Downloads666Last 12 Months66Last 6 weeks15 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

Many-objective charging optimization for electric vehicles considering demand response and multi-uncertainties based on Markov chain and information gap decision theory

Introduction. This is a sequel to my paper [1]. The present developments are largely independent of the previous results except in so far as given in the Appendix. Theorem 1 shows a kind of solidarity among the states of a recurrent class; it generalizes a classical result due to Kolmogorov and permits a classification of recurrent states and classes. In ?2 some relations involving the mean recurrence and first passage times are given. In ??3-5 sequences of random variables associated in a natural way with a Markov chain are studied. Theorem 2 is a generalized ergodic theorem which applies to any recurrent class, positive or null. It turns out that in a null class there is a set of numbers which plays the role of stationary absolute probabilities. In the case of a recurrent random walk with independent, stationary steps these numbers are all equal to one and the result is particularly simple. Theorem 3 shows that the kind of solidarity exhibited in Theorem 1 persists in such a sequence; it leads to the clarification of certain conditions stated by Doblin(2) in connection with his central limit theorem. Using a fundamental idea due to Doblin, the weak and strong laws of large numbers, the central limit theorem, the law of the iterated logarithm, and the limit theorems for the maxima of the associated sequence are proved very simply. Owing to the great simplicity of the method it is the conditions of validity of these limit theorems that should deserve attention. Among other things, we shall show by an example that a certain set of conditions, attributed to Kolmogorov, is in reality not sufficient for the validity of the central limit theorem. Furthermore, conditions of validity for the strong limit theorems and the limit theorems for the maxima are obtained by a rather natural strengthening of corresponding conditions for the weak limit theorems. A word about the connection of these conditions with martingale theory closes the paper. 1. The sequence of random variables {Xn}, n=0, 1, 2, * , forms a denumerable Markov chain with stationary transition probabilities. The states will be denoted by the non-negative integers(3) 0, 1, 2, * * . The n-step transition probability from the state i to the state j will be denoted by P(n) (P(l) = Pij). Thus we have

A major part of this book discusses the use of stochastic models for atmospheric convection, namely through the use of a stochastic model for CIN and a stochastic multicloud model which pertains to tracking the statistics of clouds of various types. These models rely on the theory of stochastic processes and Markov chains in particular. Here, we provide a brief introduction to this topic to help the reader better appreciate and comprehend the cloud models. The expert reader can skip this chapter. It is intended to readers with no or rather very little background in probability theory and stochastic processes. Nonetheless, we assume that the reader is familiar with the basic notions of probability distributions and random variables.