Introduction to Stochastic Processes, Markov Chains, and Monte Carlo Simulation

Graduates in biology or genetics learn Mendel’s laws and Hardy–Weinberg equilibrium as students, and they know, use, and sometimes teach these concepts. However, it is unusual to learn about stochastic processes during the graduate studies of these topics, although the applications of Markov chains in the fields of genetics or biology are increasing. In this paper, and from Mendel’s Laws, we are going to obtain an example of the Hardy–Weinberg Equilibrium that can be used to teach it. In the demonstration, stochastic processes theory and Markov chains will be used, in order to develop the transition matrix. In this way, we obtain an example that can be used to illustrate an explanation that involves, through Mendel’s laws, Hardy–Weinberg equilibrium for degree students, and stochastic processes for advanced students in biology or genetic sciences (predoctoral students at university). In this example, we are going to use the Scientific Workplace version 6.0 software.

Previous article Next article Second-Order Markov and Reciprocal Stationary Gaussian ProcessesR. N. MiroshinR. N. Miroshinhttps://doi.org/10.1137/1124099PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAbout[1] S. N. Bernshtei˘n, On the dependencies among random variablesCollected Works, Vol. IV, Nauka, Moscow, 1964, 235–254, (In Russian.) Google Scholar[2] J. L. Doob, Stochastic processes, John Wiley & Sons Inc., New York, 1953viii+654 MR0058896 0053.26802 Google Scholar[3] Benton Jamison, Reciprocal processes: The stationary Gaussian case, Ann. Math. Statist., 41 (1970), 1624–1630 MR0267637 0248.60030 CrossrefGoogle Scholar[4] I. I. Gikhman and , A. V. Skorokhod, The Theory of Stochastic Processes, Vol. I, Springer-Verlag, New York, 1974 CrossrefGoogle Scholar[5] E. Kamke, Differentialgleichungen. Lösungsmethoden and Lösungen, Vol. 1, Chelsea, New York, 1974 Google Scholar[6] E. Wong, Some results concerning the zero-crossings of Gaussian noise, SIAM J. Appl. Math., 14 (1966), 1246–1254 10.1137/0114099 MR0207059 0147.15905 LinkGoogle Scholar[7] E. Wong, The distribution of intervals between zeros for a stationary Gaussian process, SIAM J. Appl. Math., 18 (1970), 67–73 10.1137/0118008 MR0256456 0221.60024 LinkGoogle Scholar[8] R. N. Miroshin, Convergence of Rice and Longuet-Higgins series for a Wong process, Theory Prob. Applications, 21 (1976), 863–866 10.1137/1121103 0374.60049 LinkGoogle Scholar[9] R. N. Miroshin, Asymptotics of the second factorial moment of the number of upward crossings of the line $kt+a$ by a Gaussian stationary process, Vestnik Leningrad. Univ., 23 (1968), 109–120 MR0235612 0165.52602 Google Scholar[10] M. Krein, Sur le problème du prolongement des fonctions hermitiennes positives et continues, C. R. (Doklady) Acad. Sci. URSS (N.S.), 26 (1940), 17–22 MR0004333 0022.35302 Google Scholar Previous article Next article FiguresRelatedReferencesCited byDetails Characterization of Multivariate Stationary Gaussian Reciprocal DiffusionsJournal of Multivariate Analysis, Vol. 62, No. 1 Cross Ref Gaussian reciprocal diffusions and positive definite sturm-liouville operators4 April 2007 | Stochastics and Stochastic Reports, Vol. 55, No. 3-4 Cross Ref Gaussian reciprocal processes revisitedStatistics & Probability Letters, Vol. 12, No. 4 Cross Ref Reciprocal covariance solutions of some matrix differential equationsStochastic Processes and their Applications, Vol. 37, No. 1 Cross Ref Characterization and Classification of Gaussian Second Order Reciprocal Processes Cross Ref Gaussian reciprocal processes and self-adjoint stochastic differential equations of second order2 May 2007 | Stochastics and Stochastic Reports, Vol. 34, No. 1-2 Cross Ref Reciprocal diffusions and stochastic differential equations of second order ∗Stochastics, Vol. 24, No. 4 Cross Ref Multivariate reciprocal stationary Gaussian processesJournal of Multivariate Analysis, Vol. 23, No. 1 Cross Ref On Miroshin’s Second-Order Reciprocal ProcessesJulia Abrahams3 August 2006 | SIAM Journal on Applied Mathematics, Vol. 44, No. 1AbstractPDF (316 KB)The Use of Rice SeriesR. N. Miroshin28 July 2006 | Theory of Probability & Its Applications, Vol. 28, No. 4AbstractPDF (1011 KB)The zero-crossing problem for some nonstationary Gaussian processes (Corresp.)IEEE Transactions on Information Theory, Vol. 28, No. 4 Cross Ref Convergence of Longuet-Higgins Series for Stationary Gaussian Markov Processes of First OrderR. N. Miroshin17 July 2006 | Theory of Probability & Its Applications, Vol. 26, No. 1AbstractPDF (1546 KB) Volume 24, Issue 4| 1980Theory of Probability & Its Applications History Submitted:11 July 1977Published online:17 July 2006 InformationCopyright © Society for Industrial and Applied MathematicsPDF Download Article & Publication DataArticle DOI:10.1137/1124099Article page range:pp. 845-852ISSN (print):0040-585XISSN (online):1095-7219Publisher:Society for Industrial and Applied Mathematics

Commonly shape and material optimization for complex structures having complicated boundary and loading conditions requires FE analysis in order to determine accurately enough (especially for composite structures) objective functions. The solutions of such class of problems become difficult as we use probabilistic algorithms. Therefore, a general method using the Markov chain and stochastic equations is proposed to reduce the total number of iterations (computations) that are necessary to obtain a global optimum (or a quasi-global). The proposed method can be applied both for isotropic and anisotropic (composite laminates) structures. The method is illustrated by a numerical example.

In many applications, successive observations of a process, say X 1, X 2,…, have an inherent time component associated with them. For example, the X i could be the state of the weather at a particular location on the i th day, counting from some fixed day. In a simplistic model, the state of the weather could be “dry” or “wet,” quantified as, say, 0 and 1. It is hard to believe that in such an example, the sequence X 1, X 2,… could be mutually independent. The question then arises how to model the dependence among the X i ′s. Probabilists have numerous dependency models. A particular model that has earned a very special status is called the Markov chain. In a Markov chain model, we assume that the future, given the entire past and the present state of a process, depends only on the present state. In the weather example, suppose we want to assign a probability that tomorrow, say March 10, will be dry, and suppose that we have available to us the precipitation history for each day from January 1 to March 9. In a Markov chain model, our probability that March 10 will be dry will depend only on the state of the weather on March 9, even though the entire past precipitation history was available to us. As simple as it sounds, Markov chains are enormously useful in applications, perhaps more than any other specific dependency model. They also are independently relevant to statistical computing in very important ways. The topic has an incredibly rich and well-developed theory, with links to many other topics in probability theory. Familiarity with basic Markov chain terminology and theory is often considered essential for anyone interested in studying statistics and probability.We present an introduction to basic Markov chain theory in this chapter.KeywordsMarkov ChainStationary DistributionMarkov Chain ModelTransition Probability MatrixStationary Markov ChainThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The Perron-Frobenius Projection in the Theory of Graphs, Digraphs, Designs and Stochastic Processes

The notion of 'stopping times' is a useful one in probability theory; it can be applied to both classical problems and fresh ones. This book presents this technique in the context of the directed set, stochastic processes indexed by directed sets, and many applications in probability, analysis and ergodic theory. Martingales and related processes are considered from several points of view. The book opens with a discussion of pointwise and stochastic convergence of processes, with concise proofs arising from the method of stochastic convergence. Later, the rewording of Vitali covering conditions in terms of stopping times clarifies connections with the theory of stochastic processes. Solutions are presented here for nearly all the open problems in the Krickeberg convergence theory for martingales and submartingales indexed by directed set. Another theme of the book is the unification of martingale and ergodic theorems.

With the advancement of modern agricultural technology and the expansion of large-scale production, this article aims to solve the difficulties in plant disease and pest control through the application of artificial intelligence and automation technology, and provide accurate disease and pest warning mechanisms. This study first conducted a detailed identification and classification of plant disease and pest warning mechanisms, and established a dynamic model of disease and pests based on the environmental factors and symptoms of affected areas. On this basis, using the isomorphism relationship between generalized stochastic Petri nets and Markov chains, a plant disease and pest diagnosis model based on generalized stochastic Petri nets and an equivalent Markov chain model were constructed. The simulation results show that different combinations of infection rates have a significant impact on the probability of meeting treatment standards, with the combination of moderate and severe infection rates having the greatest impact on the probability of meeting treatment standards, while the impact of mild infection rates is relatively small. By comprehensively analyzing the interaction between mild, moderate, and severe infection rates, the critical zone surface under different disease and pest warning thresholds was obtained. Through actual data verification, the generalized stochastic Petri net model can effectively quantify the dynamic characteristics of disease and pest propagation. Combined with the equivalent analysis of Markov chains, it can provide key thresholds and decision support for disease and pest warning. This method provides a theoretical basis for automated monitoring and precise control of pests and diseases in large-scale agricultural planting, and it has high practical application value.

The El Niño-Southern Oscillation (ENSO) is the most prominent climate system in the tropical Pacific. However, its simulation, including the amplitude, phase locking, and asymmetry of its two phases, is not well reproduced by the current climate system models. In this study, the sensitivity of the ENSO simulation to the convection schemes is discussed using the Nanjing University of Information Science and Technology Earth System version 3.0 (NESM3) model. Three convection schemes, including the default, the default coupled with the stochastic multicloud model (SMCM), and the default used in the Coupled Model Intercomparison Project Phase 6 (CMIP6), are implemented. The model results reveal that the low-level cloud cover and surface net shortwave radiation are best represented over the tropical Pacific in the model containing the SMCM. The simulations of the ENSO behavior’s response to changes in the convection scheme are not uniform. The model results reveal that the model containing the SMCM performs best in terms of simulating the seasonal cycle of the sea surface temperature anomaly along the equatorial Pacific, the phase locking, and the power spectrum of ENSO but with a modest ENSO amplitude. Compared to the model containing the default convection scheme, the coupling of the default scheme and the SMCM provides a good simulation of the ENSO’s asymmetry, while the model containing the CMIP6 convection scheme outperforms the others in terms of the simulation of the ENSO’s amplitude. Two atmospheric feedback processes were further discussed to investigate the factors controlling the ENSO’s amplitude. The analyses revealed that the strongest positive atmospheric Bjerknes feedback and the thermodynamic damping of the surface net heat flux occurred in the model containing the CMIP6 convection scheme, suggesting that the atmospheric Bjerknes feedback may overwhelm the heat flux damping feedback when determining the ENSO’s amplitude. The results of this study demonstrate that perfectly modeling and predicting the ENSO is not simple, and it is still a large challenge and issue for the entire model community in the future.

At first there was the Markov property. The theory of stochastic processes, which can be considered as an exten- sion of probability theory, allows the modeling of the evolution of systems through the time. It cannot be properly understood just as pure mathemat- ics, separated from the body of experience and examples that have brought it to life. The theory of stochastic processes entered a period of intensive develop- ment, which is not finished yet, when the idea of the Markov property was brought in. Not even a serious study of the renewal processes is possible without using the strong tool of Markov processes. The modern theory of Markov processes has its origins in the studies by A. A: Markov (1856-1922) of sequences of experiments connected in a chain and in the attempts to describe mathematically the physical phenomenon known as Brownian mo- tion. Later, many generalizations (in fact all kinds of weakenings of the Markov property) of Markov type stochastic processes were proposed. Some of them have led to new classes of stochastic processes and useful applications. Let us mention some of them: systems with complete connections [90, 91, 45, 86]; K-dependent Markov processes [44]; semi-Markov processes, and so forth. The semi-Markov processes generalize the renewal processes as well as the Markov jump processes and have numerous applications, especially in relia- bility.

Hopfield neural networks (HNN) are a class of densely connected single layer nonlinear networks of perceptrons. The network's energy function is defined through a learning procedure so that its minima coincide with states from a predefined set. However, because of the network's nonlinearity a number of undesirable local energy minima emerge from the learning procedure. This has shown to significantly effect the network's performance. In this work we present a stochastic process enhanced bipolar HNN. Presence of the stochastic process in the network enables us to describe its evolution using the Markov chains theory. When faced with a fixed network topology, the desired final distribution of states can be reached by modulating the network's stochastic process. Guided by the desired final distribution, we propose a general L/sup 2/ norm error density function optimization criterion for enhancement of the Hopfield neural network performance. This criterion can also be viewed in terms of stability intervals associated with the desired and non-desired stable states of the network. Because of the complexity of the general criterion we relax the optimization to the set of non-desired states. We further formulate a stochastic process design based on the stability intervals, which satisfies the optimization criterion and results in enhanced performance of the network. Our experimental simulations confirm the predicted improvement in performance.

The goal of this paper is to develop a general method to establish conditional ergodicity of infinite-dimensional Markov chains. Given a Markov chain in a product space, we aim to understand the ergodic properties of its conditional distributions given one of the components. Such questions play a fundamental role in the ergodic theory of nonlinear filters. In the setting of Harris chains, conditional ergodicity has been established under general nondegeneracy assumptions. Unfortunately, Markov chains in infinite-dimensional state spaces are rarely amenable to the classical theory of Harris chains due to the singularity of their transition probabilities, while topological and functional methods that have been developed in the ergodic theory of infinite-dimensional Markov chains are not well suited to the investigation of conditional distributions. We must therefore develop new measure-theoretic tools in the ergodic theory of Markov chains that enable the investigation of conditional ergodicity for infinite dimensional or weak-* ergodic processes. To this end, we first develop local counterparts of zero-two laws that arise in the theory of Harris chains. These results give rise to ergodic theorems for Markov chains that admit asymptotic couplings or that are locally mixing in the sense of H. F\"{o}llmer, and to a non-Markovian ergodic theorem for stationary absolutely regular sequences. We proceed to show that local ergodicity is inherited by conditioning on a nondegenerate observation process. This is used to prove stability and unique ergodicity of the nonlinear filter. Finally, we show that our abstract results can be applied to infinite-dimensional Markov processes that arise in several settings, including dissipative stochastic partial differential equations, stochastic spin systems and stochastic differential delay equations.

Purpose. To improve the operational reliability and safety of mining facilities by updating the existing methods and developing new ones for determining the residual life; to analyse the causes of the most typical defects in mining buildings and structures and, based on the results, to estimate the residual life using probabilistic methods based on Markov chains. Research methods. General scientific research methods were used: statistical, probability theory, information theory, Markov chain theory, formalisation, analysis (including retrospective) - during solution development at the level of individual structural elements; basic principles of system organisation and system analysis - during the construction of a diagnostic model. Results. A new method of determining the residual life and the categories of technical state of buildings and structures based on a Markov chain is proposed. Recommendations for systematising the interconnection of subsystems, their technical state and relative reliability of the facility as a whole are formulated. Scientific novelty. Scientific novelty. The developed methodology, based on Markov chains, allows not only to thoroughly analyze the history of defects and their interrelationships, but also to take into account the probability of new defects in structural elements based on previous data, which in turn makes it possible to manage the technical condition of building structures. Practical significance. The proposed methodology for analysing and assessing the residual life based on Markov chains reflects in-depth perceptions of the complex processes occurring in buildings and structures of the mining industry. The Markov chain-based methodology both increases the efficiency of inspection and diagnostics of facilities and simplifies this process, reducing labour costs and workload. As a result, it improves the speed and accuracy of maintenance and repair decisions, thereby reducing the risk of accidents

Fractional Brownian Motions, Fractional Noises and Applications

Markov chainMonteCarlo (MCMC)methodshavebeenaround for almost as longasMonte Carlo techniques, even though their impact on statistics was not truly felt until the very early 1990s, except in the specialized fields of spatial statistics and image analysis, where those methods appeared earlier. The emergence of Markov based techniques in physics is a story that remains untold within this survey (see Landau and Binder, 2005). Also, we will not enter into a description of MCMC techniques, unless they have some historical link, as the remainder of this volume covers the technical aspects.Acomprehensive treatment with further references can also be found in Robert and Casella (2004). We will distinguish between the introduction of Metropolis-Hastings based algorithmsand those related to Gibbs sampling, since they each stem from radically different origins, even though their mathematical justification via Markov chain theory is the same. Tracing the development of Monte Carlo methods, we will also briefly mention what we might call the “second-generation MCMC revolution.” Starting in the mid to late 1990s, this includes the development of particle filters, reversible jump and perfect sampling, and concludes with more current work on population or sequential Monte Carlo and regeneration and the computing of “honest” standard errors. As mentioned above, the realization that Markov chains could be used in a wide varietyof situations only came (tomainstream statisticians)withGelfand and Smith (1990), despite earlier publications in the statistical literature such as Hastings (1970), Geman and Geman (1984), and Tanner and Wong (1987). Several reasons can be advanced: lack of computing machinery (think of the computers of 1970!), or background onMarkov chains, or hesitation to trust in the practicality of themethod. It thus required visionary researchers like Gelfand and Smith to convince the community, supported by papers that demonstrated, through a series of applications, that the method was easy to understand, easy to implement and practical (Gelfand et al., 1990, 1992; Smith and Gelfand, 1992; Wakefield et al., 1994). The rapid emergence of the dedicated BUGS (Bayesian inference using Gibbs sampling) software as early as 1991, when a paper on BUGS was presented at the Valencia meeting, was another compelling argument for adopting, at large, MCMC algorithms.∗Monte Carlo methods were born in Los Alamos, New Mexico, during World War II, eventually resulting in theMetropolis algorithm in the early 1950s. WhileMonte Carlo methods were in use by that time, MCMC was brought closer to statistical practicality by the work of Hastings in the 1970s. What can be reasonably seen as the first MCMC algorithm is what we now call theMetropolis algorithm, published by Metropolis et al. (1953). It emanates from the same group of scientists who produced the Monte Carlo method, namely the research scientists of Los Alamos, mostly physicists working on mathematical physics and the atomic bomb. MCMC algorithms therefore date back to the same time as the development of regular(MC only) Monte Carlo methods, which are usually traced to Ulam and von Neumann in the late 1940s. StanislawUlam associates the original ideawith an intractable combinatorial computation he attempted in 1946 (calculating the probability of winning at the solitaire card game). This ideawas enthusiastically adopted by John vonNeumann for implementationwith direct applications to neutron diffusion, the name “Monte Carlo” being suggested by Nicholas Metropolis. Eckhardt (1987) describes these early Monte Carlo developments, and Hitchcock (2003) gives a brief history of the Metropolis algorithm. These occurrences very closely coincide with the appearance of the very first general-purpose digital computer, the ENIAC,which came to life in February 1946, after three years of construction. The Monte Carlo method was set up by von Neumann, who was using it on thermonuclear and fission problems as early as 1947. That same year, Ulam and vonNeumann invented inversion andaccept-reject techniques (also recounted inEckhardt, 1987) to simulate from nonuniform distributions. Without computers, a rudimentary version invented by Fermi in the 1930s went unrecognized (Metropolis, 1987). Note also that, as early as 1949, a symposiumonMonteCarlowas supported byRand, theNational Bureau of Standards, and theOakRidge laboratory and thatMetropolis andUlam (1949) published the very first paper about the Monte Carlo method.

This thesis is centered on the theory of stochastic processes and their applications in biological systems characterized by a complex environment. Three case studies have been modeled by the use of the three fundamental tools of stochastic processes: the master equation (ME), the stochastic differential equation (SDE) and the partial differential equation (PDE). The principal approach here applied to deal with complexity is the characterization of the system by means of probability distributions describing each a parameter of the model or the introduction of fractional order derivatives to include non-local and memory effects maintaining the linearity in the equations. In Chapter 1 we briefly review the theory of stochastic processes. In Chapter 2 we derive a birth-death process master equation to test if Long Interspersed Elements (LINEs) can be modeled according to the neutral theory of biodiversity. In Chapter 3 we derive a model of anomalous diffusion based on a Langevin approach in which anomalous behavior arise in the asymptotic intermediate state as a consequence of the heterogeneity of the system, from the superposition of Ornstein-Uhlenback processes. In Chapter 4 we propose an extension of the cable equation, used to describe anomalous diffusion phenomena as the signal conduction in spiny dendrites, by introducing a Caputo time fractional derivative.