A Probabilistic Model for Automatic Segmentation of the Esophagus in 3-D CT Scans

In this thesis, the properties of some non-standard Markov chain models and their corresponding parameter estimation methods are investigated. Several practical applications and extensions are also discussed.
\n
\nThe estimation of model parameters plays a key role in the real-world applications of Markov chain models. Some widely used estimation methods for Markov chain models are based on the existence of stationary vectors. In this thesis, some weaker sufficient conditions for the existence of stationary vectors for highorder Markov chain models, multivariate Markov chain models and high-order multivariate Markov chain models are proposed. Furthermore, for multivariate Markov chain models, a new estimation method based on minimizing the prediction error is proposed. Numerical experiments are conducted to demonstrate the efficiency of the proposed estimation methods with an application in demand prediction.
\n
\nHidden Markov Model (HMM) is a bivariate stochastic process such that one of the process is hidden and the other is observable. The distribution of observable sequence depends on the hidden sequence. In a traditional HMM, the hidden states directly affect the observable states but not vice versa. However, in reality, observable sequence may also have effect on the hidden sequence. For this reason, the concept of Interactive Hidden Markov Model (IHMM) is introduced, whose key idea is that the transitions of the hidden states depend on the observable states too. In this thesis, efforts are devoted in building a highorder IHMM where the probability laws governing both observable and hidden states can be written as a pair of high-order stochastic difference equations. We also propose a new model by capturing the effect of observable sequence on the hidden sequence through using the threshold principle. In this case, reference probability methods are adopted in estimating the optimal model parameters, while for unknown threshold parameter, Akaike Information Criterion (AIC) is used. We explore asset allocation problems from both domestic and foreign perspective where asset price dynamics follows autoregressive HMM. The object of an investor is not only to maximize the expected utility of the terminal wealth, but also to ensure that the risk of the portfolio described by the Value-at-Risk (VaR) does not exceed a specified level.
\n
\nIn many decision processes, fuzziness is a major source of imprecision. As a perception of usual Markov chains, the definition of fuzzy Markov chains is introduced. Compared to traditional Markov chain models, fuzzy Markov chains are relatively new and many properties of them are still unknown. Due to the potential applications of fuzzy Markov chains, we provide some characterizations to ensure the ergodicity of these chains under both max-min and max-product compositions.

Simple SummaryUnderstanding and predicting vector population and related disease dynamics, is crucial for gaining insight into the abundance and dynamics of arthropod disease vectors, and for the design of effective vector control strategies. Several mathematical and standard epidemiological models have been proposed in studying vector transmitted infectious disease dynamics. However, most models are of deterministic nature and are not able to estimate other relevant metrics such as the probability of vector population emergence as well as the probability and expected time to reach certain population and/or infection state. Here we are focusing on stochastic modeling of mosquito abundance data using weather driven Markov chains (MCs) and are particularly interested in estimating transition probabilities (TPs) between different population levels. A MC model is based on the assumption that the future state of the variable is only dependent on the present state and is suitable in cases of short and noisy data characterized by a complex and random behavior. The aim is to introduce and generalize a formulation of conditional Markov chain models (CMSs) for predicting probability transition estimates of arthropod vector populations. In this context, first we present the basic principles and assumptions behind Markov chain modeling approach, with an intuitive interpretation of the integration of conditional Markov chains (CMCs) and then demonstrate the usefulness of the approach in predicting the abundance of Culex sp. We conclude that the conditional Markov chain technique is recommended as viable for modeling populations that explicit random dynamics and predict their future evolution. Although, the Markov models generated in this work provide an accurate abstraction of the vector disease progress observed within the dataset used for their generation, we envision the current approach as an entry point into the medical entomology literature and methods for predicting arthropod vector diseases dynamics.Understanding and predicting mosquito population dynamics is crucial for gaining insight into the abundance of arthropod disease vectors and for the design of effective vector control strategies. In this work, a climate-conditioned Markov chain (CMC) model was developed and applied for the first time to predict the dynamics of vectors of important medical diseases. Temporal changes in mosquito population profiles were generated to simulate the probabilities of a high population impact. The simulated transition probabilities of the mosquito populations achieved from the trained model are very near to the observed data transitions that have been used to parameterize and validate the model. Thus, the CMC model satisfactorily describes the temporal evolution of the mosquito population process. In general, our numerical results, when temperature is considered as the driver of change, indicate that it is more likely for the population system to move into a state of high population level when the former is a state of a lower population level than the opposite. Field data on frequencies of successive mosquito population levels, which were not used for the data inferred MC modeling, were assembled to obtain an empirical intensity transition matrix and the frequencies observed. Our findings match to a certain degree the empirical results in which the probabilities follow analogous patterns while no significant differences were observed between the transition matrices of the CMC model and the validation data (ChiSq = 14.58013, df = 24, p = 0.9324451). The proposed modeling approach is a valuable eco-epidemiological study. Moreover, compared to traditional Markov chains, the benefit of the current CMC model is that it takes into account the stochastic conditional properties of ecological-related climate variables. The current modeling approach could save costs and time in establishing vector eradication programs and mosquito surveillance programs.

In many applications, successive observations of a process, say X 1, X 2,…, have an inherent time component associated with them. For example, the X i could be the state of the weather at a particular location on the i th day, counting from some fixed day. In a simplistic model, the state of the weather could be “dry” or “wet,” quantified as, say, 0 and 1. It is hard to believe that in such an example, the sequence X 1, X 2,… could be mutually independent. The question then arises how to model the dependence among the X i ′s. Probabilists have numerous dependency models. A particular model that has earned a very special status is called the Markov chain. In a Markov chain model, we assume that the future, given the entire past and the present state of a process, depends only on the present state. In the weather example, suppose we want to assign a probability that tomorrow, say March 10, will be dry, and suppose that we have available to us the precipitation history for each day from January 1 to March 9. In a Markov chain model, our probability that March 10 will be dry will depend only on the state of the weather on March 9, even though the entire past precipitation history was available to us. As simple as it sounds, Markov chains are enormously useful in applications, perhaps more than any other specific dependency model. They also are independently relevant to statistical computing in very important ways. The topic has an incredibly rich and well-developed theory, with links to many other topics in probability theory. Familiarity with basic Markov chain terminology and theory is often considered essential for anyone interested in studying statistics and probability.We present an introduction to basic Markov chain theory in this chapter.KeywordsMarkov ChainStationary DistributionMarkov Chain ModelTransition Probability MatrixStationary Markov ChainThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

An environmental problem which is of concern across the globe nowadays is air pollution. The extent of air pollution is often studied based on data on the observed level of air pollution. Although the analysis of air pollution data that is available in the literature is numerous, studies on the dynamics of air pollution with the allowance for spatial interaction effects through the use of the Markov chain model are very limited. Accordingly, this study aims to explore the potential impact of spatial dependence over time and space on the distribution of air pollution based on the spatial Markov chain (SMC) model using the longitudinal air pollution index (API) data. This SMC model is pertinent to be applied since the daily data of API from 2012 to 2014 that have been gathered from 37 different air quality stations in Peninsular Malaysia is found to exhibit the property of spatial autocorrelation. Based on the spatial transition probability matrices found from the SMC model, specific characteristics of air pollution are studied in the regional context. These characteristics are the long-run proportion and the mean first passage time for each state of air pollution. It is found that the probability for a particular station's state to remain good is 0.814 if its neighbors are in a good state of air pollution and 0.7082 if its neighbors are in a moderate state. For a particular station having neighbors in a good state of air pollution, the proportion of time for it to continue being in a good state is 0.6. This proportion reduces to 0.4, 0.01, and 0 for the cell of moderate, unhealthy, and very unhealthy states, respectively. In addition, there exists a significant spatial dependence of API, indicating that air pollution for a particular station is dependent on the states of the neighboring stations.

<p>Mixed traffic systems are dynamically complex since there are many parameters and variables that influence the interactions between the different kinds of vehicles. Modeling the behavior of vehicles, especially motorcycle which has erratic behavior is still being developed continuously, especially in developing countries which have heterogeneous traffic. To get a better understanding of motorcycle behavior, one can look at maneuvers performed by drivers. In this research, we tried to build a model of motorcycle movement which only focused on maneuver action to avoid the obstacle along with the trajectories using a Markov Chain approach. In Markov Chain, the maneuver of motorcycle will described by state transition. The state transition model is depend on probability function which will use for determine what action will be executed next. The maneuver of motorcycle using Markov Chain model was validated by comparing the analytical result with the naturalistic data, with similarity is calculated using MSE. In order to know how good our proposed method can describe the maneuver of motorcycle, we try to compare the MSE of the trajectory based on Markov Chain model with those using polynomial approach. The MSE results showed the performance of Markov Chain Model give the smallest MSE which 0.7666 about 0.24 better than 4th order polynomial.</p>

기존의 정상성 Markov Chain 모형은 자료 자체의 Markov 특성만을 고려하여 모의하는 기법으로서 수자원 설계에서 여러 가지 목적으로 이용되어 지고 있다. 그러나 일강수량의 천이확률 및 매개변수 등이 과거와 일정하다는 정상성을 기본 가정으로 하기 때문에 평균의 변동성 등과 같은 외부충격을 모형에 적용할 수 없다. 이러한 관점에서 본 연구의 가장 큰 목적은 기존일강수량 모형을 외부인자를 받아들일 수 있는 모형으로 개발하는 것이다. 즉, Markov Chain 모형의 매개변수인 천이확률과 확률분포형의 매개변수 등을 연결함수(link function)를 통해 외부인자와 연동하도록 하였으며 정준상관분석을 통해 매개변수를 추정하였다. 개발된 모형을 서울지방 1961-2006년까지의 일강수량 자료를 대상으로 검증하는 절차를 가졌다. 추정된 결과를 보면 계절강수량의 특성뿐만 아니라 일강수량의 특성 또한 적절하게 모의되는 것을 확인할 수 있다. 따라서 본 연구에서 개발된 모형은 GCM 예측결과를 입력자료로 활용한다면 일강수계열의 장단기 모의를 위한 downscaling 기법으로 사용될 수 있다. 또한, 기후변화 시나리오가 입력자료로 이용된다면 기후변화에 따른 수자원 영향 평가를 위한 downscaling 기법으로 활용이 가능할 것으로 판단된다. A stationary Markov chain model is a stochastic process with the Markov property. Having the Markov property means that, given the present state, future states are independent of the past states. The Markov chain model has been widely used for water resources design as a main tool. A main assumption of the stationary Markov model is that statistical properties remain the same for all times. Hence, the stationary Markov chain model basically can not consider the changes of mean or variance. In this regard, a primary objective of this study is to develop a model which is able to make use of exogenous variables. The regression based link functions are employed to dynamically update model parameters given the exogenous variables, and the model parameters are estimated by canonical correlation analysis. The proposed model is applied to daily rainfall series at Seoul station having 46 years data from 1961 to 2006. The model shows a capability to reproduce daily and seasonal characteristics simultaneously. Therefore, the proposed model can be used as a short or mid-term prediction tool if elaborate GCM forecasts are used as a predictor. Also, the nonstationary Markov chain model can be applied to climate change studies if GCM based climate change scenarios are provided as inputs.

RM3 A Comparison Between Markov Chains and System Dynamics Modeling for the Estimation of Metabolic Syndrome Costs in a Public Health Care Delivery Organization in MÉxico

ABSTRACTQuantitative damage detection in composite single lap joints during fatigue is difficult and not practical with many traditional experimental methods. The objective of this study is to combine a new experimental analysis method with existing stochastic models that is capable of taking the scatter inherent in composite fatigue data into account for better representation of damage evolution in single‐lap joints. This study presents a new quantitative approach where thermoelastic stress analysis (TSA), a remote sensing and non‐destructive evaluation technique, is used for in situ continuous monitoring for fatigue damage detection of Fibre Reinfoced Polymeric (FRP) S2‐glass/E733FR single‐lap joints. Image analysis algorithms are developed to capture the debonding fronts, and Markov Chain models (MCMs) are developed to capture the fatigue damage evolution in composite lap‐joints based on the experimental analysis. The TSA damage index at the last fatigue cycle is used to calibrate the MCMs, and damage predictions using the MCMs are then examined at different fatigue cycles. Methods to construct a predictive stochastic S–N curve utilizing the MCMs are discussed. The proposed IR‐TSA with MCMs is shown to be very effective in predicting the damage evolution and allowed constructing S–N curves that take the variability into account for damage in composite single‐lap joints.

A well-reasoned model of data movie in single molecule localization microscopy (SMLM) is desired. A model of data movie can be decomposed into a model of emitter activation process and a model of data frame. In this paper, we focus on Markov chain modeling and analyzing of emitter activation process for both cycled and continuous illuminations. First, a two-phase Markov chain is proposed to model the activation process for a pair of conjugated activator and emitter with cycled illumination. By converting the frame-based Markov chain into several cycle-based Markov chains, the stationary state distribution in the photoactivatable period is derived. Further obtained are several formulas that capture the characterization of the two-phase Markov chain. Second, the Markov chain and analytical result are extended to the continuous illumination where an emitter is excited continuously in all frames. Finally, incorporating the model of emitter activation process with our previous model of data frame, the model of data movie for both cycled and continuous illuminations in 3D and 2D imaging are simulated by custom codes. It is shown that the model can synthesize data movies well and the analytical formulas predict the simulation results accurately. The models provide a means to be broadly utilized in generating well-reasoned data movies for training of neural networks and evaluation of localization algorithms.

This paper explores the inter-connectedness of the evolution of transportation networks and land use through the application of a Markov Chain mode. The model investigates how individual cells, with both land use and transportation network attributes, change over time. While this paper does not consider land use density directly, it does not consider land use type, and as cells change, it can be concluded that some form of development is likely to be occurring. The next section of the paper outlines the Markov Chain model and a discussion of the Twin Cities data used in the study follows. The subsequent section develops the transition probability matrices used by the Markov Chain. Those matrices are analyzed in order to understand the empirical regularities that appear in the data. They are then applied to both assess the predictive ability of the Markov Chain model (comparing what the model would predict with what actually happened), and then applying the mode to forecast future changes the Twin Cities area. The paper concludes with some suggestions for future research.

A simple formulation and solution to the replacement problem: A practical tool to assess the economic cow value, the value of a new pregnancy, and the cost of a pregnancy loss

PurposeThe purpose of this paper is to check the consistency of the IEC 61508 standard formula related to the average failure frequency (PFH: the probability of dangerous failure per hour) for a commonly used safety instrumented system (SIS) architecture in the process industry: 2-out-of-3 voting (2oo3), also known as Triple Modular Redundancy (TMR).Design/methodology/approachIEC 61508 standard provided PFH formulas for different SIS architectures, without explanations, assuming that the SIS puts the equipment under control into a safe state on the detection of dangerous failure. This assumption renders the use of classical reliability approaches such as fault trees and reliability block diagrams impractical for PFH calculation. That said, the consistency verification was performed thanks to a dynamic and flexible reliability approach, namely Markov chains following these steps: (1) developing the multi-phase Markov chains (MPMC) model for 2oo3 configuration, (2) deducing the related classical Markov chains (CMC) model and (3) deriving a new PFH formula for the 2oo3 architecture based on the CMC model and thoroughly comparing it to that given in the IEC 61508. Moreover, 2oo3 architecture has been modeled through Petri nets for numerical comparison purposes. That comparison has been carried out between the numerical results obtained from IEC 61508 formula, the newly derived formula, Markov chains and Petri nets models.FindingsThe newly obtained formula for 2oo3 configuration contains extra terms compared with the IEC 61508 one. Therefore, this latter formula induces an underestimated PFH results, which is dangerous from a safety point of view. This fact was corroborated by the numerical comparison.Research limitations/implicationsThis paper does not consider the different configurations given in IEC 61508.Originality/valueIn our knowledge, no verification works have been conducted before on the IEC 61508 PFH formulas with shutdown capability. Therefore, the nonaccuracy of the PFH formula related to the 2oo3 has not been stated before. This paper proposes a new and more accurate formula.

A Novel Markov Model for Near-Term Railway Delay Prediction

The Markov Chains model was applied on a sample of citation data to predict the shifting trends of scientific inquiry from science and technology to social science subjects [7]. The validity of the findings awaits verification by future data. The same technique has recently been utilized to predict the movements of authors among nine subtopics in ethnomusicology [11]. Although a validation was attempted, unfortunately, inadequate data rendered the comparison inconclusive. In the present study, publications from the 1961–1970 Index were used to model the process of subfield shifts in a physics journal. It was assumed that authors movements among subfields of physics as shown in the subject headings assigned would reflect the subject shifts of the journal. The rate of change between the initial probability distribution and the limiting probability distribution were used to predict developmental trends. Finally, the subject distribution of publications appearing in the next ten-year Index validated the prediction. It is reasonable to conclude that the Markov Chains model appears to predict the shifts of research trends in a specialty journal.

The stochastic nature of wind energy generation introduces uncertainties and risk in generation schedules computed using optimal power flow (OPF). This risk is quantified as expected energy not served (EENS) and computed via an error distribution found for each hourly forecast. This thesis produces an accurate method of estimating EENS that is also suitable for real-time OPF calculation. This thesis examines two statistical predictive models used to forecast hourly production of wind energy generators (WEGs), Markov chain model, and auto-regressive moving-average (ARMA) model, and their effects on EENS. Persistence model is used as a benchmark for comparison. For persistence and ARMA models, both Gaussian and Cauchy error distributions are used to compute EENS via a closed-form solution that reduces computational complexity. Markov chain and ARMA both provide accurate forecasts of WEG power generation though Markov Chain model performs significantly better. The Markov chain model also produces the most accurate EENS estimate of the three models.