Continuous time Markov chain based website navigability measure

This review article is concerned to provide a global context to several works on the fitting of continuous time nonhomogeneous Markov chains with finite state space and also to point out some selected aspects of two techniques previously introduced—estimation and calibration—relevant for applications and used to fit a continuous time Markov chain model to data by the adequate selection of parameters. The denomination estimation suits the procedure better when statistical techniques—e.g., maximum likelihood estimators—are employed, while calibration covers the case where, for instance, some optimisation technique finds a best approximation parameter to ensure good model fitting. For completeness, we provide a short summary of well-known important notions and results formulated for nonhomogeneous Markov chains that, in general, can be transferred to the homogeneous case. Then, as an illustration for the homogeneous case, we present a selected Billingsley’s result on parameter estimation for irreducible chains with finite state space. In the nonhomogeneous case, we quote two recent results, one of the calibration type and the other with more of a statistical flavour. We provide an ample set of bibliographic references so that the reader wanting to pursue her/his studies will be able to do so more easily and productively.

Mathematical model of the transmission of tuberculosis infection was widely studied to capture the transient behavior of the disease transmission. In this study we model the dynamic of the disease by considering an epidemiological model called SEIIR model to capture the deterministic behavior of the disease. We also applied a continuous-time Markov chain model to take into consideration the randomness of the system. In the deterministic model, a disease-free equilibrium point and a basic reproduction number of the model are found which are mainly influenced by the contact rate of susceptible individuals with infective individuals. Other parameters such as progression rate of the latent individuals to be infectious individuals and the treatment rate of latent individuals also influence not only the deterministic model but also the stochastic sample paths. For a certain critical value of the treatment rate of latent class, deterministic and stochastic solutions show different behavior at the final time of observation. A high degree of randomness is observed in the latent and infected class (hospitalize or not-hospitalized). While in the susceptible class, effect of randomness is almost not observed. This suggests the robustness of deterministic model of susceptible class to the stochastic perturbations.

The answering communities, such as Yahoo! Answers, offer great intelligence to help people solve questions. Participants can express their judgements towards answers and the system also keeps a record for every user. Retrieving Question-Answer pairs (QA pairs) extracted from these forums can improve the quality of Question-Answering (QA) systems. In this paper, we propose a Collaborative Ranking (ColRank) algorithm employing the Continuous Markov Chain Model (CMCM) to combine the quality of QA pairs and relationships among them. Empirical results show that the innovative algorithm is effective and outperform the state of art Question-Answering baselines.

An Introduction to Markov Chain Monte Carlo

We treat a model describing the continued formation and growth of mutant biological populations. At each transition time of a Poisson process a new mutant population begins its evolution with a fixed number of elements and evolves according to the laws of a continuous time positive recurrent Markov Chain $Y(t)$ with stationary transition probabilities $P_{ik}(t), i, k = 0,1,2,\cdots, t \geqq 0$. Our principal concern is the asymptotic behavior of moments and of the distribution function of the functional $S(t) = \{$number of different sizes of mutant populations at time $t\}$. When the recurrence time distribution to any state of the Markov Chain $Y(t)$ has a finite second moment, the moments of $S(t)$ and limit behavior of its distribution function are controlled by the stationary measure associated with $Y(t)$. When the second moment of the recurrence time distribution is infinite, then a local limit theorem and speed of convergence estimate for $P_{ik}(t)$ with $k = k(t) \rightarrow \infty, t \rightarrow \infty$ are required to establish asymptotic formulas for moments of $S(t)$.

Strong approximation theorems for density dependent Markov chains

We study new formulae based on Lya- punov exponents for entropy, mutual information, and capacity of finite state discrete time Markov chan- nels. We also develop a method for directly com- puting mutual information and entropy using contin- uous state space Markov chains. We show that the entropy rate for a symbol sequence is equal to the primary Lyapunov exponent for a product of random matrices. We then develop a continuous state space Markov chain formulation that allows us to directly compute entropy rates as expectations with respect to the Markov chain's stationary distribution. We also show that the stationary distribution is a continuous function of the input symbol dynamics. This continu- ity allows the channel capacity to be written in terms of Lyapunov exponents.

Library Website Design and Development: Trends and Best Practices is a how-to guide written specifically for librarians and library technologists who are designing or redesigning their library website. Whether in academic, public, or special libraries, library websites are created as a service to users – a digital branch of the physical library where users can find and access the information they require. As such, library website designers grapple with meeting library-specific needs and concerns while also designing a website that looks modern and on trend. This book provides library website designers with foundational knowledge of the standards and best practices that apply to all websites, but also delves into the current trends of modern library websites specifically. Outlining the process of creating a well-organized, accessible, and user-friendly website for library users, the book starts with needs assessment and content organization, continues through site navigation and user experience design, and closes with a look at website analytics and the process of ongoing maintenance and assessment. Library Website Design and Development: Trends and Best Practices provides practicing web librarians with an inclusive step-by-step guide to all of the topics inherent in the website design and development process, while also taking a focused look at the unique needs of library websites. Each chapter in this book covers the foundational knowledge needed for an aspect of website design and is supplemented by a list of additional resources that go into further depth on each topic.

In this article, a multiuser cooperative nonorthogonal multiple access (NOMA) network is considered. In order to improve their battery lifetime, the near-user (NU) Internet of Things (IoT) nodes relay information to the far-user (FU) IoT nodes using only the energy harvested from the ambience. The harvested energy at all the NU IoT nodes is stored in energy buffers. To improve the throughput performance, the best NU and best FU (BNBF) user selection scheme is employed. Moreover, unlike conventional orthogonal multiple access (OMA) networks, an adaptive NOMA network is considered in this article that switches between direct NOMA, NOMA with relaying, and OMA modes to maximize throughput. To improve accuracy and reduce the computational complexity, the energy buffer states at the NU IoT nodes are modeled using a continuous state-space Markov chain (CSMC) instead of discrete state-space Markov chain (DSMC). We derive the limiting distributions of the stored energy in energy buffers with either the best effort policy (BEP) or the on–off policy (OOP) applied for buffer energy management. Expressions for throughput are derived for both harvest-store-use (HSU) and harvest-use (HU) architectures. The Monte Carlo simulations are used to validate the derived analytical expressions.

State space models is a very general class of time series models capable of modelling dependent observations in a natural and interpretable way. Inference in such models has been studied by Bickel, Ritov and Rydén, who consider hidden Markov models, which are special kinds of state space models, and prove that the maximum likelihood estimator is asymptotically normal under mild regularity conditions. In this paper we generalize the results of Bickel, Ritov and Rydén to state space models, where the latent process is a continuous state Markov chain satisfying regularity conditions, which are fulfilled if the latent process takes values in a compact space.

This paper consider the (BMAP1, BMAP2)/(PH1, PH2)/N retrial queue with finite-position buffer. The behavior of the system is described in terms of continuous time multi-dimensional Markov chain. Arriving type I calls find all servers busy and join the buffer, if the positions of the buffer are insufficient, they can go to orbit. Arriving type II calls find all servers busy and join the orbit directly. Each server can provide two types heterogeneous services with Phase-type (PH) time distribution to every arriving call (including types I and II calls), arriving calls have an option to choose either type of services. The model is quite general enough to cover most of the systems in communication networks. We derive the ergodicity condition, the stationary distribution and the main performance characteristics of the system. The effects of various parameters on the system performance measures are illustrated numerically.

This paper considers a singularly perturbed hybrid system whose state equations are governed by a stochastic switching process, which is singularly perturbed and is modeled as a near decomposable continuous time finite state Markov chain (FSMC). The decomposition of the system and the switching process together into slow and fast subsystems is investigated. An approximate model for the slow subsystem over the interval of fast transitions within each group is derived and the mean-squared error between the model and the actual subsystem is quantified. The stability of the slow mode subsystem is studied and two stability criteria are introduced. The behavior of the fast subsystem depending on the relative size of perturbation parameters is analyzed. Finally an example is used to illustrate the aforementioned techniques.

BackgroundPre-existing cellular immunity has been recognized as one of the key factors in determining the outcome of influenza infection by reducing the likelihood of clinical disease and mitigates illness. Whether, and to what extent, the effect of this self-protective mechanism can be captured in the population dynamics of an influenza epidemic has not been addressed.MethodsWe applied previous findings regarding T-cell cross-reactivity between the 2009 pandemic H1N1 strain and seasonal H1N1 strains to investigate the possible changes in the magnitude and peak time of the epidemic. Continuous Monte-Carlo Markov Chain (MCMC) model was employed to simulate the role of pre-existing immunity on the dynamical behavior of epidemic peak.ResultsFrom the MCMC model simulations, we observed that, as the size of subpopulation with partially effective pre-existing immunity increases, the mean magnitude of the epidemic peak decreases, while the mean time to reach the peak increases. However, the corresponding ranges of these variations are relatively small.ConclusionsOur study concludes that the effective role of pre-existing immunity in alleviating disease outcomes (e.g., hospitalization) of novel influenza virus remains largely undetectable in population dynamics of an epidemic. The model outcome suggests that rapid clinical investigations on T-cell assays remain crucial for determining the protection level conferred by pre-existing cellular responses in the face of an emerging influenza virus.

This paper reports on the development of a wear prediction model based on stochastic filtering and hidden Markov theory. It is assumed that observations at discrete time points are available such as metal concentrations from oil-based monitoring, which are related to the true underlying state of the system which is unobservable. The system state is represented by a generic term of wear which is modelled by a continuous hidden Markov Chain using a Beta distribution. We formulated a recursive model to predict the current and future system state given past observed monitoring information to date. The model is useful to wear-based monitoring such as oil analysis. Numerical examples are presented in the paper based on simulated and real data.

Currently, a part of the R statistical software is developed in order to deal with spatial models. More specifically, some available packages allow the user to analyse categorical spatial random patterns. However, only the spMC package considers a viewpoint based on transition probabilities between locations. Through the use of this package it is possible to analyse the spatial variability of data, make inference, predict and simulate the categorical classes in unobserved sites. An example is presented by analysing the well-known Swiss Jura data set.