Mixed Vehicle Flow At Signalized Intersection: Markov Chain Analysis

The production of active carbon from bituminous coal

In Chapter 7 we considered Markov chains as a means to model stochastic DES for which explicit closed-form solutions can be obtained. Then, in Chapter 8, we saw how special classes of Markov chains (mostly, birth-death chains) can be used to model queueing systems. We pointed out, however, that queueing theory is largely “descriptive” in nature; that is, its main objective is to evaluate the behavior of queueing systems operating under a particular set of rules. On the other hand, we are often interested in “prescriptive” techniques, based on which we can make decisions regarding the “best” way to operate a system and ultimately control its performance. In this chapter, we describe some such techniques for Markov chains. Our main objective is to introduce the framework known as Markov Decision Theory, and to present some key results and techniques which can be used to control DES modeled as Markov chains. At the heart of these techniques is dynamic programming, which has played a critical role in both deterministic and stochastic control theory since the 1960s. The material in this chapter is more advanced than that of previous ones, it involves some results that were published in the research literature fairly recently, and it demands slightly higher mathematical sophistication. The results, however, should be quite gratifying for the reader, as they lead to the solution of some basic problems from everyday life experience, or related to the

We present in this paper a mathematical method based on stochastic comparisons of Markov chains in order to compute in a mobile network performance measures that are important Q parameters for users: the dropping handover of voice and the blocking probability of a new voice call. The key idea of this methodology is that given a complex system represented by a multi-dimensional Markov chain which is too large to be solved, we propose to reduce the state space, and so to define a new Markov chain which is a simplified version of the original one. This reduced Markov chain is defined as an aggregated one, which represents a stochastic bound for performance measures written as increasing reward function on the stationary distribution. The main steps of the construction of the aggregated Markov chain applied into mobile networks are presented in this paper. As the number of mobile users with different kinds of applications increases, the associated model is more complex and it can be represented by multi-dimensional continuous time Markov chain with a very large size. Thus, we define an aggregated Markov chain represented by a multi-dimensional birth and death process which is very easy to solve. We have proved that there is a weak ordering between the original Markov chain and the aggregated one using increasing set formalism. We have computed upper bounds of dropping handover and blocking probability for different values of input parameters. Numerical results prove that upper bounds give good results and so the stochastic methodology is an interesting mathematical tool for the performance evaluation of complex systems. Keywords: Mobile Networks, Quality of Service, Stochastic ordering, stochastic comparisons, Continuous Time Markov Chains.

Numerical analysis of rational processes beyond Markov chains

This paper describes a Bayesian approach to determining the order of a finite state Markov chain whose transition probabilities are themselves governed by a homogeneous finite state Markov chain. It extends previous work on homogeneous Markov chains to more general and applicable hidden Markov models. The method we describe uses a Markov chain Monte Carlo algorithm to obtain samples from the (posterior) distribution for both the order of Markov dependence in the observed sequence and the other governing model parameters. These samples allow coherent inferences to be made straightforwardly in contrast to those which use information criteria. The methods are illustrated by their application to both simulated and real data sets.

Demographic dynamics is formally equivalent to the dynamics of a Markov chain, as is true of some nonlinear dynamical systems. Convergence to demographic equilibrium can be studied in terms of convergence in the Markov chain. Tuljapurkar (1982) showed that population entropy (Kolmogorov-Sinai entropy) provides information on the rate of this convergence. This paper begins by considering finite state Markov chains, providing elementary proofs of the relationship between convergence rate and entropy, and discusses in detail the uses and limitations of entropy as a convergence measure; these results also apply to Markovian dynamical systems. Next, new qualitative and quantitative arguments are used to discuss the demographic meaning of entropy. An exact relationship is established giving population entropy in terms of the eigenvalues of the Leslie matrix characteristic equation. Finally, the significance of imprimitive and periodic limits is discussed in relation to population entropy.

Stiffness-tolerant methods for transient analysis of stiff Markov chains

The authors study the performance of a non-blocking space-division packet switch, which experiences nonuniform traffic intensities that also change as a function of time. A finite-state Markov chain is used as an underlying process to govern the time variation of traffic for the entire switch. The packet arrivals at each input form an independent Bernoulli process modulated by the underlying Markov chain. Also, the output address of each packet is independently and randomly assigned with probability distributions, which are gain modulated by the Markov chain. Provided that the traffic at each output is not dominated by individual inputs, it is shown that the service time of each input queue for sufficiently large switches can be characterized by an independent Markov-modulated phase-type process. A matrix geometric solution for the resultant quasi-birth-death-type queuing process is presented. The maximum throughput is obtained at the system saturation. The performance of the switch is numerically examined under various traffic conditions. A contention priority scheme is also proposed to improve the switch performance. >

We consider the valuation of credit default swaps (CDSs) under an extended version of Merton’s structural model for a firm’s corporate liabilities. In particular, the interest rate process of a money market account, the appreciation rate, and the volatility of the firm’s value have switching dynamics governed by a finite-state Markov chain in continuous time. The states of the Markov chain are deemed to represent the states of an economy. The shift from one economic state to another may be attributed to certain factors that affect the profits or earnings of a firm; examples of such factors include changes in business conditions, corporate decisions, company operations, management strategies, macroeconomic conditions, and business cycles. In this article, the Esscher transform, which is a well-known tool in actuarial science, is employed to determine an equivalent martingale measure for the valuation problem in the incomplete market setting. Systems of coupled partial differential equations (PDEs) satisfied by the real-world and risk-neutral default probabilities are derived. The consequences for the swap rate of a CDS brought about by the regimeswitching effect of the firm’s value are investigated via a numerical example for the case of a two-state Markov chain. We perform sensitivity analyses for the real-world default probability and the swap rate when different model parameters vary. We also investigate the accuracy and efficiency of the PDE approach by comparing the numerical results from the PDE approach to those from the Monte Carlo simulation.

On fast aggregation of Markov chain functionals using stochastic complementation

We consider a singularly perturbed (finite state) Markov chain and provide a complete characterization of the fundamental matrix. In particular, we obtain a formula for the regular part simpler than a previous formula obtained by Schweitzer, and the singular part is obtained via a reduction process similar to Delebecque's reduction for the stationary distribution. In contrast to previous approaches, one works with aggregate Markov chains of much smaller dimension than the original chain, an essential feature for practical computation. An application to mean first-passage times is also presented.

This paper presents a comparative analysis of Markov chain and K-means clustering techniques for constructing representative drive cycles in the optimisation of powertrain modules for electric vehicles (EVs). As part of the EU funded POWERDRIVE (Power electronics optimisation for next generation electric vehicle components) project, the research focuses on a case study conducted in Dublin, Ireland, aiming to develop an accurate and efficient drive cycle that captures the unique driving patterns and characteristics of the city. The Markov chain technique utilises historical driving data to model the transition probabilities between different driving states, while the K-means clustering technique groups similar driving patterns based on key parameters. The analysis evaluates the effectiveness of both approaches in terms of their ability to accurately represent real-world driving behaviour and their computational efficiency. The results of the study provide insights into the strengths and limitations of each technique, enabling researchers and practitioners to make informed decisions when selecting an appropriate methodology for drive cycle construction. The findings contribute to the ongoing efforts in developing optimised powertrain modules for EVs and advancing the field of electric vehicle technology.

Article Free Access Share on Exact sampling and approximate counting techniques Author: Mark Huber School of Operations Research and Industrial Engineering, Cornell University, Ithaca, NY School of Operations Research and Industrial Engineering, Cornell University, Ithaca, NYView Profile Authors Info & Claims STOC '98: Proceedings of the thirtieth annual ACM symposium on Theory of computingMay 1998 Pages 31–40https://doi.org/10.1145/276698.276709Published:23 May 1998Publication History 25citation666DownloadsMetricsTotal Citations25Total Downloads666Last 12 Months66Last 6 weeks15 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

This work analyzes distributed linear averaging within a connected network of sensors that each track the stationary distribution of an ergodic Markov chain with a slowly switching regime. Our approach is based on a two-time-scale stochastic approximation. A hyperparameter modeled as a Markov chain on a slower time-scale modulates the regime of each observed Markov chain. The average of all currently observed stationary distributions constitutes the average-consensus estimate to be reached by all sensors. Assuming the Markov chains do not share a common stationary distribution conditioned on their regime, then under the proposed linear averaging algorithm, the exchange graph conditions required for the sequence of sensor state values to converge weakly to the average-consensus are obtained. Estimation of a weighted average of all observed stationary distributions, not only the current ones, is proved feasible over a long-run time horizon, provided an additional communication condition holds. The sensor st...

The problem of sampling from the stationary distribution of a Markov chain finds widespread applications in a variety of fields. The time required for a Markov chain to converge to its stationary distribution is known as the classical mixing time. In this article, we deal with analog quantum algorithms for mixing. First, we provide an analog quantum algorithm that, given a Markov chain, allows us to sample from its stationary distribution in a time that scales as the sum of the square root of the classical mixing time and the square root of the classical hitting time. Our algorithm makes use of the framework of interpolated quantum walks and relies on Hamiltonian evolution in conjunction with von Neumann measurements. There also exists a different notion for quantum mixing: the problem of sampling from the limiting distribution of quantum walks, defined in a time-averaged sense. In this scenario, the quantum mixing time is defined as the time required to sample from a distribution that is close to this limiting distribution. Recently, we provided an upper bound on the quantum mixing time for Erd\ifmmode \mbox{\H{o}}\else \H{o}\fi{}s-R\'enyi random graphs [Phys. Rev. Lett. 124, 050501 (2020)]. Here, we also extend and expand upon our findings therein. Namely, we provide an intuitive understanding of the state-of-the-art random matrix theory tools used to derive our results. In particular, for our analysis we require information about macroscopic, mesoscopic, and microscopic statistics of eigenvalues of random matrices which we highlight here. Furthermore, we provide numerical simulations that corroborate our analytical findings and extend this notion of mixing from simple graphs to any ergodic, reversible, Markov chain.