On fast aggregation of Markov chain functionals using stochastic complementation

An algorithm for exact parametric analysis of stochastic Petri nets is presented. The algorithm is derived from the theory of decomposition and aggregation of Markov chains. The transition rate of interest is confined into a diagonal submatrix of the associated Markov chain by row and column permutations. Every time a new value is assigned to the transition, a smaller Markov chain is analyzed. As a result, the computational cost is greatly reduced.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Data processing in high energy physics experiments is a multi-tiered process in which raw detector signals are first processed locally into physics objects, and then collated into event records which can be scrutinized by a fast online trigger system. The resulting selection of events are reconstructed and pass through a number of software filters before arriving at a final offline analysis where hard physical constants are extracted. Although sophisticated statistical data analysis techniques are routinely employed high energy physics, the use of statistical signal processing in the field is has until now been rare. Our paper will begin with an overview of a typical high energy physics data acquisition system, outlining the technologies and tradeoffs involved at each stage. We will then proceed to argue that the dominant roles of model dependence and systematic errors in final physics analyses render statistical signal processing techniques largely inapplicable at this level. We observe, however, that at the low-level pattern recognition and event reconstruction levels, statistical signal processing techniques have been making inroads in high energy physics for a number of years, and examples from the literature will be cited. The viability of the technique for second level triggers will be assessed. Parallels to other other approaches, such as neural networks, will also be drawn. It will be argued that the falling cost of computing hardware favors the growth of statistical signal processing methods in high energy physics

In this paper, we consider the relationship between risk-sensitivity and information. Product estimators are introduced as a generalization of maximum a posteriori probability (MAP) estimator for hidden Markov models. We study the relationship between the inclusion of higher order moments, the underlying dynamics and the availability of information. Asymptotic periodicity of these estimators and the relationship between risk and information is studied via simulation.

In this paper, we present a framework for a mixed estimation scheme for hidden Markov models (HMM). A robust estimation scheme is first presented using the minimax method that minimizes a worst case cost for HMMs with bounded uncertainties. Then we present a mixed estimation scheme that minimizes a risk-neutral cost with a constraint on the worst-case cost. Some simulation results are also presented to compare these different estimation schemes in cases of uncertainties in the noise model.

A stochastic version of Expectation Maximization algorithm for better estimation of Hidden Markov Model

In this paper, we consider the relationship between risk-sensitivity and information. Product estimators are introduced as a generalization of maximum a posteriori probability (MAP) estimator for hidden Markov models. We study the relationship between the inclusion of higher order moments, the underlying dynamics and the availability of information. Asymptotic periodicity of these estimators and the relationship between risk and information is studied via simulation.

The estimation of Hidden Markov Models has attracted a lot of attention recently. The purpose of this paper is to lay the foundation for a new approach for the analysis of the maximum-likelihood estimation of HMM-s, using representation of HMM-s due to Borkar (1993). A useful connection between the estimation theory of HMM-s and linear stochastic systems is established via the theory of L-mixing processes. The results are potentially useful for deriving strong approximation results, which are in turn applicable to analyze adaptive predictors and change detection methods.

Markovian jumping systems have arisen naturally in the mathematical modeling of phenomena spanning disciplines in the social sciences, natural sciences, and engineering. This kind of stochastic dynamical systems can be employed to model the dynamics when parameters are subject to random abrupt changes due to sudden environment changes, subsystem switching, system noises, executor faults, and so forth. Much attention has been given to modeling, optimization, and real applications of such stochastic dynamical systems in the literature in recent years. As the advanced control and optimization will provide a basis for the design and application of such stochastic systems, these advanced techniques would result in substantial and sustainable benefits. The accepted papers in this special issue include stochastic stability, stabilization, stochastic control optimization, system modeling and identification methods, predictive control, signal processing, robust filtering, multiagent systems, networked control systems, time-delayed systems, neural networks, the Takagi-Sugeno fuzzy systems, simulated annealing, and fault detection methods.

In this paper, an image fusion technique based on statistical signal processing is studied. Without the multiscale transform (MST) for the original images, the fusion is directly made via using Expectation-Maximization (EM) algorithm. Unlike the standard EM fusion algorithm, we only use the neighboring pixels in the window to estimate the centered pixel. The improvement of the iterative procedure is helpful to obtain the final fusion result effectively. At last, some performance evaluations are suggested and applied to compare the performance of these fusion methods. Consequently, the obtained results and comparisons indicate that the method here has significant advantage. Keywords-image fusion; statistical signal processing; EM algorithm

We assume that a Poisson flow of vehicles arrives at isolated signalized intersection, and each vehicle, independently of others, represents a random number X of passenger car units (PCU’s). We analyze numerically the stationary distribution of the queue process {Zn}, where Zn is the number of PCU’s in a queue at the beginning of the n-th red phase, n → ∞. We approximate the number Yn of PCU’s arriving during one red-green cycle by a two-parameter Negative Binomial Distribution (NBD). The well-known fact is that {Zn} follow an infinite-state Markov chain. We approximate its stationary distribution using a finite-state Markov chain. We show numerically that there is a strong dependence of the mean queue length E[Zn] in equilibrium on the input distribution of Yn and, in particular, on the ”over dispersion” parameter γ= Var[Yn ]/E[Yn ]. For Poisson input, γ = 1. γ &gt; 1 indicates presence of heavy-tailed input. In reality it means that a relatively large ”portion” of PCU’s, considerably exceeding the average, may arrive with high probability during one red-green cycle. Empirical formulas are presented for an accurate estimation of mean queue length as a function of load and g of the input flow. Using the Markov chain technique, we analyze the mean ”virtual” delay time for a car which always arrives at the beginning of the red phase.

This paper describes a Bayesian approach to determining the order of a finite state Markov chain whose transition probabilities are themselves governed by a homogeneous finite state Markov chain. It extends previous work on homogeneous Markov chains to more general and applicable hidden Markov models. The method we describe uses a Markov chain Monte Carlo algorithm to obtain samples from the (posterior) distribution for both the order of Markov dependence in the observed sequence and the other governing model parameters. These samples allow coherent inferences to be made straightforwardly in contrast to those which use information criteria. The methods are illustrated by their application to both simulated and real data sets.

Demographic dynamics is formally equivalent to the dynamics of a Markov chain, as is true of some nonlinear dynamical systems. Convergence to demographic equilibrium can be studied in terms of convergence in the Markov chain. Tuljapurkar (1982) showed that population entropy (Kolmogorov-Sinai entropy) provides information on the rate of this convergence. This paper begins by considering finite state Markov chains, providing elementary proofs of the relationship between convergence rate and entropy, and discusses in detail the uses and limitations of entropy as a convergence measure; these results also apply to Markovian dynamical systems. Next, new qualitative and quantitative arguments are used to discuss the demographic meaning of entropy. An exact relationship is established giving population entropy in terms of the eigenvalues of the Leslie matrix characteristic equation. Finally, the significance of imprimitive and periodic limits is discussed in relation to population entropy.

Solving Markov chains is, in general, difficult if the state space of the chain is very large (or infinite) and lacking a simple repeating structure. One alternative to solving such chains is to construct models that are simple to analyze and provide bounds for a reward function of interest. We present a new bounding method for Markov chains inspired by Markov reward theory: Our method constructs bounds by redirecting selected sets of transitions, facilitating an intuitive interpretation of the modifications of the original system. We show that our method is compatible with strong aggregation of Markov chains; thus we can obtain bounds for an initial chain by analyzing a much smaller chain. We illustrate our method by using it to prove monotonicity results and bounds for assemble-to-order systems.

The authors study the performance of a non-blocking space-division packet switch, which experiences nonuniform traffic intensities that also change as a function of time. A finite-state Markov chain is used as an underlying process to govern the time variation of traffic for the entire switch. The packet arrivals at each input form an independent Bernoulli process modulated by the underlying Markov chain. Also, the output address of each packet is independently and randomly assigned with probability distributions, which are gain modulated by the Markov chain. Provided that the traffic at each output is not dominated by individual inputs, it is shown that the service time of each input queue for sufficiently large switches can be characterized by an independent Markov-modulated phase-type process. A matrix geometric solution for the resultant quasi-birth-death-type queuing process is presented. The maximum throughput is obtained at the system saturation. The performance of the switch is numerically examined under various traffic conditions. A contention priority scheme is also proposed to improve the switch performance. >

We consider the valuation of credit default swaps (CDSs) under an extended version of Merton’s structural model for a firm’s corporate liabilities. In particular, the interest rate process of a money market account, the appreciation rate, and the volatility of the firm’s value have switching dynamics governed by a finite-state Markov chain in continuous time. The states of the Markov chain are deemed to represent the states of an economy. The shift from one economic state to another may be attributed to certain factors that affect the profits or earnings of a firm; examples of such factors include changes in business conditions, corporate decisions, company operations, management strategies, macroeconomic conditions, and business cycles. In this article, the Esscher transform, which is a well-known tool in actuarial science, is employed to determine an equivalent martingale measure for the valuation problem in the incomplete market setting. Systems of coupled partial differential equations (PDEs) satisfied by the real-world and risk-neutral default probabilities are derived. The consequences for the swap rate of a CDS brought about by the regimeswitching effect of the firm’s value are investigated via a numerical example for the case of a two-state Markov chain. We perform sensitivity analyses for the real-world default probability and the swap rate when different model parameters vary. We also investigate the accuracy and efficiency of the PDE approach by comparing the numerical results from the PDE approach to those from the Monte Carlo simulation.